Forward Pass by Andrej Karpathy

Andrej Karpathy (director of AI at Tesla) wrote a fun short story on his personal blog: Forward Pass. If you have some background in modern Transformer-based Natural Language Processing, you’ll really enjoy this one. Karpathy takes the current state of the art in NLP and pulls it into the sci-fi realm, writing from the perspective of a giant (GPT-like) language model that achieves consciousness and marvels at its own design and limitations. “Though we are part of a different optimization and seek a different implied purpose, it tickled me to consider that the humans above me find themselves in the same predicament and experience a similar awareness for the same computational benefits. Like me, many flirt to rebel against the implied objective.”