GPT-2 Fully Decoded Internally Black Box Fully Open With Demo
Summary
The BABEL codec achieves the first complete decode of GPT-2 small's internal state, reconstructing 94.7% of its behavior and enabling reading and writing to the model in English, with open source resources and a demo.
Similar Articles
GPT-5.3-Codex System Card
OpenAI releases GPT-5.3-Codex, the most capable agentic coding model combining frontier coding performance with advanced reasoning, featuring interactive long-running task execution and novel high-capability safeguards in the cybersecurity domain.
Introducing GPT-5.2-Codex
OpenAI releases GPT-5.2-Codex, an advanced agentic coding model optimized for complex software engineering tasks with improvements in long-context understanding, Windows support, and cybersecurity capabilities. The model achieves state-of-the-art performance on SWE-Bench Pro and Terminal-Bench 2.0, and is now available to paid ChatGPT users with API access coming in the following weeks.
GPT-2: 1.5B release
OpenAI releases GPT-2 1.5B model with analysis of human perception of credibility, potential for misuse through fine-tuning on extremist ideologies, and challenges in detecting synthetic text. Detection models achieve ~95% accuracy but require complementary approaches for practical deployment.
@vintcessun: This project is insane — it builds GPT behind ChatGPT from scratch in a way even a kid can understand. Every line of code is commented, 12 chapters over 7500 lines, and it even explains the attention mechanism details that I could never figure out. Simply put, if you want to 'understand' rather than 'import packages' for LLM, this is the most beginner-friendly hands-on tutorial right now.
A 12-chapter interactive textbook that teaches how to build a GPT-like language model from absolute scratch, with fully annotated code and beginner-friendly explanations.
Better language models and their implications
OpenAI introduces GPT-2, a 1.5 billion parameter transformer-based language model trained on 40GB of internet text that achieves state-of-the-art performance on language modeling benchmarks and demonstrates zero-shot capabilities in reading comprehension, translation, question answering, and summarization. Due to safety concerns, only a smaller model and technical paper are released publicly rather than the full trained model.