context-free-grammars

This paper provides a theoretical analysis of deep transformers' ability to model hierarchical structures using bounded-depth context-free grammars, constructing explicit positional-attention transformers that encode grammatical states in linearly separable subspaces.

This paper presents an efficient method for guiding LLM text generation with regular expressions and context-free grammars with minimal overhead, implemented in the open-source Python library Outlines.