structured-generation

A high-throughput inference engine for structured information extraction on Apple Silicon using MLX, offering parallel constrained decoding with 5.6x to 7.0x latency reductions and 100% schema validity.

Introduces Deco-G, a decoding framework that separates format adherence from problem-solving in LLMs, using a Format Estimation Module to ensure compliance without degrading reasoning. Achieves improved accuracy on mathematical reasoning, event extraction, and LLM-as-a-judge tasks.

An educational blog post explaining how SGLang works, including its runtime, frontend language, RadixAttention mechanism, and comparison to vLLM.

This paper characterizes when conformal risk control can certify structured LLM outputs, proving impossibility bounds and analyzing certification hierarchies across different bounds. Empirical validation on six open-weight models shows that hard configurations are uncertifiable at low risk levels but practical certification is achievable at relaxed targets.

LatticeBridge proposes a twisted sequential Monte Carlo decoder for structured sequence generation that improves constraint satisfaction by treating the problem as rare-event inference, outperforming greedy and beam baselines on CommonGen, E2E NLG, and WikiBio.

A new hybrid decoding framework called In-Writing is proposed, which delays constraint application until after a trigger token, combining free-form reasoning with structured generation for improved accuracy in classification and reasoning tasks.

This paper introduces Ishigaki-IDS-Bench, a benchmark for evaluating LLMs' ability to generate Information Delivery Specification (IDS) XML from BIM information requirements. Evaluation of 10 LLMs shows best models achieve 65.6% macro F1 for content agreement but only 27.7% pass the Content audit, indicating struggles with standard and vocabulary constraints.

DALM proposes a domain-algebraic language model that generates text under exact structural constraints derived from a domain lattice, addressing hallucination by organizing knowledge into separate domain fibers with algebraic guarantees. The model uses three-phase structured denoising (domain → relation → concept) with domain-annotated training data to prevent cross-domain contamination.

Outlines is a Python library that guarantees structured outputs (JSON, Pydantic models, etc.) from LLMs during generation, working across multiple model providers to eliminate parsing errors.