Tag
This paper introduces the trie automaton, a specialized constrained decoding mechanism for finite sets that precomputes token masks via Aho-Corasick matching, achieving up to 29x end-to-end throughput improvements over XGrammar in vLLM batch serving while guaranteeing 100% output validity.
The author presents a method using GBNF grammars in llama.cpp to constrain local AI model outputs, ensuring valid JSON. A compiler is built to convert tool schemas into grammar rules, with per-turn narrowing for relevant tools.
Rapid-MLX 0.11.0 brings major performance gains with prefix-cache and response caching, supports new model families including HY3 295B MoE and Qwen3-Coder-Next 80B, introduces structured output with guaranteed valid tool calls, and adds seamless integration with MCP servers for autonomous agent workflows.
A study shows that when LLMs are asked to output in JSON format, their answer diversity collapses significantly compared to plain chat, with modal answers becoming more common and distinctive models losing half their uniqueness.
The paper introduces OrderBench, a benchmark for restaurant ordering LLM agents that evaluates semantic reliability beyond schema validity, demonstrating that structured output modes can achieve perfect schema validity while still having high semantic error rates.
This paper evaluates nine open-weight small language models (135M to 3B parameters) on a structured benchmark and shows that parameter-efficient fine-tuning significantly improves accuracy, making them viable for local deployment in structured niche workloads.
AnySearch is a search infrastructure designed for AI agents. It supports real-time web search and vertical domain search, outputting structured Markdown that agents can directly use, improving search efficiency.
The article shares production learnings for reliably generating structured JSON output from LLMs, covering methods like JSON mode, schema validation, and retry loops, achieving 99.5% validity.
This paper investigates how message format (e.g., free text, JSON, triples) affects information loss across multiple hops in LLM agent relays, finding that format effects depend on the relay model's capability and that structure preserves content faithfully but does not correct errors.
This paper investigates how sign-branched repetition penalties cause structured-output corruption and gauge dependence across different models and inference frameworks, providing measurements and comparisons with alternative repetition controls.
Discusses methods for generating structured outputs from large language models using constrained sampling techniques.
IFStruct, an instruction-following benchmark for structured output by Liquid AI, now has a leaderboard on Hugging Face, aimed at improving small models for local execution and correct tool use.
ifstruct is an instruction-following benchmark for structured output by Liquid AI, designed to push the field toward better small models that can run locally.
A practical technique for improving structured output generation from LLMs by feeding validation errors back into retry prompts, allowing the model to self-correct rather than blindly retrying. The method involves describing the error in model-friendly terms and providing the previous output for editing.
The author shares five patterns that consistently kill AI agents: too many jobs per agent, no human-in-the-loop for destructive actions, unstructured outputs, no spend caps, and lack of uncertainty escalation paths. Practical guardrails and a checklist for reliable agent deployment are provided.
This paper identifies and analyzes 'tool suppression' in open-weight LLMs when both tool calling and JSON schema constraints are simultaneously enabled, proposing the Constraint Priority Inversion hypothesis and a mitigation strategy called Transparent Two-Pass Execution.
Analyzes how DiffusionGemma's bidirectional attention and parallel block generation could potentially yield higher valid tool call rates due to its ability to revise tokens, even though its base quality is lower than Gemma 4.
Discusses various methods to optimize DiffusionGemma inference, reduce hallucination, and improve performance for tool use and agents, including entropy-bounded sampling, schema scaffolding, and retrieval during denoising.
This article explains how to use GRPO to fine-tune an LLM (Qwen3-8B) for reliable JSON structured output, improving schema accuracy from 62% to 82%, surpassing GPT-4.1's 58%.
Experiments with a live agent processing market data through a governed runtime revealed three surprises: prompt structure drives execution reliability over reasoning quality; structured output can influence agent decisions; and separating reasoning and extraction into two calls maintains high parse success. The findings suggest governance belongs at the execution boundary, not on freeform reasoning.