Tag
The paper introduces OrderBench, a benchmark for restaurant ordering LLM agents that evaluates semantic reliability beyond schema validity, demonstrating that structured output modes can achieve perfect schema validity while still having high semantic error rates.
Object Aligner is an open-source Python library that deterministically scores two JSON objects by recursively aligning their trees, using Hungarian algorithm for unordered collections and sequence alignment for ordered ones. It introduces referential alignment for graphs/hypergraphs and can be used as a reward function in LLM prompt optimization.
GolemUI is a declarative form engine that defines forms as JSON schemas, offering accessibility, i18n, and multi-framework support out of the box.
Datalab releases lift, a model that extracts structured JSON from PDFs and images using schema-constrained decoding, with local and hosted inference options.
This article explains how to use GRPO to fine-tune an LLM (Qwen3-8B) for reliable JSON structured output, improving schema accuracy from 62% to 82%, surpassing GPT-4.1's 58%.
A developer shares lessons from building a local document-to-JSON extractor using llama3.2 3B on Ollama, highlighting that deterministic post-processing and schema-constrained outputs matter more than model size, while seeking feedback on hallucination and context truncation issues with long documents.
Open Envelope provides an open JSON Schema for defining AI agent teams with roles, handoffs, human checkpoints, and access policies, aiming for portability across frameworks.
The author shares findings on the lack of portability for JSON Schema structured outputs across AI providers like OpenAI, Gemini, and Anthropic, highlighting inconsistencies in constraint enforcement and offering practical advice for robust integration.