Building Food Metadata with LLM Juries
Summary
A method using LLMs as a jury to build or validate food metadata, improving data quality and consistency.
Similar Articles
When Do LLMs Actually Help? Evaluating LLMs as Data Quality Annotators
This study evaluates LLMs as data quality annotators on e-commerce tasks, finding they outperform baselines when background knowledge is required but offer limited advantages for tasks with strong lexical signals, while demonstrating high consistency across runs.
@dair_ai: // Your LLM judge disagrees with the experts // LLM Judges can be tricky to build. Here is an interesting showcasing wh…
The paper presents UPHELD, a benchmark with extensive human annotations for evaluating conversational LLMs, and a Mixture-of-Judges framework that enhances evaluation accuracy by 30%.
Judge, Retrieve, or Abstain: Uncertainty-Guarded LLM Judging with Provable Risk Guarantees
This paper proposes a risk-controlled framework for using LLMs as judges in factual evaluation, calibrating uncertainty thresholds to maintain a user-specified error rate and routing to retrieval-augmented mode when needed, achieving higher coverage with provable reliability guarantees.
Multilingual and Multimodal LLMs in the Wild: Building for Low-Resource Languages
This tutorial paper provides an overview of building multilingual and multimodal LLMs for low-resource languages, covering data creation, model alignment, fine-tuning, and evaluation, with a focus on practical recipes and hands-on resources.
Augmenting Human Evaluation with LLM Judges: How Many Human Reviews Do You Need?
This paper proposes a two-stage sampling design where LLM evaluations are used to augment, rather than replace, human ratings, and provides guidance on determining sample sizes for human and LLM reviews using a doubly robust estimator from missing data literature.