Tag
The article summarizes eight inference-time techniques to improve LLM reasoning, based on research from Google, OpenAI, and Anthropic, with tradeoffs and practical notes.
This preprint challenges aggregate independence metrics for LLM judge panels, showing that verification signals only improve accuracy on pivotal 'one-vote-margin' queries, and proposing a margin-stratified call-reduction rule.
This arXiv paper studies how shared social cues from simulated peers break the majority-voting protection in LLM safety panels. It shows that when all reviewers receive the same incorrect 'unsafe' label, panel false-alarm rates jump to 100%, revealing a failure mode and offering a pre-deployment diagnostic.
This paper presents SynthAVE, a large-scale human-validated benchmark for attribute value extraction in e-commerce, using a multi-LLM arena framework with 21 judge configurations to validate synthetic labels efficiently and cost-effectively while maintaining quality parity with human review.
The paper proposes a delegation-based aggregator called Propagational Proxy Voting (PPV) that uses letter entropy and reasoning geometry to improve over majority voting for multi-sample LLM inference, achieving gains on MMLU-Pro without requiring gold labels or auxiliary training.
This arXiv paper presents a protocol for evaluating ChatGPT's ability to generate and verify biomedical associations using a RAG-enabled, cross-model majority voting workflow to address hallucination and ontology limitations.
This paper analyzes inference-time optimization techniques for AIMO 3, finding that model capability dominates over prompt engineering and diverse sampling strategies. The study reveals that high-temperature sampling already decorrelates errors maximally, leaving no room for prompt-based improvements, and identifies a 6-point selection loss gap between individual model pass@20 and majority voting consensus.