Tag
This preprint challenges aggregate independence metrics for LLM judge panels, showing that verification signals only improve accuracy on pivotal 'one-vote-margin' queries, and proposing a margin-stratified call-reduction rule.
This arXiv paper studies how shared social cues from simulated peers break the majority-voting protection in LLM safety panels. It shows that when all reviewers receive the same incorrect 'unsafe' label, panel false-alarm rates jump to 100%, revealing a failure mode and offering a pre-deployment diagnostic.
This paper presents SynthAVE, a large-scale human-validated benchmark for attribute value extraction in e-commerce, using a multi-LLM arena framework with 21 judge configurations to validate synthetic labels efficiently and cost-effectively while maintaining quality parity with human review.
The paper proposes a delegation-based aggregator called Propagational Proxy Voting (PPV) that uses letter entropy and reasoning geometry to improve over majority voting for multi-sample LLM inference, achieving gains on MMLU-Pro without requiring gold labels or auxiliary training.
This arXiv paper presents a protocol for evaluating ChatGPT's ability to generate and verify biomedical associations using a RAG-enabled, cross-model majority voting workflow to address hallucination and ontology limitations.
This paper analyzes inference-time optimization techniques for AIMO 3, finding that model capability dominates over prompt engineering and diverse sampling strategies. The study reveals that high-temperature sampling already decorrelates errors maximally, leaving no room for prompt-based improvements, and identifies a 6-point selection loss gap between individual model pass@20 and majority voting consensus.