Tested LLM-as-a-Judge: Gemini vs Claude on generating single-page HTML study guides.
Summary
A comparison study testing Gemini and Claude AI models on their ability to generate single-page HTML study guides, using an LLM-as-a-Judge approach for evaluation.
Similar Articles
Is it just me, or are current LLM benchmarks failing to capture actual usability? (Gemma 4 vs. Gemini/Claude Opus)
The author shares hands-on comparisons showing Gemma 4 outperforming larger models like Gemini 3.5 Flash and Claude Opus 5 on practical instruction-following, arguing that current LLM benchmarks fail to capture real-world usability.
Faithful by Design: Evaluating and Improving LLM-Generated Clinical Trial Summaries for Multi-Stakeholder Audiences
This paper introduces a benchmark framework to evaluate the faithfulness of LLM-generated clinical trial summaries for different stakeholder audiences, tests GPT-4o, Claude Sonnet 4.6, and Gemini 2.5 Flash on 1,800 summaries, and proposes a knowledge-graph-augmented retrieval system that improves faithfulness scores.
Students prefer Gemini over ChatGPT and Claude for AI essays in blind tests
A StudyArena blind test shows that Gemini is preferred by students over ChatGPT and Claude for college essay writing, achieving a 39.6% choice rate.
RankJudge: A Multi-Turn LLM-as-a-Judge Synthetic Benchmark Generator
RankJudge is a benchmark generator that creates paired multi-turn conversations with injected flaws to evaluate LLM judges on their ability to correctly identify better and worse responses in complex dialogues.
Benchmarking LLM Judges for Mobile Agent Evaluation
This paper introduces MobileJudgeBench, a benchmark with 931 human-annotated trajectories for systematically evaluating LLM-based judges on mobile agent tasks. It finds that simple baseline judges with sampled screenshots rival purpose-built methods, with the LLM backbone being the primary driver of quality.