Tag
The post claims that OpenAI Codex assigns a downgraded gpt-5.5-mini model to domestic users at the Ultra reasoning level, provides a detection prompt, and calls on users to reply 1 or 2 to measure the downgrade rate for domestic users.
Simon Willison introduces smevals, a small eval suite from Prime Radiant for evaluating models, prompts, and harnesses, with commands to run evals, grade results, and serve static HTML reports.
Arize Phoenix demonstrates using PXI to run an experiment comparing system prompt vs schema-aware prompt with a programmatic code evaluator, avoiding the need for an LLM judge.
Built an automated QA/eval engine for agent prompts called Baseline that treats prompts like software for regression testing, allowing non-coders to define rubrics and automatically optimize prompts. Currently in limited beta with a 30-day free trial.
This article introduces a new natural-language testing system for AI agents that uses simulated isolates to automatically generate multi-turn simulations and evaluate agent behavior, helping developers catch regressions from prompt changes.
A post highlights AIfiesta.ai, a tool that displays responses from multiple AI models (ChatGPT, Gemini, Claude) to the same prompt simultaneously, each in its own column.
User tested Grok's image generation function and found that the first time it successfully generated a complete image, but the second time it missed part of the prompt content, resulting in an incomplete generation.