Measuring switching a production workflow from GPT-5.3-codex to Minimax M3

Reddit r/AI_Agents News

Summary

A production team migrated their QA agent from GPT-5.3-codex to MiniMax M3, finding that while the new model uses more tokens per task, its lower per-token price led to a 55% median cost reduction. The post also highlights the importance of inference provider selection and hidden reasoning tokens affecting effective pricing.

We run browser-driving QA agents in production: they execute test plans against real applications in devcontainers. In late June we migrated our highest-volume agent from GPT-5.3-codex to MiniMax M3. Because we meter every agent step, the migration left us with a clean natural experiment across both models on the same workload: over half a trillion tokens in total. TL;DR: median LLM cost per pipeline run fell 55%. MiniMax needed 2.5x the tokens to do the same work, and the choice of inference provider mattered almost as much as the choice of model. Cost per run fell by more than half Cost per test-runner step, indexed to the codex median (= 1.00): Percentile GPT-5.3-codex MiniMax M3 Change p50 1.00 0.40 −60% p90 2.16 1.39 −36% p99 3.66 3.05 −17% The effect holds across the full distribution rather than only at the median. Per pipeline run (a run contains several steps), the median fell 55%. The model we adopted is the less efficient one The result we did not anticipate: by every physical measure, GPT-5.3-codex does the same work with less. On a matched sample (156 succeeded steps per model, same repositories, same kinds of test batches, reconstructed from session transcripts): Median per test batch GPT-5.3-codex MiniMax M3 Ratio Agent turns 94 116 1.23x Tool calls 118 134 1.14x Output + reasoning tokens 21.5k 46.8k 2.18x Input tokens (incl. cache reads) 5.4M 13.5M 2.51x Cost (indexed) 1.00 0.46 0.46x An evaluation ranked on turns or tokens to completion would have chosen codex. Our invoices chose MiniMax: at 3-12x lower per-token prices, the inefficiency is absorbed with room to spare. Token efficiency and cost efficiency are different axes, and they can point in opposite directions. Reasoning tokens put codex 58% above its list price We fit our per-step bills against recorded token counts (under 2% residual error) to get effective per-Mtok rates: Rate ($/Mtok) List Measured effective GPT-5.3-codex input $1.75 $1.72 GPT-5.3-codex cached input $0.175 $0.19 GPT-5.3-codex output $14.00 $22.12 MiniMax M3 input $0.30 $0.33 MiniMax M3 cached input $0.06 $0.06 MiniMax M3 output $1.20 $1.05 The discrepancy is reasoning tokens: they are billed as output but excluded from visible token counts. MiniMax reasons in-band, and its effective rates match its published prices almost exactly. Derive effective per-token rates from invoices, not from pricing pages. What the savings cost Metric GPT-5.3-codex MiniMax M3 Median step duration (wall-clock) 23 min 29 min Step failure rate (share of terminal states) 2.4% 4.5% Per-repo savings vs codex era baseline −81% to −13% Durations include non-LLM work (browser automation, test execution) that did not change between eras. Retries absorb most of the additional failures before they become user-visible, but retries are not free. They are part of why realized savings (55%) are smaller than the raw 3-12x price gap. The per-repository spread is real: context-heavy repositories with long agent sessions save the most, because cache reads dominate their bill; output-heavy repositories save far less. The same model is not the same model on a different provider "Open weights" suggests a commodity: the same model wherever you buy it. In practice, sampling settings, quantization, context handling, and above all prompt-cache behavior vary by provider, and most of those parameters are not visible to you. We tested GLM-5.2 and MiniMax M3 on Vercel AI Gateway, OpenRouter, and Together AI: Vercel AI Gateway had by far the worst cache hit rate for both models and scored worst on our internal evals, against the same underlying providers reachable through OpenRouter. OpenRouter routing to Together AI was substantially worse than calling Together AI directly, on both cache rate and reliability. Same weights, same named provider, different behavior depending on the layer in between. Our working hypothesis is that serving settings do not survive the trip through aggregation layers intact. We ended up going direct. For agent workloads, cache behavior is not a nicety: about 97% of our input tokens are cache reads. A provider (or layer) that quietly degrades caching multiplies your bill by more than the model choice does.
Original Article

Similar Articles

The MiniMax-M2 Series: Mini Activations Unleashing Max Real-World Intelligence

Hugging Face Daily Papers

The MiniMax-M2 series introduces Mixture-of-Experts language models that achieve high performance on agentic tasks with minimal activated parameters (9.8B per token out of 229.9B total), leveraging agent-driven data pipelines, a scalable RL system called Forge, and a checkpoint that takes early steps toward self-evolution.