OpenAI is shutting down GPT-4o and GPT-4.1 on Jan 14. We tested the cheaper Luna models task by task first. Results were mixed.

If you still have GPT-4o, GPT-4o mini, GPT-4.1, or GPT-4.1 mini in production, you have until January 14. Fine-tunes go too. OpenAI’s recommended replacement for all four is GPT-6 Luna. Our team at BotsCrew has been moving part of a production workload from GPT-5.2 to GPT-5.6 Luna, so we already had numbers on how the cheaper Luna models behave. Nazarii, one of our AI engineers, ran the evals. Short version: it depends a lot on the task. How we tested A few hundred requests and conversations run on both models A separate LLM as judge, blind to which model wrote what: is it correct, is it grounded in our data, which answer is better Separate checks on tool use and actual cost per call Results (GPT-5.6 Luna vs GPT-5.2) Message classification: same quality, ~88% cheaper Data extraction for analytics: same, ~85% cheaper Templated content with RAG: better, fewer invented facts, ~83% cheaper Short conversation summaries: better, the judge picked it most of the time, ~91% cheaper Long user conversations with lots of tools: worse. It guessed where 5.2 said “can’t verify that.” (~79% cheaper) Guardrails: worse. It missed roughly 1 in 10 harmful messages. 5.2 missed none. So the first four moved. Conversations and guardrails stayed put. Stuff that bit us Both Luna models only run at the default temperature. Setting it isn’t supported. GPT-6 Luna, through Chat Completions, only does tool calling with reasoning_effort “none”. 5.6 Luna doesn’t care. Prompts barely changed, except for one task where Luna insisted on Title Casing Every Single Headline. If you’re migrating off 4o/4.1 Go one request type at a time, with evals before and after Ask questions your data can’t answer. That’s where the cheaper model started making things up For guardrails, count misses, not accuracy. Accuracy can go up while misses go up, too Test long dialogues. Cheaper models slip as context grows Check real spend. Cache and reasoning tokens change the bill GPT-6 Luna is ~95% cheaper than 5.2 but noticeably weaker than 5.6 Luna in our experience, so we’d only use it where price matters most. That said, these are our numbers on our workload. As Nazarii put it: don’t trust the price page, the benchmarks, or even our evals. Run the models on your own use cases before you switch anything.