Well it finally happened: we’re not using models because of cost

Hey all! I’ve been working for a large company (Fortune 500) for 3 years now. We’re an “AI First” company. When I onboarded it was made explicit to me that it was absolutely imperative that I was not only allowed to use Artificial Intelligence in my work but the expectation was that all my work would involve AI. My org was quite serious about the process. For a full year we all had training. Developer, managers, sales people. Everyone was trained on the tooling. We were “full steam ahead”. Everyone was given Copilot and Claude. We had biweekly demos of people sharing use cases for AI in various projects around the company. We eventually had a very large project using AI. It was a pilot project to discover a workflow for AI and see what its capabilities were. Unfortunately, the project failed. The project was a rewrite of a legacy application. We asked the Agents to define the rules of the system “as written” by the code. However, the Agents constantly missed small details and had issues just writing a clear specification of the code written. The thinking was the rewrite would be first, essentially using the legacy code to define a specification document for the new system. We couldn’t even get there. The business rules were apparently too complex. Nonetheless, leadership’s position was still optimistic. They saw the project as the first step in a series of steps. Our lead architects were to take what they learned and apply that to future opportunities. In the meantime, we continued using AI to solve our problems from the day to day. Personally, I kept having mixed results. It was amazing for tiny refactors but for larger things it would just do insane stuff. The most egregious so far was SQL code that was dropping constraints on all the tables the code was doing inserts and deletes into. Which is, special. Of course we have proper dev practices so that code never saw production but it was not uncommon for generated code to sometimes just be weird. Finally, today I find out we’re pulling back on AI. We have lost access to Claude and we’re being told to limit usage because of costs. The company is still encouraging AI usage but we have stopped the training and demos. Our architects have told us to just use older models for tasks. The reason this hit my radar was because Ed Zitron, who has reported extensively on AI has been saying for a while that AI will be in trouble because the costs will be too high and now I’m seeing a lot more conversations about the costs of AI at work. If AI is to have any success long term they’re going to need to get the costs down.