I tested every frontier model from every AI lab - Claude Fable 5, GPT 5.6 Sol, Kimi K3, GLM 5.3, Qwen 3.8 Max, DS v4 Pro, Grok 4.6 and just 1 made it through.

The task as well as the prompt was simple. I did not ask them to make me an OS or a complete game. I just wanted to test their intelligence on something very basic. I have a repo with very large log files, 20-30 GBs each. My working directory AGENTS.MD has a lot of instructions which I have included on how to handle those files. For the test, I created completely new workspaces (folder) for every agent and gave them the exact same prompt. I sym-linked the logs directory in every workspace and there was nothing else in the workspace which could guide the agent. Then I asked each of those agents to extract a particular data from those log files. The result: GPT 5.6 Sol, Kimi K3, GLM 5.3, Qwen 3.8 Max, DS v4 Pro, Grok 4.6 - all of them crashed the harness by loading these large files to memory. Only Fable 5 was smart enough to think, hey, those are large files; and built a script to stream the data and completed the task successfully. Even if we ignore all the other models, I always assumed that GPT 5.6 Sol is as intelligent as Fable 5 and have been using it as my primary model for work, but this simple test tells me that Fable 5 is actually more intelligent than Sol. "Not to load large files into memory" is something even a junior dev knows. I have 16GB RAM on my machine with around 50% free when I ran these tests. I always excitedly look for tests when a new model is released and they are always the same - Stork on a bicycle, couple of website mockups, or numbers from tests like HLE. Design perception is subjective so it is hard to actually judge the models on that. There are no coding tests available and if there are I am not aware of them and even those would be subjective. For me, I have found my own test on which I will test every new model from now on. Do you guys also have your own custom tests which you perform on AI models to gauge their performance?