A.L.F.R.E.D. - 2B models with template can match 35B Models with 4x less speed

https://preview.redd.it/qd7drw8mqfdh1.png?width=2223&format=png&auto=webp&s=4df7e6f4002cf6c6508463f40ebdb6a39c6d1805 https://preview.redd.it/86s7i9dpqfdh1.png?width=2411&format=png&auto=webp&s=3f69ccc368fe0212fb4392f14b0d60e38261e186 So My idea is to instead of spending bigger models compute on easier tasks, why we just not "distill" bigger models knowledge into small ones and if we hit this task again, just route to small model, which already knows how to acts, just requires different context for example: "set alarm for 7:00" our system does not have this pattern in db so we call big model to resolve query and after success, we ask again same model to create pattern from it. when second time we hit the same request "set alarm for 8:30" instead of asking big model, reading skill, going step by step and validating (spending roughly 7k token on simple task) we just pass this query to small model with pattern and it executes operation, which is ~1k token, no skills, anything. Just like human brain operates, delegating with reasoning or reflexing. I know it is more or less like routing, but it adds extra steps to make everything even smaller, more deterministical, which gives us speed and more validation I call it A.L.F.R.E.D. Adaptive Local-First Routing and Execution Distillation Here is all the benchmarks, thesis fully and so on https://github.com/LeonardoDaviti/alfred