Use context profiler to optimize your LLM calls and reduce token use

Hi all. After getting inspired at the local PyCon conference, I am working on a new tool for LLM applications and coding agents - a context window profiler: [https://github.com/RimantasZ/contextspy](https://github.com/RimantasZ/contextspy) All the talk now is how to reduce token usage (to either reduce API costs, or speed up local inference), and there are a myriad of tools aimed at solving this automatically - from caveman mode to various token compressors. ContextSpy is a profiler tool for analysing context usage of LLM applications. It is implemented as a local proxy that sits between your coding agent and the LLM API. It records every request and breaks down where the input tokens are going — system prompt, tool definitions, file contents, conversation history, and so on — so you can see how the context window is actually being used. This approach allows optimising token use from the other side - similar to how CPU or memory profiler is used to identify performance bottlenecks or memory leaks, ContexSpy allows reviewing what is in the context and making a decision if all that info is really necessary. It is still in the early stages of development, so any feedback is very welcome - be it someone testing it in their setup, registering some issues (of which there are still plenty), dropping a comment here, or placing a star to keep me going through those sleepless after-work hours :) https://preview.redd.it/kfpp1mryku6h1.png?width=4060&format=png&auto=webp&s=05b2afc5182559a4471860aed573f246e1ee4e82 https://preview.redd.it/lpvlnjmzku6h1.png?width=3254&format=png&auto=webp&s=a986915efb1bbdacbcc1105055e4f572b942783c