Tag
MiniCache is a program caching framework that reuses computation across similar requests by parameterizing Program-of-Thought programs, using small models for semantic variable extraction and speculative drafting to improve LLM inference efficiency.