llm-cascade

Tag

Cards List
#llm-cascade

Why naive cosine routing fails: Using Absorbing Markov Chains over a 768D embedding space for <2ms zero-token intent triage (before waking the LLM)

Reddit r/LocalLLaMA ↗ · 2026-10-16

介绍一种基于768维嵌入空间上的吸收马尔可夫链(Absorbing Markov Chain)的语义分流引擎,通过在60节点领域流形上做随机游走来替代朴素余弦相似度路由,实现<2ms的零token意图分类,从而在唤醒LLM之前完成路由。文中给出JevBench基准测试结果:四层级联系统(Gevva0)在240个条目上达到100%召回、89.17%精度,平均p50延迟46ms,并配备否定/语法、流形邻近OOD和熵与边界三重防护网。

0 favorites 0 likes
#llm-cascade

How Often Should a Recommender Call an LLM? Value-Weighted Routing, Monitoring, and Seasonal Robustness

arXiv cs.AI ↗ · 2026-07-29 Cached

This paper introduces value-router, a simulation study for cost-aware routing between cheap heuristics and expensive LLM calls in recommender systems, showing that value-weighted routing improves precision and handles seasonal demand surges with adaptive budgets.

0 favorites 0 likes
← Back to home

Submit Feedback