Tag
作者认为为每个智能体固定绑定单一模型将显得奇怪，因为同一次智能体运行中的不同环节需要不同的智能水平，简单任务保持低成本，困难决策则临时调用前沿模型，使智能体呈现异构化而非模型本身同质化。
作者指出企业 AI 项目中 Agent 失败往往不是模型质量问题，而是上下文、工具描述、权限、状态管理和可观测性等周边环节的缺陷，并提出 Agent 架构应包含验证、执行、观测和人工兜底等完整流程。
This paper analyzes the mechanics of long-context hybrid LLMs combining full attention with sliding-window (SWA) or gated linear attention (GLA/GDN), identifying a Seesaw Effect in context extension and proposing Sliding-Window Linear Attention that achieves 16x training-free length extrapolation with 100% NIAH-SK1 accuracy at 64k context.
A developer is building an agentic Text-to-SQL pipeline for querying financial databases and seeks community feedback on the architecture's robustness, security, and performance.
The author argues against making LLMs the central component in agent runtimes, proposing a split between deterministic and semantic paths for handling requests, and seeks feedback on this architectural idea.
The article releases FP8-quantized weights for the Qwen3.8-Flash-Next model, introducing architectural innovations like Hybrid Attention with QSA, Gated Residual, and N-gram Embedding to improve efficiency in large language models.
Gemini provides a systems-engineering analysis of Dex's architecture, highlighting its stateless transformer core augmented by a state-management system to solve LLM amnesia, and distinguishes engineering achievements from ontological claims.
Sebastian Raschka provides an architectural overview of the open-weight Kimi K3 model, highlighting its scaling from 48B to 2.8T parameters, new LatentMoE and attention residual components, removal of RoPE in favor of NoPE, and native multimodal support. The model emphasizes inference efficiency and matches frontier performance.
Looped transformers reuse the same layers across multiple passes to trade parameter count for compute, achieving better reasoning with fewer weights. The article traces the idea back to the Universal Transformer (2018) and explains why it initially failed due to scaling laws and timing.
The article discusses how LLMs have grown increasingly complex, moving beyond simple transformer stacks to incorporate diverse attention variants, mixture-of-experts, and multimodal encoders, drawing parallels with recommendation systems and emphasizing the need for composable kernel optimization like FlexAttention.
A Stanford professor delivered a public lecture providing a comprehensive breakdown of how modern LLMs like GPT, Claude, and LLaMA are built under the hood, making advanced architecture accessible to the public.
A tweet criticizes current LLM architecture for wasteful recomputation due to order-dependent context, and proposes encoding context units separately to enable order-invariant, efficient caching and generation.
This article provides a comprehensive step-by-step breakdown of how modern Large Language Models like ChatGPT and Claude are built from scratch, covering data collection, tokenization, transformer architectures, training, alignment, and deployment.
The article shares a personal experience using Cursor and Opus 4.7 to generate videos, highlighting the agent's rigorous self-correction process. It then transitions into a technical discussion on 'Agent = Model + Harness,' arguing that engineering systems like ratchets and context management are more critical to AI agent performance than the underlying model alone.
The author describes improving AI agent reliability by replacing a single general-purpose agent with a four-agent workflow specializing in intake, research, action, and review. This shift prioritized system predictability and easier debugging over raw autonomy.
This paper introduces TIDE, a method that addresses the Rare Token and Contextual Collapse problems in LLMs by injecting token identity into every layer via Embedding Memory. The authors demonstrate theoretical and empirical improvements across language modeling and downstream tasks.
A social media post discusses the technical implication of applying RoPE rotation directly to KV caches, noting that it leaks positional information into the value matrix V.
This interactive tool visualizes the mathematical underpinnings of transformer models through dataflow graphs, covering architectures from GPT-2 to Qwen 3.6 and various attention mechanisms.
StageMem proposes a lifecycle-managed memory framework for language models that organizes memory into transient, working, and durable stages with explicit confidence and strength metrics, treating memory as a stateful process rather than a static store to better manage retention and forgetting under bounded capacity.
Stanford University offers a 1.5-hour lecture on LLM architecture covering fundamental concepts and design principles of large language models.