Tag
Juzi (Leo Feng) shares his first step in doing GEO in the AI era: get his personal website in shape so every AI can get to know him, while exchanging links with fellow group members—playing around with a “Web 2.0-era version of GEO.”
Intel and AMD jointly announced they will collaborate on x86 CPU software optimizations for PyTorch via oneDNN, with AMD contributing its ZenDNN improvements upstream and joining the UXL Foundation as a steering member.
The article shares a dedicated chapter on inference from the book 'How to scale your model', covering key optimization techniques for transformer models such as KV cache and latency analysis.
Splash 1.1.0 has been released with GGUF quantization support, MLX import, and optimized kernels for Apple Silicon, enabling efficient local AI inference at high speeds.
PrismQuant introduces a quantizer-aware rotation framework that aligns activation eigenspace with grouped INT4 quantizers, achieving state-of-the-art performance on large language models like Llama and Qwen, with significant speedups in inference.
A user shares their experience replacing Claude AI with Qwen 3.8 Next on local hardware, discussing performance and plans to optimize other models like DeepSeek V4 and GLM Flash.
The article describes how to use the 'make tune-kernels' tool in Splash to optimize AI model decoding on 40-core M5 Max chips, achieving up to 20% speed improvement by tuning kernel layouts for specific hardware.
Mara Chain 提出了一种新的 AI 系统自动进化框架，将被淘汰的候选方案及其失败分析保留下来迭代优化，而非直接丢弃。在 AppWorld、TerminalBench 2.1 和 MuSiQue 等基准上，以更少的 rollout 预算显著超越了 GEPA、AHE 等先前优化器，并在 GLM-5、DeepSeek、Qwen 等多种骨干模型上验证了通用性。
The tweet argues that AI optimizes existing informatization, but human economic activities will shift toward incremental informatization for marginal benefits, while organizational forms like companies will persist.
DPara is a parallel speculative decoding framework that eliminates probabilistic fallback by precomputing draft representations, achieving average speedups of 3.21× to 3.52× over autoregressive decoding on Qwen3 models.
The tweet discusses how providers tune quantization and speculative decoding strategies post-launch based on usage patterns and hardware, and introduces an interactive tutorial on speculative decoding for the NeurIPS Education Track.
Claude has optimized the inference performance of over 30 open-source biomolecular models, boosting their average running speed by 4 times with custom GPU software and open-sourced code.
The article explains the engineering aspects of Mixture-of-Experts (MoE) inference, detailing token routing, expert batching, GPU distribution, and performance trade-offs for efficient serving.
Anthropic optimized inference for over 30 open-source biomolecular models using Claude, achieving an average 4x speedup, and is open-sourcing the optimization code. They are also partnering with Adaptyv Bio on a protein design competition.
Intel releases OpenVINO 2026.4 with expanded AI model support, performance enhancements like multi-token prediction, and new features for profiling and inference across CPUs, GPUs, and NPUs.
Meta Engineering extends FlashAttention-4 with MXFP8 support for NVIDIA Blackwell, achieving up to 2.85 PFLOP/s forward performance and integrating into production training workflows like GEM.
This article explains how the prefilling and decoding phases operate in AI models and their effects on generation speed and performance, aiding in understanding the reasons behind AI response latency.
Cherenkov is a new inference engine for Apple Silicon that enables efficient memory-constrained inference of large AI models like Qwen3.8-Flash-Next using predictive expert streaming and mixed-precision execution.
React Native ExecuTorch v0.10 achieves significant speedups, up to 92x for AI models like FastSAM, with TypeScript pipelines for better inspectability and cross-platform support.
A study demonstrates a 46% error reduction by routing requests across 44 LLMs, optimizing performance on 16 benchmarks like TerminalBench and LiveCodeBench.