Tag
A tool for lossless compression of prompt caches designed specifically for AI agents.
Mimo and DeepSeek have optimized AI models to achieve low pricing with 2-3x profit margins, as detailed in their official blog.
Superpowers 6 open-source project shows that AI can not only write code but also autonomously optimize development workflows (such as auditing, merging tasks, reducing waste). This marks the beginning of AI managing its own workflow, more rigorously than human managers. The article emphasizes that an honest evaluation system (eval) is key to avoiding self-deception.
This guide explains the discipline of AI inference engineering, covering the split between prefill and decoding phases, the shift from closed to open models, and optimization techniques for latency, throughput, and cost.
A discussion on how harness designs can reduce token costs by structuring information instead of feeding everything into a language model's context, citing an example of an RLM agent processing many lines of logs with few active tokens.
Explores the speculative idea of optimizing human interaction with LLMs by conforming to their native communication patterns, such as using neuralese, rather than forcing them to adapt to human language.
The Recursive team released an automated AI research system that can autonomously complete the research loop, surpassing existing human community solutions on multiple benchmarks. For example, on NanoGPT Speedrun it compressed training time from 79.7 seconds to 77.5 seconds, and on SOL-ExecBench it improved the score to 0.754.
Discussion about rewriting parallelism to improve kernel performance using CuTe DSL and tile programming models for the FA4 (FlashAttention 4) kernel.
The tweet explains that running AI models is really about running optimized kernels, and that inference engines and their kernel implementations are critical for performance, not just the model or hardware.
Speculative decoding, inspired by 1990s CPU branch prediction, is now used by Anthropic, Google, and Meta to speed up LLM inference 2-3x. It uses a small model to guess future tokens and a large model to verify them in parallel, avoiding idle GPU time during decoding.
The article argues that companies are collections of algorithms and AI will soon optimize every component, leading to a wave of consulting-led transparency and efficiency.
The article summarizes Andrej Karpathy's advice on reducing AI coding costs by optimizing context usage, avoiding overpowered models for simple tasks, and implementing efficient routing strategies.
Maxime Rivest argues that compound AI systems for images are undervalued and suggests leveraging optimization frameworks like DSPy and GEPA to automate pipeline creation involving SAM and classifiers.
Hermes Agent demonstrates self-improvement capabilities by observing its own performance, identifying inefficiencies, and rewriting its skills to achieve a 3x speedup and 80% cost reduction in just two iterations.
The author demonstrates that migrating a LivePortrait implementation from MPS to Apple's MLX framework on an M5 Max chip results in significantly better performance and speed.
An enterprise agent developer discusses the trade-offs of using open-source models like Ling 1T 2.6, highlighting the high overhead of optimization and benchmarking compared to proprietary APIs.
RunInfra is a service that allows users to describe their AI model requirements and receive an optimized AI model.
A developer adapted Karpathy's autoresearch framework for automated software engineering, implementing multiple optimizations that yielded remarkable results.
DeepMind's Deep Tank AI system optimizes the growth of 2D semiconductors, achieving crystals measuring 130 micrometers—surpassing the 100-micrometer target—and significantly accelerating the parameter search process for material fabrication.