Tag
The article argues that LLMs and AI tools are lowering the barrier for software performance optimizations, enabling custom adaptations like JIT compilers and regex engines that were previously too costly to implement.
The post details running the Ornith-1.5-9B AI model on an RTX 3060 with 200k context, achieving high inference speeds using advanced quantization techniques.
The article discusses post-training strategies for AI to reduce costs and improve accuracy in applied use-cases, emphasizing the advantage for companies with deep vertical expertise in enterprise workflows.
The article argues that enterprises should optimize AI intelligence spend by using appropriate-sized models and hybrid systems for different tasks, rather than defaulting to expensive frontier models for all applications.
The article describes GenOS, a custom multi-agent framework that autonomously evolved Rust algorithms to solve the NP-Hard Reverse Game of Life problem, discovering three optimization paradigms and proving a mathematical limit of 378/400.
A user highlights Buun's work on optimizing AI models, achieving high-speed inference of Qwen 3.6 on a single 3090 GPU and developing DFlash2 for Qwen 3.8.
A tool for lossless compression of prompt caches designed specifically for AI agents.
Mimo and DeepSeek have optimized AI models to achieve low pricing with 2-3x profit margins, as detailed in their official blog.
Superpowers 6 open-source project shows that AI can not only write code but also autonomously optimize development workflows (such as auditing, merging tasks, reducing waste). This marks the beginning of AI managing its own workflow, more rigorously than human managers. The article emphasizes that an honest evaluation system (eval) is key to avoiding self-deception.
This guide explains the discipline of AI inference engineering, covering the split between prefill and decoding phases, the shift from closed to open models, and optimization techniques for latency, throughput, and cost.
A discussion on how harness designs can reduce token costs by structuring information instead of feeding everything into a language model's context, citing an example of an RLM agent processing many lines of logs with few active tokens.
Explores the speculative idea of optimizing human interaction with LLMs by conforming to their native communication patterns, such as using neuralese, rather than forcing them to adapt to human language.
The Recursive team released an automated AI research system that can autonomously complete the research loop, surpassing existing human community solutions on multiple benchmarks. For example, on NanoGPT Speedrun it compressed training time from 79.7 seconds to 77.5 seconds, and on SOL-ExecBench it improved the score to 0.754.
Discussion about rewriting parallelism to improve kernel performance using CuTe DSL and tile programming models for the FA4 (FlashAttention 4) kernel.
The tweet explains that running AI models is really about running optimized kernels, and that inference engines and their kernel implementations are critical for performance, not just the model or hardware.
Speculative decoding, inspired by 1990s CPU branch prediction, is now used by Anthropic, Google, and Meta to speed up LLM inference 2-3x. It uses a small model to guess future tokens and a large model to verify them in parallel, avoiding idle GPU time during decoding.
The article argues that companies are collections of algorithms and AI will soon optimize every component, leading to a wave of consulting-led transparency and efficiency.
The article summarizes Andrej Karpathy's advice on reducing AI coding costs by optimizing context usage, avoiding overpowered models for simple tasks, and implementing efficient routing strategies.
Maxime Rivest argues that compound AI systems for images are undervalued and suggests leveraging optimization frameworks like DSPy and GEPA to automate pipeline creation involving SAM and classifiers.
Hermes Agent demonstrates self-improvement capabilities by observing its own performance, identifying inefficiencies, and rewriting its skills to achieve a 3x speedup and 80% cost reduction in just two iterations.