optimization

Transformers and ggml_org collaborate to integrate GGML kernels into the transformers library, enabling GGUF file loading at performance levels comparable to llama.cpp.

Casey Muratori emphasizes the importance of learning assembly language for software optimization in a discussion shared by Gergely Orosz.

This paper formulates speculative decoding in Mixture-of-Experts models as a Stochastic Shortest Path problem and uses a diagnostic Oracle to demonstrate that optimal decisions follow a necessary condition balancing cost and progress.

This paper introduces a hierarchical Bayesian optimization framework for optimizing aircraft-based multi-agent systems-of-systems, applied to wildfire suppression, improving search efficiency and robustness in complex design spaces.

ZoAQ presents an adaptive zeroth-order optimization method that reuses past queries to reduce evaluation counts, saving 43–48% of queries in synthetic tasks and achieving high success in black-box attacks.

The paper proposes a correlation-aware structured pruning method for large language models, explicitly modeling cross-unit dependencies to improve pruning decisions and achieve competitive accuracy-efficiency trade-offs.

Switching the order of question and context in prompts for local Qwen models improved accuracy from 89% to 100% and reduced latency from ~400 ms to ~80 ms on a decision benchmark.

This paper proposes PainNAS, a block-based Neural Architecture Search method that efficiently reduces computational complexity in Leave-One-Subject-Out evaluation for automated pain assessment systems.

This paper introduces Stiefel-AdamW, a geometry-aware optimizer for linear factorization blocks in deep learning that enhances stability and performance, validated on models like GPT2, ViT, and Mistral 7B.

This article explains why the backpropagation algorithm in neural networks is implemented in a backward pass, clarifying the efficiency advantages over a forward computation based on the chain rule.

An experiment demonstrated that a quantized Qwen 3.8 27B model, running locally on a single RTX 3090 GPU, autonomously pursued optimizing CUDA inference for over three weeks, producing functional kernels and benchmarks while maintaining coherent long-term goal-following.

laya.cpp is an optimized C++ implementation for near-instant decision making, built on ggml with custom CUDA kernels, demonstrating significant speed improvements over Python for the Laya model.

This article presents a technique for solving Sudoku puzzles using graph coloring, attributed to Astra.

The article introduces Jev, a decision model from TypeSafe, integrated with Hermes to handle small, non-thinking decisions for AI agents quickly and cheaply, optimizing costs and performance.

Jev introduced on-the-fly compression for AI agents, which scores each tool call to retain important content and delete irrelevant data directly, eliminating the need for model-based summarization and making context compression faster and more lightweight.

The article describes how using the SVE2 SIMD instruction on ARM processors can optimize JSON parsing in the simdjson library by leveraging a character matching instruction for faster performance.

This tweet explains how core AI operations like attention and convolution are based on foundational mathematics such as linear algebra and probability, emphasizing the value of understanding these operations over model names.

This Twitter thread introduces Hugging Face's Kernels, a tool that allows users to select and replace optimized kernel implementations for supported layers in AI models without rewriting the entire model.

This article presents a multi-hour investigation into speeding up local inference for Qwen MoE models using llama.cpp, including experimental results, patches, benchmarks, and reproduction guides, with some workloads showing substantial gains.

The paper characterizes the resource and performance dynamics of LLM-based AI agents across tasks like question answering and coding, revealing bottlenecks and proposing optimizations that improve latency by up to 5.4×.