optimization

vLLM v0.28.0 is an updated release of the open-source library for fast and efficient LLM inference and serving, featuring enhancements like PagedAttention and broad hardware support.

The author released mainline-compatible imatrix quants for the Qwen3.8-Flash-Next model, offering 20–30GB smaller sizes with competitive PPL, and includes a separate ROCm build for AMD hardware.

Someone has optimized the Minimax H3 Max AI model to run faster than real time, creating a Rick and Morty style interdimensional cable stream that frequently gets taken down.

This article presents official 1-bit quantization builds for the Hy4 preview model, offering GGUF files with reduced sizes and instructions for running on patched llama.cpp.

A social media post listing top AI models for 2026 hardware configurations, such as lfm-2.6B for 8GB setups and Ornith models for higher-end devices.

The paper explores whether improved coordination between humans and large language models can reduce token costs without modifying the model itself.

The author creates a satirical standard called cats.txt to critique the weak evidence supporting llms.txt in GEO, showing it passes flawed tests similar to current practices.

The author details converting the scheme-rs Scheme implementation to a CPS-based JIT compiler and applying three optimizations, including beta-reduction, to achieve a 25x performance improvement.

This paper introduces a composite metric for quantization in small large language models that balances speed and quality by combining information retention and throughput gains, evaluated on models like Gemma 3 1B.

The paper proves that in stochastic SCAFFOLD for federated learning, client averaging suppresses the leading stationary bias but leaves a client-independent second-order bias component due to control fluctuations and nonquadratic curvature.

This paper proposes an approach to enhance distributed training over wide area networks by leveraging multicast technology and in-line FPGAs, along with an optimization framework for synchronization schedules, to maximize information exchange and reduce performance gaps.

This paper analyzes how finite Newton-Schulz iterations in the Muon optimizer benefit nonsmooth nonconvex optimization by smoothing the polar map, providing convergence guarantees that match best-known bounds.

Introduces HarnessOpt-Bench to measure recursive self-improvement in AI, evaluating 5 frontier models on 4 tasks and finding that model choice has a greater impact than coding harness choice.

Engrams are an architectural innovation that uses N-gram tables to offload memorization from transformer models, allowing smaller models to reason better by freeing up parameters for computation.

This paper surveys 200 works on LLM quantization, formalizing the 'Great Inversion' principle that contrasts energy concentration in coding with within-group flattening in quantization, and offers a guide for transform selection based on deployment regimes and formats like MXFP4 and NVFP4.

This paper introduces an AI-enhanced method for RF interference suppression using transformer models with a Finite Scalar Quantization tokenizer to boost performance and minimize latency, validated via audio metrics in digital signal scenarios.

The paper resolves contradictions in using neural networks for derivative-free optimization by identifying three key factors—role, radius, and room—that determine when learned local models improve performance in simulation optimization.

AutoVerifier introduces a residual-guided non-parametric optimization method to learn and apply biases in answer verification, improving accuracy over state-of-the-art verifiers.

Successfully ran the 111GB Qwen3.8-Flash-Next GGUF model on dual RTX 4090 with 96GB VRAM, achieving a generation speed of 66 tokens/s through PLE and sparse attention optimizations, with detailed explanations of quantization and runtime configuration.

The article introduces a method to serve Markdown content directly to AI agents using HTTP Accept headers, reducing token usage, improving retrieval efficiency, and lowering latency for AI processing.