Tag
The author argues that while LLMs make certain optimizations cheaper, software can still be slow due to factors like increased tolerance for latency in asynchronous workflows and other practical constraints.
An experiment in Seoul showed that deleting 68% of an AI's memory file, specifically adjectives, improved response quality, using OpenAI to grade Claude's memory in a systematic evaluation.
Bitcoin Core pull request reduces index size from 68GB to 26GB, optimizing performance for the next release.
A new open-source project called FreeToken has been released, featuring a research paper and GitHub repository, with tests showing efficient token processing speeds of around 100 tokens per second on consumer hardware.
LLVM 23 delivers a 6.75% reduction in compile-time for -O3 builds, driven by key improvements in ADT hash maps, sets, and other components.
Kent C. Dodds criticizes unnecessary performance optimizations in React, such as using virtualized lists for layouts that the browser can handle trivially.
Sharp v22.1 template reduces Qwen model output tokens by 42% without speed loss, integrated into an NInfer fork for 5090 via C++ modifications.
The article explains how to reduce C++ template bloat by factoring out type-dependent portions of functions, using a practical example with Windows DispatcherQueueHandler to de-templatize code.
This paper explores optimization landscapes in complex neural networks using Kähler geometry and information manifolds, providing theoretical guarantees on descent paths and analyzing effects of Calabi-Yau metrics.
The paper empirically characterizes the learning geometry of hybrid quantum forecasting models, comparing them to classical baselines using Neural Tangent Kernel dynamics and other metrics, showing that similar generalization can emerge from different optimization trajectories.
This paper introduces Best Prefix Selection (BPS), an algorithm for optimal skill selection in LLM agents that provides provable bicriteria guarantees and outperforms existing methods on task success while reducing token usage.
This paper introduces a method for optimizing LLM judge panels by classifying judges as copies, complements, or specialists, and using a role-conditioned allocation policy to route and stop evaluations efficiently based on validation gain thresholds.
This paper introduces a learning-based method to accelerate Mixed-Integer Linear Programming solvers by predicting early-to-final solution consistency, achieving substantial performance gains with Gurobi and zero-shot transfer to SCIP.
Axon is a domain-specific language for shape-safe and framework-agnostic LLM architectures, enabling write-once, run everywhere with benchmarks showing significant speedups over existing frameworks.
The paper introduces a subjectivity coefficient to address limitations in accuracy-based evaluation for LLM-based social simulation, proposes Subjectivity-Adaptive soft-Label Training (SALT) for optimization, and constructs the SubjSim benchmark to evaluate against full response distributions.
This article discusses the parallels between Soviet economic planning and modern data science practices, focusing on issues like resource allocation and simplifying assumptions based on historical books.
The paper proposes a compute-efficient two-step hyperparameter transfer framework to predict optimal learning rates for large Mixture-of-Experts models, enabling efficient pretraining without costly hyperparameter sweeps.
This draft pull request introduces AVX2 optimizations to speed up large batch size prompt processing for IQ models in llama.cpp, an open-source tool for efficient LLM inference.
The article explains an optimized method for parsing IPv4 addresses in C# using AVX-512 SIMD instructions, leveraging .NET 10's masked loads to achieve high performance with UTF-16 strings.
This paper re-examines the edge of stability in deep learning optimization, proposing a new formulation based on directional Hessian and gradient-alignment score for more accurate predictions and diagnostic tools.