Tag
World Model Optimizer is an open-source CLI tool that optimizes AI models from agent traces and serves them with a router to maintain frontier model quality at reduced cost.
A 1-bit quantized version of the Hy3 295B model achieves 2.2x faster inference speed compared to the cloud API with no quality loss.
DFlash is a method that accelerates Qwen3.6 27B model inference by 2.2x without quality degradation.
A fix splitting a fused operation into two matmuls allows Gemma 4 to run on a 13-year-old dual-Xeon CPU at 5.2 tok/s, using only 4B of 26B weights per token.
Unsloth releases a new version of Qwen3.6 27B NVFP4 quantization scheme, introducing FP8_E4M3 intermediate precision layer and refined weight protection, achieving 2.5x speed improvement on 24GB VRAM, while improving accuracy and tool-calling capabilities.
This paper proposes a latency-oriented neural network learning method that uses zerorized batch normalization to optimize DNN architectures for edge systems under strict latency constraints. Experiments show significant latency reduction with minimal accuracy loss on NVIDIA Jetson devices.
Explains how KV caching speeds up LLM inference by eliminating redundant recomputation of attention keys and values, trading off speed for memory, and introduces production-scale cache management challenges.
A workflow strategy for capturing and reusing AI model outputs, including setting quality bars, strategy roadmaps, knowledge notes, and reasoning records to reduce reliance on frontier models.
LivePortrait has been distilled into a model that can run at 25 frames per second directly in the browser, as shown in a Hugging Face Spaces demo by slperez.
A discussion on using applications to enhance the effectiveness of smaller AI models on larger tasks, balancing efficiency and performance.
This paper systematically studies layer-wise contribution in RL post-training for LLMs, finding that training a single middle transformer layer can recover or even surpass full-parameter RL gains, with consistent patterns across models and tasks.
Nvidia's optimized Qwen 3.6 27B NVFP4 model achieves 41% faster single-session inference and 23-25% faster concurrent inference on DGX Spark compared to Unsloth's version.
SkillOpt turns AI agent skill editing from manual modification into a training process, improving agent reliability without changing model weights, achieving consistent gains across benchmarks.
DeepSeek proposes the DSpark technique, which implements speculative decoding by inserting a mini Transformer after the Final RMSNorm, boosting large model output speed by 60%-85%.
Whisper large-v3-turbo has been compressed to 368 MB using Q3_K-matched quantization-aware training, with multilingual word error rate results reported.
Explains that super weights in large language models arise from the SoftMax-Attention interaction creating a 'Nothing Dump' token that serves as a stable reference point; removing these weights cripples performance.
A new paper investigates whether it's better to prune a larger LLM or train a small LLM from scratch, finding that pruning provides more than just a good initialization.
Prime Intellect releases prime-rl v0.6.0, enabling efficient reinforcement learning at trillion-parameter scale on large Mixture-of-Experts models, with sub-5-minute step times and optimizations for asynchronous RL.
Discusses using NVFP4 4-bit floating point weights for maximum performance, achieved via in-house quantization from FP8 using NVIDIA ModelOpt, highlighting the data format's dual scale factors for high dynamic range.
GLM-5.2 adopts MTP (Multi-Token Prediction) technology to accelerate inference and fixes a training-inference discrepancy in GLM-5.1's MTP that caused KV cache mixing issues.