model-optimization

Tag

Cards List
#model-optimization

@DailyDoseOfDS_: 6 parallelism strategies for LLM inference in production: (popular LLM interview question) Running a model across four …

X AI KOLs Timeline ↗ · 2d ago Cached

Explains six parallelism strategies for running LLMs in production — data, tensor, pipeline, context, expert, and hybrid parallelism — highlighting each one's trade-offs and guidance on choosing based on measured bottlenecks like memory, TTFT, and TPOT.

0 favorites 0 likes
#model-optimization

@rikiyatakehi: We asked GPT-6 Astra to edit Qwen3.5-0.8B weights to improve math. It directly changed only 0.016% of weights, fixing 0…

X AI KOLs Timeline ↗ · 2026-11-02 Cached

A blog post introducing "Weight-Space Programming," where frontier LLM GPT-6 Astra natively reads and edits neural network weights—changing only 0.016% of Qwen3.5-0.8B's weights to boost MATH-500 from 7.9% to 48.1% and tripling IFEval, without gradients or training.

0 favorites 0 likes
#model-optimization

Work Agents: Systems, Capabilities, and Optimization

Papers with Code Trending ↗ · 2026-10-26 Cached

该论文提出了一个统一的“工作智能体”(work agents)框架,将基础模型与运行时框架结合以可靠执行复杂数字任务,引入持久性与开放性两个维度,并系统分析能力族、模型与框架优化及评估方法。

0 favorites 0 likes
#model-optimization

HF Viewer device farm!

Reddit r/LocalLLaMA ↗ · 2026-10-21

HF Viewer 创作者宣布推出 embedl.com,一个收录 1000+ 针对边缘设备优化模型的平台,用户可通过免费的真实边缘设备农场(如 NVIDIA Orin、AGX Thor、RTX GPU)直接运行模型并获得详细性能分析。

0 favorites 0 likes
#model-optimization

Strata + Qwen 3.8 Flash Next: fast, but what's it costing in quality?

Reddit r/LocalLLaMA ↗ · 2026-10-17

一位用户质疑 Strata 的高性能宣传:虽然 tokens per second 和针尖测试数据亮眼,但其剪枝专家(pruned experts)与量化是否牺牲实际输出质量尚未经过验证,建议用 llama.cpp 上的相同 GGUF 来对照测试。

0 favorites 0 likes
#model-optimization

@woosuk_k: Congrats to @reflection_ai on the release! This time, we let AI agents drive model optimization autonomously, with perf…

X AI KOLs Timeline ↗ · 2026-10-16 Cached

Woosuk Kim congratulates Reflection AI on releasing Reflection-Beam-1.0, a 501B-total-parameter (23B active) MoE model with 1M-token context, hybrid attention, and Harmony-format reasoning, highlighting AI agents autonomously driving model optimization with tracked performance gains and Day-0 vLLM/SkyRL support.

0 favorites 0 likes
#model-optimization

From Structure to Validity: Mechanism-Guided Generation and Refinement of LLM Rubrics

arXiv cs.CL ↗ · 2026-10-12 Cached

本文系统分析了LLM评估中rubric设计的机制,涵盖标准组织、操作化与分数聚合,并提出ARTS-Hybrid方法(自适应树构建+语义优先的响应校准细化),在评估性能和rubric质量上均有提升,且生成的rubric可作为模型优化的有效信号。

0 favorites 0 likes
#model-optimization

"Strata" for GLM5.3 Flash is here for some! Project Maya

Reddit r/LocalLLaMA ↗ · 2026-10-11

Project Maya 是一个开源工具,可在本地 GPU 上运行 321B 参数的 GLM-5.3-Flash 模型,通过分层存储(GPU/RAM/NVMe)和自动调优实现单卡 30+ tokens/s 的推理速度,并提供 OpenAI/Anthropic 兼容 API。

0 favorites 0 likes
#model-optimization

@Jason_Young1231: Tibo says that Astra and 6.1sol have seen a 50% speed boost. I tested it out, and uh... 20tps -> 30tps is indeed +50%

X AI KOLs Timeline ↗ · 2026-10-06 Cached

Tibo announces a ~50% speed optimization for GPT-6 Astra and GPT-6.1 Sol across OpenAI products and third-party partners using Sign in With ChatGPT, and a user's test confirms a jump from ~20 tps to ~30 tps.

0 favorites 0 likes
#model-optimization

@lxfater: With 64GB RAM, the 100-billion-parameter model is already deployed. We've got the hammer — now where's the nail?

X AI KOLs Timeline ↗ · 2026-10-04 Cached

A 12GB GPU with 64GB of system RAM is enough to run a 100-billion-parameter large model: the open-source project Strata can run quantized versions of Qwen3.8-Flash-Next — about 94 tokens/s with the Q2_0 quantization and about 53 tokens/s with IQ3_S on an RTX 5070. The author remarks that the hammer is ready; the question is where the nail (i.e., the application scenario) is.

1 favorites 1 likes
#model-optimization

@mylifcc: Discovered a free AI god-tier resource worth a million bucks, and surprisingly, a lot of people still don't know about …

X AI KOLs Timeline ↗ · 2026-10-04 Cached

MIT 6.5940: TinyML and Efficient AI Computing, taught by MIT Associate Professor Song Han, has open-sourced all its slides and videos — covering model compression, TinyML deployment, LLM and diffusion acceleration, and efficient training, with five hands-on labs.

0 favorites 0 likes
#model-optimization

Is all the work that's being put into Qwen3.8 Flash Next going to set us up for a very quick uplift to Qwen4?

Reddit r/LocalLLaMA ↗ · 2026-10-04

The author asks whether the extensive inference optimizations going into Qwen3.8 Flash Next will position Alibaba to quickly uplift those gains to Qwen4 when it releases.

0 favorites 0 likes
#model-optimization

@anemll: Effect of V-compression (int8) in KV-cache on prefill Qwen 3.8-27B, ANE M6 ANEMLL-FORGE. Compression (Quant-dequant) is…

X AI KOLs Following ↗ · 2026-10-02 Cached

ANEMLL-FORGE reports on applying int8 V-compression to the KV-cache of Qwen 3.8-27B during prefill on Apple's ANE M6, with compression and dequantization performed on ANE to yield memory I/O savings.

0 favorites 0 likes
#model-optimization

@ibab: The team at Ivo is incredibly talented and moves at light speed. It's been a pleasure to help them take their contract …

X AI KOLs Following ↗ · 2026-10-01 Cached

River AI promotes its frontier AI development stack, claiming it helped Ivo's contract review agent jump from 70% to 91% on the LAB benchmark at lower cost than competitors, with RL training and inference on optimized open-weight models via River Cloud or on-prem clusters.

0 favorites 0 likes
#model-optimization

Dwarfstar quants

Reddit r/LocalLLaMA ↗ · 2026-10-01

A user shares early impressions of Dwarfstar, a tool offering bespoke quantization techniques for running large language models locally, reporting fast performance for Qwen3-8B on an M3 Ultra 96GB Mac Studio with leftover memory headroom.

0 favorites 0 likes
#model-optimization

The harness doesn't make the model smarter. It stops it from repeatedly becoming stupider.

Reddit r/AI_Agents ↗ · 2026-09-30

The article argues that harnesses for AI models don't boost innate reasoning but steer models to maintain performance, with examples showing a significant improvement on ARC-AGI-3 through state retention and compaction.

0 favorites 0 likes
#model-optimization

@MaximeRivest: Here is when and how to finetune a specialized decision model (aka classifier) that is better then Opus, faster then Je…

X AI KOLs Timeline ↗ · 2026-09-27 Cached

The article explains when and how to finetune a specialized decision model classifier that outperforms Opus, is faster than Jev, and runs on user devices.

0 favorites 0 likes
#model-optimization

@Anbeeld: BeeLlama v0.4.7 is out! You can now save gigabytes of VRAM when using MTP and DFlash! In the example shown in the image…

X AI KOLs Timeline ↗ · 2026-09-25

BeeLlama v0.4.7 is released, offering gigabytes of VRAM savings through independent drafter ubatch sizing for MTP and DFlash, along with major ROCm and CUDA performance enhancements for AI inference.

0 favorites 0 likes
#model-optimization

@_akhaliq: Pruna-Qwen-Image-2.1 a set of a few-step LoRA adapters that make Qwen-Image-2.1 up to 6.3× faster for image generation …

X AI KOLs Timeline ↗ · 2026-09-24 Cached

Pruna-Qwen-Image-2.1 is a set of LoRA adapters that significantly speed up image generation and editing in Qwen-Image-2.1, reducing steps from 40 to 5 or 8 for up to 6.3x faster performance.

0 favorites 0 likes
#model-optimization

Dynamic Quantiser - a way to make your own high quality dynamic quants

Reddit r/LocalLLaMA ↗ · 2026-09-22

Dynamic Quantiser is an open-source tool that enables users to create custom and high-quality dynamic quantizations of LLM GGUF files, minimizing cosine deviation for better accuracy compared to standard quants.

0 favorites 0 likes
Next →
← Back to home

Submit Feedback