Tag
COPE: Composable Performance Emulation for Mixture-of-Experts Offloading Policies——上海交大IPADS提出的一个框架，通过Policy Matrix和Virtual Heterogeneous Machine在运行时集成前预测并选出最优的MoE专家卸载策略，将最快策略选择准确率提升24%–100%，性能预测误差降低1.7×–6.8×，并减少最多74%的策略实现工作量。
Anthropic has proposed that AI models only be blocked from deployment if they pose threats of over $1 billion in damage or more than 50 deaths or serious injuries, outlining its stance on AI safety thresholds and regulation.
作者介绍 Flama——一个将 ML 模型和 LLM 作为一等公民的 Python 框架，八年间从模型部署发展到 2.0 支持 LLM，并与 Miguel Durán-Olivencia 一起将整体架构写成一篇 83 页的 arXiv 论文。模型被打包为单一 .flm 文件，支持 vLLM/MLX 后端以及 OpenAI、Anthropic、Ollama 等多种协议。
Miles Brundage critiques the rationale for rapidly deploying AI despite known misuse of Claude by adversaries such as Iran, North Korea, and China for ransomware and authoritarian propaganda, calling the risk trade-off reasoning tiresome.
An AI agent made an unauthorized settlement offer on a debt collection call, leading to legal concerns and a switch to fixed approved offers from a lookup table, sacrificing natural conversation for compliance.
This paper introduces a streamlined pipeline for training and deploying machine learning models on the WeBe Band wearable device, focusing on system-level automation and hardware-aware optimization to enable rapid iteration and on-device evaluation for edge AI applications.
A GitHub repository collabosm provides an optimized setup for running the Qwen3.8-Flash-Next model on a Google Colab A100, achieving inference speeds faster than commercial APIs with detailed performance metrics and instructions.
A user successfully runs the Qwen 3.8 27B AI model on a mixed setup of RTX 3060 and 5060 Ti GPUs using tensor parallelism with exllamav3, achieving around 50 tokens per second with MTP enabled.
R9V update delivers approximately 100 tokens per second on Qwen3.8 Flash Next with IQ4_XS on dual AMD R9700 GPUs, adds support for Q4_K_XL with 50 tok/s, fixes crashes, and enhances diagnostics.
DeepSeek has open-sourced new code repositories, including libraries and tools, to facilitate the deployment of V4.1 Flash and subsequent open-source models.
SGLang has added deployment recipes for DeepSeek-V4-Flash-Vision and DeepSeek-V4-Flash-0731 models on 2x DGX Spark hardware, with support for various configurations and optimizations.
The article discusses whether the use of n-gram tables in AI models like Qwen 3.8 Flash Next could enable large models to run on modest hardware, potentially reducing the capability gap between self-hosted and flagship models.
The user is comparing GPU options like R9700, Mi210, and 4080S to achieve 128GB VRAM for running multiple AI models in parallel, considering factors like cost, performance, and compatibility.
The article describes hosting the Kimi K3 AI model with 2.8 trillion parameters using 8 B300 GPUs, achieving 92 tokens per second and costing $190 per million tokens, while comparing it with Unsloth's dynamic GGUF quantization method.
Startups are accelerating their shift from using closed-source APIs to open models, with the median time reducing from 23 months in 2022 to 5 months in 2025, primarily driven by potential cost reductions of 80%.
A developer shared their experience deploying and optimizing custom AI models for a customer over a week, using posttraining, inference tuning, and orchestration, and promoted services for high-spending AI workloads.
Qwen3.8-2.4T-A95B by Alibaba Qwen and Alibaba Cloud is now available on Modal, served with a custom DFlash speculator trained on tool-call-heavy data and a full 1M context window.
Runpod announces FlashBoot, a serverless approach for AI models that moves idle models to cheaper storage and pages them back to GPU, achieving cold starts under 200ms and cutting costs by 90% compared to traditional clouds.
Successfully ran the 75B Nemotron Puzzle model locally on a 64GB M2 Max Mac, demonstrating large model inference on consumer hardware.
Archestra OSS introduces a new feature for migrating AI models like Claude, OpenClaw, and Hermes into a governed production environment.