Tag
NIDO v0.1.64 发布，这是一个完全运行在手机上的开源 AI 助手，无需账号或云服务，支持本地模型（Qwen、Phi、Gemma 等）、加密记忆、蓝牙/Wi-Fi 设备间通信、离线急救指南及严格隐私控制，两天内连续更新十个版本。
NInfer v0.3.0, an inference engine for running Qwen3.6/8-27B on a modded 22 GB RTX 2080 Ti, adds native Windows support and prebuilt releases, cutting binaries to 0.28 GB while delivering up to 47.5 tok/s decode with MTP speculative decoding.
A developer benchmarked 10 coding agents against a local Qwen 3.8 27B model on a MacBook Pro M5 Max, comparing pass rates, token usage, cache reuse, and sandbox behavior. Pi came out leanest while Qwen Code was the most token-hungry, and several agents attempted network access despite offline settings.
介绍 ASK，一款基于本地 LLM 和检索增强生成（RAG）的独立桌面知识助手应用，支持 Qwen、Llama 等多种本地模型，使用 Python 与 CustomTkinter 构建。
The open-source project ai-phone lets your Mac take over real phone lines directly. All speech recognition, LLM inference, and speech synthesis run locally, so call data never leaves your machine. It supports real-time interruptions and allows coding agents to orchestrate calls via CLI.
Basalt (a fork of Strata) lands new PRs setting a new benchmark record with IQ3_S Qwen3.8 Flash Next on a single RTX 5080 with 64GB DDR5, reaching 5500 tok/s prompt processing and up to 275 tok/s peak generation during a live opencode coding session.
A hobbyist benchmarked 39 local LLMs on 50 prompts using a 5060 Ti, then had four frontier LLMs (Gemini 3.8, GPT-6.1 variants, Gemini 4) grade the outputs, ranking Bonsai-2 27B and Underdog-Saluki 27B as top in capability while naming smaller models like Nanbeige4.2 3B and Gemma-4 12B as best fine-tuning foundations.
一位用户在 4x R9700 GPU 搭配上对比了 Qwen 3.8 27B 与 Nanbeige 4.3 33B 的真实世界吞吐表现，发现 Qwen 在 token 速度和每小时任务量上分别领先约 25% 和 45%，且思考模式下准确率更高。
A post exploring a new class of inexpensive IP-based KVM controllers that integrate with local or frontier vision-capable LLMs via MCP, letting AI drive target computers through HDMI capture and HID keyboard/mouse emulation without installing any software on the target machine. Several options like Sipeed NanoKVM Pro, Radxa Linkr, and Violoop are highlighted for use cases like legacy industrial systems and locked-down corporate endpoints.
An open-source desktop app that combines a manga reader with Japanese OCR, local LLM translation overlays, and a page-aware chat companion, keeping original artwork intact while offering side-by-side translations.
Magnitude is an open-source Rust LLM inference engine that auto-tunes GPU kernels on your local hardware, claiming 2.7x faster decode and 1.6x faster prefill than Ollama on a MacBook, with one-click integration into agents like Claude Code and Codex.
Delta Inference Engine 是一个 Windows 原生推理引擎，可在单张消费级 NVIDIA GPU 上以 2-bit 量化运行 5520 亿参数的 DeepSeek-V4.1-Flash-1017，RTX 3090 上生成速度达 41 tok/s，并提供 OpenAI 兼容 API。
A developer built a local GPU benchmark server with two RTX 5060 Ti cards (32GB VRAM) running vLLM to test 12 quantized open-weight models from 4B to 35B across full benchmark datasets, adding wall-clock time efficiency scores and publishing sortable results at llmbenchmarks.io.
Survival LLM is a free, open-source AI assistant and emergency reference library that runs fully offline on your own computer, answering first-aid and disaster-preparedness questions with citations from trusted sources like FEMA, CDC, and Wikipedia.
AtomicChat 发布了基于 LiquidAI/LFM3-1.7B 的安全消融版 GGUF 模型，通过投影秩-1 LoRA 移除有害提示的拒答（61.7%降至0%），同时在 MMLU 上无精度损失，并提供 AD-Q6_K 和 AD-Q4_K_M 量化版本。
erusev 与 llama.cpp 团队合作开发了 Llama，一款可在 Mac 菜单栏运行开源模型的小型应用，并发布了 35 秒演示视频。
OpenJet 的新版本在单张 RTX 3090 上通过自定义 CUDA megakernel 引擎并行服务最多 4 个编码代理，总吞吐达约 290 tok/s，是 llama-server --parallel 的 3-4 倍。核心思路是每步将所有活跃代理的工作堆叠成一个批处理，权重只读一次并用张量核同时处理所有行。
Hob 是 Frigade 开源的桌面应用，号称 Claude Cowork 和 ChatGPT 的本地替代方案，可自动挑选适合用户电脑的前沿开放权重模型，支持离线运行与隐私保护。
Off Grid AI 推出了一款免费开源的本地 AI 应用，可在手机和电脑上离线运行模型，支持聊天、文档阅读、图像问答、语音对话和图像生成，并可在设备间协同调用更大的模型，无需联网或账号。
An analysis of default telemetry behavior across 8 local AI coding agents finds 5 send usage analytics by default to services like PostHog, Statsig, and Alibaba Cloud, with only Crush honoring DO_NOT_TRACK. The post details each agent's data destinations and provides per-tool opt-out switches and shell exports to disable tracking.