local-deployment

This paper evaluates nine open-weight small language models (135M to 3B parameters) on a structured benchmark and shows that parameter-efficient fine-tuning significantly improves accuracy, making them viable for local deployment in structured niche workloads.

Qwythos-9B is a 9B super reasoning model based on the Qwen3.5-9B base, further trained with 500 million+ Claude Mythos reasoning traces. It natively supports 1M long context and tool calling, designed for local deployment by security researchers. It significantly outperforms the original base model on MMLU and mathematical reasoning.

The author shares personal views from participating in training a vertical medical model at a top domestic hospital, highlighting core challenges such as medical data not leaving the hospital, high cost of on-premises deployment, and weak willingness to pay, and suggests partnering with hardware vendors. They also note that general medical models (like Baichuan) already perform well.

Emphasizes the importance of open-source AI models for domain-specific tasks, local deployment, and continuous improvement, advocating for owning intelligence rather than renting it.

Anthropic has launched Claude Science, an AI workbench for scientists with over 60 built-in research skills. It supports local deployment and HPC clusters, and can autonomously draft computing tasks and review results.

Released on Hugging Face, Qwopus-3.6-35B-A3B-Coder is a fine-tuned Mixture-of-Experts coding agent model designed for efficient, low-latency local execution with reduced token waste in agentic workflows.

This article delves into the memory architecture of AI agents, using the Hermes four-layer memory model as an example, analyzes the limitations of the Claude Code memory system, and introduces how to add local memory capabilities to agents through open-source frameworks (such as EverOS), providing detailed deployment steps.

Community efforts, including a hybrid quantization approach by dnhkng, have enabled vLLM and SGLang to support GLM-5.2 with MTP heads, boosting local inference speed from 2 token/s to over 43 token/s on dual GH200 hardware. The challenge involved managing DSA-based MTP and quantization compatibility.

This tweet points out that you can deploy the GLM-5.2 model locally for 128,000 yuan.

A thread recommending an article covering everything you need to know to run large language models locally at home and maintain ownership of your AI.

GLM 5.2 is a frontier open-source coding model that performs near Claude Opus quality on coding tasks, with excellent tool calling, planning, and local deployment capabilities, at no cost.

Google Gemma open models can now be used to deploy local coding agents directly on a laptop, enabling offline execution and faster development workflows.

This paper proposes a fully local AI cascade for de-identifying educational dialogue, combining a recall-first candidate proposer with a contextual Redact/Keep reviewer. The approach achieves high accuracy without sending data to external APIs, outperforming both smaller local models and commercial APIs on math tutoring transcripts.

The author tested the three tiers of PP-OCRv6 models and provided open-source tools for local deployment. They demonstrated performance comparisons of each model on OmniDocBench and real-world scenarios, emphasizing the advantages of lightweight specialized models for OCR tasks.

Baidu Wenxin releases PP-OCRv6, offering three model tiers: Tiny, Small, and Medium, supporting over 50 languages. The Tiny version is only 1.5MB and can run locally in a browser, with the fastest single-image inference at 97ms, proving that small specialized models can outperform large models on OCR tasks.

Microsoft launches Fara-7B, an efficient Computer Use Agent with only 7B parameters, surpassing larger models on web tasks, supporting pure local deployment, and achieving low-cost desktop automation.

The MiniMax-M3 PRISM Dynamic-Quant recipe compresses a 428B parameter model from ~450GB to 119GB using per-tensor sensitivity ranking, with plans to prune further to 60-80GB for local deployment.

A 12-billion-parameter multimodal model has been released as open source. It features a unified Transformer architecture and requires only 16GB of RAM to run. It performs exceptionally well across multiple benchmarks, supports a 256K context window, and works with over 140 languages.

H Company released Holo-3.1-35B-A3B-NVFP4, an open-source computer-use model that achieves up to 195 tokens per second on a single DGX Spark node, outperforming larger models like Qwen3.5-397B and Kimi-K2.5.

A developer built a fully local AI workflow using 3 Mac Minis and 9 AI agents, achieving zero cloud service bills while earning $10k/month.