ollama

Manning Books is offering a free ebook, 'Local AI and Small Language Models', featuring chapters on small language models, CPU inference with GGUF and llama.cpp, quantization, LoRA/DoRA specialization, and building private local AI apps with Ollama's Python library.

Nexus-CLI 是一个纯 Go 编写的命令行工具与本地 API 代理服务器，可通过免费的 DeepSeek 网页账号桥接 DeepSeek R1/V3 模型，支持 Claude Code、Ollama、OpenAI 及 Anthropic 兼容接口。

A small open model, qwen3-4b-deslop, is a Qwen3-4B-Instruct fine-tune trained on a MacBook Air that removes common AI writing tics (em dashes, 'not X, but Y', lists of three), cutting the slop score from 121 to 12.5 per 1000 words while keeping factual accuracy, and it runs locally via Ollama.

EpochGo is an open-source, local-first multi-agent AutoML engine that uses an offline Ollama LLM (qwen2.5-coder:7b) as an orchestrator over deterministic Scikit-Learn pipelines, with AST-based validation and auto-generated FastAPI serving endpoints, running 100% offline on consumer hardware.

Dodaios 的维护者展示了一款 Apache 2.0 开源桌面 AI 代理应用，可在本地 Ollama 模型（Qwen3.5:9b 和 Qwen3.8-16gb）上运行多代理任务，并坦诚说明了本地模型在工具调用上的局限。

Pluto 是一款面向 Apple Silicon Mac 的本地优先、开源会议助手，无需机器人即可录音，本地转写并把对话整理为可检索的工作记忆，支持 Ollama 本地模型和可选云端 AI。

用户惊叹于在 M3 Max 64GB Mac 上通过 oMLX 成功运行 Qwen3.8-Flash-Next-oQ4e-mtp 这个约 100GB 的模型，58GB 分配给 GPU，并支持最高 130k 上下文窗口，展示了本地大模型推理性能的显著提升。

HIVE (Hive Inspired Virtual Entity) 是一个开源研究系统，通过九个从不同角度分析的 forager agent、一个 queen verdict 以及共享记忆（comb），并利用 Unpaywall 与 DOI 检索的已验证白皮书，为 LLM 输出提供可信度置信值，减少幻觉。系统可在本地运行（Ollama + Qwen3.6 35B），在 M1 Max 64GB 上设计与测试。

Row-Bot 5.0 demo running qwen3.8:27b locally in Ollama on a single GPU, analyzing bills, a lease, and an insurance policy to spot a water-bill leak and an illegal rent increase, entirely offline with no cloud APIs.

Neco is a local AI project built on Ollama and Open WebUI that treats the LLM as a persistent presence on the host machine, with read-only system awareness, a background daemon that generates idle thoughts, and experiments in episodic memory and evolving state.

A user benchmarks local agent models on dual RTX 5070 Tis, finding that Ornith 1.5 35B-A3B (a ~3B-active-parameter MoE) matches Qwen3.8 27B on their tests while running about 3x faster (~180 tok/s) and scaling to 512K/1M contexts with YaRN.

This paper presents CourseChat, an on-premises multi-course RAG tutoring system for undergraduate business education, sharing six isolated courses across two edge AI hosts running Ollama-served local LLMs. Model bake-offs show that 8B/12B/7B models pass classroom speed gates while larger ones fail, supporting a hardware-software co-design approach rather than assuming bigger models are optimal.

UNIXAI Version 3.6 is a minimalist single-script bash harness that turns any local Ollama model into an autonomous conversational agent, featuring an endless reflection loop, message queue trimming, and built-in shell tool-calling.

The post benchmarks a zero-GPU, CPU-only string-normalization entropy metric (Spanda) against semantic entropy across 1.5B–120B local and frontier models, finding that 7B–27B models are well-served by cheap exact-match entropy while 120B models collapse into identical confident hallucinations that even NLI models miss.

Ollama added support for Jev-like decision models such as Nimble, running entirely locally for tasks like ticket triaging, model routing, and content moderation via a new local /v1/systemone API.

Running a Discord agent powered by OpenClaw and GLM 5.3 Flash for 23 hours led to failures at around 65% of the 1M context window, exposing issues with caching and model performance under heavy use.

A user shares detailed benchmarking data and personal insights on running AI models locally with varying GPU power limits, evaluating models like gemma4 and qwen3.5 on a modest hardware setup.

The user has implemented nvfp4 KV cache support for the Qwen3.8 model on a heterogeneous GPU setup using custom CUDA kernels and quantization to optimize performance.

Otis is an open-source AI agent that provides a minimal, privacy-focused experience for running local and hosted open-weight models, automatically recommending and downloading models based on hardware.

This article introduces a full book on building production-ready multi-agent AI systems using LangGraph, MCP, A2A, and Ollama, with working code and real-world applications.