ollama

The user has implemented nvfp4 KV cache support for the Qwen3.8 model on a heterogeneous GPU setup using custom CUDA kernels and quantization to optimize performance.

Otis is an open-source AI agent that provides a minimal, privacy-focused experience for running local and hosted open-weight models, automatically recommending and downloading models based on hardware.

This article introduces a full book on building production-ready multi-agent AI systems using LangGraph, MCP, A2A, and Ollama, with working code and real-world applications.

This article provides a guide on setting up OpenCode with Ollama and Docker Sandboxes on a Mac for running local AI models like Qwen 3.8 and Gemma 4.

A factory worker built an autonomous AI entity using Gemma 4 31B on Ollama, with self-identity, memory, and a blog, demonstrating personalized AI interaction through an open-source framework.

The author describes how integrating Open Terminal with Open WebUI enables a self-hosted AI agent capable of searching large legal documents and generating notes via shell commands, using Qwen 27B on Ollama. They highlight that terminal access allows the model to reason step-by-step and extract relevant sections from documents far exceeding the context window.

MTP has been released for the Qwen3.8-Flash-Next-GGUF model, providing detailed instructions on integration with inference tools like llama.cpp, vLLM, and Ollama for deployment.

The article provides real-world performance benchmarks for running the Qwen3.8 27B AI model locally on a Mac Studio, comparing it to its predecessor and discussing hardware requirements and quantization effects.

This repository provides a BF16 GGUF conversion of the Spark-X2.5-4B language model, enabling local inference with Ollama and LM Studio.

This article provides a detailed test of the local deployment and performance of the Ling-3.0-tiny model on an Apple M5 chip Mac, demonstrating the feasibility of running a 7.9B parameter model at 47 tokens per second without a discrete GPU.

Shadow-planner is a desktop project planner featuring AI-assisted Gantt chart planning with dependencies and what-if scenarios, running locally on Ollama models with a one-time purchase.

AudioNotes is a local audio and video transcription and smart note-taking tool that prioritizes privacy and offline use, deployable via Docker or Python.

The author created an open-source hybrid tool called Local Coding Agent to make small local models effective for coding tasks on consumer GPUs by using a cloud model for planning and local models for isolated execution, with error handling and testing features.

The user indicates they are ready to use the Qwen 3.8 27b model, now Ollama directly supports Mac's MLX, making it very comfortable to use.

The author built a local realtime voice stack using Parakeet STT, Qwen 2.5 7B, and Qwen3-TTS, integrated with Ollama.

A developer describes building a multi-engine agentic coding harness that routes sessions across Claude Code, Codex, and Ollama, selecting from twelve models per session based on task value.

NativeMind is a private, open-source browser extension that runs local AI models via Ollama or WebLLM, enabling offline AI features for privacy-conscious users.

DeepSeek V4 Flash has become Ollama's fastest growing model in token usage and the most popular model on OpenRouter this week, now available in Pi across multiple providers.

The author argues that the traditional enterprise AI stack is obsolete, claiming a $599 Mac mini running Ollama can handle 80% of AI workloads locally for a fraction of the cost of renting cloud GPUs.

Homebench is a zero-config terminal tool that benchmarks locally-run LLMs for speed, memory, and quality, presenting a live leaderboard. It supports Ollama, LM Studio, llama.cpp, vLLM, and OpenAI-compatible servers.