I got tired of Apple's configurator, so I built one for local AI

Reddit r/LocalLLaMA Tools
local-llm mac-hardware configurator llama-bench inference-benchmark open-project

Summary

作者因不满 Apple 官方配置器，开发了 modelfit.io——一个专为购买 Mac 运行本地 LLM 的用户设计的配置工具，可实时显示所选 Mac 配置能运行哪些模型及预计推理速度。

I was buying a Mac mini and kept going back and forth between configs, trying to guess which one would actually run the models I want. So I built a configurator for people buying a Mac to run local LLMs: https://modelfit.io/ You can clearly see all the best llms it can read and at what efficiency all updating in real time according to the configuration you pick. You pick a Mac and a config, and it shows which models fit in memory and an estimated speed for each. The speed estimates come from a model fitted on real llama-bench runs, and every number shows its typical error, so you know how much to trust it. Apple list prices update daily. So I built my own configurator especially for people like me that are purchasing a mac in order to do some local ai. It's my own project. I'd love feedback, especially from people who can compare the estimates with their own numbers. Also Was hesitating between m6 32gb and m5pro 48gb, I'll pick the m5pro what do you think about that choice ?
Original Article

Similar Articles

@sitinme: There's a pretty interesting open-source project called Cider, specifically designed to accelerate local AI inference on Macs with Apple Silicon chips. Many people buy a Mac mini or MacBook Pro and want to run models locally, but often encounter issues like insufficient speed and high memory usage. Actually...

X AI KOLs Timeline

Cider is an open-source project designed for Apple Silicon Macs, accelerating local AI inference by fully leveraging the computing power of M-series chips. It is compatible with the MLX ecosystem, supports models like Qwen and Llama, and is easy to install.

@cevenif: For those running local LLMs on Macs, here's a tool worth watching — Rapid-MLX. It delivers 2-4x faster inference on M-series chips than Ollama, thanks to being built directly on Apple's MLX framework for more thorough utilization of the chip architecture. Key highlights: KV cache pruning plus…

X AI KOLs Timeline

Rapid-MLX is a local LLM inference tool optimized for Apple M-series chips. Built on the MLX framework, it achieves 2 to 4 times faster inference than Ollama, supports multiple models, tool calling, and an OpenAI API-compatible interface.

My local model setup on an M4 Pro Mac Mini

Hacker News Top

The author describes their local AI model setup on an M4 Pro Mac mini, using models like Qwen and Gemma with tools such as oMLX and Tailscale to achieve data privacy, cost predictability, and offline capability.