I built an app to efficiently run decision models on the Apple Neural Engine with Shortcuts actions for OS automations

Reddit r/LocalLLaMA Tools
apple-neural-engine shortcuts on-device-inference decision-models macos open-source coreml

Summary

Tinkerbox 是一个可将 LiquidAI d1-3B 等决策模型高效运行在 Apple Neural Engine 上的应用，支持通过 macOS Shortcuts 触发器实现无代码自动化，延迟低至约 40ms，并计划集成 vLLM/Qwen3.5 系列模型。

LiquidAI d1-3B is currently on the Pareto frontier of the Decision Index 0.3.2 https://huggingface.co/spaces/multimodalart/jev-decision-index, and you can run it on the ANE with around 40 ms latency on warm short-context (<256 tokens) decisions, as measured on M5 Pro. With Shortcuts integration, you can run efficient decisions on events such as incoming email and iMessages, folder and file modifications, time of day and alarms, and other triggers without writing a single line of code. Decision models are a good fit for ANE because they don't use token-by-token autoregressive decoding, which it struggles with due to relatively low memory bandwidth and static buffer limitations. Tinkerbox's implementation of d1-3B uses a tree structure to evaluate multiple decisions in a single pass, further reducing latency. Source code: https://github.com/WeGoToMars/tinkerbox AOT-compiled models: https://huggingface.co/wegotomars/d1-3B-CoreAI-ANE Download the app: https://github.com/WeGoToMars/tinkerbox/releases I don't have access to any other Macs, so please report your results and any issues you encounter. I'm also taking model and feature suggestions! Embeddings, TTS, STT, and image segmentation models are on the roadmap. This isn't a one-off vibecoded repo, I want to continue supporting it and make it a real framework for ANE-optimized inference. I'm currently working on integrating vLLM d3 models (based on Qwen3.5), which would also allow you to use most other Qwen3.5 fine-tunes with minimal modification. 0.8B (lite), 2B (nano), and 4B (mini) should be available soon!
Original Article

Similar Articles

@sitinme: There's a pretty interesting open-source project called Cider, specifically designed to accelerate local AI inference on Macs with Apple Silicon chips. Many people buy a Mac mini or MacBook Pro and want to run models locally, but often encounter issues like insufficient speed and high memory usage. Actually...

X AI KOLs Timeline

Cider is an open-source project designed for Apple Silicon Macs, accelerating local AI inference by fully leveraging the computing power of M-series chips. It is compatible with the MLX ecosystem, supports models like Qwen and Llama, and is easy to install.

@cevenif: For those running local LLMs on Macs, here's a tool worth watching — Rapid-MLX. It delivers 2-4x faster inference on M-series chips than Ollama, thanks to being built directly on Apple's MLX framework for more thorough utilization of the chip architecture. Key highlights: KV cache pruning plus…

X AI KOLs Timeline

Rapid-MLX is a local LLM inference tool optimized for Apple M-series chips. Built on the MLX framework, it achieves 2 to 4 times faster inference than Ollama, supports multiple models, tool calling, and an OpenAI API-compatible interface.