@akshay_pachaar: Apple finally did it. Its new framework, Core AI, runs models entirely on Apple silicon, so inference happens on the us…

X AI KOLs Following 06/09/26, 06:36 PM Tools

apple core-ai on-device swift open-source models inference

Summary

Apple released Core AI, a new framework that runs AI models entirely on Apple silicon devices (iPhone, iPad, Mac, Vision Pro) with zero server calls. It includes a memory-safe Swift API, model export recipes for PyTorch, an optimizer, and debugging tools, supporting models like Qwen, Mistral, and SAM3.

Apple finally did it. Its new framework, Core AI, runs models entirely on Apple silicon, so inference happens on the user's device with zero server calls and zero token bills. That means Qwen, Mistral, and SAM3 running natively across iPhone, iPad, Mac, and Vision Pro. It's a memory-safe Swift API that compiles models ahead of time for near-instant load. Pulling one in takes a few lines: let segmenter = try await ImageSegmenter(resourcesAt: sam3ModelURL) let response = try await segmenter.segment(image: inputImage, prompt: "flower") The launch goes beyond the runtime, though. It ships curated open models packaged for Swift, PyTorch extensions to convert your own, and an optimizer that shrinks models layer by layer with minimal accuracy loss. There's also a macOS debugger that profiles performance and traces behavior back to your original Python, plus Xcode tools to validate models before they ship. For any team that wanted real on-device AI without a cloud bill attached to every user, this is the answer. Models repo: https://github.com/apple/coreai-models…

Original Article

View Cached Full Text

Cached at: 06/10/26, 09:47 AM

Apple finally did it.

Its new framework, Core AI, runs models entirely on Apple silicon, so inference happens on the user’s device with zero server calls and zero token bills.

That means Qwen, Mistral, and SAM3 running natively across iPhone, iPad, Mac, and Vision Pro.

It’s a memory-safe Swift API that compiles models ahead of time for near-instant load. Pulling one in takes a few lines:

let segmenter = try await ImageSegmenter(resourcesAt: sam3ModelURL) let response = try await segmenter.segment(image: inputImage, prompt: “flower”)

The launch goes beyond the runtime, though.

It ships curated open models packaged for Swift, PyTorch extensions to convert your own, and an optimizer that shrinks models layer by layer with minimal accuracy loss.

There’s also a macOS debugger that profiles performance and traces behavior back to your original Python, plus Xcode tools to validate models before they ship.

For any team that wanted real on-device AI without a cloud bill attached to every user, this is the answer.

Models repo: https://github.com/apple/coreai-models…

apple/coreai-models

Source: https://github.com/apple/coreai-models

Core AI Models

Model export recipes, Python primitives, and Swift runtime utilities for building on-device AI with Core AI.

The main components include:

Model export — Recipes to export popular open source models from Hugging Face and other sources to Core AI format.
Reusable primitives — Python building blocks for authoring custom Core AI models in PyTorch.
Runtime utilities — Swift package built on top of Core AI framework to run models on macOS and iOS.
Skills — Plugins to help coding agents leverage Core AI effectively.

Directory	What’s inside
`models/`	Model catalog with README and export recipes.
`python/`	Python primitives for authoring and utilities for exporting models.
`swift/`	Swift package (`coreai-models`): runtime utilities to integrate Core AI models in your app.
`skills/`	Pluggable skills that enable coding agents to leverage Core AI more effectively.

Requirements

If you haven’t installed uv, install it by

brew install uv

curl -LsSf https://astral.sh/uv/install.sh | sh

Once installed successfully, refer to the README.md for each model or family of models, in models folder for their exporting recipe.

Requirements (running and app integration)

macOS and iOS 27.0+
Xcode 27.0+

Core AI models are exported as standalone .aimodel files for integration into apps via the Core AI framework.

Some models require additional resources. Language models require a tokenizer, for instance, and diffusion models run multiple models in sequence as part of a single pipeline. For these cases, export recipes in this repo produce a resource folder containing one or more .aimodel files alongside any required resources. The Swift package in this repo provides runtime utilities for integrating these into an app.

Command line interface (CLI) tools are also included for running exported models directly on a Mac (requires Xcode 27.0+). See each model’s README for available tools and example invocations.

Explore supported models

Find supported models by

git clone https://github.com/apple/coreai-models.git && cd coreai-models
uv run coreai.model.registry --list-models

Run uv run coreai.model.registry --help for details.

Agent Skills

This repo includes a plugin with skills to enable coding agents to use Core AI like an expert.

Available skills

Skill	Description
`working‑with‑coreai`	End-to-end workflow for deploying PyTorch models on Apple silicon, covering export with `coreai-torch` and running with the Core AI runtime.
`model‑authoring`	Empirical rules for authoring PyTorch models for on-device execution on Apple platforms, covering BC1S layout, op compatibility, KV cache patterns, precision rules, MoE, and common issues.
`model‑compression‑exploration`	Systematically explore weight compression configurations (quantization and palettization) for a PyTorch model using `coreai-opt`.

Install

Installation differs depending on your coding agent of choice.

Claude Code

/plugin marketplace add [email protected]:apple/coreai-models.git

Alternatively, register the marketplace from a local git checkout:

/plugin marketplace add /path/to/coreai-models

Install the plugin:

/plugin install coreai-skills@coreai-models

Codex CLI

codex plugin marketplace add https://github.com/apple/coreai-models

Alternatively, register the marketplace from a local git checkout:

codex plugin marketplace add /path/to/coreai-models

Install the plugin:

codex plugin add coreai-skills@coreai-models

Gemini CLI

Install the extension from a local directory:

gemini extensions install /path/to/coreai-models/skills

Once installed, the skills activate automatically based on your task context, or you can invoke them explicitly.

Contributing

We are not accepting code contributions at this time

Core AI Models is focused on maintaining a curated, well-tested gallery of models and a reliable Swift package. We are not accepting pull requests at launch while we learn how the community uses this project.

If you open a pull request, it will be closed. This is not a reflection of the quality of your contribution but it is a deliberate scope decision for this release.

What we do welcome

We actively want your feedback! GitHub Issues are open for:

Bug reports — if something in the Python scripts or Swift utilities does not work as expected
Model requests — if you have ideas for models you would like to see, or improvements to the workflow or Swift utilities

Use the issue templates to get started.

Support

GitHub Issues — Feedback, bug reports, and feature requests

License

This project is licensed under the BSD 3-Clause License.