@RisingSayak: Found a faster kernel? You shouldn’t need to rewrite your model to use it. With Kernels, you can choose which kernel ru…

Found a faster kernel? You shouldn’t need to rewrite your model to use it.

With 🤗 Kernels, you can choose which kernel runs a supported layer and replace its forward() with an optimized implementation.

Here’s how 🧵

1/5

First, make a layer swappable with a decorator:

use_kernel_forward_from_hub("SiluAndMul")

Then register a mapping that says: “For this layer, on this device, use this kernel from the Hub.”

You choose the implementation.

2/5

Now apply it:

kernelize(model, mode=Mode.INFERENCE)

Kernels walks through the model & replaces forward() on the eligible layers with registered kernels.

The other layers stay untouched.

You can also configure things with granulaity (device, capability specificity).

3/5

Need one kernel for inference and another for training? You can specify both.

You can also choose different kernels for different device types.

You can register those choices in a mapping, either globally or locally.

4/5

That’s the useful bit: trying another compatible kernel becomes a mapping change, not a model rewrite.

Swap it in. Benchmark your workload. Keep what actually helps.

Docs: https://huggingface.co/docs/kernels/en/layers…

5/5

No

🔥 “best-fit” kernels should always just populate!

Speaking of, any luck w further ROCm support @RisingSayak? Happy I got to contribute for qwen-series on DGX Spark, want to show Strix Halo some love too :)

Compile works. Export — we haven’t tested.