Rigel: Reverse-Engineering the Metal 4.1 Tensor Compute Path on the Apple M4 Max GPU

# Reverse-Engineering the Metal 4.1 Tensor Compute Path on the Apple M4 Max GPU
Source: [https://arxiv.org/html/2606.12765](https://arxiv.org/html/2606.12765)
\(June 2026\)

###### Abstract

Apple’s Metal 4\.1 exposes a tensor compute path: the Metal Performance Primitives \(MPP\)matmul2doperation overcooperative\_tensorfragments, whose*interface*is documented but whose*hardware behavior is deliberately hidden*\. The specification states which data\-type rows are supported, never whether they are hardware\-accelerated, where the operation physically executes, what its accumulator width is, or how it partitions matrix fragments across threads\. We presentRigel, an empirical characterization of this path on a single Apple M4 Max \(a pre–neural\-accelerator generation\)\. Using a checksum\-gated, provenance\-tracked microbenchmark harness,Rigelrecovers eleven facts the v4\.1 specification hides or contradicts\. The headline finding: the Metal 4\.1 fp8 \(E4M3\)matmul2dis*emulated*, not accelerated: it sustains0\.94×0\.94\\timesthe throughput of fp16 despite reading half the operand bytes, so on M4 it is a memory\-footprint feature, not a performance feature\. We further show, via a three\-signal triangulation \(throughput ceiling, comparison againstsimdgroup\_matrix, and per\-rail power attribution\), thatmatmul2dexecutes entirely on the GPU shader cores with no dedicated matrix datapath and no evidence of Apple Neural Engine routing; that it accumulates in≥\\geqfp32; and we reconstruct the opaque8×88\{\\times\}8cooperative\_tensorfragment layout Apple documents nowhere\. Acting on the characterization, a hand\-fused GEMM \+ bias \+ GELU kernel beats the decomposed path by\+6\.5\+6\.5–12\.9%12\.9\\%in the cache\-resident regime\. All findings are reproducible from committed MIT\-licensed code and per\-cell CSVs\.

## 1Introduction

Quantized and attention\-heavy machine\-learning workloads increasingly run on consumer Apple Silicon GPUs\. Metal 4\.1 \(shipping in Xcode 27 / macOS 27\.0\) adds a tensor compute path \(the Metal Performance Primitives \(MPP\)matmul2doperation, which contracts twotensoroperands through device\-distributedcooperative\_tensorfragments\) and a low\-precision frontier of fp8, fp4, and block\-scaled MXFP4 formats\. For anyone deploying an LLM or vision model on a Mac, three questions decide everything:*where*does a tensor op execute,*how fast*is each low\-precision format, and*what*numerical guarantees does it provide\. The Metal Shading Language Specification\[[1](https://arxiv.org/html/2606.12765#bib.bib13)\]answers none of them\.

The problem:The specification documents the*interface*but hides the*hardware behavior*\. Its feature tables state that a data\-type row is*supported*; they never state whether it is*accelerated*\. It declares thecooperative\_tensorlayout “opaque” and “device specific” \(§2\.22\.3\.1\)\. It is silent on the dispatch target ofmatmul2d, on accumulator width, and on the microarchitectural alignment constraints a kernel must satisfy\. A developer cannot tell, from the specification alone, whether fp8 will be twice as fast as fp16 or no faster at all\.

The gap:No public source characterizes the Metal 4\.1 tensor path on a specific Apple GPU\. Workload\-level studies benchmark Apple Silicon end to end\[[4](https://arxiv.org/html/2606.12765#bib.bib10),[5](https://arxiv.org/html/2606.12765#bib.bib11)\]but do not open the tensor primitive itself\. The community is split on a load\-bearing question: whether Apple’s only “tensor core” is the legacysimdgroup\_matrixinstruction \(an ALU\-utilization optimization\) or a dedicated matrix unit analogous to NVIDIA tensor cores\[[8](https://arxiv.org/html/2606.12765#bib.bib9),[12](https://arxiv.org/html/2606.12765#bib.bib8)\]\. The M4 Max sits*before*the GPU “Neural Accelerators” Apple reports for the A19/M5 family\.

The insight:These hidden facts are*recoverable empirically*, in the microbenchmarking tradition that dissected NVIDIA GPUs\[[7](https://arxiv.org/html/2606.12765#bib.bib6),[6](https://arxiv.org/html/2606.12765#bib.bib7)\], and on a pre–neural\-accelerator chip the strong prior is that the fp8/fp4 rows are functionally supported but*emulated*\. We adopt a discovery bar: a result counts only if it ishiddenorcontradictedrelative to the specification; facts a careful reader could derive from the specification are controls, not contributions\.

Contributions:

1. 1\.A checksum\-gated, provenance\-tracked harness \([Section3](https://arxiv.org/html/2606.12765#S3)\) that turns each claim into a reproducible cell with a float64 reference and a pre\-registered cheap\-baseline falsification gate\.
2. 2\.Theheadline: fp8 \(E4M3\)matmul2dis*emulated*on M4 Max \(0\.94×0\.94\\timesfp16 throughput\), making it a footprint feature, not a perf feature \([Section5](https://arxiv.org/html/2606.12765#S5)\)\.
3. 3\.A three\-signal dispatch\-target result \([Section4](https://arxiv.org/html/2606.12765#S4)\):matmul2druns on the GPU shader cores via the legacysimdgroup\_matrixpath, with no dedicated matrix unit and no evidence of ANE routing\.
4. 4\.Reconstruction of the opaque8×88\{\\times\}8cooperative\_tensorfragment layout \([Section7](https://arxiv.org/html/2606.12765#S7)\), plus the accumulator width \(≥\\geqfp32\) and the fp8/fp4 numeric semantics\.
5. 5\.Acontradicted\-class version gate: Metal 4\.1 requires Xcode 27*and*macOS 27\.0 \(not the specification’s “Xcode 26\.1\+”\), and a 4\.1 binary refuses to load on macOS 26\.5 \([Section2](https://arxiv.org/html/2606.12765#S2)\)\.
6. 6\.The eleven hidden/contradicted findings as a single legible catalog \([Table1](https://arxiv.org/html/2606.12765#S1.T1)\), the natural home for low\-visibility constraints \(the 128\-byte alignment, the SFINAE footgun, the version gate\)\.
7. 7\.An optimization study \([Section8](https://arxiv.org/html/2606.12765#S8)\): a hand\-fused GEMM\+bias\+GELU kernel beats the decomposed path by\+6\.5\+6\.5–12\.9%12\.9\\%in the cache\-resident regime \(the win decays asO​\(n3\)O\(n^\{3\}\)compute dominates\), while a fused\-attention study shows the win does not extend to operations that forgo the matrix units\.

Table 1:The eleven findings that meet our discovery bar: each is eitherhiddenby the Metal 4\.1 specification or directlycontradictsit, and none can be derived from the document alone\. Eight are hidden \(H\) and three are contradicted \(C\)\. Controls, such as the OCP\-conformant fp8 and fp4 grids, are excluded\. The “Spec\. basis” column names what the specification says, or fails to say, about each point\.
## 2Background

This section gives the reader the three things needed to follow the results: what the Metal 4\.1 tensor path actually is, the version gate that controls access to it, and the device on which we measure\.

The Metal 4\.1 tensor path:The primitive at the centre of this paper is the Metal Performance Primitives operationmatmul2d, which multiplies two matrices into a result,C=A⋅BC=A\\cdot B\. Its operands aretensorobjects, and Metal offers two flavours that matter here\. Atensor\_inlineis a lightweight, non\-owning view over an ordinarydevicebuffer, created inside the shader and describing only a contiguous region; a host\-boundtensor\_handleis instead constructed on the CPU and can carry explicit strides\. The destination may additionally be declared acooperative\_tensor, a register fragment that the threads of an execution scope hold collectively, never writing it to device memory, and whose internal layout the specification deliberately leaves opaque\. Each call is shaped by a descriptor,matmul2d\_descriptor\(m,n,k\), that fixes the output tile a single threadgroup produces\. Passingdynamic\_extentas the contraction lengthkktells the operation to loop over that dimension internally, rather than expecting the caller to drive it\. The execution scope, writtenexecution\_simdgroups<N\>, then bindsNNSIMD\-groups of 32 threads each to compute one tile cooperatively\. These few types, and the hardware behaviours the specification omits for them, are the entire subject of the paper\.

The version gate:Before any tensor feature can be measured, a binary that uses it must compile*and*run, and here the specification is already inaccurate\. It states that Metal 4\.1 is available from Xcode 26\.1 onward, but we found access to be gated on both the toolchain and the operating system\. Xcode 26\.5 and the 26\.6 release candidate still emit Metal 4\.0; only Xcode 27 produces the language\-version marker\_\_METAL\_VERSION\_\_ 410\. A library compiled at version 4\.1 under Xcode 27 then refuses to load on macOS 26\.5, failing at runtime with “language version 4\.1 is not supported on this OS\.” Exercising the fp8 and fp4 paths therefore requires Xcode 27 and macOS 27\.0 together, not Xcode 26\.1 as documented\. This is the first of the contradicted\-class findings catalogued in[Table1](https://arxiv.org/html/2606.12765#S1.T1)\.

Device under test:All measurements are taken on a single Apple M4 Max \(40 GPU cores, 64 GB of unified memory, in a Mac Studio\) running macOS 27\.0, build 26A5353q\. The M4 generation predates the GPU “Neural Accelerators” that Apple introduced with the A19 and M5 parts, so the expectation going in is a chip that supports the low\-precision formats functionally but does not accelerate them; the results bear this out\. Every number is reported “on M4 Max”, we make no claim about other Apple GPUs, and we treat cross\-generation behaviour as future work\.

## 3Methodology

A characterization is only as trustworthy as its defences against fooling itself, so before presenting any result we describe howRigelearns confidence in a measurement\. The harness is split across two languages on purpose\. Thin Objective\-C\+\+ hosts do nothing but allocate buffers, dispatch a Metal kernel, and time it, while a dependency\-light Python layer owns everything that has to be checked: the schema of a measurement cell, the per\-data\-type error tolerances, the correctness gate, and the provenance record\. A number reaches this paper only after passing the four disciplines below\.

The checksum gate:The first question about any GPU kernel is whether it computed the right answer, and low precision makes that question subtle\. An fp8 result differs from a double\-precision reference for two reasons \(the*inputs*were quantized, and the*accumulation*lost precision\), but only the second tells us anything about the hardware\. We therefore compute the reference in float64 from the exact same quantized bytes the GPU consumed, so the input\-quantization error cancels and the residual bounds the kernel’s accumulation error alone\. Any cell whose error exceeds its per\-data\-type tolerance is markedgarbageand dropped from every average rather than quietly blended in\. As an independent cross\-check, a separate Python re\-verifier re\-derives each pass or fail verdict directly from the committed CSV; run against real harness output, it agreed on all fifty cells with zero mismatches\.

Provenance:For a result to be citable months later it has to be reconstructible, so before a run begins the harness records a configuration hash, the code commit, a hash of the input\-generation recipe, the random seed, and a full capture of the environment \(device, GPU core count, OS build, and toolchain versions\)\. A run that is missing any of these is not used\. Every figure in the paper is then regenerated from its committed CSV by a singlemake reproducetarget that needs no GPU, so a reader can redraw our plots from our data\.

The cheap\-baseline gate:The central risk in any “the tensor path is fast” story is that the speed comes from something mundane \(a better loop, or simply the ordinary ALU\) rather than from special hardware\. We guard against this by refusing to make a performance claim until the simplest non\-mechanism explanation has actually been run and has failed to account for the effect\. Four baselines stand in for those mundane explanations: \(A\) the scalar\-ALU roofline, \(B\) a naive threadgroup\-tiled GEMM that uses no tensor primitive, \(C\) the legacysimdgroup\_matrixinstruction, and \(D\) the stock fp16 MPP path\. The thresholds are fixed in advance: a kernel must beat the strongest baseline by at least1\.10×1\.10\\timesto count as a win, and a low\-precision format must reach1\.5×1\.5\\timesits matched fp16 throughput to count as*accelerated*rather than merely*emulated*\.

Reporting:Each throughput cell is measured over at least thirty repetitions \(hereR=300R\{=\}300\) after discarding warm\-up runs, and we report the median together with a bootstrapped95%95\\%confidence interval \(seed\-fixed,10410^\{4\}resamples\); a single\-threadgroup “does it run” timing is never reported as throughput\. The confidence interval is not boilerplate\. As the next paragraph shows, the M4 GPU clock is bimodal at small problem sizes, and there the width of the interval is itself the finding\.

That bimodality is itself worth characterizing\. Per\-rep throughput splits at cache\-resident sizes \([Figure1](https://arxiv.org/html/2606.12765#S3.F1)\)\. At5123512^\{3\}theR=300R\{=\}300samples split into two near\-equal clusters at∼\\sim4\.65 and∼\\sim6\.5 TFLOP/s \(145 vs\. 155 reps\), giving a wide bootstrapped CI of\[5\.21,5\.96\]\[5\.21,5\.96\]TFLOP/s;102431024^\{3\}runs stably at∼\\sim12\.7 TFLOP/s with a rare low\-clock tail \(8/3008/300reps at∼\\sim6\.4 TFLOP/s\);204832048^\{3\}holds a stable max clock \(14\.8014\.80TFLOP/s, CI\[14\.79,14\.80\]\[14\.79,14\.80\]\)\. The mechanism is clock\-state: a short kernel finishes around the GPU’s clock ramp and lands in either a low or high P\-state, while a long kernel sustains the high state\. This clock\-state cliff is why every throughput claim in the paper carries a bootstrapped CI rather than a bare median: a point estimate at5123512^\{3\}would silently report either4\.654\.65or6\.56\.5TFLOP/s depending on luck\.

![Refer to caption](https://arxiv.org/html/2606.12765v1/figures/h5_clock_bimodality.png)Figure 1:Clock\-state bimodality\.\(a\) Median fp16 throughput with bootstrapped95%95\\%CI: the interval is wide at5123512^\{3\}and razor\-thin once the clock is stable\. \(b\) Per\-rep distributions \(R=300R\{=\}300\):5123512^\{3\}is bimodal \(two clock states\), larger sizes are unimodal\.
## 4Where doesmatmul2dexecute?

The dispatch target is the load\-bearing question: ifmatmul2dlowers onto the existing GPU ALU/simdgroup\_matrixpipeline, the fp8 rows cannot be accelerated without dedicated silicon\. We triangulate with three independent signals, the bar the specification’s own discovery problem demands\.

Signal 1, throughput ceiling:A checksum\-verified tiled fp16matmul2dsustains14\.814\.8TFLOP/s at204832048^\{3\}\([Figure2\(a\)](https://arxiv.org/html/2606.12765#S4.F2.sf1)\),≈46%\\approx 46\\%of the∼\\sim32 TFLOP/s scalar\-ALU fp16 roofline, never above it, so there is no hidden datapath beating the ALU ceiling\.

Signal 2, comparison:A four\-way GEMM shootout \([Figure4](https://arxiv.org/html/2606.12765#S5.F4)\) shows MPP beats a naive threadgroup\-tiled GEMM by2\.92\.9–5\.5×5\.5\\times\(the primitive matters\), but beatssimdgroup\_matrixby only1\.051\.05–1\.21×1\.21\\times\. Such proximity is consistent with MPP lowering onto thesimdgroup\_matrixand FP32\-ALU path rather than onto a separate unit\.

Signal 3, power attribution:Under a sustainedmatmul2dload, the GPU rail draws∼\\sim42 W while the GPU hardware\-active\-residency hits100%100\\%\([Figure2\(b\)](https://arxiv.org/html/2606.12765#S4.F2.sf2)\); read via a root\-free IOReport “Energy Model” reader, the GPU energy rail rises0\.27→47\.60\.27\{\\to\}47\.6W\. The*caveat*: on this macOS 27\.0 beta neitherpowermetricsnor IOReport meters CPU/ANE power \(a CPU at84%84\\%residency still reads0mW\), and no CoreML workload would execute on the ANE \(vision\-tower ANE compilation fails\), so the ANE\-exclusion half of the argument is indirect \(it rests on Signal 2: an ANE offload would*add*throughput beyond what the GPU\-only path delivers, and it does not\)\.

Taken together, these three signals settle the question\. On M4 Max,matmul2dexecutes on the GPU shader cores, on the samesimdgroup\_matrixand FP32\-ALU path used by any other shader, with no dedicated matrix datapath\. This converts the contested community claim into a measured fact*on this chip*and is the mechanism behind the fp8 result below\.

![Refer to caption](https://arxiv.org/html/2606.12765v1/figures/tiled_throughput_vs_size.png)\(a\)fp16 throughput \(Signal 1\)\.
![Refer to caption](https://arxiv.org/html/2606.12765v1/figures/h1_power_attribution.png)\(b\)GPU power \+ residency \(Signal 3\)\.

Figure 2:Dispatch\-target signals\.\(a\) Tiled fp16matmul2dpeaks at14\.814\.8TFLOP/s,≈\\approx46% of the scalar\-ALU roofline; nothing beats the ALU ceiling\. Plotted points are medians; the5123512^\{3\}value is clock\-state\-dependent \(see[Figure1](https://arxiv.org/html/2606.12765#S3.F1)for the bimodality\)\. \(b\) Under a sustainedmatmul2dload the GPU rail draws∼\\sim42 W at100%100\\%hardware\-active residency\.![Refer to caption](https://arxiv.org/html/2606.12765v1/figures/h4_fp8_vs_fp16.png)Figure 3:The headline: fp8 is emulated\.fp8 \(E4M3\)matmul2dtracks*below*fp16 at every size \(peaks at0\.94×0\.94\\times\), far under the1\.5×1\.5\\timesacceleration threshold: it is emulated\.
## 5Is fp8 accelerated or emulated?

We build identical tiled GEMMs differing only in operand dtype \(fp16 vs\. E4M3\), same layout, tile, and execution scope, both checksum bit\-exact\. fp8 sustains0\.8710\.871/0\.9270\.927/0\.941×0\.941\\timesfp16 at512512/10241024/20482048\([Figure3](https://arxiv.org/html/2606.12765#S4.F3)\), peaking at0\.94×0\.94\\times, nowhere near the pre\-registered1\.5×1\.5\\timesacceleration bar\. The argument is decisive: fp8 operands are one byte versus fp16’s two, so a memory\-bound kernel would be*faster*in fp8; fp8 is*slower*, so the matmul is compute\-bound and fp8 carries pure unpack overhead with no low\-precision datapath\. This is exactly the prediction for pre–neural\-accelerator silicon, and contrasts sharply with Hopper, where fp8 tensor cores roughly double fp16 throughput\[[11](https://arxiv.org/html/2606.12765#bib.bib3)\]\.On M4 Max, the Metal 4\.1 fp8matmul2dis a memory\-footprint and dynamic\-range feature, not a throughput feature; the speedup waits for M5\. Per the specification’s own framing, a falsified “faster” claim is a publishable characterization finding, and it is the practical result that most changes how a quantized model should be deployed on M4\.

![Refer to caption](https://arxiv.org/html/2606.12765v1/figures/gemm_shootout.png)Figure 4:Four\-way GEMM shootout\.Against the deprecatedsimdgroup\_matrixpath, MPP wins by only1\.051\.05to1\.21×1\.21\\times; against a naive tiled GEMM, by2\.92\.9to5\.5×5\.5\\times\.
## 6Numeric semantics and hidden constraints

Beyond throughput, a developer needs to know how the tensor path behaves numerically: how wide its accumulator is, what its low\-precision formats can represent, and what undocumented constraints a kernel must satisfy to compile at all\. This section settles each in turn\.

The width of the internal accumulator is invisible in an ordinary GEMM, so we expose it with a deliberately hostile cell\. The contraction has depthK=128K\{=\}128and consists of one large term,448448, followed by127127small terms of0\.06250\.0625, whose exact sum is455\.9375455\.9375\. That sum cannot be represented as a final fp16 value, and a kernel that accumulated in fp16 would lose every small term: the spacing between representable fp16 values near448448is0\.250\.25, larger than the terms being added, so the running total would stay pinned at448448, a1\.74%1\.74\\%error\. The fp8matmul2dinstead returns455\.9375455\.9375bit\-exactly \([Figure5\(a\)](https://arxiv.org/html/2606.12765#S6.F5.sf1)\), which is only possible if it accumulates in at least fp32\. This matches the at\-least\-fp32 convention documented for NVIDIA tensor cores\[[12](https://arxiv.org/html/2606.12765#bib.bib8)\], but Apple’s specification never states it for Metal\.

The low\-precision formats themselves are well behaved\. On M4 the fp8 \(E4M3, E5M2\) and fp4 \(E2M1\) representable sets match the OCP specifications\[[9](https://arxiv.org/html/2606.12765#bib.bib4),[10](https://arxiv.org/html/2606.12765#bib.bib5)\]exactly, with no divergence across all256256bit patterns\. The one behaviour worth flagging is saturation\. E5M2 is the only float format whose default saturation mode isnone, so on overflow it yields±∞\\pm\\infty; E4M3, by contrast, has no infinity encoding and is forced to clamp to±448\\pm 448, saturating even an\+∞\+\\inftyinput to448448\([Figure5\(b\)](https://arxiv.org/html/2606.12765#S6.F5.sf2)\)\.

A more practical surprise is an undocumented alignment rule\. A host\-boundtensor\_inlinerequires its inner stride to be aligned to128128*bytes*, which means the contracted dimension must be a multiple of128128for fp8 \(one byte per element\), of6464for half, and of3232for float\. The constraint is enforced only through a SFINAE note in the headers and appears nowhere in the specification, so a naive fp8 port of a working half kernel fails to compile, with a misleading “no matching constructor” error\.

Finally, the specification’s own advice deserves scrutiny\. It tells developers to abandonsimdgroup\_matrixin favour of tensors and MPP, but offers no evidence\. We find that MPP does win at every size, yet by only1\.051\.05to1\.21×1\.21\\times\([Figure4](https://arxiv.org/html/2606.12765#S5.F4)\); the deprecated path stays within about20%20\\%and ties at small problems\. The guidance is therefore directionally correct but overstated\.

![Refer to caption](https://arxiv.org/html/2606.12765v1/figures/fp8_matmul_accumulator.png)\(a\)Accumulator≥\\geqfp32\.
![Refer to caption](https://arxiv.org/html/2606.12765v1/figures/fp8_saturation_h7.png)\(b\)E4M3 saturation\.

Figure 5:Numeric semantics\.\(a\) The fp8matmul2drecovers a sum a sequential fp16 accumulator collapses, so it accumulates in≥\\geqfp32\. \(b\) E4M3 clamps to±448\\pm 448\(no infinity\), even saturating an\+∞\+\\inftyinput\.![Refer to caption](https://arxiv.org/html/2606.12765v1/figures/coop_layout_32x32.png)Figure 6:The reconstructed fragment layout\.The opaquecooperative\_tensorpartition is an8×88\{\\times\}8base fragment tiled across the output; each lane owns the same cell in every8×88\{\\times\}8block\. Inputs and output share this swizzle\.
## 7The opaque fragment layout

The specification declares thecooperative\_tensorpartition “device specific” and documents no layout, unlike NVIDIA’s, whose WMMA fragment layout has been recovered by microbenchmarking\[[12](https://arxiv.org/html/2606.12765#bib.bib8)\]\. We recover Apple’s by a pure layout query: instantiating the destination fragment and dumpingget\_capacity,is\_valid\_element, andget\_multidimensional\_indexper lane, with no GEMM\. For a single SIMD\-group the partition is an exact tiling: per\-lane capacity2​\(M/8\)​\(N/8\)2\\,\(M/8\)\(N/8\), the layout a repeating8×88\{\\times\}8base fragment \([Figure6](https://arxiv.org/html/2606.12765#S6.F6)\) in which a lane owns two vertically adjacent rows of one column of its quadrant, replicated across every8×88\{\\times\}8tile\. The input \(A and B\) fragments share the identical8×88\{\\times\}8swizzle: the partition is*unified*across operands and output\. This is the Apple\-Silicon analog of a documented tensor\-core fragment layout, and it is the lever for layout\-aware optimization\.

## 8Acting on the characterization

The characterization is not an end in itself; it tells us where a hand\-written kernel can beat the stock path and where it cannot\. This section turns three of the findings into concrete kernel decisions: the contiguity limits oftensor\_inline, the cooperative\-tensor destination, and the recovered fragment layout\. It reports one optimization that clears our win bar and one that, instructively, does not\.

The first decision concerns layout\.tensor\_inlineis contiguous\-only and cannot express a strided GEMM sub\-tile: itsslice\(\)does not offset the operation’s reads, so only the\(0,0\)\(0,0\)tile computes correctly\. A full tiled GEMM is nevertheless reachable without the host\-bound MTLTensor and MTL4 stack, by choosing a contiguous\-tilen​tntlayout in whichBBis pre\-transposed andCCis written tile\-contiguously; this is how we obtain the14\.814\.8TFLOP/s baseline of[Figure2\(a\)](https://arxiv.org/html/2606.12765#S4.F2.sf1)\.

The optimization that wins is epilogue fusion\. Routing a*bare*GEMM through acooperative\_tensordestination matches the host\-bound path exactly, because there is no per\-contraction round trip to skip\. The payoff is instead in the epilogue: fusing the bias and GELU in\-register on the fragment, using the recovered fragment layout, and storing the result once beats the decomposed GEMM\-then\-bias\-then\-GELU path by\+12\.9%\+12\.9\\%at102431024^\{3\}and\+6\.5%\+6\.5\\%at204832048^\{3\}\([Figure7](https://arxiv.org/html/2606.12765#S8.F7)\), which clears our1\.10×1\.10\\timesbar in the size range that matters for LLM inference; two fused operations beat one at every size\. Each epilogue round trip avoided isO​\(n2\)O\(n^\{2\}\)of memory traffic, so the win is largest for cache\-resident problems and shrinks asO​\(n3\)O\(n^\{3\}\)compute comes to dominate\. This is the same IO\-avoidance principle behind tiled\-kernel compilers\[[13](https://arxiv.org/html/2606.12765#bib.bib12)\]and FlashAttention\[[2](https://arxiv.org/html/2606.12765#bib.bib1)\]\.

The same principle does not extend to attention, and the reason is instructive\. A correct scalar FlashAttention that uses online softmax and never materializes theS×SS\{\\times\}Sscore matrix turns out to be about9×9\\times*slower*than a decomposed path that keeps thesimdgroup\_matrixGEMMs \([Figure8](https://arxiv.org/html/2606.12765#S8.F8)\), because fusing the softmax forces the matmuls off the matrix units and onto scalar code\. The matrix\-unit and split\-K FlashAttention variants we then built are correct and progressively faster, but all remain3\.63\.6to5×5\\timesslower than the decomposed path: at these sequence lengths the M4’s ample unified\-memory bandwidth makes theS×SS\{\\times\}Sround trip cheap, and our flash kernels are overhead\-bound by small tiles and barriers\. The lesson sharpens the paper’s thesis\. Cooperative\-tensor fusion pays off for cheap epilogues that keep the matrix units busy, not for fusions that replace them, and a production\-grade FlashAttention for Apple Silicon\[[3](https://arxiv.org/html/2606.12765#bib.bib2),[11](https://arxiv.org/html/2606.12765#bib.bib3)\]remains open work\.

![Refer to caption](https://arxiv.org/html/2606.12765v1/figures/rq11_fusion_win.png)Figure 7:Epilogue fusion\.Fusing bias\+\+GELU via acooperative\_tensordestination beats the decomposed path; two fused ops beat one and clear the1\.10×1\.10\\timesship bar atD≤1024D\\leq 1024\.![Refer to caption](https://arxiv.org/html/2606.12765v1/figures/rq11_attention.png)Figure 8:Attention: a negative result\.Scalar, matrix\-unit, and split\-K flash each improve, but all lose to a decomposed path on M4 atS≤8192S\\leq 8192: theS×SS\{\\times\}Sround trip is cheap on unified memory and our flash kernels are overhead\-bound\.
## 9Related work

GPU microarchitecture reverse\-engineering:Rigelextends a microbenchmarking lineage that recovers undocumented GPU internals: Jiaet al\.dissect NVIDIA Volta\[[7](https://arxiv.org/html/2606.12765#bib.bib6)\]and Turing\[[6](https://arxiv.org/html/2606.12765#bib.bib7)\]via microbenchmarks and ISA disassembly\. We apply the same empirical stance to Apple’s Metal tensor path, where the obstacle is not an undocumented ISA but a deliberately opaque high\-level primitive\.

Tensor\-core characterization:The closest prior art characterizes NVIDIA tensor cores: Markidiset al\.\[[8](https://arxiv.org/html/2606.12765#bib.bib9)\]quantify WMMA programmability, performance, and mixed\-precision loss; Sunet al\.\[[12](https://arxiv.org/html/2606.12765#bib.bib8)\]dissect tensor\-core latency, throughput, and—most relevant to us—the numeric behavior and per\-thread fragment layout of the MMA\. Our results are the Apple\-Silicon counterpoint: where these works characterize a dedicated matrix unit, we find M4 Max has none \([Section4](https://arxiv.org/html/2606.12765#S4)\), recover the analogouscooperative\_tensorfragment layout Apple documents nowhere \([Section7](https://arxiv.org/html/2606.12765#S7)\), and show the fp8 row is emulated rather than accelerated \([Section5](https://arxiv.org/html/2606.12765#S5)\)\.

Apple Silicon characterization:Recent work benchmarks Apple Silicon at the workload level: Fenget al\.\[[4](https://arxiv.org/html/2606.12765#bib.bib10)\]profile end\-to\-end ML training and attribute the NVIDIA gap to system factors; Hübneret al\.\[[5](https://arxiv.org/html/2606.12765#bib.bib11)\]evaluate M1–M4 for HPC with STREAM and SGEMM FLOPS\. These measure delivered performance; none opens the Metal 4\.1 tensor primitive itself—its dispatch target, low\-precision acceleration, accumulator width, or fragment layout—which isRigel’s contribution\.

Low\-precision formats and kernel generation:M4 implements the OCP FP8\[[9](https://arxiv.org/html/2606.12765#bib.bib4)\]and Microscaling\[[10](https://arxiv.org/html/2606.12765#bib.bib5)\]representable sets exactly; our contribution is to quantify whether the hardware*accelerates*them \(it does not, on M4\)\. Tiled\-kernel compilers\[[13](https://arxiv.org/html/2606.12765#bib.bib12)\]and the FlashAttention line\[[2](https://arxiv.org/html/2606.12765#bib.bib1),[3](https://arxiv.org/html/2606.12765#bib.bib2),[11](https://arxiv.org/html/2606.12765#bib.bib3)\]motivate our optimization study; FA\-3 in particular exploits Hopper fp8 tensor cores, precisely the hardware support M4 lacks\.

## 10Discussion and limitations

Results are on a*single*M4 Max under a*beta*OS; cross\-generation claims are out of scope and are the natural M5 follow\-up\. The macOS 27\.0 beta blocks direct ANE attribution \(CPU/ANE power unreported by both available tools; the ANE compiler rejects test models\), so[Section4](https://arxiv.org/html/2606.12765#S4)’s ANE\-exclusion is indirect pending a stable OS\. The attention kernels are research\-grade; a production FlashAttention \(register tiling, multiple SIMD\-groups per block\) may yet win, though M4’s strong decomposed path and ample bandwidth set a high bar\.

## 11Conclusion

Rigelreverse\-engineers what Metal 4\.1 hides on M4 Max:matmul2druns on the GPU shader path with no dedicated matrix unit; fp8 is emulated \(0\.94×0\.94\\timesfp16\), a footprint feature not a perf feature; the accumulator is≥\\geqfp32; the opaque8×88\{\\times\}8fragment layout is recovered; and epilogue fusion built on that layout wins by\+6\.5\+6\.5–12\.9%12\.9\\%in the cache\-resident regime\. The vendor documents an interface; we document the hardware, reproducibly, from committed code\. To that end, the microbenchmark harness, the Metal kernels, the per\-cell CSVs, and the figure\-regeneration scripts are all released under the MIT license, and every figure in this paper redraws from a committed CSV through a singlemake reproducetarget\.

## References

- \[1\]Apple Inc\.\(2026\)Metal shading language specification, version 4\.1\.Note:Apple Developer Documentation; archived copy dated 2026\-06\-04Cited by:[§1](https://arxiv.org/html/2606.12765#S1.p1.1)\.
- \[2\]T\. Dao, D\. Y\. Fu, S\. Ermon, A\. Rudra, and C\. Ré\(2022\)FlashAttention: fast and memory\-efficient exact attention with IO\-awareness\.InAdvances in Neural Information Processing Systems \(NeurIPS\),Note:arXiv:2205\.14135Cited by:[§8](https://arxiv.org/html/2606.12765#S8.p3.7),[§9](https://arxiv.org/html/2606.12765#S9.p4.1)\.
- \[3\]T\. Dao\(2023\)FlashAttention\-2: faster attention with better parallelism and work partitioning\.arXiv preprint arXiv:2307\.08691\.Cited by:[§8](https://arxiv.org/html/2606.12765#S8.p4.5),[§9](https://arxiv.org/html/2606.12765#S9.p4.1)\.
- \[4\]D\. Feng, Z\. Xu, R\. Wang, and F\. X\. Lin\(2025\)Profiling Apple Silicon performance for ML training\.arXiv preprint arXiv:2501\.14925\.Cited by:[§1](https://arxiv.org/html/2606.12765#S1.p3.1),[§9](https://arxiv.org/html/2606.12765#S9.p3.1)\.
- \[5\]P\. Hübner, A\. Hu, I\. Peng, and S\. Markidis\(2025\)Apple vs\. oranges: evaluating the Apple Silicon M\-series SoCs for HPC performance and efficiency\.arXiv preprint arXiv:2502\.05317\.Cited by:[§1](https://arxiv.org/html/2606.12765#S1.p3.1),[§9](https://arxiv.org/html/2606.12765#S9.p3.1)\.
- \[6\]Z\. Jia, M\. Maggioni, J\. Smith, and D\. P\. Scarpazza\(2019\)Dissecting the NVidia Turing T4 GPU via microbenchmarking\.arXiv preprint arXiv:1903\.07486\.Cited by:[§1](https://arxiv.org/html/2606.12765#S1.p4.1),[§9](https://arxiv.org/html/2606.12765#S9.p1.1)\.
- \[7\]Z\. Jia, M\. Maggioni, B\. Staiger, and D\. P\. Scarpazza\(2018\)Dissecting the NVIDIA Volta GPU architecture via microbenchmarking\.arXiv preprint arXiv:1804\.06826\.Cited by:[§1](https://arxiv.org/html/2606.12765#S1.p4.1),[§9](https://arxiv.org/html/2606.12765#S9.p1.1)\.
- \[8\]S\. Markidis, S\. W\. D\. Chien, E\. Laure, I\. B\. Peng, and J\. S\. Vetter\(2018\)NVIDIA tensor core programmability, performance & precision\.InIEEE International Parallel and Distributed Processing Symposium Workshops \(IPDPSW\),Note:arXiv:1803\.04014Cited by:[§1](https://arxiv.org/html/2606.12765#S1.p3.1),[§9](https://arxiv.org/html/2606.12765#S9.p2.1)\.
- \[9\]P\. Micikevicius, D\. Stosic, N\. Burgess, M\. Cornea, P\. Dubey, R\. Grisenthwaite, S\. Ha, A\. Heinecke, P\. Judd, J\. Kamalu, N\. Mellempudi, S\. Oberman, M\. Shoeybi, M\. Siu, and H\. Wu\(2022\)FP8 formats for deep learning\.arXiv preprint arXiv:2209\.05433\.Cited by:[§6](https://arxiv.org/html/2606.12765#S6.p3.5),[§9](https://arxiv.org/html/2606.12765#S9.p4.1)\.
- \[10\]Open Compute Project\(2023\)OCP microscaling formats \(MX\) specification, version 1\.0\.Technical reportOpen Compute Project Foundation\.Cited by:[§6](https://arxiv.org/html/2606.12765#S6.p3.5),[§9](https://arxiv.org/html/2606.12765#S9.p4.1)\.
- \[11\]J\. Shah, G\. Bikshandi, Y\. Zhang, V\. Thakkar, P\. Ramani, and T\. Dao\(2024\)FlashAttention\-3: fast and accurate attention with asynchrony and low\-precision\.InAdvances in Neural Information Processing Systems \(NeurIPS\),Note:arXiv:2407\.08608Cited by:[§5](https://arxiv.org/html/2606.12765#S5.p1.8),[§8](https://arxiv.org/html/2606.12765#S8.p4.5),[§9](https://arxiv.org/html/2606.12765#S9.p4.1)\.
- \[12\]W\. Sun, A\. Li, T\. Geng, S\. Stuijk, and H\. Corporaal\(2023\)Dissecting tensor cores via microbenchmarks: latency, throughput and numeric behaviors\.IEEE Transactions on Parallel and Distributed Systems34\(1\),pp\. 246–261\.Note:arXiv:2206\.02874External Links:[Document](https://dx.doi.org/10.1109/TPDS.2022.3217824)Cited by:[§1](https://arxiv.org/html/2606.12765#S1.p3.1),[§6](https://arxiv.org/html/2606.12765#S6.p2.10),[§7](https://arxiv.org/html/2606.12765#S7.p1.4),[§9](https://arxiv.org/html/2606.12765#S9.p2.1)\.
- \[13\]P\. Tillet, H\. T\. Kung, and D\. Cox\(2019\)Triton: an intermediate language and compiler for tiled neural network computations\.InProc\. 3rd ACM SIGPLAN Int\. Workshop on Machine Learning and Programming Languages \(MAPL\),External Links:[Document](https://dx.doi.org/10.1145/3315508.3329973)Cited by:[§8](https://arxiv.org/html/2606.12765#S8.p3.7),[§9](https://arxiv.org/html/2606.12765#S9.p4.1)\.