Neuromorphic Diffusion Language Models: Addressing Compute and Memory Bottlenecks via Sparsity and Block Denoising

# Neuromorphic Diffusion Language Models: Addressing Compute and Memory Bottlenecks via Sparsity and Block Denoising
Source: [https://arxiv.org/html/2607.24841](https://arxiv.org/html/2607.24841)
Dengyu Wu1, Clement Ruah2, Jiechen Chen1, Bipin Rajendran2, and Osvaldo Simeone2 1Department of Engineering, King’s College London, London, UK 2Institute for Intelligent Networked Systems \(INSI\), Northeastern University London, London, UK E\-mails:\{dengyu\.wu, jiechen\.chen\}@kcl\.ac\.uk, \{c\.ruah, b\.rajendran, o\.simeone\}@nulondon\.ac\.ukThis work was supported by the European Research Council \(ERC\) under the European Union’s Horizon Europe Programme \(grant agreement No\. 101198347\), by Open Fellowships of the EPSRC \(EP/W024101/1, EP/X011356/1\), by the EPSRC project \(EP/X011852/1\)\.

###### Abstract

Autoregressive \(AR\)large language models \(LLMs\)are inherently inefficient at inference time because each generated token requires accessing the full set of model parameters, leading to low operational intensity and high energy consumption\. Masked diffusion language models \(MDLMs\) partially address this limitation for memory\-bound settings by allowing multiple tokens to be generated per parameter access\. In order to further enhance inference efficiency on modern platforms with extensive in\-chip memory, this work proposes neuromorphic MDLMs \(N\-MDLMs\), which integrate block diffusion with spike\-based neuromorphic computation to jointly improve throughput and energy efficiency\. While block diffusion increases token throughput by producing multiple tokens per parameter access, spike\-induced sparsity reduces effective parameter traffic and computations by skipping inactive channels\. To analyze the synergistic effect of sparsity and diffusion, we develop a token\-level roofline\-inspired model that captures the combined impact of block\-parallel generation and spike sparsity on decoding efficiency\. Experimental results on translation tasks show that, thanks to spike\-induced sparsity, N\-MDLMs achieve substantial improvements in energy efficiency and throughput even in compute\-bound platforms for which MDLMs would fail to improve over AR\-LLMs\.

## IIntroduction

Large language models \(LLMs\)are increasingly deployed in latency\-sensitive and cost\-constrained settings, where inference efficiency, in terms of memory utilization, throughput, and energy consumption, determines system performance\. However, conventional*autoregressive*\(AR\)LLMsrequire repeatedly accessing the full set of model parameters, resulting in limited operational intensity and low throughput\[bi2026rooflinebench,verhelst2025keep\]\.

To alleviate these limitations,*masked diffusion language models*\(MDLMs\) generate multiple tokens per parameter access\[arriola2025block\]\. However, energy efficiency is still constrained by the use of transformer models in each denoising step\. Furthermore, throughput benefits hinge on the system operating in a memory\-bound regime, and may not translate to modern inference platforms with substantial in\-chip memory\[appuswamy2024breakthrough,abts2022software\]\.

![Refer to caption](https://arxiv.org/html/2607.24841v1/Figures/intro_spike_arch.png)Figure 1:In the proposed neuromorphic masked diffusion language model \(N\-MDLM\), a neuromorphic encoder processes the clean prefix tokens and produces neuromorphic contextual representations\. The neuromorphic decoder iteratively refines a block of masked tokens through denoising steps while reusing the neuromorphic representations from the encoder\. Each neuromorphic transformer layer is implemented with IF neuron dynamics \(MLP = multilayer perceptron\)\.*Neuromorphic computing*principles have recently been shown to offer potential benefits in reducing energy consumption for standardARtransformers\[zhou2022spikformer,simeone2026modern,pan2025spikingbrain\]\. Neuromorphic models operate in an event\-driven manner, with computation triggered by events, also known as spikes, representing discrete activations\. In the absence of a spike, tailored hardware deployments can avoid computation and memory access, benefiting from dynamic sparsity\[simeone2026modern\]\.

In this work, as shown in Fig\.[1](https://arxiv.org/html/2607.24841#S1.F1), we introduce*neuromorphic masked diffusion language models*\(N\-MDLMs\), a model class that enhances throughput via parallel denoising steps, while further improving inference efficiency via the dynamic sparsity afforded by neuromorphic computing\. As illustrated in Fig\.[2](https://arxiv.org/html/2607.24841#S1.F2), while conventionalAR\-LLMsgenerate one token per parameter access,MDLMsamortize this cost across multiple tokens within a block\[arriola2025block\]\.N\-MDLMsfurther enhance operational intensity through event\-driven sparsity, thereby reducing both parameter traffic and computational cost\. Even on modern neuromorphic hardware\[davies2018loihi,appuswamy2024breakthrough\]andLLMinference accelerators\[abts2022software\]that co\-locate memory and compute inside the chip, this can shift the operation towards a memory\-bound regime that makes parallel generation via diffusion both throughput and energy efficient\.

![Refer to caption](https://arxiv.org/html/2607.24841v1/Figures/introduction.png)Figure 2:FromAR\-LLMto the proposedN\-MDLM: \(a\) AnAR\-LLMrequires accessing the full set of model parameters for each generated token\. \(b\) AnMDLMgeneratesBBtokens per parameter access, increasing the number of operations per access proportionally toBB\. \(c\) The proposedN\-MDLMfurther exploits spiking sparsity, such that only active channels trigger synaptic operations, thereby reducing both parameter traffic and computational cost\.The main contributions of this work are as follows:

- •*NeuromorphicMDLM:*We introduceN\-MDLMs, a novel class of neuromorphicLLMsobtained via conversion of a conventionalMDLM\[arriola2025block\]based on quantization andintegrate\-and\-fire \(IF\)spiking models\[pan2025spikingbrain\]\.
- •*Analysis:*We develop a token\-level roofline\-inspired analytical framework that characterizes the achievable decoding efficiency of N\-MDLMs as a function of block size and activation sparsity, capturing the joint and synergistic effects of block\-parallel generation and event\-driven computation on both computation and memory access\.
- •*Experimental validation:*We validate the proposed framework on translation tasks, demonstrating that N\-MDLMs achieve improved token throughput and energy efficiency compared to AR\-LLMs and conventional MDLMs, while maintaining competitive task performance\. The gains are seen to arise from a synergistic use of sparsity, to alleviate bottlenecks due to computational limitations, and diffusion, to mitigate throughput constraints due to memory transfer\.

## IIPreliminaries

In this section, we start by reviewing AR\-LLMs and MDLMs, as well as the IF neuron model\.

### II\-AAutoregressive vs\. Masked Diffusion Language Models

AR\-LLMsfactorize the joint distributionpθ​\(x\)p\_\{\\theta\}\(x\)over the token sequencex=\(x1,…,xL\)x=\(x\_\{1\},\\ldots,x\_\{L\}\)as

pθ​\(x\)=∏l=1Lpθ​\(xl∣x<l\),p\_\{\\theta\}\(x\)=\\prod\_\{l=1\}^\{L\}p\_\{\\theta\}\(x\_\{l\}\\mid x\_\{<l\}\),\(1\)whereθ\\thetadenotes the model parameters and each tokenxlx\_\{l\}takes values in the vocabulary𝒱\\mathcal\{V\}\. In \([1](https://arxiv.org/html/2607.24841#S2.E1)\), the conditional distributionpθ​\(xl\|x<l\)p\_\{\\theta\}\(x\_\{l\}\|x\_\{<l\}\)assigns a probability to tokenxlx\_\{l\}given the prefixx<l=\(x1,…,xl−1\)x\_\{<l\}=\(x\_\{1\},\\ldots,x\_\{l\-1\}\)consisting of previous tokens\. The factorization in \([1](https://arxiv.org/html/2607.24841#S2.E1)\) induces a strictly sequential inference procedure such that at each decoding step the model produces a single tokenxlx\_\{l\}\. Accordingly, as illustrated in Fig\.[2](https://arxiv.org/html/2607.24841#S1.F2)\(a\), each parameter access yields one output token, limiting operational intensity and throughput\.

MDLMs\[arriola2025block\]relax the sequential operation by grouping tokens into blocks of sizeBBtokens and generating each block in parallel via a number of denoising steps \(see Section[III\-A](https://arxiv.org/html/2607.24841#S3.SS1)\)\. Specifically, the sequencex=\(x1,…,xL\)x=\(x\_\{1\},\\ldots,x\_\{L\}\)is partitioned into blocksxb=\(xb,1,…,xb,B\)x^\{b\}=\(x\_\{b,1\},\\ldots,x\_\{b,B\}\)withb=1,…,⌈L/B⌉b=1,\\ldots,\\lceil L/B\\rceil, and the joint distribution is factorized at the block level as

pθ​\(x\)=∏b=1⌈L/B⌉pθ​\(xb∣x<b\),p\_\{\\theta\}\(x\)=\\prod\_\{b=1\}^\{\\lceil L/B\\rceil\}p\_\{\\theta\}\(x^\{b\}\\mid x^\{<b\}\),\(2\)wherex<b=\(x1,…,xb−1\)x^\{<b\}=\(x^\{1\},\\ldots,x^\{b\-1\}\)denotes the previously generated blocks\.

### II\-BNeuromorphic Computing Principles

Neuromorphic computing systems represent information using discrete spike events generated by neuron dynamics\. A commonly used neuron model is the*integrate\-and\-fire*\(IF\) neuron with a soft\-reset mechanism\[wu2025optimizing\]\. For any neuron, a spike event, corresponding to outputst=1s\_\{t\}=1, is generated when the internal state variableVtV\_\{t\}crosses a thresholdVthV\_\{\\mathrm\{th\}\}as

st=\{1,Vt≥Vth,0,Vt<Vth\.s\_\{t\}=\\begin\{cases\}1,&V\_\{t\}\\geq V\_\{\\mathrm\{th\}\},\\\\ 0,&V\_\{t\}<V\_\{\\mathrm\{th\}\}\.\\end\{cases\}\(3\)Given an input synaptic currentItI\_\{t\}, the state variable evolves as

Vt\+1=Vt\+It−st​Vth\.V\_\{t\+1\}=V\_\{t\}\+I\_\{t\}\-s\_\{t\}V\_\{\\mathrm\{th\}\}\.\(4\)The synaptic currentItI\_\{t\}results from incoming spikes from presynaptic neurons interacting with their synaptic weights\.

Under event\-driven execution, synaptic operations are triggered only when spikes occur\. Consequently, inactive neurons incur neither computation nor memory access, leading to*dynamic sparsity*in both compute and parameter traffic\[davies2018loihi\]\.

## IIINeuromorphic Masked Diffusion Models

In this section, we introduceN\-MDLMs, a neuromorphic block diffusionLLMconstructed by converting existing conventional encoder\-decoder\-basedMDLMs\[pan2025spikingbrain\]\.

### III\-AArchitecture

As illustrated in Fig\.[1](https://arxiv.org/html/2607.24841#S1.F1), the proposedN\-MDLMfollows a block diffusion generation paradigm in which multiple tokens are predicted within a block through an iterative denoising process\. We specifically adopt the encoder\-decoder architecture from\[arriola2025encoderdecoder\]\. In it, the encoder processes the current context to produce a representation that serves as conditioning information for the decoder during the denoising process\.

At encoding stagebbbased on factorization \([2](https://arxiv.org/html/2607.24841#S2.E2)\), the model generates blockxbx^\{b\}given contextx<b=\(x1,…,xb−1\)x^\{<b\}=\(x^\{1\},\\ldots,x^\{b\-1\}\)\. The encoderfθenc​\(⋅\)f\_\{\\theta\_\{\\mathrm\{enc\}\}\}\(\\cdot\)first processes the contextx<bx^\{<b\}to produce the latent representation

hb=fθenc​\(x<b\)\.h^\{b\}=f\_\{\\theta\_\{\\mathrm\{enc\}\}\}\(x^\{<b\}\)\.\(5\)
Using the latent representationhbh^\{b\}, generation at the decoder starts from initially masked tokens representationz0b∈ℝB×Dz\_\{0\}^\{b\}\\in\\mathbb\{R\}^\{B\\times D\}, whereDDdenotes the hidden dimension of the transformer\. The matrixz0bz\_\{0\}^\{b\}contains by column the embedding of the special mask token for allBBtokens in the block\. This initial uninformative signal is iteratively refined overSSdenoising steps indexed ass=1,…,Ss=1,\\ldots,S\. At each denoising stepss,B/SB/Stokens, with ratioB/SB/Sassumed to be an integer, are selected for remaining masked position to be unmasked\[arriola2025block\]\. To this end, the transformer decoderfθdec​\(⋅\)f\_\{\\theta\_\{\\mathrm\{dec\}\}\}\(\\cdot\)maps the contextual representationhbh^\{b\}and the current noisy blockzsb=\(zs,1b,…,zs,Bb\)z\_\{s\}^\{b\}=\(z\_\{s,1\}^\{b\},\\ldots,z\_\{s,B\}^\{b\}\)to produce updated token embeddings

zs\+1b=fθdec​\(zsb,hb\)∈ℝB×D,z\_\{s\+1\}^\{b\}=f\_\{\\theta\_\{\\mathrm\{dec\}\}\}\(z\_\{s\}^\{b\},\\;h^\{b\}\)\\in\\mathbb\{R\}^\{B\\times D\},\(6\)withB/SB/Sadditional tokens unmasked as compared tozsbz\_\{s\}^\{b\}\.

### III\-BConversion

Starting from a pre\-trained encoder\-decoderMDLM\[pan2025spikingbrain\], we obtain an N\-MDLM via a quantization\-based conversion methodology that leverages theIFneuron model \([4](https://arxiv.org/html/2607.24841#S2.E4)\)\[pan2025spikingbrain\]\. The main principle underlying this approach is that the spiking rate of anIFneuron, or variants thereof, can match the output of a quantization function in which the quantization step size equals the thresholdVthV\_\{\\rm th\}in \([3](https://arxiv.org/html/2607.24841#S2.E3)\)\[pan2025spikingbrain\]\.

Given an activation vector𝒂=\(a1,…,aD\)\\boldsymbol\{a\}=\(a\_\{1\},\\ldots,a\_\{D\}\)produced by a layer of the encoder or decoder, a quantization function with an adaptive step size obtains a quantized integer\-valued vector𝒂^∈ℤD\\hat\{\\boldsymbol\{a\}\}\\in\\mathbb\{Z\}^\{D\}as\[pan2025spikingbrain\]

𝒂^=round​\(𝒂Vth​\(𝒂\)\),Vth​\(𝒂\)=1K​mean​\(\|𝒂\|\),\\hat\{\\boldsymbol\{a\}\}=\\mathrm\{round\}\\\!\\left\(\\frac\{\\boldsymbol\{a\}\}\{V\_\{\\mathrm\{th\}\}\(\\boldsymbol\{a\}\)\}\\right\),\\qquad V\_\{\\mathrm\{th\}\}\(\\boldsymbol\{a\}\)=\\frac\{1\}\{K\}\\,\\mathrm\{mean\}\(\|\\boldsymbol\{a\}\|\),\(7\)where the rounding functionround​\(⋅\)\\mathrm\{round\}\(\\cdot\)and the magnitude function\|⋅\|\|\\cdot\|are applied element\-wise\. The parameterKKin \([7](https://arxiv.org/html/2607.24841#S3.E7)\) controls the number of quantization levels, with larger values ofKKincreasing the resolution\.

Each quantized valuea^i\\hat\{a\}\_\{i\}is represented via bitwise coding, a temporal coding scheme using ternary spikessi,t∈\{−1,0,1\}s\_\{i,t\}\\in\\\{\-1,0,1\\\}overTTtime steps\. The spikes are related to integera^i\\hat\{a\}\_\{i\}as

a^i=∑t=1T2t−1​si,t,\\hat\{a\}\_\{i\}=\\sum\_\{t=1\}^\{T\}2^\{t\-1\}\\,s\_\{i,t\},\(8\)whereT=⌊log2⁡\(maxi⁡\|a^i\|\)⌋T=\\left\\lfloor\\log\_\{2\}\\left\(\\max\_\{i\}\|\\hat\{a\}\_\{i\}\|\\right\)\\right\\rfloordenotes the number of time steps\.

Let𝒘=\[w1,…,wD\]⊤∈ℝD\\boldsymbol\{w\}=\[w\_\{1\},\.\.\.,w\_\{D\}\]^\{\\top\}\\in\\mathbb\{R\}^\{D\}be a weight vector\. This may correspond to the row of a linear layer or to the linear projection in an MLP\. The inner product𝒘⊤​𝒂^\\boldsymbol\{w\}^\{\\top\}\\hat\{\\boldsymbol\{a\}\}for a quantized activation vector𝒂^\\hat\{\\boldsymbol\{a\}\}can be evaluated using a modified IF neuron operating on the spiking signals in \([8](https://arxiv.org/html/2607.24841#S3.E8)\)\. To this end, letIt=∑i=1Dwi​si,tI\_\{t\}=\\sum\_\{i=1\}^\{D\}w\_\{i\}s\_\{i,t\}be the input current to the IF neuron at timesteptt, and consider the modified IF neuron with dynamics

Vt\+1=Vt\+2t​It−2t​s^t​Vth​\(𝒂\),V\_\{t\+1\}=V\_\{t\}\+2^\{t\}I\_\{t\}\-2^\{t\}\\hat\{s\}\_\{t\}V\_\{\\mathrm\{th\}\}\(\\boldsymbol\{a\}\),\(9\)wheres^t\\hat\{s\}\_\{t\}denotes the emitted spike, computed as

s^t=\{1,Vt≥2t​Vth​\(𝒂\),−1,Vt≤−2t​Vth​\(𝒂\),0,otherwise\.\\hat\{s\}\_\{t\}=\\begin\{cases\}1,&V\_\{t\}\\geq 2^\{t\}V\_\{\\mathrm\{th\}\}\(\\boldsymbol\{a\}\),\\\\ \-1,&V\_\{t\}\\leq\-2^\{t\}V\_\{\\mathrm\{th\}\}\(\\boldsymbol\{a\}\),\\\\ 0,&\\text\{otherwise\}\.\\end\{cases\}\(10\)As a result, the linear transformation can be approximately reconstructed as\[pan2025spikingbrain\]

𝒘⊤​𝒂^≈Vth​\(𝒂\)​∑t=1T2t−1​s^i,t\.\\boldsymbol\{w\}^\{\\top\}\\hat\{\\boldsymbol\{a\}\}\\approx V\_\{\\mathrm\{th\}\}\(\\boldsymbol\{a\}\)\\sum\_\{t=1\}^\{T\}2^\{t\-1\}\\,\\hat\{s\}\_\{i,t\}\.\(11\)

## IVPerformance Analysis

Compared to conventionalAR\-LLMs,N\-MDLMscan modulate data access and computation via the choice of block sizeBB, amortizing the parameter accesses per block, and the quantization hyperparameterKK, controlling the sparsity of generated spikes\. In this section, we study the effect of block sizeBBand sparsity hyperparameterKKon the per\-token number of operations and memory transfer requirements, yielding an estimated token throughput, within a roofline\-inspired framework\[verhelst2025keep\]\.

### IV\-AComputational and Memory\-Transfer Load

Following\[bi2026rooflinebench\], we assume that the cost of a forward pass in all models is dominated by operations in linear layers\. For parallel computation ofBBtokens, these are represented by operations of the formY=X​W⊤∈ℝB​T×DY=XW^\{\\top\}\\in\\mathbb\{R\}^\{BT\\times D\}, with parameter matrixW∈ℝD×DW\\in\\mathbb\{R\}^\{D\\times D\}and inputX∈ℝB​T×DX\\in\\mathbb\{R\}^\{BT\\times D\}obtained by concatenating the spike train encodingBBconsecutive tokens overTTtime steps\. We formally define the sparsity level as the probabilityσ=P​\(xt,d=0\)∈\[0,1\]\\sigma=P\(x\_\{t,d\}=0\)\\in\[0,1\]that an elementxt,dx\_\{t,d\}from matrixXXis zero, assuming all entriesxt,dx\_\{t,d\}to be independent and identically distributed for allt∈\{1,…,B​T\}t\\in\\\{1,\.\.\.,BT\\\}andd∈\{1,…,D\}d\\in\\\{1,\.\.\.,D\\\}\. The sparsity levelσ\\sigmais controlled by the hyperparameterKK, with a largerKKgenerally reducingσ\\sigma\[pan2025spikingbrain\]\.

We further make the following event\-driven computation assumptions\[davies2018loihi\]:\(i\)\(i\)only spike eventsxt,d=1x\_\{t,d\}=1contribute to the computation load; and\(i​i\)\(ii\)thedd\-th row of matrixWWis loaded only if there is at least one spikext,d=1x\_\{t,d\}=1within the corresponding batch channel sequenceXd=\{xt,d\}t∈\{1,…,B​T\}X\_\{d\}=\\\{x\_\{t,d\}\\\}\_\{t\\in\\\{1,\.\.\.,BT\\\}\}\.

#### IV\-A1Per\-Token Computational Load

Assumption\(i\)\(i\)implies that each element in the outputYYneeds on average2​D​\(1−σ\)2D\(1\-\\sigma\)operations, counting additions and multiplications separately\[bi2026rooflinebench\]\. Accordingly, the total average number of operations per token is given by

Nops​\(B,σ\)=2​T​D2​\(1−σ\)\.N\_\{\\mathrm\{ops\}\}\(B,\\sigma\)=2TD^\{2\}\(1\-\\sigma\)\.\(12\)

#### IV\-A2Per\-Token Memory\-Transfer Load

Assumption\(i​i\)\(ii\)implies that each row of matrixWWis loaded with probability1−σB​T1\-\\sigma^\{BT\}\. Accordingly, the average number of bits to be loaded from memory per token is

Ndata​\(B,σ\)=D2​\(1−σB​T\)B​bW,N\_\{\\mathrm\{data\}\}\(B,\\sigma\)=\\frac\{D^\{2\}\(1\-\\sigma^\{BT\}\)\}\{B\}b\_\{W\},\(13\)wherebW≥1b\_\{W\}\\geq 1is the number of bits per weight\. Note that we have ignored the cost of transferringXXandYYfrom and to memory as, in practice, it is negligible compared to the cost of loading the weights\[bi2026rooflinebench\]\. The quantities in \([12](https://arxiv.org/html/2607.24841#S4.E12)\) and \([13](https://arxiv.org/html/2607.24841#S4.E13)\) also apply to linear layers inAR\-LLMs, withB=1B=1andσ=0\\sigma=0, andMDLMs, withB≥1B\\geq 1andσ=0\\sigma=0\.

### IV\-BToken Throughput

For a hardware platform capable of producingLopsL\_\{\\mathrm\{ops\}\}operations per second and of movingLdataL\_\{\\mathrm\{data\}\}bits per second, the theoretical attainable token latency \(TL\) for aN\-MDLMlinear layer is

TL​\(B,σ\)=max⁡\{Nops​\(B,σ\)Lops⏟compute\-bound,Ndata​\(B,σ\)Ldata⏟memory\-bound\},\\mathrm\{TL\}\(B,\\sigma\)=\\max\\bigg\\\{\\underbrace\{\\frac\{N\_\{\\mathrm\{ops\}\}\(B,\\sigma\)\}\{L\_\{\\mathrm\{ops\}\}\}\}\_\{\\text\{compute\-bound\}\},\\underbrace\{\\frac\{N\_\{\\mathrm\{data\}\}\(B,\\sigma\)\}\{L\_\{\\mathrm\{data\}\}\}\}\_\{\\text\{memory\-bound\}\}\\bigg\\\},\(14\)which is measured in seconds per token\. The end\-to\-end latency of the model can then be approximated by summing the individual latenciesTL​\(B,σ\)\\mathrm\{TL\}\(B,\\sigma\)of each linear layer during a forward pass\. When the equalityTL​\(B,σ\)=Nops​\(B,σ\)/Lops\\mathrm\{TL\}\(B,\\sigma\)=N\_\{\\mathrm\{ops\}\}\(B,\\sigma\)/L\_\{\\mathrm\{ops\}\}holds, the system is said to be in a*compute\-bound regime*, and throughput gains can only come from an increase in the computation speedLopsL\_\{\\mathrm\{ops\}\}\. Conversely, when the equationTL​\(B,σ\)=Ndata​\(B,σ\)/\(B​Ldata\)\\mathrm\{TL\}\(B,\\sigma\)=N\_\{\\mathrm\{data\}\}\(B,\\sigma\)/\(BL\_\{\\mathrm\{data\}\}\)holds, the system is in a*memory\-bound regime*, and throughput can be improved by increasing the memory bandwidthLdataL\_\{\\mathrm\{data\}\}\.

The expression \([14](https://arxiv.org/html/2607.24841#S4.E14)\) highlights a*regime\-dependent*tradeoff between the block sizeBBand the sparsity levelσ\\sigma: when the system is compute\-bound, the latency gains inN\-MDLMsare linearly proportional to the sparsity levelσ\\sigmaregardless of the block sizeBB\. In contrast, when the system is in a memory\-bound regime, increasing the block sizeBBreduces the benefits of sparsity, with the effective data\-loading sparsity reduced toσB​T<σ\\sigma^\{BT\}<\\sigma\. Moreover, increasing the block sizeBBsteers the system towards a compute\-bound regime, while increasing the sparsity levelσ\\sigmasteers it towards a memory\-bound regime\. Therefore, in order to get maximal benefits from sparsity,N\-MDLMsshould ideally run on systems with high memory bandwidthLdataL\_\{\\mathrm\{data\}\}, such as near\-memory compute neuromorphic devices\[davies2018loihi,appuswamy2024breakthrough\], using a larger block sizeBBas a means to ease the system into compute\-bound regimes\.

## VExperiments

In this section, we validate the performance ofN\-MDLMsvia experimental results on theWMT 14 de\-entranslation task\[bojar\-etal\-2014\-findings\]\. All experiments are conducted on an NVIDIA DGX Sparkgraphics processing unit \(GPU\)\.

### V\-ASetup

For translation experiments on theWMT 14 de\-endataset\[bojar\-etal\-2014\-findings\]we adopt theefficient encoder\-decoder diffusion \(E2D2\)architecture in\[arriola2025encoderdecoder\]withNenc=28N\_\{\\mathrm\{enc\}\}=28encoder layers andNdec=4N\_\{\\mathrm\{dec\}\}=4decoder layers, embedding dimensionDemb=512D\_\{\\mathrm\{emb\}\}=512and hidden intermediate dimensionDhid=1536D\_\{\\mathrm\{hid\}\}=1536\(approximately 250M parameters\)\[arriola2025encoderdecoder\]\. TheN\-MDLMintroduced in Sec\.[III\-A](https://arxiv.org/html/2607.24841#S3.SS1)is simulated onGPUusing bitwise coding forT=8T=8time steps\. After applying the conversion pipeline described in Section[III\-B](https://arxiv.org/html/2607.24841#S3.SS2)starting from the pretrained model released by\[arriola2025encoderdecoder\], we perform an additional round of fine\-tuning\. Fine\-tuning uses the Adam optimizer over10001000steps, with batch size1616and learning rate3×10−53\\times 10^\{\-5\}on theWMT 14 de\-endataset\. In the experiments, we consider the settingS=BS=B, so that one token is unmasked at each denoising step\.

### V\-BPerformance Metrics

We evaluate the performance of all models in terms of token throughput \(token/s\), energy per token \(J/token\), and accuracy\. For throughput and energy, we leverage the analysis in Sec\.[IV](https://arxiv.org/html/2607.24841#S4), together with sparsity estimates obtained by running the model on the mentionedGPUplatform\.

#### V\-B1Sparsity, Per\-Token Computational Load, and Per\-Token Memory\-Transfer Load

In order to apply the analysis in Sec\.[IV](https://arxiv.org/html/2607.24841#S4), we empirically measure the spike sparsityσ\\sigmaat each linear layer operation during inference\. In turn, the collected sparsity values are used to estimate theoretical number of spike operationsNops​\(B,σ\)N\_\{\\mathrm\{ops\}\}\(B,\\sigma\)\(OPs\) in \([12](https://arxiv.org/html/2607.24841#S4.E12)\) and data movementNdata​\(B,σ\)N\_\{\\mathrm\{data\}\}\(B,\\sigma\)\(bits\) in \([13](https://arxiv.org/html/2607.24841#S4.E13)\)\. We set the precision of weights tobW=16b\_\{W\}=16bits\.

To evaluate the end\-to\-end performance of the model, we denote asNopsenc​\(n,m\)N^\{\\mathrm\{enc\}\(n,m\)\}\_\{\\mathrm\{ops\}\}andNdataenc​\(n,m\)N^\{\\mathrm\{enc\}\(n,m\)\}\_\{\\mathrm\{data\}\}the respective number of estimated operations and memory transfers at themm\-th linear layer of thenn\-th encoding layer, forn∈\{1,…,Nenc\}n\\in\\\{1,\.\.\.,N\_\{\\mathrm\{enc\}\}\\\}andm∈\{1,…,Menc​\(n\)\}m\\in\\\{1,\.\.\.,M\_\{\\mathrm\{enc\}\}\(n\)\\\}, whereMenc​\(n\)M\_\{\\mathrm\{enc\}\}\(n\)denotes the number of linear layers at encoding layernn\. Similarly, we denote asNopsdec​\(n,m,k\)N^\{\\mathrm\{dec\}\(n,m,k\)\}\_\{\\mathrm\{ops\}\}andNdatadec​\(n,m,k\)N^\{\\mathrm\{dec\}\(n,m,k\)\}\_\{\\mathrm\{data\}\}the values estimated at the decoding layers forn∈\{1,…,Ndec\}n\\in\\\{1,\.\.\.,N\_\{\\mathrm\{dec\}\}\\\}andm∈\{1,…,Mdec​\(n\)\}m\\in\\\{1,\.\.\.,M\_\{\\mathrm\{dec\}\}\(n\)\\\}, wherek∈\{1,…,S\}k\\in\\\{1,\.\.\.,S\\\}indexes the denoising step\. Generally, for a fixed layer position\(n,m\)\(n,m\), the valuesNopsdec​\(n,m,k\)N^\{\\mathrm\{dec\}\(n,m,k\)\}\_\{\\mathrm\{ops\}\}andNdataenc​\(n,m,k\)N^\{\\mathrm\{enc\}\(n,m,k\)\}\_\{\\mathrm\{data\}\}can vary across denoising stepsSS, as each step is characterized by a different sparsity level\. These quantities are used to evaluate token throughput and energy per token, as discussed next\.

#### V\-B2Token Throughput

Using the estimated number of computation operations and data transfers, we derive the corresponding token latencies using \([14](https://arxiv.org/html/2607.24841#S4.E14)\) asTLenc​\(n,m\)=max⁡\{Nopsenc​\(n,m\)/Lops,Ndataenc​\(n,m\)/Ldata\}\\mathrm\{TL\}^\{\\mathrm\{enc\}\(n,m\)\}=\\max\\\{N^\{\\mathrm\{enc\}\(n,m\)\}\_\{\\mathrm\{ops\}\}/L\_\{\\mathrm\{ops\}\},N^\{\\mathrm\{enc\}\(n,m\)\}\_\{\\mathrm\{data\}\}/L\_\{\\mathrm\{data\}\}\\\}andTLdec​\(n,m,k\)=max⁡\{Nopsdec​\(n,m,k\)/Lops,Ndatadec​\(n,m,k\)/Ldata\}\\mathrm\{TL\}^\{\\mathrm\{dec\}\(n,m,k\)\}=\\max\\\{N^\{\\mathrm\{dec\}\(n,m,k\)\}\_\{\\mathrm\{ops\}\}/L\_\{\\mathrm\{ops\}\},N^\{\\mathrm\{dec\}\(n,m,k\)\}\_\{\\mathrm\{data\}\}/L\_\{\\mathrm\{data\}\}\\\}for each individual linear operation in the encoder and the decoder, respectively\. Recall that these quantities depend on the computational throughputLopsL\_\{\\mathrm\{ops\}\}and memory bandwidthLdataL\_\{\\mathrm\{data\}\}\. The estimated end\-to\-end token throughput of the system is then computed by summing the latencies of individual layers as

TTe2e=\(∑n=1Nenc∑m=1Menc​\(n\)TLenc​\(n,m\)\+∑n=1Ndec∑m=1Mdec​\(n\)∑k=1STLdec​\(n,m,k\)\)−1,\\mathrm\{TT\}^\{\\mathrm\{e2e\}\}=\\bigg\(\\sum\_\{n=1\}^\{N\_\{\\mathrm\{enc\}\}\}\\sum\_\{m=1\}^\{M\_\{\\mathrm\{enc\}\}\(n\)\}\\mathrm\{TL\}^\{\\mathrm\{enc\}\(n,m\)\}\\\\ \+\\sum\_\{n=1\}^\{N\_\{\\mathrm\{dec\}\}\}\\sum\_\{m=1\}^\{M\_\{\\mathrm\{dec\}\}\(n\)\}\\sum\_\{k=1\}^\{S\}\\mathrm\{TL\}^\{\\mathrm\{dec\}\(n,m,k\)\}\\bigg\)^\{\-1\},\(15\)which is measured in tokens per second\.

#### V\-B3Energy per Token

Similar to token throughput, the energy per token is estimated from the number of operations and memory transfers at the linear layers\. Given the energyEopsE\_\{\\mathrm\{ops\}\}consumed per operation and the energyEdataE\_\{\\mathrm\{data\}\}required per bit transferred from memory, the total energy consumed per token is estimated as\[verhelst2025keep\]

Ee2e=∑n=1Nenc∑m=1Menc​\(n\)\{Nopsenc​\(n,m\)​Eops\+Ndataenc​\(n,m\)​Edata\}\+∑n=1Ndec∑m=1Mdec​\(n\)∑k=1S\{Nopsdec​\(n,m,k\)​Eops\+Ndatadec​\(n,m,k\)​Edata\}\.E^\{\\mathrm\{e2e\}\}=\\sum\_\{n=1\}^\{N\_\{\\mathrm\{enc\}\}\}\\sum\_\{m=1\}^\{M\_\{\\mathrm\{enc\}\}\(n\)\}\\left\\\{N^\{\\mathrm\{enc\}\(n,m\)\}\_\{\\mathrm\{ops\}\}E\_\{\\mathrm\{ops\}\}\+N^\{\\mathrm\{enc\}\(n,m\)\}\_\{\\mathrm\{data\}\}E\_\{\\mathrm\{data\}\}\\right\\\}\\\\ \+\\sum\_\{n=1\}^\{N\_\{\\mathrm\{dec\}\}\}\\sum\_\{m=1\}^\{M\_\{\\mathrm\{dec\}\}\(n\)\}\\sum\_\{k=1\}^\{S\}\\left\\\{N^\{\\mathrm\{dec\}\(n,m,k\)\}\_\{\\mathrm\{ops\}\}E\_\{\\mathrm\{ops\}\}\+N^\{\\mathrm\{dec\}\(n,m,k\)\}\_\{\\mathrm\{data\}\}E\_\{\\mathrm\{data\}\}\\right\\\}\.\(16\)

#### V\-B4In\-chip Memory vs\. Off\-chip Memory Systems

TABLE I:Hardware parameters for representative off\-chip memory system \(OCMS\) and in\-chip memory system \(ICMS\)\.The benefits of MDLMs were demonstrated in\[arriola2025encoderdecoder\]for conventional memory\-bound platforms, like GPUs, that are characterized by an off\-chip memory system \(OCMS\)\. In this work, we first validate the benefits of MDLMs on OCMS by adopting the hardware parameters, dictating computation rateLopsL\_\{\\mathrm\{ops\}\}, memory bandwidthLdataL\_\{\\mathrm\{data\}\}, energy per operationEopsE\_\{\\mathrm\{ops\}\}, and energy per memory transferEdataE\_\{\\mathrm\{data\}\}, in Table[I](https://arxiv.org/html/2607.24841#S5.T1), which are obtained from GPU metrics for tensor cores andhigh bandwidth memory \(HBM\)\[nvidia2020a100\]\.

Using this validation as a benchmark, we are interested in evaluating throughput and energy performance levels on anin\-chip memory system \(ICMS\), which is aligned with neuromorphic hardware\[davies2018loihi,appuswamy2024breakthrough\]andLLMinference accelerators\[abts2022software\]that co\-locate memory and compute inside the chip\. Accordingly, the parameters for theICMSin Table[I](https://arxiv.org/html/2607.24841#S5.T1)correspond to a significantly smaller compute\-to\-bandwidth ratio, i\.e\.,Lops/Ldata<1L\_\{\\mathrm\{ops\}\}/L\_\{\\mathrm\{data\}\}<1OP/b\. Energy consumption parameters in Table[I](https://arxiv.org/html/2607.24841#S5.T1)are inspired by the numbers of in\-chip and off\-chip memory access in\[murmann2020mixed\]\.

### V\-CResults

![Refer to caption](https://arxiv.org/html/2607.24841v1/Figures/throughput_B.png)Figure 3:Normalized throughput versus block sizeBBfor AR\-LLM, MDLM, N\-AR\-LLM, and N\-MDLM, when running on ICMS \(solid lines\) and OCMS \(dashed lines\)\. Throughput, controlled by the sparsity parameterKKin \([7](https://arxiv.org/html/2607.24841#S3.E7)\), is computed using \([15](https://arxiv.org/html/2607.24841#S5.E15)\) based on the computational and memory\-transfer loads in \([12](https://arxiv.org/html/2607.24841#S4.E12)\) and \([13](https://arxiv.org/html/2607.24841#S4.E13)\)\. The throughput is normalized with respect to the AR\-LLM baseline of the corresponding hardware\.In Fig\.[3](https://arxiv.org/html/2607.24841#S5.F3), we plot the end\-to\-end throughput computed using \([15](https://arxiv.org/html/2607.24841#S5.E15)\) as a function of the block sizeBBfor AR\-LLM, MDLM, and N\-MDLM with different values of the parameterKKin \([7](https://arxiv.org/html/2607.24841#S3.E7)\)\. Recall that smaller values ofKKindicate more sparsity\. We also consider a neuromorphic version of AR\-LLM, referred to as N\-AR\-LLM, which applies the same conversion pipeline described in Sec\.[III\-B](https://arxiv.org/html/2607.24841#S3.SS2)starting from the AR\-LLM\. The throughput is normalized with respect to the AR\-LLM baseline so as to highlight the improvements achieved via diffusion and via event\-driven processing\. Following the discussion in Sec\.[V\-B](https://arxiv.org/html/2607.24841#S5.SS2), we focus on the ICMS configuration, but we also show the performance of AR\-LLM and MDLM for the OCMS as reference, reflecting a conventional GPU\-like implementation for standard models\.

As reported in\[arriola2025block\], in the memory\-bound regime of OCMS, MDLM is seen to have an increasing throughput as a function of the block sizeBB\(dashed line\)\. In contrast, in the compute\-bound regime of ICMS, increasing the block sizeBBdoes not yield gains for MDLM \(solid line\)\. This is because, in a compute\-bound regime, the throughput is limited by the compute rateLopsL\_\{\\mathrm\{ops\}\}, and it does not benefit from amortizing memory access via diffusion\.

By introducing sparsity, N\-MDLM reduces both computation and memory traffic\. This lowers the arithmetic intensity, i\.e\., the number of operations per memory access, steering the system toward a memory\-bound regime\. As a result, in the ICMS configuration, N\-MDLM is able to extract performance gains on hardware where plain MDLM shows little improvement\. In particular, the throughput of N\-MDLM is seen in Fig\.[3](https://arxiv.org/html/2607.24841#S5.F3)to increase with block sizeBB, eventually saturating when the system enters a compute\-bound regime\.

![Refer to caption](https://arxiv.org/html/2607.24841v1/Figures/throughput_energy.png)Figure 4:Normalized throughput versus energy per token for AR\-LLM, MDLM withB∈\{2,4,8\}B\\in\\\{2,4,8\\\}, N\-AR\-LLM, and N\-MDLM \(B=\{2,4,8\}B=\\\{2,4,8\\\}\), when running on ICMS \(solid lines\) and OCMS \(dashed lines\)\. Throughput is computed using \([15](https://arxiv.org/html/2607.24841#S5.E15)\), while energy per token is obtained from \([16](https://arxiv.org/html/2607.24841#S5.E16)\)\. The throughput is normalized with respect to the AR\-LLM baseline of the corresponding hardware\.Fig\.[4](https://arxiv.org/html/2607.24841#S5.F4)illustrates the relationship between throughput, computed via \([14](https://arxiv.org/html/2607.24841#S4.E14)\), and energy per token, computed via \([16](https://arxiv.org/html/2607.24841#S5.E16)\), for AR\-LLM, N\-AR\-LLM, MDLM, and N\-MDLM with different values of the sparsity\-inducing parameterKK\. The points for MDLM and N\-MDLM are obtained by varying the block sizeB∈\{2,4,8\}B\\in\\\{2,4,8\\\}\(larger values ofBBcorrespond to larger energy consumption\), while AR\-LLM and N\-AR\-LLM correspond toB=1B=1\. Moving from AR\-LLM to MDLM, energy consumption decreases due to improved amortization of memory access\. In fact, as shown by the hardware parameters in Table[I](https://arxiv.org/html/2607.24841#S5.T1), memory access incurs a significantly higher energy cost than computation\. Therefore, reducing memory traffic is the primary driver of energy efficiency gains in AR\-LLM and MDLM\.

N\-MDLM further reduces energy consumption, with the lowest energy achieved at the highest sparsity level, i\.e\.,K=1K=1\. This trend reflects the effect of event\-driven computation, whereby operations are only triggered by spike events\.

![Refer to caption](https://arxiv.org/html/2607.24841v1/Figures/normalized_performances.png)Figure 5:BLEU, energy per token, and throughput across AR\-LLM, MDLM \(B=4B\{=\}4\), and*neuromorphic masked diffusion language models*\(N\-MDLMs\) \(B=4B\{=\}4\) in ICMS\. Throughput and energy are obtained from \([14](https://arxiv.org/html/2607.24841#S4.E14)\) and \([16](https://arxiv.org/html/2607.24841#S5.E16)\), respectively\. All three metrics are independently normalized to the interval\[0,1\]\[0,1\]using their maximum values across configurations\.Finally, Fig\.[5](https://arxiv.org/html/2607.24841#S5.F5)compares normalized throughput, computed via \([15](https://arxiv.org/html/2607.24841#S5.E15)\), energy per token, computed via \([16](https://arxiv.org/html/2607.24841#S5.E16)\), and BLEU, a measure of accuracy for the given task\[bojar\-etal\-2014\-findings\], for AR\-LLM, MDLM \(B=4B=4\), and N\-MDLM \(B=4B=4\)\. The results show that*neuromorphic masked diffusion language models*\(N\-MDLMs\) outperform AR\-LLM and MDLM in terms of both throughput and energy efficiency, with only a slight drop in BLEU, indicating that the efficiency gains are achieved with minimal impact on translation quality\.

Overall, the results highlight a trade\-off between block sizeBBand sparsity level controlled byKK\. Augmenting the block sizeBBimproves the throughput of a memory\-bound system up to the point where it becomes compute bound\. It is in this regime that sparsity becomes effective in improving throughput\. In the considered ICMS, this is achieved forB=4B=4in N\-MDLM models withK=1K=1orK=2K=2\. Augmenting the block sizeBBbeyond this point does not yield additional throughput gains\.

With respect to energy, where the cost is dominated by memory access, the system always gains from augmentingBBto amortize memory access\.

## VIConclusions

This paper investigated the integration of masked diffusion language models with neuromorphic computing principles for efficient large language model inference\. Specifically, we proposedN\-MDLMs, which combines block\-parallel diffusion decoding with spike\-based sparse computation\. While block diffusion increases the number of tokens generated per parameter access, spike\-induced sparsity reduces effective synaptic operations by skipping inactive neurons\. We further developed a roofline\-inspired analysis that characterizes models in terms of throughput, energy, and accuracy\. Experiments on translation tasks show that the proposed framework improves decoding throughput and energy efficiency while maintaining task performance\. The gains are particularly significant in compute\-bound regimes, where plain MDLMs provide limited improvement\. By combining block parallelism with sparsity,*neuromorphic masked diffusion language models*\(N\-MDLMs\) reduce the computational load and enable throughput gains\. Future work will focus on validation on neuromorphic hardware and on adaptive strategies to jointly optimizeBBandKKunder varying system constraints\.

## References