Gradient-Free Training of Spiking Neural Networks via Low-Rank Evolution Strategies

# Gradient-Free Training of Spiking Neural Networks via Low-Rank Evolution Strategies
Source: [https://arxiv.org/html/2605.30361](https://arxiv.org/html/2605.30361)
Sachit Ramesha GowdaShunya Research sachit@shunyaresearch\.systems

###### Abstract

Spiking Neural Networks \(SNNs\) offer compelling energy efficiency on neuromorphic hardware, yet their training remains challenging because the discrete spike threshold is non\-differentiable\. Surrogate\-gradient methods sidestep this by approximating the derivative, but they impose backpropagation infrastructure that is incompatible with on\-chip learning\. Evolution Strategies \(ES\) are a natural gradient\-free alternative, yet their computational cost scales with the number of parameters, making them impractical for large weight matrices\.

We present a method for training SNNs usingEggroll, a low\-rank factorisation of ES perturbations that reduces per\-generation memory from𝒪​\(m​n\)\\mathcal\{O\}\(mn\)to𝒪​\(r​\(m\+n\)\)\\mathcal\{O\}\(r\(m\{\+\}n\)\)\. CombiningEggrollwith a Leaky Integrate\-and\-Fire SNN on N\-MNIST, we demonstrate that gradient\-free training achieves 79\.21% test accuracy while reducing per\-generation wall\-clock time by 2\.23×\\timesrelative to full\-rank ES\. Our results demonstrate EGGROLL is viable for SNN training, with a clear accuracy\-speed tradeoff, compatible with training on neuromorphic hardware without surrogate gradients\.

## 1Introduction

Energy Efficiency is a huge focus of modern AI research\. A lot of research is focused on reducing the energy usage of AI architecture\. Intel’s Loihi\(Davieset al\.,[2018](https://arxiv.org/html/2605.30361#bib.bib6)\)showed us that SNNs could be a potential solution on real hardware\. The dominant paradaigm in current AI is the transformer architecture\(Vaswaniet al\.,[2017](https://arxiv.org/html/2605.30361#bib.bib2)\)\. Transformers are trained on GPUs and use hundreds of watts in their training process\. While the artificial neural networks with transformers show remarkable performance, they use a significant amount of energy\. The human brain, in comparison, is extremely efficient, only using 20 watts\. SNNs attempt to capture this efficiency: neurons integrate inputs over time and fire only when a membrane potential threshold is crossed, enabling sparse, event\-driven computation on dedicated neuromorphic chips such as Intel Loihi\(Davieset al\.,[2018](https://arxiv.org/html/2605.30361#bib.bib6)\)and IBM TrueNorth\(Merollaet al\.,[2014](https://arxiv.org/html/2605.30361#bib.bib7)\)\.

#### The training problem\.

Despite their efficiency during inference, SNNs are still quite difficult to train\. The spiking generation function\(Heavside step\) has a zero gradient almost everywhere\. This makes it impossible to do standard backpropagation, since it requires calculating the gradient\. The common solution to this is using surrogate gradients\(Neftciet al\.,[2019](https://arxiv.org/html/2605.30361#bib.bib5)\)\. Surrogate gradients work by replacing the true gradient of a function with an approximation, which can be differentiated\. This allows backpropagation to proceed through layers where it would otherwise be impossible\. However, surrogate gradients still require autograd infrastructure, which is not compatible with on chip learning on neuromorphic hardware\.

#### Evolution Strategies\.

ES\(Salimanset al\.,[2017](https://arxiv.org/html/2605.30361#bib.bib4)\)is a group of loosely bio inspired training methods that are alternatives to backpropagation that do not require gradients to be calculated\. In ES, weight vectors are randomly perturbed and then a fitness evaluation is done on these perturbations\. However these methods are very computationally expensive, incurringO​\(P​m​n\)O\(Pmn\)memory for each generation, for a weight matrix of dimensionsm​×​nm×nand populationPP, making it very challenging to scale on larger networks\.

#### Our contribution\.

We integrateEggroll\(Sarkar and others,[2025](https://arxiv.org/html/2605.30361#bib.bib1)\)— which replaces each full\-rank perturbation with a low\-rank product𝐀𝐁⊤\\mathbf\{AB\}^\{\\top\},𝐀∈ℝm×r\\mathbf\{A\}\\in\\mathbb\{R\}^\{m\\times r\},𝐁∈ℝn×r\\mathbf\{B\}\\in\\mathbb\{R\}^\{n\\times r\}— into the SNN training loop\. This reduces per\-generation cost to𝒪​\(r​\(m\+n\)\)\\mathcal\{O\}\(r\(m\{\+\}n\)\)while preserving the gradient\-free property\.

Concretely, our contributions are:

1. 1\.we characterize EGGROLL’s behavior on non\-differentiable spike functionsEggrollwith SNNs, enabling gradient\-free training without surrogate approximations \(Section[3](https://arxiv.org/html/2605.30361#S3)\)\.
2. 2\.A rank ablation showing that accuracy remains relatively stable asrrdecreases, with a clear efficiency–accuracy Pareto frontier \(Section[4](https://arxiv.org/html/2605.30361#S4)\)\.
3. 3\.Comparison against vanilla ES and surrogate\-gradient BPTT on N\-MNIST\. N\-MNIST is a native neuromorphic dataset, with no rate\-coding approximation used, making this a more principled evaluation than the one done on a static MNIST dataset\. \(Section[4](https://arxiv.org/html/2605.30361#S4)\)\.

## 2Background

### 2\.1N\-MNIST Dataset

The N\-MNIST dataset\(Orchardet al\.,[2015](https://arxiv.org/html/2605.30361#bib.bib3)\)or Neuromorphic MNIST is a spiking version of the popular MNIST dataset that contains static images of hand drawn numbers\. It consists of the same 60,000 training and 10,000 test samples as the original MNIST dataset\. It was created by mounting the ATIS sensor on a motorized pan tilt unit and moving it while it recorded MNIST images on an LCD display\. The sensor outputs asynchronous events with 2 polarities: ON events \(higher brightness\) and OFF events \(lower brightness\)\. This is a better fit for SNNs than the regular MNIST dataset because the data is already a spike train, so no rate coding approximation is required\. Due to this, the SNN processes the actual sensor output instead of the surrogate encoding\.

### 2\.2Spiking Neural Networks and the LIF Model

We model each neuron as a Leaky Integrate\-and\-Fire unit\. At each discrete timesteptt, the membrane potentialVtV\_\{t\}evolves as

Vt=α​Vt−1\+𝐰⊤​𝐬t−1,V\_\{t\}=\\alpha V\_\{t\-1\}\+\\mathbf\{w\}^\{\\top\}\\mathbf\{s\}\_\{t\-1\},\(1\)whereα∈\(0,1\)\\alpha\\in\(0,1\)is the membrane decay constant \(leak factor\),𝐰\\mathbf\{w\}the synaptic weight vector, and𝐬t−1\\mathbf\{s\}\_\{t\-1\}the binary spike vector from the previous layer\. A spike is emitted whenVt≥VthV\_\{t\}\\geq V\_\{\\text\{th\}\}, after whichVtV\_\{t\}resets toVreset=0V\_\{\\text\{reset\}\}=0\. The network output is the spike count \(firing rate\) overTTtimesteps\.

### 2\.3Surrogate Gradient Training

Because the spike functionΘ​\(V−Vth\)\\Theta\(V\-V\_\{\\text\{th\}\}\)is a Heaviside step, its true derivative is zero almost everywhere\. Surrogate gradient methods\(Neftciet al\.,[2019](https://arxiv.org/html/2605.30361#bib.bib5)\)substitute a smooth proxyΘ^′\\hat\{\\Theta\}^\{\\prime\}— commonly the fast sigmoid — only during the backward pass, leaving the forward pass unchanged\. This recovers gradient flow at the cost of introducing an approximation bias and requiring full backpropagation\-through\-time \(BPTT\)\. Implementing this method requires autograd, which as mentioned earlier, is not compatible with on\-chip learning, and makes it unsuitable for neuromorphic hardware\.

### 2\.4OpenAI Evolution Strategies

Salimanset al\.\([2017](https://arxiv.org/html/2605.30361#bib.bib4)\)propose estimating the gradient of expected fitnessJ​\(𝜽\)J\(\\boldsymbol\{\\theta\}\)as

∇𝜽J≈12​P​σ​∑i=1P\[F​\(𝜽\+σ​𝜺i\)−F​\(𝜽−σ​𝜺i\)\]​𝜺i,\\nabla\_\{\\boldsymbol\{\\theta\}\}J\\approx\\frac\{1\}\{2P\\sigma\}\\sum\_\{i=1\}^\{P\}\\bigl\[F\(\\boldsymbol\{\\theta\}\+\\sigma\\boldsymbol\{\\varepsilon\}\_\{i\}\)\-F\(\\boldsymbol\{\\theta\}\-\\sigma\\boldsymbol\{\\varepsilon\}\_\{i\}\)\\bigr\]\\boldsymbol\{\\varepsilon\}\_\{i\},\(2\)where𝜺i∼𝒩​\(𝟎,𝐈\)\\boldsymbol\{\\varepsilon\}\_\{i\}\\sim\\mathcal\{N\}\(\\mathbf\{0\},\\mathbf\{I\}\),PPis the population size, andσ\\sigmathe perturbation scale\. Antithetic \(mirrored\) sampling reduces variance with no extra sampling budget\. The estimate is passed to Adam\(Kingma and Ba,[2014](https://arxiv.org/html/2605.30361#bib.bib13)\)as if it were a true gradient\. The biggest issue with ES is that its memory cost is𝒪​\(P​m​n\)\\mathcal\{O\}\(Pmn\), making it slow for large models\.

### 2\.5EGGROLL: Low\-Rank Evolution Strategies

Sarkar and others\([2025](https://arxiv.org/html/2605.30361#bib.bib1)\)observe that sampling a full perturbation matrix𝐄∈ℝm×n\\mathbf\{E\}\\in\\mathbb\{R\}^\{m\\times n\}is wasteful: for populationPP, the perturbation tensor requires𝒪​\(P​m​n\)\\mathcal\{O\}\(Pmn\)memory\.Eggrollinstead draws

𝐄i=1r​𝐀i​𝐁i⊤,𝐀i∈ℝm×r,𝐁i∈ℝn×r,\\mathbf\{E\}\_\{i\}=\\frac\{1\}\{\\sqrt\{r\}\}\\mathbf\{A\}\_\{i\}\\mathbf\{B\}\_\{i\}^\{\\top\},\\qquad\\mathbf\{A\}\_\{i\}\\in\\mathbb\{R\}^\{m\\times r\},\\;\\mathbf\{B\}\_\{i\}\\in\\mathbb\{R\}^\{n\\times r\},\(3\)normalising byr\\sqrt\{r\}so that each entry has unit variance regardless of rank, without which the variance scales byrr, rendering theσ\\sigmahyperparameter obsolete\. Additionally, EGGROLL reconstructs noise on demand using a counter based deterministic random number generator\(RNG\), so that the perturbations do not have to be stored in memory\. This trick helps EGGROLL work on the billion parameter scale\. The gradient estimate is then reconstructed via the\(diag⁡\(𝐟\)​𝐀\)⊤​𝐁\(\\operatorname\{diag\}\(\\mathbf\{f\}\)\\mathbf\{A\}\)^\{\\top\}\\mathbf\{B\}formulation \(§4\.2 ofSarkar and others\([2025](https://arxiv.org/html/2605.30361#bib.bib1)\)\), which never materialises individual perturbation matrices\. Memory per generation drops to𝒪​\(r​\(m\+n\)\)\\mathcal\{O\}\(r\(m\{\+\}n\)\)\.

## 3Method

### 3\.1Network Architecture

We use a two\-layer LIF network:

2312→64→LIF10→LIFoutput\.2312\\xrightarrow\{\}64\\xrightarrow\{\\text\{LIF\}\}10\\xrightarrow\{\\text\{LIF\}\}\\text\{output\}\.\(4\)The input is of size 2312 since N\-MNIST has a size of34​X​3434X34pixels, and 2 polarities for each pixel\. Each LIF layer shares a common membrane decayβ\\beta\. Initial weights are sampled from𝒩​\(0,0\.32\)\\mathcal\{N\}\(0,0\.3^\{2\}\)\. The output is the mean spike rate overTTtimesteps, converted to class probabilities via softmax for fitness evaluation\. Biases are perturbed with independent 1\-D Gaussian factors to maintain equivalence with full\-rank ES whenr=min⁡\(m,n\)r=\\min\(m,n\)\. Unlike static\-image SNNs, the forward pass feeds a different event frame to each timestep not the same image repeated T times\. Event counts are clamped to\[0,1\]\[0,1\]to prevent a single pixel dominating membrane potential\.

### 3\.2EGGROLL Integration

Algorithm[1](https://arxiv.org/html/2605.30361#alg1)details the training loop\. Instead of backpropogating through the SNN the usual way, the algorithm starts with network weights and creates many perturbed copies of the network which are subsequently run on a minibatch\. The performance of each copy is measured and that score is then converted into a gradient like signal which updates the weights with Adam\. The key departure from vanilla ES is the batched forward pass in Line[6](https://arxiv.org/html/2605.30361#alg1.l6): allPPperturbed networks are evaluated simultaneously by expanding the data tensor along a population dimension, avoiding a sequential Python loop over population members\. The factors𝐀\\mathbf\{A\},𝐁\\mathbf\{B\}are regenerated from a stored seed rather than cached, keeping GPU memory independent ofPP\. Centered rank normalization is done to make the gradient scale unaffected by absolute fitness values across batches\. Antithetic sampling is also used to reduce the variance\.

Algorithm 1Eggrolltraining for an SNN1:rank

rr, population

PP, scale

σ\\sigma, learning rate

η\\eta, generations

GG, timesteps

TT
2:Initialise weights

𝜽=\{𝐖1,𝐛1,𝐖2,𝐛2\}\\boldsymbol\{\\theta\}=\\\{\\mathbf\{W\}\_\{1\},\\mathbf\{b\}\_\{1\},\\mathbf\{W\}\_\{2\},\\mathbf\{b\}\_\{2\}\\\}from

𝒩​\(0,0\.32\)\\mathcal\{N\}\(0,0\.3^\{2\}\)
3:for

g=1,…,Gg=1,\\ldots,Gdo

4:Sample seed

ss; draw data mini\-batch

\(𝐗,𝐲\)\(\\mathbf\{X\},\\mathbf\{y\}\)
5:Reconstruct

𝐀i,𝐁i,𝐜i\\mathbf\{A\}\_\{i\},\\mathbf\{B\}\_\{i\},\\mathbf\{c\}\_\{i\}from

ssfor

i=1,…,Pi=1,\\ldots,P
6:Batched forward:evaluate all

PPperturbed nets on

𝐗\\mathbf\{X\}
7:Collect antithetic rewards

ri\+r\_\{i\}^\{\+\},

ri−r\_\{i\}^\{\-\}for each population member

8:Apply centered rank normalisation to

\{ri\+,ri−\}\\\{r\_\{i\}^\{\+\},r\_\{i\}^\{\-\}\\\}
9:Compute gradient

∇^\\hat\{\\nabla\}via

\(diag⁡\(𝐟\)​𝐀\)⊤​𝐁\(\\operatorname\{diag\}\(\\mathbf\{f\}\)\\mathbf\{A\}\)^\{\\top\}\\mathbf\{B\}
10:Update

𝜽←𝜽−η⋅Adam​\(∇^\)\\boldsymbol\{\\theta\}\\leftarrow\\boldsymbol\{\\theta\}\-\\eta\\cdot\\text\{Adam\}\(\\hat\{\\nabla\}\)
11:endfor

### 3\.3Fitness Function

We use negative cross\-entropy \(log\-likelihood\) as the fitness signal, rather than raw classification accuracy:

F​\(𝜽\)=−1B​∑b=1Blog⁡exp⁡\(y^b,yb\)∑cexp⁡\(y^b,c\),F\(\\boldsymbol\{\\theta\}\)=\-\\frac\{1\}\{B\}\\sum\_\{b=1\}^\{B\}\\log\\frac\{\\exp\(\\hat\{y\}\_\{b,y\_\{b\}\}\)\}\{\\sum\_\{c\}\\exp\(\\hat\{y\}\_\{b,c\}\)\},\(5\)wherey^b,c\\hat\{y\}\_\{b,c\}is the spike rate for classccon examplebb\. Log\-likelihood provides a smooth landscape for ES to navigate; raw accuracy \(a step function over fitness values\) gives no gradient signal between threshold crossings\.

## 4Experiments

### 4\.1Setup

#### Dataset\.

N\-MNIST\(Orchardet al\.,[2015](https://arxiv.org/html/2605.30361#bib.bib3)\): 54,000 training / 6,000 validation / 10,000 test images, generator seed 0\. Sensor size34​X​3434X34, 2 polarities\. No rate coding is used, the event frames are the spiking input\.

#### Baselines\.

- •Vanilla ES: full\-rank Gaussian perturbations, sequential evaluation, identical hyperparameters\.
- •Surrogate\-gradient BPTT: fast\-sigmoid surrogate, Adam optimiser, same architecture\.

#### Compute\.

All experiments run on a single NVIDIA RTX 3070 Ti\. Wall\-clock times are averaged over three seeds\.

### 4\.2Main Results

Table 1:Test accuracy and per\-generation wall\-clock time\. Mean±\\pmstd over three seeds\.The key takeaway from[Table 1](https://arxiv.org/html/2605.30361#S4.T1)is the significant speedup that EGGROLL provides to ES, for almost no tradeoff in accuracy\. Even with a rank of 1, the difference in accuracy is very minute relative to the speedup provided by EGGROLL\. Though Surrogate BPTT has a higher accuracy than ES, it is significantly slower\.

### 4\.3Rank Ablation

Figure[1](https://arxiv.org/html/2605.30361#S4.F1)plots validation accuracy and per\-generation wall\-clock time as functions of rankrr\.

![Refer to caption](https://arxiv.org/html/2605.30361v1/x1.png)\(a\)Accuracy vs\. rank\.
![Refer to caption](https://arxiv.org/html/2605.30361v1/x2.png)\(b\)Wall\-clock vs\. rank\.

Figure 1:Rank ablation on N\-MNIST\.The accuracy remains relatively stable across ranks, with the best performance occurring atr=1r=1andr=4r=4, with larger variance atr=2r=2andr=8r=8\. Wall clock time is lowest atr=1r=1\. It increases forr=2r=2, and remains relatively stable across the other 2 ranks\.

### 4\.4Convergence Curves

Figure[2](https://arxiv.org/html/2605.30361#S4.F2)plots the convergence of different ES implementations\(Naive and with EGGROLL\) across generations\. Figure[3](https://arxiv.org/html/2605.30361#S4.F3)plots the average wall clock time for vanilla ES and EGGROLL\.

![Refer to caption](https://arxiv.org/html/2605.30361v1/x3.png)Figure 2:Validation accuracy over generations for all methods \(mean±\\pmstd, three seeds\)\. Dashed line: surrogate BPTT final accuracy\.![Refer to caption](https://arxiv.org/html/2605.30361v1/x4.png)Figure 3:Wall clock time\(in seconds\) for EGGROLL implementations and vanilla ESEGGROLL withr=4r=4converges slightly faster than the other EGGROLL variants, withr=1r=1slightly behind, but overall the difference in convergence speed is modest\. The variance bands \(mean±\\pmstd over three seeds\) are generally narrow forEggrollwith small ranks, indicating more stable convergence than several higher\-rank or full\-rank baselines\. EGGROLL achieves substantially lower wall\-clock training time due to its reduced per\-generation cost\. In particular EGGROLL reaches comparable validation accuracy in fewer seconds than vanilla ES despite similar generation\-level convergence behavior\.

## 5Discussion

#### Accuracy gap\.

Eggrollachieves 79\.21% accuracy vs\. 94\.03% for surrogate BPTT\. We attribute the gap to our limited compute budget as a result of which we could only run 100 generations for the ES training\(naive and EGGROLL\)\. Additionally with ES, the gradient estimate can be noisy because of stochastic perturbations and mini\-batch variation\. Due to our limited compute budget, we used a lightweight two\-layer fully connected SNN, which keeps the parameter count modest, but limits our representational capacity relative to deeper architectures\. Per\-seed results \(Appendix[A](https://arxiv.org/html/2605.30361#A1)\) show occasional training collapse atr=2r=2,r=8r=8, suggesting sensitivity to initialization that warrants further study\.

#### Rank and expressiveness\.

The rank ablation \(Figure[1](https://arxiv.org/html/2605.30361#S4.F1)\) shows that evenr=1r=1retains substantial accuracy, suggesting the useful gradient directions lie in a low\-dimensional subspace\. This is consistent with the flat minima hypothesis\(Hochreiter and Schmidhuber,[1997](https://arxiv.org/html/2605.30361#bib.bib10)\)and with findings on intrinsic dimensionality in neural networks\(Liet al\.,[2018](https://arxiv.org/html/2605.30361#bib.bib14)\)\. The ablations are empirical evidence for this in SNNs\.

#### Limitations\.

Our experiments are limited to N\-MNIST with a two\-layer, fully connected network with no convolutions, and 10 fixed time bins which may discard some temporal resolution\. Additionally we have not tested this on real neuromorphic hardware, and have not measured the energy consumption\. Scaling to CIFAR10\-DVS\(Liet al\.,[2017](https://arxiv.org/html/2605.30361#bib.bib9)\)and N\-Caltech101\(Orchardet al\.,[2015](https://arxiv.org/html/2605.30361#bib.bib3)\)is the next step, since these are more complex than N\-MNIST\. Three seeds is insufficient for definitive rank claims; we report per\-seed results in Appendix[A](https://arxiv.org/html/2605.30361#A1)for transparency\.

## 6Related Work

#### SNN training methods\.

Spike\-Timing\-Dependent Plasticity \(STDP\) is a local Hebbian rule that requires no global error signal\(Bi and Poo,[1998](https://arxiv.org/html/2605.30361#bib.bib18)\)but struggles to match backprop accuracy\. Surrogate gradients\(Neftciet al\.,[2019](https://arxiv.org/html/2605.30361#bib.bib5); Bellecet al\.,[2020](https://arxiv.org/html/2605.30361#bib.bib15)\)are currently the dominant approach for deep SNNs, achieving state\-of\-the\-art accuracy but requiring autograd\. Conversion methods\(Caoet al\.,[2015](https://arxiv.org/html/2605.30361#bib.bib16)\)train an ANN and convert weights, but incur long simulation times\.

#### Evolution Strategies for neural networks\.

Salimanset al\.\([2017](https://arxiv.org/html/2605.30361#bib.bib4)\)showed ES competitive with RL on Atari\.Suchet al\.\([2017](https://arxiv.org/html/2605.30361#bib.bib17)\)demonstrated ES on deep networks via compact perturbation seeds\. PEPG\(Sehnkeet al\.,[2010](https://arxiv.org/html/2605.30361#bib.bib8)\)and CMA\-ES\(Hansen,[2016](https://arxiv.org/html/2605.30361#bib.bib19)\)are closely related natural\-gradient variants\.Eggroll\(Sarkar and others,[2025](https://arxiv.org/html/2605.30361#bib.bib1)\)is, to our knowledge, the first explicit low\-rank factorisation of ES perturbations shown to scale to large models; our work is the first to apply it to SNNs\.

#### Neuromorphic hardware constraints\.

Loihi\(Davieset al\.,[2018](https://arxiv.org/html/2605.30361#bib.bib6)\)and BrainScaleS\(Schemmelet al\.,[2010](https://arxiv.org/html/2605.30361#bib.bib20)\)support on\-chip learning but only with local rules or very limited precision\. Gradient\-free methods are a natural fit for these constraints, motivating this line of work\.

#### EGGROLL

Sarkar and others\([2025](https://arxiv.org/html/2605.30361#bib.bib1)\)showed that EGGROLL can work on language models and RNNs\. As far as we know, our work is first application of EGGROLL on SNNs, that poses the challenge of non differentiable spikes\.

## 7Conclusion

Training on SNNs is challenging due to its discrete spike threshold\. We have demonstrated thatEggroll— low\-rank Evolution Strategies — can train SNNs on N\-MNIST without surrogate gradients or backpropagation, achieving 79\.21% test accuracy at 2\.23×\\timesthe speed of vanilla ES\. The rankr=4r=4offers the best accuracy\-per\-second tradeoff in our setup\.

#### Future work\.

The most immediate extension is scaling to more complex neuromorphic benchmarks, like N\-Caltech101\(Orchardet al\.,[2015](https://arxiv.org/html/2605.30361#bib.bib3)\)and DVS\-CIFAR10\(Orchardet al\.,[2015](https://arxiv.org/html/2605.30361#bib.bib3)\)with a convolutional SNN\. Beyond accuracy, a key open question is whetherEggroll\-trained SNNs produce sparser spike trains than surrogate\-gradient methods, which would directly translate to energy savings on neuromorphic hardware\. Finally, combiningEggrollwith local plasticity rules for the early layers — reserving global ES updates for the readout layer — may offer the best of both worlds\.

## References

- G\. Bellec, F\. Scherr, A\. Subramoney,et al\.\(2020\)A solution to the learning dilemma for recurrent networks of spiking neurons\.Nature Communications11\(1\),pp\. 3625\.Cited by:[§6](https://arxiv.org/html/2605.30361#S6.SS0.SSS0.Px1.p1.1)\.
- G\. Bi and M\. Poo \(1998\)Synaptic modifications in cultured hippocampal neurons: dependence on spike timing, synaptic strength, and postsynaptic cell type\.Journal of Neuroscience18\(24\),pp\. 10464–10472\.Cited by:[§6](https://arxiv.org/html/2605.30361#S6.SS0.SSS0.Px1.p1.1)\.
- Y\. Cao, Y\. Chen, and D\. Khosla \(2015\)Spiking deep convolutional neural networks for energy\-efficient object recognition\.Vol\.113,pp\. 54–66\.Cited by:[§6](https://arxiv.org/html/2605.30361#S6.SS0.SSS0.Px1.p1.1)\.
- M\. Davies, N\. Srinivasa, T\. Lin,et al\.\(2018\)Loihi: a neuromorphic manycore processor with on\-chip learning\.IEEE Micro38\(1\),pp\. 82–99\.Cited by:[§1](https://arxiv.org/html/2605.30361#S1.p1.1),[§6](https://arxiv.org/html/2605.30361#S6.SS0.SSS0.Px3.p1.1)\.
- N\. Hansen \(2016\)The cma evolution strategy: a tutorial\.arXiv preprint arXiv:1604\.00772\.Cited by:[§6](https://arxiv.org/html/2605.30361#S6.SS0.SSS0.Px2.p1.1)\.
- S\. Hochreiter and J\. Schmidhuber \(1997\)Flat minima\.Neural Computation9\(1\),pp\. 1–42\.Cited by:[§5](https://arxiv.org/html/2605.30361#S5.SS0.SSS0.Px2.p1.1)\.
- D\. P\. Kingma and J\. Ba \(2014\)Adam: a method for stochastic optimization\.arXiv preprint arXiv:1412\.6980\.Cited by:[§2\.4](https://arxiv.org/html/2605.30361#S2.SS4.p1.5)\.
- C\. Li, H\. Farkhoor, R\. Liu, and J\. Yosinski \(2018\)Measuring the intrinsic dimension of objective landscapes\.InInternational Conference on Learning Representations,Cited by:[§5](https://arxiv.org/html/2605.30361#S5.SS0.SSS0.Px2.p1.1)\.
- H\. Li, H\. Liu, X\. Ji, G\. Li, and L\. Shi \(2017\)CIFAR10\-dvs: an event\-stream dataset for object classification\.Frontiers in NeuroscienceVolume 11 \- 2017\.External Links:[Link](https://www.frontiersin.org/journals/neuroscience/articles/10.3389/fnins.2017.00309),[Document](https://dx.doi.org/10.3389/fnins.2017.00309),ISSN 1662\-453XCited by:[§5](https://arxiv.org/html/2605.30361#S5.SS0.SSS0.Px3.p1.1)\.
- P\. A\. Merolla, J\. V\. Arthur, R\. Alvarez\-Icaza,et al\.\(2014\)A million spiking\-neuron integrated circuit with a scalable communication network and interface\.Science345\(6197\),pp\. 668–673\.Cited by:[§1](https://arxiv.org/html/2605.30361#S1.p1.1)\.
- E\. O\. Neftci, H\. Mostafa, and F\. Zenke \(2019\)Surrogate gradient learning in spiking neural networks\.IEEE Signal Processing Magazine36\(6\),pp\. 51–63\.Cited by:[§1](https://arxiv.org/html/2605.30361#S1.SS0.SSS0.Px1.p1.1),[§2\.3](https://arxiv.org/html/2605.30361#S2.SS3.p1.2),[§6](https://arxiv.org/html/2605.30361#S6.SS0.SSS0.Px1.p1.1)\.
- G\. Orchard, A\. Jayawant, G\. K\. Cohen, and N\. Thakor \(2015\)Converting static image datasets to spiking neuromorphic datasets using saccades\.Frontiers in NeuroscienceVolume 9 \- 2015\.External Links:[Link](https://www.frontiersin.org/journals/neuroscience/articles/10.3389/fnins.2015.00437),[Document](https://dx.doi.org/10.3389/fnins.2015.00437),ISSN 1662\-453XCited by:[§2\.1](https://arxiv.org/html/2605.30361#S2.SS1.p1.1),[§4\.1](https://arxiv.org/html/2605.30361#S4.SS1.SSS0.Px1.p1.1),[§5](https://arxiv.org/html/2605.30361#S5.SS0.SSS0.Px3.p1.1),[§7](https://arxiv.org/html/2605.30361#S7.SS0.SSS0.Px1.p1.1)\.
- T\. Salimans, J\. Ho, X\. Chen, S\. Sidor, and I\. Sutskever \(2017\)Evolution strategies as a scalable alternative to reinforcement learning\.arXiv preprint arXiv:1703\.03864\.Cited by:[§1](https://arxiv.org/html/2605.30361#S1.SS0.SSS0.Px2.p1.3),[§2\.4](https://arxiv.org/html/2605.30361#S2.SS4.p1.1),[§6](https://arxiv.org/html/2605.30361#S6.SS0.SSS0.Px2.p1.1)\.
- S\. Sarkaret al\.\(2025\)Evolution strategies at the hyperscale\.arXiv preprint arXiv:2511\.16652\.Cited by:[Appendix B](https://arxiv.org/html/2605.30361#A2.p1.1),[§1](https://arxiv.org/html/2605.30361#S1.SS0.SSS0.Px3.p1.4),[§2\.5](https://arxiv.org/html/2605.30361#S2.SS5.p1.3),[§2\.5](https://arxiv.org/html/2605.30361#S2.SS5.p1.8),[§6](https://arxiv.org/html/2605.30361#S6.SS0.SSS0.Px2.p1.1),[§6](https://arxiv.org/html/2605.30361#S6.SS0.SSS0.Px4.p1.1)\.
- J\. Schemmel, D\. Brüderle, A\. Grübl,et al\.\(2010\)A wafer\-scale neuromorphic hardware system for large\-scale neural modeling\.InProceedings of the IEEE International Symposium on Circuits and Systems,pp\. 1947–1950\.Cited by:[§6](https://arxiv.org/html/2605.30361#S6.SS0.SSS0.Px3.p1.1)\.
- F\. Sehnke, C\. Osendorfer, T\. Rückstieß, A\. Graves, J\. Peters, and J\. Schmidhuber \(2010\)Parameter\-exploring policy gradients\.Neural Networks23\(4\),pp\. 551–559\.Note:The 18th International Conference on Artificial Neural Networks, ICANN 2008External Links:ISSN 0893\-6080,[Document](https://dx.doi.org/https%3A//doi.org/10.1016/j.neunet.2009.12.004),[Link](https://www.sciencedirect.com/science/article/pii/S0893608009003220)Cited by:[§6](https://arxiv.org/html/2605.30361#S6.SS0.SSS0.Px2.p1.1)\.
- F\. P\. Such, V\. Madhavan, E\. Conti,et al\.\(2017\)Deep neuroevolution: genetic algorithms are a competitive alternative for training deep neural networks for reinforcement learning\.Cited by:[§6](https://arxiv.org/html/2605.30361#S6.SS0.SSS0.Px2.p1.1)\.
- A\. Vaswani, N\. Shazeer, N\. Parmar, J\. Uszkoreit, L\. Jones, A\. N\. Gomez, L\. Kaiser, and I\. Polosukhin \(2017\)Attention is all you need\.CoRRabs/1706\.03762\.External Links:[Link](http://arxiv.org/abs/1706.03762),1706\.03762Cited by:[§1](https://arxiv.org/html/2605.30361#S1.p1.1)\.

## Appendix AVariance Analysis Across Seeds

Table 2:Per\-seed accuracy and average training time across three seeds\.
## Appendix BEGGROLL Gradient Derivation

FollowingSarkar and others\[[2025](https://arxiv.org/html/2605.30361#bib.bib1)\]§4\.2, the gradient estimate for weight matrix𝐖\\mathbf\{W\}under low\-rank perturbations is:

∇^𝐖​J\\displaystyle\\hat\{\\nabla\}\_\{\\mathbf\{W\}\}J=12​P​σ​r​∑i=1Pfi​𝐀i​𝐁i⊤\\displaystyle=\\frac\{1\}\{2P\\sigma\\sqrt\{r\}\}\\sum\_\{i=1\}^\{P\}f\_\{i\}\\mathbf\{A\}\_\{i\}\\mathbf\{B\}\_\{i\}^\{\\top\}=12​P​σ​r​\(diag⁡\(𝐟\)​𝐀\)⊤​𝐁,\\displaystyle=\\frac\{1\}\{2P\\sigma\\sqrt\{r\}\}\\bigl\(\\operatorname\{diag\}\(\\mathbf\{f\}\)\\,\\mathbf\{A\}\\bigr\)^\{\\top\}\\mathbf\{B\},\(6\)where𝐀∈ℝP×m×r\\mathbf\{A\}\\in\\mathbb\{R\}^\{P\\times m\\times r\}and𝐁∈ℝP×n×r\\mathbf\{B\}\\in\\mathbb\{R\}^\{P\\times n\\times r\}are stacked factor tensors, andfi=ri\+−ri−f\_\{i\}=r\_\{i\}^\{\+\}\-r\_\{i\}^\{\-\}is the antithetic rank difference\. Equation \([6](https://arxiv.org/html/2605.30361#A2.E6)\) requires only two batched matrix multiplications, never materialising theP×m×nP\\times m\\times nperturbation tensor\.