NeuroPB: Scaling Neural Decoding with Pretrained Behavioral Representations

# NeuroPB: Scaling Neural Decoding with Pretrained Behavioral Representations
Source: [https://arxiv.org/html/2608.04389](https://arxiv.org/html/2608.04389)
###### Abstract

Decoding continuous motor trajectories from neural activity is essential for developing practical brain–computer interfaces \(BCIs\)\. However, current neural decoders are constrained by the limited scale and heterogeneity of neural recordings\. In contrast, behavioral data can be collected more readily and at substantially larger scale from humans, animals, simulations, and robotic systems\. Here, we introduce NeuroPB, a framework that scales neural decoding by transferring knowledge from pretrained behavioral representations\. NeuroPB first pretrains a motor encoder on large\-scale motor behavior data and then aligns neural activity with the resulting behavioral representation space using a limited set of paired neural–behavioral recordings\. A neural encoder and lightweight motor decoder are subsequently optimized to reconstruct continuous movement from the aligned neural representations\. Across multiple macaque motor datasets, behavioral pretraining improves trajectory decoding, including an 11%R2R^\{2\}increase on center\-out and 8% on random\-target compared with training the motor encoder from scratch\. Notably, pretraining on robotic trajectories achieves performance comparable to pretraining on macaque trajectories, demonstrating that transferable kinematic structure is shared across biological and artificial models\. Moreover, decoding performance improves as the scale and diversity of robotic pretraining data increase, when the amount of neural data is fixed\. Pretraining also enhances generalization across recording sessions, subjects, and motor tasks, with only 10% calibration needed to match training from scratch\. Overall, these results establish behavioral pretraining as a scalable source for neural decoding and provide a promising route toward high\-performance and calibration\-efficient BCIs under limited neural data\.

## Introduction

Brain\-computer interfaces \(BCIs\) provide a direct pathway for translating neural activity into executable commands, enabling users to interact with prosthetic limbs, robotic systems, computers, and other external devices\(Willseyet al\.[2025a](https://arxiv.org/html/2608.04389#bib.bib1)\)\. Among the capabilities required by practical BCIs, decoding continuous motor behavior is particularly important because natural interaction depends not only on identifying a discrete intention, but also on reconstructing how a movement evolves over time\(Leeet al\.[2025](https://arxiv.org/html/2608.04389#bib.bib2); Willseyet al\.[2025b](https://arxiv.org/html/2608.04389#bib.bib3)\)\. Despite substantial progress, accurately decoding continuous behavioral trajectories from neural activity remains challenging\. Neural activity is heterogeneous across recording sessions, individuals, and motor tasks, while collecting sufficiently large and well\-annotated neural datasets is expensive and technically demanding\.

![Refer to caption](https://arxiv.org/html/2608.04389v1/intro.png)Figure 1:Conceptual overview of NeuroPB\. NeuroPB aligns neural activity with a behavioral representation pretrained on scalable macaque and robotic trajectories, enabling continuous trajectory decoding with generalization across recording sessions, subjects, and motor tasks\.Existing neural decoding methods learn direct mappings from neural activity to movement trajectories using end\-to\-end models\(Pandarinathet al\.[2018](https://arxiv.org/html/2608.04389#bib.bib5); Glaseret al\.[2020](https://arxiv.org/html/2608.04389#bib.bib7); Dyeret al\.[2017](https://arxiv.org/html/2608.04389#bib.bib16)\)\. Although these methods achieve good performance within the training distribution, their performance often degrades when applied to unseen recording sessions, subjects, or motor tasks\. To improve generalization, recent studies have explored large\-scale neural pretraining\(Zhanget al\.[2025](https://arxiv.org/html/2608.04389#bib.bib9); Azabouet al\.[2025](https://arxiv.org/html/2608.04389#bib.bib17)\)and self\-supervised learning\(Schneideret al\.[2023](https://arxiv.org/html/2608.04389#bib.bib15)\)\. These approaches aim to learn general neural representations before adapting the model to a downstream decoding task\. However, their scalability remains constrained by the limited availability and pronounced heterogeneity of neural recordings across subjects, sessions, brain regions, and acquisition devices\. Consequently, increasing the scale of neural pretraining alone may not provide a readily scalable solution\. This limitation motivates us to hypothesize: rather than relying exclusively on scarce and heterogeneous neural recordings, can prior knowledge about behavior itself be used to guide the learning of more transferable neural representations?

In this work, we approach neural decoding from the behavioral side\. Recent advances in embodied AI have promoted large, diverse, and standardized manipulation trajectories\(Liuet al\.[2023](https://arxiv.org/html/2608.04389#bib.bib18); Walkeet al\.[2023](https://arxiv.org/html/2608.04389#bib.bib20); O’Neillet al\.[2024](https://arxiv.org/html/2608.04389#bib.bib19)\)\. Compared with neural recordings, robotic behavior can be collected at substantially larger scales\(Yeet al\.[2023](https://arxiv.org/html/2608.04389#bib.bib8); O’Neillet al\.[2024](https://arxiv.org/html/2608.04389#bib.bib19)\)\. Biological movements, in contrast, naturally contain trial\-to\-trial variability\(Fishbachet al\.[2007](https://arxiv.org/html/2608.04389#bib.bib21)\)\. Despite their differences in embodiment, biological and artificial systems share motor control objectives and can exhibit reusable behavioral structure\(Merelet al\.[2019](https://arxiv.org/html/2608.04389#bib.bib22)\)\. We therefore hypothesize that representations learned from large\-scale behavioral trajectories can provide transferable motor priors for neural decoding\.

To investigate this hypothesis, we proposeNeuroPB, aNeuron decoding framework withPretrainedBehavioral representation that bridges neural activity with representations extracted by a pretrained behavior model\. NeuroPB uses a pretrained motor encoder to define a structured behavioral representation space\. During training, a neural encoder maps neural activity into this space, allowing pretrained behavioral representations to serve as anchors that guide the neural encoder toward capturing motor\-relevant information\. The resulting neural representation is then passed to a motor decoder to reconstruct the corresponding continuous movement\. At inference time, the pretrained motor encoder is no longer required: the neural encoder directly extracts a motor\-informed representation from the neural signal, and the motor decoder generates the predicted trajectory\. In this way, NeuroPB transfers behavioral knowledge to the neural decoding model without requiring behavioral inputs during deployment\.

We evaluate NeuroPB on multiple macaque neural datasets covering different recording sessions, subjects, and motor tasks\. Our experiments show that pretrained behavioral representations consistently facilitate neural trajectory decoding and that scaling trajectory pretraining leads to stronger overall performance\. Notably, trajectory encoders pretrained on large\-scale robotic data provide effective priors for decoding biological movements, supporting the existence of transferable behavioral structure across embodiments\. NeuroPB also exhibits improved generalization to unseen sessions, subjects, and tasks, while requiring fewer paired neural and trajectory samples during calibration\. The main contributions are summarized as follows:

- •We propose theNeuroPB, a neural decoding framework that uses pretrained behavioral representations as anchors to learn motor\-informed neural representations\.
- •We demonstrate that behavioral pretraining consistently outperforms training the framework from scratch, establishing the importance of behavioral pretraining for neural trajectory decoding\.
- •We investigate the scaling of the pretrained motor encoder and show that larger\-scale and diverse behavioral datasets yield better decoding performance\.
- •We evaluate NeuroPB on macaque datasets, demonstrating improved calibration efficiency and robust generalization across recording sessions, subjects, and motor tasks\.

## Related Work

### Motor Neural Decoding

Motor neural decoding seeks to infer behavioral variables from neural population activity\. Early approaches map neural activity to movement trajectories, such as Wiener filters\(Carmenaet al\.[2003](https://arxiv.org/html/2608.04389#bib.bib4)\), Kalman filters\(Glaseret al\.[2020](https://arxiv.org/html/2608.04389#bib.bib7)\), statistical models\(Pandarinathet al\.[2018](https://arxiv.org/html/2608.04389#bib.bib5)\), recurrent networks\(Ye and Pandarinath[2021](https://arxiv.org/html/2608.04389#bib.bib23)\), sequential autoencoders\(Keshtkaranet al\.[2022](https://arxiv.org/html/2608.04389#bib.bib14)\), and other deep architectures\(Tianet al\.[2026](https://arxiv.org/html/2608.04389#bib.bib24)\)\. Although these methods can achieve high decoding accuracy within the recording sessions, they often generalize poorly to unseen sessions, subjects, or motor tasks\. To improve generalization, recent studies have pretrained neural encoders on recordings collected from multiple sessions, subjects, and tasks\. NEDS learns generalized latent population dynamics across datasets\(Zhanget al\.[2025](https://arxiv.org/html/2608.04389#bib.bib9)\), CEBRA constructs behaviorally structured neural representations using contrastive learning\(Schneideret al\.[2023](https://arxiv.org/html/2608.04389#bib.bib15)\), and large\-scale frameworks such as POYO\+ and NDT3 integrate heterogeneous neural recordings through neural pretraining\(Azabouet al\.[2025](https://arxiv.org/html/2608.04389#bib.bib17); Yeet al\.[2026](https://arxiv.org/html/2608.04389#bib.bib25)\)\. These approaches demonstrate that increasing the scale of neural pretraining can improve adaptation to new recording sessions\. Nevertheless, their scalability remains dependent on the availability of large, high\-quality neural datasets, which are costly to collect and heterogeneous across subjects, sessions, brain regions, and acquisition devices\. Our work therefore explores a complementary direction: learning transferable behavioral representations from large\-scale behavioral trajectories and using them to guide neural representation learning with limited paired neural–behavioral data\.

### Contrastive Learning in Neural Decoding

Contrastive learning provides an effective mechanism for neural decoding by bringing matched neural–target representations closer while separating mismatched samples, thereby emphasizing information shared across modalities\. Inspired by the CLIP framework for large\-scale vision–language alignment\(Radfordet al\.[2021](https://arxiv.org/html/2608.04389#bib.bib13)\), this strategy has been increasingly adopted to connect neural signals with behaviorally meaningful representation spaces\. CEBRA uses contrastive sampling conditioned on behavior or time to learn consistent neural embeddings across sessions, animals, and recording modalities\(Schneideret al\.[2023](https://arxiv.org/html/2608.04389#bib.bib15)\)\. Contrastive alignment is employed between neural and speech representations for speech decoding\(Défossezet al\.[2023](https://arxiv.org/html/2608.04389#bib.bib26); Zhanget al\.[2026b](https://arxiv.org/html/2608.04389#bib.bib27)\)\. Similar ideas have been explored in visual decoding, where aligned EEG/fMRI responses with image embeddings are used for object recognition\(Songet al\.[2023](https://arxiv.org/html/2608.04389#bib.bib28); Zhanget al\.[2026a](https://arxiv.org/html/2608.04389#bib.bib30); Xiaet al\.[2024](https://arxiv.org/html/2608.04389#bib.bib29)\)\. Unlike prior work that relies primarily on paired neural–target data, our framework also leverages independently collected, large\-scale robotic trajectories to pretrain a scalable behavioral representation, which is then aligned with neural activity for transferable motor decoding\. This design enables neural decoding to benefit from scalable behavioral data beyond biological recordings\.

![Refer to caption](https://arxiv.org/html/2608.04389v1/framework.png)Figure 2:\(A\) A motor encoder is pretrained on masked trajectory reconstruction using macaque or robotic trajectories and then frozen to produce behavioral representations\. \(B\) Contrastive learning aligns paired neural and behavioral representations while separating unpaired samples\. \(C\) During inference, the neural encoder maps spike activity into the neural representation, from which a motor decoder reconstructs movement trajectories\.

## Method

Our framework learns transferable neural representations by using a pretrained behavioral representation space as behavioral supervision\. As illustrated in Fig\.[2](https://arxiv.org/html/2608.04389#Sx2.F2), the method consists of three stages\. First, the motor encoder is pretrained through masked trajectory reconstruction using macaque trajectories, scale\-matched robotic trajectories, or large\-scale robotic trajectories, while a randomly initialized encoder serves as the scratch condition\. The resulting motor encoder is then kept frozen\. Second, the neural encoder is optimized with a contrastive objective that aligns matched neural and behavioral representations while separating mismatched pairs\. Finally, a lightweight motor decoder reconstructs the continuous movement trajectory from the aligned neural representation\. Detailed network architectures are provided in the Appendix\.

### Motor Encoder

We construct the motor encoder based on the Transformer structure, which learns structured movement representations through masked trajectory reconstruction\(Wenet al\.[2023](https://arxiv.org/html/2608.04389#bib.bib10)\)\. Given a trajectory segment

𝐏=\[𝐩1,𝐩2,…,𝐩T\],𝐩t∈ℝdp,\\mathbf\{P\}=\[\\mathbf\{p\}\_\{1\},\\mathbf\{p\}\_\{2\},\\ldots,\\mathbf\{p\}\_\{T\}\],\\qquad\\mathbf\{p\}\_\{t\}\\in\\mathbb\{R\}^\{d\_\{p\}\},\(1\)where𝐩t\\mathbf\{p\}\_\{t\}denotes the movement position at time stepttanddp=2d\_\{p\}=2for the\(x,y\)\(x,y\), a subset of time stepsℳ⊆\{1,…,T\}\\mathcal\{M\}\\subseteq\\\{1,\\ldots,T\\\}is randomly masked\. The input token at each time step is defined as

𝐞~t=\{Epos​\(𝐩t\)\+Etime​\(t\),t∉ℳ,𝐞mask\+Etime​\(t\),t∈ℳ,\\widetilde\{\\mathbf\{e\}\}\_\{t\}=\\begin\{cases\}E\_\{\\mathrm\{pos\}\}\(\\mathbf\{p\}\_\{t\}\)\+E\_\{\\mathrm\{time\}\}\(t\),&t\\notin\\mathcal\{M\},\\\\\[4\.0pt\] \\mathbf\{e\}\_\{\\mathrm\{mask\}\}\+E\_\{\\mathrm\{time\}\}\(t\),&t\\in\\mathcal\{M\},\\end\{cases\}\(2\)whereEpos​\(⋅\)E\_\{\\mathrm\{pos\}\}\(\\cdot\)projects trajectory coordinates into the model dimension,Etime​\(⋅\)E\_\{\\mathrm\{time\}\}\(\\cdot\)encodes temporal order, and𝐞mask\\mathbf\{e\}\_\{\\mathrm\{mask\}\}is a learnable mask embedding\. The resulting token sequence is processed by the motor encoderfθmf\_\{\\theta\_\{m\}\}:

𝐇m=fθm​\(𝐞~1,𝐞~2,…,𝐞~T\),\\mathbf\{H\}^\{m\}=f\_\{\\theta\_\{m\}\}\\left\(\\widetilde\{\\mathbf\{e\}\}\_\{1\},\\widetilde\{\\mathbf\{e\}\}\_\{2\},\\ldots,\\widetilde\{\\mathbf\{e\}\}\_\{T\}\\right\),\(3\)where

𝐇m=\[𝐡1m,𝐡2m,…,𝐡Tm\]∈ℝT×D\\mathbf\{H\}^\{m\}=\[\\mathbf\{h\}^\{m\}\_\{1\},\\mathbf\{h\}^\{m\}\_\{2\},\\ldots,\\mathbf\{h\}^\{m\}\_\{T\}\]\\in\\mathbb\{R\}^\{T\\times D\}\(4\)denotes the contextualized trajectory tokens\. A lightweight prediction headgϕg\_\{\\phi\}reconstructs the masked trajectory coordinates:

𝐩^t=gϕ​\(𝐡tm\),t∈ℳ\.\\widehat\{\\mathbf\{p\}\}\_\{t\}=g\_\{\\phi\}\\left\(\\mathbf\{h\}^\{m\}\_\{t\}\\right\),\\qquad t\\in\\mathcal\{M\}\.\(5\)The motor encoder is pretrained using the masked trajectory reconstruction objective

ℒtraj=1\|ℳ\|​∑t∈ℳ‖𝐩^t−𝐩t‖22\.\\mathcal\{L\}\_\{\\mathrm\{traj\}\}=\\frac\{1\}\{\|\\mathcal\{M\}\|\}\\sum\_\{t\\in\\mathcal\{M\}\}\\left\\\|\\widehat\{\\mathbf\{p\}\}\_\{t\}\-\\mathbf\{p\}\_\{t\}\\right\\\|\_\{2\}^\{2\}\.\(6\)
The behavioral representation of the trajectory is obtained by aggregating the contextualized trajectory tokens:

𝐳m=Pool⁡\(𝐇m\),𝐳¯m=𝐳m‖𝐳m‖2,\\mathbf\{z\}^\{m\}=\\operatorname\{Pool\}\\left\(\\mathbf\{H\}^\{m\}\\right\),\\qquad\\overline\{\\mathbf\{z\}\}^\{m\}=\\frac\{\\mathbf\{z\}^\{m\}\}\{\\left\\\|\\mathbf\{z\}^\{m\}\\right\\\|\_\{2\}\},\(7\)wherePool⁡\(⋅\)\\operatorname\{Pool\}\(\\cdot\)denotes token aggregation and𝐳¯m\\overline\{\\mathbf\{z\}\}^\{m\}is the normalized behavioral representation used as the target for neural–behavioral alignment\.

To disentangle the effects of behavioral source and pretraining scale, we evaluated four motor encoder initialization strategies: training from scratch \(Scratch\), pretraining on macaque trajectories \(Macaque\-PT\), pretraining on a scale\-matched robotic trajectory subset \(Robot\-Matched\-PT\), and pretraining on the large\-scale LIBERO\-100 robotic trajectory dataset \(Robot\-Large\-PT\)\. In Scratch, the motor encoder was randomly initialized and jointly optimized during neural alignment\. Macaque\-PT, Robot\-Matched\-PT, and Robot\-Large\-PT mean pretrained with macaque trajectories, robotic trajectories matched in size with macaque trajectories, and larger\-scale robotic trajectories, respectively\. Then, keep frozen during neural alignment\. Comparing Macaque\-PT with Robot\-Matched\-PT isolates the effect of behavioral source under a matched data scale, whereas comparing Robot\-Matched\-PT with Robot\-Large\-PT evaluates the effect of increasing robotic pretraining scale\.

### Neural Encoder

We construct the neural encoder following the Perceiver\-based POYO architecture\(Azabouet al\.[2023](https://arxiv.org/html/2608.04389#bib.bib6); Jaegleet al\.[2021](https://arxiv.org/html/2608.04389#bib.bib12)\), which represents neural population activity as a sequence of timestamped spike events and compresses the variable\-length sequence into a fixed number of latent tokens\. For a neural window of durationTT, let

𝒮=\{\(ui,ti\)\}i=1M\\mathcal\{S\}=\\left\\\{\(u\_\{i\},t\_\{i\}\)\\right\\\}\_\{i=1\}^\{M\}\(8\)denote theMMobserved spike events, whereuiu\_\{i\}is the identity of the neural unit that generated theii\-th spike andti∈\[0,T\]t\_\{i\}\\in\[0,T\]is its timestamp\. Each spike event is represented using a learnable unit embedding:

𝐱i=Eunit​\(ui\),𝐗=\[𝐱1,𝐱2,…,𝐱M\]∈ℝM×D\.\\mathbf\{x\}\_\{i\}=E\_\{\\mathrm\{unit\}\}\(u\_\{i\}\),\\qquad\\mathbf\{X\}=\[\\mathbf\{x\}\_\{1\},\\mathbf\{x\}\_\{2\},\\ldots,\\mathbf\{x\}\_\{M\}\]\\in\\mathbb\{R\}^\{M\\times D\}\.\(9\)Temporal information is incorporated through rotary positional embeddings\(Suet al\.[2024](https://arxiv.org/html/2608.04389#bib.bib11)\), allowing the attention operation to depend on the relative timing between spike events and latent tokens\. For adapt to different numbers of spikes varies across windows, sessions, and neural populations, we introduceNNlatent tokens

𝐙0=\[𝐳0,1,𝐳0,2,…,𝐳0,N\]∈ℝN×D,N≪M,\\mathbf\{Z\}\_\{0\}=\[\\mathbf\{z\}\_\{0,1\},\\mathbf\{z\}\_\{0,2\},\\ldots,\\mathbf\{z\}\_\{0,N\}\]\\in\\mathbb\{R\}^\{N\\times D\},\\qquad N\\ll M,\(10\)whose timestamps are uniformly distributed over the neural context window\. The latent tokens first aggregate information from the spike\-event sequence through cross\-attention:

𝐙1=CrossAttn⁡\(𝐙0,𝐗\)\.\\mathbf\{Z\}\_\{1\}=\\operatorname\{CrossAttn\}\\left\(\\mathbf\{Z\}\_\{0\},\\mathbf\{X\}\\right\)\.\(11\)More specifically, the cross\-attention operation is given by

softmax⁡\(RoPE\(𝐙0𝐖q,𝝉z\)RoPE\(𝐗𝐖k,𝝉x\)⊤dk\)​𝐗𝐖v,\\displaystyle\\operatorname\{softmax\}\\left\(\\frac\{\\operatorname\{RoPE\}\\left\(\\mathbf\{Z\}\_\{0\}\\mathbf\{W\}\_\{q\},\\boldsymbol\{\\tau\}\_\{z\}\\right\)\\operatorname\{RoPE\}\\left\(\\mathbf\{X\}\\mathbf\{W\}\_\{k\},\\boldsymbol\{\\tau\}\_\{x\}\\right\)^\{\\top\}\}\{\\sqrt\{d\_\{k\}\}\}\\right\)\\mathbf\{X\}\\mathbf\{W\}\_\{v\},\(12\)where𝝉x\\boldsymbol\{\\tau\}\_\{x\}and𝝉z\\boldsymbol\{\\tau\}\_\{z\}denote the timestamps assigned to the spike tokens and latent tokens, respectively, and𝐖q\\mathbf\{W\}\_\{q\},𝐖k\\mathbf\{W\}\_\{k\}, and𝐖v\\mathbf\{W\}\_\{v\}are learnable projection matrices\. The compressed latent sequence is subsequently processed byL−1L\-1Transformer self\-attention blocks, The output of the final self\-attention block as the neural representation:

𝐙n=𝐙L=\[𝐳L,1,𝐳L,2,…,𝐳L,N\]∈ℝN×D\.\\mathbf\{Z\}^\{n\}=\\mathbf\{Z\}\_\{L\}=\[\\mathbf\{z\}\_\{L,1\},\\mathbf\{z\}\_\{L,2\},\\ldots,\\mathbf\{z\}\_\{L,N\}\]\\in\\mathbb\{R\}^\{N\\times D\}\.\(13\)Thus,𝐙L\\mathbf\{Z\}\_\{L\}summarizes the temporal and population\-level structure of the neural activity while maintaining a fixed dimensionality across recording sessions with different numbers and identities of neural units\. For modality alignment, the latent tokens are aggregated and transformed by a learnable neural projection headgng\_\{n\}:

𝐳n=Pool⁡\(𝐙n\),𝐳~n=gn​\(𝐳n\),\\mathbf\{z\}^\{n\}=\\operatorname\{Pool\}\\left\(\\mathbf\{Z\}^\{n\}\\right\),\\qquad\\widetilde\{\\mathbf\{z\}\}^\{n\}=g\_\{n\}\\left\(\\mathbf\{z\}^\{n\}\\right\),\(14\)followed byℓ2\\ell\_\{2\}normalization:

𝐳¯n=𝐳~n‖𝐳~n‖2\.\\overline\{\\mathbf\{z\}\}^\{n\}=\\frac\{\\widetilde\{\\mathbf\{z\}\}^\{n\}\}\{\\left\\\|\\widetilde\{\\mathbf\{z\}\}^\{n\}\\right\\\|\_\{2\}\}\.\(15\)The resulting representation𝐳¯n\\overline\{\\mathbf\{z\}\}^\{n\}is aligned with the corresponding behavioral representation produced by the frozen motor encoder\.

### Modality Alignment

After obtaining neural and behavioral representations, we align the two modalities using a CLIP\-style symmetric contrastive objective\(Radfordet al\.[2021](https://arxiv.org/html/2608.04389#bib.bib13)\)\. Given a mini\-batch ofBBtemporally paired neural and behavioral samples,

\{\(𝒮i,𝐏i\)\}i=1B,\\left\\\{\\left\(\\mathcal\{S\}\_\{i\},\\mathbf\{P\}\_\{i\}\\right\)\\right\\\}\_\{i=1\}^\{B\},\(16\)the neural encoder and frozen motor encoder get normalized behavioral representation and neural representation,

\{\(𝐳¯im,𝐳¯in\)\}i=1B,\\left\\\{\\left\(\\overline\{\\mathbf\{z\}\}^\{m\}\_\{i\},\\overline\{\\mathbf\{z\}\}^\{n\}\_\{i\}\\right\)\\right\\\}\_\{i=1\}^\{B\},\(17\)We compute the cross\-modal similarity between theii\-th neural representation and thejj\-th behavioral representation using scaled cosine similarity:

si​j=\(𝐳¯in\)⊤​𝐳¯jmτ,s\_\{ij\}=\\frac\{\\left\(\\overline\{\\mathbf\{z\}\}^\{n\}\_\{i\}\\right\)^\{\\top\}\\overline\{\\mathbf\{z\}\}^\{m\}\_\{j\}\}\{\\tau\},\(18\)whereτ\\tauis a temperature parameter controlling the concentration of the similarity distribution\. For each neural representation, its temporally paired behavioral representation is treated as the positive sample, whereas the remaining behavioral representations in the mini\-batch are treated as negative samples\. The neural\-to\-motor contrastive loss is defined as

ℒn→m=−1B​∑i=1Blog⁡exp⁡\(si​i\)∑j=1Bexp⁡\(si​j\)\.\\mathcal\{L\}\_\{n\\rightarrow m\}=\-\\frac\{1\}\{B\}\\sum\_\{i=1\}^\{B\}\\log\\frac\{\\exp\(s\_\{ii\}\)\}\{\\sum\_\{j=1\}^\{B\}\\exp\(s\_\{ij\}\)\}\.\(19\)Similarly, by treating each behavioral representation as a query and the neural representations as candidates, the motor\-to\-neural contrastive loss is

ℒm→n=−1B​∑i=1Blog⁡exp⁡\(si​i\)∑j=1Bexp⁡\(sj​i\)\.\\mathcal\{L\}\_\{m\\rightarrow n\}=\-\\frac\{1\}\{B\}\\sum\_\{i=1\}^\{B\}\\log\\frac\{\\exp\(s\_\{ii\}\)\}\{\\sum\_\{j=1\}^\{B\}\\exp\(s\_\{ji\}\)\}\.\(20\)The final modality\-alignment objective is the symmetric average of the two directions:

ℒalign=12​\(ℒn→m\+ℒm→n\)\.\\mathcal\{L\}\_\{\\mathrm\{align\}\}=\\frac\{1\}\{2\}\\left\(\\mathcal\{L\}\_\{n\\rightarrow m\}\+\\mathcal\{L\}\_\{m\\rightarrow n\}\\right\)\.\(21\)
During modality alignment, the pretrained motor encoder remains frozen, whereas the neural encoder and the neural projection headgng\_\{n\}are optimized\. The objective increases the similarity between temporally paired neural and behavioral representations while reducing the similarity between mismatched pairs\. Consequently, the neural encoder is encouraged to preserve behaviorally relevant information shared with the pretrained behavioral representation space while suppressing recording\-specific variability across units and sessions\.

### Motor Decoder

The motor decoder reconstructs the continuous movement trajectory from the aligned neural representation\. Given the pooled neural representation𝐳¯in\\overline\{\\mathbf\{z\}\}^\{n\}\_\{i\}, we employ a lightweight multilayer perceptrondψd\_\{\\psi\}to predict the trajectory:

𝐏^i=dψ​\(𝐳¯in\)\\widehat\{\\mathbf\{P\}\}\_\{i\}=d\_\{\\psi\}\\left\(\\overline\{\\mathbf\{z\}\}^\{n\}\_\{i\}\\right\)\(22\)It was trained jointly with the neural encoder; the total objective was

ℒ=ℒalign\+ℒmotor\\mathcal\{L\}=\\mathcal\{L\}\_\{\\mathrm\{align\}\}\+\\mathcal\{L\}\_\{\\mathrm\{motor\}\}\(23\)whereℒmotor\\mathcal\{L\}\_\{\\mathrm\{motor\}\}was the mean\-squared error between the predicted and ground truth positionsℒmotor=MSE​\(𝐏^,𝐏\)\\mathcal\{L\}\_\{\\mathrm\{motor\}\}=\\mathrm\{MSE\}\(\\widehat\{\\mathbf\{P\}\},\\mathbf\{P\}\)\.

Table 1:Comparison with baselines on single\-session\.

## Experiments and Results

### Datasets

We evaluated our method on two datasets, a large\-scale collection of electrophysiological and behavioral recordings from four rhesus macaques performing two\-dimensional reaching tasks\(Perichet al\.[2018](https://arxiv.org/html/2608.04389#bib.bib31)\)\. Neural activity was recorded using chronically implanted multielectrode arrays in the primary motor cortex \(M1\) and dorsal premotor cortex \(PMd\), with some sessions containing simultaneous recordings from both regions and up to 192 electrodes\. It contains two behavioral paradigms: center\-out reaching \(CO\), in which the macaque moved a cursor from a central location toward one of eight peripheral targets arranged radially, and random\-target reaching \(RT\), in which the macaque continuously acquired targets presented at varying spatial locations\. The dataset provides spike events together with synchronized cursor position and task\-related metadata\. The recordings span different collection sessions, subjects, and task structures, making them suitable for evaluating both within\-domain decoding and generalization across sessions, subjects, and motor tasks\.

### Implement Details

Our method was implemented in PyTorch on an RTX 4090 GPU\. We used the SparseLamb optimizer with a weight decay of10−410^\{\-4\}\. The maximum learning rate was3\.125×10−4×128=0\.043\.125\\times 10^\{\-4\}\\times 128=0\.04, where 128 is the batch size\. The learning rate followed a one\-cycle schedule with cosine annealing\. After training, the checkpoint with the highest average validation metric was evaluated once on the held\-out test set\. Additional experimental details are provided in the Appendix\. In all experiments, the training, validation, and test sets were defined by the predefined split\-specific sampling intervals in the dataset configuration, with around 10% holdout for validation and 20% holdout for testing\. The 1s window is used to sample from the training intervals\. Validation and test windows used 50% overlap within the same split\. For cross experiments, the split was performed at the corresponding session level, such that all data from a given session were assigned to only one partition\.

### Comparison with Baselines

We compared NeuroPB with several classical and recent models on the center\-out \(CO\) and random\-target \(RT\) tasks\. Table[1](https://arxiv.org/html/2608.04389#Sx3.T1)shows the performance from Wiener Filter\(Carmenaet al\.[2003](https://arxiv.org/html/2608.04389#bib.bib4)\), Smoothing, DenseNN, RNN\(Glaseret al\.[2020](https://arxiv.org/html/2608.04389#bib.bib7)\), and LFADS\(Pandarinathet al\.[2018](https://arxiv.org/html/2608.04389#bib.bib5)\), which were reproduced using the same protocol\. NEDS\(Zhanget al\.[2025](https://arxiv.org/html/2608.04389#bib.bib9)\), NDT2\(Yeet al\.[2023](https://arxiv.org/html/2608.04389#bib.bib8)\), and POYO\(Azabouet al\.[2023](https://arxiv.org/html/2608.04389#bib.bib6)\)results were obtained from other publications\(Ryooet al\.[2026](https://arxiv.org/html/2608.04389#bib.bib32); Maoet al\.[2026](https://arxiv.org/html/2608.04389#bib.bib33)\)\. NeuroPB achieved the state\-of\-the\-art decoding performance on both datasets, reaching 94\.78%±1\.20%R2R^\{2\}on CO and 84\.75%±1\.71%R2R^\{2\}on RT\. The results indicate that NeuroPB remains effective for tasks with different movement structures\. While several baselines performed reasonably well on the more structured CO task, their performance generally decreased on RT, which contains more variable target locations and trajectories\. NeuroPB maintained strong performance on both tasks and showed low variability across evaluation sessions\. These findings demonstrate that aligning neural activity with a pretrained behavioral representation provides a more accurate and robust basis for continuous trajectory decoding than direct end\-to\-end decoding or neural\-only representation learning\.

![Refer to caption](https://arxiv.org/html/2608.04389v1/result1.png)Figure 3:Robot\-Large\-PT significantly outperforms Scratch on CO and RT and produces trajectories that more closely match the ground truth\. \*\*\* denotesp<0\.001p<0\.001\.
### Effect of Pretrained Motor Encoder

In the overall evaluation, NeuroPB uses the Robot\-Large\-PT motor encoder\. To isolate the contribution of behavioral pretraining, we compared this initialization with Scratch, which means the motor encoder was initialized randomly and jointly optimized with neural alignment\. As shown in the left panel of Fig\.[3](https://arxiv.org/html/2608.04389#Sx4.F3), behavioral pretraining substantially improved decoding performance on both datasets\. Statistical significance was assessed using a two\-sided paired t\-test\. On CO, Robot\-Large\-PT achieved anR2R^\{2\}of 94\.78% compared with 84\.19% for scratch\. On RT, Robot\-Large\-PT achieved 84\.75%, outperforming scratch at 76\.29%\. In addition to improving the average decoding accuracy, pretraining markedly reduced performance variability across evaluations, suggesting that the pretrained behavioral representation provides a more stable target space for neural–behavioral alignment\.

The representative trajectory examples further illustrate this improvement in the right panel of Fig\.[3](https://arxiv.org/html/2608.04389#Sx4.F3)\. Predictions produced with the scratch motor encoder exhibit larger deviations from the ground\-truth movement paths, whereas those produced with Robot\-Large\-PT more closely follow the overall direction and geometry of the movements toward the target\. Together, these results demonstrate that the performance of NeuroPB does not arise solely from its alignment architecture: behavioral pretraining is a critical component for learning accurate and robust neural trajectory decoders\.

![Refer to caption](https://arxiv.org/html/2608.04389v1/scaled_fig.png)Figure 4:The left panel schematically illustrates the relative pretraining\-data scales\. The right panel shows comparable performance for Macaque\-PT and Robot\-Matched\-PT, while Robot\-Large\-PT achieves the best decoding performance\.
### Scaled Pretrained Motor Encoder

We next examined whether the improvements from behavioral pretraining were determined primarily by the source of the trajectories or by the scale of the pretraining data\. We compared three pretrained motor\-encoder initializations: Macaque\-PT, Robot\-Matched\-PT, and Robot\-Large\-PT\. Macaque\-PT was pretrained on macaque trajectories, whereas Robot\-Matched\-PT was pretrained on trajectories from LIBERO\-Spatial\. The macaque and LIBERO\-Spatial pretraining sets were matched in size, allowing the influence of behavioral source to be evaluated while approximately controlling for data scale\. Robot\-Large\-PT was instead pretrained on the substantially larger LIBERO\-100 dataset, enabling us to evaluate the effect of scaling robotic trajectory pretraining\. Notably, the left panel of Fig\.[4](https://arxiv.org/html/2608.04389#Sx4.F4)provides only a conceptual illustration of the relative pretraining conditions and dataset scales\. It is not derived from the measured decoding scores\.

As shown quantitatively in the right panel of Fig\.[4](https://arxiv.org/html/2608.04389#Sx4.F4), Macaque\-PT and Robot\-Matched\-PT achieved comparable decoding performance despite the considerable difference between biological reaching movements and robotic manipulation trajectories\. This result suggests that the motor encoder learns transferable kinematic structures that are not restricted to the system from which the trajectories were collected\. More importantly, Robot\-Large\-PT provided the stronger overall performance, outperforming both scale\-matched conditions\. Thus, although macaque and robotic trajectories are both effective sources of behavioral supervision, increasing the scale and diversity of robotic pretraining data produces a stronger and more transferable behavioral representation\. These findings support the premise of NeuroPB: readily available behavioral data from artificial systems can complement scarce neural recordings, and its benefit can be further increased by scaling and enriching behavioral pretraining\.

![Refer to caption](https://arxiv.org/html/2608.04389v1/_calibration.png)Figure 5:\(A\) Calibration\-efficiency validation under cross\-session, cross\-subject, cross\-task including CO\-RT and RT\-CO\. \(B\) Estimated calibration fraction required by each pretrained model to match the performance of Scratch using 100% of the data\.
### Calibration\-Efficiency Validation

We next evaluated whether behavioral pretraining reduces the amount of target\-domain neural data required for adaptation across sessions, subjects, and motor tasks; the result is shown in Fig\.[5](https://arxiv.org/html/2608.04389#Sx4.F5)A\. Under each transfer setting, only the newly introduced unit and session embeddings were optimized using the fraction of target training data\. In contrast, the scratch baseline initialized the entire model randomly and optimized all components using each calibration subset\. More detailed calibration settings are provided in the Appendix\. Across all four settings, the Robot\-Large\-PT achieved the strongest performance in the low\-data regime\. Its advantage was particularly evident with only 1%–10% of the target data, where it consistently exceeded the Robot\-Matched\-PT, Macaque\-PT, and scratch models\. The performance gap gradually narrowed as more calibration data became available, but Robot\-Large\-PT remained competitive at full calibration\.

The calibration\-efficiency summary in Fig\.[5](https://arxiv.org/html/2608.04389#Sx4.F5)B further quantifies this advantage by estimating the fraction of target data required to match the performance of the scratch model trained on 100% of the target data\. Robot\-Large\-PT required less calibration data for generalization\. In comparison, Macaque\-PT and Robot\-Matched\-PT share a similar fraction of calibration data, and exhibited similar generalization and both substantially improved calibration efficiency relative to training from scratch\. Increasing the scale and diversity of behavioral pretraining data produces more transferable behavioral representations, even when the pretraining trajectories originate from robotic rather than biological systems\.

![Refer to caption](https://arxiv.org/html/2608.04389v1/visualization.png)Figure 6:\(A\) Pairwise similarity between neural and behavioral representations\. \(B\) PCA visualization of neural representations before and after alignment, colored by movement direction and marked by recording session\. Alignment improves directional organization while reducing session\-dependent variation\.
### Visualization Analysis

To examine whether neural activity was aligned with the pretrained behavioral space, we computed the pairwise cosine similarity between neural and behavioral representations\. The resulting similarity matrix in Fig\.[6](https://arxiv.org/html/2608.04389#Sx4.F6)A exhibited a clear diagonal structure, indicating that each neural representation was most similar to its corresponding behavioral representation, while mismatched pairs generally showed lower similarity\. This pattern provides direct representation\-level evidence that the alignment objective successfully maps neural activity into a behaviorally meaningful embedding space\.

To further examine how motor alignment reshapes the learned neural representation, we visualized held\-out neural windows from six recording sessions using PCA in Fig\.[6](https://arxiv.org/html/2608.04389#Sx4.F6)B\. Before alignment, the neural representations were dominated by session\-specific variability: samples from different recording sessions showed noticeable separation, whereas representations associated with different reach directions were substantially intermixed\. After projection into the pretrained behavioral representation space, samples became more clearly structured according to reach direction, while marker shapes corresponding to different sessions were broadly intermingled within the same direction\-related regions\. These results suggest that motor alignment suppresses session\-dependent nuisance variation while preserving behaviorally relevant movement information, thereby producing a more session\-invariant neural representation\.

### Ablation Studies

We conducted ablation experiments on the CO dataset to quantify the contribution of contrastive alignment and the key components of the neural encoder, as summarized in Table[2](https://arxiv.org/html/2608.04389#Sx4.T2)\. We validated that removing the motor encoder structure produced the largest degradation, withR2R^\{2\}of 0\.8296±\\pm0\.0424\. This substantial decrease, together with the increased performance variability, demonstrates that alignment with the pretrained behavioral representation space is critical for learning accurate and stable neural representations\. Among the neural\-encoder components, removing unit encoding caused the largest performance reduction, yielding anR2R^\{2\}of 0\.8825±\\pm0\.0155, which highlights the importance of preserving neuron\-specific information\. Removing RoPE reduced performance to 0\.9011±\\pm0\.0090, confirming the value of explicitly modeling the relative temporal structure of spike sequences\. Excluding latent self\-attention resulted in anR2R^\{2\}of 0\.9065±\\pm0\.0133, indicating that interactions among latent tokens contribute to population\-level representation learning\. Removing cross\-attention produced a smaller but consistent decrease to 0\.9182±\\pm0\.0107, demonstrating its role in selectively aggregating spike\-token information\. Overall, these results show that behavioral contrastive alignment provides the largest contribution, while unit identity, temporal encoding, cross\-attention, and latent self\-attention jointly support effective neural trajectory decoding\.

Table 2:Ablation performance
### Conclusion

We introduced NeuroPB, a neural decoding framework that transfers knowledge from pretrained behavioral representations to motor trajectory decoding\. By aligning neural activity with a frozen behavioral representation space, NeuroPB consistently outperformed its scratch\-trained counterpart\. Macaque and scale\-matched robotic trajectories provided comparable pretraining benefits, while large\-scale robotic pretraining achieved the best performance, improved generalization, and calibration efficiency across sessions, subjects, and tasks\. This study is limited to offline evaluation using invasive macaque recordings\. Future work will extend behavioral pretraining and neural alignment to human recordings, including non\-invasive neural signals, and evaluate NeuroPB in real\-time closed\-loop applications\.

## References

- M\. Azabou, V\. Arora, V\. Ganesh, X\. Mao, S\. Nachimuthu, M\. Mendelson, B\. Richards, M\. Perich, G\. Lajoie, and E\. Dyer \(2023\)A unified, scalable framework for neural population decoding\.Advances in Neural Information Processing Systems36,pp\. 44937–44956\.Cited by:[Neural Encoder](https://arxiv.org/html/2608.04389#Sx3.SSx2.p1.1),[Comparison with Baselines](https://arxiv.org/html/2608.04389#Sx4.SSx3.p1.2)\.
- M\. Azabou, K\. Pan, V\. Arora, I\. Knight, E\. Dyer, and B\. A\. Richards \(2025\)Multi\-session, multi\-task neural decoding from distinct cell\-types and brain regions\.InInternational Conference on Learning Representations,Vol\.2025,pp\. 59654–59677\.Cited by:[Introduction](https://arxiv.org/html/2608.04389#Sx1.p2.1),[Motor Neural Decoding](https://arxiv.org/html/2608.04389#Sx2.SSx1.p1.1)\.
- J\. M\. Carmena, M\. A\. Lebedev, R\. E\. Crist, J\. E\. O’Doherty, D\. M\. Santucci, D\. F\. Dimitrov, P\. G\. Patil, C\. S\. Henriquez, and M\. A\. L\. Nicolelis \(2003\)Learning to control a brain–machine interface for reaching and grasping by primates\.PLoS Biology1\(2\),pp\. e42\.Cited by:[Motor Neural Decoding](https://arxiv.org/html/2608.04389#Sx2.SSx1.p1.1),[Comparison with Baselines](https://arxiv.org/html/2608.04389#Sx4.SSx3.p1.2)\.
- A\. Défossez, C\. Caucheteux, J\. Rapin, O\. Kabeli, and J\. King \(2023\)Decoding speech perception from non\-invasive brain recordings\.Nature Machine Intelligence5\(10\),pp\. 1097–1107\.Cited by:[Contrastive Learning in Neural Decoding](https://arxiv.org/html/2608.04389#Sx2.SSx2.p1.1)\.
- E\. L\. Dyer, M\. Gheshlaghi Azar, M\. G\. Perich, H\. L\. Fernandes, S\. Naufel, L\. E\. Miller, and K\. P\. Körding \(2017\)A cryptography\-based approach for movement decoding\.Nature Biomedical Engineering1\(12\),pp\. 967–976\.Cited by:[Introduction](https://arxiv.org/html/2608.04389#Sx1.p2.1)\.
- A\. Fishbach, S\. A\. Roy, C\. Bastianen, L\. E\. Miller, and J\. C\. Houk \(2007\)Deciding when and how to correct a movement: discrete submovements as a decision making process\.Experimental brain research177\(1\),pp\. 45–63\.Cited by:[Introduction](https://arxiv.org/html/2608.04389#Sx1.p3.1)\.
- J\. I\. Glaser, A\. S\. Benjamin, R\. H\. Chowdhury, M\. G\. Perich, L\. E\. Miller, and K\. P\. Kording \(2020\)Machine learning for neural decoding\.eneuro7\(4\)\.Cited by:[Introduction](https://arxiv.org/html/2608.04389#Sx1.p2.1),[Motor Neural Decoding](https://arxiv.org/html/2608.04389#Sx2.SSx1.p1.1),[Comparison with Baselines](https://arxiv.org/html/2608.04389#Sx4.SSx3.p1.2)\.
- A\. Jaegle, S\. Borgeaud, J\. Alayrac, C\. Doersch, C\. Ionescu, D\. Ding, S\. Koppula, D\. Zoran, A\. Brock, E\. Shelhamer,et al\.\(2021\)Perceiver io: a general architecture for structured inputs & outputs\.arXiv preprint arXiv:2107\.14795\.Cited by:[Neural Encoder](https://arxiv.org/html/2608.04389#Sx3.SSx2.p1.1)\.
- M\. R\. Keshtkaran, A\. R\. Sedler, R\. H\. Chowdhury, R\. Tandon, D\. Basrai, S\. L\. Nguyen, H\. Sohn, M\. Jazayeri, L\. E\. Miller, and C\. Pandarinath \(2022\)A large\-scale neural network training framework for generalized estimation of single\-trial population dynamics\.Nature methods19\(12\),pp\. 1572–1577\.Cited by:[Motor Neural Decoding](https://arxiv.org/html/2608.04389#Sx2.SSx1.p1.1)\.
- J\. Y\. Lee, S\. Lee, A\. Mishra, X\. Yan, B\. McMahan, B\. Gaisford, C\. Kobashigawa, M\. Qu, C\. Xie, and J\. C\. Kao \(2025\)Brain–computer interface control with artificial intelligence copilots\.Nature machine intelligence7\(9\),pp\. 1510–1523\.Cited by:[Introduction](https://arxiv.org/html/2608.04389#Sx1.p1.1)\.
- B\. Liu, Y\. Zhu, C\. Gao, Y\. Feng, Q\. Liu, Y\. Zhu, and P\. Stone \(2023\)Libero: benchmarking knowledge transfer for lifelong robot learning\.Advances in Neural Information Processing Systems36,pp\. 44776–44791\.Cited by:[Introduction](https://arxiv.org/html/2608.04389#Sx1.p3.1)\.
- X\. Mao, N\. H\. Krishna, A\. H\. Ryoo, M\. G\. Perich, and G\. Lajoie \(2026\)Leveraging unlabelled data for generalizable neural population decoding\.arXiv preprint arXiv:2607\.14086\.Cited by:[Comparison with Baselines](https://arxiv.org/html/2608.04389#Sx4.SSx3.p1.2)\.
- J\. Merel, M\. Botvinick, and G\. Wayne \(2019\)Hierarchical motor control in mammals and machines\.Nature communications10\(1\),pp\. 5489\.Cited by:[Introduction](https://arxiv.org/html/2608.04389#Sx1.p3.1)\.
- A\. O’Neill, A\. Rehman, A\. Maddukuri, A\. Gupta, A\. Padalkar, A\. Lee, A\. Pooley, A\. Gupta, A\. Mandlekar, A\. Jain,et al\.\(2024\)Open x\-embodiment: robotic learning datasets and rt\-x models: open x\-embodiment collaboration 0\.In2024 IEEE International Conference on Robotics and Automation \(ICRA\),pp\. 6892–6903\.Cited by:[Introduction](https://arxiv.org/html/2608.04389#Sx1.p3.1)\.
- C\. Pandarinath, D\. J\. O’Shea, J\. Collins, R\. Jozefowicz, S\. D\. Stavisky, J\. C\. Kao, E\. M\. Trautmann, M\. T\. Kaufman, S\. I\. Ryu, L\. R\. Hochberg,et al\.\(2018\)Inferring single\-trial neural population dynamics using sequential auto\-encoders\.Nature Methods15\(10\),pp\. 805–815\.Cited by:[Introduction](https://arxiv.org/html/2608.04389#Sx1.p2.1),[Motor Neural Decoding](https://arxiv.org/html/2608.04389#Sx2.SSx1.p1.1),[Comparison with Baselines](https://arxiv.org/html/2608.04389#Sx4.SSx3.p1.2)\.
- M\. G\. Perich, J\. A\. Gallego, and L\. E\. Miller \(2018\)A neural population mechanism for rapid learning\.Neuron100\(4\),pp\. 964–976\.Cited by:[Datasets](https://arxiv.org/html/2608.04389#Sx4.SSx1.p1.1)\.
- A\. Radford, J\. W\. Kim, C\. Hallacy, A\. Ramesh, G\. Goh, S\. Agarwal, G\. Sastry, A\. Askell, P\. Mishkin, J\. Clark,et al\.\(2021\)Learning transferable visual models from natural language supervision\.InInternational Conference on Machine Learning,pp\. 8748–8763\.Cited by:[Contrastive Learning in Neural Decoding](https://arxiv.org/html/2608.04389#Sx2.SSx2.p1.1),[Modality Alignment](https://arxiv.org/html/2608.04389#Sx3.SSx3.p1.1)\.
- A\. H\. Ryoo, N\. H\. Krishna, X\. Mao, M\. Azabou, E\. L\. Dyer, M\. G\. Perich, and G\. Lajoie \(2026\)Generalizable, real\-time neural decoding with hybrid state\-space models\.Advances in Neural Information Processing Systems38,pp\. 51764–51791\.Cited by:[Comparison with Baselines](https://arxiv.org/html/2608.04389#Sx4.SSx3.p1.2)\.
- S\. Schneider, J\. H\. Lee, and M\. W\. Mathis \(2023\)Learnable latent embeddings for joint behavioural and neural analysis\.Nature617\(7960\),pp\. 360–368\.Cited by:[Introduction](https://arxiv.org/html/2608.04389#Sx1.p2.1),[Motor Neural Decoding](https://arxiv.org/html/2608.04389#Sx2.SSx1.p1.1),[Contrastive Learning in Neural Decoding](https://arxiv.org/html/2608.04389#Sx2.SSx2.p1.1)\.
- Y\. Song, B\. Liu, X\. Li, N\. Shi, Y\. Wang, and X\. Gao \(2023\)Decoding natural images from EEG for object recognition\.arXiv preprint arXiv:2308\.13234\.Cited by:[Contrastive Learning in Neural Decoding](https://arxiv.org/html/2608.04389#Sx2.SSx2.p1.1)\.
- J\. Su, M\. Ahmed, Y\. Lu, S\. Pan, W\. Bo, and Y\. Liu \(2024\)Roformer: enhanced transformer with rotary position embedding\.Neurocomputing568,pp\. 127063\.Cited by:[Neural Encoder](https://arxiv.org/html/2608.04389#Sx3.SSx2.p1.6)\.
- K\. Tian, S\. Zhao, Y\. Zhang, and S\. Yu \(2026\)Multi\-dimensional neural decoding with orthogonal representations for brain\-computer interfaces\.InProceedings of the AAAI Conference on Artificial Intelligence,Vol\.40,pp\. 2092–2100\.Cited by:[Motor Neural Decoding](https://arxiv.org/html/2608.04389#Sx2.SSx1.p1.1)\.
- H\. R\. Walke, K\. Black, T\. Z\. Zhao, Q\. Vuong, C\. Zheng, P\. Hansen\-Estruch, A\. W\. He, V\. Myers, M\. J\. Kim, M\. Du,et al\.\(2023\)Bridgedata v2: a dataset for robot learning at scale\.InConference on Robot Learning,pp\. 1723–1736\.Cited by:[Introduction](https://arxiv.org/html/2608.04389#Sx1.p3.1)\.
- C\. Wen, X\. Lin, J\. So, K\. Chen, Q\. Dou, Y\. Gao, and P\. Abbeel \(2023\)Any\-point trajectory modeling for policy learning\.arXiv preprint arXiv:2401\.00025\.Cited by:[Motor Encoder](https://arxiv.org/html/2608.04389#Sx3.SSx1.p1.11)\.
- M\. S\. Willsey, N\. P\. Shah, D\. T\. Avansino, N\. V\. Hahn, R\. M\. Jamiolkowski, F\. B\. Kamdar, L\. R\. Hochberg, F\. R\. Willett, and J\. M\. Henderson \(2025a\)A high\-performance brain–computer interface for finger decoding and quadcopter game control in an individual with paralysis\.Nature Medicine,pp\. 1–9\.Cited by:[Introduction](https://arxiv.org/html/2608.04389#Sx1.p1.1)\.
- M\. S\. Willsey, N\. P\. Shah, D\. T\. Avansino, N\. V\. Hahn, R\. M\. Jamiolkowski, F\. B\. Kamdar, L\. R\. Hochberg, F\. R\. Willett, and J\. M\. Henderson \(2025b\)A high\-performance brain–computer interface for finger decoding and quadcopter game control in an individual with paralysis\.Nature Medicine31\(1\),pp\. 96–104\.Cited by:[Introduction](https://arxiv.org/html/2608.04389#Sx1.p1.1)\.
- W\. Xia, R\. de Charette, C\. Oztireli, and J\. Xue \(2024\)UMBRAE: unified multimodal brain decoding\.InEuropean Conference on Computer Vision,pp\. 242–259\.Cited by:[Contrastive Learning in Neural Decoding](https://arxiv.org/html/2608.04389#Sx2.SSx2.p1.1)\.
- J\. Ye, J\. Collinger, L\. Wehbe, and R\. Gaunt \(2023\)Neural data transformer 2: multi\-context pretraining for neural spiking activity\.Advances in Neural Information Processing Systems36,pp\. 80352–80374\.Cited by:[Introduction](https://arxiv.org/html/2608.04389#Sx1.p3.1),[Comparison with Baselines](https://arxiv.org/html/2608.04389#Sx4.SSx3.p1.2)\.
- J\. Ye and C\. Pandarinath \(2021\)Representation learning for neural population activity with neural data transformers\.arXiv preprint arXiv:2108\.01210\.Cited by:[Motor Neural Decoding](https://arxiv.org/html/2608.04389#Sx2.SSx1.p1.1)\.
- J\. Ye, F\. Rizzoglio, X\. Ma, A\. Smoulder, H\. Mao, G\. Blumenthal, W\. Hockeimer, N\. Kunigk, D\. Moore, P\. Marino,et al\.\(2026\)A generalist intracortical motor decoder\.Advances in Neural Information Processing Systems38,pp\. 110547–110587\.Cited by:[Motor Neural Decoding](https://arxiv.org/html/2608.04389#Sx2.SSx1.p1.1)\.
- W\. Zhang, S\. Wang, Y\. Su, X\. Li, C\. Zhang, and S\. Zhong \(2026a\)Neurobridge: bio\-inspired self\-supervised eeg\-to\-image decoding via cognitive priors and bidirectional semantic alignment\.InProceedings of the AAAI Conference on Artificial Intelligence,Vol\.40,pp\. 18028–18036\.Cited by:[Contrastive Learning in Neural Decoding](https://arxiv.org/html/2608.04389#Sx2.SSx2.p1.1)\.
- Y\. Zhang, L\. He, C\. Fan, T\. Liu, H\. Yu, T\. Le, J\. Li, S\. Linderman, L\. Duncker, F\. R\. Willett,et al\.\(2026b\)A cross\-species neural foundation model for end\-to\-end speech decoding\.InThe Fourteenth International Conference on Learning Representations,Cited by:[Contrastive Learning in Neural Decoding](https://arxiv.org/html/2608.04389#Sx2.SSx2.p1.1)\.
- Y\. Zhang, Y\. Wang, M\. Azabou, A\. Andre, Z\. Wang, H\. Lyu, T\. I\. B\. Laboratory, E\. Dyer, L\. Paninski, and C\. Hurwitz \(2025\)Neural encoding and decoding at scale\.ArXiv,pp\. arXiv–2504\.Cited by:[Introduction](https://arxiv.org/html/2608.04389#Sx1.p2.1),[Motor Neural Decoding](https://arxiv.org/html/2608.04389#Sx2.SSx1.p1.1),[Comparison with Baselines](https://arxiv.org/html/2608.04389#Sx4.SSx3.p1.2)\.