Accelerating Hierarchical Sparse Predictive Coding with Hybrid Amortized Inference

# Accelerating Hierarchical Sparse Predictive Coding with Hybrid Amortized Inference
Source: [https://arxiv.org/html/2606.27802](https://arxiv.org/html/2606.27802)
\\corrauthor

\[1\]Kazuhisa Fujitakazu@spikingneuron\.net

Kazuhisa FujitaDepartment of Clinical Engineering, Komatsu University, 10\-10 Doihara\-Machi, Komatsu, Ishikawa, Japan 923\-0921

###### Abstract

Hierarchical predictive coding provides an interpretable framework for perception as error\-driven inference in multi\-layer generative models, while sparse coding imposes parsimonious latent representations through explicit sparsity constraints\. Their combination yields hierarchical sparse predictive coding models with appealing computational and neuroscientific properties, but practical use is often limited by the cost of iterative latent inference\. In such models, each input may require many recurrent refinement steps before a useful sparse representation is obtained, and this burden becomes more severe as the hierarchy deepens\. We study this bottleneck by holding the hierarchical sparse energy fixed and varying the inference procedure\. The comparison includes four schemes: classical iterative inference based on ISTA, an accelerated MFISTA reference, structurally informed amortized inference using a LISTA\-style bottom\-up encoder adapted to the hierarchical model, and a hybrid method in which this fast amortized initialization is followed by a small number of corrective energy\-based refinement steps\. Under this shared objective, we measure reconstruction quality, sparsity, latency, and stability on static image benchmarks\. The results show that a shallow LISTA\-style initializer plus short corrective recurrence improves over pure amortization while remaining much faster than long iterative inference\.

## 1Introduction

Perception can be formulated as inference over latent causes\. A useful model should recover those causes efficiently and reliably, while keeping the representation interpretable\. Predictive coding frames this process as hierarchical error minimization: top\-down predictions are compared with bottom\-up inputs, and residual prediction errors propagate through the hierarchy\(Rao and Ballard,,[1999](https://arxiv.org/html/2606.27802#bib.bib24); Friston and Kiebel,,[2009](https://arxiv.org/html/2606.27802#bib.bib14)\)\. Sparse coding instead explains sensory signals with a small number of active coefficients in an overcomplete dictionary\. This yields efficient and biologically meaningful representations; on natural images, sparse coding recovers localized, oriented, band\-pass features resembling simple\-cell receptive fields in primary visual cortex\(Olshausen and Field,,[1996](https://arxiv.org/html/2606.27802#bib.bib22)\)\.

These traditions are compatible\. Predictive coding provides hierarchical generative structure and local error\-driven inference; sparse coding provides an explicit sparsity prior and competition among units\. Their combination has led to hierarchical sparse predictive coding models, most notably Sparse Deep Predictive Coding \(SDPC\), which integrates recurrent sparse inference within layers with feedforward and feedback interactions across layers\(Boutin et al\.,,[2021](https://arxiv.org/html/2606.27802#bib.bib9)\)\. SDPC can learn oriented low\-level receptive fields, more complex higher\-level features, and context\-sensitive effects such as contour integration, supporting hierarchical sparse predictive coding as both a computational and neuroscientific framework\(Boutin et al\.,,[2021](https://arxiv.org/html/2606.27802#bib.bib9)\)\.

The main practical weakness is the inference engine\. Standard formulations require sparsity\-regularized inverse problems to be solved for each input, often by ISTA or related proximal methods\(Daubechies et al\.,,[2004](https://arxiv.org/html/2606.27802#bib.bib11);[Beck and Teboulle, 2009a,](https://arxiv.org/html/2606.27802#bib.bib6);[Beck and Teboulle, 2009b,](https://arxiv.org/html/2606.27802#bib.bib7)\)\. These methods are principled, but they can be expensive: each new input may need many refinement steps before reaching a useful latent state\. The cost becomes more serious in deeper hierarchies, where latency, instability, and repeated inner\-loop optimization can dominate the system\. Recurrent sparse inference has also been studied as a neural computation model with thresholding dynamics and local competition\(Rozell et al\.,,[2008](https://arxiv.org/html/2606.27802#bib.bib27)\)\.

A natural improvement to this bottleneck is amortization, especially when the amortized map retains structure from the original sparse optimization problem\. In sparse inference, Learning ISTA \(LISTA\) replaces many ISTA\-like optimization steps with a learned feedforward network obtained by algorithm unrolling, yielding a structurally informed amortized predictor of approximate sparse codes\(Gregor and LeCun,,[2010](https://arxiv.org/html/2606.27802#bib.bib16)\)\. In predictive coding more broadly, hybrid predictive coding has shown how amortized and iterative inference can optimize one objective, combining fast feedforward initialization with recurrent refinement\(Tschantz et al\.,,[2023](https://arxiv.org/html/2606.27802#bib.bib28)\)\.

We study how to equip hierarchical sparse predictive coding for static images with faster and more stable inference\. We treat the inference mechanism, rather than the image architecture or sparse objective, as the experimental variable\. The objective, model family, and evaluation protocol are held fixed so that iterative, LISTA\-style amortized, and hybrid amortized\-plus\-iterative inference can be compared under the same energy\-based formulation\.

The paper contributes a fixed\-objective benchmark for hierarchical sparse predictive coding on static images, a unified comparison of iterative, LISTA\-style amortized, and Hybrid inference, and an empirical analysis of their quality–latency–stability trade\-offs\.

## 2Related Work

### 2\.1Sparse coding and dictionary learning

Sparse coding provides a foundational framework for efficient representation learning\. It frames perception as reconstruction from a small number of active features in a large, overcomplete dictionary\. The seminal work ofOlshausen and Field, \([1996](https://arxiv.org/html/2606.27802#bib.bib22)\)showed that sparse coding trained on natural images learns basis functions resembling receptive fields in primary visual cortex\. It has therefore become established as both a useful feature\-extraction method and a biologically plausible model of early vision\.

A large body of later work focused on scalable optimization and dictionary learning\. Iterative shrinkage\-thresholding methods and their accelerated variants became standard tools for solving the underlyingℓ1\\ell\_\{1\}\-regularized inverse problems\([Beck and Teboulle, 2009a,](https://arxiv.org/html/2606.27802#bib.bib6);[Beck and Teboulle, 2009b,](https://arxiv.org/html/2606.27802#bib.bib7)\)\. Related work developed online dictionary learning for sparse coding practical at larger scale\(Mairal et al\.,,[2010](https://arxiv.org/html/2606.27802#bib.bib20)\)\. Other work also extended patch\-based sparse coding to convolutional settings, which are better suited to images because they model translation structure more directly\(Bristow et al\.,,[2013](https://arxiv.org/html/2606.27802#bib.bib10)\)\. These developments made sparse coding much more practical, but they did not remove its central inference bottleneck: obtaining sparse latent codes still usually requires iterative optimization for each input\.

### 2\.2Predictive coding and free\-energy\-based inference

Predictive coding is often formulated as a framework for approximate inference in hierarchical generative models\(Rao and Ballard,,[1999](https://arxiv.org/html/2606.27802#bib.bib24); Friston,,[2005](https://arxiv.org/html/2606.27802#bib.bib13); Friston and Kiebel,,[2009](https://arxiv.org/html/2606.27802#bib.bib14)\)\. In classical hierarchical formulations, higher cortical areas send predictions to lower areas, while lower areas return the mismatch between predicted and observed activity, that is, prediction errors\(Rao and Ballard,,[1999](https://arxiv.org/html/2606.27802#bib.bib24); Friston,,[2005](https://arxiv.org/html/2606.27802#bib.bib13); Bastos et al\.,,[2012](https://arxiv.org/html/2606.27802#bib.bib5)\)\. Recurrent exchanges between predictions and errors then iteratively update latent representations so that they better explain sensory input\(Rao and Ballard,,[1999](https://arxiv.org/html/2606.27802#bib.bib24); Friston,,[2005](https://arxiv.org/html/2606.27802#bib.bib13)\)\. This line of work was later generalized within the free\-energy framework, in which perceptual inference is written as the minimization of a variational objective under a hierarchical generative model\(Friston and Kiebel,,[2009](https://arxiv.org/html/2606.27802#bib.bib14)\)\. These formulations are attractive in computational neuroscience because both inference and learning can be described in terms of local message passing based on prediction errors\(Friston,,[2005](https://arxiv.org/html/2606.27802#bib.bib13); Bastos et al\.,,[2012](https://arxiv.org/html/2606.27802#bib.bib5)\)\. Later work also showed that predictive\-coding\-style networks can approximate backpropagation under suitable conditions\(Whittington and Bogacz,,[2017](https://arxiv.org/html/2606.27802#bib.bib29); Rosenbaum,,[2022](https://arxiv.org/html/2606.27802#bib.bib26)\)\. Related top\-down modulation mechanisms have also been studied in biologically motivated visual models, including dynamic gating of task\-relevant information in primary visual cortex and categorization\-oriented interactions between higher and lower visual areas\(Kamiyama et al\.,,[2016](https://arxiv.org/html/2606.27802#bib.bib17); Abe et al\.,,[2018](https://arxiv.org/html/2606.27802#bib.bib1); Kashimori et al\.,,[2007](https://arxiv.org/html/2606.27802#bib.bib18)\)\.

Here, predictive coding matters as both a neuroscientific theory and an algorithmic template\. It provides hierarchical structure, local error\-driven updates, and a natural iterative inference procedure\. However, standard predictive coding networks require multiple recurrent steps per input, making deep inference costly and sometimes difficult to stabilize\(Tschantz et al\.,,[2023](https://arxiv.org/html/2606.27802#bib.bib28)\)\.

### 2\.3Bridging predictive coding and sparse hierarchical generative models

Several works have highlighted the compatibility between predictive coding and sparse coding\. Predictive coding contributes hierarchical generative structure and error\-driven inference; sparse coding contributes an explicit sparsity prior and competition among latent units\. This has motivated models that embed sparse latent inference in predictive\-coding\-style top\-down and bottom\-up interactions\.

A representative example is Sparse Deep Predictive Coding \(SDPC\), which combines predictive coding across layers with sparse recurrent inference within each layer in a hierarchical convolutional architecture\(Boutin et al\.,,[2021](https://arxiv.org/html/2606.27802#bib.bib9)\)\. SDPC can learn oriented low\-level receptive fields, structured higher\-level representations, and context\-sensitive effects such as contour integration\. It therefore shows that predictive coding and sparse coding can be unified in a single model family for realistic image data\. At the same time, it illustrates the main practical limitation of this approach: inference remains recurrent and expensive, especially as model depth and complexity increase\(Boutin et al\.,,[2021](https://arxiv.org/html/2606.27802#bib.bib9)\)\.

### 2\.4Algorithm unrolling and learned sparse inference

A separate but closely related line of work comes from algorithm unrolling for sparse inference\. LISTA showed that the iterations of ISTA can be unfolded into a finite\-depth neural network whose parameters are learned from data, thereby replacing slow iterative optimization with fast feedforward \(i\.e\., amortized\) inference that approximates sparse codes\(Gregor and LeCun,,[2010](https://arxiv.org/html/2606.27802#bib.bib16)\)\. Later work clarified and expanded this design space\. For example, learned\-step\-size variants of unfolded ISTA were shown to be competitive with state\-of\-the\-art learned sparse\-inference networks when the target solutions are sufficiently sparse\(Ablin et al\.,,[2019](https://arxiv.org/html/2606.27802#bib.bib3)\)\. Analytic LISTA \(ALISTA\) further showed that strong performance can be obtained even when only the step\-sizes and thresholds are learned\(Liu et al\.,,[2019](https://arxiv.org/html/2606.27802#bib.bib19)\)\.

Despite this progress, most unrolled sparse coding work has focused on single\-layer sparse recovery rather than hierarchical predictive coding with explicit top\-down generative interactions\. This leaves an open question: how should fast amortized inference be combined with recurrent error\-correcting refinement in a multi\-layer sparse generative model?

### 2\.5Hybrid predictive coding and amortized\-plus\-iterative inference

Recent work in predictive coding has begun to address this question from the viewpoint of amortized inference\. Hybrid Predictive Coding \(HPC\) interprets feedforward inference as amortized inference and recurrent predictive\-coding updates as iterative refinement under the same objective\(Tschantz et al\.,,[2023](https://arxiv.org/html/2606.27802#bib.bib28)\)\. It formalizes a dual\-process view: fast feedforward pathways provide cheap initial beliefs, while recurrent dynamics improve accuracy, context sensitivity, and robustness when more refinement is needed\.

HPC provides a strong conceptual precedent, but it does not focus on hierarchical sparse coding with explicitℓ1\\ell\_\{1\}\-type latent sparsity for static image representation\. Furthermore, while HPC typically relies on unstructured black\-box neural networks for the amortized pathway, our approach employs a structurally informed LISTA\-style encoder with dictionary\-derived initialization and learned shrinkage blocks\. HPC establishes the general principle of combining amortized and iterative inference in predictive coding, while sparse coding provides tools for sparse objectives\. This combination has been less directly studied for hierarchical sparse image models\.

### 2\.6Positioning of the present work

Our work is closest to SDPC on the modeling side and to LISTA and HPC on the inference side\. The emphasis is different\. Relative to SDPC, we vary the inference engine while keeping the sparse generative model fixed\. Relative to LISTA and related unrolling methods, we study hierarchical latent inference rather than single\-layer sparse recovery\. Relative to HPC, we specialize amortized\-plus\-iterative inference to hierarchical sparse coding for static images, where reconstruction quality, sparsity, latency, and stability can be compared under one sparse energy\.

We therefore focus on a narrower question: how amortized and iterative inference should be combined when the sparse objective and hierarchical generative model are held fixed\.

## 3Methods

### 3\.1Overview

We compare four inference schemes for the hierarchical sparse model defined below: \(i\) ISTA\-style iterative inference as the basic iterative baseline\(Daubechies et al\.,,[2004](https://arxiv.org/html/2606.27802#bib.bib11);[Beck and Teboulle, 2009a,](https://arxiv.org/html/2606.27802#bib.bib6)\), \(ii\) MFISTA as a monotone accelerated iterative baseline\([Beck and Teboulle, 2009a,](https://arxiv.org/html/2606.27802#bib.bib6)\), \(iii\) a shared\-parameter LISTA\-style amortized bottom\-up encoder adapted to the hierarchical model\(Gregor and LeCun,,[2010](https://arxiv.org/html/2606.27802#bib.bib16)\), and \(iv\) a proposed Hybrid method in which this encoder provides an initialization that is further refined by a small number of ISTA\-style steps\. All four use the same hierarchical sparse objective\.

### 3\.2Hierarchical sparse generative model

Let𝒙∈ℝn0\\bm\{x\}\\in\\mathbb\{R\}^\{n\_\{0\}\}denote an observed image represented as a column vector, and let𝒂ℓ∈ℝnℓ\\bm\{a\}\_\{\\ell\}\\in\\mathbb\{R\}^\{n\_\{\\ell\}\}denote the latent code at layerℓ∈\{1,…,L\}\\ell\\in\\\{1,\\dots,L\\\}\. For each layer,𝑫ℓ∈ℝnℓ−1×nℓ\\bm\{D\}\_\{\\ell\}\\in\\mathbb\{R\}^\{n\_\{\\ell\-1\}\\times n\_\{\\ell\}\}denotes a dictionary matrix\. The hierarchical generative model is

𝒙\\displaystyle\\bm\{x\}≈𝑫1​𝒂1,\\displaystyle\\approx\\bm\{D\}\_\{1\}\\bm\{a\}\_\{1\},\(1\)𝒂ℓ−1\\displaystyle\\bm\{a\}\_\{\\ell\-1\}≈𝑫ℓ​𝒂ℓ,ℓ=2,…,L\.\\displaystyle\\approx\\bm\{D\}\_\{\\ell\}\\bm\{a\}\_\{\\ell\},\\qquad\\ell=2,\\dots,L\.\(2\)Equivalently, the layerwise residuals are

𝒓1\\displaystyle\\bm\{r\}\_\{1\}=𝒙−𝑫1​𝒂1,\\displaystyle=\\bm\{x\}\-\\bm\{D\}\_\{1\}\\bm\{a\}\_\{1\},\(3\)𝒓ℓ\\displaystyle\\bm\{r\}\_\{\\ell\}=𝒂ℓ−1−𝑫ℓ​𝒂ℓ,ℓ=2,…,L\.\\displaystyle=\\bm\{a\}\_\{\\ell\-1\}\-\\bm\{D\}\_\{\\ell\}\\bm\{a\}\_\{\\ell\},\\qquad\\ell=2,\\dots,L\.\(4\)Thus, the first layer explains the sensory input, while deeper layers explain the activities of lower layers\.

### 3\.3Hierarchical sparse energy

We define the single\-sample energy as

E​\(𝒙,\{𝒂ℓ\},\{𝑫ℓ\}\)=12​‖𝒙−𝑫1​𝒂1‖22\+∑ℓ=2Lβℓ−12​‖𝒂ℓ−1−𝑫ℓ​𝒂ℓ‖22\+∑ℓ=1Lλℓ​‖𝒂ℓ‖1,E\(\\bm\{x\},\\\{\\bm\{a\}\_\{\\ell\}\\\},\\\{\\bm\{D\}\_\{\\ell\}\\\}\)=\\frac\{1\}\{2\}\\\|\\bm\{x\}\-\\bm\{D\}\_\{1\}\\bm\{a\}\_\{1\}\\\|\_\{2\}^\{2\}\+\\sum\_\{\\ell=2\}^\{L\}\\frac\{\\beta\_\{\\ell\-1\}\}\{2\}\\\|\\bm\{a\}\_\{\\ell\-1\}\-\\bm\{D\}\_\{\\ell\}\\bm\{a\}\_\{\\ell\}\\\|\_\{2\}^\{2\}\+\\sum\_\{\\ell=1\}^\{L\}\\lambda\_\{\\ell\}\\\|\\bm\{a\}\_\{\\ell\}\\\|\_\{1\},\(5\)whereλℓ\>0\\lambda\_\{\\ell\}\>0controls sparsity andβℓ−1\>0\\beta\_\{\\ell\-1\}\>0controls inter\-layer coupling\. The quadratic terms penalize input reconstruction error and inconsistency between adjacent latent layers\. Theℓ1\\ell\_\{1\}terms encourage sparse latent codes like lasso regression\. Thus, the objective extends standard sparse coding, reconstruction error plus sparsity, to a coupled hierarchical setting\(Olshausen and Field,,[1996](https://arxiv.org/html/2606.27802#bib.bib22); Aberdam et al\.,,[2019](https://arxiv.org/html/2606.27802#bib.bib2)\)\.

For a mini\-batch\{𝒙\(b\)\}b=1B\\\{\\bm\{x\}^\{\(b\)\}\\\}\_\{b=1\}^\{B\}whereBBis the batch size and𝒙\(b\)\\bm\{x\}^\{\(b\)\}denotes thebb\-th sample in the batch, the training objective is the batch average of the same energy:

ℒ=1B​∑b=1B\[12​‖𝒙\(b\)−𝑫1​𝒂1\(b\)‖22\+∑ℓ=2Lβℓ−12​‖𝒂ℓ−1\(b\)−𝑫ℓ​𝒂ℓ\(b\)‖22\+∑ℓ=1Lλℓ​‖𝒂ℓ\(b\)‖1\]\.\\mathcal\{L\}=\\frac\{1\}\{B\}\\sum\_\{b=1\}^\{B\}\\left\[\\frac\{1\}\{2\}\\\|\\bm\{x\}^\{\(b\)\}\-\\bm\{D\}\_\{1\}\\bm\{a\}\_\{1\}^\{\(b\)\}\\\|\_\{2\}^\{2\}\+\\sum\_\{\\ell=2\}^\{L\}\\frac\{\\beta\_\{\\ell\-1\}\}\{2\}\\\|\\bm\{a\}\_\{\\ell\-1\}^\{\(b\)\}\-\\bm\{D\}\_\{\\ell\}\\bm\{a\}\_\{\\ell\}^\{\(b\)\}\\\|\_\{2\}^\{2\}\+\\sum\_\{\\ell=1\}^\{L\}\\lambda\_\{\\ell\}\\\|\\bm\{a\}\_\{\\ell\}^\{\(b\)\}\\\|\_\{1\}\\right\]\.\(6\)

### 3\.4ISTA\-style latent inference

With dictionaries fixed, latent inference is posed as the minimization of Eq\. \([5](https://arxiv.org/html/2606.27802#S3.E5)\) with respect to\{𝒂ℓ\}ℓ=1L\\\{\\bm\{a\}\_\{\\ell\}\\\}\_\{\\ell=1\}^\{L\}\. Our basic iterative baseline is an ISTA\-style proximal\-gradient method: each refinement step computes layerwise gradients for the smooth part of the hierarchical energy and then applies soft\-thresholding to the latent codes\(Daubechies et al\.,,[2004](https://arxiv.org/html/2606.27802#bib.bib11);[Beck and Teboulle, 2009a,](https://arxiv.org/html/2606.27802#bib.bib6)\)\.

Letffdenote the smooth part of Eq\. \([5](https://arxiv.org/html/2606.27802#S3.E5)\), i\.e\. the same objective without theℓ1\\ell\_\{1\}penalties\. In column\-vector form, the layerwise gradients are as follows\.

Forℓ=1\\ell=1,

∇𝒂1f=𝑫1⊤​\(𝑫1​𝒂1−𝒙\)\+β1​\(𝒂1−𝑫2​𝒂2\),\\nabla\_\{\\bm\{a\}\_\{1\}\}f=\\bm\{D\}\_\{1\}^\{\\top\}\(\\bm\{D\}\_\{1\}\\bm\{a\}\_\{1\}\-\\bm\{x\}\)\+\\beta\_\{1\}\(\\bm\{a\}\_\{1\}\-\\bm\{D\}\_\{2\}\\bm\{a\}\_\{2\}\),\(7\)whenL\>1L\>1, and the second term is omitted whenL=1L=1\.

For1<ℓ<L1<\\ell<L,

∇𝒂ℓf=βℓ−1​𝑫ℓ⊤​\(𝑫ℓ​𝒂ℓ−𝒂ℓ−1\)\+βℓ​\(𝒂ℓ−𝑫ℓ\+1​𝒂ℓ\+1\)\.\\nabla\_\{\\bm\{a\}\_\{\\ell\}\}f=\\beta\_\{\\ell\-1\}\\bm\{D\}\_\{\\ell\}^\{\\top\}\(\\bm\{D\}\_\{\\ell\}\\bm\{a\}\_\{\\ell\}\-\\bm\{a\}\_\{\\ell\-1\}\)\+\\beta\_\{\\ell\}\(\\bm\{a\}\_\{\\ell\}\-\\bm\{D\}\_\{\\ell\+1\}\\bm\{a\}\_\{\\ell\+1\}\)\.\(8\)
Forℓ=L\\ell=L,

∇𝒂Lf=βL−1​𝑫L⊤​\(𝑫L​𝒂L−𝒂L−1\)\.\\nabla\_\{\\bm\{a\}\_\{L\}\}f=\\beta\_\{L\-1\}\\bm\{D\}\_\{L\}^\{\\top\}\(\\bm\{D\}\_\{L\}\\bm\{a\}\_\{L\}\-\\bm\{a\}\_\{L\-1\}\)\.\(9\)
We then apply the proximal update

𝒂ℓ←𝒮θℓ​\(𝒂ℓ−ηℓ​∇𝒂ℓf\),\\bm\{a\}\_\{\\ell\}\\leftarrow\\mathcal\{S\}\_\{\\theta\_\{\\ell\}\}\\\!\\left\(\\bm\{a\}\_\{\\ell\}\-\\eta\_\{\\ell\}\\nabla\_\{\\bm\{a\}\_\{\\ell\}\}f\\right\),\(10\)where𝒮θ\\mathcal\{S\}\_\{\\theta\}is the soft\-thresholding operator

𝒮θ​\(v\)=sign​\(v\)​max⁡\(\|v\|−θ,0\),\\mathcal\{S\}\_\{\\theta\}\(v\)=\\mathrm\{sign\}\(v\)\\max\(\|v\|\-\\theta,0\),\(11\)applied elementwise, andηℓ\\eta\_\{\\ell\}andθℓ\\theta\_\{\\ell\}are the step size and threshold for layerℓ\\ell, respectively\.

In implementation, all layer gradients are evaluated at the current iterate, and the proximal updates are then applied within the same refinement step\. The iterative baseline is therefore a block\-Jacobi proximal\-gradient refinement on the joint latent variables\([Beck and Teboulle, 2009a,](https://arxiv.org/html/2606.27802#bib.bib6);[Beck and Teboulle, 2009b,](https://arxiv.org/html/2606.27802#bib.bib7); Parikh and Boyd,,[2014](https://arxiv.org/html/2606.27802#bib.bib23); Xu,,[2018](https://arxiv.org/html/2606.27802#bib.bib30)\), not a Gauss–Seidel or cyclic block\-coordinate method\(Beck and Tetruashvili,,[2013](https://arxiv.org/html/2606.27802#bib.bib8); Xu,,[2018](https://arxiv.org/html/2606.27802#bib.bib30)\)\. We use “ISTA\-style” to emphasize the gradient\-plus\-soft\-thresholding update form, while distinguishing this simultaneous multi\-layer update from a literal cyclic application of classical single\-block ISTA\.

In standard proximal\-gradient methods, fixed step sizes are determined by the Lipschitz constant of the full gradient\([Beck and Teboulle, 2009a,](https://arxiv.org/html/2606.27802#bib.bib6);[Beck and Teboulle, 2009b,](https://arxiv.org/html/2606.27802#bib.bib7); Parikh and Boyd,,[2014](https://arxiv.org/html/2606.27802#bib.bib23)\)\. Similarly, block\-coordinate variants use block\-specific Lipschitz constants for each partial gradient\(Nesterov,,[2012](https://arxiv.org/html/2606.27802#bib.bib21); Richtárik and Takác̆,,[2014](https://arxiv.org/html/2606.27802#bib.bib25)\)\. Following this principle, we determine layerwise step sizes based on the block\-Lipschitz constants of the partial gradients for the smooth term\.

For our smooth objective, the relevant block Hessians are

∇a1​a12f\\displaystyle\\nabla^\{2\}\_\{a\_\{1\}a\_\{1\}\}f=𝑫1⊤​𝑫1\+β1​𝑰n1\(L\>1\),\\displaystyle=\\bm\{D\}\_\{1\}^\{\\top\}\\bm\{D\}\_\{1\}\+\\beta\_\{1\}\\bm\{I\}\_\{n\_\{1\}\}\\qquad\(L\>1\),\(12\)∇aℓ​aℓ2f\\displaystyle\\nabla^\{2\}\_\{a\_\{\\ell\}a\_\{\\ell\}\}f=βℓ−1​𝑫ℓ⊤​𝑫ℓ\+βℓ​𝑰nℓ,\(1<ℓ<L\),\\displaystyle=\\beta\_\{\\ell\-1\}\\bm\{D\}\_\{\\ell\}^\{\\top\}\\bm\{D\}\_\{\\ell\}\+\\beta\_\{\\ell\}\\bm\{I\}\_\{n\_\{\\ell\}\},\\qquad\(1<\\ell<L\),\(13\)∇aL​aL2f\\displaystyle\\nabla^\{2\}\_\{a\_\{L\}a\_\{L\}\}f=βL−1​𝑫L⊤​𝑫L\.\\displaystyle=\\beta\_\{L\-1\}\\bm\{D\}\_\{L\}^\{\\top\}\\bm\{D\}\_\{L\}\.\(14\)Taking the spectral norm of each block Hessian, the corresponding block\-Lipschitz constants are

L1\\displaystyle L\_\{1\}=‖𝑫1‖spec2\+β1\(L\>1\),\\displaystyle=\\\|\\bm\{D\}\_\{1\}\\\|\_\{\\mathrm\{spec\}\}^\{2\}\+\\beta\_\{1\}\\qquad\(L\>1\),\(15\)Lℓ\\displaystyle L\_\{\\ell\}=βℓ−1​‖𝑫ℓ‖spec2\+βℓ\(1<ℓ<L\),\\displaystyle=\\beta\_\{\\ell\-1\}\\\|\\bm\{D\}\_\{\\ell\}\\\|\_\{\\mathrm\{spec\}\}^\{2\}\+\\beta\_\{\\ell\}\\qquad\(1<\\ell<L\),\(16\)LL\\displaystyle L\_\{L\}=βL−1​‖𝑫L‖spec2,\\displaystyle=\\beta\_\{L\-1\}\\\|\\bm\{D\}\_\{L\}\\\|\_\{\\mathrm\{spec\}\}^\{2\},\(17\)and for the single\-layer caseL1=‖𝑫1‖spec2L\_\{1\}=\\\|\\bm\{D\}\_\{1\}\\\|\_\{\\mathrm\{spec\}\}^\{2\}\. Here,∥⋅∥spec\\\|\\cdot\\\|\_\{\\mathrm\{spec\}\}is the spectral norm, so‖𝑫ℓ‖spec2=λmax​\(𝑫ℓ⊤​𝑫ℓ\)\\\|\\bm\{D\}\_\{\\ell\}\\\|\_\{\\mathrm\{spec\}\}^\{2\}=\\lambda\_\{\\max\}\(\\bm\{D\}\_\{\\ell\}^\{\\top\}\\bm\{D\}\_\{\\ell\}\)is the largest eigenvalue of𝑫ℓ⊤​𝑫ℓ\\bm\{D\}\_\{\\ell\}^\{\\top\}\\bm\{D\}\_\{\\ell\}\. In practice,‖𝑫ℓ‖spec2\\\|\\bm\{D\}\_\{\\ell\}\\\|\_\{\\mathrm\{spec\}\}^\{2\}is estimated by 10 power iterations on𝑫ℓ⊤​𝑫ℓ\\bm\{D\}\_\{\\ell\}^\{\\top\}\\bm\{D\}\_\{\\ell\}from a fixed all\-ones initial vector, followed by a Rayleigh\-quotient estimate\(Demmel,,[1997](https://arxiv.org/html/2606.27802#bib.bib12); Golub and van Loan,,[2013](https://arxiv.org/html/2606.27802#bib.bib15)\)\.

The actual step sizes are

ηℓ=ηscaleLℓ,θℓ=ηℓ​λℓ,\\eta\_\{\\ell\}=\\frac\{\\eta\_\{\\mathrm\{scale\}\}\}\{L\_\{\\ell\}\},\\qquad\\theta\_\{\\ell\}=\\eta\_\{\\ell\}\\lambda\_\{\\ell\},\(18\)whereηscale\\eta\_\{\\mathrm\{scale\}\}is a single global scaling hyperparameter that controls the overall speed of inference\. Because these layerwise updates are applied simultaneously to coupled latent variables, this blockwise rule should be read as a practical scaling choice rather than a monotonicity guarantee for the full joint objective\.

In training, the step sizes are recomputed from the current detached dictionaries\. When dictionaries are fixed at evaluation time, the same rule may be precomputed once and then reused for all inputs\.

### 3\.5LISTA\-style amortized encoder

For fast approximate inference, we use a bottom\-up amortized encoder inspired by LISTA\(Gregor and LeCun,,[2010](https://arxiv.org/html/2606.27802#bib.bib16)\)\. Classical LISTA unfolds ISTA into a learned finite\-depth sparse\-coding network for a single sparse\-coding problem\. Here, we use the same idea as a structurally informed feed\-forward inference module adapted to the hierarchical sparse model\. This module is used both as the amortized initializer for Hybrid and as the LISTA\-style inference method for hierarchical sparse coding in the comparisons below\.

The encoder is composed ofLLlayerwise blocks\. Each block maps the current layer input to an approximate latent code for that layer, and the resulting code is then passed to the next block in a bottom\-up manner:

𝒂1\\displaystyle\\bm\{a\}\_\{1\}=f1​\(𝒙\),\\displaystyle=f\_\{1\}\(\\bm\{x\}\),\(19\)𝒂2\\displaystyle\\bm\{a\}\_\{2\}=f2​\(𝒂1\),\\displaystyle=f\_\{2\}\(\\bm\{a\}\_\{1\}\),\(20\)⋮\\displaystyle\\vdots\(21\)𝒂L\\displaystyle\\bm\{a\}\_\{L\}=fL​\(𝒂L−1\)\.\\displaystyle=f\_\{L\}\(\\bm\{a\}\_\{L\-1\}\)\.\(22\)Thus, the encoder defines a feed\-forward LISTA\-style amortized approximation to the hierarchical latent codes\. It should not be read as an exact unrolling of proximal\-gradient inference for the full coupled hierarchical objective\. Instead, it is a layerwise bottom\-up initializer whose blocks have ISTA\-like shrinkage structure; the full reconstruction and inter\-layer coupling terms are enforced directly only by the iterative refinement stage and by the training loss\.

In the main experiments, we use the*shared\-parameter*variant\. For a given layer input𝒖ℓ\\bm\{u\}\_\{\\ell\}, one encoder block first computes

𝑩ℓ=Wx,ℓ​𝒖ℓ,\\bm\{B\}\_\{\\ell\}=W\_\{x,\\ell\}\\bm\{u\}\_\{\\ell\},\(23\)whereWx,ℓW\_\{x,\\ell\}is an encoder weight matrix, and then applies a fixed number of shrinkage stages:

𝒂ℓ\(0\)\\displaystyle\\bm\{a\}^\{\(0\)\}\_\{\\ell\}=𝒮ϑℓ​\(𝑩ℓ\),\\displaystyle=\\mathcal\{S\}\_\{\\vartheta\_\{\\ell\}\}\(\\bm\{B\}\_\{\\ell\}\),\(24\)𝒂ℓ\(t\+1\)\\displaystyle\\bm\{a\}^\{\(t\+1\)\}\_\{\\ell\}=𝒮ϑℓ​\(𝑩ℓ\+Wa,ℓ​𝒂ℓ\(t\)\),t=0,…,K−2,\\displaystyle=\\mathcal\{S\}\_\{\\vartheta\_\{\\ell\}\}\\\!\\left\(\\bm\{B\}\_\{\\ell\}\+W\_\{a,\\ell\}\\bm\{a\}^\{\(t\)\}\_\{\\ell\}\\right\),\\qquad t=0,\\dots,K\-2,\(25\)whereKKis the total number of LISTA\-style shrinkage stages in the block, the final output is𝒂ℓ\(K−1\)\\bm\{a\}^\{\(K\-1\)\}\_\{\\ell\}, andWa,ℓW\_\{a,\\ell\}is another encoder weight matrix\. Thus,K=1K=1means that only the initial shrinkage stage is applied\. Here,𝒮ϑℓ\\mathcal\{S\}\_\{\\vartheta\_\{\\ell\}\}denotes elementwise soft\-thresholding with the learned LISTA thresholdϑℓ\\vartheta\_\{\\ell\}, distinct from the ISTA refinement thresholdθℓ\\theta\_\{\\ell\}in Eq\. \([18](https://arxiv.org/html/2606.27802#S3.E18)\)\. The same parameters\(Wx,ℓ,Wa,ℓ,ϑℓ\)\(W\_\{x,\\ell\},W\_\{a,\\ell\},\\vartheta\_\{\\ell\}\)are shared across all internal shrinkage stages within a block\. To ensure non\-negative thresholds, we reparameterize

ϑℓ=softplus​\(ρℓ\),\\vartheta\_\{\\ell\}=\\mathrm\{softplus\}\(\\rho\_\{\\ell\}\),\(26\)whereρℓ\\rho\_\{\\ell\}is an unconstrained encoder parameter\.

During training and evaluation, LISTA\-style amortized inference uses the current encoder weights and thresholds directly\. Its shrinkage stages in Eq\. \([25](https://arxiv.org/html/2606.27802#S3.E25)\) do not recompute an ISTA step size; any step\-size\-like scaling is learned inWx,ℓW\_\{x,\\ell\},Wa,ℓW\_\{a,\\ell\}, andρℓ\\rho\_\{\\ell\}\. Pure LISTA returns this feed\-forward code directly\. In the Hybrid variant, the same forward pass produces an initial latent code, then refines it with ISTA\-style updates under the full hierarchical energy\.

### 3\.6Hybrid inference

The proposed method combines amortized latent\-code initialization with iterative refinement\. Here “initialization” means the starting latent codes for the current input, not initialization of encoder parameters\. Hybrid first runs the LISTA\-style amortized encoder and then applies ISTA\-style refinement\. Given𝒙\\bm\{x\}, the bottom\-up initialization is

\{𝒂ℓ\(0\)\}ℓ=1L=LISTA​\(𝒙;ϕ\),\\\{\\bm\{a\}\_\{\\ell\}^\{\(0\)\}\\\}\_\{\\ell=1\}^\{L\}=\\mathrm\{LISTA\}\(\\bm\{x\};\\phi\),\(27\)whereϕ\\phidenotes the encoder parameters\. This forward pass gives a fast approximate latent code\. Other amortized architectures could in principle be substituted\.

Starting from this initialization, we then performTrefT\_\{\\mathrm\{ref\}\}ISTA\-style refinement steps using Eq\. \([10](https://arxiv.org/html/2606.27802#S3.E10)\)\. The final inferred codes are

𝒂^ℓ=𝒂ℓ\(Tref\),ℓ=1,…,L\.\\hat\{\\bm\{a\}\}\_\{\\ell\}=\\bm\{a\}\_\{\\ell\}^\{\(T\_\{\\mathrm\{ref\}\}\)\},\\qquad\\ell=1,\\dots,L\.\(28\)These refinement steps correct the initial amortized codes using gradients and shrinkage operations derived from the full hierarchical sparse energy\.

Thus, the proposed Hybrid method is defined as:

> bottom\-up amortized initialization with a shared LISTA\-style encoder, followed by a small number of ISTA\-style refinement steps under the full hierarchical sparse energy\.

The motivation is simple: the amortized encoder provides a useful initial estimate with low cost, and iterative refinement corrects it by minimizing the full hierarchical sparse energy\. LISTA uses the encoder output directly; Hybrid corrects it with full\-energy recurrence\.

### 3\.7Training procedure and gradient flow

#### Alternating training view

Training alternates between latent inference and parameter updates\. This design is best viewed as an alternating\-optimization\-style approximation:

1. 1\.infer latent codes while holding dictionaries fixed;
2. 2\.evaluate the batch loss using the learnable dictionaries;
3. 3\.update dictionaries, and when applicable encoder parameters, using that final loss\.

The inferred codes are therefore treated as the result of optimizing latent variables with fixed dictionaries\. Dictionary updates are driven by the final sparse\-coding loss, without backpropagating dictionary gradients through every refinement step, which reduces memory use and improves training stability\.

#### Dictionary gradient flow

During latent inference, the dictionaries are treated as fixed\. In the implementation, the learnable dictionaries\{𝑫ℓ\}ℓ=1L\\\{\\bm\{D\}\_\{\\ell\}\\\}\_\{\\ell=1\}^\{L\}are first converted into detached inference dictionaries,

𝑫~ℓ=stopgrad⁡\(𝑫ℓ\),ℓ=1,…,L,\\tilde\{\\bm\{D\}\}\_\{\\ell\}=\\operatorname\{stopgrad\}\(\\bm\{D\}\_\{\\ell\}\),\\qquad\\ell=1,\\dots,L,\(29\)and the inference engine receives\{𝑫~ℓ\}ℓ=1L\\\{\\tilde\{\\bm\{D\}\}\_\{\\ell\}\\\}\_\{\\ell=1\}^\{L\}rather than the original parameters\. Conceptually, the inferred codes are computed as

\{𝒂^ℓ\}ℓ=1L=Infer⁡\(𝒙;\{𝑫~ℓ\}ℓ=1L\)\.\\\{\\hat\{\\bm\{a\}\}\_\{\\ell\}\\\}\_\{\\ell=1\}^\{L\}=\\operatorname\{Infer\}\\left\(\\bm\{x\};\\\{\\tilde\{\\bm\{D\}\}\_\{\\ell\}\\\}\_\{\\ell=1\}^\{L\}\\right\)\.\(30\)Thus, inference uses the current dictionary values, but the computation graph does not keep a gradient path from the inferred codes back to the dictionary parameters through the inference trajectory\.

After inference, the final mini\-batch loss is evaluated with the original learnable dictionaries, not with the detached copies:

ℒ=E​\(𝒙,\{𝒂^ℓ\}ℓ=1L,\{𝑫ℓ\}ℓ=1L\)\.\\mathcal\{L\}=E\\left\(\\bm\{x\},\\\{\\hat\{\\bm\{a\}\}\_\{\\ell\}\\\}\_\{\\ell=1\}^\{L\},\\\{\\bm\{D\}\_\{\\ell\}\\\}\_\{\\ell=1\}^\{L\}\\right\)\.\(31\)Dictionary learning is therefore driven by the explicit dependence of the final loss on\{𝑫ℓ\}ℓ=1L\\\{\\bm\{D\}\_\{\\ell\}\\\}\_\{\\ell=1\}^\{L\}\. If the detached dictionaries were also used in this final loss, the loss would contain no dictionary\-gradient path and the dictionaries would not be learned from the reconstruction and inter\-layer consistency terms\. The inferred codes are treated as constants when updating dictionaries\. Equivalently, the update differentiates the final energy with respect to dictionary parameters, but not through the iterative inference procedure\.

#### Encoder gradient flow

In Hybrid, the LISTA\-style encoder provides the initial codes for refinement\. These codes are not detached, so gradients from the final refined codes can flow back to encoder parameters\. The dictionaries used inside refinement are detached, so the refinement trajectory does not provide a gradient path back to dictionary parameters\. The appendix\-only Hybrid\-MFISTA ablation uses the same encoder\-gradient convention\.

After refinement, the final loss is evaluated using the refined codes and the original learnable dictionaries\. Dictionary parameters are updated through their explicit appearance in this loss, whereas encoder parameters are updated through their influence on the refined codes\. The Hybrid encoder is trained to produce initial codes that become good solutions after refinement\.

#### Learned parameter groups

The learnable dictionary parameters are

𝚯D=\{𝑫ℓ\}ℓ=1L\.\\bm\{\\Theta\}\_\{D\}=\\\{\\bm\{D\}\_\{\\ell\}\\\}\_\{\\ell=1\}^\{L\}\.\(32\)For LISTA and Hybrid, the model also contains amortized encoder parameters, denoted by

𝚯E=ϕ\.\\bm\{\\Theta\}\_\{E\}=\\phi\.\(33\)ISTA and MFISTA do not contain such encoder parameters, since they compute latent codes only by iterative inference\.

Parameter learning uses standard gradient\-based optimization\. In all modes,𝚯D\\bm\{\\Theta\}\_\{D\}is updated from the final loss\. In LISTA and Hybrid,𝚯E\\bm\{\\Theta\}\_\{E\}is updated from the same loss\. Both parameter groups are optimized with Adam using separate learning rates\. Thus, the same hierarchical sparse objective trains the dictionaries and, when present, the amortized encoder\.

LISTA and Hybrid are trained as separate models\. They share the same encoder architecture and initialization rule, but their learned encoder parameters are not shared\. The appendix\-only Hybrid\-MFISTA ablation is also trained separately, using the Hybrid encoder setup while replacing only the refinement rule\.

#### Dictionary normalization

After each parameter update, all dictionary atoms are normalized:

𝒅ℓ,m←𝒅ℓ,mmax⁡\(‖𝒅ℓ,m‖2,δ\),\\bm\{d\}\_\{\\ell,m\}\\leftarrow\\frac\{\\bm\{d\}\_\{\\ell,m\}\}\{\\max\(\\\|\\bm\{d\}\_\{\\ell,m\}\\\|\_\{2\},\\delta\)\},\(34\)where𝒅ℓ,m\\bm\{d\}\_\{\\ell,m\}denotes themm\-th atom of dictionary𝑫ℓ\\bm\{D\}\_\{\\ell\}andδ\>0\\delta\>0is a small constant\.

#### Encoder initialization

Before training, the generative dictionaries are randomly initialized and column\-normalized\. For LISTA and Hybrid, the LISTA\-style encoder parameters are then initialized once from these dictionaries\. This dictionary\-derived initialization is not repeated during later inference or evaluation\. For layerℓ\\ell, let𝑫ℓ\\bm\{D\}\_\{\\ell\}be the initial local dictionary and let

Lℓenc≈‖𝑫ℓ‖spec2L\_\{\\ell\}^\{\\mathrm\{enc\}\}\\approx\\\|\\bm\{D\}\_\{\\ell\}\\\|\_\{\\mathrm\{spec\}\}^\{2\}\(35\)be the spectral estimate used for this encoder initialization\. This estimate uses the same 10\-step power\-iteration procedure as the inference step\-size rule\. We set

ηℓenc=ηscaleLℓenc,\\eta\_\{\\ell\}^\{\\mathrm\{enc\}\}=\\frac\{\\eta\_\{\\mathrm\{scale\}\}\}\{L\_\{\\ell\}^\{\\mathrm\{enc\}\}\},\(36\)and initialize

Wx,ℓ\\displaystyle W\_\{x,\\ell\}←ηℓenc​𝑫ℓ⊤,\\displaystyle\\leftarrow\\eta\_\{\\ell\}^\{\\mathrm\{enc\}\}\\bm\{D\}\_\{\\ell\}^\{\\top\},\(37\)Wa,ℓ\\displaystyle W\_\{a,\\ell\}←𝑰nℓ−ηℓenc​𝑫ℓ⊤​𝑫ℓ,\\displaystyle\\leftarrow\\bm\{I\}\_\{n\_\{\\ell\}\}\-\\eta\_\{\\ell\}^\{\\mathrm\{enc\}\}\\bm\{D\}\_\{\\ell\}^\{\\top\}\\bm\{D\}\_\{\\ell\},\(38\)ϑℓ\\displaystyle\\vartheta\_\{\\ell\}←ηℓenc​λℓ\.\\displaystyle\\leftarrow\\eta\_\{\\ell\}^\{\\mathrm\{enc\}\}\\lambda\_\{\\ell\}\.\(39\)In the softplus parameterization, this last assignment is implemented by settingρℓ\\rho\_\{\\ell\}so thatsoftplus​\(ρℓ\)=ηℓenc​λℓ\\mathrm\{softplus\}\(\\rho\_\{\\ell\}\)=\\eta\_\{\\ell\}^\{\\mathrm\{enc\}\}\\lambda\_\{\\ell\}\.

After this one\-time initialization, the encoder is not reinitialized as dictionaries change\. Instead,Wx,ℓW\_\{x,\\ell\},Wa,ℓW\_\{a,\\ell\}, andρℓ\\rho\_\{\\ell\}are learned by minimizing the same hierarchical sparse loss in Eq\. \([6](https://arxiv.org/html/2606.27802#S3.E6)\)\. Thus,LℓencL\_\{\\ell\}^\{\\mathrm\{enc\}\}andηℓenc\\eta\_\{\\ell\}^\{\\mathrm\{enc\}\}serve only as initialization quantities, not as step\-size parameters used during later LISTA inference\. No teacher\-code imitation loss is used; LISTA is trained end\-to\-end under the shared sparse objective\.

#### Overall training procedure

Training proceeds as follows:

1. 1\.sample a mini\-batch of images;
2. 2\.flatten images into vectors and optionally remove the per\-sample DC component;
3. 3\.infer latent codes using one of the four inference engines: ISTA\-style iterative inference, MFISTA, shared LISTA, or Hybrid;
4. 4\.compute the batch\-averaged hierarchical sparse loss in Eq\. \([6](https://arxiv.org/html/2606.27802#S3.E6)\);
5. 5\.update dictionaries and, when applicable, encoder parameters by gradient\-based optimization;
6. 6\.normalize all dictionary atoms\.

The generative model, dictionary capacity, sparsity coefficients, and evaluation protocol are fixed across methods\. The main difference is the inference engine in Step 3, with LISTA and Hybrid additionally introducing amortized encoder parameters trained under the same sparse objective\.

### 3\.8Algorithm summaries

The complete implementation\-level procedures for Hybrid inference and training are summarized in Algorithm[1](https://arxiv.org/html/2606.27802#alg1)and Algorithm[2](https://arxiv.org/html/2606.27802#alg2), respectively; the shared model, encoder block, step\-size rule, and fixed inference budgets are specified in the surrounding method and experimental\-protocol sections\. The gradients in the method description above are written for a single column\-vector sample\. In Algorithm[1](https://arxiv.org/html/2606.27802#alg1), batches are written in the implementation convention with samples as rows, so𝒂ℓ\\bm\{a\}\_\{\\ell\}and𝒈ℓ\\bm\{g\}\_\{\\ell\}denoteB×nℓB\\times n\_\{\\ell\}matrices and thebb\-th row of𝒈ℓ\\bm\{g\}\_\{\\ell\}is the transpose of the corresponding column\-vector gradient∇𝒂ℓf\\nabla\_\{\\bm\{a\}\_\{\\ell\}\}f\.

Algorithm 1Hybrid inference forLL\-layer hierarchical sparse coding1:input batch

𝑿∈ℝB×n0\\bm\{X\}\\in\\mathbb\{R\}^\{B\\times n\_\{0\}\}, dictionaries

\{𝑫ℓ\}ℓ=1L\\\{\\bm\{D\}\_\{\\ell\}\\\}\_\{\\ell=1\}^\{L\}with

𝑫ℓ∈ℝnℓ−1×nℓ\\bm\{D\}\_\{\\ell\}\\in\\mathbb\{R\}^\{n\_\{\\ell\-1\}\\times n\_\{\\ell\}\}, sparsity weights

\{λℓ\}ℓ=1L\\\{\\lambda\_\{\\ell\}\\\}\_\{\\ell=1\}^\{L\}, coupling weights

\{βℓ\}ℓ=1L−1\\\{\\beta\_\{\\ell\}\\\}\_\{\\ell=1\}^\{L\-1\}, amortized encoder, LISTA\-style shrinkage stages

KK, refinement steps

TrefT\_\{\\mathrm\{ref\}\}, scale parameter

ηscale\\eta\_\{\\mathrm\{scale\}\}
2:latent codes

\{𝒂ℓ\}ℓ=1L\\\{\\bm\{a\}\_\{\\ell\}\\\}\_\{\\ell=1\}^\{L\}
3:Step 1: amortized bottom\-up initialization with shared LISTA blocks

4:

𝑼←𝑿\\bm\{U\}\\leftarrow\\bm\{X\}
5:for

ℓ=1\\ell=1to

LLdo

6:

𝑩ℓ←𝑼​Wx,ℓ⊤\\bm\{B\}\_\{\\ell\}\\leftarrow\\bm\{U\}W\_\{x,\\ell\}^\{\\top\}
7:

𝒂ℓ←𝒮ϑℓ​\(𝑩ℓ\)\\bm\{a\}\_\{\\ell\}\\leftarrow\\mathcal\{S\}\_\{\\vartheta\_\{\\ell\}\}\(\\bm\{B\}\_\{\\ell\}\)
8:for

k=1k=1to

K−1K\-1do

9:

𝒂ℓ←𝒮ϑℓ​\(𝑩ℓ\+𝒂ℓ​Wa,ℓ⊤\)\\bm\{a\}\_\{\\ell\}\\leftarrow\\mathcal\{S\}\_\{\\vartheta\_\{\\ell\}\}\(\\bm\{B\}\_\{\\ell\}\+\\bm\{a\}\_\{\\ell\}W\_\{a,\\ell\}^\{\\top\}\)
10:endfor

11:

𝑼←𝒂ℓ\\bm\{U\}\\leftarrow\\bm\{a\}\_\{\\ell\}
12:endfor

13:Step 2: layerwise step sizes

14:for

ℓ=1\\ell=1to

LLdo

15:if

ℓ=1\\ell=1then

16:

L1←‖𝑫1‖spec2L\_\{1\}\\leftarrow\\\|\\bm\{D\}\_\{1\}\\\|\_\{\\mathrm\{spec\}\}^\{2\}
17:if

L\>1L\>1then

18:

L1←L1\+β1L\_\{1\}\\leftarrow L\_\{1\}\+\\beta\_\{1\}
19:endif

20:elseif

ℓ=L\\ell=Lthen

21:

LL←βL−1​‖𝑫L‖spec2L\_\{L\}\\leftarrow\\beta\_\{L\-1\}\\\|\\bm\{D\}\_\{L\}\\\|\_\{\\mathrm\{spec\}\}^\{2\}
22:else

23:

Lℓ←βℓ−1​‖𝑫ℓ‖spec2\+βℓL\_\{\\ell\}\\leftarrow\\beta\_\{\\ell\-1\}\\\|\\bm\{D\}\_\{\\ell\}\\\|\_\{\\mathrm\{spec\}\}^\{2\}\+\\beta\_\{\\ell\}
24:endif

25:

ηℓ←ηscale/Lℓ\\eta\_\{\\ell\}\\leftarrow\\eta\_\{\\mathrm\{scale\}\}/L\_\{\\ell\}
26:

θℓ←ηℓ​λℓ\\theta\_\{\\ell\}\\leftarrow\\eta\_\{\\ell\}\\lambda\_\{\\ell\}
27:endfor

28:Step 3: ISTA\-style refinement with simultaneous gradient evaluation

29:for

t=1t=1to

TrefT\_\{\\mathrm\{ref\}\}do

30:for

ℓ=1\\ell=1to

LLdo

31:if

ℓ=1\\ell=1then

32:

𝑿rec←𝒂1​𝑫1⊤\\bm\{X\}\_\{\\mathrm\{rec\}\}\\leftarrow\\bm\{a\}\_\{1\}\\bm\{D\}\_\{1\}^\{\\top\}
33:

𝒈ℓ←\(𝑿rec−𝑿\)​𝑫1\\bm\{g\}\_\{\\ell\}\\leftarrow\(\\bm\{X\}\_\{\\mathrm\{rec\}\}\-\\bm\{X\}\)\\bm\{D\}\_\{1\}
34:if

L\>1L\>1then

35:

𝑷upper←𝒂2​𝑫2⊤\\bm\{P\}\_\{\\mathrm\{upper\}\}\\leftarrow\\bm\{a\}\_\{2\}\\bm\{D\}\_\{2\}^\{\\top\}
36:

𝒈ℓ←𝒈ℓ\+β1​\(𝒂1−𝑷upper\)\\bm\{g\}\_\{\\ell\}\\leftarrow\\bm\{g\}\_\{\\ell\}\+\\beta\_\{1\}\(\\bm\{a\}\_\{1\}\-\\bm\{P\}\_\{\\mathrm\{upper\}\}\)
37:endif

38:elseif

ℓ=L\\ell=Lthen

39:

𝑷lower←𝒂L​𝑫L⊤\\bm\{P\}\_\{\\mathrm\{lower\}\}\\leftarrow\\bm\{a\}\_\{L\}\\bm\{D\}\_\{L\}^\{\\top\}
40:

𝒈ℓ←βL−1​\(𝑷lower−𝒂L−1\)​𝑫L\\bm\{g\}\_\{\\ell\}\\leftarrow\\beta\_\{L\-1\}\(\\bm\{P\}\_\{\\mathrm\{lower\}\}\-\\bm\{a\}\_\{L\-1\}\)\\bm\{D\}\_\{L\}
41:else

42:

𝑷lower←𝒂ℓ​𝑫ℓ⊤\\bm\{P\}\_\{\\mathrm\{lower\}\}\\leftarrow\\bm\{a\}\_\{\\ell\}\\bm\{D\}\_\{\\ell\}^\{\\top\}
43:

𝑷upper←𝒂ℓ\+1​𝑫ℓ\+1⊤\\bm\{P\}\_\{\\mathrm\{upper\}\}\\leftarrow\\bm\{a\}\_\{\\ell\+1\}\\bm\{D\}\_\{\\ell\+1\}^\{\\top\}
44:

𝒈lower←βℓ−1​\(𝑷lower−𝒂ℓ−1\)​𝑫ℓ\\bm\{g\}\_\{\\mathrm\{lower\}\}\\leftarrow\\beta\_\{\\ell\-1\}\(\\bm\{P\}\_\{\\mathrm\{lower\}\}\-\\bm\{a\}\_\{\\ell\-1\}\)\\bm\{D\}\_\{\\ell\}
45:

𝒈upper←βℓ​\(𝒂ℓ−𝑷upper\)\\bm\{g\}\_\{\\mathrm\{upper\}\}\\leftarrow\\beta\_\{\\ell\}\(\\bm\{a\}\_\{\\ell\}\-\\bm\{P\}\_\{\\mathrm\{upper\}\}\)
46:

𝒈ℓ←𝒈lower\+𝒈upper\\bm\{g\}\_\{\\ell\}\\leftarrow\\bm\{g\}\_\{\\mathrm\{lower\}\}\+\\bm\{g\}\_\{\\mathrm\{upper\}\}
47:endif

48:endfor

49:for

ℓ=1\\ell=1to

LLdo

50:

𝒂ℓ←𝒮θℓ​\(𝒂ℓ−ηℓ​𝒈ℓ\)\\bm\{a\}\_\{\\ell\}\\leftarrow\\mathcal\{S\}\_\{\\theta\_\{\\ell\}\}\(\\bm\{a\}\_\{\\ell\}\-\\eta\_\{\\ell\}\\bm\{g\}\_\{\\ell\}\)
51:endfor

52:endfor

53:return

\{𝒂ℓ\}ℓ=1L\\\{\\bm\{a\}\_\{\\ell\}\\\}\_\{\\ell=1\}^\{L\}

Algorithm 2Training procedure used in the implementation1:training set

𝒟\\mathcal\{D\}, model mode

mm, dictionaries

\{𝑫ℓ\}ℓ=1L\\\{\\bm\{D\}\_\{\\ell\}\\\}\_\{\\ell=1\}^\{L\}, optional encoder parameters

ϕ\\phi
2:

m∈\{ista,mfista,lista,hybrid,m\\in\\\{\\texttt\{ista\},\\texttt\{mfista\},\\texttt\{lista\},\\texttt\{hybrid\},
3:

hybrid\_mfista\}\\texttt\{hybrid\\\_mfista\}\\\}
4:trained dictionaries

\{𝑫ℓ\}ℓ=1L\\\{\\bm\{D\}\_\{\\ell\}\\\}\_\{\\ell=1\}^\{L\}and, when applicable, encoder parameters

ϕ\\phi
5:initialize dictionaries

\{𝑫ℓ\}\\\{\\bm\{D\}\_\{\\ell\}\\\}randomly

6:for

ℓ=1\\ell=1to

LLdo

7:normalize columns of

𝑫ℓ\\bm\{D\}\_\{\\ell\}
8:endfor

9:if

mmuses a LISTA\-style encoderthen

10:initialize encoder parameters from current dictionaries

11:endif

12:forepoch = 1 to

NepochN\_\{\\mathrm\{epoch\}\}do

13:formini\-batch

𝑿\\bm\{X\}from

𝒟\\mathcal\{D\}do

14:flatten input images into row vectors

𝑿∈ℝB×n0\\bm\{X\}\\in\\mathbb\{R\}^\{B\\times n\_\{0\}\}
15:ifDC centering is enabledthen

16:subtract per\-sample mean from each row of

𝑿\\bm\{X\}
17:endif

18:

𝑫~ℓ←detach​\(𝑫ℓ\)\\tilde\{\\bm\{D\}\}\_\{\\ell\}\\leftarrow\\mathrm\{detach\}\(\\bm\{D\}\_\{\\ell\}\)for all

ℓ\\ell
19:infer latent codes

\{𝒂ℓ\}\\\{\\bm\{a\}\_\{\\ell\}\\\}using the selected inference engine with

\{𝑫~ℓ\}\\\{\\tilde\{\\bm\{D\}\}\_\{\\ell\}\\\}
20:compute batch\-averaged energy with the original, non\-detached dictionaries

21:

ℒ=1B​∑b=1B\[12​‖𝒙\(b\)−𝑫1​𝒂1\(b\)‖22\+∑ℓ=2Lβℓ−12​‖𝒂ℓ−1\(b\)−𝑫ℓ​𝒂ℓ\(b\)‖22\+∑ℓ=1Lλℓ​‖𝒂ℓ\(b\)‖1\]\\mathcal\{L\}=\\frac\{1\}\{B\}\\sum\_\{b=1\}^\{B\}\\left\[\\frac\{1\}\{2\}\\\|\\bm\{x\}^\{\(b\)\}\-\\bm\{D\}\_\{1\}\\bm\{a\}\_\{1\}^\{\(b\)\}\\\|\_\{2\}^\{2\}\+\\sum\_\{\\ell=2\}^\{L\}\\frac\{\\beta\_\{\\ell\-1\}\}\{2\}\\\|\\bm\{a\}\_\{\\ell\-1\}^\{\(b\)\}\-\\bm\{D\}\_\{\\ell\}\\bm\{a\}\_\{\\ell\}^\{\(b\)\}\\\|\_\{2\}^\{2\}\+\\sum\_\{\\ell=1\}^\{L\}\\lambda\_\{\\ell\}\\\|\\bm\{a\}\_\{\\ell\}^\{\(b\)\}\\\|\_\{1\}\\right\]
22:take an Adam step on

ℒ\\mathcal\{L\}to update

\{𝑫ℓ\}ℓ=1L\\\{\\bm\{D\}\_\{\\ell\}\\\}\_\{\\ell=1\}^\{L\}
23:ifencoder existsthen

24:take an Adam step on

ℒ\\mathcal\{L\}to update

ϕ\\phi
25:endif

26:for

ℓ=1\\ell=1to

LLdo

27:normalize columns of

𝑫ℓ\\bm\{D\}\_\{\\ell\}
28:endfor

29:endfor

30:endfor

31:return

\{𝑫ℓ\}ℓ=1L\\\{\\bm\{D\}\_\{\\ell\}\\\}\_\{\\ell=1\}^\{L\}and

ϕ\\phi

## 4Experimental Setup

The experiments evaluate*inference mechanisms*for the hierarchical sparse generative model\. We specify the datasets, shared hyperparameters, inference\-budget sweeps, latency measurements, and other plotted quantities used to generate the reported figures\. The experimental design is organized around three questions:

1. 1\.How do ISTA, MFISTA, LISTA, and the proposed Hybrid method compare in final test loss, reconstruction error, and inference latency?
2. 2\.How does this comparison change as the hierarchical model becomes deeper?
3. 3\.Are the observed trends stable under changes in sparsity strength and implementation\-level choices in the refinement step size?

### 4\.1Datasets

The main experiments use three static\-image benchmarks: MNIST, Fashion\-MNIST, and CIFAR\-10 Gray\. MNIST provides a simple reference regime, Fashion\-MNIST adds more structured grayscale images, and CIFAR\-10 Gray, a grayscale version of CIFAR\-10, tests whether the same inference trade\-off persists on compact natural images\. Appendix[C](https://arxiv.org/html/2606.27802#A3)adds BSDS500 Patch as a natural\-image patch check connected to the classical sparse\-coding setting; it consists of16×1616\\times 16patches derived from the BSDS500 natural\-image dataset\(Arbelaez et al\.,,[2011](https://arxiv.org/html/2606.27802#bib.bib4)\)\.

For MNIST, Fashion\-MNIST, and CIFAR\-10 Gray, we keep the official test split untouched and reserve10%10\\%of the official training split for validation using a fixed random seed\. The validation split is used for model monitoring and epoch\-wise convergence curves\. Final quality metrics for method comparisons are computed on the held\-out test split\. The dedicated latency\-only pass also uses test batches\.

Table[1](https://arxiv.org/html/2606.27802#S4.T1)summarizes the actual train/validation/test partitions used for the main experiments\.

Table 1:Dataset summary\.
### 4\.2Compared methods and protocol

The primary comparison includes four inference schemes: ISTA, MFISTA, LISTA, and Hybrid\. ISTA and MFISTA are iterative reference methods, LISTA denotes the LISTA\-style amortized encoder baseline adapted to the hierarchical model, and Hybrid uses the same amortized initialization followed by ISTA\-style refinement\. Hybrid\-MFISTA, which replaces the Hybrid refinement stage with MFISTA\-style refinement, is not part of the primary comparison; it is treated only as an appendix additional ablation and reported in Table[4](https://arxiv.org/html/2606.27802#A2.T4)\. The algorithmic details of these methods are given in the Methods section\.

ISTA and MFISTA are not intended to be low\-latency competitors in a pure accuracy contest\. With enough iterations, iterative solvers are expected to approach strong sparse\-code solutions, but this comes with substantially higher inference cost\. We therefore use them mainly to anchor the high\-computation end of the quality–latency curve\. The default iteration counts should be read as practical operating points rather than separately tuned accuracy optima for each iterative solver\. The central comparison is not whether Hybrid dominates fully iterated solvers in final accuracy, but whether amortized initialization plus a small number of corrective steps improves on pure amortization without incurring the cost of full iterative inference\.

All four primary methods use the same sparse energy, data splits, dictionary architecture, optimizer settings, and evaluation code\. The ablation experiments then vary one named factor at a time, such as hierarchy depth \(Fig\.[4](https://arxiv.org/html/2606.27802#S5.F4)\), sparsity strength \(Fig\.[5](https://arxiv.org/html/2606.27802#S5.F5)\), or Hybrid implementation choices \(Fig\.[6](https://arxiv.org/html/2606.27802#S5.F6)\)\.

Throughout the paper,*inference budget*means the number of method\-specific inference steps used to produce a latent code, whereas*latency*means the measured wall\-clock time in ms/sample\. For ISTA and MFISTA, this budget is the number of iterative update steps\. For LISTA\-style amortized inference, it is the number of learned shrinkage stagesKKinside each layerwise encoder block, not an unrolling depth of the full coupled hierarchical objective\. For Hybrid, it is specified by the pair of amortized stages and refinement steps\(K,Tref\)\(K,T\_\{\\mathrm\{ref\}\}\)\. In the main quality\-versus\-budget comparison, the Hybrid amortized depth is fixed atK=1K=1and onlyTrefT\_\{\\mathrm\{ref\}\}is varied; Fig\.[2](https://arxiv.org/html/2606.27802#S5.F2)then reports the measured latency of those sameK=1K=1Hybrid settings together with the corresponding ISTA, MFISTA, and LISTA budget points\. The separate Hybrid allocation analysis in Section[5\.2](https://arxiv.org/html/2606.27802#S5.SS2)variesKKandTrefT\_\{\\mathrm\{ref\}\}independently\.

Unless otherwise stated, final test loss and reconstruction error are aggregated over the completed runs from the target seed set\{0,1,2\}\\\{0,1,2\\\}and plotted as mean values with standard deviation error bars\. Epoch\-wise convergence curves are the exception: they plot validation loss and validation reconstruction error during training\. When latency is plotted, the value is the median ms/sample from the dedicated latency evaluation matched to the same method and inference\-budget setting\.

The dedicated latency evaluation uses seed0, batch size 1, no additional training epochs, 100 warm\-up batches, and at most 500 timed test batches\. This pass measures latency for the same architectures and inference budgets without running training\. Quality metrics come from the corresponding training\-based runs\.

### 4\.3Fixed hyperparameters and ablations

The default model is a two\-layer hierarchy with latent dimensions\(256,64\)\(256,64\)\. Thus the first dictionary maps the flattened input to 256 coefficients, and the second maps 256 coefficients to 64 coefficients; the input dimension is dataset dependent \(784 for28×2828\\times 28grayscale images, 1024 for CIFAR\-10 Gray, and 256 for16×1616\\times 16BSDS500 patches\)\. Unless an ablation changes them, the sparsity weights are tied across layers,λℓ=0\.05\\lambda\_\{\\ell\}=0\.05, and the inter\-layer coupling isβ1=1\.0\\beta\_\{1\}=1\.0\. Dictionary parameters and, when present, encoder parameters are optimized with Adam using learning rates10−310^\{\-3\}for both𝚯D\\bm\{\\Theta\}\_\{D\}and𝚯E\\bm\{\\Theta\}\_\{E\}\. We useηscale=1\.0\\eta\_\{\\mathrm\{scale\}\}=1\.0, batch size 256, 25 training epochs, and no per\-sample DC centering\.

These defaults should be read as balanced working settings identified from preliminary sweeps, not as dataset\-specific or method\-specific optima\. In choosing them, we prioritized a regime that was stable across the tested datasets and methods, computationally feasible for the full method–dataset–ablation grid, and still representative of a nontrivial two\-layer sparse hierarchy\. The sparsity value in particular is not a reconstruction\-optimal value: reducing sparsity pressure can improve reconstruction error at the cost of denser codes\. We therefore treatλ=0\.05\\lambda=0\.05as a useful sparse\-coding regime rather than as an optimum of a single scalar performance measure\. The role of departures from this default is assessed through the plotted depth, sparsity, step\-size, and inference\-budget sweeps\.

The depth\-scaling experiment uses the following script\-defined layer schedules:

L=1L=1\(256\)\(256\)L=2L=2\(256,64\)\(256,64\)L=4L=4\(256,192,128,64\)\(256,192,128,64\)L=6L=6\(256,224,192,160,128,64\)\(256,224,192,160,128,64\)L=8L=8\(256,240,224,192,160,128,96,64\)\(256,240,224,192,160,128,96,64\)This sweep keepsλℓ=0\.05\\lambda\_\{\\ell\}=0\.05andβℓ=1\.0\\beta\_\{\\ell\}=1\.0wherever applicable\. The sparsity ablation sweeps tied coefficientsλ∈\{0\.02,0\.05,0\.10,0\.20\}\\lambda\\in\\\{0\.02,0\.05,0\.10,0\.20\\\}, and the step\-size ablation sweepsηscale∈\{0\.5,1\.0,1\.5\}\\eta\_\{\\mathrm\{scale\}\}\\in\\\{0\.5,1\.0,1\.5\\\}\. Separate inference\-budget sweeps for ISTA, MFISTA, LISTA, and Hybrid are used for the quality\-versus\-budget comparison in Fig\.[1](https://arxiv.org/html/2606.27802#S5.F1)and the budget–latency mapping in Fig\.[2](https://arxiv.org/html/2606.27802#S5.F2), so the comparison is not tied only to the single default inference budget in Table[2](https://arxiv.org/html/2606.27802#S4.T2)\. For ISTA and MFISTA, each additional inference step also adds computation time; the latency plot therefore shows which quality gains come at the cost of slower inference\.

For readability, the default operating point used throughout the main comparisons is summarized in Table[2](https://arxiv.org/html/2606.27802#S4.T2)\.

Table 2:Default experimental hyperparameters used in the main comparisons\.
### 4\.4Evaluation metrics

Because all methods optimize or approximate the same sparse generative objective, the primary plotted quantities are two quality metrics and one timing metric\. For final method comparisons, the quality metrics are computed on the held\-out test split after training\. The timing metric is computed separately on test batches by the dedicated latency\-only protocol\.

The first final quality metric,*test loss*, is the full batch\-averaged hierarchical sparse energy in Eq\. \([6](https://arxiv.org/html/2606.27802#S3.E6)\)\. It includes the input reconstruction term, the inter\-layer consistency terms, and theℓ1\\ell\_\{1\}sparsity penalties\. The second final quality metric,*reconstruction error*, is evaluated on the same test examples but keeps only the input\-space term12​‖𝒙−𝑫1​𝒂1‖22\\frac\{1\}\{2\}\\\|\\bm\{x\}\-\\bm\{D\}\_\{1\}\\bm\{a\}\_\{1\}\\\|\_\{2\}^\{2\}\. Thus, test loss and reconstruction error differ in the component of the objective being measured, not in the data split\. Both are averaged over the full test split after training\. In contrast, the convergence figures use the validation split and plot the same two quantities over epochs\. In the sparsity sweep, we additionally plot the mean active fraction on the test split, defined as the fraction of inferred coefficients with magnitude greater than10−310^\{\-3\}\.

For depth\-scaling and ablation experiments, stability is assessed from the plotted loss, reconstruction error, and between\-seed variability\. We also provide qualitative visualizations of learned first\-layer dictionary atoms and representative reconstructions as diagnostic checks on the learned sparse representation\.

### 4\.5Implementation details

All programs are implemented in Python and PyTorch\. The source code and experiment scripts are publicly available at[https://github\.com/KazuhisaFujita/HSC](https://github.com/KazuhisaFujita/HSC)\. The accompanying repository provides the core scripts and a demo notebook\.

Only the dedicated latency evaluation is run under a fixed hardware and execution setting: a single\-threaded run on a machine with 64 GB RAM and an AMD Ryzen 9900X CPU, using the same batch\-size convention and summary format across methods\. The training\-based experiments used for final test quality metrics were run with parallel processing on different hardware environments\. Therefore, the paper uses the dedicated latency pass for timing comparisons, while quality metrics are taken from the corresponding training\-based runs\.

## 5Results

We evaluate Hybrid by comparing inference budget, measured latency, budget allocation, depth scaling, sparsity strength, and step\-size sensitivity under the fixed protocol described above\.

### 5\.1Quality versus inference budget and latency

Figure[1](https://arxiv.org/html/2606.27802#S5.F1)compares the four inference schemes at a fixed two\-layer hierarchy\. Columns correspond to datasets, and rows show the two test metrics: full sparse\-coding loss and reconstruction error\. The horizontal axis is the method\-specific inference budget\. For ISTA and MFISTA this is the number of iterative updates, for LISTA\-style amortized inference it is the number of learned shrinkage stages per layerwise encoder block, and for the displayed Hybrid curve it isK\+TrefK\+T\_\{\\mathrm\{ref\}\}with the amortized depth fixed atK=1K=1\. Each marker is the mean over completed seeds at one inference setting, with error bars showing between\-seed variation in quality\.

The main pattern is that LISTA\-style amortized inference gives the lowest\-budget solution, but additional layerwise shrinkage stages do not reliably close the quality gap\. On Fashion\-MNIST and CIFAR\-10 Gray in particular, deeper amortized encoder points can have worse test loss or reconstruction error than the shallowK=1K=1point\. By contrast, theK=1K=1Hybrid curve improves on the corresponding amortized baseline with only a few corrective updates\. Across MNIST, Fashion\-MNIST, and CIFAR\-10 Gray, these Hybrid points reduce both test loss and reconstruction error relative toK=1K=1LISTA\-style inference, and additional refinement usually maintains or slightly improves the gains asTrefT\_\{\\mathrm\{ref\}\}increases from 1 to 7\.

![Refer to caption](https://arxiv.org/html/2606.27802v1/x1.png)Figure 1:Quality versus inference budget for ISTA, MFISTA, LISTA, and Hybrid at a fixed hierarchical depth\. Columns show datasets; rows show test loss and reconstruction error\. For Hybrid, the amortized depth is fixed atK=1K=1and the curve varies onlyTrefT\_\{\\mathrm\{ref\}\}\.Figure[2](https://arxiv.org/html/2606.27802#S5.F2)then maps these same inference settings to measured latency\. The quality gains above are not free, but they are obtained in a much lower\-latency region than the fully iterative references\. LISTA is the fastest method, with one\-stage LISTA around0\.030\.03ms/sample in the measured setting\. Adding Hybrid refinement raises latency to roughly0\.070\.07–0\.080\.08ms/sample for one corrective step and about0\.330\.33–0\.370\.37ms/sample for seven corrective steps\. This is still far below the 100\-step ISTA and MFISTA settings, which are several milliseconds per sample on the same plots\.

Together, Figs\.[1](https://arxiv.org/html/2606.27802#S5.F1)and[2](https://arxiv.org/html/2606.27802#S5.F2)show that the proposed method is not simply a compromise on an abstract budget axis\. In measured wall\-clock terms, short Hybrid refinement gives a practical operating point between LISTA and fully iterative inference: it consistently improves over the shallow amortized baseline, while remaining separated from long ISTA/MFISTA runs by a large latency margin\.

![Refer to caption](https://arxiv.org/html/2606.27802v1/x2.png)Figure 2:Measured latency as a function of inference budget for the same fixed\-depth settings used in Fig\.[1](https://arxiv.org/html/2606.27802#S5.F1)\. For Hybrid, the amortized depth is fixed atK=1K=1and the budget changes throughTrefT\_\{\\mathrm\{ref\}\}\.
### 5\.2How Hybrid inference budget should be allocated

The next question is where the Hybrid inference budget should go\. Figure[3](https://arxiv.org/html/2606.27802#S5.F3)separates the two components of Hybrid inference: the number of LISTA\-style amortized shrinkage stagesKKinside the layerwise encoder and the number of corrective refinement stepsTrefT\_\{\\mathrm\{ref\}\}\. The horizontal axis variesTrefT\_\{\\mathrm\{ref\}\}, while different curves correspond to different values ofKK\.

The main result is that increasingKKis not the reliable way to improve Hybrid inference\. On Fashion\-MNIST and CIFAR\-10 Gray, using more amortized shrinkage stages without any refinement \(Tref=0T\_\{\\mathrm\{ref\}\}=0\) does not reliably improve test loss or reconstruction error, and can be worse than the shallowK=1K=1initializer\. By contrast,K=1K=1already provides a useful initialization, and adding iterative refinement consistently improves the code from that starting point\.

Most of the benefit of refinement appears after only a small number of steps\. This indicates that the LISTA initializer is not solving the sparse inference problem by itself, but it places the latent variables close enough that corrective ISTA\-style updates can reach a good regime much faster than inference from a generic initialization\. In this sense, the effective Hybrid allocation is not a deep amortized encoder followed by heavy recurrence, but a shallow amortized initializer followed by a few refinement steps\.

![Refer to caption](https://arxiv.org/html/2606.27802v1/x3.png)Figure 3:Disentangling the Hybrid inference budget across the main datasets\. Different points correspond to different combinations of amortized depthKKand corrective refinementTrefT\_\{\\mathrm\{ref\}\}, showing how allocation choices affect quality rather than holding a single total budget fixed\.Table[3](https://arxiv.org/html/2606.27802#S5.T3)makes this allocation issue explicit on Fashion\-MNIST, where the contrast is easy to see\. Rows are organized by the actual pair\(K,Tref\)\(K,T\_\{\\mathrm\{ref\}\}\)rather than only by the total count, and are restricted to settings with matching dedicated latency measurements under the default two\-layer hierarchy\. This exposes a point that is hidden by a one\-dimensional budget axis: increasing amortized depth is not a reliable substitute for refinement\. At total count66, for example,\(1,5\)\(1,5\)gives lower test loss and reconstruction error than\(5,1\)\(5,1\), although\(5,1\)\(5,1\)has lower latency\. The balanced allocation\(3,3\)\(3,3\)lies between these two in quality, with test loss close to\(1,5\)\(1,5\)but reconstruction error closer to\(5,1\)\(5,1\)\. Thus, even when the total count is fixed, allocating more steps to refinement tends to improve quality, whereas allocating more stages to amortized inference can reduce latency\.

Table 3:Representative Fashion\-MNIST slice of the Hybrid inference\-budget allocation sweep, restricted to settings with a matching dedicated latency measurement under the default two\-layer hierarchy\. Lower is better for test loss, reconstruction error, and latency\.
### 5\.3Depth scaling and stability

We next examine the depth\-scaling plot\. Figure[4](https://arxiv.org/html/2606.27802#S5.F4)shows how quality, latency, and between\-seed variability change as the hierarchy becomes deeper under the layer schedules specified in the experimental setup\.

The depth sweep shows that latency grows with depth, but the quality and between\-seed variability depend strongly on the inference method\. MFISTA serves as a comparatively stable accelerated reference, whereas the ISTA\-style baseline shows large loss variance in some regimes, including MNIST at depths 4 and 6\. These patterns indicate that increasing hierarchy depth affects not only runtime but also the reliability of the inferred sparse codes\.

Pure LISTA shows a different failure mode\. On Fashion\-MNIST, increasing depth beyond the shallow settings does not clearly improve the final metrics, and the between\-seed variance is larger at depth 8\. On CIFAR\-10 Gray, LISTA remains fast but shows a clearer quality gap than on MNIST\. This suggests that a single amortized forward pass is less reliable on Fashion\-MNIST and CIFAR\-10 Gray than on MNIST, making recurrent refinement more useful\.

Hybrid remains much faster than the iterative solvers and is usually more stable than pure LISTA and the unaccelerated ISTA\-style baseline, although it can become variable at some deeper CIFAR\-10 Gray settings\. Its refinement stage improves over LISTA on Fashion\-MNIST and preserves competitive quality on most CIFAR\-10 Gray depths, while avoiding the largest failures of the iterative baseline\.

MNIST generally behaves as an easier regime for the strongest methods: apart from standard ISTA’s increased variance, the depth\-induced gap between valid solvers remains smaller than on Fashion\-MNIST\. This reinforces the interpretation that the main benefit of Hybrid is not merely on the easiest data, but on datasets where pure amortized inference shows a clearer quality or stability gap\.

Overall, the depth sweep shows that Hybrid is not merely a latency compromise: it often improves stability over pure amortization and is generally more stable than unaccelerated ISTA in the tested settings\.

![Refer to caption](https://arxiv.org/html/2606.27802v1/x4.png)Figure 4:Depth scaling and stability across the main datasets\. The panels show how objective quality and runtime change as the hierarchy deepens, making the method\-wise trade\-offs easier to compare than in a compressed aggregate table\.
### 5\.4Effect of sparsity strength and step\-size scaling

We next ask whether the Hybrid behavior depends on one particular sparsity setting\. Figure[5](https://arxiv.org/html/2606.27802#S5.F5)varies the tied sparsity coefficientλ\\lambdawhile keeping the rest of the default setup fixed\. As expected, increasingλ\\lambdareduces the mean active fraction and generally increases reconstruction error, so this sweep should be read as a change in sparsity regime rather than as a search for the best reconstruction setting\.

The important point is that the Hybrid comparison does not collapse when the code becomes denser or sparser\. Across the plottedλ\\lambdavalues, Hybrid remains a low\-latency alternative to the iterative solvers while usually improving on pure LISTA in reconstruction error\. This supports the interpretation that the benefit of adding short refinement is not specific to the single default valueλ=0\.05\\lambda=0\.05\.

![Refer to caption](https://arxiv.org/html/2606.27802v1/x5.png)Figure 5:Effect of sparsity strength on reconstruction error and mean active fraction\.Figure[6](https://arxiv.org/html/2606.27802#S5.F6)then tests whether the refinement result is an artifact of one step\-size scale\. This is a sensitivity check rather than an exhaustive tuning experiment: the sameη\\eta\-scale is varied for ISTA, MFISTA, and Hybrid, and the resulting reconstruction error is compared\.

Hybrid is moderately sensitive to this scale, withηscale=1\.0\\eta\_\{\\mathrm\{scale\}\}=1\.0generally giving the cleanest operating point in the plotted settings, but the method is not uniquely dependent on an exact value\. Smaller or larger scales can worsen the reconstruction error, yet the Hybrid points remain in the same qualitative regime\. In contrast, unaccelerated ISTA is more visibly affected by poor step\-size choices, especially when the scale is increased\. Thus the ablation suggests that Hybrid benefits from a reasonable refinement step size, but its main behavior is not explained solely by fine tuning that parameter\.

![Refer to caption](https://arxiv.org/html/2606.27802v1/x6.png)Figure 6:Effect of step\-size scaling on ISTA, MFISTA, and Hybrid\.

## 6Discussion

The experiments show that the main issue is not only how many inference steps are used, but how those steps are organized\. Pure amortization gives the lowest latency, but can leave a quality gap on Fashion\-MNIST, CIFAR\-10 Gray, and deeper hierarchies\. Fully iterative inference remains the high\-computation reference\. In the tested settings, Hybrid is best interpreted as inference\-budget reallocation: a shallow amortized pass moves the latent state into a useful region, and a small number of corrective updates recovers quality that pure amortization leaves behind\.

This interpretation is strongest in the Hybrid budget\-allocation experiment\. Increasing the number of LISTA\-style shrinkage stages inside the layerwise encoder is not a reliable substitute for refinement: in some settings, using more amortized stages without any refinement \(Tref=0T\_\{\\mathrm\{ref\}\}=0\) gives worse test loss or reconstruction error than the shallowK=1K=1initializer\. By contrast,K=1K=1is already a useful initializer, and most of the refinement gain appears after only a few corrective steps\. For this model class, the more effective allocation is cheap initialization followed by limited error correction\.

The shallow LISTA\-style encoder is already a useful amortized initializer, because even a single learned shrinkage stage maps the input to a data\-dependent sparse code\. However, increasing the number of layerwise amortized stages is not equivalent to performing corrective inference under the full hierarchical energy\. Additional shared LISTA\-style stages repeatedly transform the bottom\-up code within each layer, whereas Hybrid refinement explicitly uses reconstruction and inter\-layer prediction errors from the coupled objective\. This explains whyK=1K=1can be competitive, while largerKKdoes not reliably replace iterative refinement\.

The depth, sparsity, and step\-size checks clarify the scope of this pattern\. Depth scaling shows that Hybrid can improve stability over pure LISTA and the unaccelerated ISTA\-style baseline, but it is not uniformly stable at every depth\. The sparsity sweep shows that the qualitative behavior is not tied to one value ofλ\\lambda\. The step\-size ablation shows moderate sensitivity, especially compared with a poorly scaled iterative baseline, but does not suggest that the Hybrid result is explained only by fine tuning a single refinement parameter\.

The present experiments also leave several limitations that motivate future work\. The implementation is not a fully local biological learning rule, and the observed stability of Hybrid inference should not be interpreted as a general convergence guarantee\. More importantly, the advantage of Hybrid inference should not be attributed to a single mechanism\. Its performance likely reflects the interaction between amortized initialization, sparse iterative refinement, step\-size control, thresholding, hierarchical coupling, and dictionary normalization\. The present study evaluates a deliberately simple Hybrid instantiation in order to isolate the benefit of combining feed\-forward initialization with a small number of refinement steps under a shared sparse\-coding objective\.

Future work should test whether this hybrid principle scales beyond the fully connected static\-image setting studied here\. One direction is convolutional sparse coding and larger image models, where dictionary structure and spatial weight sharing may change both reconstruction quality and inference cost\(Bristow et al\.,,[2013](https://arxiv.org/html/2606.27802#bib.bib10); Boutin et al\.,,[2021](https://arxiv.org/html/2606.27802#bib.bib9)\)\. Another direction is to combine the present framework with more advanced components from modern sparse\-coding and algorithm\-unrolling methods, such as structured amortized encoders, ALISTA\-style parameterizations\(Liu et al\.,,[2019](https://arxiv.org/html/2606.27802#bib.bib19)\), adaptive inference budgets, or safeguarded refinement schemes\. These extensions may improve the quality–latency trade\-off in more demanding domains without changing the central idea of using amortized inference as a fast initializer for sparse iterative correction\.

Finally, the present results prepare the intended extension to temporal data\. Because this paper focuses on static images and reconstruction\-oriented sparse inference, it does not establish claims about temporal prediction, belief dynamics, or active inference\. Introducing temporal latent dynamics, sequential observations, or video inputs would therefore provide a natural test of whether the proposed hybrid inference principle remains useful when data arrive continuously and latency constraints are tighter\.

## 7Conclusion

We studied hierarchical sparse predictive coding for static images under a fixed sparse generative objective\. Rather than proposing a new energy function, representation family, or biological learning rule, we compared how iterative, amortized, and hybrid inference procedures change the behavior of the same model\.

Pure iterative methods preserve the sparse generative semantics but are costly, while pure amortized inference is fast but can lose quality or stability\. Hybrid provides a practical middle regime by using cheap amortized initialization followed by a small amount of corrective refinement\.

This work provides a controlled comparison of accelerated hierarchical sparse inference within one energy\-based framework\.

## Acknowledgements

The author used coding agents, including Codex and GitHub Copilot, with GPT\-5\.5, GPT\-5\.4, Qwen 3\.6 27B, and Qwen 3\.6 35B to help revise wording, improve clarity, check the consistency of the manuscript, and write and revise program code\. The scientific ideas, experimental design, implementation, analysis, interpretation, and final text were reviewed and approved by the author, who takes full responsibility for the content of the paper\.

## References

- Abe et al\., \(2018\)Abe, Y\., Fujita, K\., and Kashimori, Y\. \(2018\)\.Visual and category representations shaped by the interaction between inferior temporal and prefrontal cortices\.Cognitive Computation, 10:687 – 702\.
- Aberdam et al\., \(2019\)Aberdam, A\., Sulam, J\., and Elad, M\. \(2019\)\.Multi\-layer sparse coding: The holistic way\.SIAM Journal on Mathematics of Data Science, 1\(1\):46–77\.
- Ablin et al\., \(2019\)Ablin, P\., Moreau, T\., Massias, M\., and Gramfort, A\. \(2019\)\.Learning step sizes for unfolded sparse coding\.InAdvances in Neural Information Processing Systems 32 \(NeurIPS 2019\), pages 13100–13110\.
- Arbelaez et al\., \(2011\)Arbelaez, P\., Maire, M\., Fowlkes, C\., and Malik, J\. \(2011\)\.Contour detection and hierarchical image segmentation\.IEEE Trans\. Pattern Anal\. Mach\. Intell\., 33\(5\):898–916\.
- Bastos et al\., \(2012\)Bastos, A\., Usrey, W\., Adams, R\., Mangun, G\., Fries, P\., and Friston, K\. \(2012\)\.Canonical microcircuits for predictive coding\.Neuron, 76\(4\):695–711\.
- \(6\)Beck, A\. and Teboulle, M\. \(2009a\)\.Fast gradient\-based algorithms for constrained total variation image denoising and deblurring problems\.IEEE Transactions on Image Processing, 18\(11\):2419–2434\.
- \(7\)Beck, A\. and Teboulle, M\. \(2009b\)\.A fast iterative shrinkage\-thresholding algorithm for linear inverse problems\.SIAM Journal on Imaging Sciences, 2\(1\):183–202\.
- Beck and Tetruashvili, \(2013\)Beck, A\. and Tetruashvili, L\. \(2013\)\.On the convergence of block coordinate descent type methods\.SIAM Journal on Optimization, 23\(4\):2037–2060\.
- Boutin et al\., \(2021\)Boutin, V\., Franciosini, A\., Chavane, F\., Ruffier, F\., and Perrinet, L\. \(2021\)\.Sparse deep predictive coding captures contour integration capabilities of the early visual system\.PLOS Computational Biology, 17\(1\):1–31\.
- Bristow et al\., \(2013\)Bristow, H\., Eriksson, A\., and Lucey, S\. \(2013\)\.Fast convolutional sparse coding\.InProceedings of the IEEE Conference on Computer Vision and Pattern Recognition \(CVPR\), pages 391–398\.
- Daubechies et al\., \(2004\)Daubechies, I\., Defrise, M\., and De Mol, C\. \(2004\)\.An iterative thresholding algorithm for linear inverse problems with a sparsity constraint\.Communications on Pure and Applied Mathematics, 57\(11\):1413–1457\.
- Demmel, \(1997\)Demmel, J\. W\. \(1997\)\.Applied Numerical Linear Algebra\.Society for Industrial and Applied Mathematics\.
- Friston, \(2005\)Friston, K\. J\. \(2005\)\.A theory of cortical responses\.Philosophical Transactions of the Royal Society B: Biological Sciences, 360:815 – 836\.
- Friston and Kiebel, \(2009\)Friston, K\. J\. and Kiebel, S\. J\. \(2009\)\.Predictive coding under the free\-energy principle\.Philosophical Transactions of the Royal Society B: Biological Sciences, 364:1211 – 1221\.
- Golub and van Loan, \(2013\)Golub, G\. H\. and van Loan, C\. F\. \(2013\)\.Matrix Computations\.JHU Press, fourth edition\.
- Gregor and LeCun, \(2010\)Gregor, K\. and LeCun, Y\. \(2010\)\.Learning fast approximations of sparse coding\.InProceedings of the 27th International Conference on International Conference on Machine Learning, ICML’10, page 399–406, Madison, WI, USA\. Omnipress\.
- Kamiyama et al\., \(2016\)Kamiyama, A\., Fujita, K\., and Kashimori, Y\. \(2016\)\.A neural mechanism of dynamic gating of task\-relevant information by top\-down influence in primary visual cortex\.Biosystems, 150:138–148\.
- Kashimori et al\., \(2007\)Kashimori, Y\., Ichinose, Y\., and Fujita, K\. \(2007\)\.A functional role of interaction between it cortex and pf cortex in visual categorization task\.Neurocomputing, 70\(10\):1813–1818\.Computational Neuroscience: Trends in Research 2007\.
- Liu et al\., \(2019\)Liu, J\., Chen, X\., Wang, Z\., and Yin, W\. \(2019\)\.Alista: Analytic weights are as good as learned weights in lista\.InInternational Conference on Learning Representations\.
- Mairal et al\., \(2010\)Mairal, J\., Bach, F\., Ponce, J\., and Sapiro, G\. \(2010\)\.Online learning for matrix factorization and sparse coding\.J\. Mach\. Learn\. Res\., 11:19–60\.
- Nesterov, \(2012\)Nesterov, Y\. \(2012\)\.Efficiency of coordinate descent methods on huge\-scale optimization problems\.SIAM Journal on Optimization, 22\(2\):341–362\.
- Olshausen and Field, \(1996\)Olshausen, B\. A\. and Field, D\. J\. \(1996\)\.Emergence of simple\-cell receptive field properties by learning a sparse code for natural images\.Nature, 381:607–609\.
- Parikh and Boyd, \(2014\)Parikh, N\. and Boyd, S\. \(2014\)\.Proximal algorithms\.Found\. Trends Optim\., 1\(3\):127–239\.
- Rao and Ballard, \(1999\)Rao, R\. and Ballard, D\. \(1999\)\.Predictive coding in the visual cortex: a functional interpretation of some extra\-classical receptive\-field effects\.Nature neuroscience, 2:79–87\.
- Richtárik and Takác̆, \(2014\)Richtárik, P\. and Takác̆, M\. \(2014\)\.Iteration complexity of randomized block\-coordinate descent methods for minimizing a composite function\.Math\. Program\., 144\(1–2\):1–38\.
- Rosenbaum, \(2022\)Rosenbaum, R\. \(2022\)\.On the relationship between predictive coding and backpropagation\.PLOS ONE, 17\(3\):1–27\.
- Rozell et al\., \(2008\)Rozell, C\. J\., Johnson, D\. H\., Baraniuk, R\. G\., and Olshausen, B\. A\. \(2008\)\.Sparse coding via thresholding and local competition in neural circuits\.Neural Comput\., 20\(10\):2526–2563\.
- Tschantz et al\., \(2023\)Tschantz, A\., Millidge, B\., Seth, A\. K\., and Buckley, C\. L\. \(2023\)\.Hybrid predictive coding: Inferring, fast and slow\.PLOS Computational Biology, 19\(8\):1–31\.
- Whittington and Bogacz, \(2017\)Whittington, J\. C\. R\. and Bogacz, R\. \(2017\)\.An approximation of the error backpropagation algorithm in a predictive coding network with local hebbian synaptic plasticity\.Neural Computation, 29\(5\):1229–1262\.
- Xu, \(2018\)Xu, Y\. \(2018\)\.Hybrid jacobian and gauss–seidel proximal block coordinate update methods for linearly constrained convex programming\.SIAM Journal on Optimization, 28\(1\):646–670\.

## Appendix AMFISTA\-style accelerated baseline for hierarchical sparse coding

MFISTA is included only as an accelerated iterative reference\. It uses the same hierarchical sparse energy and the same proximal shrinkage operation as the ISTA\-style baseline, but differs in two implementation details\. First, the gradient step is taken from an extrapolated latent code rather than from the currently accepted latent code\. Second, the resulting joint latent\-code candidate is accepted only if it lowers the full energy\. This acceptance rule applies to the inferred codes, not to the dictionary parameters\.

Let𝒂¯\(t\)=\(𝒂¯1\(t\),…,𝒂¯L\(t\)\)\\bar\{\\bm\{a\}\}^\{\(t\)\}=\(\\bar\{\\bm\{a\}\}\_\{1\}^\{\(t\)\},\\dots,\\bar\{\\bm\{a\}\}\_\{L\}^\{\(t\)\}\)denote the accepted latent codes at iterationtt, let𝒚\(t\)=\(𝒚1\(t\),…,𝒚L\(t\)\)\\bm\{y\}^\{\(t\)\}=\(\\bm\{y\}\_\{1\}^\{\(t\)\},\\dots,\\bm\{y\}\_\{L\}^\{\(t\)\}\)denote the extrapolated codes, and letsts\_\{t\}be the FISTA momentum scalar\. We initialize the momentum ass0=1s\_\{0\}=1and set the first extrapolated code equal to the initial accepted code,𝒚\(0\)=𝒂¯\(0\)\\bm\{y\}^\{\(0\)\}=\\bar\{\\bm\{a\}\}^\{\(0\)\}\. We first compute the same layerwise ISTA step\-size estimates\(\{ηℓ\}ℓ=1L\)\(\\\{\\eta\_\{\\ell\}\\\}\_\{\\ell=1\}^\{L\}\)as in Eq\. \([18](https://arxiv.org/html/2606.27802#S3.E18)\), using the current dictionaries, coupling weights, andηscale\\eta\_\{\\mathrm\{scale\}\}\. For the accelerated baseline, we then collapse these layerwise estimates to a single fixed step size shared across all latent blocks:

η=minℓ⁡ηℓ,θℓ=η​λℓ\.\\eta=\\min\_\{\\ell\}\\eta\_\{\\ell\},\\qquad\\theta\_\{\\ell\}=\\eta\\lambda\_\{\\ell\}\.\(40\)This choice is conservative relative to the layerwise ISTA updates, although it should not be interpreted as an exact global Lipschitz step\-size guarantee for the fully coupled joint latent objective\.

At each iteration, MFISTA forms the same kind of soft\-thresholded proximal candidate as ISTA, but at the extrapolated point:

𝒛ℓ\(t\)=𝒮θℓ​\(𝒚ℓ\(t\)−η​∇𝒂ℓf​\(𝒚\(t\)\)\),ℓ=1,…,L\.\\bm\{z\}\_\{\\ell\}^\{\(t\)\}=\\mathcal\{S\}\_\{\\theta\_\{\\ell\}\}\\left\(\\bm\{y\}\_\{\\ell\}^\{\(t\)\}\-\\eta\\nabla\_\{\\bm\{a\}\_\{\\ell\}\}f\(\\bm\{y\}^\{\(t\)\}\)\\right\),\\qquad\\ell=1,\\ldots,L\.\(41\)Here,∇𝒂ℓf​\(𝒚\(t\)\)\\nabla\_\{\\bm\{a\}\_\{\\ell\}\}f\(\\bm\{y\}^\{\(t\)\}\)denotes theℓ\\ell\-th block gradient of the same smooth energy used in ISTA, evaluated at the full extrapolated code𝒚\(t\)\\bm\{y\}^\{\(t\)\}\. Equivalently, it is obtained from the appropriate layerwise expression in Eqs\. \([7](https://arxiv.org/html/2606.27802#S3.E7)\)–\([9](https://arxiv.org/html/2606.27802#S3.E9)\), according to whetherℓ=1\\ell=1,1<ℓ<L1<\\ell<L, orℓ=L\\ell=L, after substituting𝒂j=𝒚j\(t\)\\bm\{a\}\_\{j\}=\\bm\{y\}\_\{j\}^\{\(t\)\}for all latent blocks\. The full candidate vector𝒛\(t\)=\(𝒛1\(t\),…,𝒛L\(t\)\)\\bm\{z\}^\{\(t\)\}=\(\\bm\{z\}\_\{1\}^\{\(t\)\},\\dots,\\bm\{z\}\_\{L\}^\{\(t\)\}\)is then compared with the previously accepted joint latent code:

𝒂¯\(t\+1\)=arg⁡min𝒖∈\{𝒂¯\(t\),𝒛\(t\)\}⁡E​\(𝒖\),\\bar\{\\bm\{a\}\}^\{\(t\+1\)\}=\\arg\\min\_\{\\bm\{u\}\\in\\\{\\bar\{\\bm\{a\}\}^\{\(t\)\},\\bm\{z\}^\{\(t\)\}\\\}\}E\(\\bm\{u\}\),\(42\)so the accepted energy is monotone non\-increasing over MFISTA inference iterations\. The momentum scalar is updated by

st\+1=1\+1\+4​st22\.s\_\{t\+1\}=\\frac\{1\+\\sqrt\{1\+4s\_\{t\}^\{2\}\}\}\{2\}\.\(43\)The next extrapolated latent code is formed layerwise via

𝒚ℓ\(t\+1\)=𝒂¯ℓ\(t\+1\)\+stst\+1​\(𝒛ℓ\(t\)−𝒂¯ℓ\(t\+1\)\)\+st−1st\+1​\(𝒂¯ℓ\(t\+1\)−𝒂¯ℓ\(t\)\),ℓ=1,…,L\.\\bm\{y\}\_\{\\ell\}^\{\(t\+1\)\}=\\bar\{\\bm\{a\}\}\_\{\\ell\}^\{\(t\+1\)\}\+\\frac\{s\_\{t\}\}\{s\_\{t\+1\}\}\\left\(\\bm\{z\}\_\{\\ell\}^\{\(t\)\}\-\\bar\{\\bm\{a\}\}\_\{\\ell\}^\{\(t\+1\)\}\\right\)\+\\frac\{s\_\{t\}\-1\}\{s\_\{t\+1\}\}\\left\(\\bar\{\\bm\{a\}\}\_\{\\ell\}^\{\(t\+1\)\}\-\\bar\{\\bm\{a\}\}\_\{\\ell\}^\{\(t\)\}\\right\),\\qquad\\ell=1,\\dots,L\.\(44\)
This baseline is introduced for empirical comparison, not as a claim that the coupled hierarchical objective inherits the full guarantees of classical convex MFISTA\. Its role is to provide an accelerated iterative reference under the same energy and inference\-budget protocol\. Hybrid instead uses ISTA\-style refinement, which aligns more directly with the ISTA\-inspired shrinkage structure of the LISTA\-style amortized encoder\.

## Appendix BSupplementary Results for the Main Experiments

This appendix section provides supplementary results for the main experimental scope: method\-specific inference\-budget sweeps and auxiliary variant analyses\. All comparisons below use the same objective, architecture family, and training protocol as the main text\. The appendix\-only BSDS500 Patch check is displayed in Section[C](https://arxiv.org/html/2606.27802#A3)\.

### B\.1Training\-time convergence in epoch space

Figure[7](https://arxiv.org/html/2606.27802#A2.F7)shows the training\-time evolution of validation loss and reconstruction error over the 25\-epoch schedule used in the main experiments\. This check is intended to assess whether the shared schedule is long enough to expose the method\-wise differences used in the reported comparisons, rather than to claim full optimization convergence\.

The curves show that all methods improve rapidly in the early epochs and are comparatively flat by the end of training\. Thus, the 25\-epoch schedule appears adequate for comparing methods under this fixed protocol\. The remaining separation among methods is already visible late in training: LISTA remains the fastest amortized baseline but does not close the quality gap entirely, while Hybrid moves closer to the iterative reference regime after refinement\.

![Refer to caption](https://arxiv.org/html/2606.27802v1/x7.png)Figure 7:Training\-time convergence curves across optimization methods\. The panels show validation loss and reconstruction error as functions of training epoch under the shared core setting\. Validation loss is the full hierarchical sparse energy, whereas reconstruction error denotes only the input\-space reconstruction term\.
### B\.2Additional ablation: Hybrid\-MFISTA refinement

The proposed Hybrid method uses ISTA\-style refinement\. As an additional ablation, we also tested Hybrid\-MFISTA, which replaces only the refinement stage with the MFISTA procedure in Appendix[A](https://arxiv.org/html/2606.27802#A1)\. The amortized initialization, encoder\-gradient convention, and training protocol are unchanged\.

In the full set of method\-wise epoch curves in Fig\.[7](https://arxiv.org/html/2606.27802#A2.F7), the Hybrid\-MFISTA curves track the original Hybrid method closely over training epochs\. Table[4](https://arxiv.org/html/2606.27802#A2.T4)shows that Hybrid\-MFISTA is sometimes competitive but does not consistently outperform Hybrid\. MFISTA\-style refinement adds latency without a clear objective\-quality gain\. This ablation supports using the simpler ISTA\-style refinement rule in the main Hybrid method\.

Table 4:Appendix additional ablation comparing the main Hybrid method with Hybrid\-MFISTA refinement\.

## Appendix CAppendix\-only dataset experiments

The main text is intentionally limited to three datasets\. Here we add BSDS500 Patch as an*appendix\-only*natural\-image patch check\. This experiment broadens scope and connects the comparison to the classical natural\-image sparse\-coding setting, rather than adding a new core claim\. The implementation samples16×1616\\times 16patches from the official BSDS500 train, validation, and test image directories, using 54000, 6000, and 10000 patches for the three splits, respectively\.

![Refer to caption](https://arxiv.org/html/2606.27802v1/x8.png)Figure 8:BSDS500 Patch appendix check\. Left column: validation loss and reconstruction error over training epochs\. Right column: test loss and reconstruction error versus inference budget\. For Hybrid in the budget panels, the amortized depth is fixed atK=1K=1, matching the main quality\-versus\-budget comparison\.Figure[8](https://arxiv.org/html/2606.27802#A3.F8)checks two points\. First, the epoch curves rule out an obvious training\-schedule mismatch: the methods improve rapidly and flatten within the common 25\-epoch schedule\. Second, the budget panels ask whether the main quality\-versus\-inference\-budget comparison remains recognizable on natural\-image patches\. The BSDS500 Patch result is more nuanced than the main grayscale benchmarks, but the same broad trade\-off remains visible: pure amortization is cheapest, stronger iterative inference can improve final quality, and short Hybrid refinement occupies an intermediate operating regime\.