VisionHOPE: Visual Backbones as Self-Modifying Learning Systems

Hugging Face Daily Papers Papers
self-modifying-systems visual-backbone academic-research computer-vision neural-networks

Summary

VisionHOPE introduces a visual backbone that operates as a self-modifying learning system, where memory and learning rules co-evolve during image processing, achieving competitive results on benchmarks like ImageNet-1K, COCO, and ADE20K.

Visual backbones have evolved from Convolutional Neural Networks (CNNs) with local aggregation to Vision Transformers (ViTs) with global interactions, State-Space Models (SSMs) with input-dependent state transitions, and Test-Time Training (TTT) layers that adapt an inner learner while processing an image. Across this progression, visual computation has become increasingly adaptive to each input, yet the rules governing that adaptation remain largely prescribed by the trained backbone. We introduce VisionHOPE, the first generic visual backbone formulated as a self-modifying learning system, in which what the model remembers and how it learns co-evolve within an image. Building on the self-referential construction of Nested Learning (NL), VisionHOPE realizes this co-evolution through five coupled memories that store content, generate key and value representations, and govern learning rate and retention. These memories evolve jointly as visual context accumulates along each scan. However, directly applying the unconstrained self-referential update to a visual backbone leads to instability. We therefore derive a stability-matched step-size control scheme that combines a soft cap on self-referential injection with a spectral clamp on the retained memory transition, and prove that the resulting memory dynamics are non-expansive along each scan. For two-dimensional feature maps, we adapt NL's chunk formulation by aligning chunks with image rows and columns across four directional scans. The proposed VisionHOPE achieves competitive results on ImageNet-1K, COCO, and ADE20K, establishing self-modifying learning systems as a practical foundation for general-purpose visual backbones. The code is available at https://github.com/PSRben/VisionHOPE.
Original Article
View Cached Full Text

Cached at: 09/29/26, 08:12 PM

Paper page - VisionHOPE: Visual Backbones as Self-Modifying Learning Systems

Source: https://huggingface.co/papers/2609.33325

Abstract

VisualbackboneshaveevolvedfromConvolutionalNeuralNetworks(CNNs)withlocalaggregationtoVisionTransformers(ViTs)withglobalinteractions,State-SpaceModels(SSMs)withinput-dependentstatetransitions,andTest-TimeTraining(TTT)layersthatadaptaninnerlearnerwhileprocessinganimage.Acrossthisprogression,visualcomputationhasbecomeincreasinglyadaptivetoeachinput,yettherulesgoverningthatadaptationremainlargelyprescribedbythetrainedbackbone.WeintroduceVisionHOPE,thefirstgenericvisualbackboneformulatedasaself-modifyinglearningsystem,inwhichwhatthemodelremembersandhowitlearnsco-evolvewithinanimage.Buildingontheself-referentialconstructionofNestedLearning(NL),VisionHOPErealizesthisco-evolutionthroughfivecoupledmemoriesthatstorecontent,generatekeyandvaluerepresentations,andgovernlearningrateandretention.Thesememoriesevolvejointlyasvisualcontextaccumulatesalongeachscan.However,directlyapplyingtheunconstrainedself-referentialupdatetoavisualbackboneleadstoinstability.Wethereforederiveastability-matchedstep-sizecontrolschemethatcombinesasoftcaponself-referentialinjectionwithaspectralclampontheretainedmemorytransition,andprovethattheresultingmemorydynamicsarenon-expansivealongeachscan.Fortwo-dimensionalfeaturemaps,weadaptNL’schunkformulationbyaligningchunkswithimagerowsandcolumnsacrossfourdirectionalscans.TheproposedVisionHOPEachievescompetitiveresultsonImageNet-1K,COCO,andADE20K,establishingself-modifyinglearningsystemsasapracticalfoundationforgeneral-purposevisualbackbones.Thecodeisavailableathttps://github.com/PSRben/VisionHOPE.

View arXiv pageView PDFGitHub59Add to collection

Get this paper in your agent:

hf papers read 2609\.33325

Don’t have the latest CLI?curl \-LsSf https://hf\.co/cli/install\.sh \| bash

Models citing this paper1

#### PSRben/VisionHOPE Image Classification• Updatedabout 4 hours ago • 135

Datasets citing this paper0

No dataset linking this paper

Cite arxiv.org/abs/2609.33325 in a dataset README.md to link it from this page.

Spaces citing this paper0

No Space linking this paper

Cite arxiv.org/abs/2609.33325 in a Space README.md to link it from this page.

Collections including this paper0

No Collection including this paper

Add this paper to acollectionto link it from this page.

Similar Articles

PSRben/VisionHOPE

Hugging Face Models Trending

PSRben released official pretrained VisionHOPE-T/S/B visual backbone checkpoints on Hugging Face, covering ImageNet-1K classification, COCO detection/segmentation, and ADE20K semantic segmentation for the paper 'VisionHOPE: Visual Backbones as Self-Modifying Learning Systems'.

LoopVL: Recurrent Visual Intelligence

Hugging Face Daily Papers

LoopVL extends Loop Transformers to vision-language models, iteratively updating a unified multimodal state through shared modules (Module-Loop and Model-Loop) and outperforming similarly sized non-recurrent models on understanding and reasoning benchmarks, with notable 'Visual Aha Moments' in attention.

AtlasVA: Self-Evolving Visual Skill Memory for Teacher-Free VLM Agents

Hugging Face Daily Papers

AtlasVA is a teacher-free visual skill memory framework for vision-language model agents that uses spatial heatmaps, visual exemplars, and symbolic text skills to improve spatial decision-making in long-horizon tasks, outperforming baselines on several benchmarks.

Stateful Visual Encoders for Vision-Language Models

Hugging Face Daily Papers

This paper introduces a stateful visual encoder for vision-language models that conditions visual representations on prior features, enabling better visual comparison in multi-image and agentic settings. The method shows consistent improvements across tasks such as cross-image spatial aggregation and longitudinal radiology.