神经理想与神经编码：用于神经网络分类和特征解释的代数框架

# An Algebraic Framework for Neural Network Classification and Feature Interpretation
Source: [https://arxiv.org/html/2609.30279](https://arxiv.org/html/2609.30279)
## Neural Ideals and Neural Codes: An Algebraic Framework for Neural Network Classification and Feature InterpretationThanks:The first author would like to thank the Department of Education, Government of India, for the financial assistance\.

Venkata Subbaiah YerrapatiAddress:\(Yerrapati\) Department of Mathematics, S\. V\. National Institute of Technology, Surat\-7, Gujarat, IndiaEmail address:[yvsmath@gmail\.com](mailto:[email protected])Rahul DixitAddress:\(Dixit\) Department of Artificial Intelligence, S\. V\. National Institute of Technology, Surat\-7, Gujarat, IndiaEmail address:[rahuldixit@aid\.svnit\.ac\.in](mailto:[email protected])andAjay Kumar ShuklaAddress:\(Shukla\) Department of Mathematics, S\. V\. National Institute of Technology, Surat\-7, Gujarat, IndiaEmail address:[aks@amhd\.svnit\.ac\.in](mailto:[email protected])

Date: August 20, 2026

###### Abstract\.

Understanding the features captured by the hidden layers of neural networks is a fundamental challenge in machine learning, despite their widespread success across various classification problems\. In this work, we propose an algebraic framework for examining neural networks that model classification problems\. Certain results, such as the correspondence between the neural network and neural ideals, algorithms for computing the neural ideals, and a stabilization theorem that enables approximation of the neural ideals, are first established\. As an application to the framework, we present algorithms to identify and interpret the features captured by each hidden\-layer neuron\. Along with these theoretical developments, the practical performance has been demonstrated on the MNIST digit dataset, and the results highlight the pivotal role of neural ideals as a mathematical and computational tool for analyzing the features captured by neural networks\. Further, we develop an interactive software that builds on the presented framework to visualize the features captured by each neuron\. This tool is available at[https://github\.com/yvs1967/neural\-network\-representation\-explorer/](https://github.com/yvs1967/neural-network-representation-explorer/)

###### Key words and phrases:

Neural Ideals, Neural codes, Deep Learning Interpretability, Feature Interpretation

###### 2020 Mathematics Subject Classification

13P25, 68T07

## 1\.Introduction

Artificial neural networks\[[6](https://arxiv.org/html/2609.30279#bib.bib8)\]have emerged as one of the most successful computational paradigms for addressing complex classification problems arising in computer vision, natural language processing, scientific computing, and related areas\[[4](https://arxiv.org/html/2609.30279#bib.bib5)\]\. Despite their empirical success, the mathematical understanding of the internal representations learned by such models remains limited\[[9](https://arxiv.org/html/2609.30279#bib.bib6)\]\. The classification mechanism of a trained neural network is typically governed by compositions of affine transformations and nonlinear activation functions acting in high\-dimensional spaces\[[4](https://arxiv.org/html/2609.30279#bib.bib5)\]\. Although these architectures achieve remarkable predictive performance, the relationship between hidden\-layer activations and the final classification decision is often difficult to characterize mathematically\[[8](https://arxiv.org/html/2609.30279#bib.bib7)\]\. Consequently, the development of rigorous mathematical frameworks for analysing the internal structure and decision\-making process of neural networks has become an important research direction\[[9](https://arxiv.org/html/2609.30279#bib.bib6),[8](https://arxiv.org/html/2609.30279#bib.bib7)\]\.

One possible approach to this problem arises from the interaction of commutative algebra, algebraic geometry, and data science\. When an input is propagated through a feedforward neural network, the hidden neurons produce a pattern of activations that may be viewed as a binary firing configuration\. Similar binary activation patterns appear in mathematical neuroscience, where they are studied through the theory of combinatorial neural codes\. In this setting, Curto*et al\.*\[[3](https://arxiv.org/html/2609.30279#bib.bib1)\]introduced neural rings and neural ideals as algebraic objects associated with neural codes\. Given a neural codeC⊆𝔽2mC\\subseteq\\mathbb\{F\}\_\{2\}^\{m\}, the corresponding neural idealJCJ\_\{C\}encodes the combinatorial structure ofCCwithin a Boolean polynomial ring\. These algebraic constructions have proved useful in the study of biological neural systems and stimulus–response relationships\. However, their application to artificial neural networks and modern classification architectures has received comparatively little attention\.

The objective of the present work is to develop an algebraic framework that connects neural network classification with the theory of neural ideals\. We associate to each target class a collection of binary activation patterns obtained from the hidden neurons of the network and introduce the notion of an*allowed code space*\. This construction enables the translation of activation information into the Boolean quotient ring

R=𝔽2\[x1,…,xm\]/⟨xi2−xi\|1≤i≤m⟩\.R=\\mathbb\{F\}\_\{2\}\[x\_\{1\},\\ldots,x\_\{m\}\]/\\langle x\_\{i\}^\{2\}\-x\_\{i\}\\;\|\\;1\\leq i\\leq m\\rangle\.Using pseudo\-monomial generators, we construct class\-specific neural ideals whose algebraic structure reflects the admissible activation patterns associated with a given classification outcome\.

The proposed formulation establishes a correspondence between neural network classification and polynomial ideal theory\. In particular, the classification behaviour of a trained network is represented through Boolean polynomial relations determined by the associated neural ideal\. We further investigate the computational aspects of constructing these ideals from finite collections of training samples and establish a stabilization result showing that the resulting ideals become invariant after the inclusion of sufficiently many samples\.

The main contributions of this work are summarized as follows:

1. \(1\)We introduce the notion of an*allowed code space*associated with a target class of a trained feedforward neural network\.
2. \(2\)We adapt the theory of neural ideals to artificial neural network classification and construct class\-specific neural ideals from hidden\-layer activation patterns\.
3. \(3\)We establish a classification–ideal correspondence that characterizes class membership through the generators of the associated neural ideal\.
4. \(4\)We develop algorithms for constructing threshold maps, computing neural codes, determining allowed code spaces, and deriving neural ideals from neural network architectures\.
5. \(5\)We prove a stabilization theorem that guarantees the eventual invariance of the neural ideals obtained from sufficiently large collections of training samples\.
6. \(6\)We demonstrate the applicability of the proposed framework through the XOR classification problem and the MNIST handwritten digit dataset, illustrating its usefulness for analysing hidden representations and feature structures\.
7. \(7\)We develop an interactive software framework for neural\-code generation, allowed code\-space construction, neural\-ideal analysis, and visualization of learned neural representations\.

The remainder of the paper is organized as follows\. Section[2](https://arxiv.org/html/2609.30279#S2)reviews the necessary background on feedforward neural networks, Boolean polynomial rings, and neural ideals\. Section[3](https://arxiv.org/html/2609.30279#S3)contains the principal theoretical results together with the proposed algorithms\. Section[4](https://arxiv.org/html/2609.30279#S4)presents applications of the framework to the XOR classification problem and the MNIST handwritten digit dataset\.

## 2\.Preliminaries

In this section, we review the definitions and results required for later sections\. Basic concepts are taken from\[[2](https://arxiv.org/html/2609.30279#bib.bib2)\], while a detailed treatment of neural rings and neural ideals, including relevant theorems and proofs, can be found in\[[3](https://arxiv.org/html/2609.30279#bib.bib1)\]\.

###### Definition 2\.1\(Activation Function\)\.

Letnnbe a positive integer\. Any functionσ:ℝ→ℝ\\sigma:\\mathbb\{R\}\\to\\mathbb\{R\}is called an*activation function*and the associated*coordinate\-wise activation function*σn:ℝn→ℝn\\sigma^\{n\}:\\mathbb\{R\}^\{n\}\\to\\mathbb\{R\}^\{n\}is defined byσn​\(x1,x2,…,xn\)=\(σ⁡\(x1\),σ⁡\(x2\),…,σ⁡\(xn\)\)\\sigma^\{n\}\(x\_\{1\},x\_\{2\},\\ldots,x\_\{n\}\)=\\left\(\\sigma\(x\_\{1\}\),\\sigma\(x\_\{2\}\),\\ldots,\\sigma\(x\_\{n\}\)\\right\)\.

###### Definition 2\.2\(Feedforward Neural Network\)\.

Letn0,n1,…,nLn\_\{0\},n\_\{1\},\\ldots,n\_\{L\}be positive integers\. For eachi=1,2,…,Li=1,2,\\ldots,L, letAi:ℝni−1→ℝniA\_\{i\}:\\mathbb\{R\}^\{n\_\{i\-1\}\}\\to\\mathbb\{R\}^\{n\_\{i\}\}be an affine map defined byAi​\(x\)=Wi​x\+biA\_\{i\}\(x\)=W\_\{i\}x\+b\_\{i\}, whereWi∈ℝni×ni−1W\_\{i\}\\in\\mathbb\{R\}^\{n\_\{i\}\\times n\_\{i\-1\}\}is a weight matrix andbi∈ℝnib\_\{i\}\\in\\mathbb\{R\}^\{n\_\{i\}\}is a bias vector\. Letσi:ℝni→ℝni\\sigma\_\{i\}:\\mathbb\{R\}^\{n\_\{i\}\}\\to\\mathbb\{R\}^\{n\_\{i\}\}be a coordinate\-wise activation function\. A feedforward neural network of depthLLis the functionf:ℝn0→ℝnLf:\\mathbb\{R\}^\{n\_\{0\}\}\\to\\mathbb\{R\}^\{n\_\{L\}\}defined byf\(x\)=\(σL∘AL\)∘\(σL−1∘AL−1\)∘⋯∘\(σ1∘A1\)\(x\)f\(x\)=\(\\sigma\_\{L\}\\circ A\_\{L\}\)\\circ\(\\sigma\_\{L\-1\}\\circ A\_\{L\-1\}\)\\circ\\cdots\\circ\(\\sigma\_\{1\}\\circ A\_\{1\}\)\(x\), for allx∈ℝn0x\\in\\mathbb\{R\}^\{n\_\{0\}\}\.

###### Definition 2\.3\(Boolean Polynomial Ring\)\.

Let𝔽2\\mathbb\{F\}\_\{2\}denote the field of two elements and𝔽2​\[x1,…,xn\]\\mathbb\{F\}\_\{2\}\[x\_\{1\},\\ldots,x\_\{n\}\]be the polynomial ring over the field𝔽2\\mathbb\{F\}\_\{2\}\. The*Boolean polynomial ring*is the quotient ring𝔽2​\[x1,…,xn\]⟨xi2−xi∣fori=1,2,…,n⟩\\frac\{\\mathbb\{F\}\_\{2\}\[x\_\{1\},\\ldots,x\_\{n\}\]\}\{\\langle x\_\{i\}^\{2\}\-x\_\{i\}\\mid\\text\{ for \}i=1,2,\\ldots,n\\rangle\}\.

###### Definition 2\.4\(Neural Code\)\.

A*neural code*CCis a subset of𝔽2m\\mathbb\{F\}\_\{2\}^\{m\}and a*codeword*is any element in𝔽2m\\mathbb\{F\}\_\{2\}^\{m\}\.

###### Definition 2\.5\(Pseudo\-monomial\)\.

A polynomialf∈𝔽2​\[x1,x2,…,xm\]f\\in\\mathbb\{F\}\_\{2\}\[x\_\{1\},x\_\{2\},\\ldots,x\_\{m\}\]is called a*pseudo\-monomial*iff=∏i∈σxi​∏j∈τ\(1−xj\)f=\\prod\_\{i\\in\\sigma\}x\_\{i\}\\prod\_\{j\\in\\tau\}\(1\-x\_\{j\}\), whereσ,τ⊆\{1,2,…,m\}\\sigma,\\tau\\subseteq\\\{1,2,\\ldots,m\\\}such thatσ∩τ=∅\\sigma\\cap\\tau=\\emptyset\.

###### Definition 2\.6\(Characteristic Polynomial\)\.

Letv=\(v1,…,vm\)∈\{0,1\}mv=\(v\_\{1\},\\ldots,v\_\{m\}\)\\in\\\{0,1\\\}^\{m\}be a binary vector\. The*characteristic polynomial*ofvvis the pseudo\-monomialρv∈𝔽2​\[x1,…,xm\]\\rho\_\{v\}\\in\\mathbb\{F\}\_\{2\}\[x\_\{1\},\\ldots,x\_\{m\}\]defined byρv​\(x\)=∏vi=1xi​∏vj=0\(1−xj\)\\rho\_\{v\}\(x\)=\\prod\_\{v\_\{i\}=1\}x\_\{i\}\\prod\_\{v\_\{j\}=0\}\(1\-x\_\{j\}\)\.

###### Definition 2\.7\(Neural Ideal\)\.

LetC⊆\{0,1\}mC\\subseteq\\\{0,1\\\}^\{m\}be a neural code\. The*neural ideal*associated withCCis the ideal

JC=⟨ρv∣v∉C⟩\+⟨xi2−xi∣i=1,…,m⟩⊆𝔽2\[x1,…,xm\],J\_\{C\}=\\langle\\rho\_\{v\}\\mid v\\notin C\\rangle\+\\langle x\_\{i\}^\{2\}\-x\_\{i\}\\mid i=1,\\ldots,m\\rangle\\subseteq\\mathbb\{F\}\_\{2\}\[x\_\{1\},\\ldots,x\_\{m\}\],where, for eachv=\(v1,…,vm\)∈\{0,1\}mv=\(v\_\{1\},\\ldots,v\_\{m\}\)\\in\\\{0,1\\\}^\{m\},ρv=∏vi=1xi​∏vj=0\(1−xj\)\\rho\_\{v\}=\\prod\_\{v\_\{i\}=1\}x\_\{i\}\\prod\_\{v\_\{j\}=0\}\(1\-x\_\{j\}\)is the characteristic polynomial ofvv\.

###### Lemma 2\.8\(Chapter 8\[[1](https://arxiv.org/html/2609.30279#bib.bib3)\]\)\.

Let𝕂\\mathbb\{K\}be a finite field andR=𝕂\[x1,…,xm\]/⟨xi2−xi∣i=1,…,m⟩R=\\mathbb\{K\}\[x\_\{1\},\\ldots,x\_\{m\}\]\\Big/\\left\\langle x\_\{i\}^\{2\}\-x\_\{i\}\\mid i=1,\\ldots,m\\right\\rangle\. IfI1⊇I2⊇I3⊇⋯I\_\{1\}\\supseteq I\_\{2\}\\supseteq I\_\{3\}\\supseteq\\cdotsis a descending chain of ideals inRR, then there exists an integern0n\_\{0\}such thatIn=In0I\_\{n\}=I\_\{n\_\{0\}\}, for alln≥n0n\\geq n\_\{0\}\.

## 3\.Main Results

###### Definition 3\.1\(Allowed code space\)\.

Let𝒩\\mathcal\{N\}be a neural network defined on an input space𝒳⊆ℝd\\mathcal\{X\}\\subseteq\\mathbb\{R\}^\{d\}that classifies the inputs intokk\-discrete classes\. Lety∈\{1,2,…,k\}y\\in\\\{1,2,\\ldots,k\\\}denote a specific target class andΦ𝒩:ℝd→𝔽2m\\Phi\_\{\\mathcal\{N\}\}:\\mathbb\{R\}^\{d\}\\to\\mathbb\{F\}\_\{2\}^\{m\}denote the threshold map associated with themmneurons of𝒩\\mathcal\{N\}\. The set

\(1\)C\(y\)=\{𝐱∈𝔽2m∣Φ𝒩\(𝐯\)=𝐱,for some𝐯∈𝒳\}C\(y\)=\\\{\\mathbf\{x\}\\in\\mathbb\{F\}\_\{2\}^\{m\}\\mid\\Phi\_\{\\mathcal\{N\}\}\(\\mathbf\{v\}\)=\\mathbf\{x\},\\text\{ for some \}\\mathbf\{v\}\\in\\mathcal\{X\}\\\}is theallowed code spaceof𝒩\\mathcal\{N\}corresponding to the classyy\.

###### Theorem 3\.2\.

Let𝒩:ℝd→ℝk\\mathcal\{N\}:\\mathbb\{R\}^\{d\}\\to\\mathbb\{R\}^\{k\}be a trained artificial feedforward neural network defined to classify intokkdiscrete classes\. Letyybe a class among thekkdiscrete classes andJ𝒩​\(y\)\\mathrm\{J\}\_\{\\mathcal\{N\}\}\(y\)be the neural ideal in the quotient ring𝔽2\[x1,x2,…,xm\]/⟨xi2−xi∣1≤i≤m⟩\\mathbb\{F\}\_\{2\}\[x\_\{1\},x\_\{2\},\\ldots,x\_\{m\}\]/\\langle x\_\{i\}^\{2\}\-x\_\{i\}\\mid 1\\leq i\\leq m\\rangle, wheremmis number of neurons in𝒩\\mathcal\{N\}\. LetGy=\{g1,g2,…,gl\}G\_\{y\}=\\\{g\_\{1\},g\_\{2\},\\ldots,g\_\{l\}\\\}be the generating set ofJ𝒩⁡\(y\)\\mathrm\{J\}\_\{\\mathcal\{N\}\(y\)\}\. Furthermore, letΦ:ℝd→𝔽2m\\Phi:\\mathbb\{R\}^\{d\}\\to\\mathbb\{F\}\_\{2\}^\{m\}be the function that maps the input vectorv∈ℝdv\\in\\mathbb\{R\}^\{d\}to a code wordx∈𝔽2mx\\in\\mathbb\{F\}\_\{2\}^\{m\}\. Given an input vectorv0∈ℝdv\_\{0\}\\in\\mathbb\{R\}^\{d\}andx0x\_\{0\}be its corresponding code word, then𝒩⁡\(v0\)=y\\mathcal\{N\}\(v\_\{0\}\)=yif and only ifg⁡\(x0\)=0g\(x\_\{0\}\)=0, for allg∈Gyg\\in G\_\{y\}\.

###### Proof\.

Letv0∈ℝdv\_\{0\}\\in\\mathbb\{R\}^\{d\}andx0x\_\{0\}be its corresponding neural code obtained by the threshold mapΦ\\Phi\. Due to definition of the allowed code space, we havex0∈C⁡\(y\)x\_\{0\}\\in C\(y\)\. The neural idealJ𝒩​\(y\)\\mathrm\{J\}\_\{\\mathcal\{N\}\}\(y\)contains all the polynomialf∈Rf\\in\\mathrm\{R\}such thatf⁡\(x\)=0f\(x\)=0, for allx∈C⁡\(y\)x\\in C\(y\)\. SinceGy⊆J𝒩​\(y\)G\_\{y\}\\subseteq\\mathrm\{J\}\_\{\\mathcal\{N\}\}\(y\), it follows that for everyx∈C⁡\(y\)x\\in C\(y\),gi​\(x\)=0g\_\{i\}\(x\)=0, fori=1,2,…,li=1,2,\\ldots,l\. In particular forx=x0x=x\_\{0\}, we obtaingi​\(x0\)=0g\_\{i\}\(x\_\{0\}\)=0, fori=1,2,…,li=1,2,\\ldots,l\. To prove the converse, letv0∈ℝdv\_\{0\}\\in\\mathbb\{R\}^\{d\}be the input vector that maps to an elementx0x\_\{0\}of𝔽2m\\mathbb\{F\}\_\{2\}^\{m\}satisfying

\(2\)g⁡\(x0\)=0,∀g∈Gy\.g\(x\_\{0\}\)=0,\\ \\forall\\ g\\in G\_\{y\}\.Suppose that the neural network does not classify the targeted class i\.e\.𝒩⁡\(v0\)=z\\mathcal\{N\}\(v\_\{0\}\)=zwherez≠yz\\neq y\. By the definition of the allowed code space, we havex0∉C⁡\(y\)x\_\{0\}\\notin C\(y\)which implies thatx0∈C​\(y\)cx\_\{0\}\\in C\(y\)^\{c\}\. As the class specific neural idealJ𝒩​\(y\)\\mathrm\{J\}\_\{\\mathcal\{N\}\}\(y\)is generated by the set consists of unique pseudo monomialsmσm\_\{\\sigma\}for every forbidden stateσ∈C​\(y\)c\\sigma\\in C\(y\)^\{c\}and field equations, it implies that there exists a unique pseudo monomialmx0​\(x\)m\_\{x\_\{0\}\}\(x\)contained in the generating set ofJ𝒩​\(y\)\\mathrm\{J\}\_\{\\mathcal\{N\}\}\(y\)obtained byx0x\_\{0\}and evaluating atx=x0x=x\_\{0\}, we get

\(3\)mx0\(x0\)=\(Πl:xl=11\)\(Πo:xo=0\(1−0\)\)=1m\_\{x\_\{0\}\}\(x\_\{0\}\)=\\left\(\\Pi\_\{l:x\_\{l\}=1\}1\\right\)\\left\(\\Pi\_\{o:x\_\{o\}=0\}\(1\-0\)\\right\)=1mx0​\(x\)m\_\{x\_\{0\}\}\(x\)belongs to the idealJ𝒩​\(y\)\\mathrm\{J\}\_\{\\mathcal\{N\}\}\(y\)andGyG\_\{y\}is a generating set ofJ𝒩​\(y\)\\mathrm\{J\}\_\{\\mathcal\{N\}\}\(y\)which means thatmx0​\(x\)m\_\{x\_\{0\}\}\(x\)can be expressed as a linear combination of elements ofGyG\_\{y\}with polynomial coefficientshi​\(x\)∈Rh\_\{i\}\(x\)\\in\\mathrm\{R\}\.

\(4\)mx0​\(x\)=∑i=1lhi​\(x\)⋅gi​\(x\)m\_\{x\_\{0\}\}\(x\)=\\sum\_\{i=1\}^\{l\}h\_\{i\}\(x\)\\cdot g\_\{i\}\(x\)Substitutingx=x0x=x\_\{0\}in equation \([4](https://arxiv.org/html/2609.30279#S3.E4)\), we obtainmx0​\(x0\)=∑i=1lhi​\(x0\)⋅gi​\(x0\)m\_\{x\_\{0\}\}\(x\_\{0\}\)=\\sum\_\{i=1\}^\{l\}h\_\{i\}\(x\_\{0\}\)\\cdot g\_\{i\}\(x\_\{0\}\)\. By hypothesis statementgi​\(x0\)=0g\_\{i\}\(x\_\{0\}\)=0, fori=1,2,…,li=1,2,\\ldots,l, equation \([3](https://arxiv.org/html/2609.30279#S3.E3)\) and the equation \([4](https://arxiv.org/html/2609.30279#S3.E4)\) leads to a contradiction to the assumption thatx0∈C​\(y\)cx\_\{0\}\\in C\(y\)^\{c\}\. Therefore, our assumption is not correct andx0∈C⁡\(y\)x\_\{0\}\\in C\(y\)finishing the argument\. ∎

Theorem[3\.2](https://arxiv.org/html/2609.30279#S3.Thmtheorem2)implies that whenever a neural ideal corresponding to a specific class exists, its generating set characterizes whether a given input vector is classified into that class by the neural network\. However, we do not have a concrete algorithm for determining the neural ideal explicitly\. In an attempt to study this idea, we introduce few definitions and subsequently present an algorithm applicable to certain specific classes of neural networks\.

###### Definition 3\.3\(Threshold Map\)\.

Let𝒩\\mathcal\{N\}be a neural network defined on an input space𝒳⊆ℝd\\mathcal\{X\}\\subseteq\\mathbb\{R\}^\{d\}\. Athreshold mapassociated with the neural network𝒩\\mathcal\{N\}is a functionΦ𝒩:ℝd→𝔽2s\\Phi\_\{\\mathcal\{N\}\}:\\mathbb\{R\}^\{d\}\\to\\mathbb\{F\}\_\{2\}^\{s\}that maps each input vectorv∈ℝdv\\in\\mathbb\{R\}^\{d\}to a binary discrete vector in𝔽2s\\mathbb\{F\}\_\{2\}^\{s\}, wheressis the number of neurons in the neural network𝒩\\mathcal\{N\}\.

### 3\.1\.Proposed Algorithms

The only assumption required for the algorithm is that the neural network accepts binary inputs, namely \(0\) or \(1\)\.

#### 3\.1\.1\.Algorithm : Threshold Map of an Artificial Feedforward Neural Network

1. \(1\)Start with an artificial feedforward neural network𝒩\\mathcal\{N\}\.
2. \(2\)For a given layerLLwithssneurons, and for each neuron in this layer, obtain its bias valuebjb\_\{j\}, for1≤j≤s1\\leq j\\leq s\.
3. \(3\)Compute the threshold valuesθj\\theta\_\{j\}of the layer byθj=−bj\\theta\_\{j\}=\-b\_\{j\}\.
4. \(4\)Define the threshold mapΦ𝒩,L:ℝs→𝔽2s\\Phi\_\{\\mathcal\{N\},L\}:\\mathbb\{R\}^\{s\}\\to\\mathbb\{F\}\_\{2\}^\{s\}of the layerLLby the following coordinate map \(5\)Φ𝒩,L​\(v\)j=\{1if​∑kWj,k​vk≥θj0otherwise\\Phi\_\{\\mathcal\{N\},L\}\(v\)\_\{j\}=\\begin\{cases\}1&\\text\{if \}\\sum\_\{k\}W\_\{j,k\}v\_\{k\}\\geq\\theta\_\{j\}\\\\ 0&\\text\{otherwise\}\\end\{cases\}, for1≤j≤s1\\leq j\\leq s\.
5. \(5\)Concatenate the threshold maps of each layerΦ𝒩,L\\Phi\_\{\\mathcal\{N\},L\}to obtain the threshold mapΦ𝒩\\Phi\_\{\\mathcal\{N\}\}\.

#### 3\.1\.2\.Algorithm : Neural code of an input vector of an Artificial Feedforward Neural Network

1. \(1\)Start with an artificial feedforward neural network𝒩\\mathcal\{N\}\.
2. \(2\)Compute the threshold mapΦ\\Phiusing Algorithm[3\.1\.1](https://arxiv.org/html/2609.30279#S3.SS1.SSS1)\.
3. \(3\)For an input vectorvv, compute the output vector of each layerLL, denoted byoL​\(v\)o\_\{L\}\(v\)\.
4. \(4\)Concatenate the output vectors of each layer to obtain output vector \(6\)O⁡\(v\)=\[oL1​\(v\),oL2​\(v\),…,oLm​\(v\)\]T,O\(v\)=\[o\_\{L\_\{1\}\}\(v\),o\_\{L\_\{2\}\}\(v\),\\dots,o\_\{L\_\{m\}\}\(v\)\]^\{T\},wheremmdenotes the number of neurons in𝒩\\mathcal\{N\}\.
5. \(5\)Compute the neural code for the input vectorvvby evaluating the threshold map atO⁡\(v\)O\(v\)\.

#### 3\.1\.3\.Algorithm for Computing the Allowed Code Space of a Class

1. \(1\)Start with an artificial feedforward neural network𝒩\\mathcal\{N\}and a setΩ\\Omegaconsisting of all vectors from the training data whose target class isyy, whereyyis one of the classes that𝒩\\mathcal\{N\}classifies\.
2. \(2\)Compute the threshold mapΦ\\Phifor𝒩\\mathcal\{N\}using Algorithm[3\.1\.1](https://arxiv.org/html/2609.30279#S3.SS1.SSS1)\.
3. \(3\)For each input vectorv∈Ωv\\in\\Omega, compute its corresponding neural code by using Algorithm[3\.1\.2](https://arxiv.org/html/2609.30279#S3.SS1.SSS2)\.
4. \(4\)Collect all the computed neural codes into a single set\.
5. \(5\)Define this set as the allowed code space for classyy\.

For a given neural network and a target class, Algorithm[3\.1\.3](https://arxiv.org/html/2609.30279#S3.SS1.SSS3)is used to determine the corresponding allowed code space\. From the Algorithm in\[[3](https://arxiv.org/html/2609.30279#bib.bib1)\], we compute the canonical form of the neural ideal associated with the target class classified by the neural network\.

The computational complexity of the proposed method depends on two primary factors: the cardinality of the data points and the total number of neurons within the network architecture\. We focus exclusively on addressing the constraints of the first case to compute the corresponding neural ideals\. While the second case falls outside the scope of this study and is left for future investigation\.

###### Lemma 3\.5\.

Let𝒩:ℝd→ℝk\\mathcal\{N\}:\\mathbb\{R\}^\{d\}\\to\\mathbb\{R\}^\{k\}be a trained artificial feedforward neural network, and letΦ:ℝd→𝔽2m\\Phi:\\mathbb\{R\}^\{d\}\\to\\mathbb\{F\}\_\{2\}^\{m\}be its associated threshold map\. LetΩ⊆ℝd\\Omega\\subseteq\\mathbb\{R\}^\{d\}be the input space corresponding to a target classyy\. For any subset of inputsD⊆ΩD\\subseteq\\Omega, let𝒜⁡\(D\)\\mathcal\{A\}\(D\)andℱ⁡\(D\)\\mathcal\{F\}\(D\)denote the set of allowed and forbidden neural codes, respectively\. IfD1D\_\{1\}andD2D\_\{2\}are two subsets of inputs such thatD1⊆D2⊆ΩD\_\{1\}\\subseteq D\_\{2\}\\subseteq\\Omega, thenJ𝒩D2​\(y\)⊆J𝒩D1​\(y\)\\mathrm\{J\}\_\{\\mathcal\{N\}\_\{D\_\{2\}\}\}\(y\)\\subseteq\\mathrm\{J\}\_\{\\mathcal\{N\}\_\{D\_\{1\}\}\}\(y\)\.

###### Proof\.

LetD1⊆D2⊆ΩD\_\{1\}\\subseteq D\_\{2\}\\subseteq\\Omega\. By the definition of the allowed code space, we have𝒜⁡\(D1\)⊆𝒜⁡\(D2\)\\mathcal\{A\}\(D\_\{1\}\)\\subseteq\\mathcal\{A\}\(D\_\{2\}\)\. By taking the complements in𝔽2m\\mathbb\{F\}\_\{2\}^\{m\}to find the forbidden states, we obtainℱ⁡\(D2\)⊆ℱ⁡\(D1\)\\mathcal\{F\}\(D\_\{2\}\)\\subseteq\\mathcal\{F\}\(D\_\{1\}\)\. Sinceℱ⁡\(D2\)\\mathcal\{F\}\(D\_\{2\}\)is a subset ofℱ⁡\(D1\)\\mathcal\{F\}\(D\_\{1\}\), a polynomialffthat vanishes on all elements ofℱ⁡\(D1\)\\mathcal\{F\}\(D\_\{1\}\)must necessarily vanish on all elements ofℱ⁡\(D2\)\\mathcal\{F\}\(D\_\{2\}\)\. Therefore,J𝒩D2​\(y\)⊆J𝒩D1​\(y\)\\mathrm\{J\}\_\{\\mathcal\{N\}\_\{D\_\{2\}\}\}\(y\)\\subseteq\\mathrm\{J\}\_\{\\mathcal\{N\}\_\{D\_\{1\}\}\}\(y\)\. This completes our argument\. ∎

###### Theorem 3\.6\.

Let𝒩:ℝd→ℝk\\mathcal\{N\}:\\mathbb\{R\}^\{d\}\\to\\mathbb\{R\}^\{k\}be a trained artificial feedforward neural network defined to classify intokkdiscrete classes\. LetΦ:ℝd→𝔽2m\\Phi:\\mathbb\{R\}^\{d\}\\to\\mathbb\{F\}\_\{2\}^\{m\}be the function that maps the input vectorv∈ℝdv\\in\\mathbb\{R\}^\{d\}to a neural codex∈𝔽2mx\\in\\mathbb\{F\}\_\{2\}^\{m\}, wheremmis the number of neurons in𝒩\\mathcal\{N\}\. LetGy=\{g1,g2,…,gl\}G\_\{y\}=\\\{g\_\{1\},g\_\{2\},\\ldots,g\_\{l\}\\\}be the generating set ofJ𝒩⁡\(y\)\\mathrm\{J\}\_\{\\mathcal\{N\}\(y\)\}\. LetΩ⊆ℝd\\Omega\\subseteq\\mathbb\{R\}^\{d\}be the space of inputs of classyythat are used to train the neural network𝒩\\mathcal\{N\}containing very large amount of elements\. Furthermore, letyybe a class among thekkdiscrete classes andJ𝒩Ω​\(y\)\\mathrm\{J\}\_\{\\mathcal\{N\}\_\{\\Omega\}\}\(y\)be the neural ideal of𝒩\\mathcal\{N\}\. Then there exists a natural numberN0N\_\{0\}such that for everyΩ0⊆Ω\\Omega\_\{0\}\\subseteq\\Omegacontaining at leastN0N\_\{0\}elementsJ𝒩​\(y\)=J𝒩Ω0​\(y\)J\_\{\\mathcal\{N\}\}\(y\)=J\_\{\\mathcal\{N\}\_\{\\Omega\_\{0\}\}\}\(y\), whereJ𝒩Ω0​\(y\)J\_\{\\mathcal\{N\}\_\{\\Omega\_\{0\}\}\}\(y\)is the neural ideal of𝒩\\mathcal\{N\}constructed only with the elements ofΩ0\\Omega\_\{0\}\.

###### Proof\.

LetΔ1⊆Δ2⊆⋯⊆Ω\\Delta\_\{1\}\\subseteq\\Delta\_\{2\}\\subseteq\\cdots\\subseteq\\Omegabe a sequence of subsets, whereΔi\\Delta\_\{i\}consists ofiielements\. By Lemma[3\.5](https://arxiv.org/html/2609.30279#S3.Thmtheorem5), this nested sequence induces a nested descending chain of neural ideals inS=S=\.

\(7\)J𝒩Δ1​\(y\)⊇J𝒩Δ2​\(y\)⊇J𝒩Δ3​\(y\)⊇⋯⊇J𝒩Ω​\(y\)J\_\{\\mathcal\{N\}\_\{\\Delta\_\{1\}\}\(y\)\}\\supseteq J\_\{\\mathcal\{N\}\_\{\\Delta\_\{2\}\}\(y\)\}\\supseteq J\_\{\\mathcal\{N\}\_\{\\Delta\_\{3\}\}\(y\)\}\\supseteq\\cdots\\supseteq J\_\{\\mathcal\{N\}\_\{\\Omega\}\}\(y\)
Since𝔽2\[x1,x2,…,xm\]/⟨xi2−xi∣1≤i≤m⟩\\mathbb\{F\}\_\{2\}\[x\_\{1\},x\_\{2\},\\ldots,x\_\{m\}\]/\\langle x\_\{i\}^\{2\}\-x\_\{i\}\\mid 1\\leq i\\leq m\\rangleis finite, wheremmis number of neurons in𝒩\\mathcal\{N\}\. By Lemma[2\.8](https://arxiv.org/html/2609.30279#S2.Thmtheorem8), the descending chain of ideals \([7](https://arxiv.org/html/2609.30279#S3.E7)\) must stabilize after a finite number of steps\. Thus, there exists a natural numberN0N\_\{0\}such that∀n≥N0\\forall\\,n\\geq N\_\{0\}

\(8\)JΔn​\(y\)=J𝒩ΔN0​\(y\)J\_\{\\Delta\_\{n\}\}\(y\)=J\_\{\\mathcal\{N\}\_\{\\Delta\_\{N\_\{0\}\}\}\}\(y\)
LetΩ0⊆Ω\\Omega\_\{0\}\\subseteq\\Omegabe an arbitrary subset containing at leastN0N\_\{0\}elements\. By Lemma[3\.5](https://arxiv.org/html/2609.30279#S3.Thmtheorem5), we haveJ𝒩Ω0​\(y\)⊇J𝒩Ω​\(y\)J\_\{\\mathcal\{N\}\_\{\\Omega\_\{0\}\}\}\(y\)\\supseteq J\_\{\\mathcal\{N\}\_\{\\Omega\}\}\(y\)\. IfΩ0\\Omega\_\{0\}contains at leastN0N\_\{0\}elements, from \([8](https://arxiv.org/html/2609.30279#S3.E8)\) we obtainJ𝒩Ω0​\(y\)⊆J𝒩ΩN0​\(y\)=J𝒩Ω​\(y\)J\_\{\\mathcal\{N\}\_\{\\Omega\_\{0\}\}\}\(y\)\\subseteq J\_\{\\mathcal\{N\}\_\{\\Omega\_\{N\_\{0\}\}\}\}\(y\)=J\_\{\\mathcal\{N\}\_\{\\Omega\}\}\(y\)\. Combining both inclusions, we getJ𝒩Ω​\(y\)=J𝒩Ω0​\(y\)J\_\{\\mathcal\{N\}\_\{\\Omega\}\}\(y\)=J\_\{\\mathcal\{N\}\_\{\\Omega\_\{0\}\}\}\(y\)\. This completes the proof\. ∎

## 4\.Applications

In this section, we illustrate the proposed framework through two classification problems of different scales\. We first consider the classical XOR classification problem\[[7](https://arxiv.org/html/2609.30279#bib.bib9)\], where the associated neural ideal can be computed explicitly and interpreted in terms of the activation behavior of the network\. We then apply the same methodology to the MNIST handwritten digit dataset in order to examine the algebraic structure of hidden representations learned by a larger neural network\.

These examples demonstrate how neural codes, allowed code spaces, and neural ideals can be used to provide an algebraic description of neural network classification while retaining information about the internal representations learned by the model\.

### 4\.1\.XOR Classification

We illustrate the proposed framework using the classical XOR classification problem\. Although simple in structure, XOR represents a fundamental non\-linearly separable classification task and provides a convenient setting for explicitly constructing neural codes, allowed code spaces, and neural ideals\.

Consider a feedforward neural network consisting of five neurons, denoted byx1,x2,…,x5x\_\{1\},x\_\{2\},\\ldots,x\_\{5\}, arranged into input, hidden, and output layers\. The network architecture together with the corresponding weights and activation thresholds is shown in Figure[1](https://arxiv.org/html/2609.30279#S4.F1)\.

x1x\_\{1\}x2x\_\{2\}x3x\_\{3\}x4x\_\{4\}x5x\_\{5\}w13=1w\_\{13\}\\\!=\\\!1w14=1w\_\{14\}\\\!=\\\!1w23=1w\_\{23\}\\\!=\\\!1w24=1w\_\{24\}\\\!=\\\!1w35=1w\_\{35\}\\\!=\\\!1w45=−2w\_\{45\}\\\!=\\\!\-2θ3=1\\theta\_\{3\}=1θ4=2\\theta\_\{4\}=2θ5=1\\theta\_\{5\}=1InputLayerHiddenLayerOutputLayerFigure 1\.The 5\-neuron feedforward network layout optimized for XOR classification, including corresponding weights \(wi​jw\_\{ij\}\) and activation thresholds \(θi\\theta\_\{i\}\)\.It is straightforward to verify that the network realizes the XOR classification function\. We now analyze the same network through the algebraic framework developed in the previous section\. The objective is to determine the neural codes generated by the network, construct the corresponding allowed code space, and derive the associated neural ideal\.

Since the output neuron is determined by the activation patterns of the input and hidden neurons, we focus on the first four neurons\. Each neuron can either be active or inactive, and hence every activation pattern can be represented by a binary vector in𝔽24\\mathbb\{F\}\_\{2\}^\{4\}\.

Using Algorithm[3\.1\.1](https://arxiv.org/html/2609.30279#S3.SS1.SSS1)and Algorithm[3\.1\.2](https://arxiv.org/html/2609.30279#S3.SS1.SSS2), we first compute the neural code associated with each input vector\. To illustrate the procedure, consider the input vector\(1,0\)\(1,0\)\.

1. \(1\)For the neuronx3x\_\{3\}, w13​x1\+w23​x2=1≥θ3=1\.w\_\{13\}x\_\{1\}\+w\_\{23\}x\_\{2\}=1\\geq\\theta\_\{3\}=1\.Hencex3x\_\{3\}activates and thereforex3=1x\_\{3\}=1\.
2. \(2\)For the neuronx4x\_\{4\}, w14​x1\+w24​x2=1<θ4=2\.w\_\{14\}x\_\{1\}\+w\_\{24\}x\_\{2\}=1<\\theta\_\{4\}=2\.Hencex4x\_\{4\}does not activate and thereforex4=0x\_\{4\}=0\.

Consequently, the neural code associated with the input\(1,0\)\(1,0\)is\(1,0,1,0\)\(1,0,1,0\)\. Repeating the same computation for the remaining input configurations yields the neural codes listed in Table[1](https://arxiv.org/html/2609.30279#S4.T1)\.

Table 1\.Neural codes of XORWe next determine the allowed code space corresponding to the positive XOR class\. By Definition 3\.1, the allowed code space consists of all neural codes associated with inputs classified into Class𝟏\\mathbf\{1\}\. From Table[1](https://arxiv.org/html/2609.30279#S4.T1), we obtain

C⁡\(𝟏\)=\{\(1,0,1,0\),\(0,1,1,0\)\}\.C\(\\mathbf\{1\}\)=\\\{\(1,0,1,0\),\(0,1,1,0\)\\\}\.
The corresponding neural ideal is constructed from the forbidden states, namely those binary vectors that do not belong to the allowed code space\. Since𝔽24\\mathbb\{F\}\_\{2\}^\{4\}contains24=162^\{4\}=16binary vectors and only two belong toC⁡\(𝟏\)C\(\\mathbf\{1\}\), the remaining fourteen vectors constitute the forbidden code space

C​\(𝟏\)c=𝔽24∖C⁡\(𝟏\)\.C\(\\mathbf\{1\}\)^\{c\}=\\mathbb\{F\}\_\{2\}^\{4\}\\setminus C\(\\mathbf\{1\}\)\.
For each forbidden vector𝐚=\(a1,a2,a3,a4\)\\mathbf\{a\}=\(a\_\{1\},a\_\{2\},a\_\{3\},a\_\{4\}\), we associate the pseudo\-monomial

∏i=14\(1−ai−xi\)\.\\prod\_\{i=1\}^\{4\}\(1\-a\_\{i\}\-x\_\{i\}\)\.
Rather than working with all forbidden states individually, we identify the algebraic relations satisfied by every neural code belonging toC⁡\(𝟏\)C\(\\mathbf\{1\}\)\. Since all computations take place over𝔽2\\mathbb\{F\}\_\{2\}, the Boolean relationsxi2=xix\_\{i\}^\{2\}=x\_\{i\}hold for eachi∈\{1,2,3,4\}i\\in\\\{1,2,3,4\\\}\.

The neural codes belonging to the class𝟏\\mathbf\{1\}satisfy the following structural constraints\.

1. \(1\)For everya=\(a1,a2,a3,a4\)∈C⁡\(𝟏\)a=\(a\_\{1\},a\_\{2\},a\_\{3\},a\_\{4\}\)\\in C\(\\mathbf\{1\}\), we havea4=0a\_\{4\}=0\. Hence every state witha4=1a\_\{4\}=1is forbidden, giving rise to the polynomial relationx4x\_\{4\}\.
2. \(2\)For everya=\(a1,a2,a3,a4\)∈C⁡\(𝟏\)a=\(a\_\{1\},a\_\{2\},a\_\{3\},a\_\{4\}\)\\in C\(\\mathbf\{1\}\), we havea3=1a\_\{3\}=1\. Hence every state witha3=0a\_\{3\}=0is forbidden, yielding the relation1−x31\-x\_\{3\}\.
3. \(3\)For everya=\(a1,a2,a3,a4\)∈C⁡\(𝟏\)a=\(a\_\{1\},a\_\{2\},a\_\{3\},a\_\{4\}\)\\in C\(\\mathbf\{1\}\), the coordinatesa1a\_\{1\}anda2a\_\{2\}are distinct\. Consequently, the following configurations are forbidden: 1. \(a\)a1=a2=0a\_\{1\}=a\_\{2\}=0, yielding the relation \(1−x1\)​\(1−x2\),\(1\-x\_\{1\}\)\(1\-x\_\{2\}\), 2. \(b\)a1=a2=1a\_\{1\}=a\_\{2\}=1, yielding the relation Sincex1​x2=0x\_\{1\}x\_\{2\}=0, the relation \(1−x1\)​\(1−x2\)\(1\-x\_\{1\}\)\(1\-x\_\{2\}\)simplifies to

Combining these relations together with the Boolean relationsxi2−xix\_\{i\}^\{2\}\-x\_\{i\}, fori∈\{1,2,3,4\}i\\in\\\{1,2,3,4\\\}, we obtain the neural ideal

J𝒩​\(𝟏\)=⟨1−x1−x2,x1​x2,1−x3,x4⟩\.J\_\{\\mathcal\{N\}\}\(\\mathbf\{1\}\)=\\left\\langle 1\-x\_\{1\}\-x\_\{2\},\\;x\_\{1\}x\_\{2\},\\;1\-x\_\{3\},\\;x\_\{4\}\\right\\rangle\.
The neural ideal provides a complete algebraic description of the XOR classification mechanism\. Each generator corresponds to a structural constraint satisfied by every admissible neural code associated with the target class\.

1. \(1\)The generatorx4x\_\{4\}indicates that the AND feature represented by the neuronx4x\_\{4\}must remain inactive\.
2. \(2\)The generator1−x31\-x\_\{3\}implies that the OR feature represented by the neuronx3x\_\{3\}must remain active\.
3. \(3\)The generator1−x1−x21\-x\_\{1\}\-x\_\{2\}encodes the defining XOR condition that exactly one of the two input neurons is active\.

Thus, the classification behaviour of the network can be recovered directly from the algebraic relations contained in the neural ideal\.

### 4\.2\.Feature Interpretation through Neural Codes

One of the important aspects of the proposed algebraic framework is its ability to provide insight into the internal representations learned by a neural network\. Although neural networks are often regarded as black\-box models, the neural codes introduced in this work provide a structured description of hidden\-layer activations\. By associating activation patterns with the corresponding input samples, it becomes possible to investigate the features captured by individual neurons and to obtain an interpretable description of their functional roles within the network\.

The central idea is that neurons exhibiting similar activation behaviour correspond to specific visual or structural patterns present in the input data\. Consequently, the combinatorial information encoded by neural codes can be used not only for classification analysis but also for understanding the features learned by the network\. The following procedure describes the feature interpretation framework employed in this work\.

#### 4\.2\.1\.Algorithm: Feature Visualization through Neural Codes

1. \(1\)Begin with a trained feedforward neural network together with its training dataset\.
2. \(2\)Compute the threshold map using Algorithm[3\.1\.1](https://arxiv.org/html/2609.30279#S3.SS1.SSS1)\.
3. \(3\)Apply Algorithm[3\.1\.2](https://arxiv.org/html/2609.30279#S3.SS1.SSS2)to determine the neural codes associated with the input samples\.
4. \(4\)Select a hidden neuron and identify all input samples for which the neuron remains active\.
5. \(5\)Collect the corresponding input images and compute their average\. The resulting image provides a visual representation of the feature associated with the selected neuron\.
6. \(6\)Similarly, compute the average of all input images for which the neuron remains inactive\. This image represents the complementary suppression feature associated with the neuron\.
7. \(7\)Compare the activation and suppression patterns to obtain an interpretable description of the feature learned by the neuron\.

### 4\.3\.MNIST Handwritten Digit Classification

To demonstrate the applicability of the proposed framework on a practical classification task, we consider a feedforward neural network trained on the MNIST handwritten digit dataset\[[5](https://arxiv.org/html/2609.30279#bib.bib4)\]\. The network maps input images represented as vectors inℝ784\\mathbb\{R\}^\{784\}to one of the ten digit classes\.

The neural network consists of the following layers\.

1. \(1\)A first hidden layer containing6464neurons followed by a ReLU activation\.
2. \(2\)A second hidden layer containing1616neurons followed by a ReLU activation\.
3. \(3\)An output layer consisting of1010neurons responsible for the final classification\.

The model is trained using the Adam optimizer for ten epochs\. Following training, the activation thresholds associated with the hidden neurons are determined\. For a neuron with bias parameterbb, the corresponding threshold is given by−b\-b, which represents the hyperplane at which the activation state of the neuron changes\.

Using Algorithm[3\.1\.3](https://arxiv.org/html/2609.30279#S3.SS1.SSS3), each input image is transformed into a binary activation pattern, referred to as its neural code\. Images belonging to the same class tend to generate similar neural codes, indicating that the network organizes visually related inputs into common combinatorial regions of the hidden representation space\. The collection of neural codes corresponding to a particular class forms the associated allowed code space\.

For a fixed target class, such as the digit33, letC3C\_\{3\}denote the set of neural codes generated by images belonging to that class\. The complement ofC3C\_\{3\}within the space of all binary activation states yields the corresponding forbidden code space, denoted byℱ\\mathcal\{F\}\. Each forbidden code

𝐚=\(a1,…,am\)\\mathbf\{a\}=\(a\_\{1\},\\ldots,a\_\{m\}\)determines the pseudo\-monomial

∏k=1m\(ak​xk\+\(1−ak\)​\(1−xk\)\)\.\\prod\_\{k=1\}^\{m\}\\left\(a\_\{k\}x\_\{k\}\+\(1\-a\_\{k\}\)\(1\-x\_\{k\}\)\\right\)\.
The neural ideal associated with the target class is generated by these pseudo\-monomials\. Consequently, the neural ideal provides an algebraic description of the combinatorial constraints satisfied by the hidden representations of that class\.

While neural ideals characterize hidden representations algebraically, understanding their practical significance requires relating hidden neurons back to the image domain\. To accomplish this, we employ the feature interpretation procedure introduced in the previous subsection\.

For each hidden neuron, all training images that activate the neuron are collected and averaged\. The resulting image represents the visual structure most strongly associated with that neuron\. Similarly, averaging the images for which the neuron remains inactive produces a complementary suppression feature\. The difference between these two averages yields a feature balance map that highlights image regions contributing positively or negatively to neuron activation\. These reconstructed visualizations provide an empirical interpretation of the features captured by the hidden neurons\.

To further investigate how the learned hidden representations contribute to the final prediction, we analyze the weight matrix connecting the hidden neurons to the output layer\. Figure[2](https://arxiv.org/html/2609.30279#S4.F2)presents the corresponding heatmap, where positive weights indicate evidence supporting a class, whereas negative weights indicate evidence opposing that class\.

![Refer to caption](https://arxiv.org/html/2609.30279v1/features.png)\(a\)Features learned by hidden neurons 8, 9, 10\.
![Refer to caption](https://arxiv.org/html/2609.30279v1/weight_matrix.png)\(b\)Weight matrix connecting hidden neurons to the output layer\.

Figure 2\.Visualization of learned features and output\-layer weights\.The heatmap indicates that the classifier does not rely on any single hidden neuron\. Instead, each digit class is characterized by a distinct combination of supporting and suppressing features distributed across multiple neurons\. The final classification decision therefore emerges from the collective contribution of the hidden representations learned by the network\.

- •Digit 0\.The weight matrix indicates that neuronsx6x\_\{6\}andx8x\_\{8\}contribute positively towards the classification of digit00, whereas neuronsx4x\_\{4\}andx5x\_\{5\}contribute negatively\. This suggests that the features associated withx6x\_\{6\}andx8x\_\{8\}are more frequently observed in images of digit00, while the patterns represented byx4x\_\{4\}andx5x\_\{5\}are less compatible with this class\.
- •Digit 1\.Digit11receives strong positive contributions from neuronsx2x\_\{2\},x6x\_\{6\}, andx7x\_\{7\}, whereas neuronx5x\_\{5\}is strongly suppressed\. This behaviour indicates that the hidden representations encoded byx2x\_\{2\},x6x\_\{6\}, andx7x\_\{7\}play an important role in distinguishing images belonging to this class\.
- •Digit 3\.The classification of digit33exhibits a strong positive dependence on neuronx3x\_\{3\}, while neuronx8x\_\{8\}contributes negatively\. This observation suggests that the representation of digit33is determined by a combination of supporting and suppressing features rather than by the activation of a single neuron\.
- •Digit 6\.Digit66is associated with some of the strongest negative weights in the entire matrix, particularly for neuronsx1x\_\{1\},x4x\_\{4\}, andx5x\_\{5\}\. At the same time, neuronsx2x\_\{2\}andx3x\_\{3\}contribute positively\. This indicates that the network identifies digit66through a selective balance between favourable and unfavourable hidden representations\.
- •Digit 9\.The classification of digit99is characterized by negative contributions from neuronsx2x\_\{2\}andx3x\_\{3\}, together with positive support from neuronx1x\_\{1\}\. Such behaviour illustrates how the network differentiates visually similar digit classes through variations in their hidden activation patterns\.

### 4\.4\.Software Implementation and Interactive Exploration Framework

To demonstrate the practical applicability of the proposed framework, we developed an interactive software tool, namely the*Neural Network Representation Explorer*\. The tool implements the complete pipeline introduced in this work, including threshold\-map construction, neural\-code generation, allowed code space computation, neural ideal analysis, and feature interpretation\.

The primary objective of the tool is to provide an interpretable interface for studying the internal representations learned by trained neural networks\. Given a trained model and the corresponding dataset, the framework automatically computes the binary activation patterns associated with hidden neurons and constructs the corresponding neural\-code representation\. These codes are subsequently used to identify class\-specific allowed code spaces and their associated algebraic structures\.

Figure[3](https://arxiv.org/html/2609.30279#S4.F3)presents the graphical interface of the developed system\. The left panel visualizes the neural network architecture together with the learned inter\-layer connections\. Positive and negative weights are displayed using distinct colour schemes, allowing the contribution of individual neurons to be examined visually\. The right panel provides detailed information about the selected output node, including the activation state, threshold metric, allowed code\-space information, and receptive\-field relationships\.

![Refer to caption](https://arxiv.org/html/2609.30279v1/tool_overview.png)Figure 3\.Graphical interface of the Neural Network Representation Explorer implementing the proposed framework\. The interface visualizes network connectivity, activation states, threshold\-map information, allowed code spaces, and class\-specific feature representations\.An important component of the framework is the construction of feature balance maps\. For a selected neuron or output class, the tool computes the average activation pattern of the corresponding input samples and contrasts it with the average pattern of suppressed samples\. The resulting visualization highlights image regions that contribute positively or negatively to neuron activation\. Consequently, the learned representations can be interpreted directly in the input domain\.

Figure[4](https://arxiv.org/html/2609.30279#S4.F4)illustrates representative feature balance maps obtained for several output classes of the MNIST dataset\. Although the network is trained solely for classification, the generated visualizations reveal class\-dependent structural patterns learned by the hidden layers\. Distinct activation regions can be observed for different digit classes, indicating that the neural\-code representation preserves meaningful information regarding the internal organization of the network\.

![Refer to caption](https://arxiv.org/html/2609.30279v1/digit_feature_maps.png)Figure 4\.Feature balance maps generated using the proposed framework for different digit classes\. Red regions indicate activation\-supporting structures, whereas blue regions correspond to suppressive regions\. The visualizations provide an interpretable description of the representations learned by the network\.The developed software serves as an implementation of the theoretical framework proposed in this paper\. Beyond classification analysis, it enables interactive exploration of neural codes, activation patterns, class\-specific code spaces, and learned representations\. This demonstrates that the proposed algebraic formulation can be translated into a practical tool for investigating the behaviour of trained neural networks\.

However, this approach requires a significant amount of computational power to represent larger neural networks\.

## 5\.Conclusion

This work introduced an algebraic framework for analyzing neural network representations using neural codes and neural ideals\. By constructing class\-specific allowed code spaces from hidden\-layer activation patterns, we established a direct connection between neural network classification and Boolean polynomial relations\. The proposed framework provides a mathematical description of the decision\-making process while simultaneously enabling the interpretation of hidden representations through neural codes\.

In addition to the theoretical results, we developed algorithms for threshold\-map construction, neural\-code generation, allowed code\-space computation, and neural\-ideal derivation\. A stabilization result was established to guarantee the invariance of neural ideals obtained from sufficiently large collections of training samples\. The XOR and MNIST experiments demonstrated that the framework can capture both classification behaviour and feature\-level representations in a unified algebraic setting\.

To facilitate practical exploration, an interactive software framework was developed for visualizing network connectivity, activation patterns, allowed code spaces, neural ideals, and feature balance maps\. The software illustrates how the proposed mathematical framework can be translated into an interpretable analysis tool for trained neural networks\. The presented results indicate that neural ideals offer a promising direction for studying representation learning, interpretability, and the internal structure of neural network models\.

## Data availability

## References

- \[1\]M\. F\. Atiyah and I\. G\. Macdonald\(2016\)Introduction to commutative algebra\.Student economy edition edition,Addison\-Wesley Series in Mathematics,Westview Press,Boulder\.External Links:ISBN 978\-0\-8133\-5018\-9Cited by:[Lemma 2\.8](https://arxiv.org/html/2609.30279#S2.Thmtheorem8)\.
- \[2\]D\. A\. Cox, J\. Little, and D\. O’Shea\(2025\)Ideals, varieties, and algorithms: an introduction to computational algebraic geometry and commutative algebra\.5th edition edition,Undergraduate Texts in Mathematics,Springer,Cham\.External Links:ISBN 978\-3\-031\-91840\-7,[Document](https://dx.doi.org/10.1007/978-3-031-91841-4)Cited by:[§2](https://arxiv.org/html/2609.30279#S2.p1.1)\.
- \[3\]C\. Curto, V\. Itskov, A\. Veliz\-Cuba, and N\. Youngs\(2013\)The neural ring: an algebraic tool for analyzing the intrinsic structure of neural codes\.Bulletin of Mathematical Biology75\(9\),pp\. 1571–1611\.External Links:ISSN 1522\-9602,[Document](https://dx.doi.org/10.1007/s11538-013-9860-3),[Link](https://doi.org/10.1007/s11538-013-9860-3)Cited by:[§1](https://arxiv.org/html/2609.30279#S1.p2.1),[§2](https://arxiv.org/html/2609.30279#S2.p1.1),[§3\.1\.3](https://arxiv.org/html/2609.30279#S3.SS1.SSS3.p2.1)\.
- \[4\]Y\. LeCun, Y\. Bengio, and G\. Hinton\(2015\)Deep learning\.Nature521\(7553\),pp\. 436–444\.External Links:ISSN 1476\-4687,[Document](https://dx.doi.org/10.1038/nature14539),[Link](https://doi.org/10.1038/nature14539)Cited by:[§1](https://arxiv.org/html/2609.30279#S1.p1.1)\.
- \[5\]Y\. LeCun and C\. Cortes\(2010\)MNIST handwritten digit database\.Note:http://yann\.lecun\.com/exdb/mnist/External Links:[Link](http://yann.lecun.com/exdb/mnist/)Cited by:[§4\.3](https://arxiv.org/html/2609.30279#S4.SS3.p1.1)\.
- \[6\]W\. S\. McCulloch and W\. Pitts\(1943\)A logical calculus of the ideas immanent in nervous activity\.The Bulletin of Mathematical Biophysics5\(4\),pp\. 115–133\.External Links:ISSN 1522\-9602,[Document](https://dx.doi.org/10.1007/BF02478259),[Link](https://doi.org/10.1007/BF02478259)Cited by:[§1](https://arxiv.org/html/2609.30279#S1.p1.1)\.
- \[7\]M\. Minsky and S\. A\. Papert\(2017\)Perceptrons: an introduction to computational geometry\.Note:Reissue of the 1969 Expanded EditionExternal Links:ISBN 0262534770,[Link](https://direct.mit.edu/books/monograph/3132/PerceptronsAn-Introduction-to-Computational)Cited by:[§4](https://arxiv.org/html/2609.30279#S4.p1.1)\.
- \[8\]V\. Papyan, X\. Y\. Han, and D\. L\. Donoho\(2020\)Prevalence of neural collapse during the terminal phase of deep learning training\.Proceedings of the National Academy of Sciences117\(40\),pp\. 24652–24663\.External Links:[Document](https://dx.doi.org/10.1073/pnas.2015509117)Cited by:[§1](https://arxiv.org/html/2609.30279#S1.p1.1)\.
- \[9\]N\. Suh and G\. Cheng\(2025\)A survey on statistical theory of deep learning: approximation, training dynamics, and generative models\.Annual Review of Statistics and Its Application12,pp\. 177–207\.External Links:[Document](https://dx.doi.org/10.1146/annurev-statistics-040522-013920)Cited by:[§1](https://arxiv.org/html/2609.30279#S1.p1.1)\.