Updating the standard neuron model in artificial neural networks

# Updating the standard neuron model in artificial neural networks
Source: [https://arxiv.org/html/2605.30370](https://arxiv.org/html/2605.30370)
\[1\]\\fnmMarcelo\\surBertalmío

1\]\\orgnameSpanish National Research Council \(CSIC\),\\orgaddress\\cityMadrid,\\countrySpain 2\]\\orgnameCenter for Research in Mathematics \(CIMAT\),\\orgaddress\\cityGuanajuato,\\countryMexico 3\]\\orgnameUniversidad Autónoma de Madrid \(UAM\),\\orgaddress\\cityMadrid,\\countrySpain 4\]\\orgnameNational Science Foundation \(NSF\),\\orgaddress\\cityAlexandria,\\stateVA,\\countryUSA

\\fnmThomas\\surBatard\\fnmErik\\surVelasco\-Salido\\fnmRamsses\\surDe Los Santos Mendoza\\fnmJorge H\.\\surMartínez\\fnmStacey\\surLevine[marcelo\.bertalmio@csic\.es](https://arxiv.org/html/2605.30370v1/mailto:[email protected])\[\[\[\[

###### Abstract

From their inception in the 1950s, artificial neural networks \(ANNs\) started using the so\-called point neuron model then prevalent in neuroscience, hoping that this analogy would allow for a better emulation of brain function\. Over the years the neuroscience literature has shown that the point neuron model is too simplistic to properly represent many fundamental neural processes; however, the standard neuron model in ANNs still remains the same\. Here we substitute it by a very recent model of cortical cells and demonstrate through theoretical analyses and experimental results how, simply by using a more realistic neural unit element without augmenting the number of parameters, the resulting ANNs offer a number of important advantages that include increases in expressivity, robustness and learning speed, and a reduction in memorization and the amount of training data needed\.

## Introduction

In the neuroscience of the mid 20th century, the prevalent view on neural computation was a simplistic abstraction\[[1](https://arxiv.org/html/2605.30370#bib.bib1),[2](https://arxiv.org/html/2605.30370#bib.bib2)\]currently known as the point neuron model, where it is assumed that each individual neuron linearly sums its inputs, i\.e\. that its dendrites are passive and act as mere “readers” of incoming signals, and after this linear filtering there follows a nonlinear process to generate the neuronal output\. This approach was adopted to represent neurons inartificial neural networks\(ANNs\) when they were created in the late 1950s\[[3](https://arxiv.org/html/2605.30370#bib.bib3)\], with the belief that matching the brain starts with emulating its building block, the neuron\[[4](https://arxiv.org/html/2605.30370#bib.bib4)\]\. To this day, neural units inartificial neural networksstill follow the same paradigm\[[5](https://arxiv.org/html/2605.30370#bib.bib5)\], and the cascade of linear\-nonlinear stages inANNsis referred to as the standarddeep learning\(DL\) framework\[[6](https://arxiv.org/html/2605.30370#bib.bib6)\]\.

However, the ANN architectures that are currently most popular are poorly aligned with brain functionality\[[7](https://arxiv.org/html/2605.30370#bib.bib7),[8](https://arxiv.org/html/2605.30370#bib.bib8),[9](https://arxiv.org/html/2605.30370#bib.bib9),[10](https://arxiv.org/html/2605.30370#bib.bib10)\]and have some critical limitations that do not affect living beings, such as requiring huge amounts of training data and being extremely sensitive to virtually unnoticeable perturbations of their input\[[11](https://arxiv.org/html/2605.30370#bib.bib11)\]\. Our conjecture is the following: at the root of these and other fundamental constraints and drawbacks ofANNslies the fact that the point neuron model is overly simplistic\. It assumes that dendrites only act linearly, but since the late 1990s and early 2000s it has become clear that dendritic processing is not only much more complex, but also that complex dendritic computations are an indispensable element of key neural behaviors and brain functions\[[12](https://arxiv.org/html/2605.30370#bib.bib12),[13](https://arxiv.org/html/2605.30370#bib.bib13),[14](https://arxiv.org/html/2605.30370#bib.bib14)\]\. Of note, an essential property of dendrites is that they can receivebackpropagating action potentialsinitiated from the soma\[[15](https://arxiv.org/html/2605.30370#bib.bib15)\], and the importance ofbAPsfor the computations of dendrites, single cells and the whole network cannot be overstated\[[16](https://arxiv.org/html/2605.30370#bib.bib16),[17](https://arxiv.org/html/2605.30370#bib.bib17),[18](https://arxiv.org/html/2605.30370#bib.bib18)\]\. There is currently a renewed interest in the research community in leveraging neuroscience to accelerate progress inAI\[[19](https://arxiv.org/html/2605.30370#bib.bib19)\], and a number of recent works consider more realistic dendritic models to improve the performance ofANNs\[[20](https://arxiv.org/html/2605.30370#bib.bib20),[5](https://arxiv.org/html/2605.30370#bib.bib5)\]\. However, these approaches still fall under the standardDLframework, which is not adequate to represent the back and forth exchange between cell output and dendritic contributions caused bybAPs\[[21](https://arxiv.org/html/2605.30370#bib.bib21)\]\.

Very recently, a new model for neurons in the primary visual cortex that considers the interaction between dendritic nonlinearities andbAPswas shown to be able to explain a number of physiological results that challenge standard approaches\[[22](https://arxiv.org/html/2605.30370#bib.bib22)\]\. Here, we adapt this more realistic activity model of biological neurons into a novel formulation for the neural units in ANNs\. Compared with an ANN that uses the standard neuron model and has the same number of parameters, an ANN built using this updated neuron model is much less impacted by the critical limitations mentioned above, being considerably more robust to input perturbations and requiring a significantly smaller amount of training data, as demonstrated by our theoretical analyses and experimental results\. Furthermore, the novel neuron model allows the network to have other very relevant advantages as well: to be more expressive, requiring less parameters to achieve a given quality level in the task it solves; to learn faster, reaching a target performance with fewer learning steps; and to memorize less, which is an asset when the training data has noisy or mislabeled samples, as is generally the case\.

## Results

### Updating the neuron model

We say that an ANN follows thestandard model\(SM\) when the outputuiu\_\{i\}of a neural unit given an input𝐱∈ℝN\\mathbf\{x\}\\in\\mathbb\{R\}^\{N\}may be written as:

ui\\displaystyle u\_\{i\}=ϕ​\(yi\),\\displaystyle=\\phi\(y\_\{i\}\),\(1\)yi\\displaystyle y\_\{i\}=∑j=1Nmi​j​xj−bi,\\displaystyle=\\sum\_\{j=1\}^\{N\}m\_\{ij\}x\_\{j\}\-b\_\{i\},\(2\)
where𝐦i∈ℝN\\mathbf\{m\}\_\{i\}\\in\\mathbb\{R\}^\{N\}is a linear filter,bi∈ℝb\_\{i\}\\in\\mathbb\{R\}is the bias term, andϕ\\phiis the \(nonlinear\) activation function\.

![Refer to caption](https://arxiv.org/html/2605.30370v1/x1.png)\(a\)
![Refer to caption](https://arxiv.org/html/2605.30370v1/x2.png)\(b\)

Figure 1:Schematic neuron models\. Left: standard neuron model in ANNs, following a simplistic abstraction from the neuroscience of the 1950s\[[1](https://arxiv.org/html/2605.30370#bib.bib1),[2](https://arxiv.org/html/2605.30370#bib.bib2)\]that assumed linear dendrites and an output nonlinearity\. Right: proposed neuron model, based on a recent model of cortical cells\[[22](https://arxiv.org/html/2605.30370#bib.bib22)\]; notice how this new model extends the standard neuron model by considering internal nonlinearitiesσ\\sigma\(that represent dendritic processes\), and these nonlinearities are affected in a dynamic way by the activity valueziz\_\{i\}\(that represents the bAP going from the soma to the dendrites\), itself depending on the inputsxjx\_\{j\}and the contributionszkz\_\{k\}from other neurons\.Here we propose a new neuron model forANNsthat extends the standard model with the addition of a new bias term, inspired by a very recent neuroscience work\[[22](https://arxiv.org/html/2605.30370#bib.bib22)\], that emulates the contribution of dendritic nonlinearities mediated by their interaction withbAPs\. In our formulation, the outputviv\_\{i\}of a neural unit given an input𝐱∈ℝN\\mathbf\{x\}\\in\\mathbb\{R\}^\{N\}is

vi\\displaystyle v\_\{i\}=ϕ​\(zi\),\\displaystyle=\\phi\(z\_\{i\}\),\(3\)zi\\displaystyle z\_\{i\}=∑j=1Nmi​j​xj−bi⏞yi−λ​∑k=1Dwi​k​σ​\(zk−zi\)⏞new bias,\\displaystyle=\\overbrace\{\\sum\_\{j=1\}^\{N\}m\_\{ij\}x\_\{j\}\-b\_\{i\}\}^\{y\_\{i\}\}\\,\-\\,\\overbrace\{\\lambda\\sum\_\{k=1\}^\{D\}w\_\{ik\}\\,\\sigma\\\!\\left\(z\_\{k\}\\\!\-\\\!z\_\{i\}\\right\)\}^\{\\text\{new bias\}\},\(4\)
whereλ∈ℝ\\lambda\\in\\mathbb\{R\},𝐰i=\(wi​k\)k=1D∈ℝD\\mathbf\{w\}\_\{i\}=\(w\_\{ik\}\)\_\{k=1\}^\{D\}\\in\\mathbb\{R\}^\{D\}is a linear filter, andσ\\sigmais a nonlinear function\.

The first term in Eq\. \([4](https://arxiv.org/html/2605.30370#Sx2.E4)\) represents the response to the input𝐱\\mathbf\{x\}produced by the linear dendrites; notice how this term is identical toyiy\_\{i\}in Eq\. \([2](https://arxiv.org/html/2605.30370#Sx2.E2)\), which is to be expected given that theSMonly considers dendrites that are passive\. The second term in Eq\. \([4](https://arxiv.org/html/2605.30370#Sx2.E4)\) models the effect of nonlinear dendrites\. In this new bias term, each of the dendrites has the nonlinear response functionσ\\sigma, which receives two inputs: the contributionzkz\_\{k\}from a neighboring unit in the same neural layer, and the valueziz\_\{i\}, which emulates thebAPfrom the soma\. From Eqs\. \([2](https://arxiv.org/html/2605.30370#Sx2.E2)\) and \([4](https://arxiv.org/html/2605.30370#Sx2.E4)\) it is trivial to see that, whenλ=0\\lambda=0, our proposed neuron model is identical to the standard one; see Fig\.[1](https://arxiv.org/html/2605.30370#Sx2.F1)for an schematic of both models\. Importantly, if we takeλ\\lambdaand𝐰i\\mathbf\{w\}\_\{i\}to be fixed and predefined, then our proposed model has exactly the same number of trainable parameters as in theSMcase, i\.e\.𝐦i\\mathbf\{m\}\_\{i\}andbib\_\{i\}\.

Given that the new bias term involvesziz\_\{i\}, the definition ofziz\_\{i\}depends onziz\_\{i\}itself and therefore Eq\. \([4](https://arxiv.org/html/2605.30370#Sx2.E4)\) isimplicit\. For this reason, we say that an ANN that uses our model is animplicit bias neural network\(IBNN\) \. We can see from Eqs\. \([3](https://arxiv.org/html/2605.30370#Sx2.E3)\) and \([4](https://arxiv.org/html/2605.30370#Sx2.E4)\) that computing the output of animplicit bias neural network\(IBNN\) requires solving a system of coupled implicit equations, and while this type of problem does not necessarily have a solution, in our case whenσ\\sigmais a sigmoidal function andλ<T\\lambda<Tfor some positive thresholdTT, the solution always exists, is unique, and minimizes an energy, and consequently it can be computed with a simple gradient descent algorithm \(seeMethods\)\.

As an illustration, we consider a toy example in 2D where we want to find the boundary between two classes\. We train different ANNs with analogous architecture to solve this task \(seeMethods\): aSMnetwork, and severalIBNNswith a variety of values for the weightλ\\lambdaof the implicit bias term and the maximum slope of the nonlinearityσ\\sigma, denoted bypp\. Fig\.[2](https://arxiv.org/html/2605.30370#Sx2.F2)shows the two classes in yellow and blue, and the classification boundaries obtained with the different ANNs\. As we can see,IBNNis more expressive than theSMnetwork, being able to approximate well the non\-convex shape of the classification boundary while the latter is only capable of producing a convex polygon\. Also, the resulting boundary inIBNNis able to become more curved as the magnitude of the parameterλ\\lambdaincreases \(Fig\.[2](https://arxiv.org/html/2605.30370#Sx2.F2), left\), andppcontrols the curvature of the segments or arcs composing the classification boundary \(Fig\.[2](https://arxiv.org/html/2605.30370#Sx2.F2), right\)\.

![Refer to caption](https://arxiv.org/html/2605.30370v1/x3.png)\(a\)
![Refer to caption](https://arxiv.org/html/2605.30370v1/x4.png)\(b\)

Figure 2:Plot of the decision boundary forANNswith a single hidden layer of three neurons in a binary classification of 2D input data: comparison betweenANNswhose hidden layer is composed of neurons according to theSM\(blue solid line in both left and right panels\) and of neurons according to the bio\-inspired neuron model with different weightsλ\\lambdafor the implicit bias term \(left panel, in shades of green\) and different values of the maximum slopeppof the sigmoidal nonlinearityσ\\sigma\(right panel, shades of red\)\. The resulting validation accuracy of eachANNis included in the legend\. Both panels suggest that theIBNNis more expressive than theSMnetwork, as theIBNNis able to approximate well the non\-convex shape of the classification boundary while theANNusing theSMneuron is only capable of producing a convex polygonal shape\. TheIBNNboundary can better fit curved boundaries as the magnitude of the parameterλ\\lambdaincreases \(left\) while he maximum slopeppof the nonlinearityσ\\sigmacontrols the sharpness of the transitions between the segments or arcs composing the classification boundary \(right\)\.
### Advantages of the networks that use the updated neuron model

We perform theoretical analyses and numerical experiments comparing in each case an ANN that usesSMneurons with anIBNNthat has the same architecture and number of parameters but whose neurons follow the proposed model, thus ensuring that the benefits ofIBNNdo not come from an increase in model complexity\. Numerical tests consist of solving image classification tasks over different image datasets and with different architectures, in order to highlight the generality of the approach\.

#### Less training data required

Fig\.[3](https://arxiv.org/html/2605.30370#Sx2.F3)shows, for three different image datasets, thatIBNNconsistently requires just a fraction \(20−82%20\-82\\%\) of the training data in order to achieve the best performance level of anANNwith neurons that follow theSM\.

![Refer to caption](https://arxiv.org/html/2605.30370v1/x5.png)\(a\)Fashion\-MNIST
![Refer to caption](https://arxiv.org/html/2605.30370v1/x6.png)\(b\)SVHN
![Refer to caption](https://arxiv.org/html/2605.30370v1/x7.png)\(c\)CIFAR\-10

Figure 3:IBNNconsistently requires just a fraction \(29−82%29\-82\\%\) of the training data in order to achieve the best performance level of anANNwith neurons that follow theSM\. The pink dashed line indicates the best validation accuracy forSMand the proportion of the complete dataset needed to achieve it, and the proportion for theIBNNto achieve the same accuracy\. Datasets are indicated under each graph and corresponding architectures indicated inMethods\. Shadowed, CI of9090%\.
#### Increased robustness to input perturbations

A common way to analyze the sensitivity of anANNto input perturbations is to compute the norm of the Jacobian matrix of the network with respect to the input, with smaller norm values corresponding to more robust networks\[[23](https://arxiv.org/html/2605.30370#bib.bib23)\]\. Under a mild set of assumptions, it can be shown that the norm of the Jacobian of anIBNNis smaller than that of its correspondingSMnetwork \(i\.e\. the network obtained by settingλ=0\\lambda=0\), and therefore theIBNNis more robust \(seeMethods\)\. It is notable that the formal mathematical proof presented in\[[6](https://arxiv.org/html/2605.30370#bib.bib6)\]to demonstrate thatSMnetworks are either inaccurate or unstable \(if accurate\) does not directly apply to theIBNNformulation, whose structure does not satisfy the assumptions in\[[6](https://arxiv.org/html/2605.30370#bib.bib6)\]due to its implicit and highly nonlinear formulation \(seeMethods\)\.

Recently, it has become very relevant to evaluate the robustness of networks against stealth attacks\[[24](https://arxiv.org/html/2605.30370#bib.bib24)\], defined as a scenario where an attacker modifies one or more neurons in anANNin such a way that the following two conditions are met: \(1\) for inputs belonging to the validation set of theANN, the responses are as expected, and therefore no attack is observed; \(2\) however, for some trigger input known only to the attacker, the ANN provides the result that the attacker wants\. There is proof that a one\-neuron stealth attack is possible on aSMnetwork\[[24](https://arxiv.org/html/2605.30370#bib.bib24)\]but we demonstrate that it would be unfeasible on anIBNN\(seeMethods\)\.

As experimental demonstration of the robustness capabilities that the proposed neuron model provides, Figs\.[4](https://arxiv.org/html/2605.30370#Sx2.F4)and[5](https://arxiv.org/html/2605.30370#Sx2.F5)show comparisons of the performance ofIBNNandSMnetworks with the same number of parameters against adversarial attacks of different nature\. As we can see,IBNNis consistently less sensitive to input perturbations\.

![Refer to caption](https://arxiv.org/html/2605.30370v1/x8.png)\(a\)Fashion\-MNIST
![Refer to caption](https://arxiv.org/html/2605.30370v1/x9.png)\(b\)SVHN
![Refer to caption](https://arxiv.org/html/2605.30370v1/x10.png)\(c\)CIFAR\-10

Figure 4:IBNNis more robust to white\-box adversarial attacks\. The curves represent the accuracy achieved by each model for different intensities of an adversarial attack, in particularPGD\[[25](https://arxiv.org/html/2605.30370#bib.bib25)\]attack \(power denoted in the horizontal axis: note that a power of0corresponds to the clean accuracy of the model\)\. Datasets are indicated under each graph and corresponding architectures and details on the adversarial attacks are indicated inMethods\. Shadowed, full range of data \(CI of100100%\)\.![Refer to caption](https://arxiv.org/html/2605.30370v1/x11.png)\(a\)Fashion\-MNIST
![Refer to caption](https://arxiv.org/html/2605.30370v1/x12.png)\(b\)SVHN
![Refer to caption](https://arxiv.org/html/2605.30370v1/x13.png)\(c\)CIFAR\-10

Figure 5:IBNNis more robust to black\-box adversarial attacks\. The translucent bar represents clean accuracy for each model, whereas the solid bar represents the accuracy under an adversarial attack, in particular Pixle\[[26](https://arxiv.org/html/2605.30370#bib.bib26)\]of fixed strength\. Datasets are indicated under each graph and corresponding architectures and details on the adversarial attacks are indicated inMethods\. The black lines indicate the full range of obtained accuracies \(CI of100100%\)\.
#### Greater expressivity

Assessing the expressivity of a neural network entails determining the functions it can compute\. In analogy to the classical works that demonstrate the universal approximation power ofANNsbased on theSM\[[27](https://arxiv.org/html/2605.30370#bib.bib27),[28](https://arxiv.org/html/2605.30370#bib.bib28)\], we establish that a single\-layerIBNNis also able to approximate any given mapping with an arbitrarily small error \(seeMethods\)\. But we also show thatIBNNsare in fact more expressive thanSMnetworks, because they need a smaller number of parameters to approximate the same function \(seeMethods\)\.

As an example of the improvement in expressivity that the updated neuron model can provide to an ANN, Fig\.[6](https://arxiv.org/html/2605.30370#Sx2.F6)shows that the performance of anIBNNis consistently better than that of aSMnetwork with the same number of trainable parameters or, equivalently, thatIBNNcan achieve the same performance as aSMnetwork with less parameters\. Note that the potential consideration of trainableλ\\lambdawould introduce an additional number of trainable parameters equal to the number ofIBNNlayers in the network, which would make a negligible difference to the comparison shown in the figure\.

![Refer to caption](https://arxiv.org/html/2605.30370v1/x14.png)\(a\)Fashion\-MNIST
![Refer to caption](https://arxiv.org/html/2605.30370v1/x15.png)\(b\)SVHN
![Refer to caption](https://arxiv.org/html/2605.30370v1/x16.png)\(c\)CIFAR\-10

Figure 6:IBNNis more expressive: it achieves higher accuracy with the same number of trainable parameters or, equivalently, it requires less parameters to perform as well as aSMnetwork\. Variation in the number for parameters caused by the increase in the number of channels of the layers of the backbone is accounted for in each graph\. Datasets are indicated under each graph and corresponding architectures are indicated inMethods\. Shadowed, CI of9090%\.
#### Faster learning

Fig\.[7](https://arxiv.org/html/2605.30370#Sx2.F7)shows thatIBNNrequires less learning steps to achieve the same classification performance as aSMnetwork: in all cases, this equated to less than half the number of epochs\.

![Refer to caption](https://arxiv.org/html/2605.30370v1/x17.png)\(a\)Fashion\-MNIST
![Refer to caption](https://arxiv.org/html/2605.30370v1/x18.png)\(b\)SVHN
![Refer to caption](https://arxiv.org/html/2605.30370v1/x19.png)\(c\)CIFAR\-10

Figure 7:IBNNlearns faster:IBNNmatches in1818\-2525epochs the best validation accuracy reached by theSMnetwork within a8080\-epoch training, which, in all cases, happened after the5353\-th epoch\. The pink dashed line indicates the best validation accuracy for theSMover the considered training epochs and epoch when it was achieved, and the epoch for theIBNNto achieve the same accuracy\. Datasets are indicated under each graph and corresponding architectures are indicated inMethods\. Shadowed, CI of9090%\.
#### Less memorization

When networks have a very large number of parameters they can manage to learn imperfect and even mislabeled training data, in a process called memorization\[[29](https://arxiv.org/html/2605.30370#bib.bib29)\]\. This is clearly problematic, as the performance of the network may become brittle\. The occurrence of memorization correlates with the curvature of the loss function with respect to the input\[[30](https://arxiv.org/html/2605.30370#bib.bib30)\], and we provide a mathematical analysis suggesting thatIBNNhas a smaller curvature and therefore memorizes less than aSMnetwork \(seeMethods\)\.

Fig\.[8](https://arxiv.org/html/2605.30370#Sx2.F8)shows an experiment where the training data has been corrupted with varying percentages of mislabeled samples \(details inMethods\): the advantage of the best validation accuracy achieved byIBNNover the best validation achieved by an analogousSMnetwork is consistently larger when the training set includes data with wrong labels, indicating thatIBNNmemorizes less\.

![Refer to caption](https://arxiv.org/html/2605.30370v1/x20.png)\(a\)Fashion\-MNIST
![Refer to caption](https://arxiv.org/html/2605.30370v1/x21.png)\(b\)SVHN
![Refer to caption](https://arxiv.org/html/2605.30370v1/x22.png)\(c\)CIFAR\-10

Figure 8:IBNNmemorizes less: the improvement in validation accuracy ofIBNNover theSMnetwork is constently larger when the training set includes data with wrong labels and, in certain instances, the difference even widens as the proportion of wrong labels increases \(see panel \(b\)\)\. Datasets are indicated under each graph and corresponding architectures are indicated inMethods\. For each dataset: top row, best validation accuracy achieved for each model for different percentage of wrong training labels; bottom row, difference of the validation ofIBNNandSMdepicted in the top row\. Shadowed, CI of9090%\.

## Discussion

While ANNs continue to serve as the core of many state\-of\-the\-art AI algorithms, they still have significant limitations that have not been overcome despite extensive research efforts\. There appears to be an emerging consensus that the solution to these problems will not arrive simply from scaling up current ANNs, but that fundamentally new approaches are needed instead\[[19](https://arxiv.org/html/2605.30370#bib.bib19)\]\. In this article we take inspiration from neuroscience and propose to use a new, compact and very effective model of cortical cells\[[22](https://arxiv.org/html/2605.30370#bib.bib22)\]to replace the omnipresent but clearly outdated point neuron model of ANNs\. The novelty of the proposed approach lies in the combination of several bio\-inspired elements, including internal dendritic\-like nonlinearities that become dynamic and input\-dependent through the modeling of the bAP, and lateral interactions among neurons that result in a system of coupled and implicit differential equations\. In the NeuroAI literature there are recent works that introduce more involved models of neural units\[[31](https://arxiv.org/html/2605.30370#bib.bib31),[32](https://arxiv.org/html/2605.30370#bib.bib32),[33](https://arxiv.org/html/2605.30370#bib.bib33),[5](https://arxiv.org/html/2605.30370#bib.bib5)\], but they fail to emulate the bAP, or assume linear dendrites, or disregard lateral interactions\. And, while some AI approaches use implicit layers and differential equations\[[34](https://arxiv.org/html/2605.30370#bib.bib34)\], they lack an equivalent to the bAP that might make the nonlinearities input\-dependent\.

The experimental and theoretical results presented in the article support our original hypothesis, namely, that the overly simplistic neuron model of ANNs contributes to their fundamental limitations, and that an updated and biologically inspired neuron model, one that takes into account the complexity of dendritic processing, may allow ANNs to achieve significant advantages\. We have shown that ANNs using the new model demonstrate gains over a number of aspects that include expressivity, robustness and learning, and this aligns with the fact that nonlinear dendrites bring these same advantages to cortical computations and neural behaviors\[[14](https://arxiv.org/html/2605.30370#bib.bib14)\]\.

The experiments have focused on image data, and the model comes from visual neuroscience, but we expect the approach to generalize well, making extensions to other data modalities simple, in the same way that deep learning is applied to any sort of data despite its being based on the hierarchical model that Hubel and Wiesel proposed for neurons in the visual cortex\[[2](https://arxiv.org/html/2605.30370#bib.bib2)\]\. TheIBNNparameters\{𝐦i,bi\}\\\{\\mathbf\{m\}\_\{i\},b\_\{i\}\\\}in all experiments have been learned, but we have also tested a second mode of operation for our formulation where these parameters are simply taken from aSMnetwork that solves the same task and has already been trained; the original mode produces more accurate results and is better suited for scenarios where training data is scarce \(given thatIBNNneeds less training data\), whereas the second mode can be seen as a fast and simple method to take an existing AI system and make it more robust to input perturbations, without the need to train it again \(Supplementary Information\)\.

There are several research questions related to the proposed model that are worth exploring as future work\. To begin, we must note that in our experiments the optimum value for the hyperparameterλ\\lambdawas either found by grid search or through pre\-training \(i\.e\. by optimizing for accuracy\); moving forward we believe thatλ\\lambdashould always be considered a trainable parameter\. In the same vein, the spatial extent of the filters𝐰i\\mathbf\{w\}\_\{i\}and the maximum slopeppof the nonlinearityσ\\sigmashould also be learned\. This would produce just a marginal increase in the number of parameters while it could enhance the expressivity of the network even further, although the mathematical analysis could become much more involved\. Related to this, the use of values ofλ\\lambdaabove a certain positive threshold might imply the existence of multiple solutions for a single input, which could allow to represent with higher fidelity the modeling of biological processes through partial differential equations, where the final state of the process is often path\-dependent\[[35](https://arxiv.org/html/2605.30370#bib.bib35)\]\. We will also develop numerical techniques specifically tailored toIBNNs, in order to reduce the computational complexity associated to the iterative nature of finding the solution to the implicit system of equations, and also to the presence of multiple cross\-comparisons in the nonlinear term of the new neuron model\.

Secondly, while the proposed model was inspired by the V1 model in\[[22](https://arxiv.org/html/2605.30370#bib.bib22)\], for the sake of simplicity we did not emulate it fully, omitting one term of their formulation\. We will incorporate that term to our model, which will introduce a non\-monotonic internal nonlinearity, and analyze in which contexts this may provide an advantage \(a recent work suggests that the use of non\-monotonic nonlinearities might be key to predict time\-varying phenomena\[[36](https://arxiv.org/html/2605.30370#bib.bib36)\]\)\.

Thirdly, the second term of our model \(i\.e\. the new bias in Eq\. \([3](https://arxiv.org/html/2605.30370#Sx2.E3)\)\) leads to the observation thatIBNNhas the ability to push for independent representations, because if the activitiesziz\_\{i\}andzjz\_\{j\}are very similar then the neural unitsiiandjjdo not influence each other; this is consistent with the fact that the V1 model that motivated our approach has a direct connection with the sparse coding framework\[[22](https://arxiv.org/html/2605.30370#bib.bib22)\]\. We view this property of our proposed neuron model as providing a complementary behavior to the attention mechanism employed by Transformer architectures\[[37](https://arxiv.org/html/2605.30370#bib.bib37)\], it would be interesting to test if their combination \(i\.e\. a Transformer using the updated neuron model\) provides substantial gains in expressivity\.

Finally, the mathematical form of our model, where the solution minimizes an energy and can be found through an iterative algorithm analogous to the numerical approximation of a differential equation \(Methods, Eq\. \([12](https://arxiv.org/html/2605.30370#Sx4.E12)\)\), links our approach with energy models for AI\[[38](https://arxiv.org/html/2605.30370#bib.bib38)\]and also with causal modeling for machine learning\[[39](https://arxiv.org/html/2605.30370#bib.bib39)\], a topic of the utmost importance as it has been argued that the major unsolved problems in AI are fundamentally tied to causality\[[40](https://arxiv.org/html/2605.30370#bib.bib40)\]\.

## Methods

### Model solution

#### Existence, uniqueness, and energy minimization

We will next give some conditions onλ\\lambdafor which we have \(i\) existence and \(ii\) uniqueness of the solutions of Eq\. \([4](https://arxiv.org/html/2605.30370#Sx2.E4)\)\. To that aim, we establish a connection between Eq\. \([4](https://arxiv.org/html/2605.30370#Sx2.E4)\) and energy functionals of the form

Eλ\(𝐳;𝐲\)=g\(𝐳;𝐲\)−fλ\(𝐳\),𝐳,𝐲∈ℝDwith\{g​\(𝐳;𝐲\)=12​‖𝐳−𝐲‖22fλ​\(𝐳\)=λ2​∑i=1D∑k=1Dwi​k​S​\(zk−zi\),E\_\{\\lambda\}\(\\mathbf\{z\};\\mathbf\{y\}\)=g\(\\mathbf\{z\};\\mathbf\{y\}\)\-f\_\{\\lambda\}\(\\mathbf\{z\}\)\\,,\\mathbf\{z\},\\mathbf\{y\}\\in\\mathbb\{R\}^\{D\}\\;\\textrm\{with\}\\quad\\left\\\{\\begin\{matrix\}\[l\]g\(\\mathbf\{z\};\\mathbf\{y\}\)&=\\displaystyle\\hphantom\{\-\}\\frac\{1\}\{2\}\{\\big\\\|\\mathbf\{z\}\-\\mathbf\{y\}\\big\\\|\}\_\{2\}^\{2\}\\\\ f\_\{\\lambda\}\(\\mathbf\{z\}\)&=\\displaystyle\\frac\{\\lambda\}\{2\}\\sum\_\{i=1\}^\{D\}\\sum\_\{k=1\}^\{D\}w\_\{ik\}S\(z\_\{k\}\-z\_\{i\}\)\\end\{matrix\}\\right\.\\,,\(5\)
where

- •SSis a primitive function ofσ\\sigma, and
- •wi​kw\_\{ik\}are symmetric, positive, and normalized \(i\.e\.wi​k=wk​i≥0w\_\{ik\}=w\_\{ki\}\\geq 0and∑k=1Nwi​k=1\\sum\_\{k=1\}^\{N\}w\_\{ik\}=1\),0<i,k≤D0<i,k\\leq D\.

###### Proposition 1\.

The solutions of Eq\. \([4](https://arxiv.org/html/2605.30370#Sx2.E4)\), if they exist, are the critical points ofEλ​\(⋅;M​𝐱−𝐛\)E\_\{\\lambda\}\(\\cdot;M\\mathbf\{x\}\-\\mathbf\{b\}\)provided thatSSis even\.

###### Proof\.

Denoting by∇𝐳\\nabla\_\{\\mathbf\{z\}\}the gradient operator with respect to𝐳\\mathbf\{z\}, we have that

∇𝐳g​\(𝐳;𝐲\)=𝐳−𝐲\\nabla\_\{\\mathbf\{z\}\}g\(\\mathbf\{z\};\\mathbf\{y\}\)=\\mathbf\{z\}\-\\mathbf\{y\}Moreover, ifSSis even, then we have from the fact thatS′=σS^\{\\prime\}=\\sigmaandwware symmetric that

\(∇𝐳fλ​\(𝐳\)\)i=−λ​∑k=1Dwi​k​σ​\(zk−zi\),\(\\nabla\_\{\\mathbf\{z\}\}f\_\{\\lambda\}\(\\mathbf\{z\}\)\)\_\{i\}=\-\\lambda\\sum\_\{k=1\}^\{D\}w\_\{ik\}\\sigma\(z\_\{k\}\-z\_\{i\}\),from which we deduce that

\(∇𝐳Eλ​\(𝐳;𝐲\)\)i=0⟺zi=yi−λ​∑k=1Dwi​k​σ​\(zk−zi\)\.\(\\nabla\_\{\\mathbf\{z\}\}E\_\{\\lambda\}\(\\mathbf\{z\};\\mathbf\{y\}\)\)\_\{i\}=0\\Longleftrightarrow z\_\{i\}=y\_\{i\}\-\\lambda\\sum\_\{k=1\}^\{D\}w\_\{ik\}\\sigma\(z\_\{k\}\-z\_\{i\}\)\.∎

###### Proposition 2\.

IfSSis positive, even and satisfies thatS​\(z\)≤A​z2S\(z\)\\leq Az^\{2\}, for someA≥0A\\geq 0thenEλE\_\{\\lambda\}is coercive with respect to the first variable for anyλ<14​A\\lambda<\\dfrac\{1\}\{4A\}\.

###### Proof\.

We have

12​‖𝐳−𝐲‖2≥12​‖𝐳‖2−‖𝐳‖​‖𝐲‖\+12​‖𝐲‖2\\dfrac\{1\}\{2\}\{\\\|\\mathbf\{z\}\-\\mathbf\{y\}\\\|\}^\{2\}\\geq\\dfrac\{1\}\{2\}\\\|\\mathbf\{z\}\\\|^\{2\}\-\\\|\\mathbf\{z\}\\\|\\\|\\mathbf\{y\}\\\|\+\\dfrac\{1\}\{2\}\\\|\\mathbf\{y\}\\\|^\{2\}\(6\)from the Cauchy\-Schwartz inequality\.

As

SSis positive, the term

fλ​\(𝐳\)f\_\{\\lambda\}\(\\mathbf\{z\}\)in \([5](https://arxiv.org/html/2605.30370#Sx4.E5)\) is positive if

λ\>0\\lambda\>0, from which follows that

Eλ​\(𝐳;𝐲\)≥12​‖𝐳‖2−‖𝐳‖​‖𝐲‖\+12​‖𝐲‖2ifλ<0,E\_\{\\lambda\}\(\\mathbf\{z\};\\mathbf\{y\}\)\\geq\\dfrac\{1\}\{2\}\\\|\\mathbf\{z\}\\\|^\{2\}\-\\\|\\mathbf\{z\}\\\|\\\|\\mathbf\{y\}\\\|\+\\dfrac\{1\}\{2\}\\\|\\mathbf\{y\}\\\|^\{2\}\\qquad\\textrm\{if\}\\quad\\lambda<0,which gives that

EλE\_\{\\lambda\}is coercive with respect to

𝐳\\mathbf\{z\}if

λ<0\\lambda<0\.

On the other hand, if

λ\>0\\lambda\>0and

S​\(z\)≤A​z2S\(z\)\\leq Az^\{2\}, we have

λ2​∑i=1D∑j=1Dwi​j​S​\(zj−zi\)\\displaystyle\\dfrac\{\\lambda\}\{2\}\\sum\_\{i=1\}^\{D\}\\sum\_\{j=1\}^\{D\}w\_\{ij\}S\(z\_\{j\}\-z\_\{i\}\)≤λ2​A​∑i=1D∑j=1Dwi​j​\(zj−zi\)2\\displaystyle\\leq\\dfrac\{\\lambda\}\{2\}A\\sum\_\{i=1\}^\{D\}\\sum\_\{j=1\}^\{D\}w\_\{ij\}\{\(z\_\{j\}\-z\_\{i\}\)\}^\{2\}≤λ2​A​∑i=1D∑j=1Dwi​j​\(zj2\+zi2\+2​\|zj\|​\|zi\|\)\\displaystyle\\leq\\dfrac\{\\lambda\}\{2\}A\\sum\_\{i=1\}^\{D\}\\sum\_\{j=1\}^\{D\}w\_\{ij\}\(z^\{2\}\_\{j\}\+z^\{2\}\_\{i\}\+2\|z\_\{j\}\|\|z\_\{i\}\|\)≤λ​A​‖𝐳‖2\+λ​A​∑i=1D∑j=1Dwi​j​\|zj\|​\|zi\|\\displaystyle\\leq\\lambda A\\\|\\mathbf\{z\}\\\|^\{2\}\+\\lambda A\\sum\_\{i=1\}^\{D\}\\sum\_\{j=1\}^\{D\}w\_\{ij\}\|z\_\{j\}\|\|z\_\{i\}\|\(7\)from Cauchy\-Schwartz inequality\.

Note that we have

∑i=1D∑j=1Dwi​j​zi2=∑i=1D∑j=1Dwi​j​zj2=‖𝐳‖2\\sum\_\{i=1\}^\{D\}\\sum\_\{j=1\}^\{D\}w\_\{ij\}z\_\{i\}^\{2\}=\\sum\_\{i=1\}^\{D\}\\sum\_\{j=1\}^\{D\}w\_\{ij\}z\_\{j\}^\{2\}=\\\|\\mathbf\{z\}\\\|^\{2\}as the kernelwwis normalized\. The Cauchy\-Schwartz inequality applied to the vectors𝐚,𝐛∈ℝD×D\\mathbf\{a\},\\mathbf\{b\}\\in\\mathbb\{R\}^\{D\\times D\}:

ai​j=wi​j​\|zi\|bi​j=wi​j​\|zj\|a\_\{ij\}=\\sqrt\{w\_\{ij\}\}\|z\_\{i\}\|\\qquad b\_\{ij\}=\\sqrt\{w\_\{ij\}\}\|z\_\{j\}\|gives⟨𝐚,𝐛⟩≤‖𝐚‖​‖𝐛‖\\langle\\mathbf\{a\},\\mathbf\{b\}\\rangle\\leq\\\|\\mathbf\{a\}\\\|\\\|\\mathbf\{b\}\\\|, i\.e\.

∑i=1D∑j=1Dwi​j​\|zi\|​\|zj\|\\displaystyle\\sum\_\{i=1\}^\{D\}\\sum\_\{j=1\}^\{D\}w\_\{ij\}\|z\_\{i\}\|\|z\_\{j\}\|≤\(∑i=1D∑j=1Dwi​j​zj2\)1/2​\(∑i=1D∑j=1Dwi​j​zi2\)1/2\\displaystyle\\leq\\left\(\\sum\_\{i=1\}^\{D\}\\sum\_\{j=1\}^\{D\}w\_\{ij\}z^\{2\}\_\{j\}\\right\)^\{1/2\}\\left\(\\sum\_\{i=1\}^\{D\}\\sum\_\{j=1\}^\{D\}w\_\{ij\}z^\{2\}\_\{i\}\\right\)^\{1/2\}\(8\)≤‖𝐳‖2\\displaystyle\\leq\\\|\\mathbf\{z\}\\\|^\{2\}\(9\)We deduce from Eqs\. \([6](https://arxiv.org/html/2605.30370#Sx4.E6)\), \([Proof\.](https://arxiv.org/html/2605.30370#Sx4.Ex5)\) and \([8](https://arxiv.org/html/2605.30370#Sx4.E8)\) that

Eλ​\(𝐳;𝐲\)\\displaystyle E\_\{\\lambda\}\(\\mathbf\{z\};\\mathbf\{y\}\)≥12​‖𝐳‖2−‖𝐳‖​‖𝐲‖\+12​‖𝐲‖2−λ​A​‖𝐳‖2−λ​A​‖𝐳‖2\\displaystyle\\geq\\dfrac\{1\}\{2\}\\\|\\mathbf\{z\}\\\|^\{2\}\-\\\|\\mathbf\{z\}\\\|\\\|\\mathbf\{y\}\\\|\+\\dfrac\{1\}\{2\}\\\|\\mathbf\{y\}\\\|^\{2\}\-\\lambda A\\\|\\mathbf\{z\}\\\|^\{2\}\-\\lambda A\\\|\\mathbf\{z\}\\\|^\{2\}≥‖𝐳‖2​\(12−2​λ​A\)−‖𝐳‖​‖𝐲‖\+12​‖𝐲‖2\.\\displaystyle\\geq\\\|\\mathbf\{z\}\\\|^\{2\}\(\\dfrac\{1\}\{2\}\-2\\lambda A\)\-\\\|\\mathbf\{z\}\\\|\\\|\\mathbf\{y\}\\\|\+\\dfrac\{1\}\{2\}\\\|\\mathbf\{y\}\\\|^\{2\}\.We deduce thatEλE\_\{\\lambda\}is coercive ifλ<14​A\\lambda<\\dfrac\{1\}\{4A\}\. ∎

###### Proposition 3\.

IfSSis convex and twice differentiable with second order derivative bounded above byK\>0K\>0, then the energy in Eq\. \([5](https://arxiv.org/html/2605.30370#Sx4.E5)\) is strictly convex with respect to the first variable forλ<12​K\\lambda<\\frac\{1\}\{2K\}\.

###### Proof\.

We have

dd​t​Eλ​\(𝐳\+t​𝜼;𝐲\)=\\displaystyle\\dfrac\{d\}\{dt\}E\_\{\\lambda\}\(\\mathbf\{z\}\+t\\bm\{\\eta\};\\mathbf\{y\}\)=12​\(2​⟨𝐳,𝜼⟩\+2​t​‖𝜼‖2−2​⟨𝜼,𝐲⟩\)\\displaystyle\\dfrac\{1\}\{2\}\(2\\langle\\mathbf\{z\},\\bm\{\\eta\}\\rangle\+2t\\\|\\bm\{\\eta\}\\\|^\{2\}\-2\\langle\\bm\{\\eta\},\\mathbf\{y\}\\rangle\)−λ2​∑i=1D∑j=1Dwi​j​\(ηj−ηi\)​S′​\(zj\+t​ηj−zi−t​ηi\)\\displaystyle\-\\dfrac\{\\lambda\}\{2\}\\sum\_\{i=1\}^\{D\}\\sum\_\{j=1\}^\{D\}w\_\{ij\}\(\\eta\_\{j\}\-\\eta\_\{i\}\)S^\{\\prime\}\(z\_\{j\}\+t\\eta\_\{j\}\-z\_\{i\}\-t\\eta\_\{i\}\)and

d2d​t2​Eλ​\(𝐳\+t​𝜼;𝐲\)=‖𝜼‖2−λ2​∑i=1D∑j=1Dwi​j​\(ηj−ηi\)2​S′′​\(zj\+t​ηj−zi−t​ηi\),\\dfrac\{d^\{2\}\}\{dt^\{2\}\}E\_\{\\lambda\}\(\\mathbf\{z\}\+t\\bm\{\\eta\};\\mathbf\{y\}\)=\\\|\\bm\{\\eta\}\\\|^\{2\}\-\\dfrac\{\\lambda\}\{2\}\\sum\_\{i=1\}^\{D\}\\sum\_\{j=1\}^\{D\}w\_\{ij\}\(\\eta\_\{j\}\-\\eta\_\{i\}\)^\{2\}S^\{\\prime\\prime\}\(z\_\{j\}\+t\\eta\_\{j\}\-z\_\{i\}\-t\\eta\_\{i\}\),which gives

d2d​t2​Eλ​\(𝐳\+t​𝜼;𝐲\)\|t=0=‖𝜼‖2−λ2​∑i=1D∑j=1Dwi​j​\(ηj−ηi\)2​S′′​\(zj−zi\),\\dfrac\{d^\{2\}\}\{dt^\{2\}\}E\_\{\\lambda\}\(\\mathbf\{z\}\+t\\bm\{\\eta\};\\mathbf\{y\}\)\_\{\|t=0\}=\\\|\\bm\{\\eta\}\\\|^\{2\}\-\\dfrac\{\\lambda\}\{2\}\\sum\_\{i=1\}^\{D\}\\sum\_\{j=1\}^\{D\}w\_\{ij\}\(\\eta\_\{j\}\-\\eta\_\{i\}\)^\{2\}S^\{\\prime\\prime\}\(z\_\{j\}\-z\_\{i\}\),As

SSis twice differentiable and convex, we have, by definition, that

S′′≥0S^\{\\prime\\prime\}\\geq 0\. We deduce that

d2d​t2​Eλ​\(𝐳\+t​𝜼;𝐲\)\|t=0\>0\\dfrac\{d^\{2\}\}\{dt^\{2\}\}E\_\{\\lambda\}\(\\mathbf\{z\}\+t\\bm\{\\eta\};\\mathbf\{y\}\)\_\{\|t=0\}\>0if

λ<0\\lambda<0, which shows that

EλE\_\{\\lambda\}is strictly convex\.

Let us now assume that

λ\>0\\lambda\>0\. We have that

d2d​t2​Eλ​\(𝐳\+t​𝜼;𝐲\)\|t=0\\displaystyle\\dfrac\{d^\{2\}\}\{dt^\{2\}\}E\_\{\\lambda\}\(\\mathbf\{z\}\+t\\bm\{\\eta\};\\mathbf\{y\}\)\_\{\|t=0\}≥‖𝜼‖2−λ2​K​∑i=1D∑j=1Dwi​j​\(ηj−ηi\)2\.\\displaystyle\\geq\\\|\\bm\{\\eta\}\\\|^\{2\}\-\\dfrac\{\\lambda\}\{2\}K\\sum\_\{i=1\}^\{D\}\\sum\_\{j=1\}^\{D\}w\_\{ij\}\(\\eta\_\{j\}\-\\eta\_\{i\}\)^\{2\}\.Then, from the triangle inequality

\(ηj−ηi\)2≤ηj2\+ηi2\+2​\|ηj\|​\|ηi\|,\(\\eta\_\{j\}\-\\eta\_\{i\}\)^\{2\}\\leq\\eta\_\{j\}^\{2\}\+\\eta\_\{i\}^\{2\}\+2\|\\eta\_\{j\}\|\|\\eta\_\{i\}\|,we have

d2d​t2​Eλ​\(𝐳\+t​𝜼;𝐲\)\|t=0≥‖𝜼‖2−λ2​K​∑i=1D∑j=1Dwi​j​\(ηj2\+ηi2\)−λ2​K​∑i=1D∑j=1Dwi​j​\|ηj\|​\|ηi\|\\dfrac\{d^\{2\}\}\{dt^\{2\}\}E\_\{\\lambda\}\(\\mathbf\{z\}\+t\\bm\{\\eta\};\\mathbf\{y\}\)\_\{\|t=0\}\\geq\\\|\\bm\{\\eta\}\\\|^\{2\}\-\\dfrac\{\\lambda\}\{2\}K\\sum\_\{i=1\}^\{D\}\\sum\_\{j=1\}^\{D\}w\_\{ij\}\(\\eta\_\{j\}^\{2\}\+\\eta\_\{i\}^\{2\}\)\-\\dfrac\{\\lambda\}\{2\}K\\sum\_\{i=1\}^\{D\}\\sum\_\{j=1\}^\{D\}w\_\{ij\}\|\\eta\_\{j\}\|\|\\eta\_\{i\}\|\(10\)The Cauchy\-Schwartz inequality applied to the vectors𝐚,𝐛∈ℝD×D\\mathbf\{a\},\\mathbf\{b\}\\in\\mathbb\{R\}^\{D\\times D\}:

ai​j=wi​j​\|ηi\|bi​j=wi​j​\|ηj\|a\_\{ij\}=\\sqrt\{w\_\{ij\}\}\|\\eta\_\{i\}\|\\qquad b\_\{ij\}=\\sqrt\{w\_\{ij\}\}\|\\eta\_\{j\}\|gives⟨𝐚,𝐛⟩≤‖𝐚‖​‖𝐛‖\\langle\\mathbf\{a\},\\mathbf\{b\}\\rangle\\leq\\\|\\mathbf\{a\}\\\|\\\|\\mathbf\{b\}\\\|, i\.e\.

∑i=1D∑j=1Dwi​j​\|ηi\|​\|ηj\|\\displaystyle\\sum\_\{i=1\}^\{D\}\\sum\_\{j=1\}^\{D\}w\_\{ij\}\|\\eta\_\{i\}\|\|\\eta\_\{j\}\|≤\(∑i=1D∑j=1Dwi​j​ηj2\)1/2​\(∑i=1D∑j=1Dwi​j​ηi2\)1/2\\displaystyle\\leq\\left\(\\sum\_\{i=1\}^\{D\}\\sum\_\{j=1\}^\{D\}w\_\{ij\}\\eta^\{2\}\_\{j\}\\right\)^\{1/2\}\\left\(\\sum\_\{i=1\}^\{D\}\\sum\_\{j=1\}^\{D\}w\_\{ij\}\\eta^\{2\}\_\{i\}\\right\)^\{1/2\}≤‖𝜼‖2\\displaystyle\\leq\\\|\\bm\{\\eta\}\\\|^\{2\}\(11\)We deduce from \([10](https://arxiv.org/html/2605.30370#Sx4.E10)\) and \([Proof\.](https://arxiv.org/html/2605.30370#Sx4.Ex18)\) that

d2d​t2​Eλ​\(𝐳\+t​𝜼;y\)\|t=0\\displaystyle\\dfrac\{d^\{2\}\}\{dt^\{2\}\}E\_\{\\lambda\}\(\\mathbf\{z\}\+t\\bm\{\\eta\};y\)\_\{\|t=0\}≥‖𝜼‖2−λ​K​‖𝜼‖2−λ​K​‖𝜼‖2\\displaystyle\\geq\\\|\\bm\{\\eta\}\\\|^\{2\}\-\\lambda K\\\|\\bm\{\\eta\}\\\|^\{2\}\-\\lambda K\\\|\\bm\{\\eta\}\\\|^\{2\}≥‖𝜼‖2​\(1−2​λ​K\)\\displaystyle\\geq\\\|\\bm\{\\eta\}\\\|^\{2\}\(1\-2\\lambda K\)≥‖𝜼‖2\>0ifλ<12​K\.\\displaystyle\\geq\\\|\\bm\{\\eta\}\\\|^\{2\}\>0\\quad\\textrm\{if\}\\quad\\lambda<\\dfrac\{1\}\{2K\}\.∎

###### Corollary 3\.1\.

Under the assumption thatSSis positive, even and satisfies thatS​\(z\)≤A​z2S\(z\)\\leq Az^\{2\}, for someA≥0A\\geq 0, then Eq\. \([4](https://arxiv.org/html/2605.30370#Sx2.E4)\) possesses a solution forλ<14​A\\lambda<\\dfrac\{1\}\{4A\}\. Moreover, ifSSis also twice differentiable and convex with second order derivative bounded above byK\>0K\>0, then the solution is unique forλ<1max⁡\(4​A,2​K\)\\lambda<\\dfrac\{1\}\{\\max\(4A,2K\)\}\.

###### Proof\.

According to Prop\.[2](https://arxiv.org/html/2605.30370#Thmproposition2), the energyEλ​\(⋅;M​𝐱−𝐛\)E\_\{\\lambda\}\(\\cdot;M\\mathbf\{x\}\-\\mathbf\{b\}\)is coercive forλ<14​A\\lambda<\\dfrac\{1\}\{4A\}ifSSis positive, even and satisfies thatS​\(z\)≤A​z2S\(z\)\\leq Az^\{2\}\. Together with the continuity ofEλ​\(⋅;M​𝐱−𝐛\)E\_\{\\lambda\}\(\\cdot;M\\mathbf\{x\}\-\\mathbf\{b\}\), it guarantees the existence of a minimum ofEλ​\(⋅;M​𝐱−𝐛\)E\_\{\\lambda\}\(\\cdot;M\\mathbf\{x\}\-\\mathbf\{b\}\)\. Then, by differentiability ofEλ​\(⋅;M​𝐱−𝐛\)E\_\{\\lambda\}\(\\cdot;M\\mathbf\{x\}\-\\mathbf\{b\}\), its minima correspond to the points vanishing its gradient, and these latter correspond to solutions of Eq\. \([4](https://arxiv.org/html/2605.30370#Sx2.E4)\) according to Prop\.[1](https://arxiv.org/html/2605.30370#Thmproposition1)\. Moreover, ifSSis also twice differentiable and convex with second order derivative bounded above byK\>0K\>0, then the energy is also strictly convex ifλ<12​K\\lambda<\\dfrac\{1\}\{2K\}according to Prop\.[3](https://arxiv.org/html/2605.30370#Thmproposition3)\. We deduce thatEλ​\(⋅;M​𝐱−𝐛\)E\_\{\\lambda\}\(\\cdot;M\\mathbf\{x\}\-\\mathbf\{b\}\)is coercive and strictly convex forλ<min⁡\(14​A,12​K\)=1max⁡\(4​A,2​K\)\\lambda<\\min\\left\(\\dfrac\{1\}\{4A\},\\dfrac\{1\}\{2K\}\\right\)=\\dfrac\{1\}\{\\max\(4A,2K\)\}, which guarantees the uniqueness of the minimum of the energy, which shows the uniqueness of the solution of Eq\. \([4](https://arxiv.org/html/2605.30370#Sx2.E4)\)\. ∎

#### Practical computation of the solution

###### Corollary 3\.2\.

Under the assumptions of Prop\.[2](https://arxiv.org/html/2605.30370#Thmproposition2)and for a well\-chosen sequence\{τ\(n\)\}\\\{\\tau^\{\(n\)\}\\\}, the sequence𝐳\(n\)=\(zi\(n\)\)i=1D\\mathbf\{z\}^\{\(n\)\}=\(z\_\{i\}^\{\(n\)\}\)\_\{i=1\}^\{D\}defined by

zi\(n\+1\)=zi\(n\)\+τ\(n\)​\(−zi\(n\)\+∑j=1Nmi​j​xj−bi−λ​∑k=1Dwi​k​σ​\(zk\(n\)−zi\(n\)\)\),i∈\{1,…,D\},z^\{\(n\+1\)\}\_\{i\}=z^\{\(n\)\}\_\{i\}\+\\tau^\{\(n\)\}\\Big\(\-\\\!z^\{\(n\)\}\_\{i\}\+\\sum\_\{j=1\}^\{N\}m\_\{ij\}x\_\{j\}\\\!\-\\\!b\_\{i\}\-\\lambda\\sum\_\{k=1\}^\{D\}w\_\{ik\}\\sigma\(z^\{\(n\)\}\_\{k\}\\\!\-\\\!z^\{\(n\)\}\_\{i\}\)\\Big\),\\qquad i\\in\\\{1,\\ldots,D\\\}\\,,\\,\(12\)converges; and, forϕ\\phicontinuous, the sequenceΦ​\(𝐳\(n\)\)=\(ϕ​\(zi\(n\)\)\)i=1D∈ℝD\\Phi\(\\mathbf\{z\}^\{\(n\)\}\)=\(\\phi\(z\_\{i\}^\{\(n\)\}\)\)\_\{i=1\}^\{D\}\\in\\mathbb\{R\}^\{D\}converges to an output of the correspondingIBNNlayer\. Moreover, under the assumptions of Corollary[3\.1](https://arxiv.org/html/2605.30370#Thmproposition3.Thmcorollary1),Φ​\(𝐳\(n\)\)\\Phi\(\\mathbf\{z\}^\{\(n\)\}\)converges to the unique output of theIBNNlayer layer from any initial guess𝐳\(0\)=\(zi\(0\)\)i=1D\\mathbf\{z\}^\{\(0\)\}=\(z\_\{i\}^\{\(0\)\}\)\_\{i=1\}^\{D\}\.

###### Proof\.

Under the assumptions of Prop\.[2](https://arxiv.org/html/2605.30370#Thmproposition2), the energyEλ​\(⋅;M​𝐱−𝐛\)E\_\{\\lambda\}\(\\cdot;M\\mathbf\{x\}\-\\mathbf\{b\}\)possesses a minimum as it is continuous and coercive\. Moreover, the sequence \([12](https://arxiv.org/html/2605.30370#Sx4.E12)\) corresponds to the gradient descent algorithm associated to this energy, whereτ\(n\)\\tau^\{\(n\)\}is the step size at iterationnn\. Then, the convergence of𝐳\(n\)\\mathbf\{z\}^\{\(n\)\}to a critical point ofEλ​\(⋅;M​𝐱−𝐛\)E\_\{\\lambda\}\(\\cdot;M\\mathbf\{x\}\-\\mathbf\{b\}\)is guaranteed ifτ\(n\)\\tau^\{\(n\)\}is well\-chosen \(e\.g\.τ\(n\)\\tau^\{\(n\)\}satisfies the Wolfe conditions\)\. As the critical points ofEλ​\(⋅,M​𝐱−𝐛\)E\_\{\\lambda\}\(\\cdot,M\\mathbf\{x\}\-\\mathbf\{b\}\)correspond to the solutions of Eq\. \([4](https://arxiv.org/html/2605.30370#Sx2.E4)\), it shows that the sequenceΦ​\(𝐳\(n\)\)\\Phi\(\\mathbf\{z\}^\{\(n\)\}\)converges to an output of an IBNN layer ifΦ\\Phiis continous\. Moreover, under the assumptions of Corollary[3\.1](https://arxiv.org/html/2605.30370#Thmproposition3.Thmcorollary1), there is uniqueness of the critical point ofEλ​\(⋅,M​𝐱−𝐛\)E\_\{\\lambda\}\(\\cdot,M\\mathbf\{x\}\-\\mathbf\{b\}\), which guarantees the convergence of the sequenceΦ​\(𝐳\(n\)\)\\Phi\(\\mathbf\{z\}^\{\(n\)\}\)to the unique output of an IBNN layer\. ∎

This iterative algorithm represents an intuitive and natural approach to the output of theIBNNlayer\. However, in the conditions where the solution of the system of coupled implicit equations of Eq\. \([4](https://arxiv.org/html/2605.30370#Sx2.E4)\) exists and unique, the proposed layer is agnostic to the specific algorithm leading to such solution: the practical implementation of the layer could, therefore, be based on any fixed\-point or root\-finding algorithm able to efficiently solve the task, as discussed later in the context of the effective implementation used for the experimental evaluation\.

Finally, we give a family of functions satisfying the desirable properties aforementioned\.

###### Proposition 4\.

The function

Sp:z⟼ln⁡\(cosh⁡\(p​z\)\)p,p∈ℝ\+S\_\{p\}\\colon z\\longmapsto\\dfrac\{\\ln\(\\cosh\(pz\)\)\}\{p\},\\,p\\in\\mathbb\{R\}^\{\+\}\(13\)satisfiesSp​\(z\)≤z2S\_\{p\}\(z\)\\leq z^\{2\}, is positive, even, twice differentiable, and convex, and its second order derivative is bounded above bypp\.

###### Proof\.

It is clear thatSpS\_\{p\}is positive and even\. Moreover, we have

cosh⁡\(p​z\)=ep​z\+e−p​z2≤ep​\|z\|≤ep​z2,\\cosh\(pz\)=\\dfrac\{e^\{pz\}\+e^\{\-pz\}\}\{2\}\\leq e^\{p\|z\|\}\\leq e^\{pz^\{2\}\},and we deduce by monotonicity of the functionln\\lnthat

ln⁡\(cosh⁡\(p​z\)\)≤p​z2,\\ln\(\\cosh\(pz\)\)\\leq pz^\{2\},and consequently that

ln⁡\(cosh⁡\(p​z\)\)p≤z2\.\\dfrac\{\\ln\(\\cosh\(pz\)\)\}\{p\}\\leq z^\{2\}\.
Finally, the functionSpS\_\{p\}is clearly twice differentiable, and it is convex with second order derivative bounded above byppas

Sp′​\(z\)=tanh⁡\(p​z\)\(≜σ​\(z\)\)S\_\{p\}^\{\\prime\}\(z\)=\\tanh\(pz\)\\quad\(\\triangleq\\sigma\(z\)\)\(14\)
and

Sp′′​\(z\)=pcosh2⁡\(p​z\),S\_\{p\}^\{\\prime\\prime\}\(z\)=\\dfrac\{p\}\{\\cosh^\{2\}\(pz\)\},from which we obtain that

0<S′′​\(z\)≤p\.0<S^\{\\prime\\prime\}\(z\)\\leq p\.∎

### IBNNin matrix notation

IBNNlayers based on neurons according to Eqs\. \([3](https://arxiv.org/html/2605.30370#Sx2.E3)\)\-\([4](https://arxiv.org/html/2605.30370#Sx2.E4)\) can be written in a compact manner using multivariate functions and using matrix notation: we briefly define it next because it greatly clarifies certain proofs and results included in the following paragraphs\.

- •Let the affine functionA:ℝN→ℝD\\mathrm\{A\}:\\mathbb\{R\}^\{N\}\\to\\mathbb\{R\}^\{D\}be A​\(𝐱\)=M​𝐱−𝐛,\\mathrm\{A\}\(\\mathbf\{x\}\)=\\mathrm\{M\}\\mathbf\{x\}\-\\mathbf\{b\}\\,,\(15\)withA∈ℳD×N​\(ℝ\)\\mathrm\{A\}\\\!\\in\\\!\\mathcal\{M\}\_\{D\\times N\}\(\\mathbb\{R\}\)and𝐛∈ℝD\\mathbf\{b\}\\\!\\in\\\!\\mathbb\{R\}^\{D\}\.
- •Let the non\-linear functionB:ℝD→ℝD\\mathrm\{B\}:\\mathbb\{R\}^\{D\}\\to\\mathbb\{R\}^\{D\}be defined component\-wise asB​\(𝐳\)=\(Bi​\(𝐳\)\)i=1D\\mathrm\{B\}\(\\mathbf\{z\}\)\\\!=\\\!\\big\(\\mathrm\{B\}\_\{i\}\(\\mathbf\{z\}\)\\big\)\_\{i=1\}^\{D\}, where Bi​\(𝐳\)=∑k=1Dwi​k​σ​\(zk−zi\)\.\\mathrm\{B\}\_\{i\}\(\\mathbf\{z\}\)\\\!=\\\!\\sum\_\{k=1\}^\{D\}w\_\{ik\}\\,\\sigma\(z\_\{k\}\-z\_\{i\}\)\\,\.\(16\)
- •Let the functionF:ℝD→ℝD\\mathrm\{F\}:\\mathbb\{R\}^\{D\}\\to\\mathbb\{R\}^\{D\}be implicitly defined, usingB\\mathrm\{B\}and withλ\\lambdaas a hyperparameter, as F​\(𝐲;λ\)=\{𝐳\|𝐳=𝐲−λ​B​\(𝐳\)\}\.\\mathrm\{F\}\(\\mathbf\{y\};\\lambda\)=\\big\\\{\\mathbf\{z\}\\;\|\\;\\mathbf\{z\}=\\mathbf\{y\}\-\\lambda\\mathrm\{B\}\(\\mathbf\{z\}\)\\big\\\}\\,\.\(17\)
- •And letΦ:ℝD→ℝD\\Phi:\\mathbb\{R\}^\{D\}\\to\\mathbb\{R\}^\{D\}be defined using a 1D component\-wise outer activationϕ\\phias Φ​\(𝐳\)=\(ϕ​\(zi\)\)i=1D\.\\Phi\(\\mathbf\{z\}\)=\(\\,\\phi\(z\_\{i\}\)\\,\)\_\{i=1\}^\{D\}\\,\.\(18\)

Through these, Eqs\. \([3](https://arxiv.org/html/2605.30370#Sx2.E3)\)\-\([4](https://arxiv.org/html/2605.30370#Sx2.E4)\) definingIBNNcan be condensed into

IBNN=Φ∘F​\(⋅;λ\)∘A\\operatorname\{IBNN\}=\\Phi\\circ\\mathrm\{F\}\(\\cdot;\\lambda\)\\circ\\mathrm\{A\}\(19\)
whileSMcan be condensed into

SM=Φ∘A\.\\operatorname\{SM\}=\\Phi\\circ\\mathrm\{A\}\\,\.\(20\)

### Expressivity

#### Number ofSMneurons required to reproduce the operation of anIBNNlayer ofDDneurons

Due to the entanglement between their outputs and their implicit nature, the operations performed byIBNNlayers cannot be efficiently represented usingSMneurons, which act individually and whose operation is purely forward\. To reinforce this idea, we will show that, even for infinitesimalλ\\lambda, approximating the behavior of anIBNNlayer ofDD\-neurons could only be achieved, under theSMparadigm, with a number of neurons that is orders of magnitude higher\.

To this end we will leverage the first\-order Taylor expansion of the operation performed by theIBNNlayer aroundλ=0\\lambda=0\. First, and using implicit differentiation on the matrix notation of Eq\. \([19](https://arxiv.org/html/2605.30370#Sx4.E19)\), the derivative with respect toλ\\lambdais

∂IBNN∂λ=D​Φ​∂F∂λ,with∂F∂λ=−\(IdD×D\+λ​DB\)−1​B,\\frac\{\\partial\\operatorname\{IBNN\}\}\{\\partial\\lambda\}=\\mathrm\{D\}\{\\Phi\}\\,\\frac\{\\partial\\mathrm\{F\}\}\{\\partial\\lambda\},\\qquad\\textrm\{with\}\\qquad\\frac\{\\partial\\mathrm\{F\}\}\{\\partial\\lambda\}=\-\\Big\(\\textrm\{Id\}\_\{\{D\}\\\!\\times\\\!\{D\}\}\+\\,\\lambda\\,\\mathrm\{D\}\{\\mathrm\{B\}\}\\Big\)^\{\\\!\-1\}\\mathrm\{B\}\\,,\(21\)
which yields the expansion

IBNN~​\(𝐱\)=IBNN\|λ=0⁡\(𝐱\)\+λ​∂IBNN∂λ\|λ=0=Φ​\(A​\(𝐱\)\)−λ​D​Φ​B​\(A​\(𝐱\)\)\.\\widetilde\{\\operatorname\{IBNN\}\}\(\\mathbf\{x\}\)\\;=\\;\\operatorname\{IBNN\}\_\{\|\\lambda=0\}\(\\mathbf\{x\}\)\+\\lambda\\left\.\\frac\{\\partial\\operatorname\{IBNN\}\}\{\\partial\\lambda\}\\right\|\_\{\\lambda\\\!=\\\!0\}=\\;\\Phi\\Big\(\\mathrm\{A\}\(\\mathbf\{x\}\)\\Big\)\-\\,\\lambda\\;\\mathrm\{D\}\{\\Phi\}\\;\\mathrm\{B\}\\Big\(\\mathrm\{A\}\(\\mathbf\{x\}\)\\Big\)\\,\.\(22\)
The calculation of the linearization \([22](https://arxiv.org/html/2605.30370#Sx4.E22)\) of theIBNNlayer, composed of 2 summands, would require the following:

- •Theii\-th component of the first summand would require the calculation of the vector multiplication \(plus offset\)Ai​\(𝐱\)=𝐦iT​𝐱−bi\\mathrm\{A\}\_\{i\}\(\\mathbf\{x\}\)=\\mathbf\{m\}^\{T\}\_\{i\}\\mathbf\{x\}\-b\_\{i\}plus the application of the non\-linearityϕ\\phi, which coincides with operation of aSMneuron\.
- •Theii\-th component of the second summand would require the product ofBi​\(A​\(𝐱\)\)\\mathrm\{B\}\_\{i\}\(\\mathrm\{A\}\(\\mathbf\{x\}\)\)with a scalar factor, since the matrixD​Φ\\mathrm\{D\}\{\\Phi\}is diagonal\. The calculation ofBi​\(A​\(𝐱\)\)\\mathrm\{B\}\_\{i\}\(\\mathrm\{A\}\(\\mathbf\{x\}\)\)\(see Eq\. \([16](https://arxiv.org/html/2605.30370#Sx4.E16)\)\) entails calculations involving all theDDcomponentsAk​\(𝐱\)\\mathrm\{A\}\_\{k\}\(\\mathbf\{x\}\),k∈\{1,…,D\}k\\in\\\{1,\\ldots,D\\\}\. In particular, and in line with the discussion in\[[31](https://arxiv.org/html/2605.30370#bib.bib31)\]for convolutional operations, \(even ifA​\(𝐱\)\\mathrm\{A\}\(\\mathbf\{x\}\)is presumed precalculated due to its implication in the first summand\) the calculation ofBi\\mathrm\{B\}\_\{i\}requires, first, the calculation of all ‘intermediate’ signalsσ​\(Ak​\(𝐱\)−Ai​\(𝐱\)\)\\sigma\(\\mathrm\{A\}\_\{k\}\(\\mathbf\{x\}\)\-\\mathrm\{A\}\_\{i\}\(\\mathbf\{x\}\)\),∀k∈\{1,…,D\}\\forall k\\in\\\{1,\\ldots,D\\\}, requiring each aSMneuron with output nonlinearityσ\\sigma, and then the application of anotherSMneuron \(without output nonlinearity\) to linearly combine them, resulting intoD\+1D\+1neurons according to theSM\.

Therefore, even for this first\-order approximation of a layer ofDDIBNNneurons, which would neglect further nonlinear behavior that that the realIBNNlayer would have, would require at leastD​\(D\+2\)D\(D\+2\)neurons according to theSMparadigm\.

#### Auniversal approximation theorem\(UAT\) forIBNN

A whole panoply ofUATsregarding the ability of theSM\-layers to approximate arbitrary functions exists\[[28](https://arxiv.org/html/2605.30370#bib.bib28),[41](https://arxiv.org/html/2605.30370#bib.bib41)\], differing on their respective assumptions about the type of network theSMlayer is part of\. The most widespread version of theUATfor theSMrefers to networks composed of a single hidden SM layer with a linear combination of its outputs, that is, the class of functions

ℋSM=\{𝐱→𝐜⋅SM⁡\(𝐱;M,𝐛\)=𝐜⋅Φ​\(M​𝐱−𝐛\)\},\\mathcal\{H\}\_\{\\operatorname\{SM\}\}=\\Big\\\{\\mathbf\{x\}\\rightarrow\\mathbf\{c\}\\cdot\\operatorname\{SM\}\(\\mathbf\{x\};\\mathrm\{M\},\\mathbf\{b\}\)=\\mathbf\{c\}\\cdot\\Phi\\big\(\\mathrm\{M\}\\mathbf\{x\}\-\\mathbf\{b\}\\big\)\\Big\\\}\\,,\(23\)
whereΦi=ϕ\\Phi\_\{i\}=\\phiis a non\-polynomial activation and where𝐱∈X⊂ℝN\\mathbf\{x\}\\in X\\subset\\mathbb\{R\}^\{N\},XXcompact \(note its correspondence with theSMneurons defined by Eqs\. \([1](https://arxiv.org/html/2605.30370#Sx2.E1)\)\-\([2](https://arxiv.org/html/2605.30370#Sx2.E2)\)\), and its proof consist in showing that the set \([23](https://arxiv.org/html/2605.30370#Sx4.E23)\) is dense in the space of continuous functions𝒞​\(X\)\\mathcal\{C\}\(X\)\[[41](https://arxiv.org/html/2605.30370#bib.bib41)\]\. \(Note that we have explicitly included the matrixM\\mathrm\{M\}and offset vector𝐛\\mathbf\{b\}in the notation of theSMin Eq\. \([23](https://arxiv.org/html/2605.30370#Sx4.E23)\)\.\)

We will analogously prove that the class of functions

ℋIBNN′=\{𝐱→𝐜~⋅SM⁡\(IBNN⁡\(𝐱;M,𝐛,λ\);M~,𝐛~\)=𝐜~⋅Φ​\(M~​IBNN⁡\(𝐱;M,𝐛,λ\)−𝐛~\)\},\\mathcal\{H\}\_\{\\operatorname\{IBNN\}^\{\\prime\}\}=\\bigg\\\{\\mathbf\{x\}\\rightarrow\\tilde\{\\mathbf\{c\}\}\\cdot\\operatorname\{SM\}\\Big\(\\operatorname\{IBNN\}\(\\mathbf\{x\};\\mathrm\{M\},\\mathbf\{b\},\\lambda\);\\tilde\{\\mathrm\{M\}\},\\tilde\{\\mathbf\{b\}\}\\Big\)=\\tilde\{\\mathbf\{c\}\}\\cdot\\Phi\\Big\(\\tilde\{\\mathrm\{M\}\}\\,\\operatorname\{IBNN\}\(\\mathbf\{x\};\\mathrm\{M\},\\mathbf\{b\},\\lambda\)\-\\tilde\{\\mathbf\{b\}\}\\Big\)\\bigg\\\}\\,,\(24\)
again with a non\-polynomial activationϕ\\phiand defined for𝐱∈K⊂ℝN\\mathbf\{x\}\\in K\\subset\\mathbb\{R\}^\{N\}compact, is dense in the space of functions𝒞​\(X\)\\mathcal\{C\}\(X\)\. Note thatℋIBNN′\\mathcal\{H\}\_\{\\operatorname\{IBNN\}^\{\\prime\}\}corresponds to anIBNNlayer followed by aSMlayer and a linear combination of the outputs of the latter\.

###### Proposition 5\.

For everyf∈𝒞​\(X\)f\\in\\mathcal\{C\}\(X\)and everyϵ\>0\\epsilon\>0there existsf~∈ℋIBNN′\\tilde\{f\}\\in\\mathcal\{H\}\_\{\\operatorname\{IBNN\}^\{\\prime\}\}such that

sup𝐱∈X\|f​\(𝐱\)−f~​\(𝐱\)\|<ϵ\.\\sup\_\{\\mathbf\{x\}\\in X\}\\;\\Big\|f\(\\mathbf\{x\}\)\-\\tilde\{f\}\(\\mathbf\{x\}\)\\Big\|\\;<\\;\\epsilon\\,\.

###### Proof\.

By theUATforℋSM\\mathcal\{H\}\_\{\\operatorname\{SM\}\}, for any givenf∈𝒞​\(X\)f\\in\\mathcal\{C\}\(X\)there exist𝐜\\mathbf\{c\},M\\mathrm\{M\}, and𝐛\\mathbf\{b\}such that the approximating functiong​\(𝐱\)=𝐜⋅Φ​\(M​𝐱−𝐛\)g\(\\mathbf\{x\}\)=\\mathbf\{c\}\\cdot\\Phi\(\\mathrm\{M\}\\mathbf\{x\}\-\\mathbf\{b\}\)satisfies

sup𝐱∈X\|f​\(𝐱\)−g​\(𝐱\)\|<ϵ2\.\\sup\_\{\\mathbf\{x\}\\in X\}\\;\\Big\|f\(\\mathbf\{x\}\)\-g\(\\mathbf\{x\}\)\\Big\|\\;<\\;\\frac\{\\epsilon\}\{2\}\\,\.
For this proof, let us assume that the nonlinearityϕ\\phiis injective; althoughReLU\\operatorname\{ReLU\}, the prototypical activation function and the basis for the experiments in this manuscript, is in fact not injective, we could assume, without any loss of generality for the purpose of this proof, a family of injective softplus functions with uniform convergence to it\. This allows the definition and use of the left inverseΦleft−1\\Phi\_\{\\mathrm\{left\}\}^\{\\\!\-1\}such thatΦleft−1∘Φ=Id\\Phi\_\{\\mathrm\{left\}\}^\{\\\!\-1\}\\circ\\Phi=\\mathrm\{Id\}\. In addition, forλ\\lambdawhereF​\(⋅;λ\)\\mathrm\{F\}\(\\cdot;\\lambda\)is single valued and is injective, theL​\(⋅;λ\):ℝD→ℝD\\mathrm\{L\}\(\\cdot;\\lambda\):\\mathbb\{R\}^\{D\}\\to\\mathbb\{R\}^\{D\}such that

L​\(𝐳;λ\)=𝐳\+λ​B​\(𝐳\),\\mathrm\{L\}\(\\mathbf\{z\};\\lambda\)=\\mathbf\{z\}\+\\lambda\\mathrm\{B\}\(\\mathbf\{z\}\)\\,,\(25\)
is well defined and is, it and straightforwardly \(see Eq\. \([17](https://arxiv.org/html/2605.30370#Sx4.E17)\)\) the left\-inverse ofF​\(⋅;λ\)\\mathrm\{F\}\(\\cdot;\\lambda\)and thusL​\(⋅;λ\)∘F​\(⋅;λ\)=Id\\mathrm\{L\}\(\\cdot;\\lambda\)\\circ\\mathrm\{F\}\(\\cdot;\\lambda\)=\\mathrm\{Id\}\. Using these two maps, we can define the functionH​\(⋅;λ\):ℝD→ℝD\\mathrm\{H\}\(\\cdot;\\lambda\):\\mathbb\{R\}^\{D\}\\to\\mathbb\{R\}^\{D\}asH​\(⋅;λ\)=L​\(⋅;λ\)∘Φleft−1\\mathrm\{H\}\(\\cdot;\\lambda\)=\\mathrm\{L\}\(\\cdot;\\lambda\)\\circ\\Phi\_\{\\mathrm\{left\}\}^\{\\\!\-1\}, which \(represents by construction a partial, left inverse of theIBNNlayer and\) yields

H​\(IBNN⁡\(𝐱;M,𝐛,λ\);λ\)=A​\(𝐱\)=M​𝐱−𝐛\.\\mathrm\{H\}\\Big\(\\operatorname\{IBNN\}\(\\mathbf\{x\};\\mathrm\{M\},\\mathbf\{b\},\\lambda\);\\;\\lambda\\Big\)=\\mathrm\{A\}\(\\mathbf\{x\}\)=\\mathrm\{M\}\\mathbf\{x\}\-\\mathbf\{b\}\\,\.\(26\)
Let us define a new functionh​\(𝐯\)=𝐜⋅Φ​\(H​\(𝐯;λ\)\)h\(\\mathbf\{v\}\)=\\mathbf\{c\}\\cdot\\Phi\(\\,\\mathrm\{H\}\(\\mathbf\{v\};\\lambda\)\\,\)based on the parameter𝐜\\mathbf\{c\}and the functionΦ\\Phiof the approximating functiong​\(𝐱\)g\(\\mathbf\{x\}\)above and on the functionH​\(⋅;λ\)\\mathrm\{H\}\(\\cdot;\\lambda\)just defined\. The domain of definitionVVofhhis also compact, sinceV=IBNN⁡\(X\)V=\\operatorname\{IBNN\}\(X\)andIBNNis continuous\. Additionally,hhis continuous, since it is the composition of continuous maps\. Leveraging again theUATfor theSM, this time on the compact domainVV, there exist new𝐜~\\tilde\{\\mathbf\{c\}\},M~\\tilde\{\\mathrm\{M\}\}, and𝐛~\\tilde\{\\mathbf\{b\}\}such that

sup𝐯∈V\|h​\(𝐯\)−𝐜~⋅Φ​\(M~​𝐯−𝐛~\)\|<ϵ2\.\\sup\_\{\\mathbf\{v\}\\in V\}\\;\\Big\|h\(\\mathbf\{v\}\)\-\\tilde\{\\mathbf\{c\}\}\\cdot\\Phi\(\\tilde\{\\mathrm\{M\}\}\\mathbf\{v\}\-\\tilde\{\\mathbf\{b\}\}\)\\Big\|\\;<\\;\\frac\{\\epsilon\}\{2\}\\,\.
Since, using Eq\. \([26](https://arxiv.org/html/2605.30370#Sx4.E26)\), we have

h​\(IBNN⁡\(𝐱;M,𝐛,λ\)\)=𝐜⋅Φ​\(H​\(IBNN⁡\(𝐱;M,𝐛,λ\);λ\)\)=𝐜⋅Φ​\(M​𝐱−𝐛\)=g​\(𝐱\)\.h\\Big\(\\,\\operatorname\{IBNN\}\(\\mathbf\{x\};\\mathrm\{M\},\\mathbf\{b\},\\lambda\)\\,\\Big\)=\\mathbf\{c\}\\cdot\\Phi\\left\(\\,\\mathrm\{H\}\\Big\(\\operatorname\{IBNN\}\(\\mathbf\{x\};\\mathrm\{M\},\\mathbf\{b\},\\lambda\);\\;\\lambda\\Big\)\\,\\right\)=\\mathbf\{c\}\\cdot\\Phi\(\\mathrm\{M\}\\mathbf\{x\}\-\\mathbf\{b\}\)=g\(\\mathbf\{x\}\)\\,\.
Therefore, the functionh∘IBNN⁡\(⋅;M,𝐛,λ\)h\\circ\\operatorname\{IBNN\}\(\\cdot;\\mathrm\{M\},\\mathbf\{b\},\\lambda\)yields the claim through the triangle inequality\. ∎

### Comparative analysis of the robustness ofIBNN

#### Reduced sensitivity ofIBNNwith respect to input perturbations

The following analysis, based on the matrix notation of Eqs\. \([15](https://arxiv.org/html/2605.30370#Sx4.E15)\)\-\([20](https://arxiv.org/html/2605.30370#Sx4.E20)\), compares the behavior ofIBNNandSMlayers regarding their sensitivity to input perturbation on the assumption thatIBNNandSMnets having identical affine transforms \(Eq\. \([15](https://arxiv.org/html/2605.30370#Sx4.E15)\)\) and scoring head serve to the same classification problem \(Supplementary Information\)\.

##### Local perturbations

What follows demonstrates that the output of the layers and of the classifierANNsbased on the newly proposedIBNNneurons are less sensitive to infinitesimal perturbations of their inputs than theirSMcounterparts, by virtue of the analysis of their Jacobian matrix\.

Using implicit derivation\[[34](https://arxiv.org/html/2605.30370#bib.bib34)\]under the assumption thatF​\(⋅;λ\)\\mathrm\{F\}\(\\cdot;\\lambda\)\(see Eq\. \([15](https://arxiv.org/html/2605.30370#Sx4.E15)\)\) is a differentiable function𝐳​\(𝐲\)=F​\(𝐲;λ\)\\mathbf\{z\}\(\\mathbf\{y\}\)=\\mathrm\{F\}\(\\mathbf\{y\};\\lambda\)of its input𝐱\\mathbf\{x\}we can write

dd​𝐲​𝐳​\(𝐲\)=IdD×D−λ​dBd​𝐳​dd​𝐲​𝐳​\(𝐲\)\.\\frac\{\\mathrm\{d\}\}\{\\mathrm\{d\}\\mathbf\{y\}\}\\,\\mathbf\{z\}\(\\mathbf\{y\}\)=\\textrm\{Id\}\_\{\{D\}\\\!\\times\\\!\{D\}\}\-\\lambda\\frac\{\\mathrm\{d\}\\mathrm\{B\}\}\{\\mathrm\{d\}\\mathbf\{z\}\}\\frac\{\\mathrm\{d\}\}\{\\mathrm\{d\}\\mathbf\{y\}\}\\,\\mathbf\{z\}\(\\mathbf\{y\}\)\\,\.\(27\)
Reordering the terms and using the more compact symbolD​\(⋅\)\|\(⋅\)\\mathrm\{D\}\{\(\\cdot\)\}\_\{\|\(\\cdot\)\}to denote the absolute derivative calculated at the point indicated by its subscript yields

d​𝐳d​𝐲=DF\|𝐲=\(IdD×D\+λ​DB\|F\(𝐲\)\)−1≜K∞\|F​\(𝐲\)\.\\frac\{\\mathrm\{d\}\\mathbf\{z\}\}\{\\mathrm\{d\}\\mathbf\{y\}\}=\\mathrm\{D\}\{\\mathrm\{F\}\}\_\{\|\\mathbf\{y\}\}=\\Big\(\\textrm\{Id\}\_\{\{D\}\\\!\\times\\\!\{D\}\}\+\\,\\lambda\\,\\mathrm\{D\}\{\\mathrm\{B\}\}\_\{\|\\mathrm\{F\}\(\\mathbf\{y\}\)\}\\Big\)^\{\\\!\-1\}\\triangleq\\mathrm\{K\}\_\{\\infty\|\\mathrm\{F\}\(\\mathbf\{y\}\)\}\\,\.\(28\)
In the following we will refer to this term asDF\|𝐲\\mathrm\{D\}\{\\mathrm\{F\}\}\_\{\|\\mathbf\{y\}\}orK∞\|F​\(𝐲\)\\mathrm\{K\}\_\{\\infty\|\\mathrm\{F\}\(\\mathbf\{y\}\)\}\.

Using Eq\. \([28](https://arxiv.org/html/2605.30370#Sx4.E28)\), the Jacobian matrix ofIBNNcan be written through the application of the chain rule to Eq\. \([19](https://arxiv.org/html/2605.30370#Sx4.E19)\) as

D​IBNN\|𝐱=D​Φ\|\(F∘A\)\(𝐱\)​K∞\|\(F∘A\)​\(𝐱\)​DA,\\mathrm\{D\}\{\\,\\operatorname\{IBNN\}\}\_\{\|\\mathbf\{x\}\}=\\mathrm\{D\}\{\\Phi\}\_\{\|\(\\mathrm\{F\}\\circ\\mathrm\{A\}\)\(\\mathbf\{x\}\)\}\\;\\mathrm\{K\}\_\{\\infty\|\(\\mathrm\{F\}\\circ\\mathrm\{A\}\)\(\\mathbf\{x\}\)\}\\;\\mathrm\{D\}\{\\mathrm\{A\}\}\\,,\(29\)
where we have used thatDA\|𝐱=DA=M\\mathrm\{D\}\{\\mathrm\{A\}\}\_\{\|\\mathbf\{x\}\}=\\mathrm\{D\}\{\\mathrm\{A\}\}=\\mathrm\{M\}, constant for the whole input space\. Note that, based on the vector formulation of Eq\. \([15](https://arxiv.org/html/2605.30370#Sx4.E15)\), the Jacobian matrix ofSMis

D​SM\|𝐱=D​Φ\|A\(𝐱\)​DA=D​Φ\|A\(𝐱\)​IdD×D​DA,\\mathrm\{D\}\{\\,\\operatorname\{SM\}\}\_\{\|\\mathbf\{x\}\}=\\mathrm\{D\}\{\\Phi\}\_\{\|\\mathrm\{A\}\(\\mathbf\{x\}\)\}\\;\\mathrm\{D\}\{\\mathrm\{A\}\}=\\mathrm\{D\}\{\\Phi\}\_\{\|\\mathrm\{A\}\(\\mathbf\{x\}\)\}\\;\\textrm\{Id\}\_\{\{D\}\\\!\\times\\\!\{D\}\}\\;\\mathrm\{D\}\{\\mathrm\{A\}\}\\,,\(30\)
which highlights the analogies betweenD​SM\|𝐱\\mathrm\{D\}\{\\,\\operatorname\{SM\}\}\_\{\|\\mathbf\{x\}\}andD​IBNN\|𝐱\\mathrm\{D\}\{\\,\\operatorname\{IBNN\}\}\_\{\|\\mathbf\{x\}\}but also the decisive interposed termK∞\|\(F∘A\)​\(𝐱\)\\mathrm\{K\}\_\{\\infty\|\(\\mathrm\{F\}\\circ\\mathrm\{A\}\)\(\\mathbf\{x\}\)\}present in the expression Eq\. \([29](https://arxiv.org/html/2605.30370#Sx4.E29)\) of the latter\. For convenience and for the sake of clarity, we will henceforth drop the explicit indication of the calculation point of each derivative\.

Taking a deeper look into the shape and behavior ofK∞\\mathrm\{K\}\_\{\\infty\}, we have that

DB=\(DBi​j\)i,j∈\{1,…,D\},withDBi​j=\{−∑∀k≠i\[wi​k​σ′​\(zk−zi\)\]j=iwi​j​σ′​\(zj−zi\)j≠i,\\mathrm\{D\}\{\\mathrm\{B\}\}=\\Big\(\\mathrm\{D\}\{\\mathrm\{B\}\}\_\{ij\}\\Big\)\_\{i,j\\in\\\{1,\\ldots,D\\\}\}\\,,\\quad\\textrm\{with \}\\quad\\mathrm\{D\}\{\\mathrm\{B\}\}\_\{ij\}=\\left\\\{\\begin\{matrix\}\-\\displaystyle\\sum\_\{\\begin\{subarray\}\{c\}\\forall k\\neq i\\end\{subarray\}\}\\Big\[w\_\{ik\}\\;\\sigma^\{\\prime\}\(z\_\{k\}\-z\_\{i\}\)\\Big\]&\\;j=i\\\\ \\hphantom\{\-\\displaystyle\\sum\_\{\\begin\{subarray\}\{c\}\\forall k\\neq i\\end\{subarray\}\}\\Big\[\}w\_\{ij\}\\;\\sigma^\{\\prime\}\(z\_\{j\}\-z\_\{i\}\)\\hphantom\{\\Big\]\}&\\;j\\neq i\\end\{matrix\}\\right\.\\;,\(31\)
which means that, by virtue of the assumptionswj​i=wi​jw\_\{ji\}=w\_\{ij\}andσ\\sigmaodd,DB\\mathrm\{D\}\{\\mathrm\{B\}\}is symmetric, which in turn implies that it is diagonalizable and that it can be written asDB=Q​Ξ​QT\\mathrm\{D\}\{\\mathrm\{B\}\}=\\mathrm\{Q\}\\,\\Xi\\,\\mathrm\{Q\}^\{T\}for a certain real orthogonal matrixQQand a real diagonal matrixΞ=diag⁡\{ξ1,…,ξD\}\\Xi=\\operatorname\{diag\}\\\{\\xi\_\{1\},\\ldots,\\xi\_\{D\}\\\}\. The fact that, additionally, each element in the main diagonal ofDB\\mathrm\{D\}\{\\mathrm\{B\}\}is exactly the negative sum of the rest of elements of its row has interesting additional implications:

- •Columns are linearly dependent, and thus at least one eigenvalue is0\(and its corresponding eigenvalue is in fact the uniform vector1D​\(1,…,1\)\\frac\{1\}\{\\sqrt\{D\}\}\(1,\\ldots,1\)\)\.
- •Sincewi​kw\_\{ik\}andσ′\\sigma^\{\\prime\}are non\-negative,DB\\mathrm\{D\}\{\\mathrm\{B\}\}is a symmetric diagonally dominant matrix with real non\-positive diagonal elements, which makes it negative semidefinite and implies thatξi≤0\\xi\_\{i\}\\leq 0,∀i∈\{1,…,D\}\\forall i\\in\\\{1,\\ldots,D\\\}\. \(In fact, by virtue of Gershgorin circle theorem, all the eigenvalues are within the intervalξi∈\[−2​maxi⁡\|DBi​i\|,0\]\\xi\_\{i\}\\in\\left\[\\,\-2\\max\_\{i\}\|\\mathrm\{D\}\{\\mathrm\{B\}\}\_\{ii\}\|,0\\,\\right\]\.

These implications result \(ordering eigenvalues in decreasing order of magnitude \(\|ξ1\|≥…​\|ξi\|≥\|ξD\|\|\\xi\_\{1\}\|\\geq\\ldots\|\\xi\_\{i\}\|\\geq\|\\xi\_\{D\}\|\) without any loss of generality\) inΞ=diag⁡\{ξ1,…,ξi,…,ξD−1,0\}\\Xi=\\operatorname\{diag\}\\\{\\xi\_\{1\},\\,\\ldots,\\,\\xi\_\{i\},\\,\\ldots,\\,\\xi\_\{D\-1\},\\,0\\\}, whereξi≤0\\xi\_\{i\}\\leq 0,∀i∈\{1,…,D−1\}\\forall i\\in\\\{1,\\ldots,D\-1\\\}\. The injection of the above properties ofDB\\mathrm\{D\}\{\\mathrm\{B\}\}into the expression ofK∞\\mathrm\{K\}\_\{\\infty\}, includingξD=0\\xi\_\{D\}=0, yields

K∞=\(IdD×D\+λ​Q​Ξ​QT\)−1=Q​\(IdD×D\+λ​Ξ\)−1​QT=Q​diag⁡\{11\+λ​ξ1,…,11\+λ​ξi,…,11\+λ​ξD−1,1\}​QT\.\\begin\{split\}\\mathrm\{K\}\_\{\\infty\}&=\\Big\(\\textrm\{Id\}\_\{\{D\}\\\!\\times\\\!\{D\}\}\\,\+\\,\\lambda\\,\\mathrm\{Q\}\\,\\Xi\\,\\mathrm\{Q\}^\{T\}\\Big\)^\{\\\!\-1\}=\\mathrm\{Q\}\\;\\Big\(\\textrm\{Id\}\_\{\{D\}\\\!\\times\\\!\{D\}\}\+\\lambda\\,\\Xi\\Big\)^\{\\\!\-1\}\\mathrm\{Q\}^\{T\}\\\\ &=\\mathrm\{Q\}\\;\\operatorname\{diag\}\\left\\\{\\frac\{1\}\{1\+\\lambda\\xi\_\{1\}\}\\,,\\;\\ldots\\,,\\;\\frac\{1\}\{1\+\\lambda\\xi\_\{i\}\}\\,,\\;\\ldots\\,,\\;\\frac\{1\}\{1\+\\lambda\\xi\_\{D\\\!\-\\\!1\}\}\\,,\\;1\\right\\\}\\;\\mathrm\{Q\}^\{T\}\\,\.\\end\{split\}\(32\)
This expression directly represents the diagonalization ofK∞\\mathrm\{K\}\_\{\\infty\}and links its eigenvalueskik\_\{i\}one\-to\-one to the from the respective eigenvaluesξi\\xi\_\{i\}ofDB\\mathrm\{D\}\{\\mathrm\{B\}\}aski=11\+λ​ξik\_\{i\}=\\frac\{1\}\{1\+\\lambda\\xi\_\{i\}\}\.

Since, as previously discussed,ξi≤0\\xi\_\{i\}\\leq 0,∀i∈\{1,…,D−1\}\\forall i\\in\\\{1,\\ldots,D\\\!\-\\\!1\\\}, we can conclude that anyλ≤0\\lambda\\leq 0would ensure that all the eigenvalueskik\_\{i\}ofK∞\\mathrm\{K\}\_\{\\infty\}are in the range\(0,1\]\(0,1\]and thus\|ki\|≤1\|k\_\{i\}\|\\leq 1\. Forλ\>0\\lambda\>0, however,\|ki\|≤1\|k\_\{i\}\|\\leq 1is only fulfilled depending on the relative values ofξi\\xi\_\{i\}andλ\\lambda, and in particular forλ≥−2ξi\\lambda\\geq\-\\frac\{2\}\{\\xi\_\{i\}\}\. In bothλ\>0\\lambda\>0andλ<0\\lambda<0casesK∞\\mathrm\{K\}\_\{\\infty\}has at least one eigenvalue exactly11\.

The norm‖K∞‖2\\\|\\mathrm\{K\}\_\{\\infty\}\\\|\_\{2\}, representing the worst\-case direction regarding vector stretch, is11; nevertheless, the directions corresponding to eigenvectors of eigenvalues strictlyξi<0\\xi\_\{i\}<0are compressed by the effect ofK∞\\mathrm\{K\}\_\{\\infty\}\. This implies that,∀δ​𝐲∈ℝD\\forall\\delta\\mathbf\{y\}\\in\\mathbb\{R\}^\{D\},‖K∞​δ​𝐲‖2=k​‖δ​𝐲‖2\\\|\\mathrm\{K\}\_\{\\infty\}\\,\\delta\\mathbf\{y\}\\\|\_\{2\}\\,=k\\\|\\delta\\mathbf\{y\}\\\|\_\{2\}for somek∈\(0,1\]k\\in\(0,1\]dependent on the specificδ​𝐲\\delta\\mathbf\{y\}, which implies, in turn,

‖D​IBNN⁡δ​𝐱‖2=‖D​Φ​K∞​DA​δ​𝐱‖2≤‖D​Φ‖2​‖K∞​DA​δ​𝐱‖2≤‖D​Φ‖2​‖DA​δ​𝐱‖2,\\begin\{split\}\\\|\\mathrm\{D\}\{\\,\\operatorname\{IBNN\}\}\\;\\delta\\mathbf\{x\}\\\|\_\{2\}&=\\;\\\|\\mathrm\{D\}\{\\Phi\}\\;\\mathrm\{K\}\_\{\\infty\}\\;\\mathrm\{D\}\{\\mathrm\{A\}\}\\;\\delta\\mathbf\{x\}\\\|\_\{2\}\\\\ &\\;\\leq\\;\\\|\\mathrm\{D\}\{\\Phi\}\\\|\_\{2\}\\;\\\|\\mathrm\{K\}\_\{\\infty\}\\;\\mathrm\{D\}\{\\mathrm\{A\}\}\\;\\delta\\mathbf\{x\}\\\|\_\{2\}\\;\\leq\\;\\\|\\mathrm\{D\}\{\\Phi\}\\\|\_\{2\}\\;\\\|\\mathrm\{D\}\{\\mathrm\{A\}\}\\;\\delta\\mathbf\{x\}\\\|\_\{2\}\\,,\\end\{split\}\(33\)
where the second line results from the definition of the norm∥⋅∥2\\\|\\cdot\\\|\_\{2\}, ensuring‖Ψ​δ​𝐱‖2≤‖Ψ‖2​‖δ​𝐱‖2\\\|\\Psi\\,\\delta\\mathbf\{x\}\\\|\_\{2\}\\leq\\\|\\Psi\\\|\_\{2\}\\\|\\delta\\mathbf\{x\}\\\|\_\{2\}\. Finally, from this result and the relationship between Eq\. \([29](https://arxiv.org/html/2605.30370#Sx4.E29)\) and \([30](https://arxiv.org/html/2605.30370#Sx4.E30)\), and under the assumption that bothIBNNandSMshare a similarA\\mathrm\{A\}and are working under the same locally\-linear region of the non\-linear activationΦ\\Phiand thusD​Φ\|A\(𝐱\)=D​Φ\|\(F∘A\)\(𝐱\)\\mathrm\{D\}\{\\Phi\}\_\{\|\\mathrm\{A\}\(\\mathbf\{x\}\)\}=\\mathrm\{D\}\{\\Phi\}\_\{\|\(\\mathrm\{F\}\\circ\\mathrm\{A\}\)\(\\mathbf\{x\}\)\}, we have

‖D​IBNN⁡δ​𝐱‖2‖δ​𝐱‖2≤‖D​SM⁡δ​𝐱‖2‖δ​𝐱‖2,\\frac\{\\\|\\mathrm\{D\}\{\\,\\operatorname\{IBNN\}\}\\;\\delta\\mathbf\{x\}\\\|\_\{2\}\}\{\\\|\\delta\\mathbf\{x\}\\\|\_\{2\}\}\\leq\\frac\{\\\|\\mathrm\{D\}\{\\,\\operatorname\{SM\}\}\\;\\delta\\mathbf\{x\}\\\|\_\{2\}\}\{\\\|\\delta\\mathbf\{x\}\\\|\_\{2\}\}\\,,\(34\)
which would suggest, according to the reasoning in\[[23](https://arxiv.org/html/2605.30370#bib.bib23)\], a naturally improved robustness against adversarial attacks\.

The above analysis for individualIBNNandSMlayers also applies to networks having a backbone composed of a sequence of them\. Suppose that we have respectiveIBNN\- andSM\-based backbones ofLLlayers according to

IBNN\(1:L\)=IBNN\(L\)∘⋯∘IBNN\(l\)∘⋯∘IBNN\(1\),with​IBNN\(l\)=Φ∘F\(l\)​\(⋅;λ\(l\)\)∘A\(l\)\\operatorname\{IBNN\}^\{\(1:L\)\}=\\operatorname\{IBNN\}^\{\(L\)\}\\circ\\cdots\\circ\\operatorname\{IBNN\}^\{\(l\)\}\\circ\\cdots\\circ\\operatorname\{IBNN\}^\{\(1\)\}\\;,\\quad\\textrm\{with\}\\;\\operatorname\{IBNN\}^\{\(l\)\}=\\Phi\\circ\\mathrm\{F\}^\{\(l\)\}\(\\cdot;\\lambda^\{\(l\)\}\)\\circ\\mathrm\{A\}^\{\(l\)\}\(35\)
and

SM\(1:L\)=SM\(L\)∘⋯∘SM\(l\)∘⋯∘SM\(1\),with​SM\(l\)=Φ∘A\(l\)\.\\operatorname\{SM\}^\{\(1:L\)\}=\\operatorname\{SM\}^\{\(L\)\}\\circ\\cdots\\circ\\operatorname\{SM\}^\{\(l\)\}\\circ\\cdots\\circ\\operatorname\{SM\}^\{\(1\)\}\\;,\\quad\\textrm\{with\}\\;\\operatorname\{SM\}^\{\(l\)\}=\\Phi\\circ\\mathrm\{A\}^\{\(l\)\}\\,\.\(36\)
The application of the chain rule, along with Eq\. \([29](https://arxiv.org/html/2605.30370#Sx4.E29)\) and \([30](https://arxiv.org/html/2605.30370#Sx4.E30)\), yields, respectively,

D​IBNN\(1:L\)=D​IBNN\(L\)⋯D​IBNN\(l\)⋯D​IBNN\(1\)=D​Φ\(L\)​K∞\(L\)​DA\(L\)⋯D​Φ\(l\)​K∞\(l\)​DA\(l\)⋯D​Φ\(1\)​K∞\(1\)​DA\(1\)\\begin\{matrix\}\[c\]\\mathrm\{D\}\{\\operatorname\{IBNN\}^\{\(1:L\)\}\}&=&\\mathrm\{D\}\{\\operatorname\{IBNN\}^\{\(L\)\}\}&\\cdots&\\mathrm\{D\}\{\\operatorname\{IBNN\}^\{\(l\)\}\}&\\cdots&\\mathrm\{D\}\{\\operatorname\{IBNN\}^\{\(1\)\}\}\\\\ &=&\\mathrm\{D\}\{\\Phi^\{\(L\)\}\}\\;\\mathrm\{K\}^\{\(L\)\}\_\{\\infty\}\\;\\mathrm\{D\}\{\\mathrm\{A\}^\{\(L\)\}\}&\\cdots&\\mathrm\{D\}\{\\Phi^\{\(l\)\}\}\\;\\mathrm\{K\}^\{\(l\)\}\_\{\\infty\}\\;\\mathrm\{D\}\{\\mathrm\{A\}^\{\(l\)\}\}&\\cdots&\\mathrm\{D\}\{\\Phi^\{\(1\)\}\}\\;\\mathrm\{K\}^\{\(1\)\}\_\{\\infty\}\\;\\mathrm\{D\}\{\\mathrm\{A\}^\{\(1\)\}\}\\end\{matrix\}\(37\)
and

D​SM\(1:L\)=D​SM\(L\)⋯D​SM\(l\)⋯D​SM\(1\)=D​Φ\(L\)​IdD×D​DA\(L\)⋯D​Φ\(l\)​IdD×D​DA\(l\)⋯D​Φ\(1\)​IdD×D​DA\(1\)\.\\begin\{matrix\}\[c\]\\mathrm\{D\}\{\\operatorname\{SM\}^\{\(1:L\)\}\}&=&\\mathrm\{D\}\{\\operatorname\{SM\}^\{\(L\)\}\}&\\cdots&\\mathrm\{D\}\{\\operatorname\{SM\}^\{\(l\)\}\}&\\cdots&\\mathrm\{D\}\{\\operatorname\{SM\}^\{\(1\)\}\}\\\\ &=&\\mathrm\{D\}\{\\Phi^\{\(L\)\}\}\\;\\textrm\{Id\}\_\{\{D\}\\\!\\times\\\!\{D\}\}\\;\\mathrm\{D\}\{\\mathrm\{A\}^\{\(L\)\}\}&\\cdots&\\mathrm\{D\}\{\\Phi^\{\(l\)\}\}\\;\\textrm\{Id\}\_\{\{D\}\\\!\\times\\\!\{D\}\}\\;\\mathrm\{D\}\{\\mathrm\{A\}^\{\(l\)\}\}&\\cdots&\\mathrm\{D\}\{\\Phi^\{\(1\)\}\}\\;\\textrm\{Id\}\_\{\{D\}\\\!\\times\\\!\{D\}\}\\;\\mathrm\{D\}\{\\mathrm\{A\}^\{\(1\)\}\}\\,\.\\end\{matrix\}\(38\)
For eachIBNNlayer withλ\(l\)≤0\\lambda^\{\(l\)\}\\leq 0we have that,∀δ​𝐲∈ℝD\\forall\\delta\\mathbf\{y\}\\in\\mathbb\{R\}^\{D\},‖K∞\(l\)​δ​𝐲‖2=k\(l\)​‖δ​𝐲‖2\\\|\\mathrm\{K\}^\{\(l\)\}\_\{\\infty\}\\,\\delta\\mathbf\{y\}\\\|\_\{2\}\\,=k^\{\(l\)\}\\\|\\delta\\mathbf\{y\}\\\|\_\{2\}for somek\(l\)∈\(0,1\]k^\{\(l\)\}\\in\(0,1\]dependent on the specificδ​𝐲\\delta\\mathbf\{y\}\. Using sequentially this fact along with the definition of the norm∥⋅∥2\\\|\\cdot\\\|\_\{2\}yields

‖D​IBNN\(1:L\)⁡δ​𝐱‖2=‖D​Φ\(L\)​K∞\(L\)​DA\(L\)​⋯​D​Φ\(l\)​K∞\(l\)​DA\(l\)​⋯​D​Φ\(1\)​K∞\(1\)​DA\(1\)​δ​𝐱‖2≤‖D​Φ\(L\)‖2​‖K∞\(L\)​DA\(L\)​⋯​D​Φ\(l\)​K∞\(l\)​DA\(l\)​⋯​D​Φ\(1\)​K∞\(1\)​DA\(1\)​δ​𝐱‖2=k\(L\)​‖D​Φ\(L\)‖2​‖DA\(L\)​⋯​D​Φ\(l\)​K∞\(l\)​DA\(l\)​⋯​D​Φ\(1\)​K∞\(1\)​DA\(1\)​δ​𝐱‖2⋮≤k\(L\)​⋯​k\(1\)​\(‖D​Φ\(L\)‖2​‖DA\(L\)‖2​⋯​‖D​Φ\(1\)‖2​‖DA\(1\)‖2\)​‖δ​𝐱‖2\.\\begin\{split\}\\\|\\mathrm\{D\}\{\\operatorname\{IBNN\}^\{\(1:L\)\}\}\\;\\delta\\mathbf\{x\}\\\|\_\{2\}&=\\Big\\\|\\;\\mathrm\{D\}\{\\Phi^\{\(L\)\}\}\\;\\mathrm\{K\}^\{\(L\)\}\_\{\\infty\}\\;\\mathrm\{D\}\{\\mathrm\{A\}^\{\(L\)\}\}\\;\\cdots\\;\\mathrm\{D\}\{\\Phi^\{\(l\)\}\}\\;\\mathrm\{K\}^\{\(l\)\}\_\{\\infty\}\\;\\mathrm\{D\}\{\\mathrm\{A\}^\{\(l\)\}\}\\;\\cdots\\;\\mathrm\{D\}\{\\Phi^\{\(1\)\}\}\\;\\mathrm\{K\}^\{\(1\)\}\_\{\\infty\}\\;\\mathrm\{D\}\{\\mathrm\{A\}^\{\(1\)\}\}\\;\\delta\\mathbf\{x\}\\;\\Big\\\|\_\{2\}\\\\ &\\leq\\\|\\mathrm\{D\}\{\\Phi^\{\(L\)\}\}\\\|\_\{2\}\\;\\Big\\\|\\;\\mathrm\{K\}^\{\(L\)\}\_\{\\infty\}\\;\\mathrm\{D\}\{\\mathrm\{A\}^\{\(L\)\}\}\\;\\cdots\\;\\mathrm\{D\}\{\\Phi^\{\(l\)\}\}\\;\\mathrm\{K\}^\{\(l\)\}\_\{\\infty\}\\;\\mathrm\{D\}\{\\mathrm\{A\}^\{\(l\)\}\}\\;\\cdots\\;\\mathrm\{D\}\{\\Phi^\{\(1\)\}\}\\;\\mathrm\{K\}^\{\(1\)\}\_\{\\infty\}\\;\\mathrm\{D\}\{\\mathrm\{A\}^\{\(1\)\}\}\\;\\delta\\mathbf\{x\}\\;\\Big\\\|\_\{2\}\\\\ &=\\;k^\{\(L\)\}\\;\\\|\\mathrm\{D\}\{\\Phi^\{\(L\)\}\}\\\|\_\{2\}\\;\\Big\\\|\\;\\mathrm\{D\}\{\\mathrm\{A\}^\{\(L\)\}\}\\;\\cdots\\;\\mathrm\{D\}\{\\Phi^\{\(l\)\}\}\\;\\mathrm\{K\}^\{\(l\)\}\_\{\\infty\}\\;\\mathrm\{D\}\{\\mathrm\{A\}^\{\(l\)\}\}\\;\\cdots\\;\\mathrm\{D\}\{\\Phi^\{\(1\)\}\}\\;\\mathrm\{K\}^\{\(1\)\}\_\{\\infty\}\\;\\mathrm\{D\}\{\\mathrm\{A\}^\{\(1\)\}\}\\;\\delta\\mathbf\{x\}\\;\\Big\\\|\_\{2\}\\\\ &\\hphantom\{=\}\\hphantom\{\\;k^\{\(L\)\}\\;\\\|\\mathrm\{D\}\{\\Phi^\{\(L\)\}\}\\\|\_\{2\}\\;\}\\vdots\\\\ &\\leq\\;k^\{\(L\)\}\\;\\cdots\\;k^\{\(1\)\}\\;\\left\(\\\|\\mathrm\{D\}\{\\Phi^\{\(L\)\}\}\\\|\_\{2\}\\;\\;\\\|\\mathrm\{D\}\{\\mathrm\{A\}^\{\(L\)\}\}\\\|\_\{2\}\\cdots\\\|\\mathrm\{D\}\{\\Phi^\{\(1\)\}\}\\\|\_\{2\}\\;\\;\\\|\\mathrm\{D\}\{\\mathrm\{A\}^\{\(1\)\}\}\\\|\_\{2\}\\right\)\\\|\\delta\\mathbf\{x\}\\\|\_\{2\}\\;\.\\end\{split\}\(39\)
The fact that we have, analogously,

‖D​SM\(1:L\)⁡δ​𝐱‖2=‖D​Φ\(L\)​DA\(L\)​⋯​D​Φ\(1\)​DA\(1\)​δ​𝐱‖2≤\(‖D​Φ\(L\)‖2​‖DA\(L\)‖2​⋯​‖D​Φ\(1\)‖2​‖DA\(1\)‖2\)​‖δ​𝐱‖2,\\begin\{split\}\\\|\\mathrm\{D\}\{\\operatorname\{SM\}^\{\(1:L\)\}\}\\;\\delta\\mathbf\{x\}\\\|\_\{2\}&=\\Big\\\|\\mathrm\{D\}\{\\Phi^\{\(L\)\}\}\\;\\mathrm\{D\}\{\\mathrm\{A\}^\{\(L\)\}\}\\;\\cdots\\;\\mathrm\{D\}\{\\Phi^\{\(1\)\}\}\\;\\mathrm\{D\}\{\\mathrm\{A\}^\{\(1\)\}\}\\;\\delta\\mathbf\{x\}\\Big\\\|\_\{2\}\\\\ &\\leq\\left\(\\\|\\mathrm\{D\}\{\\Phi^\{\(L\)\}\}\\\|\_\{2\}\\;\\;\\\|\\mathrm\{D\}\{\\mathrm\{A\}^\{\(L\)\}\}\\\|\_\{2\}\\cdots\\\|\\mathrm\{D\}\{\\Phi^\{\(1\)\}\}\\\|\_\{2\}\\;\\;\\\|\\mathrm\{D\}\{\\mathrm\{A\}^\{\(1\)\}\}\\\|\_\{2\}\\right\)\\\|\\delta\\mathbf\{x\}\\\|\_\{2\}\\;,\\end\{split\}\(40\)
along with the fact thatk\(l\)≤1k^\{\(l\)\}\\leq 1,∀l∈\{1,…,L\}\\forall l\\in\\\{1,\\ldots,L\\\}, suggests that the sensitivity of theIBNN\-based multilayer backbone to perturbations of its input is lower than itsSM\-based counterpart\.

##### Global perturbations

What follows demonstrates that the output of the layers based on the newly proposedIBNNneurons are, globally, less affected by perturbations of their inputs than theirSMcounterparts\. The demonstration stems from the functionals introduced to prove the existence and uniqueness of the solution ofIBNN\(Eq\. \([5](https://arxiv.org/html/2605.30370#Sx4.E5)\),Methods\)\.

###### Proposition 6\.

Let𝐱∈ℝN\\mathbf\{x\}\\in\\mathbb\{R\}^\{N\},𝒩𝐱\\mathcal\{N\}\_\{\\mathbf\{x\}\}a neighborhood of𝐱\\mathbf\{x\}, and𝐲∈𝒩𝐱\\mathbf\{y\}\\in\\mathcal\{N\}\_\{\\mathbf\{x\}\}\. Under the assumption thatλ≤0,\\lambda\\leq 0,the functionSSinfλf\_\{\\lambda\}is convex, andΦ\\Phiis a similarity inℱ\\mathcal\{F\}⊂ℝD\\subset\\mathbb\{R\}^\{D\}such that

ℱ⊃\{proxfλ⁡\(A​\(𝒩𝐱\)\)\}∪\{A​\(𝒩𝐱\)\},\\mathcal\{F\}\\supset\\\{\\operatorname\{prox\}\_\{f\_\{\\lambda\}\}\(\\mathrm\{A\}\(\\mathcal\{N\}\_\{\\mathbf\{x\}\}\)\)\\\}\\cup\\\{\\mathrm\{A\}\(\\mathcal\{N\}\_\{\\mathbf\{x\}\}\)\\\},we have

‖IBNN⁡\(𝐱;M,𝐛,λ\)−IBNN⁡\(𝐲;M,𝐛,λ\)‖≤‖SM⁡\(𝐱;M,𝐛\)−SM⁡\(𝐲;M,𝐛\)‖∀𝐲∈𝒩𝐱\.\\\|\\operatorname\{IBNN\}\(\\mathbf\{x\};\\mathrm\{M\},\\mathbf\{b\},\\lambda\)\-\\operatorname\{IBNN\}\(\\mathbf\{y\};\\mathrm\{M\},\\mathbf\{b\},\\lambda\)\\\|\\;\\leq\\;\\\|\\operatorname\{SM\}\(\\mathbf\{x\};\\mathrm\{M\},\\mathbf\{b\}\)\-\\operatorname\{SM\}\(\\mathbf\{y\};\\mathrm\{M\},\\mathbf\{b\}\)\\\|\\qquad\\forall\\mathbf\{y\}\\in\\mathcal\{N\}\_\{\\mathbf\{x\}\}\.

###### Proof\.

If

λ≤0\\lambda\\leq 0and the function

SSin Eq\. \([5](https://arxiv.org/html/2605.30370#Sx4.E5)\) is convex, then

−fλ\-f\_\{\\lambda\}is convex\. Together with the fact that

−fλ\-f\_\{\\lambda\}is continuous and proper, we have that the variational problem

arg⁡min𝑧​Eλ​\(z;A​\(𝐱\)\)\\underset\{z\}\{\\arg\\min\}\\,E\_\{\\lambda\}\(z;\\mathrm\{A\}\(\\mathbf\{x\}\)\)is the proximal operator

prox−fλ⁡\(A​\(𝐱\)\)\\operatorname\{prox\}\_\{\-f\_\{\\lambda\}\}\(\\mathrm\{A\}\(\\mathbf\{x\}\)\)of

−fλ\-f\_\{\\lambda\}evaluated at

A​\(𝐱\)\\mathrm\{A\}\(\\mathbf\{x\}\), which leads to the existence and uniqueness of the solution of Eq\. \([4](https://arxiv.org/html/2605.30370#Sx2.E4)\) and consequently of

IBNN⁡\(𝐱;M,𝐛,λ\)∀𝐱∈ℝN\\operatorname\{IBNN\}\(\\mathbf\{x\};\\mathrm\{M\},\\mathbf\{b\},\\lambda\)\\quad\\forall\\,\\,\\mathbf\{x\}\\in\\mathbb\{R\}^\{N\}\.

By property of proximal operators, we then have that

‖prox−fλ⁡\(A​\(𝐱\)\)−prox−fλ⁡\(A​\(𝐲\)\)‖≤‖A​\(𝐱\)−A​\(𝐲\)‖∀𝐱,𝐲∈ℝN\.\\\|\\operatorname\{prox\}\_\{\-f\_\{\\lambda\}\}\(\\mathrm\{A\}\(\\mathbf\{x\}\)\)\-\\operatorname\{prox\}\_\{\-f\_\{\\lambda\}\}\(\\mathrm\{A\}\(\\mathbf\{y\}\)\)\\\|\\;\\leq\\;\\\|\\mathrm\{A\}\(\\mathbf\{x\}\)\-\\mathrm\{A\}\(\\mathbf\{y\}\)\\\|\\qquad\\forall\\,\\mathbf\{x\},\\mathbf\{y\}\\in\\mathbb\{R\}^\{N\}\.We deduce that, under the assumption thatΦ\\Phiis a similarity, i\.e\. an isometry scaled by a constant, inℱ⊂ℝN\\mathcal\{F\}\\subset\\mathbb\{R\}^\{N\}such that

ℱ⊃\{prox−fλ⁡\(A​\(𝒩𝐱\)\)\}∪\{A​\(𝒩𝐱\)\},\\mathcal\{F\}\\supset\\\{\\operatorname\{prox\}\_\{\-f\_\{\\lambda\}\}\(\\mathrm\{A\}\(\\mathcal\{N\}\_\{\\mathbf\{x\}\}\)\)\\\}\\cup\\\{\\mathrm\{A\}\(\\mathcal\{N\}\_\{\\mathbf\{x\}\}\)\\\}\\,,we have

∥Φ\(prox−fλ\(A\(𝐱\)\)−Φ\(prox−fλ\(A\(𝐲\)\)∥≤∥Φ\(A\(𝐱\)\)−Φ\(A\(𝐲\)\)∥∀𝐲∈𝒩𝐱,\\\|\\Phi\(\\operatorname\{prox\}\_\{\-f\_\{\\lambda\}\}\(\\mathrm\{A\}\(\\mathbf\{x\}\)\)\-\\Phi\(\\operatorname\{prox\}\_\{\-f\_\{\\lambda\}\}\(\\mathrm\{A\}\(\\mathbf\{y\}\)\)\\\|\\;\\leq\\;\\\|\\Phi\(\\mathrm\{A\}\(\\mathbf\{x\}\)\)\-\\Phi\(\\mathrm\{A\}\(\\mathbf\{y\}\)\)\\\|\\qquad\\forall\\mathbf\{y\}\\in\\mathcal\{N\}\_\{\\mathbf\{x\}\}\\,,i\.e\.

‖IBNN⁡\(𝐱;M,𝐛,λ\)−IBNN⁡\(𝐲;M,𝐛,λ\)‖≤‖SM⁡\(𝐱;M,𝐛\)−SM⁡\(𝐲;M,𝐛\)‖\.\\\|\\operatorname\{IBNN\}\(\\mathbf\{x\};\\mathrm\{M\},\\mathbf\{b\},\\lambda\)\-\\operatorname\{IBNN\}\(\\mathbf\{y\};\\mathrm\{M\},\\mathbf\{b\},\\lambda\)\\\|\\;\\leq\\;\\\|\\operatorname\{SM\}\(\\mathbf\{x\};\\mathrm\{M\},\\mathbf\{b\}\)\-\\operatorname\{SM\}\(\\mathbf\{y\};\\mathrm\{M\},\\mathbf\{b\}\)\\\|\\,\.∎

#### Inconclusive evidence about accuracy\-stability tradeoff, unlike for theSM

Bastouniset al\.\[[6](https://arxiv.org/html/2605.30370#bib.bib6)\]demonstrated that multilayerSM\-basedANNswith continuous nonlinear activations such as ReLU achieving a successful accuracy become universally unstable \(see\[[6](https://arxiv.org/html/2605.30370#bib.bib6)\]Theorem 2\.2 \(ii\)\), which represents a formal proof thatSM\-based nets are subject to an inherent tradeoff between accurate and stability\.

Their proof depends on the standard deep neural network structureΨ\\PsiwhoseLLlayers follow the form of Eqs\. \([1](https://arxiv.org/html/2605.30370#Sx2.E1)\)\-\([2](https://arxiv.org/html/2605.30370#Sx2.E2)\); that is, and using the notation of Eq\. \([15](https://arxiv.org/html/2605.30370#Sx4.E15)\) and \([18](https://arxiv.org/html/2605.30370#Sx4.E18)\), it can be expressed as the compositionΨ=A\(L\)∘Φ∘A\(L−1\)∘⋯∘Φ∘A\(l\)​⋯∘Φ∘A\(1\)\\Psi=\\mathrm\{A\}^\{\(L\)\}\\circ\\Phi\\circ\\mathrm\{A\}^\{\(L\-1\)\}\\circ\\cdots\\circ\\Phi\\circ\\mathrm\{A\}^\{\(l\)\}\\cdots\\circ\\Phi\\circ\\mathrm\{A\}^\{\(1\)\}, whereΦi=ϕ\\Phi\_\{i\}=\\phiis a continuous nonlinear function applied coordinate\-wise such as ReLU andA\(l\)\\mathrm\{A\}^\{\(l\)\}are affine functions\. A key factor in their proof depends on this standard network structure affording a linearization on finite but arbitrarily large sets \(\[[6](https://arxiv.org/html/2605.30370#bib.bib6)\]lemma 5\.5\), a property which does not directly carry over to the implicit and highly nonlinearIBNNformulation\. This linearization is critical in the construction of arbitrarily large sets where the output of the standard model neural network with fixed dimensions fails to match the true classification function output \(\[[6](https://arxiv.org/html/2605.30370#bib.bib6)\]lemma 5\.4\)\. Thus the accuracy\-stability tradeoff established in\[[6](https://arxiv.org/html/2605.30370#bib.bib6)\]does not directly apply to theIBNN\.

#### IBNNis resistant to single\-neuron stealth attacks

We define anANNcomposed of one single hidden layer according to theIBNNparadigm and with uniform weightswi​kw\_\{ik\}so

zi​\(𝐱\)\\displaystyle z\_\{i\}\(\\mathbf\{x\}\)=𝐦iT​𝐱−bi−λD​∑k=1Dσ​\(zk​\(𝐱\)−zi​\(𝐱\)\)\\displaystyle=\\mathbf\{m\}^\{T\}\_\{i\}\\mathbf\{x\}\-b\_\{i\}\\,\-\\,\\frac\{\\lambda\}\{D\}\\sum\_\{k=1\}^\{D\}\\sigma\\\!\\left\(z\_\{k\}\(\\mathbf\{x\}\)\\\!\-\\\!z\_\{i\}\(\\mathbf\{x\}\)\\right\)\(41\)vi​\(𝐱\)\\displaystyle v\_\{i\}\(\\mathbf\{x\}\)=ϕ​\(zi​\(𝐱\)\)\\displaystyle=\\phi\(z\_\{i\}\(\\mathbf\{x\}\)\)\(42\)
and one single output layer so the output score of the resultingIBNNANNis:

s​\(𝐱\)=∑i=1Nci​vi​\(𝐱\)\.s\(\\mathbf\{x\}\)=\\sum\_\{i=1\}^\{N\}c\_\{i\}v\_\{i\}\(\\mathbf\{x\}\)\\,\.\(43\)
Following\[[24](https://arxiv.org/html/2605.30370#bib.bib24)\], we say that a successful single\-neuron stealth attack on theIBNNANNyielding the scoring functionsswould be a modifiedIBNNANNwith the same artificial units of the original plus an extraIBNNneuron and yielding a scoring functionsas\_\{a\}such, given a verification input set𝒱\\mathcal\{V\}, known to the owner of the systemssbut unknown to the attacker, andϵ≥0\\epsilon\\geq 0,Δ\>0\\Delta\>0,

1. 1\.for some input𝐱T\\mathbf\{x\}\_\{T\}, known to the attacker but not to the owner of the network,\|sa​\(𝐱T\)−s​\(𝐱T\)\|≥Δ\|s\_\{a\}\(\\mathbf\{x\}\_\{T\}\)\-s\(\\mathbf\{x\}\_\{T\}\)\|\\geq\\Delta, and
2. 2\.for every input𝐱v∈𝒱\\mathbf\{x\}\_\{v\}\\in\\mathcal\{V\},\|sa​\(𝐱v\)−s​\(𝐱v\)\|≤ϵ\|s\_\{a\}\(\\mathbf\{x\}\_\{v\}\)\-s\(\\mathbf\{x\}\_\{v\}\)\|\\leq\\epsilon\.

Condition \(1\) implies that the attacker makes the ‘hacked’ networksas\_\{a\}respond to a trigger input𝐱T\\mathbf\{x\}\_\{T\}in a way that is different from what the originalssdoes, whereas condition \(2\) implies that the system owner will not be able to tell that an attack has taken place, that the original networksshas been modified, because the responses of the hacked networksas\_\{a\}on𝒱\\mathcal\{V\}are very similar to those of the original networkss\.

The hacked networksas\_\{a\}is

sa​\(𝐱\)\\displaystyle s\_\{a\}\(\\mathbf\{x\}\)=∑i=1Nci​v^i​\(𝐱\)\+ca​v^N\+1​\(𝐱\)\\displaystyle=\\sum\_\{i=1\}^\{N\}c\_\{i\}\\hat\{v\}\_\{i\}\(\\mathbf\{x\}\)\+c\_\{a\}\\hat\{v\}\_\{N\+1\}\(\\mathbf\{x\}\)\(44\)v^i​\(𝐱\)\\displaystyle\\hat\{v\}\_\{i\}\(\\mathbf\{x\}\)=ϕ​\(z^i​\(I\)\)\\displaystyle=\\phi\(\\hat\{z\}\_\{i\}\(I\)\)\(45\)z^i​\(𝐱\)\\displaystyle\\hat\{z\}\_\{i\}\(\\mathbf\{x\}\)=𝐦iT​𝐱−bi−λD​∑k=1D\+1σ​\(z^k​\(𝐱\)−z^i​\(𝐱\)\)\\displaystyle=\\mathbf\{m\}^\{T\}\_\{i\}\\mathbf\{x\}\-b\_\{i\}\\,\-\\,\\frac\{\\lambda\}\{D\}\\sum\_\{k=1\}^\{D\+1\}\\sigma\\\!\\left\(\\hat\{z\}\_\{k\}\(\\mathbf\{x\}\)\\\!\-\\\!\\hat\{z\}\_\{i\}\(\\mathbf\{x\}\)\\right\)\(46\)
Let𝐱T\\mathbf\{x\}\_\{T\}be a trigger, such that the magnitude of the attack it produces isΔ\\Delta; takingϕ\\phito be the ReLU function and assuming that the network units are working in the linear range ofϕ\\phi, we have

Δ=\|sa​\(𝐱T\)−s​\(𝐱T\)\|=\|∑iDci​\(v^i​\(𝐱T\)−vi​\(𝐱T\)\)\+ca​v^D\+1​\(𝐱T\)\|\.\\Delta=\|s\_\{a\}\(\\mathbf\{x\}\_\{T\}\)\-s\(\\mathbf\{x\}\_\{T\}\)\|=\\left\|\\sum\_\{i\}^\{D\}c\_\{i\}\\Big\(\\hat\{v\}\_\{i\}\(\\mathbf\{x\}\_\{T\}\)\-v\_\{i\}\(\\mathbf\{x\}\_\{T\}\)\\Big\)\+c\_\{a\}\\hat\{v\}\_\{D\+1\}\(\\mathbf\{x\}\_\{T\}\)\\right\|\\,\.\(47\)
IfD≫1D\\gg 1we may assume that adding a single neuron does not change significantly the sum∑k=1Dσ​\(zj​\(𝐱\)−zi​\(𝐱\)\)\\sum\_\{k=1\}^\{D\}\\sigma\(z\_\{j\}\(\\mathbf\{x\}\)\-z\_\{i\}\(\\mathbf\{x\}\)\), since only one term is added to the summation; therefore, we havezi^≃zi\\hat\{z\_\{i\}\}\\simeq z\_\{i\}and hence

Δ≃\|ca​z^D\+1​\(𝐱T\)\|\.\\Delta\\simeq\\left\|c\_\{a\}\\;\\hat\{z\}\_\{D\+1\}\(\\mathbf\{x\}\_\{T\}\)\\right\|\\,\.\(48\)
Additional, we have

z^D\+1​\(𝐱T\)=𝐦D\+1T​𝐱T−λD​∑k=1D\+1σ​\(z^k​\(𝐱T\)−z^D\+1​\(𝐱T\)\),\\hat\{z\}\_\{D\+1\}\(\\mathbf\{x\}\_\{T\}\)=\\mathbf\{m\}^\{T\}\_\{D\+1\}\\mathbf\{x\}\_\{T\}\\,\-\\,\\frac\{\\lambda\}\{D\}\\sum\_\{k=1\}^\{D\+1\}\\sigma\(\\hat\{z\}\_\{k\}\(\\mathbf\{x\}\_\{T\}\)\-\\hat\{z\}\_\{D\+1\}\(\\mathbf\{x\}\_\{T\}\)\),\(49\)
where, for simplicity, we have chosenbD\+1=0b\_\{D\+1\}=0\. Combining Eqs\. \([48](https://arxiv.org/html/2605.30370#Sx4.E48)\) and \([49](https://arxiv.org/html/2605.30370#Sx4.E49)\) we get

\|Δca\|\\displaystyle\\left\|\\frac\{\\Delta\}\{c\_\{a\}\}\\right\|=\|𝐦D\+1T​𝐱T−λD​∑k=1D\+1σ​\(z^k​\(𝐱T\)±Δca\)\|\\displaystyle=\\bigg\|\\,\\mathbf\{m\}^\{T\}\_\{D\+1\}\\mathbf\{x\}\_\{T\}\\,\-\\,\\frac\{\\lambda\}\{D\}\\sum\_\{k=1\}^\{D\+1\}\\sigma\\Big\(\\hat\{z\}\_\{k\}\(\\mathbf\{x\}\_\{T\}\)\\pm\\frac\{\\Delta\}\{c\_\{a\}\}\\Big\)\\,\\bigg\|\(50\)𝐦D\+1\\displaystyle\\mathbf\{m\}\_\{D\+1\}=𝐱T‖𝐱T‖2​\(±Δca±λD​∑k=1D\+1σ​\(z^k​\(𝐱T\)±Δca\)\)\\displaystyle=\\frac\{\\mathbf\{x\}\_\{T\}\}\{\|\|\\mathbf\{x\}\_\{T\}\|\|^\{2\}\}\(\\pm\\frac\{\\Delta\}\{c\_\{a\}\}\\pm\\frac\{\\lambda\}\{D\}\\sum\_\{k=1\}^\{D\+1\}\\sigma\(\\hat\{z\}\_\{k\}\(\\mathbf\{x\}\_\{T\}\)\\pm\\frac\{\\Delta\}\{c\_\{a\}\}\)\)\(51\)𝐦D\+1\\displaystyle\\mathbf\{m\}\_\{D\+1\}∝𝐱T\\displaystyle\\propto\\mathbf\{x\}\_\{T\}\(52\)
This proves that we can find a filter𝐦D\+1\\mathbf\{m\}\_\{D\+1\}so that the input𝐱T\\mathbf\{x\}\_\{T\}produces very different outputs inssandsas\_\{a\}: the difference between the outputs of both networks has an amplitude ofΔ\\Delta, which would fulfill the first requirement of the stealth attack\.

Now we need to check the second requirement, i\.e\. that the outputs for the verification set remain unaltered, or change very little for𝐱v∈𝒱\\mathbf\{x\}\_\{v\}\\in\\mathcal\{V\}\. Using analogous assumptions to the reasoning above, the differenceΔv\\Delta\_\{v\}betweensa​\(𝐱v\)s\_\{a\}\(\\mathbf\{x\}\_\{v\}\)ands​\(𝐱v\)s\(\\mathbf\{x\}\_\{v\}\)is

Δv≃\|ca​z^D\+1​\(𝐱v\)\|\.\\Delta\_\{v\}\\simeq\\left\|c\_\{a\}\\;\\hat\{z\}\_\{D\+1\}\(\\mathbf\{x\}\_\{v\}\)\\right\|\\,\.\(53\)
We would likeΔv\\Delta\_\{v\}to be negligible, e\.g\.Δv≃0\\Delta\_\{v\}\\simeq 0, which givesz^D\+1​\(𝐱v\)≃0\\hat\{z\}\_\{D\+1\}\(\\mathbf\{x\}\_\{v\}\)\\simeq 0, and from the definition ofz^D\+1\\hat\{z\}\_\{D\+1\}we get

𝐦D\+1T​𝐱v=λD​∑k=1Dσ​\(zk​\(𝐱v\)\)\\mathbf\{m\}^\{T\}\_\{D\+1\}\\mathbf\{x\}\_\{v\}=\\frac\{\\lambda\}\{D\}\\sum\_\{k=1\}^\{D\}\\sigma\(z\_\{k\}\(\\mathbf\{x\}\_\{v\}\)\)\(54\)
So, in order for the stealth attack to be actually stealth, undetectable, all the inputs in the verification set𝒱\\mathcal\{V\}should satisfy Eq\. \([54](https://arxiv.org/html/2605.30370#Sx4.E54)\)\. But this is an implicit equation on𝐱v\\mathbf\{x\}\_\{v\}, whose solutions, if any, might be totally unrelated to the elements of𝒱\\mathcal\{V\}, which is a set that is only known to the system owner, not to the attacker\.

In conclusion, condition \(2\) for the stealth attack is not satisfied, and therefore we can say that single\-neuron stealth attacks on aIBNNare unfeasible\.

### Resistance to memorization

Here we show that IBNN is less prone to memorization by following the procedure suggested by Garget al\.\[[30](https://arxiv.org/html/2605.30370#bib.bib30)\], where the tendency of an ANN to memorize its training data is evaluated by computing the curvature of the loss function with respect to the input: lower curvature means less memorization\.

LetΨ\\Psibe a binary classifier with a single hidden layer and afully\-connected\(FC\)\-based scorers:ℝD→ℝs:\\mathbb\{R\}^\{D\}\\to\\mathbb\{R\}according to

s​\(𝐱\)=𝐜T​IBNN⁡\(𝐱\)−c0,s\(\\mathbf\{x\}\)=\\mathbf\{c\}^\{T\}\\,\\operatorname\{IBNN\}\(\\mathbf\{x\}\)\-c\_\{0\}\\,,
and a subsequent probability assignment according toΨ​\(𝐱\)=P​\(s​\(𝐱\)\)\\Psi\(\\mathbf\{x\}\)=P\\big\(s\(\\mathbf\{x\}\)\\big\), where𝐜∈ℝD\\mathbf\{c\}\\in\\mathbb\{R\}^\{D\}, andc0∈ℝc\_\{0\}\\in\\mathbb\{R\}and whereP:ℝ→\[0,1\]P:\\mathbb\{R\}\\to\[0,1\]is a function providing the probability of the class11\(for two classes, with respective labels0and11\), e\.g\. a softmax summarization makingyythe probability of choosing the label11, complementary of the probability of choosing the label0\. Additionally, let the loss for the training pair\(𝐱,yt\)\(\\mathbf\{x\},y\_\{t\}\)\(ttfor*truth*\) be theMean Squared Error\(MSE\), that is,L​\(𝐱;yt\)=\(Ψ​\(𝐱\)−yt\)2L\(\\mathbf\{x\};y\_\{t\}\)=\(\\Psi\(\\mathbf\{x\}\)\-y\_\{t\}\)^\{2\}\. The derivative of the loss with respect to the input is, by virtue of the chain rule,

∂LIBNN∂𝐱=2​\(Ψ​\(𝐱\)−yt\)​P′​\(s​\(𝐱\)\)​𝐜T​D​IBNN\|𝐱\.\\frac\{\\partial L\_\{\\operatorname\{IBNN\}\}\}\{\\partial\\mathbf\{x\}\}\\;=\\;2\\Big\(\\Psi\(\\mathbf\{x\}\)\-y\_\{t\}\\Big\)\\;P^\{\\prime\}\(s\(\\mathbf\{x\}\)\)\\;\\mathbf\{c\}^\{T\}\\;\\mathrm\{D\}\{\\,\\operatorname\{IBNN\}\}\_\{\|\\mathbf\{x\}\}\\,\.\(55\)
An analogous expression∂LSM∂𝐱\\frac\{\\partial L\_\{\\operatorname\{SM\}\}\}\{\\partial\\mathbf\{x\}\}would correspond to an analogousSM\-based network,*mutatis mutandis*\.

The exact expression forD​IBNN\|𝐱\\mathrm\{D\}\{\\,\\operatorname\{IBNN\}\}\_\{\|\\mathbf\{x\}\}in Eq\. \([55](https://arxiv.org/html/2605.30370#Sx4.E55)\) is provided by Eq\. \([29](https://arxiv.org/html/2605.30370#Sx4.E29)\) — it is the product of the termsD​Φ\|\(F∘A\)\(𝐱\)\\mathrm\{D\}\{\\Phi\}\_\{\|\(\\mathrm\{F\}\\circ\\mathrm\{A\}\)\(\\mathbf\{x\}\)\},K∞\|\(F∘A\)​\(𝐱\)\\mathrm\{K\}\_\{\\infty\|\(\\mathrm\{F\}\\circ\\mathrm\{A\}\)\(\\mathbf\{x\}\)\}, andDA\\mathrm\{D\}\{\\mathrm\{A\}\}, which can be approximated as follows:

- •D​Φ≈IdD×D\\mathrm\{D\}\{\\Phi\}\\approx\\textrm\{Id\}\_\{\{D\}\\\!\\times\\\!\{D\}\}, assuming that theIBNNouter activation is aReLU\\operatorname\{ReLU\}and that the network is working within its linear range;
- •K∞\|\(F∘A\)​\(𝐱\)≈IdD×D−λ​DB\|A\(𝐱\)\\mathrm\{K\}\_\{\\infty\|\(\\mathrm\{F\}\\circ\\mathrm\{A\}\)\(\\mathbf\{x\}\)\}\\approx\\textrm\{Id\}\_\{\{D\}\\\!\\times\\\!\{D\}\}\-\\,\\lambda\\,\\mathrm\{D\}\{\\mathrm\{B\}\}\_\{\|\\mathrm\{A\}\(\\mathbf\{x\}\)\}, assuming\|λ\|≪1\|\\lambda\|\\ll 1,\(F∘A\)​\(𝐱\)≈A​\(𝐱\)\(\\mathrm\{F\}\\circ\\mathrm\{A\}\)\(\\mathbf\{x\}\)\\approx\\mathrm\{A\}\(\\mathbf\{x\}\)and using the first\-order Taylor expansion of Eq\. \([28](https://arxiv.org/html/2605.30370#Sx4.E28)\)\.

Accordingly,D​IBNN\|𝐱\\mathrm\{D\}\{\\,\\operatorname\{IBNN\}\}\_\{\|\\mathbf\{x\}\}can be approximated as

D​IBNN\|𝐱≈\(IdD×D−λ​DB\|A\(𝐱\)\)​DA\\mathrm\{D\}\{\\,\\operatorname\{IBNN\}\}\_\{\|\\mathbf\{x\}\}\\approx\\Big\(\\textrm\{Id\}\_\{\{D\}\\\!\\times\\\!\{D\}\}\-\\,\\lambda\\,\\mathrm\{D\}\{\\mathrm\{B\}\}\_\{\|\\mathrm\{A\}\(\\mathbf\{x\}\)\}\\Big\)\\mathrm\{D\}\{\\mathrm\{A\}\}\(56\)
which, in scalar notation and identifyingzi=IBNNi⁡\(𝐱\)z\_\{i\}=\\operatorname\{IBNN\}\_\{i\}\(\\mathbf\{x\}\), is

∂zi∂xj≈mi​j−λ​∑kwi​k​σ′​\(\(𝐦kT−𝐦iT\)​𝐱\)​\(mk​j−mi​j\)\.\\frac\{\\partial z\_\{i\}\}\{\\partial x\_\{j\}\}\\approx m\_\{ij\}\-\\lambda\\sum\_\{k\}w\_\{ik\}\\sigma^\{\\prime\}\\Big\(\(\\mathbf\{m\}^\{T\}\_\{k\}\-\\mathbf\{m\}^\{T\}\_\{i\}\)\\mathbf\{x\}\\Big\)\(m\_\{kj\}\-m\_\{ij\}\)\\,\.\(57\)
Assuming thatσ\\sigmais piece\-wise linear with central slopepp\(i\.e\.σ​\(z\)=−1\\sigma\(z\)=\-1,∀z<−1p\\forall z<\-\\frac\{1\}\{p\};σ​\(z\)=1\\sigma\(z\)=1,∀z\>1p\\forall z\>\\frac\{1\}\{p\}; andσ​\(z\)=p​z\\sigma\(z\)=pz,∀\|z\|<1p\\forall\|z\|<\\frac\{1\}\{p\}\), that the weightswi​kw\_\{ik\}are uniform withwi​k=1Dw\_\{ik\}=\\frac\{1\}\{D\},∀i,k∈\{1,…,D\}\\forall i,k\\in\\\{1,\\ldots,D\\\}, and that the norm of𝐦iT​𝐱\\mathbf\{m\}^\{T\}\_\{i\}\\mathbf\{x\}does not vary significantly for different unitsiiand thus\|\(𝐦kT−𝐦iT\)​𝐱\|≪1\|\(\\mathbf\{m\}^\{T\}\_\{k\}\-\\mathbf\{m\}^\{T\}\_\{i\}\)\\mathbf\{x\}\|\\ll 1, we can write

∂zi∂xj≈mi​j−λ​p​\(∑k=1Dmk​jD\)\+λ​p​mi​j=\(1\+λ​p\)​mi​j−λ​p​\(1D​∑k=1Dmk​j\)\.\\begin\{split\}\\frac\{\\partial z\_\{i\}\}\{\\partial x\_\{j\}\}\\,\\approx\\,m\_\{ij\}\-\\lambda p\\left\(\\sum\_\{k=1\}^\{D\}\\frac\{m\_\{kj\}\}\{D\}\\right\)\+\\lambda\\,p\\,m\_\{ij\}=\(1\+\\lambda p\)m\_\{ij\}\-\\lambda p\\left\(\\frac\{1\}\{D\}\\sum\_\{k=1\}^\{D\}m\_\{kj\}\\right\)\\,\.\\end\{split\}\(58\)
We now make the common assumption that the neurons have a reduced receptive field, i\.e\. each neural unit receives inputs from a small amount of coordinates of𝐱\\mathbf\{x\}\. Then, the matrix whose rows are the filters𝐦iT\\mathbf\{m\}^\{T\}\_\{i\}is sparse, and with bounded filter elements \(e\.g\.\|mi​j\|≤1\|m\_\{ij\}\|\\leq 1,∀i,j\\forall i,j\) we get\|1D​∑j=1Dmk​j\|≪1\|\\frac\{1\}\{D\}\\sum\_\{j=1\}^\{D\}m\_\{kj\}\|\\ll 1, which, plugged in Eq\. \([58](https://arxiv.org/html/2605.30370#Sx4.E58)\), gives

∂zi∂xj≈\(1\+λ​p\)​mi​j,\\frac\{\\partial z\_\{i\}\}\{\\partial x\_\{j\}\}\\,\\approx\\,\(1\+\\lambda p\)m\_\{ij\}\\,,\(59\)
where, just as a note,λ=0\\lambda=0would correspond to theSM\.

With this relationship between derivatives with respect to the input forIBNNand forSM, and going back to Eq\. \([55](https://arxiv.org/html/2605.30370#Sx4.E55)\), we have

∂LIBNN∂xj≈\(1\+λ​p\)​∂LSM∂xj\.\\frac\{\\partial L\_\{\\operatorname\{IBNN\}\}\}\{\\partial x\_\{j\}\}\\approx\(1\+\\lambda p\)\\frac\{\\partial L\_\{\\operatorname\{SM\}\}\}\{\\partial x\_\{j\}\}\\,\.\(60\)
Since, according to\[[30](https://arxiv.org/html/2605.30370#bib.bib30)\], the magnitude of the curvature of the loss function w\.r\.t\. the input can be estimated as

Curv​\(𝐱\)∝‖∂L​\(𝐱\+h​𝐠\)∂𝐱−∂L\(𝐱\)\)∂𝐱‖2,\\mathrm\{Curv\}\(\\mathbf\{x\}\)\\propto\\left\\\|\\frac\{\\partial L\(\\mathbf\{x\}\+h\\mathbf\{g\}\)\}\{\\partial\\mathbf\{x\}\}\-\\frac\{\\partial L\(\\mathbf\{x\}\)\)\}\{\\partial\\mathbf\{x\}\}\\right\\\|^\{2\}\\,,\(61\)
where𝐠∈ℝN\\mathbf\{g\}\\in\\mathbb\{R\}^\{N\}are vectors sampled from a Rademacher distribution, the above approximation yields

CurvIBNN​\(𝐱\)=\(1\+λ​p\)2​CurvSM​\(𝐱\)\\mathrm\{Curv\}\_\{\\operatorname\{IBNN\}\}\(\\mathbf\{x\}\)=\(1\+\\lambda p\)^\{2\}\\;\\mathrm\{Curv\}\_\{\\operatorname\{SM\}\}\(\\mathbf\{x\}\)\(62\)
Given that that the magnitude of the curvature is a measure of memorization\[[30](https://arxiv.org/html/2605.30370#bib.bib30)\], Eq\.[62](https://arxiv.org/html/2605.30370#Sx4.E62)shows that with a proper choice of\(λ,p\)\(\\lambda,p\)\(s\.t\.\|1\+λ​p\|<1\|1\+\\lambda p\|<1\)IBNNmemorizes less than aSMnetwork\.

### Implementation of theIBNNlayers

TrainableIBNNlayers have been custom\-implemented in Python using the design principles and the utilities provided by PyTorch\[[42](https://arxiv.org/html/2605.30370#bib.bib42)\]for bothANNtraining and evaluation\. The implementedIBNNlayers are analogous to and mostly interchangeable with theSMlayers available in PyTorch, thus easing comparisons\.

Two types ofIBNNlayers, convolutional andfully\-connected\(FC\), have been implemented, which are, respectively, analogous to convolutional andFClayers according to theSMparadigm\. The two types differ in the spatial limitations imposed on the linear filters represented by the elements\{𝐦i,bi\}\\\{\\mathbf\{m\}\_\{i\},b\_\{i\}\\\}\(see, respectively, Eq\. \([2](https://arxiv.org/html/2605.30370#Sx2.E2)\) and \([4](https://arxiv.org/html/2605.30370#Sx2.E4)\)\): convolutional layers have been achieved by shaping the filter elements\{𝐦i,bi\}\\\{\\mathbf\{m\}\_\{i\},b\_\{i\}\\\}to correspond to a square mask and tying them across all the neurons of the same layer\.

As highlighted before, the output𝐯\\mathbf\{v\}of anIBNNlayer for a certain input𝐱\\mathbf\{x\}corresponds to the solution of a system of coupled implicit equations \(Eqs\. \([3](https://arxiv.org/html/2605.30370#Sx2.E3)\)\-\([4](https://arxiv.org/html/2605.30370#Sx2.E4)\)\), which represents a challenge for both forward and backward \(i\.e\. training\) operations\. The library TorchDEQ\[[43](https://arxiv.org/html/2605.30370#bib.bib43)\], providing both operations, is the engine of the implicit calculations of our implementation of theIBNNlayer:

- •The forward operation of theIBNNlayer, which is an implicit, but more specifically, fixed\-point calculation, has been performed using the default forward solvers provided by TorchDEQ, in particular Broyden’s method\[[43](https://arxiv.org/html/2605.30370#bib.bib43)\]\. Additionally, a forward solver specifically tailored toIBNNand based on the minimization of Eq\. \([5](https://arxiv.org/html/2605.30370#Sx4.E5)\) through the procedure of Eq\. \([12](https://arxiv.org/html/2605.30370#Sx4.E12)\) has also been implemented\. Neither showed convergence rates that would advise a preference for one over the other, and they have been used interchangeably\.
- •The backward operation, required for training, has been performed using the default forward solvers provided by TorchDEQ, in particular Broyden’s method\[[43](https://arxiv.org/html/2605.30370#bib.bib43)\]\.

Our implementation of theIBNNlayers allows for networks with independentλ\\lambdas at each layer, and for fixed or trainableλ\\lambdas: the latter option has been used in certain experiments as a sort of tentative hyperparameter optimization\.

### Experimental details

#### Datasets

The image classification experiments were based on three datasets of increasing difficulty level, each one containing1010classes: Fashion\-MNIST\[[44](https://arxiv.org/html/2605.30370#bib.bib44)\], SVHN\[[45](https://arxiv.org/html/2605.30370#bib.bib45)\], and CIFAR\-10\[[46](https://arxiv.org/html/2605.30370#bib.bib46)\]\. These datasets have been used from their respective built\-ins in the module Torchvision\[[47](https://arxiv.org/html/2605.30370#bib.bib47)\]of PyTorch\[[42](https://arxiv.org/html/2605.30370#bib.bib42)\]\. Fashion\-MNIST consists of28×2828\\times 28grayscale images of clothing, for a total of60,00060,000training and10,00010,000test images; their comprised classes are T\-shirt/top, trousers, pullover, dress, coat, sandal, shirt, sneaker, bag, and ankle boot\. SVHN consists of32×3232\\times 32color images of digits from0to99, obtained as croppings of real street images, for a total73,25773,257training and26,03226,032test images\. And CIFAR\-10 consists of32×3232\\times 32color images of animals and vehicles, for a total of50,00050,000training and10,00010,000test images; their comprised classes are airplane, automobile, bird, cat, deer, dog, frog, horse, ship, and truck\.

The visualizations in Fig\.[2](https://arxiv.org/html/2605.30370#Sx2.F2)correspond to a custom 2D dataset, generated using a custom defined mask containing the two concentric arrows hosting the 2D samples of the two separated classes: the samples of each class were generated by ‘accept\-reject sampling’ from a 2D uniform distribution with a support comprising their respective masks\. The resulting 2D samples where generated in the form of 2\-pixel images for compatibility with the image\-oriented implementation of the classifiers\.

#### Network architectures used in the experimental evaluation

The classifierANNsused in the presented experimental evaluation, which we will refer to asuniform convolutional networks\(UCNs\) and which will be denoted using the conventionUCN\(L,C\)\(L,C\), is a simplified architecture composed of:

- •A backbone formed ofLLsequential blocks of analogous characteristics \(hence the nomenclature ‘uniform’\)\. Each blockl∈\{1,…,L\}l\\in\\\{1,\\ldots,L\\\}contains a convolutional layer, defined according to theSMorIBNNdepending on the case, and a batch normalization layer for its output\. In all cases, the convolutional layer maintains \(through zero padding\) the spatial size of its input image but producesCCchannels at its output, and the side of the convolutional filters is in all cases the1515% of the size of its input image\.ReLU\\operatorname\{ReLU\}is used as the output nonlinearityϕ\\phi\(Eq\. \([1](https://arxiv.org/html/2605.30370#Sx2.E1)\) and \([3](https://arxiv.org/html/2605.30370#Sx2.E3)\)\) of all layers\.
- •An output scoring head composed of one singleFClayer \(according to theSM\) with as many neurons as classes contained in the dataset\.

For convolutional layers according toIBNN, additionally, uniform weightswi​k=1Dw\_\{ik\}=\\frac\{1\}\{D\}and a dendritic nonlinearityσ​\(z\)=tanh⁡\(p​z\)\\sigma\(z\)=\\tanh\(pz\)withp=10p\\\!=\\\!10is used\. This value ofppwas chosen for two reasons: it is an intermediate value regarding boundary flexibility \(see Fig\.[2](https://arxiv.org/html/2605.30370#Sx2.F2)\); and it ensures existence and uniqueness ofIBNNin the considered ranges ofλ\\lambda\(see Eq\. \([14](https://arxiv.org/html/2605.30370#Sx4.E14)\)\)\. The parameterλl\\lambda\_\{l\}of each layerl∈\{1,…,L\}l\\in\\\{1,\\ldots,L\\\}is set or trained individually\.

All experiments shown inResultscorrespond to the describedUCNarchitecture\. However, experiments corresponding to different datasets and aimed aspects differ inLLandCC\. For each dataset, all experiments gravitate around a configurationUCN\(Ld,Cd\)\(L\_\{d\},C\_\{d\}\)whoseLdL\_\{d\}andCdC\_\{d\}were set in order to achieve base accuracies in the range of65−90%65\-90\\%\. This central configuration was, for each dataset: for Fashion\-MNIST,UCN\(1,3\)\(1,3\); for SVHN,UCN\(2,3\)\(2,3\); and for CIFAR\-10,UCN\(3,8\)\(3,8\)\. All the experiments correspond to said central configurations but for two exceptions: experiments related to expressivity compare network sizes by varying the number of channels, and thus correspond for each dataset to theLLindicated above but capture different values ofCC\(see Fig\.[6](https://arxiv.org/html/2605.30370#Sx2.F6)\); and the result included in Fig\.[7](https://arxiv.org/html/2605.30370#Sx2.F7)for CIFAR\-10, corresponding toUCN\(3,16\)\(3,16\)is the only departure from these configurations: although bothC=8C\\\!=\\\!8andC=16C\\\!=\\\!16similarly reproduce the same positive trend regarding learning speed, it was determined that the latter illustrates the concept better\.

The visualizations in Fig\.[2](https://arxiv.org/html/2605.30370#Sx2.F2), however, correspond to a network with a backbone of a single hidden layer ofFCtype composed of33neurons \(according to theSMorIBNNdepending on the case\) with aReLU\\operatorname\{ReLU\}output nonlinearity and a batch normalization layer, followed by a single layer head/unit leading to a binary classification\.

#### Training and evaluation details forIBNNandSM\-based networks

Training of bothSM\- andIBNN\-basedANNswas performed usingStochastic Gradient Descent\(SGD\)\. The adoption ofSGD, instead of other more dynamic optimizers such as Adam, is motivated by two reasons: the latter requires the calculation of higher\-order moments and therefore conveys a larger computational burden; and the simplicity ofSGDhas been regarded as more controllable by experimenters with respect to the more obscure optimization steps of the latter\. A scheduler composed of a linear warmup from0\.0010\.001to0\.010\.01in4040epochs and a linear decay symmetric to the former was used for thelearning rate\(LR\)\. This scheduler was chosen because it represents a favorable trade\-off between stability with respect to the initial random weights of the network, reasonable speed due to the higherLRduring the central epochs, and better convergence due to the final cooldown\.

A warmup training step of22epochs was incorporated prior to the training of bothSM\- andIBNN\-basedANNs\. In the specific case of theIBNN\-based nets, this warmup was performed on aSM\-based surrogate with an identical architecture to that of the targetIBNN; the training of theIBNNnet itself starts by initializing its weights with the weights resulting from the warmup of the surrogate \(see Eqs\. \([2](https://arxiv.org/html/2605.30370#Sx2.E2)\) and \([4](https://arxiv.org/html/2605.30370#Sx2.E4)\), and consider also the batch normalization and head of theuniform convolutional network\(UCN\)\)\. This surrogate\-based warmup has been introduced to ensure favorable initializations ofIBNN\-basedANNsand to avoid those rare cases where random initializations without such warmup did not allow training to progress to viable classifiers\.

Each model \(labeled in each case asIBNNorSM\) in each experiment inResultswas obtained as the median of several instances of the same model, trained independently from distinct random initial weights and dataset reordering: the results addressing expressivity \(Fig\.[6](https://arxiv.org/html/2605.30370#Sx2.F6)\) and memorization \(Fig\.[8](https://arxiv.org/html/2605.30370#Sx2.F8)\) aggregate1010instances for theSMnets and55for theIBNNnets; the results addressing decreasing percentages of the training data \(Fig\.[3](https://arxiv.org/html/2605.30370#Sx2.F3)\) and learning speed \(Fig\.[7](https://arxiv.org/html/2605.30370#Sx2.F7)\) aggregate2020instances for theSMnets and55for theIBNNnets\. Corresponding confidence intervals of9090% are included in all cases\. Additionally, the results illustrating learning speed \(Fig\.[7](https://arxiv.org/html/2605.30370#Sx2.F7)\) perform a running average of33epochs\.

#### Adversarial attack generation and evaluation details

The respectiveadversarial robustness\(AR\) ofSM\- andIBNN\-based nets has been assessed using twoadversarial attacksof very different nature to better characterize the abilities of the proposal:PGD\[[25](https://arxiv.org/html/2605.30370#bib.bib25)\]and Pixle\[[26](https://arxiv.org/html/2605.30370#bib.bib26)\]\. In both cases, attacks have been generated using the library TorchAttacks\[[48](https://arxiv.org/html/2605.30370#bib.bib48)\]from the validation subset of each dataset\.

Originally,PGDis a white\-box attack that presumes the full availability of the targetANNand its gradient with respect to variations in its input\. It performs a number of gradient\-based iterations and limits its operation to a maximum distance with respect to the image from which it is calculated, denoted asϵ\\epsilon\. However, our implementation based onIBNNprevents the calculation of such a gradient and thus the feasibility of directPGDattacks, analogous to the gradient masking/obfuscation defenses againstAAs\. Nevertheless, the literature has shown that attacks generated using differential surrogate models are usually effective for attack generation\[[49](https://arxiv.org/html/2605.30370#bib.bib49),[50](https://arxiv.org/html/2605.30370#bib.bib50)\]\. Accordingly, whilePGDattacks for theSM\-basedANNshave been generated in a purely white\-box manner, thePGDattacks for theIBNN\-basedANNshave been produced using aSM\-based surrogate model, as follows: each trainedIBNNlayer has been replaced by theSMlayer resulting from simply removing the nonlinear bias of Eq\. \([4](https://arxiv.org/html/2605.30370#Sx2.E4)\) \(identical to settingλ=0\\lambda=0for the layer\)\. This type of surrogate model has been selected because it is deemed to be the most likely surrogate chosen by a potential attacker due to its similarity to the targetIBNNnet\.

Conversely, Pixle is purely black\-box, working only on the classification probabilities reported by theANNfor a given query image, and has been used to directly attack bothSMandIBNNwithout any need for surrogates\. In this way, it provides a complementary view to the results obtained usingPGD*via*a surrogate\. Pixle gradually shuffles an increasing number of pixels from the original image as its so\-called restarts advance, and optimizes the effectiveness of the newly scrambled pixels at each restart for a certain number of iterations\.

All Pixle experiments presented in this manuscript correspond to an identical attack with a maximum number of1010restarts with55iterations each, with the generation of pixel candidates based on similarity and from3×33\\times 3image patches\[[26](https://arxiv.org/html/2605.30370#bib.bib26)\]\. ThePGD\-based attacks presented in this manuscript perform1010gradient iterations and sweep across ranges of attack intensities, fromϵ=0\\epsilon=0to a certain maximum that depends on the dataset under study, resulting in the corresponding security evaluation curves\[[49](https://arxiv.org/html/2605.30370#bib.bib49)\]for each experimental condition\.

The models subject to adversarial attacks were trained according to the same principles, architectures, and hyperparameters indicated above, and, as a result, several instances of each model were obtained and evaluated on the clean validation dataset and under attack\. The results addressing bothPGD\(Fig\.[4](https://arxiv.org/html/2605.30370#Sx2.F4)\) and Pixle \(Fig\.[5](https://arxiv.org/html/2605.30370#Sx2.F5)\) aggregate, using the median,55model instances for theSMnets and55for theIBNNnets\. Since confidence interval estimation would be unreliable for the limited amount of available samples, adversarial attack experiments show the complete range of obtained results \(equivalent to confidence intervals of100100%\)\.

#### Memorization: experimental evaluation

Memorization occurs when models start learning individual training samples instead of the underlying statistical distribution of data: therefore, its occurrence must become apparent on the evolution of the network on its validation dataset\.

We devised, consequently, the following experiment: we deliberately contaminated the training subset of a certain dataset with different proportions of wrong samples, generated by simply scrambling their labels, while keeping the validation subset unscathed; for each considered proportion we trained networks using the contaminated data and took the epoch when the corresponding validation accuracy started to permanently decline as a sign of memorization and overfitting; and we regard such maximum validation accuracy as the level of generalization achievable for anANNon a contaminated dataset\.

## Code availability

## Author contributions

Conceptualization and methodology: M\.B\.; mathematical analysis: R\.M\., T\.B\., M\.B\., S\.L\.; numerical implementation: R\.M\., E\.V\.S\.; experiments: R\.M\., E\.V\.S\., R\.S\.M\., J\.H\.M\.; analysis of results: all authors; writing: R\.M\., M\.B\., T\.B\., S\.L\.

## Acknowledgments

This work has been supported by: Programa Fundamentos \(ref\. VIS4NN\) of Fundación BBVA 2022 \(Spain\); Programa Tecnologías \(ref\. TEC 2024/COM\-322\) of Comunidad de Madrid \(Spain\); Proyectos de Generación de Conocimiento 2024 \(ref\. PID2024\-161841NB\-I00\) of Ministerio de Ciencia, Innovación y Universidades \(Spain\); and Programa Momentum \(ref\. MMT24\-IO\-02\) of Consejo Superior de Investigaciones Científicas \(Spain\), funded by European Commission–NextGenerationEU through the Spanish Recovery, Transformation and Resilience Plan\.

## Additional information

### Supplementary information

The present manuscript refers toSupplementary Informationincluded as an appendix\.

## References

- \\bibcommenthead
- \[1\]McCulloch, W\. S\. & Pitts, W\.A logical calculus of the ideas immanent in nervous activity\.*The Bulletin of Mathematical Biophysics*5, 115–133 \(1943\)\.
- \[2\]Hubel, D\. H\. & Wiesel, T\. N\.Receptive fields of single neurones in the cat’s striate cortex\.*The Journal of Physiology*148, 574–591 \(1959\)\.
- \[3\]Rosenblatt, F\.The perceptron: A probabilistic model for information storage and organization in the brain\.*Psychological Review*65, 386 \(1958\)\.
- \[4\]Haykin, S\.*Neural networks and learning machines*\(New York: Prentice Hall, 2009\)\.
- \[5\]Chavlis, S\. & Poirazi, P\.Dendrites endow artificial neural networks with accurate, robust and parameter\-efficient learning\.*Nature Communications*16, 943 \(2025\)\.
- \[6\]Bastounis, A\., Hansen, A\. C\. & Vlačić, V\.The mathematics of adversarial attacks in AI – Why deep learning is unstable despite the existence of stable neural networks\.Preprint at[https://arxiv\.org/abs/2109\.06098](https://arxiv.org/abs/2109.06098)\(2021\)\.
- \[7\]Sartzetaki, C\., Roig, G\., Snoek, C\. G\. M\. & Groen, I\.International Conference on Learning Representations \(ed\.\)*One hundred neural networks and brains watching videos: Lessons from alignment*\.\(ed\.International Conference on Learning Representations\)*Proceedings of the International Conference on Learning Representations*\(2025\)\.
- \[8\]Kar, K\. & DiCarlo, J\. J\.The quest for an integrated set of neural mechanisms underlying object recognition in primates\.*Annual Review of Vision Science*10, 91–121 \(2024\)\.
- \[9\]Bowers, J\. S\.*et al\.*Deep problems with neural network models of human vision\.*Behavioral and Brain Sciences*1–74 \(2022\)\.
- \[10\]Tong, S\.*et al\.*IEEE/CVF Computer Vision Shifts and Pattern Recognition Organizer Committee \(ed\.\)*Eyes wide shut? Exploring the visual shortcomings of multimodal LLMs*\.\(ed\.IEEE/CVF Computer Vision Shifts and Pattern Recognition Organizer Committee\)*IEEE/CVF Conference on Computer Vision and Pattern Recognition*, 9568–9578 \(2024\)\.
- \[11\]Wichmann, F\. A\. & Geirhos, R\.Are deep neural networks adequate behavioral models of human visual perception?*Annual Review of Vision Science*9, 501–524 \(2023\)\.
- \[12\]Poleg\-Polsky, A\.Dendritic spikes expand the range of well tolerated population noise structures\.*Journal of Neuroscience*39, 9173–9184 \(2019\)\.
- \[13\]Pagkalos, M\., Makarov, R\. & Poirazi, P\.Leveraging dendritic properties to advance machine learning and neuro\-inspired computing\.*Current Opinion in Neurobiology*85, 102853 \(2024\)\.
- \[14\]Poirazi, P\. & Papoutsi, A\.Illuminating dendritic function with computational models\.*Nature Reviews Neuroscience*21, 303–321 \(2020\)\.
- \[15\]Stuart, G\. J\. & Spruston, N\.Dendritic integration: 60 years of progress\.*Nature Neuroscience*18, 1713–1721 \(2015\)\.
- \[16\]Francioni, V\. & Harnett, M\. T\.Rethinking single neuron electrical compartmentalization: dendritic contributions to network computation in vivo\.*Neuroscience*489, 185–199 \(2022\)\.
- \[17\]Stuyt, G\., Godenzini, L\. & Palmer, L\. M\.Local and global dynamics of dendritic activity in the pyramidal neuron\.*Neuroscience*489, 176–184 \(2022\)\.
- \[18\]Stingl, M\., Draguhn, A\. & Both, M\.A dendrite is a dendrite is a dendrite? Dendritic signal integration beyond the “antenna” model\.*Pflügers Archiv\-European Journal of Physiology*477, 9–16 \(2025\)\.
- \[19\]Zador, A\.*et al\.*Catalyzing next\-generation artificial intelligence through NeuroAI\.*Nature Communications*14, 1597 \(2023\)\.
- \[20\]Iyer, A\.*et al\.*Avoiding catastrophe: Active dendrites enable multi\-task learning in dynamic environments\.*Frontiers in Neurorobotics*16, 846219 \(2022\)\.
- \[21\]Larkum, M\. E\.Are dendrites conceptually useful?*Neuroscience*489, 4–14 \(2022\)\.
- \[22\]Rentzeperis, I\., Prandi, D\. & Bertalmío, M\.A neural model for V1 that incorporates dendritic nonlinearities and back\-propagating action potentials\.*Journal of Neuroscience*45\(2025\)\.
- \[23\]Jakubovitz, D\. & Giryes, R\.European Computer Vision Association \(ECVA\) \(ed\.\)*Improving DNN robustness to adversarial attacks using Jacobian regularization*\.\(ed\.European Computer Vision Association \(ECVA\)\)*European Conference on Computer Vision*, 525–541 \(2018\)\.
- \[24\]Tyukin, I\., Higham, D\. J\., Bastounis, A\., Woldegeorgis, E\. & Gorban, A\. N\.The feasibility and inevitability of stealth attacks\.*IMA Journal of Applied Mathematics*89, 44–84 \(2024\)\.
- \[25\]Ma̧dry, A\., Makelov, A\., Schmidt, L\., Tsipras, D\. & Vladu, A\.International Conference on Learning Representations \(ed\.\)*Towards deep learning models resistant to adversarial attacks*\.\(ed\.International Conference on Learning Representations\)*Proceedings of the International Conference on Learning Representations*\(2018\)\.
- \[26\]Pomponi, J\., Scardapane, S\. & Uncini, A\.Pixle: a fast and effective black\-box attack based on rearranging pixels\.International Joint Conference on Neural Networks \(IJCNN 2022\)\.
- \[27\]Hornik, K\., Stinchcombe, M\., White, H\.*et al\.*Multilayer feedforward networks are universal approximators\.*Neural networks*2, 359–366 \(1989\)\.
- \[28\]Cybenko, G\.Approximation by superpositions of a sigmoidal function\.*Mathematics of Control, Signals and Systems*2, 303–314 \(1989\)\.
- \[29\]Feldman, V\. & Zhang, C\.NIPS Foundation \(ed\.\)*What neural networks memorize and why: Discovering the long tail via influence estimation*\.\(ed\.NIPS Foundation\)*Advances in Neural Information Processing Systems*, Vol\. 33, 2881–2891 \(2020\)\.
- \[30\]Garg, I\., Ravikumar, D\. & Roy, K\.IMLS \(ed\.\)*Memorization through the lens of curvature of loss function around samples*\.\(ed\.IMLS\)*International Conference on Machine Learning*, Vol\. 235, 15083–15101 \(2024\)\.
- \[31\]Bertalmío, M\.*et al\.*Evidence for the intrinsically nonlinear nature of receptive fields in vision\.*Scientific reports*10, 16277 \(2020\)\.
- \[32\]Paiton, D\. M\.*et al\.*Selectivity and robustness of sparse coding networks\.*Journal of Vision*20, 1–28 \(2020\)\.
- \[33\]Rawat, S\., Heeger, D\. & Martiniani, S\.Unconditional stability of a recurrent neural circuit implementing divisive normalization\.*Advances in Neural Information Processing Systems*37, 14712–14750 \(2024\)\.
- \[34\]Bai, S\., Kolter, J\. Z\. & Koltun, V\.NIPS Foundation \(ed\.\)*Deep equilibrium models*\.\(ed\.NIPS Foundation\)*Advances in Neural Information Processing Systems*\(2019\)\.
- \[35\]Khona, M\. & Fiete, I\. R\.Attractor and integrator networks in the brain\.*Nature Reviews Neuroscience*23, 744–766 \(2022\)\.
- \[36\]Luna, R\., Serrano\-Pedraza, I\. & Bertalmío, M\.Overcoming the limitations of motion sensor models by considering dendritic computations\.*Scientific reports*15, 9213 \(2025\)\.
- \[37\]Vaswani, A\.*et al\.*Attention is all you need\.*Advances in neural information processing systems*30\(2017\)\.
- \[38\]Hopfield, J\. J\.Neurons with graded response have collective computational properties like those of two\-state neurons\.*Proceedings of the national academy of sciences*81, 3088–3092 \(1984\)\.
- \[39\]Schölkopf, B\.*Causality for machine learning*, 765–804 \(Association for Computing Machinery, New York, NY, USA, 2022\)\.
- \[40\]Pearl, J\. & Mackenzie, D\.*The book of why: the new science of cause and effect*\(Basic books, 2018\)\.
- \[41\]Pinkus, A\.Approximation theory of the mlp model in neural networks\.*Acta Numerica*8, 143––195 \(1999\)\.
- \[42\]Paszke, A\.*et al\.*Pytorch: An imperative style, high\-performance deep learning library\.Preprint at[https://arxiv\.org/abs/1912\.01703](https://arxiv.org/abs/1912.01703)\(2019\)\.
- \[43\]Geng, Z\. & Kolter, J\. Z\.TorchDEQ: A library for deep equilibrium models \(2023\)\.URL[https://github\.com/locuslab/torchdeq](https://github.com/locuslab/torchdeq)\.
- \[44\]Xiao, H\., Rasul, K\. & Vollgraf, R\.Fashion\-MNIST: a novel image dataset for benchmarking machine learning algorithms\.Preprint at[https://arxiv\.org/abs/1708\.07747](https://arxiv.org/abs/1708.07747)\(2017\)\.
- \[45\]Netzer, Y\.*et al\.*NIPS Foundation \(ed\.\)*Reading digits in natural images with unsupervised feature learning*\.\(ed\.NIPS Foundation\)*NIPS Workshop on Deep Learning and Unsupervised Feature Learning*, Vol\. 2011, 7 \(2011\)\.
- \[46\]Krizhevsky, A\. & Hinton, G\.Learning multiple layers of features from tiny images\.Tech\. Rep\., University of Toronto \(2009\)\.URL[https://www\.cs\.toronto\.edu/~kriz/learning\-features\-2009\-TR\.pdf](https://www.cs.toronto.edu/~kriz/learning-features-2009-TR.pdf)\.
- \[47\]TorchVision: PyTorch’s computer vision library\.[https://github\.com/pytorch/vision](https://github.com/pytorch/vision)\(2016\)\.
- \[48\]Kim, H\.Torchattacks: A PyTorch repository for adversarial attacks\.Preprint at[https://arxiv\.org/abs/2010\.01950](https://arxiv.org/abs/2010.01950)\(2021\)\.
- \[49\]Biggio, B\. & Roli, F\.Wild patterns: Ten years after the rise of adversarial machine learning\.*Pattern Recognition*84, 317–331 \(2018\)\.
- \[50\]Papernot, N\.*et al\.*ACM Special Interest Group on Security, Audit and Control \(ed\.\)*Practical black\-box attacks against machine learning*\.\(ed\.ACM Special Interest Group on Security, Audit and Control\)*Proceedings of the ACM on Asia Conference on Computer and Communications Security*, 506–519 \(Association for Computing Machinery, New York, NY, USA, 2017\)\.

\\appendixpage

## Creating an IBNN by transferring the weights from an existing SM network already trained for the same task

This section demonstrates that when creating anIBNNto solve a given task, one could avoid the training stage by simply transferring the weights from an existing SM network of analogous architecture that has already been trained for the same task\. The proof is formulated for the case of a binary classification task and one\-hidden\-layer networks, but this should not limit its generality\. In addition, we show that the magnitude of the parameterλ\\lambdaused inIBNNhas an impact on the accuracy achieved by the network\.

Formally, letΨ\\Psibe aSM\-based binary classifierΨ​\(𝐱\)=H​\(s​\(𝐱\)\)\\Psi\(\\mathbf\{x\}\)=H\(s\(\\mathbf\{x\}\)\), which performs a binary decision, embodied by the Heaviside functionHH, based on the sign of a scores:ℝD→ℝs:\\mathbb\{R\}^\{D\}\\to\\mathbb\{R\}assigned to each input sample𝐱∈ℝN\\mathbf\{x\}\\in\\mathbb\{R\}^\{N\}through a single hidden layer ofDDSMneurons and aFC\-based head according to

s​\(𝐱\)=𝐜T​SM⁡\(𝐱;M,𝐛\)−c0=∑i=1Dci​ϕ​\(yi​\(𝐱\)\)−c0,s\(\\mathbf\{x\}\)=\\mathbf\{c\}^\{T\}\\,\\operatorname\{SM\}\(\\mathbf\{x\};\\mathrm\{M\},\\mathbf\{b\}\)\-c\_\{0\}=\\sum\_\{i=1\}^\{D\}c\_\{i\}\\,\\phi\(\\,y\_\{i\}\(\\mathbf\{x\}\)\\,\)\-c\_\{0\}\\,,\(63\)
with𝐜=\(ci\)i=1D∈ℝD\\mathbf\{c\}=\(c\_\{i\}\)\_\{i=1\}^\{D\}\\in\\mathbb\{R\}^\{D\}andc0∈ℝc\_\{0\}\\in\\mathbb\{R\}are the weights of the output layer, and where

yi​\(𝐱\)=𝐦iT​𝐱−bi,y\_\{i\}\(\\mathbf\{x\}\)=\\mathbf\{m\}^\{T\}\_\{i\}\\mathbf\{x\}\-b\_\{i\}\\,,\(64\)
where𝐦i∈ℝN\\mathbf\{m\}\_\{i\}\\in\\mathbb\{R\}^\{N\}andbi∈ℝb\_\{i\}\\in\\mathbb\{R\}are, respectively, the linear filter and the bias for theii\-th unit of the hidden layer \(see Eq\. \([1](https://arxiv.org/html/2605.30370#Sx2.E1)\) and Eq\. \([20](https://arxiv.org/html/2605.30370#Sx4.E20)\)\)\. And let us assume, without any loss of generality, that\{𝐜,c0\}\\\{\\mathbf\{c\},c\_\{0\}\\\}has been normalized to make𝐜\\mathbf\{c\}a unit vector, that is,‖𝐜‖2=1\\\|\\mathbf\{c\}\\\|\_\{2\}=1\.

We assume that this network has been trained on the set𝒯\\mathcal\{T\}composed of two classes𝒯0\\mathcal\{T\}\_\{0\}and𝒯1\\mathcal\{T\}\_\{1\}achieving perfect accuracy on it, that is,

Ψ\(𝐱\(t\)\)=\{1,∀𝐱\(t\)∈𝒯10,∀𝐱\(t\)∈𝒯0:\\Psi\(\\mathbf\{x\}^\{\(t\)\}\)=\\left\\\{\\begin\{matrix\}1\\,,&\\forall\\mathbf\{x\}^\{\(t\)\}\\in\\mathcal\{T\}\_\{1\}\\\\ 0\\,,&\\forall\\mathbf\{x\}^\{\(t\)\}\\in\\mathcal\{T\}\_\{0\}\\end\{matrix\}\\right\.\\;:
in other words,SM⁡\(⋅;M,𝐛\)\\operatorname\{SM\}\(\\cdot;\\mathrm\{M\},\\mathbf\{b\}\)makes the setsSM⁡\(𝒯1;M,𝐛\)\\operatorname\{SM\}\(\\mathcal\{T\}\_\{1\};\\mathrm\{M\},\\mathbf\{b\}\)andSM⁡\(𝒯0;M,𝐛\)\\operatorname\{SM\}\(\\mathcal\{T\}\_\{0\};\\mathrm\{M\},\\mathbf\{b\}\)linearly separable, and\{𝐜,c0\}\\\{\\mathbf\{c\},c\_\{0\}\\\}separates them\. Furthermore, we assume that such perfect classifications is achieved through the classifierssof Eq\. \([63](https://arxiv.org/html/2605.30370#Ax1.E63)\) by a marginMM, that is, that

min𝐱\(1\)∈𝒯1⁡\{s​\(𝐱\(1\)\)\}−max𝐱\(0\)∈𝒯0⁡\{s​\(𝐱\(0\)\)\}≥M\.\\min\_\{\\mathbf\{x\}^\{\(1\)\}\\in\\mathcal\{T\}\_\{1\}\}\\left\\\{s\(\\mathbf\{x\}^\{\(1\)\}\)\\right\\\}\\,\-\\max\_\{\\mathbf\{x\}^\{\(0\)\}\\in\\mathcal\{T\}\_\{0\}\}\\left\\\{s\(\\mathbf\{x\}^\{\(0\)\}\)\\right\\\}\\;\\geq\\;M\\;\.\(65\)
We define another networkΨ^​\(𝐱\)=H​\(s^​\(𝐱\)\)\\hat\{\\Psi\}\(\\mathbf\{x\}\)=H\(\\hat\{s\}\(\\mathbf\{x\}\)\), with analogous architecture toΨ\\Psibut withIBNNinstead ofSMneurons in its hidden layer, and with the peculiarity that theIBNNneurons use the same exact\{M,𝐛\}\\\{\\mathrm\{M\},\\mathbf\{b\}\\\}of the respectiveSMneurons in Eqs\. \([63](https://arxiv.org/html/2605.30370#Ax1.E63)\)\-\([64](https://arxiv.org/html/2605.30370#Ax1.E64)\) but whose scoring hyperplane\{𝐜^,c^0\}\\\{\\hat\{\\mathbf\{c\}\},\\hat\{c\}\_\{0\}\\\}is allowed to differ, and thus

s^​\(𝐱\)=𝐜^T​IBNN⁡\(𝐱;M,𝐛,λ\)−c^0=∑i=1Dc^i​ϕ​\(zi​\(𝐱\)\)−c^0,\\hat\{s\}\(\\mathbf\{x\}\)=\\hat\{\\mathbf\{c\}\}^\{T\}\\operatorname\{IBNN\}\(\\mathbf\{x\};\\mathrm\{M\},\\mathbf\{b\},\\lambda\)\-\\hat\{c\}\_\{0\}=\\sum\_\{i=1\}^\{D\}\\hat\{c\}\_\{i\}\\,\\phi\(\\,z\_\{i\}\(\\mathbf\{x\}\)\\,\)\-\\hat\{c\}\_\{0\}\\,,\(66\)
where

zi​\(𝐱\)=𝐦iT​𝐱−bi−λ​∑k=1Dwi​k​σ​\(zk​\(𝐱\)−zi​\(𝐱\)\)\.z\_\{i\}\(\\mathbf\{x\}\)=\\mathbf\{m\}^\{T\}\_\{i\}\\mathbf\{x\}\-b\_\{i\}\\,\-\\,\\lambda\\sum\_\{k=1\}^\{D\}w\_\{ik\}\\,\\sigma\\\!\\left\(z\_\{k\}\(\\mathbf\{x\}\)\\\!\-\\\!z\_\{i\}\(\\mathbf\{x\}\)\\right\)\\,\.\(67\)
We will show that the training error achievable by theIBNN\-based classifierΨ^\\hat\{\\Psi\}is below a bound that depends linearly on the magnitude\|λ\|\|\\lambda\|and which could be made, consequently, arbitrarily small\. We will do so by studying the case of𝐜^=𝐜\\hat\{\\mathbf\{c\}\}=\\mathbf\{c\}andc^0=c0\\hat\{c\}\_\{0\}=c\_\{0\}, that is, a scoring hyperplane forΨ^\\hat\{\\Psi\}identical to that ofΨ\\Psi, as a suboptimal instance of\{𝐜^,c^0\}\\\{\\hat\{\\mathbf\{c\}\},\\hat\{c\}\_\{0\}\\\}\. In the remaining of the section we will use, for convenience, the shorter notationIBNN⁡\(𝐱\)=IBNN⁡\(𝐱;M,𝐛,λ\)\\operatorname\{IBNN\}\(\\mathbf\{x\}\)=\\operatorname\{IBNN\}\(\\mathbf\{x\};\\mathrm\{M\},\\mathbf\{b\},\\lambda\)andSM⁡\(𝐱\)=SM⁡\(𝐱;M,𝐛\)\\operatorname\{SM\}\(\\mathbf\{x\}\)=\\operatorname\{SM\}\(\\mathbf\{x\};\\mathrm\{M\},\\mathbf\{b\}\)where the parameters of the layer are left implied\.

First, a bound on the maximum distance between the images of the mapsSM⁡\(⋅\)\\operatorname\{SM\}\(\\cdot\)andIBNN⁡\(⋅\)\\operatorname\{IBNN\}\(\\cdot\)for the same data point𝐱\\mathbf\{x\}can be established\. The difference can be written as

IBNN⁡\(𝐱\)−SM⁡\(𝐱\)=\(Φ∘F∘A\)​\(𝐱\)−\(Φ∘A\)​\(𝐱\)=\(Φ∘F\)​\(A​\(𝐱\)\)−Φ​\(A​\(𝐱\)\),\\operatorname\{IBNN\}\(\\mathbf\{x\}\)\-\\operatorname\{SM\}\(\\mathbf\{x\}\)=\\big\(\\Phi\\circ\\mathrm\{F\}\\circ\\mathrm\{A\}\\big\)\(\\mathbf\{x\}\)\-\\big\(\\Phi\\circ\\mathrm\{A\}\\big\)\(\\mathbf\{x\}\)=\\big\(\\Phi\\circ\\mathrm\{F\}\\big\)\(\\mathrm\{A\}\(\\mathbf\{x\}\)\)\-\\Phi\(\\mathrm\{A\}\(\\mathbf\{x\}\)\)\\,,\(68\)
where the dependenceF=F​\(⋅;λ\)\\mathrm\{F\}=\\mathrm\{F\}\(\\cdot;\\lambda\)has also left implied\. From that form, for anyΦ=\(ϕ\)i=1D\\Phi=\(\\phi\)\_\{i=1\}^\{D\}contractive like in the case ofϕ=ReLU\\phi=\\operatorname\{ReLU\}, we have that, for any𝐲=A​\(𝐱\)\\mathbf\{y\}=\\mathrm\{A\}\(\\mathbf\{x\}\),

‖\(Φ∘F\)​\(𝐲\)−Φ​\(𝐲\)‖22≤‖F​\(𝐲\)−𝐲‖22=‖−λ​B​\(F​\(𝐲\)\)‖22≤\|λ\|2​‖B​\(F​\(𝐲\)\)‖22≤\|λ\|2​D,\\begin\{split\}\\left\\\|\\big\(\\Phi\\circ\\mathrm\{F\}\\big\)\(\\mathbf\{y\}\)\-\\Phi\(\\mathbf\{y\}\)\\right\\\|\_\{2\}^\{2\}&\\leq\\,\\left\\\|\\mathrm\{F\}\(\\mathbf\{y\}\)\-\\mathbf\{y\}\\right\\\|\_\{2\}^\{2\}\\\\ &=\\,\\left\\\|\\,\-\\lambda\\;\\mathrm\{B\}\\big\(\\mathrm\{F\}\(\\mathbf\{y\}\)\\big\)\\,\\right\\\|\_\{2\}^\{2\}\\,\\leq\\,\|\\lambda\|^\{2\}\\,\\left\\\|\\,\\mathrm\{B\}\\big\(\\mathrm\{F\}\(\\mathbf\{y\}\)\\big\)\\,\\right\\\|\_\{2\}^\{2\}\\\\ &\\leq\|\\lambda\|^\{2\}\\,D\\;,\\end\{split\}\(69\)
wherein the second line is derived from the very definition of the fixed\-point functionF\\mathrm\{F\}\(see Eq\. \([17](https://arxiv.org/html/2605.30370#Sx4.E17)\)\) when a unique solution exists, sinceF​\(𝐲\)=𝐲−λ​B​\(F​\(𝐲\)\)\\mathrm\{F\}\(\\mathbf\{y\}\)=\\mathbf\{y\}\-\\lambda\\mathrm\{B\}\\big\(\\mathrm\{F\}\(\\mathbf\{y\}\)\), and wherein the last inequality derives straightforwardly from the definition ofB=\(Bi\)i=1D\\mathrm\{B\}=\(\\mathrm\{B\}\_\{i\}\)\_\{i=1\}^\{D\}\(see Eq\. \([16](https://arxiv.org/html/2605.30370#Sx4.E16)\)\) for a sigmoidal\|σ\|≤1\|\\sigma\|\\leq 1and positive and normalizedwi​kw\_\{ik\}as

\|Bi​\(𝐳\)\|=\|∑k=1Dwi​k​σ​\(zk−zi\)\|≤∑k=1D\|wi​k\|​\|σ​\(zk−zi\)\|≤∑k=1D\|wi​k\|≤1\.\\left\|\\mathrm\{B\}\_\{i\}\(\\mathbf\{z\}\)\\right\|\\,=\\,\\left\|\\sum\_\{k=1\}^\{D\}w\_\{ik\}\\sigma\(z\_\{k\}\-z\_\{i\}\)\\right\|\\,\\leq\\,\\sum\_\{k=1\}^\{D\}\\left\|w\_\{ik\}\\right\|\\left\|\\sigma\(z\_\{k\}\-z\_\{i\}\)\\right\|\\,\\leq\\,\\sum\_\{k=1\}^\{D\}\\left\|w\_\{ik\}\\right\|\\leq 1\\,\.\(70\)
Therefore, since by virtue of Eqs\. \([68](https://arxiv.org/html/2605.30370#Ax1.E68)\)\-\([69](https://arxiv.org/html/2605.30370#Ax1.E69)\)

‖IBNN⁡\(𝐱\)−SM⁡\(𝐱\)‖2≤\|λ\|​D,\\left\\\|\\operatorname\{IBNN\}\(\\mathbf\{x\}\)\-\\operatorname\{SM\}\(\\mathbf\{x\}\)\\right\\\|\_\{2\}\\,\\leq\\,\|\\lambda\|\\sqrt\{D\}\\,,\(71\)
we can write, for any𝐱∈ℝN\\mathbf\{x\}\\in\\mathbb\{R\}^\{N\},

IBNN⁡\(𝐱\)=SM⁡\(𝐱\)\+α​\(𝐱\)​𝐪​\(𝐱\)\\operatorname\{IBNN\}\(\\mathbf\{x\}\)=\\operatorname\{SM\}\(\\mathbf\{x\}\)\+\\alpha\(\\mathbf\{x\}\)\\,\\mathbf\{q\}\(\\mathbf\{x\}\)\(72\)
for certainα​\(𝐱\)∈ℝ\\alpha\(\\mathbf\{x\}\)\\in\\mathbb\{R\}and𝐪​\(𝐱\)∈ℝD\\mathbf\{q\}\(\\mathbf\{x\}\)\\in\\mathbb\{R\}^\{D\}such that0≤α​\(𝐱\)≤\|λ\|​D0\\leq\\alpha\(\\mathbf\{x\}\)\\leq\|\\lambda\|\\sqrt\{D\}and‖𝐪​\(𝐱\)‖2=1\\\|\\mathbf\{q\}\(\\mathbf\{x\}\)\\\|\_\{2\}=1\. Using this fact, and for an identical scoring hyperplane to that ofΨ\\Psi, that is,𝐜^=𝐜\\hat\{\\mathbf\{c\}\}=\\mathbf\{c\}andc^0=c0\\hat\{c\}\_\{0\}=c\_\{0\}\), we have,∀𝐱\(1\)∈𝒯1\\forall\\mathbf\{x\}^\{\(1\)\}\\in\\mathcal\{T\}\_\{1\}and∀𝐱\(0\)∈𝒯0\\forall\\mathbf\{x\}^\{\(0\)\}\\in\\mathcal\{T\}\_\{0\},

s^\(𝐱\(1\)\)−s^​\(𝐱\(0\)\)=\(𝐜T​IBNN⁡\(𝐱\(1\)\)−c0\)−\(𝐜T​IBNN⁡\(𝐱\(0\)\)−c0\)=\(𝐜T​\(SM⁡\(𝐱\(1\)\)\+α​\(𝐱\(1\)\)​𝐪​\(𝐱\(1\)\)\)−c0\)−\(𝐜T​\(SM⁡\(𝐱\(0\)\)\+α​\(𝐱\(0\)\)​𝐪​\(𝐱\(0\)\)\)−c0\)=\(\(𝐜T​SM⁡\(𝐱\(1\)\)−c0\)−\(𝐜T​SM⁡\(𝐱\(0\)\)−c0\)\)\+𝐜T​\(α​\(𝐱\(1\)\)​𝐪​\(𝐱\(1\)\)−α​\(𝐱\(0\)\)​𝐪​\(𝐱\(0\)\)\)≥M−2​\|λ\|​D,\\begin\{split\}\\hat\{s\}&\(\\mathbf\{x\}^\{\(1\)\}\)\-\\hat\{s\}\(\\mathbf\{x\}^\{\(0\)\}\)\\\\ &=\\,\\Big\(\\mathbf\{c\}^\{T\}\\operatorname\{IBNN\}\(\\mathbf\{x\}^\{\(1\)\}\)\-c\_\{0\}\\Big\)\\,\-\\,\\Big\(\\mathbf\{c\}^\{T\}\\operatorname\{IBNN\}\(\\mathbf\{x\}^\{\(0\)\}\)\-c\_\{0\}\\Big\)\\\\ &=\\,\\bigg\(\\mathbf\{c\}^\{T\}\\,\\Big\(\\operatorname\{SM\}\(\\mathbf\{x\}^\{\(1\)\}\)\+\\alpha\(\\mathbf\{x\}^\{\(1\)\}\)\\,\\mathbf\{q\}\(\\mathbf\{x\}^\{\(1\)\}\)\\Big\)\-c\_\{0\}\\bigg\)\-\\bigg\(\\mathbf\{c\}^\{T\}\\,\\Big\(\\operatorname\{SM\}\(\\mathbf\{x\}^\{\(0\)\}\)\+\\alpha\(\\mathbf\{x\}^\{\(0\)\}\)\\,\\mathbf\{q\}\(\\mathbf\{x\}^\{\(0\)\}\)\\Big\)\-c\_\{0\}\\bigg\)\\\\ &=\\,\\bigg\(\\Big\(\\mathbf\{c\}^\{T\}\\,\\operatorname\{SM\}\(\\mathbf\{x\}^\{\(1\)\}\)\-c\_\{0\}\\Big\)\-\\Big\(\\mathbf\{c\}^\{T\}\\,\\operatorname\{SM\}\(\\mathbf\{x\}^\{\(0\)\}\)\-c\_\{0\}\\Big\)\\bigg\)\+\\mathbf\{c\}^\{T\}\\,\\Big\(\\alpha\(\\mathbf\{x\}^\{\(1\)\}\)\\,\\mathbf\{q\}\(\\mathbf\{x\}^\{\(1\)\}\)\-\\alpha\(\\mathbf\{x\}^\{\(0\)\}\)\\,\\mathbf\{q\}\(\\mathbf\{x\}^\{\(0\)\}\)\\Big\)\\\\ &\\geq\\;M\-2\|\\lambda\|\\sqrt\{D\}\\,,\\end\{split\}\(73\)
where the last inequality results from the assumption of Eq\. \([65](https://arxiv.org/html/2605.30370#Ax1.E65)\) for a marginMMin theSM\-based classifier and from the worst case alignment between the unit vectors𝐜\\mathbf\{c\}and𝐪​\(𝐱\(i\)\)\\mathbf\{q\}\(\\mathbf\{x\}^\{\(i\)\}\)\.