Classifying Directional Trajectories Near Criticality in the Three-State Majority-Vote Model with Deep Belief Networks and Bidirectional GRUs

# Classifying Directional Trajectories Near Criticality in the Three-State Majority-Vote Model with Deep Belief Networks and Bidirectional GRUs
Source: [https://arxiv.org/html/2608.18235](https://arxiv.org/html/2608.18235)
1stMauricio A\. ValleAffiliation:Faculty of Engineering and Sciences Universidad Adolfo Ibáñez Millennium Nucleus for Social Data Science \(SODAS\) Santiago, Chile mauricio\.valle\.b@uai\.cl2ndGonzalo A\. RuzAffiliation:Faculty of Engineering and Sciences Universidad Adolfo Ibáñez Millennium Nucleus for Social Data Science \(SODAS\) Millennium Nucleus in Data Science for Plant Resilience \(PhytoLearning\) Santiago, Chile gonzalo\.ruz@uai\.cl

###### Abstract

In this work, we investigate whether the latent representations learned by a Deep Belief Network \(DBN\) and a Bidirectional Gated Recurrent Unit \(Bi\-GRU\) can discriminate among four dynamically distinct trajectory types in the three\-state majority vote model \(MV3\): approach from disorder, approach from order, departure to disorder, and departure to order\. The DBN, pre\-trained in an unsupervised manner on static equilibrium samples via a Gaussian\-Bernoulli Restricted Boltzmann Machine input layer and architecture784→4096→225→81784\\to 4096\\to 225\\to 81, encodes each lattice snapshot into an 81\-dimensional latent vector\. A t\-SNE analysis of the DBN latent space reveals only partial separation of the four trajectory types, reflecting the fact that a model trained on static configurations cannot fully resolve directional temporal structure\. A two\-layer Bi\-GRU classifier, trained on sequences of DBN\-encoded snapshots of lengthT=50T=50, achieves near\-perfect separation of all four trajectory types in its hidden state space, as confirmed by t\-SNE visualization on both training and test sets\. Furthermore, a sliding\-window application of the trained Bi\-GRU to continuous MV3 dynamics demonstrates its ability to sense the system’s current dynamical regime in real\-time\. These results establish a principled hierarchical architecture for detecting and classifying critical transitions in agent\-based opinion dynamics models\.

###### Index Terms:

Three\-state majority\-vote model, opinion dynamics, deep belief networks, bidirectional gated recurrent units, critical transitions

## IIntroduction

Agent\-based models of opinion formation, when studied through the framework of statistical mechanics, exhibit phase transitions, bifurcations between ordered and disordered collective states, governed by a noise parameter that plays the role of social temperature\[[5](https://arxiv.org/html/2608.18235#bib.bib1),[11](https://arxiv.org/html/2608.18235#bib.bib2)\]\. These transitions are of intrinsic scientific interest because they represent tipping points: qualitative changes in collective behavior that, once crossed, are difficult to reverse\. In social contexts, analogs of such tipping points include the sudden fragmentation of a previously moderate public discourse into two irreconcilable poles, the emergence of group consensus in initially heterogeneous populations, or the destabilization of a previously stable social equilibrium\. Understanding the dynamical signatures that precede these transitions, and developing computational tools capable of detecting them from system observables, is therefore not only a challenge in nonequilibrium statistical physics, but a problem with direct relevance to the modeling of social systems and financial markets, where sequence\-based deep models have recently been used to flag impending instabilities\[[16](https://arxiv.org/html/2608.18235#bib.bib22)\]\.

The three\-state majority vote model \(MV3\) provides a minimal but non\-trivial arena for studying this problem\. In the MV3 model, each agent on a lattice may hold one of three discrete opinions, adopting the majority opinion of its neighbors with probabilityp=1−qp=1\-qand dissenting with noise probabilityqq\[[22](https://arxiv.org/html/2608.18235#bib.bib3),[12](https://arxiv.org/html/2608.18235#bib.bib4)\]\. On infinite lattices, the three\-state version is known to exhibit a discontinuous order\-disorder transition; however, on finite lattices, such as the28×2828\\times 28system studied here, the sharp discontinuity is rounded by finite\-size effects into a continuous crossover regime, producing a rich, extended critical region\. This finite\-size rounding is not a limitation but an opportunity: the system spends an extended portion of its trajectory in the vicinity of the critical point, generating the kind of slowly evolving, correlated dynamics that a temporal deep learning architecture can learn to recognize and classify\.

The application of machine learning to phase transitions has grown substantially over the past decade, beginning with the landmark demonstration that neural networks can identify phases of matter from raw spin configurations without prior knowledge of the order parameter\[[3](https://arxiv.org/html/2608.18235#bib.bib5)\]\. Subsequent work showed that even unsupervised methods, specifically the “learning by confusion” scheme, can locate critical points without labeled data\[[20](https://arxiv.org/html/2608.18235#bib.bib6)\], and that transitions can be learned directly from dynamical rather than static data\[[19](https://arxiv.org/html/2608.18235#bib.bib7)\]\. Most of these approaches, however, analyze configurations or signals at fixed parameter values rather than full temporal trajectories evolving through a phase transition\. The dynamically richer problem of anticipating a transition before it occurs, the core challenge in early warning systems, requires methods that can extract temporal structure from sequential data\. Critical slowing down near tipping points produces measurable statistical signatures in time series\[[15](https://arxiv.org/html/2608.18235#bib.bib8)\], and recurrent deep learning architectures can detect these signatures earlier and more reliably than classical indicators in ecological and climate systems\[[2](https://arxiv.org/html/2608.18235#bib.bib9)\]\. Extending these ideas to agent\-based social models is a natural but non\-trivial step, requiring careful design of both the representation learning pipeline and the temporal classification architecture\.

More recently, deep learning has been applied directly to opinion dynamics models\. Supervised dense networks, PCA, and variational autoencoders have been employed to characterize the continuous phase transition of the majority vote model, showing that the VAE reconstruction loss behaves as an order parameter and that the correlation between real and reconstructed configurations is universal at the critical point\[[13](https://arxiv.org/html/2608.18235#bib.bib10)\]\. Analogous methods applied to the Biswas–Chatterjee–Sen model confirm that unsupervised techniques can identify critical points on both square and triangular lattices without explicit knowledge of the transition\[[14](https://arxiv.org/html/2608.18235#bib.bib11)\]\. These results demonstrate that deep generative models can learn the statistical structure of opinion configurations in a phase\-sensitive way\. However, these approaches treat system states as static snapshots, and even dynamics\-based methods stop short of resolving the*direction*of a trajectory relative to the critical point, whether a system is approaching or departing criticality, and from which phase, which is precisely the information needed to anticipate a transition\.

In this work, we address this gap, using the MV3 model on a two\-dimensional square lattice with periodic boundary conditions as our test system\. We construct quasi\-static trajectory ensembles via Monte Carlo simulation with asynchronous update rules, covering four dynamically distinct trajectory types defined by their direction and origin relative to the critical point: approach from disorder, approach from order, departure toward disorder, and departure toward order\. Each trajectory constitutes a temporal sequence of lattice configurations evolving slowly through parameter space, designed to isolate the dynamical signatures of critical slowing down and symmetry breaking\. We then develop a two\-stage deep learning pipeline in which a Deep Belief Network \(DBN\) built on a Gaussian\-Bernoulli Restricted Boltzmann Machine \(GB\-RBM\) input layer first encodes high\-dimensional lattice snapshots into compact latent representations, and a Bidirectional Gated Recurrent Unit \(Bi\-GRU\) network subsequently processes these latent sequences to classify the four trajectory types\. Our contributions are threefold: \(i\) a directional trajectory ensemble that isolates the four combinations of approach and departure from both stable phases; \(ii\) a hierarchical DBN–Bi\-GRU pipeline that separates these modes through analysis of its latent representations via t\-SNE, with the DBN providing partial separation and the Bi\-GRU hidden state achieving a substantially cleaner decomposition that reflects both phase and direction; and \(iii\) a sliding\-window experiment in which the trained model tracks regime changes in real time along continuous MV3 trajectories\. Together, these results indicate that combining unsupervised representation learning with recurrent sequence modeling constitutes a principled framework for characterizing criticality dynamics in opinion systems, and a concrete step toward early warning systems for critical transitions\.

## IIProposed Method and Experiments

To study the dynamics of a system governed by the majority rule, we developed a five\-step methodology consisting of five stages illustrated in Figure[1](https://arxiv.org/html/2608.18235#S2.F1):

1. 1\.Simulating MV3 systems using Monte Carlo steps \(MCS\) on a lattice with periodic boundary conditions to obtain static equilibrium samples of the system at a given value ofpp,S⁡\(p\)S\(p\)\.
2. 2\.Training a DBN to generate latent representations in the last hidden layerL⁡\(S⁡\(p\)\)L\(S\(p\)\)of the samples simulated in the previous stage\.
3. 3\.Simulating ensembles of quasi\-static trajectories to capture the system’s criticality dynamics\. We use the notationT⁡\(ps→pe\)=\{S⁡\(pt\)\}T\(p\_\{s\}\\rightarrow p\_\{e\}\)=\\\{S\(p\_\{t\}\)\\\}to denote a trajectory consisting of a sequence of system statesS⁡\(pt\)S\(p\_\{t\}\)with different values ofptp\_\{t\}from an initial valuepip\_\{i\}to an end valuepep\_\{e\}\.
4. 4\.Encode the trajectoriesT⁡\(ps→pe\)T\(p\_\{s\}\\rightarrow p\_\{e\}\)using the DBN, which we will callL⁡\(ps→pe\)L\(p\_\{s\}\\rightarrow p\_\{e\}\)\.
5. 5\.Given the trajectory assemblyL⁡\(ps→pe\)L\(p\_\{s\}\\rightarrow p\_\{e\}\), train a GRU to encode its temporal structure in the embedded space of the last layer through multiclass supervised learning\. We will refer to these mappings in the GRU’s latent space asG⁡\(ps→pe\)G\(p\_\{s\}\\rightarrow p\_\{e\}\)\.

Fig\. 1:Methodology developed to analyze the criticality dynamics of MV3 systems\.### II\-AMV3 System and Generation of Training Dataset

We simulated the three\-state majority vote model \(MV3\) on a two\-dimensional square lattice ofN=282=784N=28^\{2\}=784agents with periodic boundary conditions, where each agentiiholds one of three discrete opinion statessi∈\{0,1,2\}s\_\{i\}\\in\\\{0,1,2\\\}\. The energy of the system is defined as

H⁡\(S\)=∑\(i,j\)δ⁡\(si,sj\),H\(S\)=\\sum\_\{\(i,j\)\}\\delta\(s\_\{i\},s\_\{j\}\),\(1\)whereδ\\deltais the Kronecker delta\. At each Monte Carlo step \(MCS\), agents are updated asynchronously: if the agent’s four nearest neighbors hold a majority opinion, the agent adopts it with probabilityp=1−qp=1\-q; otherwise it dissents and assumes a random state with probabilityq=1−pq=1\-p\. If no majority exists among neighbors, the agent adopts either competing state with equal probability\. One MCS corresponds toN=784N=784individual update attempts\.

For the static training dataset used to train the DBN, each simulation is initialized from a random configuration and evolved freely for at least 100,000 MCS to reach a steady state, after which a single snapshotS⁡\(p\)S\(p\)is recorded\. This process is repeated across a broad range of values ofpp, covering subcritical \(p<0\.75p<0\.75\), critical \(0\.75≤p≤0\.850\.75\\leq p\\leq 0\.85\), and supercritical \(p\>0\.85p\>0\.85\) regimes\. The theoretical critical value isqc≈0\.151q\_\{c\}\\approx 0\.151\(equivalentlypc≈0\.849p\_\{c\}\\approx 0\.849\)\[[11](https://arxiv.org/html/2608.18235#bib.bib2)\], but due to finite\-size effects on the28×2828\\times 28lattice, the observed critical noise value for this system isqc=0\.106q\_\{c\}=0\.106\(orp=0\.894p=0\.894\) at which a phase transition is observed, characterized by the coexistence of clusters of two or three opinion states simultaneously, giving rise to a variety of polarization possibilities\. Figure[2](https://arxiv.org/html/2608.18235#S2.F2)shows an example ofS⁡\(p\)S\(p\)systems for different values of the parameter p\. To calculate the system’s magnetization, we use vector sum in the complex plane \(similar to a Potts model withq=3q=3\) rather than a simple scalar\. Thus, each state is represented as a 2D vector\. States0=0s\_\{0\}=0corresponds to a vector pointing to the right \(0°\), states1=1s\_\{1\}=1to a vector pointing top\-left \(120°\), and states2=2s\_\{2\}=2to a vector pointing bottom\-left \(240°\)\. The magnetization is defined as:

M=1N​\|∑i=1Nei​θi\|=1N​\|∑i=1Nexp⁡\(i​2​π3​si\)\|M=\\frac\{1\}\{N\}\\left\|\\sum\_\{i=1\}^\{N\}e^\{i\\theta\_\{i\}\}\\right\|=\\frac\{1\}\{N\}\\left\|\\sum\_\{i=1\}^\{N\}\\exp\\left\(i\\frac\{2\\pi\}\{3\}s\_\{i\}\\right\)\\right\|\(2\)
whereN=784N=784, number of agents in the lattice\. The numerator indicates the magnitude \(Euclidean length\) of the net magnetization vector\. Thus, whenM≈0M\\approx 0, this indicates a system in which there is complete disagreement among the agents, whereas in the opposite case, whenM≈1M\\approx 1, this indicates general agreement on one of the three states\.

![Refer to caption](https://arxiv.org/html/2608.18235v1/fig2_pdf.png)

Fig\. 2:Examples of systemsS⁡\(p\)S\(p\)ofL2=282=784L^\{2\}=28^\{2\}=784for different values of the parameterpp, from more to less noise\.
### II\-BAsynchronous Trajectory Ensemble Generation

We construct a quasi\-static physics\-based trajectory ensemble designed to expose the learning pipeline to the full diversity of dynamical regimes associated with the MV3 phase transition\. The central motivation for a trajectory\-based approach, rather than static snapshot analysis, is that the most diagnostically relevant information about a system’s proximity to a critical point is carried by its temporal evolution, not by any single configuration\. Near the critical noiseqcq\_\{c\}, the relaxation timeτ\\tauof the system diverges asτ∝\|q−qc\|−ν​z\\tau\\propto\|q\-q\_\{c\}\|^\{\-\\nu z\}, whereν\\nuis the correlation\-length critical exponent andzzis the dynamic critical exponent\[[15](https://arxiv.org/html/2608.18235#bib.bib8),[2](https://arxiv.org/html/2608.18235#bib.bib9)\]\. This phenomenon, known as critical slowing down, means that a system traversing the transition at any finite rate will necessarily pass through a non\-equilibrium regime nearpcp\_\{c\}in which its configurations carry memory of the preceding phase\. Our ensemble is designed to systematically sample these non\-equilibrium passages, generating trajectories that encode the directional and temporal signatures of the transition that a static ensemble cannot provide\. Four trajectory modes are defined by their direction and endpoint relative to the observed critical pointpc≈0\.894p\_\{c\}\\approx 0\.894:

- •Mode 0— From Disorder to Criticality or approach from disorder \(p:0\.5→pcp\\colon 0\.5\\to p\_\{c\}, magnetization increases\)\.
- •Mode 1— From Order to Criticality or approach from order \(p:1\.0→pcp\\colon 1\.0\\to p\_\{c\}, magnetization decreases\)\.
- •Mode 2— From Criticality to Disorder or depart to disorder \(p:pc→0\.5p\\colon p\_\{c\}\\to 0\.5, magnetization decreases\); initialized after thermalization atpcp\_\{c\}\.
- •Mode 3— From Criticality to Order or depart to order \(p:pc→1\.0p\\colon p\_\{c\}\\to 1\.0, magnetization increases\); initialized after thermalization atpcp\_\{c\}\.

The four modes are chosen deliberately to include all four directional combinations of approach and departure relative to the critical point, starting from both stable phases\. This design forces the classifier to learn representations that are sensitive to thedirectionof parameter evolution and not merely to the instantaneous value of the order parameter\. Modes 0 and 3 share increasing magnetization, and Modes 1 and 2 share decreasing magnetization; distinguishing within these pairs requires information about the history and destination of the trajectory, which is precisely the temporal structure the GRU is designed to exploit\.

Each trajectoryT⁡\(ps→pe\)T\(p\_\{s\}\\rightarrow p\_\{e\}\)consists ofTTsequential snapshots\{S⁡\(pt\)\}t=1T\\\{S\(p\_\{t\}\)\\\}\_\{t=1\}^\{T\}for which at the startt=1t=1,p=psp=p\_\{s\}, and at the endt=Tt=T,p=pep=p\_\{e\}\. The control parameterppevolves step\-by\-step according to a randomized phase velocityv=Δ​p/\(T⋅u\)v=\\Delta p\\,/\\,\(T\\cdot u\), whereu∼𝒰⁡\(0\.65,0\.95\)u\\sim\\mathcal\{U\}\(0\.65,0\.95\)is drawn independently for each trajectory\. Thisasynchronousdesign, inspired by the Kibble\-Zurek framework for finite\-rate quenches\[[9](https://arxiv.org/html/2608.18235#bib.bib12)\], ensures that the critical point is crossed at a different time step in every trajectory\. As a consequence, any downstream model cannot exploit a fixed temporal index as a shortcut for phase identification; it must instead detect the structural changes in the latent representations that signal the transition\. Nearpcp\_\{c\}, the randomized velocity also naturally produces a range of non\-equilibrium conditions: faster traversals generate configurations more characteristic of the preceding phase \(analogous to the impulse regime of the Kibble\-Zurek mechanism\), while slower traversals allow the system more time to equilibrate locally\[[9](https://arxiv.org/html/2608.18235#bib.bib12)\]\. Modes 2 and 3, which depart from criticality, require an initial thermalization of 300 MCS atpcp\_\{c\}before the trajectory begins\. Once the target valuepep\_\{e\}is reached,ppis held fixed for the remaining steps, exposing the model to examples of stationary post\-transition behavior\. Figure[3](https://arxiv.org/html/2608.18235#S2.F3)shows an example of the simulated dynamics of various possible trajectories in each of the four possible modes\.

Fig\. 3:Example of non\-equilibrium phase transition dynamics\. The graph shows the magnetization evolution for 100 trajectories in each mode, withT=50T=50time steps\. The thick line represents the average\.#### Color\-Permutation Invariance and Symmetry Augmentation\.

The MV3 model’s transition rules are invariant under the simultaneous permutation of all opinion states, a symmetry formally described by the point groupC3​vC\_\{3v\}\[[1](https://arxiv.org/html/2608.18235#bib.bib20),[21](https://arxiv.org/html/2608.18235#bib.bib21)\]\. This means that the three opinion states\{0,1,2\}\\\{0,1,2\\\}are physically equivalent: no opinion label is privileged, and any global permutation of all agents’ states produces a configuration governed by the same statistical mechanics\. In social terms, this corresponds to the fact that the labels “opinion A,” “opinion B,” and “neutral” are interchangeable, what matters is the pattern of agreement and disagreement among agents, not the specific identity of the majority opinion\. This symmetry has a critical practical implication for machine learning: if the DBN and GRU are trained on trajectories in which, say, state00is always the majority in the ordered phase, the models may learn to recognize the specific color of the majority rather than the structural pattern of order itself\. Such a model would fail to generalize to trajectories in which state11or state22becomes dominant, even though these configurations are physically identical to the training examples under a relabeling of opinions\. To enforce color\-permutation invariance, each physical trajectory is augmented with two additional permuted copies via the mappingsi↦\(si\+k\)mod3s\_\{i\}\\mapsto\(s\_\{i\}\+k\)\\bmod 3fork∈\{1,2\}k\\in\\\{1,2\\\}\. For every physical simulation, this produces three latent paths through the DBN that are statistically equivalent but span all possible majority\-opinion labelings\. This augmentation triples the ensemble size without additional simulation cost and ensures that the learned representations are sensitive to the degree of order, the physically meaningful quantity, rather than to which specific opinion dominates\. Lattice states are normalized to\[0,1\]\[0,1\]via the mapping\{0,1,2\}↦\{0\.0,0\.5,1\.0\}\\\{0,1,2\\\}\\mapsto\\\{0\.0,\\,0\.5,\\,1\.0\\\}, consistent with the normalization used during DBN training\. The resulting ensemble contains approximately 60,000 snapshots\.

### II\-CDeep Belief Network and Latent Space Analysis

The DBN used in this work was pre\-trained on static equilibrium samplesS⁡\(p\)S\(p\)in a previous study\[[17](https://arxiv.org/html/2608.18235#bib.bib13)\]and its weights are held fixed throughout all experiments reported here\. The architecture consists of three stacked RBM layers: a Gaussian\-Bernoulli RBM \(GB\-RBM\) as the first layer, which handles the real\-valued three\-state inputs, followed by two Bernoulli\-Bernoulli RBM \(BB\-RBM\) layers\. The layer dimensions are784→4096→225→81784\\to 4096\\to 225\\to 81, trained greedily layer\-by\-layer using contrastive divergence\[[7](https://arxiv.org/html/2608.18235#bib.bib14)\]\. The GB\-RBM energy function is:

E⁡\(𝐯,𝐡\)=−∑i,jwi​jhjsjviσi−∑i\(vi−bi\)22​σi−∑j\(hj−cj\)22​sj,\\begin\{split\}E\(\\mathbf\{v\},\\mathbf\{h\}\)&=\-\\sum\_\{i,j\}w\_\{ij\}\\frac\{h\_\{j\}\}\{s\_\{j\}\}\\frac\{v\_\{i\}\}\{\\sigma\_\{i\}\}\-\\sum\_\{i\}\\frac\{\(v\_\{i\}\-b\_\{i\}\)^\{2\}\}\{2\\sigma\_\{i\}\}\\\\ \\qquad&\-\\sum\_\{j\}\\frac\{\(h\_\{j\}\-c\_\{j\}\)^\{2\}\}\{2s\_\{j\}\},\\end\{split\}\(3\)
whereviv\_\{i\}are the static system’s snapshots with valuesL⁡\(S⁡\(p\)\)L\(S\(p\)\)with values\{0\.0,0\.5,1\.0\}\\\{0\.0,\\,0\.5,\\,1\.0\\\}, andhjh\_\{j\}denote visible and hidden units, respectively;wi​jw\_\{ij\}are connection weights;bib\_\{i\}andcjc\_\{j\}are visible and hidden biases; andσi\\sigma\_\{i\},sjs\_\{j\}are standard deviations of visible and hidden units\[[10](https://arxiv.org/html/2608.18235#bib.bib15)\]\. The conditional probabilities for the GB\-RBM follow these equations\[[10](https://arxiv.org/html/2608.18235#bib.bib15)\]:

p⁡\(𝐯\|𝐡\)=N⁡\(𝐯\|𝐛\+∑jhj​wi​j,Σ2\),p\(\\mathbf\{v\}\|\\mathbf\{h\}\)=N\\Bigl\(\\mathbf\{v\}\|\\mathbf\{b\}\+\\sum\_\{j\}h\_\{j\}w\_\{ij\},\\Sigma^\{2\}\\Bigr\),\(4\)p⁡\(𝐡\|𝐯\)=N⁡\(𝐡\|𝐜\+∑ivi​wi​j,S2\)p\(\\mathbf\{h\}\|\\mathbf\{v\}\)=N\\Bigl\(\\mathbf\{h\}\|\\mathbf\{c\}\+\\sum\_\{i\}v\_\{i\}w\_\{ij\},S^\{2\}\\Bigr\)\(5\)
whereN⁡\(μ,Σ\)N\(\\mu,\\Sigma\)denotes the multivariate normal distribution with mean vectorμ\\muand covariance matrixΣ2\\Sigma^\{2\}, andS2S^\{2\}corresponds to the covariance matrix associated with the hidden units\. Both covariance matrices are kept fixed throughout training\[[8](https://arxiv.org/html/2608.18235#bib.bib16)\]\. The empirical covariance matrix computed from the training data is employed for the visible units, whereas for the hidden units we defineS=s​ImS=sI\_\{m\}, which implies statistical independence among hidden units\. Through experimentation, we determined thats=0\.5s=0\.5leads to the most stable convergence behavior\.

Once the GBRBM has been trained, the hidden unit activations from this first layer are fed as input to train a subsequent layer using a conventional RBM\. In other words, the hidden layer of the initial network acts as the visible layer for training the next RBM, which in this case is a Bernoulli\-Bernoulli RBM \(BBRBM\)\. For this second model, the update rules for the conditional probabilities of both visible and hidden units follow the standard formulation\[[6](https://arxiv.org/html/2608.18235#bib.bib17)\]\. Consequently, the full DBN is constructed by stacking multiple RBMs, each trained sequentially via a greedy, layer\-by\-layer unsupervised learning procedure\. Parameter estimation for every RBM is carried out using contrastive divergence \(CD\), a widely adopted learning algorithm for RBM training\[[7](https://arxiv.org/html/2608.18235#bib.bib14)\]\.

Once the DBN is trained, we feed trajectoriesT⁡\(ps→pe\)T\(p\_\{s\}\\rightarrow p\_\{e\}\)to the first layer and let the DBN encode them by propagating them forward through all three DBN layers via the learned conditional probabilitiesp⁡\(𝐡\(l\)∣𝐡\(l−1\)\)p\(\\mathbf\{h\}^\{\(l\)\}\\mid\\mathbf\{h\}^\{\(l\-1\)\}\), yielding an 81\-dimensional latent vector𝐳t∈𝐑81\\mathbf\{z\}\_\{t\}\\in\\mathbf\{R\}^\{81\}per snapshot\. Applying a model trained on static equilibrium configurations to out\-of\-equilibrium trajectory samples is a deliberate design choice: it probes whether the DBN’s learned representations remain informative about the dynamical regime beyond their training distribution\. The discriminative power of these representations is assessed through t\-SNE visualization\[[18](https://arxiv.org/html/2608.18235#bib.bib18)\]of the full ensemble of latent vectors\{𝐳t\}\\\{\\mathbf\{z\}\_\{t\}\\\}\(orL⁡\(ps→pe\)L\(p\_\{s\}\\rightarrow p\_\{e\}\)\) in Figure[4](https://arxiv.org/html/2608.18235#S2.F4), colored by trajectory mode, which reveals the degree to which the DBN latent space geometrically separates the four trajectory types prior to any temporal modeling\. At this point, it is clear that the DBN alone does not achieve robust discriminative power for each of the four trajectory modes, nor is it expected to, given that the DBN was trained on static samples rather than dynamic sample sequences\. To capture the dynamics, we use a Bi\-GRU\.

![Refer to caption](https://arxiv.org/html/2608.18235v1/fig_tsne_dbn.png)

Fig\. 4:2D t\-SNE projection of the latent representations𝐳t\\mathbf\{z\}\_\{t\}orL⁡\(ps→pe\)L\(p\_\{s\}\\rightarrow p\_\{e\}\)last layer of the DBN\. Left: The samples have been colored according to the trajectory mode to which they belong\. Right: The samples have been colored according to the system’s magnetization value\.
### II\-DGated Recurrent Unit for Trajectory Classification and Dynamic Latent Representation

To exploit the temporal structure of the encoded trajectories, each trajectory is reorganized as a sequential tensor of shape\(T×82\)\(T\\times 82\), where each row corresponds to the 81\-dimensional plus one additional unit indicating the physical magnetization of the system, which represents the DBN latent representation𝐳t\\mathbf\{z\}\_\{t\}of one snapshot\. This means that the GRU is trained to be “physics\-aware”\. The full ensemble is structured into a 3D input tensor of shape\(ntraj×T×82\)\(n\_\{\\text\{traj\}\}\\times T\\times 82\)for batch training, withT=50T=50time steps retained in full to preserve long\-range temporal correlations that distinguish trajectory modes sharing similar instantaneous magnetization levels\.

The recurrent architecture consists of two stacked Bidirectional Gated Recurrent Unit \(Bi\-GRU\) layers\[[4](https://arxiv.org/html/2608.18235#bib.bib19)\]\. Processing each sequence in both forward and backward directions allows the model to integrate contextual information from the entire trajectory at each time step, which is particularly important near the critical point where the direction of approach cannot be inferred from instantaneous features alone\. The first Bi\-GRU layer contains 64 hidden units per direction and returns the full sequence of hidden states, while the second Bi\-GRU layer contains 32 hidden units per direction\. This allows the first layer to extract low\-level temporal features \(fluctuations\) and the second to capture high\-level transition signatures\. Each layer includes a dropout at 20% regularization rate and batch normalization\. Then we add a pooling layerGlobalAveragePooling1Dwhich reduces the 3D tensor to a 2D output tensor calculating the average of all values of the lastT=50T=50time steps before passing the output to the dense layer\.

The hidden state update at each stepttfor the forward direction follows:

𝐡t=GRU​\(𝐳t,𝐡t−1g\),\\mathbf\{h\}\_\{t\}=\\text\{GRU\}\(\\mathbf\{z\}\_\{t\},\\,\\mathbf\{h\}^\{g\}\_\{t\-1\}\),\(6\)where𝐡t−1g\\mathbf\{h\}^\{g\}\_\{t\-1\}denotes the state of the hidden units of the Bi\-GRU at the previous time step, and with the backward direction processing the sequence in reverse, and the outputs of both directions are concatenated at each layer\. The final hidden state is passed to a dense layer of 32 units withReLUactivation, followed by a softmax output layer of 4 units, one for each trajectory mode:

𝐲^=softmax​\(𝐖o​ReLU​\(𝐖d​𝐡Tg\+𝐛d\)\+𝐛o\),\\hat\{\\mathbf\{y\}\}=\\text\{softmax\}\\\!\\left\(\\mathbf\{W\}\_\{o\}\\,\\text\{ReLU\}\(\\mathbf\{W\}\_\{d\}\\,\\mathbf\{h\}^\{g\}\_\{T\}\+\\mathbf\{b\}\_\{d\}\)\+\\mathbf\{b\}\_\{o\}\\right\),\(7\)where𝐖o\\mathbf\{W\}\_\{o\},𝐖d\\mathbf\{W\}\_\{d\},𝐛o\\mathbf\{b\}\_\{o\}, and𝐛d\\mathbf\{b\}\_\{d\}are learned weight matrices and bias vectors\. The model is trained as a 4\-class classifier using categorical cross\-entropy loss\.

The discriminative power of the GRU’s internal representations of the last 32 hidden units layer is evaluated through t\-SNE visualization\[[18](https://arxiv.org/html/2608.18235#bib.bib18)\]of the final hidden states𝐡T\\mathbf\{h\}\_\{T\}across the ensemble \(equivalent to theG⁡\(ps→pe\)G\(p\_\{s\}\\rightarrow p\_\{e\}\)in Figure[1](https://arxiv.org/html/2608.18235#S2.F1)\), colored by trajectory mode \(See Figure[5](https://arxiv.org/html/2608.18235#S2.F5)\)\. This analysis is performed in conjunction with the t\-SNE visualization of the DBN latent vectors described in the previous Subsection, forming a paired comparison that quantifies the additional trajectory\-type separation gained by processing the full temporal sequence beyond what the static DBN encoding provides at the snapshot level\.

![Refer to caption](https://arxiv.org/html/2608.18235v1/tsne_gru_T100_pdf.png)

Fig\. 5:2D t\-SNE projection of the latent representations𝐡tg\\mathbf\{h\}^\{g\}\_\{t\}orG⁡\(ps→pe\)G\(p\_\{s\}\\rightarrow p\_\{e\}\), the last layer of the 32 hidden units of the Bi\-GRU, usingT=50T=50time steps\. Left: the resulting projection for the train set\. Right: the resulting projection for the test set\. These projections were performed using 960 samples from the train and test set respectively, with approximately 240 samples for each of the four trajectories\.These results may justify the encouraging performance as a classifier\. The Bi\-GRU classifier was trained using a dataset of 1,200 labeled synthetic trajectories\. A stratified random split was employed, with 80% of the samples allocated to the training set and the remaining 20% reserved for testing\. The Bi\-GRU achieved an average classification accuracy of 1\.00 on the training set and 0\.99 on the test set, indicating excellent generalization performance\. The training loss converged rapidly, reaching a stable minimum within approximately five epochs, with no evidence of overfitting\. To assess the contribution of physically informed features, we trained an alternative Bi\-GRU using the same protocol but excluding the magnetization from the DBN latent representation\. Despite the removal of this information, the classifier maintained a high level of performance, achieving an average test accuracy of 0\.978\. Class\-wise evaluation revealed recall values of at least 0\.96 for all trajectory modes, while precision exceeded 0\.97 in all cases except for Mode 2 trajectories \(Crit→\\rightarrowDis\), for which the precision was 0\.91\. These results indicate that the temporal patterns encoded in the latent representations are highly informative for trajectory classification, even in the absence of explicit magnetization information\. To assess whether the DBN outperforms the MV3 system as an input to the Bi\-GRU in terms of representational capacity compared to a simpler dimension reduction technique, we applied principal component analysis \(PCA\) to all training samples\{S⁡\(pt\)\}t=1T\\\{S\(p\_\{t\}\)\\\}\_\{t=1\}^\{T\}\. We then trained the Bi\-GRU using the first 81 components, following the same training protocol\. The average classification accuracy achieved was 0\.995 and 0\.854 for the training and test sets, respectively\. We observed that on the test set, the precision is 0\.722 for Mode 2 trajectories \(Crit→\\rightarrowDis\)\. While these results are not bad, using PCA sacrifices generalization in trajectory mode discrimination compared to our original DBN\-based approach, which is vital for using the Bi\-GRU to detect the transitions in which the system finds itself\.

### II\-EApplication of the Bi\-GRU to “sense” the system’s state

Since the Bi\-GRU achieves excellent discrimination between the different types of trajectories that describe various scenarios in which the system may be entering or exiting a state of criticality, it could be useful to leverage this representational capability to “sense” the dynamics of a system that is not necessarily in equilibrium, and continuously feed sequences or “snapshots” of these dynamics into the model so that the recurrent network can indicate the system’s current state\.

For this experiment, we artificially generated long chains of MV3 using Monte Carlo simulation with a predeterminedppschedule\. This allows us to evaluate the model’s ability to detect phase changes in the system\. To illustrate the experiment, consider the following example: We generated a long trajectory of 500 steps for an MV3 system\. The system starts in equilibrium at the critical point \(p=0\.894p=0\.894\), and we proceed in five distinct phases to change the value ofppin order to produce a gradual increase in magnetization, followed by an abrupt decrease as the system enters the supercritical state\. The procedure is summarized as follows:

- •Phase 1: Criticality\. Fromt=0t=0tot=100t=100,p=0\.85p=0\.85constant\.
- •Phase 2: Ramp up\. Fromt=100t=100tot=200t=200, increasingppfrom 0\.85 to 0\.92\.
- •Phase 3: Supercriticality\. Fromt=200t=200tot=300t=300,p=0\.93p=0\.93constant\.
- •Phase 4: Quench\. Fromt=300t=300tot=350t=350, decreasingppfrom 0\.92 to 0\.70\.
- •Phase 5: Disorder or subcriticality: fromt=350t=350tot=500t=500,p=0\.70p=0\.70constant\.

Figure[6](https://arxiv.org/html/2608.18235#S2.F6)\(a\) shows the dynamics of the resulting system’s magnetization and the corresponding value ofppused to create this particular simulation\. We deliberately created an abrupt transition from a supercritical state \(high magnetization\) to a frozen state \(low magnetization\) to evaluate the Bi\-GRU’s ability to recognize this sudden change\. Once the system dynamics have been generated, each snapshotS⁡\(p,t\)S\(p,t\)is fed into the DBN to obtain the corresponding latent representation in the final layerL⁡\(S⁡\(p,t\)\)L\(S\(p,t\)\), yielding a sequence\{L⁡\(S⁡\(p,t\)\)\}t=1500\\\{L\(S\(p,t\)\)\\\}\_\{t=1\}^\{500\}of the system dynamics\. From this sequence, we create sliding windows ofT=50T=50time steps with 1\-step overlaps \(Stride = 1\), which are fed into the Bi\-GRU\. In this way, we allow the recurrent network to gradually sense, step by step, the type of trajectory \(one of the four\) on which the system is currently located\. The output of the Bi\-GRU is a probability vector generated by the softmax activation function, indicating the probability of each of the four trajectories\.

Figure[6](https://arxiv.org/html/2608.18235#S2.F6)\(b\) shows the results of the multiclass classification\. The plot confirms that the model acts as a highly sensitive phase detector rather than an anticipatory one\. It “senses” the change in real\-time by identifying the transient physics currently held within theT=50T=50window\. In Phase 1 \(t<200t<200\), even thoughp=0\.85p=0\.85\(criticality\), the slight upward drift in magnetization\|M\|\|M\|is being picked up by the Bi\-GRU as Mode 3 \(Crit→\\toOrd\)\. This shows the GRU is sensitive to the direction of the magnetization flow, not just the static value ofpp\. In the Ordered Phase Paradox \(t≈200−300t\\approx 200\-300\) is the most interesting part of the plot\. As\|M\|\|M\|stabilizes at a high value, the Bi\-GRU shifts to Mode 1 \(Ord→\\toCrit\)\. This is actually a very logical “deduction” by the model: in the training ensemble, a stable high\-magnetization state is the prerequisite for an “Order→\\toCritical” transition\. Since the system isn’t increasing in order anymore, the Bi\-GRU essentially “waits” for the decay\. In the Quench Detection \(t\>300t\>300\), the transition to Mode 2 \(Crit→\\toDis\) is remarkably sharp\. The model doesn’t predict the quench; it detects the entry of disordered frames into the sliding window\. Because the Bi\-GRU was trained with the magnetization as a 82nd feature, it likely triggers this class the moment the vector sum\|M\|\|M\|starts to tilt back toward the origin\.

Fig\. 6:System dynamics and GRU predictions\. \(a\) Evolution of the parameterpp\(orange dashed line\) and the magnetization\|M\|\|M\|\(blue line\)\. \(b\) Probabilities predicted by the Bi\-GRU for the four phase transitions\. The dotted vertical line indicates the onset of the quench\.

## IIIConclusions

This work investigated the discriminative power of a hierarchical deep learning pipeline, combining an unsupervised Deep Belief Network encoder with a supervised Bidirectional Gated Recurrent Unit classifier, for identifying four dynamically distinct trajectory types in the three\-state majority vote model\. The central finding is that neither static latent representations nor temporal sequence modeling alone is sufficient: the DBN, applied out of distribution to non\-equilibrium trajectory snapshots, produces only partial separation of the four trajectory modes in t\-SNE space, with modes sharing the same direction of magnetization change remaining difficult to distinguish\. The Bi\-GRU, by contrast, achieves near\-perfect separation of all four modes in its hidden state space, demonstrating that the temporal structure of the DBN\-encoded sequences carries the directional and historical information that instantaneous configurations cannot provide\. The sliding\-window sensing experiment further shows that the trained Bi\-GRU can track the system’s dynamical regime in real\-time along continuous MV3 trajectories, correctly identifying transitions between phases and responding sharply to abrupt quenches\. Together, these results confirm that the combination of unsupervised representation learning on equilibrium configurations with recurrent sequence modeling of non\-equilibrium trajectories constitutes an effective and physically principled framework for characterizing criticality dynamics in opinion systems\.

Some methodological limitations must be acknowledged\. First, the DBN was trained exclusively on static equilibrium samples, and its application to out\-of\-equilibrium trajectory snapshots introduces an intrinsic out\-of\-distribution risk: the learned representations may not capture all dynamically relevant features of non\-equilibrium configurations, particularly near the critical point where the system departs most strongly from its training distribution\. Second, all experiments are conducted on a finite28×2828\\times 28lattice, whose finite\-size effects round the discontinuous bulk transition into a continuous crossover and shift the effective critical point\. The degree to which the pipeline’s discriminative power generalizes to larger lattices, other network topologies, or systems with different universality classes remains an open question\.

Future work will pursue two primary directions\. The first is to reframe the Bi\-GRU training objective from classification of the current trajectory mode, asensingtask, toward anticipation of an impending transition, that is, anearly warningtask in which the model must assign elevated probability to an approaching critical crossing before it is observed in the sliding window\[[16](https://arxiv.org/html/2608.18235#bib.bib22)\]\. This requires redesigning the training labels to reflect proximity to the critical point rather than trajectory identity, and may benefit from incorporating uncertainty quantification to provide calibrated warning signals\. The second direction is to apply the full pipeline to real\-world time series from social and financial systems, such as sequences of collective sentiment indicators, social network activity patterns, or financial return volatility, where the existence and timing of critical transitions is not known a priori\. Such an application would test the extent to which representations learned from a stylized model can transfer to the richer, noisier dynamics of empirical data, and would constitute a concrete step toward machine learning–driven early warning systems for collective social behavior\.

## Acknowledgment

The authors thank ANID FONDECYT 1250386, ANID FONDECYT 1230315, ANID\-MILENIO\-NCN2024\_103, ANID\-MILENIO\-NCN2024\_047, and Centro de Modelamiento Matemático \(CMM\) FB210005, BASAL funds for centers of excellence from ANID\-Chile\.

## References

- \[1\]\(1999\)Universal behavior in an irreversible model with c 3 v symmetry\.Physical Review E60\(4\),pp\. 3666\.Cited by:[§II\-B](https://arxiv.org/html/2608.18235#S2.SS2.SSSx1.p1.1)\.
- \[2\]T\. M\. Bury, R\. Sujith, I\. Pavithran, M\. Scheffer, T\. M\. Lenton, M\. Anand, and C\. T\. Bauch\(2021\)Deep learning for early warning signals of tipping points\.Proceedings of the National Academy of Sciences118\(39\),pp\. e2106140118\.Cited by:[§I](https://arxiv.org/html/2608.18235#S1.p3.1),[§II\-B](https://arxiv.org/html/2608.18235#S2.SS2.p1.1)\.
- \[3\]J\. Carrasquilla and R\. G\. Melko\(2017\)Machine learning phases of matter\.Nature Physics13\(5\),pp\. 431–434\.Cited by:[§I](https://arxiv.org/html/2608.18235#S1.p3.1)\.
- \[4\]K\. Cho, B\. Van Merriënboer, Ç\. Gulçehre, D\. Bahdanau, F\. Bougares, H\. Schwenk, and Y\. Bengio\(2014\)Learning phrase representations using rnn encoder–decoder for statistical machine translation\.InProceedings of the 2014 conference on empirical methods in natural language processing \(EMNLP\),pp\. 1724–1734\.Cited by:[§II\-D](https://arxiv.org/html/2608.18235#S2.SS4.p2.1)\.
- \[5\]M\. J\. De Oliveira\(1992\)Isotropic majority\-vote model on a square lattice\.Journal of Statistical Physics66\(1\),pp\. 273–281\.Cited by:[§I](https://arxiv.org/html/2608.18235#S1.p1.1)\.
- \[6\]A\. Fischer and C\. Igel\(2014\)Training restricted boltzmann machines: an introduction\.Pattern Recognition47\(1\),pp\. 25–39\.Cited by:[§II\-C](https://arxiv.org/html/2608.18235#S2.SS3.p5.1)\.
- \[7\]G\. E\. Hinton\(2002\)Training products of experts by minimizing contrastive divergence\.Neural computation14\(8\),pp\. 1771–1800\.Cited by:[§II\-C](https://arxiv.org/html/2608.18235#S2.SS3.p1.1),[§II\-C](https://arxiv.org/html/2608.18235#S2.SS3.p5.1)\.
- \[8\]R\. Karakida, M\. Okada, and S\. Amari\(2016\)Dynamical analysis of contrastive divergence learning: restricted boltzmann machines with gaussian visible units\.Neural Networks79,pp\. 78–87\.Cited by:[§II\-C](https://arxiv.org/html/2608.18235#S2.SS3.p4.1)\.
- \[9\]H\. Kou and P\. Li\(2023\)Varying quench dynamics in the transverse ising chain: the kibble\-zurek, saturated, and presaturated regimes\.Physical Review B108\(21\),pp\. 214307\.Cited by:[§II\-B](https://arxiv.org/html/2608.18235#S2.SS2.p4.1)\.
- \[10\]R\. Liao, S\. Kornblith, M\. Ren, D\. J\. Fleet, and G\. Hinton\(2022\)Gaussian\-bernoulli rbms without tears\.arXiv preprint arXiv:2210\.10318\.Cited by:[§II\-C](https://arxiv.org/html/2608.18235#S2.SS3.p2.1)\.
- \[11\]F\. Lima\(2012\)Three\-state majority\-vote model on square lattice\.Physica A: Statistical Mechanics and its Applications391\(4\),pp\. 1753–1758\.Cited by:[§I](https://arxiv.org/html/2608.18235#S1.p1.1),[§II\-A](https://arxiv.org/html/2608.18235#S2.SS1.p2.1)\.
- \[12\]E\. M\. Luz and F\. W\. Lima\(2007\)Majority\-vote on directed small\-world networks\.International Journal of Modern Physics C18\(08\),pp\. 1251–1261\.Cited by:[§I](https://arxiv.org/html/2608.18235#S1.p2.1)\.
- \[13\]J\. S\. Neto, D\. Alencar, L\. Brito, G\. Alves, F\. Lima, A\. Macedo\-Filho, R\. Ferreira, and T\. Alves\(2025\)Supervised and unsupervised deep learning applied to the majority vote model\.Physica A: Statistical Mechanics and its Applications,pp\. 131208\.Cited by:[§I](https://arxiv.org/html/2608.18235#S1.p4.1)\.
- \[14\]J\. F\. Neto, D\. S\. Alencar, L\. T\. Brito, G\. A\. Alves, F\. W\. S\. Lima, A\. M\. Filho, R\. S\. Ferreira, and T\. F\. Alves\(2025\)Deep learning of the biswas–chatterjee–sen model\.Entropy27\(11\),pp\. 1173\.Cited by:[§I](https://arxiv.org/html/2608.18235#S1.p4.1)\.
- \[15\]M\. Scheffer, J\. Bascompte, W\. A\. Brock, V\. Brovkin, S\. R\. Carpenter, V\. Dakos, H\. Held, E\. H\. Van Nes, M\. Rietkerk, and G\. Sugihara\(2009\)Early\-warning signals for critical transitions\.Nature461\(7260\),pp\. 53–59\.Cited by:[§I](https://arxiv.org/html/2608.18235#S1.p3.1),[§II\-B](https://arxiv.org/html/2608.18235#S2.SS2.p1.1)\.
- \[16\]M\. A\. Valle, J\. Lavín, and F\. Urbina\(2025\)Stock market bubble warning: a restricted boltzmann machine approach using volatility–return sequences\.Applied Sciences15\(10\),pp\. 5613\.Cited by:[§I](https://arxiv.org/html/2608.18235#S1.p1.1),[§III](https://arxiv.org/html/2608.18235#S3.p3.1)\.
- \[17\]M\. A\. Valle and G\. A\. Ruz\(2026\)Exploration of the generative capabilities of boltzmann machines applied to social systems under the majority rule\.Note:Paper submitted to the IEEE International Conference on Systems, Man, and Cybernetics, Bellevue, WA, USA\.Cited by:[§II\-C](https://arxiv.org/html/2608.18235#S2.SS3.p1.1)\.
- \[18\]L\. Van der Maaten and G\. Hinton\(2008\)Visualizing data using t\-sne\.\.Journal of machine learning research9\(11\)\.Cited by:[§II\-C](https://arxiv.org/html/2608.18235#S2.SS3.p6.1),[§II\-D](https://arxiv.org/html/2608.18235#S2.SS4.p4.1)\.
- \[19\]E\. van Nieuwenburg, E\. Bairey, and G\. Refael\(2018\)Learning phase transitions from dynamics\.Physical Review B98\(6\),pp\. 060301\.Cited by:[§I](https://arxiv.org/html/2608.18235#S1.p3.1)\.
- \[20\]E\. P\. Van Nieuwenburg, Y\. Liu, and S\. D\. Huber\(2017\)Learning phase transitions by confusion\.Nature Physics13\(5\),pp\. 435–439\.Cited by:[§I](https://arxiv.org/html/2608.18235#S1.p3.1)\.
- \[21\]A\. L\. Vilela, C\. Wang, K\. P\. Nelson, and H\. E\. Stanley\(2019\)Majority\-vote model for financial markets\.Physica A: Statistical Mechanics and its Applications515,pp\. 762–770\.Cited by:[§II\-B](https://arxiv.org/html/2608.18235#S2.SS2.SSSx1.p1.1)\.
- \[22\]B\. J\. Zubillaga, A\. L\. Vilela, M\. Wang, R\. Du, G\. Dong, and H\. E\. Stanley\(2022\)Three\-state majority\-vote model on small\-world networks\.Scientific Reports12\(1\),pp\. 282\.Cited by:[§I](https://arxiv.org/html/2608.18235#S1.p2.1)\.