Adaptive Multi-Branching for Shallow Decision Tree Induction

# Adaptive Multi-Branching for Shallow Decision Tree Induction
Source: [https://arxiv.org/html/2608.29262](https://arxiv.org/html/2608.29262)
Hanul Park1, Jeonghoon Choi1, Juseong Kim1, Sanghun Sel1, Giltae Song1,2,\*Affiliation:1Department of Information Convergence Engineering, Pusan National University, Busan, Rep\. of Korea 2School of Computer Science and Engineering, Pusan National University, Busan, Rep\. of Korea \*Corresponding author:gsong@pusan\.ac\.kr hanul\.park@gmail\.com, jeonghoonchoi@pusan\.ac\.kr kjs\_0322@naver\.com, poohnim5@naver\.com

###### Abstract

Decision trees are attractive for tabular prediction tasks because each prediction follows an interpretable sequence of feature\-threshold tests\. Under a strict maximum\-depth budget, however, conventional binary trees can be under\-expressive, since each internal node makes only a single threshold decision\. We study shallow\-depth tree induction, where the goal is to improve accuracy while keeping root\-to\-leaf paths short\. We propose the Multi\-Branch Neural Decision Tree with Adaptive Pruning \(MBNDT\), a single axis\-aligned tree trained end\-to\-end with differentiable multi\-way splits\. Each internal node learns ordered thresholds over a selected feature and a branch mask that adapts its effective arity, and the trained model is converted to a deterministic single\-path tree for inference\. Across 21 OpenML binary\-classification benchmarks, MBNDT achieves the best average rank and mean balanced accuracy among depth\-constrained single\-tree baselines; a controlled ablation isolates multi\-way splitting as the source of the gain\. These gains come with an explicit trade\-off: MBNDT realizes more leaves than the other single\-tree baselines, making it best suited when accuracy under short, bounded decision paths is prioritized over minimal global tree size\.

###### Index Terms:

Decision tree, Interpretability, Explainable machine learning, Tabular data, Classification

††footnotetext:© 2026 IEEE\. Personal use of this material is permitted\. Permission from IEEE must be obtained for all other uses, in any current or future media, including reprinting/republishing this material for advertising or promotional purposes, creating new collective works, for resale or redistribution to servers or lists, or reuse of any copyrighted component of this work in other works\.## IIntroduction

Decision trees remain a central model class for tabular prediction tasks because of their intrinsic interpretability\. Unlike black\-box models, a decision tree maps each input to a deterministic root\-to\-leaf path consisting of feature\-threshold tests, allowing individual predictions to be inspected as rule\-like decision processes\. Accordingly, this path\-based structure is particularly attractive in settings where predictions must be explained, audited, or implemented through a small number of sequential decisions\[[1](https://arxiv.org/html/2608.29262#bib.bib38),[2](https://arxiv.org/html/2608.29262#bib.bib11),[3](https://arxiv.org/html/2608.29262#bib.bib1)\]\.

Although decision trees are commonly regarded as interpretable models, their interpretability depends on human cognitive factors and the intended use context; no single global complexity measure can fully capture it\[[4](https://arxiv.org/html/2608.29262#bib.bib9),[5](https://arxiv.org/html/2608.29262#bib.bib8),[2](https://arxiv.org/html/2608.29262#bib.bib11)\]\. For example, a tree may contain many leaves while assigning each instance through a short path; conversely, a globally compact tree may contain paths that are too deep for practical inspection\. Because predictions are typically interpreted one instance at a time, path length is considered as a proxy for inspection effort while reporting leaf count to expose the corresponding global\-size trade\-off\[[5](https://arxiv.org/html/2608.29262#bib.bib8),[6](https://arxiv.org/html/2608.29262#bib.bib10)\]\. Shallow trees are therefore especially relevant in settings where predictions must be manually reviewed, audited, or implemented as rule\-like decision protocols, such as high\-stakes decision support and clinical risk stratification\[[3](https://arxiv.org/html/2608.29262#bib.bib1),[7](https://arxiv.org/html/2608.29262#bib.bib14)\]\. In this paper, we study*shallow\-depth tree induction*, where the maximum decision depth is strictly constrained, e\.g\.,D≤4D\\leq 4\. In this regime, the goal is not merely to reduce the total number of leaves, but to improve predictive accuracy under a short per\-instance decision\-path budget\.

Conventional binary decision trees, however, face an expressivity bottleneck in this shallow\-depth regime\. A binary tree of depthDDcan represent at most2D2^\{D\}leaves, and each internal node partitions its region using only a single threshold decision\. WhenDDis small, this coupling between path length and partitioning capacity can make a single binary tree under\-expressive for heterogeneous tabular decision boundaries\. Specifically, greedy methods\[[8](https://arxiv.org/html/2608.29262#bib.bib2),[9](https://arxiv.org/html/2608.29262#bib.bib15)\]make locally suboptimal root\-level decisions that are difficult to correct under a small depth budget; solver\-based methods improve global search but typically operate over binary or restricted candidate split structures\[[10](https://arxiv.org/html/2608.29262#bib.bib16)\]; and differentiable tree methods often retain binary routing, soft prediction, or ensemble\-style formulations that are hard to interpret\.

Our key idea is to improve the local expressivity of each decision step by dividing a selected feature into multiple ordered intervals using a multi\-way split\. We propose the*Multi\-Branch Neural Decision Tree with Adaptive Pruning*\(MBNDT\)111Implementation: \[https://github\.com/hanulpark98/MBNDT\], an axis\-aligned shallow decision tree trained end\-to\-end by gradient\-based optimization\. Each internal node learns a feature selector and a set of strictly ordered thresholds that define a differentiable multi\-way split over the selected feature\. Moreover, MBNDT avoids node\-wise greedy split selection by jointly optimizing split features, thresholds and leaf predictions\. To avoid unnecessarily using the full branching factor at every node, MBNDT introduces learnable branch masks that adapt the effective arity of each split during training\. After training, the learned routing structure and branch masks are converted into a deterministic pruned tree for inference, yielding a single root\-to\-leaf prediction path for each instance\.

Our empirical study evaluates MBNDT against greedy, solver\-based, and gradient\-based single\-tree baselines under fixed shallow\-depth budgets\. Across the evaluated datasets, MBNDT achieves the best average rank and highest mean balanced accuracy among the depth\-constrained single\-tree learners, while sparsification ablations show that branch masks and post\-hoc pruning reduce realized leaves without degrading balanced accuracy\. These gains carry an explicit complexity trade\-off—MBNDT realizes more leaves than the other single\-tree baselines—so it is best suited to settings that prioritize accuracy under short per\-instance decision paths over minimal global tree size\.

Our contributions are as follows:

- •Adaptive multi\-way branching\.We introduce differentiable, axis\-aligned multi\-way interval splits: each internal node selects a single feature and learns ordered thresholds that partition it into several regions, with split features, thresholds, and leaf predictions optimized jointly by gradient descent rather than greedily\.
- •Structure sparsification for differentiable trees\.We adapt each node’s effective arity with learnable branch masks, control tree size during training through a differentiable leaf\-budget penalty, and apply post\-hoc train\-path pruning—together converting the nominalBB\-ary tree into a compact effective tree with one deterministic path per input\.
- •Accuracy–depth–size analysis\.We benchmark MBNDT against greedy, solver\-based, and gradient\-based single\-tree baselines under a shared depth budget, reporting predictive accuracy, decision\-path length, and realized leaves, with ablations isolating multi\-way branching and the sparsification mechanisms\.

## IIRelated Work

### II\-ARule Simplicity and Explanation Redundancy

Several studies analyze tree interpretability through the complexity of individual root\-to\-leaf rules\. Souza et al\.\[[6](https://arxiv.org/html/2608.29262#bib.bib10)\]define explanation size as the number of distinct attributes appearing along a decision path\. A complete path, however, need not be a minimal explanation: Izza et al\.\[[11](https://arxiv.org/html/2608.29262#bib.bib12)\]study redundant path conditions, while McTavish et al\.\[[12](https://arxiv.org/html/2608.29262#bib.bib13)\]show that predictively equivalent trees can induce different evaluation processes\. These concerns are complementary to our scope: we bound the number of executed decision steps rather than claim subset\-minimal explanations\.

### II\-BGreedy Decision Tree Induction

Classical decision tree learners typically use a top\-down recursive partitioning strategy: at each node, they select the feature and threshold that maximizes a local impurity reduction, such as information gain or the Gini index, and then recurse on the resulting children\. Representative algorithms include ID3\[[13](https://arxiv.org/html/2608.29262#bib.bib7)\], C4\.5\[[9](https://arxiv.org/html/2608.29262#bib.bib15)\], and CART\[[8](https://arxiv.org/html/2608.29262#bib.bib2)\]\. These methods are fast and scalable, but optimizing each split in isolation can produce globally suboptimal trees, a limitation that is particularly consequential under shallow depth budgets\[[10](https://arxiv.org/html/2608.29262#bib.bib16)\]\.

### II\-COptimal Decision Tree Search

Beyond greedy induction, optimal\-tree methods formulate tree learning as a discrete global optimization problem, typically under fixed depth, leaf, or feature budgets\. Mixed\-integer programming approaches encode routing, split selection, and leaf predictions and solve the resulting formulation using branch\-and\-bound\[[10](https://arxiv.org/html/2608.29262#bib.bib16)\]\. Specialized methods improve scalability through dynamic programming, bounds, and search reuse\[[14](https://arxiv.org/html/2608.29262#bib.bib19),[15](https://arxiv.org/html/2608.29262#bib.bib18),[16](https://arxiv.org/html/2608.29262#bib.bib21),[17](https://arxiv.org/html/2608.29262#bib.bib20),[18](https://arxiv.org/html/2608.29262#bib.bib22)\]\. Recent work also reduces reliance on coarse discretization: ConTree directly optimizes continuous\-feature thresholds using dynamic programming with branch\-and\-bound\[[19](https://arxiv.org/html/2608.29262#bib.bib23)\], while SPLIT combines bounded lookahead with greedy lower\-level splits for near\-optimal search\[[20](https://arxiv.org/html/2608.29262#bib.bib37)\]\.

### II\-DMulti\-way Splitting for Decision Trees

Multi\-way splits have long been used for categorical attributes in ID3/C4\.5\-style and CHAID trees\[[13](https://arxiv.org/html/2608.29262#bib.bib7),[9](https://arxiv.org/html/2608.29262#bib.bib15),[21](https://arxiv.org/html/2608.29262#bib.bib3)\]\. Numerical variants search for multiple thresholds at each node to obtain smaller or more expressive trees\[[22](https://arxiv.org/html/2608.29262#bib.bib5),[23](https://arxiv.org/html/2608.29262#bib.bib4)\]\. More recently, path\-based mixed\-integer formulations with column generation have been used to learn constrained optimal multiway\-split trees\[[24](https://arxiv.org/html/2608.29262#bib.bib17)\]\. The earlier node\-wise methods, however, do not optimize the structure jointly across levels, and higher arity can fragment the data and increase overfitting, potentially requiring post\-hoc pruning\.

### II\-EDifferentiable Decision Trees

Another line of work replaces node\-wise greedy induction with gradient\-based optimization\. These methods replace discrete split selection with differentiable relaxations or surrogate gradients, enabling tree parameters to be optimized by backpropagation\. Good et al\.\[[25](https://arxiv.org/html/2608.29262#bib.bib40)\]alternate sparse feature learning with differentiable tree construction to obtain compact trees\. Norouzi et al\.\[[26](https://arxiv.org/html/2608.29262#bib.bib26)\]jointly optimize oblique splits across levels and leaf parameters, while DTSemNet\[[27](https://arxiv.org/html/2608.29262#bib.bib39)\]learns hard oblique trees through a neural\-network encoding\. Soft\-routing methods use differentiable path probabilities\[[28](https://arxiv.org/html/2608.29262#bib.bib28),[29](https://arxiv.org/html/2608.29262#bib.bib30)\]\. Particularly relevant, DNDT learns feature\-wise multi\-interval cut points by gradient descent, and D3T adapts the number of cut points per feature\[[30](https://arxiv.org/html/2608.29262#bib.bib24),[31](https://arxiv.org/html/2608.29262#bib.bib25)\]\. These methods discretize features globally, whereas MBNDT places adaptive multi\-way splits at internal nodes of a recursive tree\. GradTree instead learns hard axis\-aligned trees through surrogate gradients, while GRANDE extends gradient\-trained trees to ensembles\[[32](https://arxiv.org/html/2608.29262#bib.bib32),[33](https://arxiv.org/html/2608.29262#bib.bib33)\]\. MBNDT further converts the learned structure into a deterministic pruned tree for single\-path inference\.

![Refer to caption](https://arxiv.org/html/2608.29262v1/figure1_v7.png)Fig\. 1:Overview of MBNDT with prespecified maximum branching factorB=3B=3\.Left: a nominal depth\-2 ternary tree with mask\-suppressed branches\.Right: node\-level routing, where a selected feature is partitioned by ordered thresholds into soft interval probabilities, reweighted by branch masks, and hard\-routed to a single child\.
### II\-FPositioning

The individual ingredients of MBNDT are not new in isolation: multi\-way numerical splits, gradient\-based tree training, and global threshold optimization have each been studied separately above\. Our contribution is their unification into a decision tree learner that learns ordered*multi\-way*interval splits with continuous thresholds and adapts each node’s*effective arity*, while remaining a single axis\-aligned tree\.

## IIIMBNDT: Architecture, Training, and Inference

We formulate the learning ofMBNDTas the joint optimization of the parametersθ\\thetaof a predefined decision treeTθT\_\{\\theta\}with branching factorBBat each internal node and depthDD\. An axis\-aligned node selects one input feature, and routing denotes assigning an input to one child according to the interval containing that feature value\. The learnable parameters consist of node\-wise split feature parameters, split threshold parameters, branch masks, andBDB^\{D\}leaf logits\. Figure[1](https://arxiv.org/html/2608.29262#S2.F1)illustrates the node\-level pipeline\.

Given a labeled training set𝒮=\{\(𝐱i,yi\)\}i=1n\\mathcal\{S\}=\\\{\(\\mathbf\{x\}\_\{i\},y\_\{i\}\)\\\}\_\{i=1\}^\{n\}with𝐱i∈ℝp\\mathbf\{x\}\_\{i\}\\in\\mathbb\{R\}^\{p\}and binary labelyi∈\{0,1\}y\_\{i\}\\in\\\{0,1\\\}, MBNDT outputs a scalar logitfθ​\(𝐱\)∈ℝf\_\{\\theta\}\(\\mathbf\{x\}\)\\in\\mathbb\{R\}\. Its sigmoid transformσ​\(fθ​\(𝐱\)\)\\sigma\(f\_\{\\theta\}\(\\mathbf\{x\}\)\)is interpreted as the estimated probability of the positive class\. In our experiments, we train the model using binary cross\-entropy together with a leaf\-budget regularization term defined as:

minθ⁡1n​∑i=1nℒBCE​\(yi,fθ​\(𝐱i\)\)\+ℒbudget​\(θ\),\\min\_\{\\theta\}\\;\\frac\{1\}\{n\}\\sum\_\{i=1\}^\{n\}\\mathcal\{L\}\_\{\\mathrm\{BCE\}\}\\\!\\left\(y\_\{i\},f\_\{\\theta\}\(\\mathbf\{x\}\_\{i\}\)\\right\)\\;\+\\;\\mathcal\{L\}\_\{\\mathrm\{budget\}\}\(\\theta\),\(1\)We defer the detailed form ofℒbudget\\mathcal\{L\}\_\{\\mathrm\{budget\}\}to Section[IV\-B](https://arxiv.org/html/2608.29262#S4.SS2)\.

### III\-ASplit Feature Selection

At internal nodejj, we associate nodejjwith a vector of learnable*feature logits*𝐬j∈ℝp\\mathbf\{s\}\_\{j\}\\in\\mathbb\{R\}^\{p\}, whosekk\-th entrysj,ks\_\{j,k\}represents the tendency to select featurekkat nodejj\. These logits are then converted into a sparse probability vector over features usingα\\alpha\-entmax\[[34](https://arxiv.org/html/2608.29262#bib.bib35)\]:

𝐰j=entmax1\.5⁡\(𝐬j\)\.\\mathbf\{w\}\_\{j\}\\;=\\;\\operatorname\{entmax\}\_\{1\.5\}\\\!\\left\(\\mathbf\{s\}\_\{j\}\\right\)\.\(2\)Unlike other relaxation functions \(e\.g\., softmax, Gumbel–Softmax\), entmax can assign*exact zeros*to low\-scoring features, yielding a highly sparse candidate set per node while remaining differentiable; this sparsity is often empirically associated with more stable optimization and improved interpretability in differentiable routing/tree models\[[32](https://arxiv.org/html/2608.29262#bib.bib32),[35](https://arxiv.org/html/2608.29262#bib.bib27)\]\.

To guarantee*axis\-aligned*splits, nodejjmust ultimately select a*single*feature\. We therefore harden𝐰j\\mathbf\{w\}\_\{j\}viaarg⁡max\\arg\\maxin the forward pass \(as a one\-hot indicator\) and use a straight\-through \(ST\) estimator\[[36](https://arxiv.org/html/2608.29262#bib.bib34)\]for backpropagation, following the prior studies\[[32](https://arxiv.org/html/2608.29262#bib.bib32),[37](https://arxiv.org/html/2608.29262#bib.bib36)\]:

𝐰~j=𝐰j\+sg⁡\(onehot⁡\(arg⁡maxk⁡wj,k\)−𝐰j\),\\tilde\{\\mathbf\{w\}\}\_\{j\}\\;=\\;\\mathbf\{w\}\_\{j\}\\;\+\\;\\mathrm\{sg\}\\\!\\Big\(\\mathrm\{onehot\}\(\\arg\\max\_\{k\}\\,w\_\{j,k\}\)\-\\mathbf\{w\}\_\{j\}\\Big\),\(3\)wheresg⁡\(⋅\)\\mathrm\{sg\}\(\\cdot\)denotes the stop\-gradient operator\. This construction uses the hard one\-hot choice in the forward pass, while in the backward pass it propagates gradients as if𝐰~j=𝐰j\\tilde\{\\mathbf\{w\}\}\_\{j\}=\\mathbf\{w\}\_\{j\}, computed via theentmax1\.5\\mathrm\{entmax\}\_\{1\.5\}Jacobian:

∂𝐰~j∂𝐬j≈∂𝐰j∂𝐬j=Jentmax1\.5​\(𝐬j\)\.\\frac\{\\partial\\tilde\{\\mathbf\{w\}\}\_\{j\}\}\{\\partial\\mathbf\{s\}\_\{j\}\}\\;\\approx\\;\\frac\{\\partial\\mathbf\{w\}\_\{j\}\}\{\\partial\\mathbf\{s\}\_\{j\}\}\\;=\\;J\_\{\\mathrm\{entmax\}\_\{1\.5\}\}\(\\mathbf\{s\}\_\{j\}\)\.\(4\)

### III\-BMulti\-branch routing via ordered thresholds

A standard binary decision\-tree node learns a single threshold and partitions its input into two regions\. In contrast, each internal node of MBNDT learnsB−1B\{\-\}1*ordered*thresholds and partitions the corresponding node score intoBBordered regions\. This requires the thresholdstj,1<⋯<tj,B−1t\_\{j,1\}<\\cdots<t\_\{j,B\-1\}to be strictly increasing\. Rather than enforcing this constraint via per\-iteration sorting, we parameterize the thresholds as cumulative sums ofsoftplus\\operatorname\{softplus\}\-transformed learnable gap logitsδj,r\\delta\_\{j,r\}\. For nodejj, given a learnable base thresholdtj,0t\_\{j,0\}, the ordered thresholds are defined as

gj,r=softplus⁡\(δj,r\)\>0,𝜹j∈ℝB−1,g\_\{j,r\}=\\operatorname\{softplus\}\(\\delta\_\{j,r\}\)\>0,\\quad\\boldsymbol\{\\delta\}\_\{j\}\\in\\mathbb\{R\}^\{B\-1\},\(5\)tj,k=tj,0\+∑r=1kgj,r,k=1,…,B−1,t\_\{j,k\}=t\_\{j,0\}\+\\sum\_\{r=1\}^\{k\}g\_\{j,r\},\\quad k=1,\\dots,B\-1,\(6\)
which ensurestj,1<⋯<tj,B−1t\_\{j,1\}<\\cdots<t\_\{j,B\-1\}by construction\. Thesoftplus⁡\(x\)=log⁡\(1\+ex\)\\operatorname\{softplus\}\(x\)=\\log\(1\+e^\{x\}\)transformation maps unconstrained real\-valued parameters to strictly positive gaps, thereby preventing coincident thresholds while remaining smooth for gradient\-based optimization\.

Given the ordered thresholds\{tj,k\}k=1B−1\\\{t\_\{j,k\}\\\}\_\{k=1\}^\{B\-1\}, nodejjroutes an input𝐱\\mathbf\{x\}according to its selected\-feature scorezj​\(𝐱\)=𝐱⊤​𝐰~jz\_\{j\}\(\\mathbf\{x\}\)=\\mathbf\{x\}^\{\\top\}\\tilde\{\\mathbf\{w\}\}\_\{j\}\. Because hard assignment ofzj​\(𝐱\)z\_\{j\}\(\\mathbf\{x\}\)to one of theBBintervals is non\-differentiable, we instead use a soft binning scheme based on cumulative logistic probabilities\. For a temperatureτcdf\>0\\tau\_\{\\mathrm\{cdf\}\}\>0, we define

uj,k\(𝐱\)=tj,k−zj​\(𝐱\)τcdf,k=1,…,B−1,u\_\{j,k\}\(\\mathbf\{x\}\)\\;=\\;\\frac\{t\_\{j,k\}\-z\_\{j\}\(\\mathbf\{x\}\)\}\{\\tau\_\{\\mathrm\{cdf\}\}\},\\quad k=1,\\dots,B\-1,\(7\)and define the cumulative terms asCj,k​\(𝐱\)=σ⁡\(uj,k​\(𝐱\)\)C\_\{j,k\}\(\\mathbf\{x\}\)=\\sigma\(u\_\{j,k\}\(\\mathbf\{x\}\)\), whereσ⁡\(u\)=1/\(1\+e−u\)\\sigma\(u\)=1/\(1\+e^\{\-u\}\)\. TheBBsoft routing probabilities are then obtained from adjacent differences:

πj,1​\(𝐱\)=Cj,1​\(𝐱\),πj,k​\(𝐱\)=Cj,k\(𝐱\)−Cj,k−1\(𝐱\),k=2,…,B−1,πj,B​\(𝐱\)=1−Cj,B−1​\(𝐱\)\.\\begin\{split\}\\pi\_\{j,1\}\(\\mathbf\{x\}\)&=C\_\{j,1\}\(\\mathbf\{x\}\),\\\\ \\pi\_\{j,k\}\(\\mathbf\{x\}\)&=C\_\{j,k\}\(\\mathbf\{x\}\)\-C\_\{j,k\-1\}\(\\mathbf\{x\}\),\\qquad k=2,\\dots,B\-1,\\\\ \\pi\_\{j,B\}\(\\mathbf\{x\}\)&=1\-C\_\{j,B\-1\}\(\\mathbf\{x\}\)\.\\end\{split\}\(8\)
This defines a differentiable approximation to interval membership that becomes sharper asτcdf↓0\\tau\_\{\\mathrm\{cdf\}\}\\downarrow 0\. Sinceσ\\sigmais monotone increasing andtj,1<⋯<tj,B−1t\_\{j,1\}<\\cdots<t\_\{j,B\-1\}, we haveCj,1​\(𝐱\)≤⋯≤Cj,B−1​\(𝐱\)C\_\{j,1\}\(\\mathbf\{x\}\)\\leq\\cdots\\leq C\_\{j,B\-1\}\(\\mathbf\{x\}\), implyingπj,k​\(𝐱\)≥0\\pi\_\{j,k\}\(\\mathbf\{x\}\)\\geq 0and∑k=1Bπj,k​\(𝐱\)=1\\sum\_\{k=1\}^\{B\}\\pi\_\{j,k\}\(\\mathbf\{x\}\)=1\. Thus,𝝅j​\(𝐱\)\\boldsymbol\{\\pi\}\_\{j\}\(\\mathbf\{x\}\)defines a valid routing distribution over theBBbranches\.

### III\-CAdaptive pruning via branch masks

A fullBB\-ary tree of depthDDgrows exponentially in size, even though many internal nodes may not require allBBoutgoing branches to form an effective partition\. To allow each node to adapt its effective arity, MBNDT equips every internal nodejjwith a learnable, input\-independent branch mask over itsBBoutgoing branches\. Thus,BBserves only as an upper bound on the local branching factor, and unnecessary branches can be suppressed during training\.

Specifically, for nodejj, we introduce sigmoid\-transformed branch masks𝐦j=σ⁡\(𝜼j\)\\mathbf\{m\}\_\{j\}=\\sigma\(\\boldsymbol\{\\eta\}\_\{j\}\),𝜼j∈ℝB\\boldsymbol\{\\eta\}\_\{j\}\\in\\mathbb\{R\}^\{B\}\. Given the unmasked routing probabilitiesπj,b​\(𝐱\)\\pi\_\{j,b\}\(\\mathbf\{x\}\)from Equation \([8](https://arxiv.org/html/2608.29262#S3.E8)\), the normalized masked routing probabilities become

π~j,b\(𝐱\)=𝐦j,b​πj,b​\(𝐱\)∑c=1B𝐦j,c​πj,c​\(𝐱\),b=1,…,B,\\tilde\{\\pi\}\_\{j,b\}\(\\mathbf\{x\}\)=\\frac\{\\mathbf\{m\}\_\{j,b\}\\,\\pi\_\{j,b\}\(\\mathbf\{x\}\)\}\{\\sum\_\{c=1\}^\{B\}\\mathbf\{m\}\_\{j,c\}\\,\\pi\_\{j,c\}\(\\mathbf\{x\}\)\},\\quad b=1,\\dots,B,\(9\)This preserves the partition\-of\-unity constraint while allowing nodejjto globally suppress or emphasize individual branches\. Using the masked probabilities𝝅~j​\(𝐱\)\\tilde\{\\boldsymbol\{\\pi\}\}\_\{j\}\(\\mathbf\{x\}\), we perform hard routing in the forward pass by selecting the branch with highest𝝅~j​\(𝐱\)\\tilde\{\\boldsymbol\{\\pi\}\}\_\{j\}\(\\mathbf\{x\}\)and use a straight\-through estimator:

𝐫j​\(𝐱\)=𝝅~j​\(𝐱\)\+sg⁡\(onehot⁡\(arg⁡maxk​π~j,k​\(𝐱\)\)−𝝅~j​\(𝐱\)\),\\mathbf\{r\}\_\{j\}\(\\mathbf\{x\}\)=\\tilde\{\\boldsymbol\{\\pi\}\}\_\{j\}\(\\mathbf\{x\}\)\+\\mathrm\{sg\}\\\!\\Big\(\\mathrm\{onehot\}\\\!\\big\(\\arg\\max\_\{k\}\\tilde\{\\pi\}\_\{j,k\}\(\\mathbf\{x\}\)\\big\)\-\\tilde\{\\boldsymbol\{\\pi\}\}\_\{j\}\(\\mathbf\{x\}\)\\Big\),\(10\)so that𝐫j​\(𝐱\)\\mathbf\{r\}\_\{j\}\(\\mathbf\{x\}\)is one\-hot in the forward pass, while gradients are propagated through𝝅~j​\(𝐱\)\\tilde\{\\boldsymbol\{\\pi\}\}\_\{j\}\(\\mathbf\{x\}\)during backpropagation\.

#### Pruning effect\.

The branch masks directly induce adaptive pruning under hard routing\. Ifmj,k≈0m\_\{j,k\}\\approx 0, thenπ~j,k​\(𝐱\)≈0\\tilde\{\\pi\}\_\{j,k\}\(\\mathbf\{x\}\)\\approx 0for all𝐱\\mathbf\{x\}, so branchkkis effectively never selected and its subtree becomes unreachable at inference\. More generally, the masks modify pairwise branch preferences through multiplicative biases:

π~j,k​\(𝐱\)π~j,ℓ​\(𝐱\)=𝐦j,k𝐦j,ℓ⋅πj,k​\(𝐱\)πj,ℓ​\(𝐱\),\\frac\{\\tilde\{\\pi\}\_\{j,k\}\(\\mathbf\{x\}\)\}\{\\tilde\{\\pi\}\_\{j,\\ell\}\(\\mathbf\{x\}\)\}=\\frac\{\\mathbf\{m\}\_\{j,k\}\}\{\\mathbf\{m\}\_\{j,\\ell\}\}\\cdot\\frac\{\\pi\_\{j,k\}\(\\mathbf\{x\}\)\}\{\\pi\_\{j,\\ell\}\(\\mathbf\{x\}\)\},\(11\)which implieslog⁡π~j,k​\(𝐱\)π~j,ℓ​\(𝐱\)=log⁡πj,k​\(𝐱\)πj,ℓ​\(𝐱\)\+log⁡𝐦j,k𝐦j,ℓ\\log\\frac\{\\tilde\{\\pi\}\_\{j,k\}\(\\mathbf\{x\}\)\}\{\\tilde\{\\pi\}\_\{j,\\ell\}\(\\mathbf\{x\}\)\}=\\log\\frac\{\\pi\_\{j,k\}\(\\mathbf\{x\}\)\}\{\\pi\_\{j,\\ell\}\(\\mathbf\{x\}\)\}\+\\log\\frac\{\\mathbf\{m\}\_\{j,k\}\}\{\\mathbf\{m\}\_\{j,\\ell\}\}\. Thus, the masks shift hard\-routing preferences by branch\-specific global offsets, allowing weak branches to be suppressed across the input space\. Figure[2](https://arxiv.org/html/2608.29262#S3.F2)contrasts unmasked routing probabilitiesπk​\(x\)\\pi\_\{k\}\(x\)with masked probabilitiesπ~k​\(x\)\\tilde\{\\pi\}\_\{k\}\(x\), showing how masks reweight branch preferences and suppress weak branches while the ordered thresholds remain fixed\.

![Refer to caption](https://arxiv.org/html/2608.29262v1/download.png)Fig\. 2:Illustration of the branch\-mask effect in a toyB=4B\{=\}4routing node\.Top: raw interval probabilities;Bottom: probabilities after applying branch masks and renormalizing\. Dashed lines show the original ordered\-threshold locations; masks reweight branch probabilities and can change which branches are selected under hard routing\.

### III\-DInference

Each leaf nodeℓ\\ellstores a scalar logitθℓ∈ℝ\\theta\_\{\\ell\}\\in\\mathbb\{R\}\. Since only leaf nodes store learned output logits, MBNDT computes predictions by a bottom\-up reduction from the leaves to the root\. Specifically, for each leafℓ\\ell, we initialize the subtree value asvℓ\(D\)​\(𝐱\)=θℓv^\{\(D\)\}\_\{\\ell\}\(\\mathbf\{x\}\)=\\theta\_\{\\ell\}\. Internal nodes do not store output logits; instead, they are assigned recursively computed subtree values\. Letchild⁡\(j,k\)\\mathrm\{child\}\(j,k\)denote thekk\-th child of nodejj\. For depthsd=D−1,…,0d=D\-1,\\dots,0, we compute

vj\(d\)​\(𝐱\)=∑k=1Brj,k​\(𝐱\)​vchild⁡\(j,k\)\(d\+1\)​\(𝐱\)\.v^\{\(d\)\}\_\{j\}\(\\mathbf\{x\}\)=\\sum\_\{k=1\}^\{B\}r\_\{j,k\}\(\\mathbf\{x\}\)\\,v^\{\(d\+1\)\}\_\{\\mathrm\{child\}\(j,k\)\}\(\\mathbf\{x\}\)\.\(12\)The tree logit is then defined asfθ​\(𝐱\)=vroot\(0\)​\(𝐱\)f\_\{\\theta\}\(\\mathbf\{x\}\)=v^\{\(0\)\}\_\{\\mathrm\{root\}\}\(\\mathbf\{x\}\)\. At inference,𝐫j​\(𝐱\)∈\{0,1\}B\\mathbf\{r\}\_\{j\}\(\\mathbf\{x\}\)\\in\\\{0,1\\\}^\{B\}is one\-hot, so exactly one term in the sum is active at each internal node\. Therefore, the recursion simply propagates the logit of the uniquely selected leaf upward to the root\. For binary classification, the predicted probability isσ​\(fθ​\(𝐱\)\)\\sigma\(f\_\{\\theta\}\(\\mathbf\{x\}\)\)\.

## IVStructure Sparsification in MBNDT

MBNDT supports structure sparsification at three levels: intrinsic sparsification through learned branch masks, explicit complexity control through leaf\-budget regularization, and additional compression through post\-hoc train\-path pruning\.

### IV\-AEffective sparsification from branch masks

As introduced in Section[III\-C](https://arxiv.org/html/2608.29262#S3.SS3), the branch masks multiplicatively bias routing probabilities and can suppress weak branches under hard routing\. This reduces the effective arity of internal nodes and may render entire subtrees unreachable\. We therefore distinguish the*nominal tree*from the*effective tree*, obtained by retaining only branches that remain reachable under hard routing\. A leaf is called*routable*if there exists an admissible input𝐱\\mathbf\{x\}whose hard routing decisions reach that leaf\. The effective tree is the subtree induced by all routable leaves\.

### IV\-BLeaf\-budget regularization

To further control tree complexity during training, we introduce an augmented\-Lagrangian style\[[38](https://arxiv.org/html/2608.29262#bib.bib29)\]penalty for exceeding a predefined leaf budget\. Since the exact number of hard\-routable leaves is discrete and non\-differentiable, we instead construct a differentiable surrogate from the masked soft routing probabilities in Equation \([9](https://arxiv.org/html/2608.29262#S3.E9)\)\.

Letπ~j,b​\(𝐱\)\\tilde\{\\pi\}\_\{j,b\}\(\\mathbf\{x\}\)denote the masked routing probability of branchbbat internal nodejj, and letπ~¯j,b=𝔼𝐱​\[π~j,b​\(𝐱\)\]\\bar\{\\tilde\{\\pi\}\}\_\{j,b\}=\\mathbb\{E\}\_\{\\mathbf\{x\}\}\\\!\\left\[\\tilde\{\\pi\}\_\{j,b\}\(\\mathbf\{x\}\)\\right\]denote its average over the current batch\. Motivated by effective\-number measures such as the inverse Simpson index\[[39](https://arxiv.org/html/2608.29262#bib.bib31)\], we define the effective arity of nodejjas

Neffj=1∑b=1Bπ~¯j,b2,\\mathrm\{Neff\}\_\{j\}=\\frac\{1\}\{\\sum\_\{b=1\}^\{B\}\\bar\{\\tilde\{\\pi\}\}\_\{j,b\}^\{\\,2\}\},\(13\)which is close to11when routing concentrates on a single branch and increases towardBBas branch usage becomes more distributed\. We then calculate the soft reach probabilitysj​\(𝐱\)s\_\{j\}\(\\mathbf\{x\}\)of nodejj, defined recursively bysroot​\(𝐱\)=1s\_\{\\mathrm\{root\}\}\(\\mathbf\{x\}\)=1andschild⁡\(j,b\)​\(𝐱\)=sj​\(𝐱\)​π~j,b​\(𝐱\)s\_\{\\mathrm\{child\}\(j,b\)\}\(\\mathbf\{x\}\)=s\_\{j\}\(\\mathbf\{x\}\)\\tilde\{\\pi\}\_\{j,b\}\(\\mathbf\{x\}\)\. Consequently, we useαj=𝔼𝐱​\[sj​\(𝐱\)\]\\alpha\_\{j\}=\\mathbb\{E\}\_\{\\mathbf\{x\}\}\[s\_\{j\}\(\\mathbf\{x\}\)\]and define the soft effective leaf count as

Lsoft=1\+∑jαj​\(Neffj−1\)\.L\_\{\\mathrm\{soft\}\}=1\+\\sum\_\{j\}\\alpha\_\{j\}\\bigl\(\\mathrm\{Neff\}\_\{j\}\-1\\bigr\)\.\(14\)AlthoughLsoftL\_\{\\mathrm\{soft\}\}is not identical to the exact hard\-routable leaf count, it provides a differentiable proxy for effective structural usage during optimization\. Thus, given a target leaf budgetKKwe penalize budget violations using

ℒbudget=μ​\[Lsoft−K\]\+\+ρ2​\[Lsoft−K\]\+2,\\mathcal\{L\}\_\{\\mathrm\{budget\}\}=\\mu\[L\_\{\\mathrm\{soft\}\}\-K\]\_\{\+\}\+\\frac\{\\rho\}\{2\}\[L\_\{\\mathrm\{soft\}\}\-K\]\_\{\+\}^\{2\},\(15\)where\[z\]\+=max⁡\(z,0\)\[z\]\_\{\+\}=\\max\(z,0\)so sparse trees are not penalized\. The update schedule for the coefficientsρ\\rhoandμ\\muis described in the Appendix\.

### IV\-CPost\-hoc train\-path pruning

In addition to training\-time sparsification, MBNDT applies post\-hoc train\-path pruning \(PP\) based on the hard\-routing behavior of the trained model on the training set\. At each internal node, we retain only branches that are observed at least once under hard routing, thereby obtaining a train\-path\-supported subtree\. Inputs that would otherwise select a removed branch are redirected to the nearest surviving branch in the original branch ordering, while the original leaf logits are kept unchanged\. This pruned tree is the final MBNDT model used at inference, rather than a newly retrained compact tree\.

Algorithm 1Training and construction of MBNDT0:Training set

𝒮\\mathcal\{S\}, depth

DD, maximum arity

BB, leaf budget

KK
1:Initialize a full

BB\-ary tree

TθT\_\{\\theta\}
2:foreach training minibatch

ℬ\\mathcal\{B\}do

3:foreach internal node

jjdo

4:Select one feature using the ST feature selector

5:Construct

B−1B\{\-\}1ordered thresholds

6:Compute masked branch probabilities and ST hard routing

7:endfor

8:Route each sample to a leaf and obtain

fθ​\(𝐱\)f\_\{\\theta\}\(\\mathbf\{x\}\)
9:Update

θ\\thetausing

ℒBCE\+ℒbudget\\mathcal\{L\}\_\{\\mathrm\{BCE\}\}\+\\mathcal\{L\}\_\{\\mathrm\{budget\}\}
10:endfor

11:Determine branches used by the training samples

12:Prune unused branches and redirect them to the nearest surviving branch

13:Fix feature and branch choices by their hard decisions

14:returnDeterministic pruned tree

T⋆T^\{\\star\}

## VExperiments

### V\-AExperimental setting

#### Baselines

Our single\-tree baselines are CART\[[8](https://arxiv.org/html/2608.29262#bib.bib2)\]\(greedy binary tree\), SPLIT\[[20](https://arxiv.org/html/2608.29262#bib.bib37)\]\(solver\-based shallow tree\), and GradTree\[[32](https://arxiv.org/html/2608.29262#bib.bib32)\]\(gradient\-trained tree\)\. We restrict the comparison to methods that, like MBNDT, produce a single axis\-aligned tree with one root\-to\-leaf path per input; this excludes soft and ensemble differentiable trees, whose predictions are not single\-path\. We additionally report XGBoost\[[40](https://arxiv.org/html/2608.29262#bib.bib6)\]as a strong black\-box reference, but do not treat it as a comparable single\-tree model, since its predictions aggregate many trees rather than following one decision path\.

Exact and near\-optimal tree learners such as OSDT, MurTree, and GOSDT\[[14](https://arxiv.org/html/2608.29262#bib.bib19),[16](https://arxiv.org/html/2608.29262#bib.bib21),[17](https://arxiv.org/html/2608.29262#bib.bib20)\]are comparable in form\. However, they operate over pre\-binarized feature sets, and their search cost grows steeply with the number of binarized features and training instances, making per\-split hyperparameter tuning under our shared one\-hour budget impractical on the larger benchmarks \(up to∼106\{\\sim\}10^\{6\}instances\)\. We therefore adopt SPLIT\[[20](https://arxiv.org/html/2608.29262#bib.bib37)\]as the representative of this family: its original study reports near\-optimal tree quality while substantially reducing search time relative to exact optimal\-tree solvers, making it the closest scalable proxy for exact and near\-optimal induction across all dataset sizes rather than only on the small datasets where exact solvers remain tractable\.

TABLE I:Per\-dataset test balanced accuracy on 21 binary\-classification datasets\. Values are reported as mean±\\pmstandard deviation over five dataset splits\. All single\-tree methods use a maximum\-depth budget ofD≤4D\\leq 4; XGBoost is included as a depth\-limited ensemble reference\. Ties are assigned the same competition rank and counted as wins when tied for first\.
#### Shared evaluation protocol

To evaluate models in the shallow\-tree regime, we constrain the maximum depth toD≤4D\\leq 4for all single\-tree methods\. The baseline trees are binary by construction, whereas MBNDT tunes its maximum branching factor overB∈\{3,4\}B\\in\\\{3,4\\\}\. For the remaining model\-specific hyperparameters, we follow the corresponding papers where possible; the full search spaces are reported in Tables[VIII](https://arxiv.org/html/2608.29262#A0.T8)–[XII](https://arxiv.org/html/2608.29262#A0.T12)\. We use nested stratified evaluation with fixed seeds and matching outer and inner split indices across methods\.

*Outer split:*for each dataset, five 80/20 train–test splits are generated, and each outer test set remains untouched until final evaluation\.

*Inner HPO:*within each outer\-training pool, Optuna evaluates configurations over five stratified inner folds under a shared wall\-clock budget of up to one hour per method and outer split\. Each configuration is scored byμ−0\.5​σ\\mu\-0\.5\\sigma, whereμ\\muandσ\\sigmaare the mean and standard deviation of inner\-validation balanced accuracy\. Within an inner fold, differentiable methods further reserve 20% of the inner\-training portion for early stopping, yielding a 64/16/20 parameter\-training/early\-stop/ inner\-validation partition; non\-neural methods train on the full 80% inner\-training portion because they do not require early stopping\.

*Method\-specific fitting:*all methods share the splits, depth constraint, and wall\-clock budget, but use their required preprocessing and optimization\. In particular, SPLIT uses quantile\-binarized numeric features, one\-hot encoded categoricals, bounded lookahead, and per\-solve time limits, with exact settings reported in Table[X](https://arxiv.org/html/2608.29262#A0.T10)\.

*Refit and test:*the best configuration is refit on the outer\-training pool \(with an early\-stopping reserve only for differentiable methods\) and evaluated once on the untouched outer test set\. Experiments ran on Ubuntu 24\.04\.2 LTS with an AMD Ryzen Threadripper PRO 5975WX CPU and an NVIDIA RTX 4090 GPU\.

#### MBNDT\-specific training details

For MBNDT, each Optuna trial samples only the architectural and optimizer hyperparameters listed in Table[VIII](https://arxiv.org/html/2608.29262#A0.T8)\. All remaining training choices are fixed across datasets: Adam optimization with separate learning\-rate groups, balanced binary cross\-entropy and deterministic batch\-size selection based on training\-set size\. Because differentiable tree training is sensitive to initialization, MBNDT uses random restarts and checkpoint selection during the final refit\. These choices use only the early\-stopping split drawn from the outer\-training pool; the outer test split is reserved for final reporting\. Appendix[\-A](https://arxiv.org/html/2608.29262#A0.SS1)gives the exact MBNDT random restart policy and leaf\-budget update\.

TABLE II:Structural complexity and training time for depth\-constrained single\-tree methods\. Path length, realized leaves, and time are averaged over five outer splits, with size\-stratified means\.DatasetMBNDT \(ours\)GradTreeSPLITCARTPathLeavesTime \(s\)PathLeavesTime \(s\)PathLeavesTime \(s\)PathLeavesTime \(s\)Small datasets \(n<103n<10^\{3\}\)hepatitis2\.406\.810\.732\.808\.07\.552\.947\.01\.073\.559\.4<0\.01<0\.01colic1\.664\.019\.813\.008\.812\.751\.924\.60\.703\.369\.2<0\.01<0\.01vote2\.295\.623\.272\.406\.87\.171\.102\.40\.742\.859\.0<0\.01<0\.01breast\-w2\.6218\.220\.333\.4011\.27\.672\.075\.20\.943\.3310\.6<0\.01<0\.01blood\-transfusion\-service\-center2\.269\.015\.453\.6012\.816\.832\.466\.42\.262\.487\.4<0\.01<0\.01diabetes2\.5211\.821\.521\.805\.213\.562\.676\.84\.353\.339\.0<0\.01<0\.01Mammographic\-Mass\-Data\-Set2\.527\.626\.563\.4012\.012\.062\.204\.60\.483\.328\.6<0\.01<0\.01Small mean2\.329\.019\.672\.919\.311\.082\.195\.31\.513\.189\.0<0\.01<0\.01Medium datasets \(103≤n<10410^\{3\}\\leq n<10^\{4\}\)credit\-g2\.698\.223\.913\.4012\.08\.482\.686\.40\.523\.129\.8<0\.01<0\.01qsar\-biodeg3\.3618\.621\.863\.8014\.427\.612\.636\.83\.873\.5710\.60\.01banknote\-authentication2\.9937\.427\.634\.0016\.09\.652\.937\.411\.203\.5011\.6<0\.01<0\.01steel\-plates\-fault3\.5111\.225\.343\.8014\.413\.062\.634\.02\.372\.905\.0<0\.01<0\.01kr\-vs\-kp3\.209\.022\.253\.8014\.414\.642\.696\.01\.163\.076\.60\.01spambase3\.6017\.628\.613\.6013\.621\.753\.008\.026\.923\.9513\.20\.02phoneme3\.3838\.841\.662\.406\.829\.272\.877\.612\.983\.9612\.80\.02mushroom3\.288\.643\.544\.0016\.023\.942\.957\.05\.543\.7410\.00\.01Medium mean3\.2518\.729\.353\.6013\.518\.552\.806\.78\.073\.4810\.00\.01Large datasets \(104≤n≤10510^\{4\}\\leq n\\leq 10^\{5\}\)MagicTelescope3\.6077\.075\.553\.8014\.435\.303\.007\.8260\.123\.9814\.80\.09bank\-marketing3\.3019\.0126\.503\.6012\.8115\.272\.996\.45\.833\.3410\.60\.08electricity3\.5149\.0169\.213\.0010\.0180\.123\.008\.0180\.133\.4911\.20\.04adult3\.2622\.8137\.693\.2010\.4145\.782\.714\.418\.363\.9814\.40\.05Large mean3\.4242\.0127\.243\.4011\.9119\.122\.926\.7116\.113\.7012\.80\.07Massive datasetscreditcard2\.7516\.6633\.693\.6012\.8315\.871\.424\.6251\.502\.986\.06\.24SEA\(50000\)3\.3035\.41616\.552\.809\.21502\.112\.466\.28974\.473\.9415\.81\.51Massive mean3\.0326\.01125\.123\.2011\.0908\.991\.945\.44612\.993\.4610\.93\.88Overall mean2\.9520\.6149\.133\.3011\.5120\.022\.546\.1465\.023\.4210\.30\.39

### V\-BDepth\-constrained predictive performance

Table[I](https://arxiv.org/html/2608.29262#S5.T1)reports mean and standard deviation of test balanced accuracy over the five outer splits\. Ranks and boldface are computed over the single\-tree methods only, with ties at three\-decimal precision sharing competition ranks\. Among the depth\-constrained single\-tree learners, MBNDT achieves the best average rank \(1\.481\.48\), the most wins \(1515of2121datasets\), and the highest mean balanced accuracy \(0\.8320\.832, versus0\.8220\.822for CART,0\.7900\.790for SPLIT, and0\.7620\.762for GradTree\)\. A Friedman test rejects equal performance across the four single\-tree methods \(p=2\.37×10−6p=2\.37\\times 10^\{\-6\}\), and pairwise two\-sided Wilcoxon signed\-rank tests with Holm correction confirm that MBNDT improves over each baseline individually \(Table[III](https://arxiv.org/html/2608.29262#S5.T3)\)\. The margin is largest over GradTree and SPLIT and smaller—though still significant—over greedy CART, consistent with CART being the strongest of the depth\-constrained baselines\.

TABLE III:Paired Wilcoxon tests for test balanced accuracy between MBNDT and single\-tree baselines with maximum depthD≤4D\\leq 4\.
### V\-CStructural complexity and trade\-offs

Table[II](https://arxiv.org/html/2608.29262#S5.T2)reports mean test decision\-path length, realized leaves, and final refit time after model selection, averaged over the five outer splits and broken out by dataset size\. We treat path length as a proxy for local decision cost and realized leaves as a proxy for global model size; neither metric alone establishes human interpretability\. These results make the intended trade\-off explicit\. MBNDT keeps per\-instance decision paths short \(mean2\.952\.95, shorter than GradTree and CART, though longer than SPLIT\) while realizing more leaves \(20\.620\.6\) than the binary or solver\-based baselines\. SPLIT produces the smallest trees but the lowest accuracy \(Table[I](https://arxiv.org/html/2608.29262#S5.T1)\)\. MBNDT’s refit time is higher than CART and GradTree but lower than SPLIT on average\.

### V\-DAblation studies of MBNDT

The ablations are diagnostic rather than alternate main benchmarks: the main tables use HPO\-selected MBNDT with the leaf\-budget penalty enabled, whereas Tables[IV](https://arxiv.org/html/2608.29262#S5.T4)–[V](https://arxiv.org/html/2608.29262#S5.T5)fix or remove that budget to isolate branching, masks, and post\-hoc pruning\.

TABLE IV:Branching\-factor ablation for MBNDT with fixed depthD=4D=4and leaf budgetK=16K=16\. Best counts the number of datasets on which each variant achieves the highest balanced accuracy\.#### Effect of branching factor\.

To isolate the contribution of multi\-way splitting, we run a fixed\-hyperparameter branch\-factor ablation withB∈\{2,3,4\}B\\in\\\{2,3,4\\\}\. For each outer split, we reuse the non\-architecture hyperparameters from the original HPO run, including learning rates and routing temperature, and hold the depth and soft leaf budget fixed atD=4D=4andK=16K=16\. This is therefore not a full HPO comparison across branching factors; it tests how local branching capacity affects performance under a common training and budget setting\.

Table[IV](https://arxiv.org/html/2608.29262#S5.T4)reports mean balanced accuracy, mean test path length, realized leaves, and the number of datasets on which each variant is best among the three branching factors\. Moving from binary to ternary routing raises balanced accuracy from0\.8200\.820to0\.8330\.833and slightly shortens mean path length from3\.223\.22to3\.123\.12; in the direct pairwise comparison, B3 outperforms B2 on 16 of 21 datasets\. This suggests that the improvement is not driven by longer decision paths, but by greater local partitioning capacity at fixed depth\. The cost is increased realized leaves \(9\.59\.5to16\.016\.0\)\. Increasing toB=4B=4gives similar accuracy \(0\.8320\.832\) and the shortest paths \(3\.093\.09\), but realizes more leaves \(23\.323\.3\), indicating diminishing returns beyond ternary splits under this budget\.

TABLE V:Effect of branch masks and post\-hoc train\-path pruning \(PP\) in a controlled setting without a leaf\-budget penalty\.
#### Effect of branch masks\.

Table[V](https://arxiv.org/html/2608.29262#S5.T5)separates branch masks from post\-hoc train\-path pruning \(PP\) in a controlled no\-budget setting with fixed hyperparameters; the leaf\-budget penalty is disabled because its surrogate depends on the masked routing probabilities \(Eq\.[14](https://arxiv.org/html/2608.29262#S4.E14)\) and would confound the mask\-versus\-PP attribution\. PP is the main compression mechanism: without masks, it reduces realized leaves from60\.760\.7to22\.922\.9on average \(p<10−4p<10^\{\-4\}\) and shortens mean paths from3\.323\.32to3\.063\.06, while balanced accuracy is statistically unchanged \(0\.8230\.823,p=0\.13p=0\.13\)\. Branch masks play a different role\. Without PP, masks reduce leaves only modestly \(60\.760\.7to56\.456\.4\) but raise balanced accuracy from0\.8230\.823to0\.8270\.827\(paired Wilcoxonp=0\.010p=0\.010\)\. After applying the same PP step, the masked model remains more accurate \(0\.8270\.827vs\.0\.8230\.823;p=0\.025p=0\.025, 15/21 datasets\) and slightly smaller \(20\.720\.7vs\.22\.922\.9leaves\)\.

Thus, masks are not simply a substitute for post\-hoc pruning: PP supplies most of the size reduction, whereas masks shape the routing during training and yield a small but consistent accuracy gain that survives pruning\. Overall, their combination gives the most favorable accuracy–size trade\-off, reducing realized leaves by about two\-thirds relative to the unmasked, unpruned tree\.

#### Effect of the leaf\-budget constraint\.

Table[VI](https://arxiv.org/html/2608.29262#S5.T6)reports a fixed\-budget sweep against single\-tree reference points; MBNDT values are averaged over datasets after post\-hoc pruning\. The leaf budget behaves as an effective accuracy–size control knob: increasing it fromK=4K=4toK=64K=64raises mean balanced accuracy from0\.8270\.827to0\.8360\.836while lengthening mean paths from2\.672\.67to3\.013\.01and realized leaves from13\.413\.4to20\.320\.3\.

TABLE VI:Leaf\-budget ablation for MBNDT with single\-tree reference points\. MBNDT rows vary the target leaf budgetKK; the HPO\-selected row reports the mean selectedKKacross dataset splits\. Reference methods are shown under the same depth constraint\.The effect saturates and is not strictly monotonic across datasets—K=16K=16already attains0\.8350\.835accuracy at path2\.942\.94and18\.318\.3leaves, close toK=64K=64\. The penalty is moreover regularizing rather than merely shrinking: on the dataset\-averaged means,K=16K=16dominates the no\-budget variant on all three axes \(higher accuracy, shorter paths, and fewer leaves than0\.8270\.827/3\.043\.04/20\.720\.7\)\. Relative to the single\-tree baselines, MBNDT occupies a higher\-accuracy, larger\-tree regime: SPLIT, CART, and GradTree remain more compact \(6\.16\.1,10\.310\.3, and11\.511\.5leaves\) but less accurate \(0\.7900\.790,0\.8220\.822, and0\.7620\.762\), and even the tightest budget \(K=4K=4,13\.413\.4leaves\) does not bring MBNDT below them in size\. The budget should therefore be read as a knob for trading realized size against accuracy within MBNDT, not as a mechanism that makes it globally smaller than binary or solver\-based trees\. \(The no\-budget row matches the setting of the mask/PP ablation in Table[V](https://arxiv.org/html/2608.29262#S5.T5); the HPO\-selected row corresponds to the main benchmark protocol\.\)

## VIConclusion

We studied shallow\-depth tree induction, where the complexity budget is a maximum decision depth, and proposed MBNDT, a single axis\-aligned tree that learns differentiable multi\-way splits end\-to\-end\. By increasing local partitioning capacity instead of tree depth, MBNDT achieves the highest mean balanced accuracy among depth\-constrained single\-tree learners across 21 OpenML binary\-classification benchmarks\. A branching\-factor ablation shows that multi\-way splits outperform a binary variant under the same depth and budget setting, supporting the role of multi\-way branching\. The improvement has an explicit cost—MBNDT realizes more leaves than the other single\-tree baselines—so it is best suited to settings that prioritize accuracy under bounded per\-instance decision depth over minimal global tree size\.

Several directions remain for future work\. First, MBNDT is evaluated here for binary classification; extending the differentiable multi\-way formulation to multiclass classification and regression is a natural next step\. In addition, future work should complement path length and realized leaf count with distinct\-feature and threshold/rule\-complexity measures and human\-centered studies\.

## Acknowledgment

This work was supported by the Institute of Information & Communications Technology Planning & Evaluation \(IITP\) under the Artificial Intelligence Convergence Innovation Human Resources Development \(IITP\-2026\-RS\-2023\-00254177\), the Institute of Information & Communications Technology Planning & Evaluation \(IITP\) under the Leading Generative AI Human Resources Development \(IITP\-2026\-RS\-2024\-00360227\), and the National Research Foundation of Korea \(NRF\) grant funded by the Korean government \(MSIT\) \(RS\-2026\-25555206\)\.

## References

- \[1\]\(2014\)Fifty years of classification and regression trees\.International Statistical Review82\(3\),pp\. 329–348\.Cited by:[§I](https://arxiv.org/html/2608.29262#S1.p1.1)\.
- \[2\]J\. Huysmans, K\. Dejaeger, C\. Mues, J\. Vanthienen, and B\. Baesens\(2011\)An empirical evaluation of the comprehensibility of decision table, tree and rule based predictive models\.Decision Support Systems51\(1\),pp\. 141–154\.External Links:[Document](https://dx.doi.org/10.1016/j.dss.2010.12.003)Cited by:[§I](https://arxiv.org/html/2608.29262#S1.p1.1),[§I](https://arxiv.org/html/2608.29262#S1.p2.1)\.
- \[3\]C\. Rudin\(2019\)Stop explaining black box machine learning models for high stakes decisions and use interpretable models instead\.Nature Machine Intelligence1\(5\),pp\. 206–215\.External Links:[Document](https://dx.doi.org/10.1038/s42256-019-0048-x)Cited by:[§I](https://arxiv.org/html/2608.29262#S1.p1.1),[§I](https://arxiv.org/html/2608.29262#S1.p2.1)\.
- \[4\]A\. A\. Freitas\(2014\)Comprehensible classification models: a position paper\.ACM SIGKDD explorations newsletter15\(1\),pp\. 1–10\.Cited by:[§I](https://arxiv.org/html/2608.29262#S1.p2.1)\.
- \[5\]R\. Piltaver, M\. Luštrek, M\. Gams, and S\. Martinčić\-Ipšić\(2016\)What makes classification trees comprehensible?\.Expert Systems with Applications62,pp\. 333–346\.Cited by:[§I](https://arxiv.org/html/2608.29262#S1.p2.1)\.
- \[6\]V\. F\. Souza, F\. Cicalese, E\. Laber, and M\. Molinaro\(2022\)Decision trees with short explainable rules\.InAdvances in Neural Information Processing Systems,S\. Koyejo, S\. Mohamed, A\. Agarwal, D\. Belgrave, K\. Cho, and A\. Oh \(Eds\.\),Vol\.35,pp\. 12365–12379\.Cited by:[§I](https://arxiv.org/html/2608.29262#S1.p2.1),[§II\-A](https://arxiv.org/html/2608.29262#S2.SS1.p1.1)\.
- \[7\]B\. Letham, C\. Rudin, T\. H\. McCormick, and D\. Madigan\(2015\)Interpretable classifiers using rules and Bayesian analysis: Building a better stroke prediction model\.The Annals of Applied Statistics9\(3\),pp\. 1350–1371\.Cited by:[§I](https://arxiv.org/html/2608.29262#S1.p2.1)\.
- \[8\]L\. Breiman, J\. H\. Friedman, R\. A\. Olshen, and C\. J\. Stone\(1984\)Classification and regression trees\.Wadsworth International Group\.Cited by:[§I](https://arxiv.org/html/2608.29262#S1.p3.1),[§II\-B](https://arxiv.org/html/2608.29262#S2.SS2.p1.1),[§V\-A](https://arxiv.org/html/2608.29262#S5.SS1.SSS0.Px1.p1.1)\.
- \[9\]J\. R\. Quinlan\(1993\)C4\.5: programs for machine learning\.Morgan Kaufmann\.Cited by:[§I](https://arxiv.org/html/2608.29262#S1.p3.1),[§II\-B](https://arxiv.org/html/2608.29262#S2.SS2.p1.1),[§II\-D](https://arxiv.org/html/2608.29262#S2.SS4.p1.1)\.
- \[10\]D\. Bertsimas and J\. Dunn\(2017\)Optimal classification trees\.Machine Learning106\(7\),pp\. 1039–1082\.Cited by:[§I](https://arxiv.org/html/2608.29262#S1.p3.1),[§II\-B](https://arxiv.org/html/2608.29262#S2.SS2.p1.1),[§II\-C](https://arxiv.org/html/2608.29262#S2.SS3.p1.1)\.
- \[11\]Y\. Izza, A\. Ignatiev, and J\. Marques\-Silva\(2022\)On tackling explanation redundancy in decision trees\.Journal of Artificial Intelligence Research75,pp\. 261–321\.External Links:[Document](https://dx.doi.org/10.1613/jair.1.13575)Cited by:[§II\-A](https://arxiv.org/html/2608.29262#S2.SS1.p1.1)\.
- \[12\]H\. McTavish, Z\. Boner, J\. Donnelly, M\. Seltzer, and C\. Rudin\(2025\)Leveraging predictive equivalence in decision trees\.InProceedings of the 42nd International Conference on Machine Learning,Proceedings of Machine Learning Research, Vol\.267,pp\. 43440–43475\.Cited by:[§II\-A](https://arxiv.org/html/2608.29262#S2.SS1.p1.1)\.
- \[13\]J\. R\. Quinlan\(1986\)Induction of decision trees\.Machine Learning1\(1\),pp\. 81–106\.External Links:[Document](https://dx.doi.org/10.1023/A%3A1022643204877)Cited by:[§II\-B](https://arxiv.org/html/2608.29262#S2.SS2.p1.1),[§II\-D](https://arxiv.org/html/2608.29262#S2.SS4.p1.1)\.
- \[14\]X\. Hu, C\. Rudin, and M\. Seltzer\(2019\)Optimal sparse decision trees\.InAdvances in Neural Information Processing Systems,Vol\.32\.Cited by:[§II\-C](https://arxiv.org/html/2608.29262#S2.SS3.p1.1),[§V\-A](https://arxiv.org/html/2608.29262#S5.SS1.SSS0.Px1.p2.1)\.
- \[15\]G\. Aglin, S\. Nijssen, and P\. Schaus\(2021\)PyDL8\.5: a library for learning optimal decision trees\.InProceedings of the International Joint Conference on Artificial Intelligence \(IJCAI\),pp\. 5222–5224\.Cited by:[§II\-C](https://arxiv.org/html/2608.29262#S2.SS3.p1.1)\.
- \[16\]E\. Demirović, A\. Lukina, E\. Hebrard, J\. Chan, J\. Bailey, C\. Leckie, K\. Ramamohanarao, and P\. J\. Stuckey\(2022\)MurTree: optimal decision trees via dynamic programming and search\.Journal of Machine Learning Research23\(26\),pp\. 1–47\.Cited by:[§II\-C](https://arxiv.org/html/2608.29262#S2.SS3.p1.1),[§V\-A](https://arxiv.org/html/2608.29262#S5.SS1.SSS0.Px1.p2.1)\.
- \[17\]J\. Lin, C\. Zhong, D\. Hu, C\. Rudin, and M\. Seltzer\(2020\)Generalized and scalable optimal sparse decision trees\.InProceedings of the 37th International Conference on Machine Learning,pp\. 6150–6160\.Cited by:[§II\-C](https://arxiv.org/html/2608.29262#S2.SS3.p1.1),[§V\-A](https://arxiv.org/html/2608.29262#S5.SS1.SSS0.Px1.p2.1)\.
- \[18\]J\. G\. M\. van der Linden, M\. M\. de Weerdt, and E\. Demirović\(2023\)Necessary and sufficient conditions for optimal decision trees using dynamic programming\.InAdvances in Neural Information Processing Systems,Vol\.36\.External Links:[Document](https://dx.doi.org/10.52202/075280-0404)Cited by:[§II\-C](https://arxiv.org/html/2608.29262#S2.SS3.p1.1)\.
- \[19\]C\. E\. Briţa, J\. G\. M\. van der Linden, and E\. Demirović\(2025\)Optimal classification trees for continuous feature data using dynamic programming with branch\-and\-bound\.Proceedings of the AAAI Conference on Artificial Intelligence39\(11\),pp\. 11131–11139\.External Links:[Document](https://dx.doi.org/10.1609/aaai.v39i11.33210)Cited by:[§II\-C](https://arxiv.org/html/2608.29262#S2.SS3.p1.1)\.
- \[20\]V\. Babbar, H\. Mctavish, C\. Rudin, and M\. Seltzer\(2025\)Near\-optimal decision trees in a SPLIT second\.InProceedings of the 42nd International Conference on Machine Learning,Proceedings of Machine Learning Research, Vol\.267,pp\. 2114–2175\.Cited by:[§II\-C](https://arxiv.org/html/2608.29262#S2.SS3.p1.1),[§V\-A](https://arxiv.org/html/2608.29262#S5.SS1.SSS0.Px1.p1.1),[§V\-A](https://arxiv.org/html/2608.29262#S5.SS1.SSS0.Px1.p2.1)\.
- \[21\]G\. V\. Kass\(1980\)An exploratory technique for investigating large quantities of categorical data\.Journal of the Royal Statistical Society: Series C \(Applied Statistics\)29\(2\),pp\. 119–127\.Cited by:[§II\-D](https://arxiv.org/html/2608.29262#S2.SS4.p1.1)\.
- \[22\]T\. Fulton, S\. Kasif, and S\. Salzberg\(1995\)Efficient algorithms for finding multi\-way splits for decision trees\.InProceedings of the 12th International Conference on Machine Learning,pp\. 244–251\.Cited by:[§II\-D](https://arxiv.org/html/2608.29262#S2.SS4.p1.1)\.
- \[23\]F\. Berzal, J\. Cubero, N\. Marín, and D\. Sánchez\(2004\)Building multi\-way decision trees with numerical attributes\.Information Sciences165\(1–2\),pp\. 73–90\.External Links:[Document](https://dx.doi.org/10.1016/j.ins.2003.09.018)Cited by:[§II\-D](https://arxiv.org/html/2608.29262#S2.SS4.p1.1)\.
- \[24\]S\. Subramanian and W\. Sun\(2023\)Scalable optimal multiway\-split decision trees with constraints\.Proceedings of the AAAI Conference on Artificial Intelligence37\(8\),pp\. 9891–9899\.External Links:[Document](https://dx.doi.org/10.1609/aaai.v37i8.26180)Cited by:[§II\-D](https://arxiv.org/html/2608.29262#S2.SS4.p1.1)\.
- \[25\]J\. Good, T\. Kovach, K\. Miller, and A\. Dubrawski\(2023\)Feature learning for interpretable, performant decision trees\.InAdvances in Neural Information Processing Systems,Vol\.36\.External Links:[Document](https://dx.doi.org/10.52202/075280-2907)Cited by:[§II\-E](https://arxiv.org/html/2608.29262#S2.SS5.p1.1)\.
- \[26\]M\. Norouzi, M\. Collins, M\. A\. Johnson, D\. J\. Fleet, and P\. Kohli\(2015\)Efficient non\-greedy optimization of decision trees\.InAdvances in Neural Information Processing Systems,Vol\.28\.Cited by:[§II\-E](https://arxiv.org/html/2608.29262#S2.SS5.p1.1)\.
- \[27\]S\. P\. Panda, B\. Genest, A\. Easwaran, and P\. N\. Suganthan\(2024\)Vanilla gradient descent for oblique decision trees\.InECAI 2024 – 27th European Conference on Artificial Intelligence,Cited by:[§II\-E](https://arxiv.org/html/2608.29262#S2.SS5.p1.1)\.
- \[28\]P\. Kontschieder, M\. Fiterau, A\. Criminisi, and S\. Rota Bulò\(2015\)Deep neural decision forests\.InProceedings of the IEEE International Conference on Computer Vision \(ICCV\),pp\. 1467–1475\.External Links:[Document](https://dx.doi.org/10.1109/ICCV.2015.172)Cited by:[§II\-E](https://arxiv.org/html/2608.29262#S2.SS5.p1.1)\.
- \[29\]T\. M\. Hehn, J\. F\. P\. Kooij, and F\. A\. Hamprecht\(2020\)End\-to\-end learning of decision trees and forests\.International Journal of Computer Vision128\(4\),pp\. 997–1011\.External Links:[Document](https://dx.doi.org/10.1007/s11263-019-01237-6)Cited by:[§II\-E](https://arxiv.org/html/2608.29262#S2.SS5.p1.1)\.
- \[30\]Y\. Yang, I\. Garcia Morillo, and T\. M\. Hospedales\(2018\)Deep neural decision trees\.Note:arXiv preprint arXiv:1806\.06988Cited by:[§II\-E](https://arxiv.org/html/2608.29262#S2.SS5.p1.1)\.
- \[31\]X\. Zhou, M\. Zhang, Z\. Lee, W\. Ye, and S\. Zhang\(2025\)HaDeMiF: hallucination detection and mitigation in large language models\.InThe Thirteenth International Conference on Learning Representations,External Links:[Link](https://openreview.net/forum?id=VwOYxPScxB)Cited by:[§II\-E](https://arxiv.org/html/2608.29262#S2.SS5.p1.1)\.
- \[32\]S\. Marton, S\. Lüdtke, C\. Bartelt, and H\. Stuckenschmidt\(2024\)GradTree: learning axis\-aligned decision trees with gradient descent\.InProceedings of the AAAI Conference on Artificial Intelligence,Vol\.38,pp\. 14323–14331\.Cited by:[§II\-E](https://arxiv.org/html/2608.29262#S2.SS5.p1.1),[§III\-A](https://arxiv.org/html/2608.29262#S3.SS1.p1.2),[§III\-A](https://arxiv.org/html/2608.29262#S3.SS1.p2.1),[§V\-A](https://arxiv.org/html/2608.29262#S5.SS1.SSS0.Px1.p1.1)\.
- \[33\]S\. Marton, S\. Lüdtke, C\. Bartelt, and H\. Stuckenschmidt\(2024\)GRANDE: gradient\-based decision tree ensembles for tabular data\.InInternational Conference on Learning Representations,Vol\.2024,pp\. 3811–3837\.Cited by:[§II\-E](https://arxiv.org/html/2608.29262#S2.SS5.p1.1)\.
- \[34\]B\. Peters, V\. Niculae, and A\. F\. T\. Martins\(2019\)Sparse sequence\-to\-sequence models\.InProceedings of the 57th Annual Meeting of the Association for Computational Linguistics,External Links:[Document](https://dx.doi.org/10.18653/v1/P19-1146)Cited by:[§III\-A](https://arxiv.org/html/2608.29262#S3.SS1.p1.1)\.
- \[35\]S\. Popov, S\. Morozov, and A\. Babenko\(2019\)Neural oblivious decision ensembles for deep learning on tabular data\.Note:arXiv preprint arXiv:1909\.06312Cited by:[§III\-A](https://arxiv.org/html/2608.29262#S3.SS1.p1.2)\.
- \[36\]Y\. Bengio, N\. Léonard, and A\. Courville\(2013\)Estimating or propagating gradients through stochastic neurons for conditional computation\.Note:arXiv preprint arXiv:1308\.3432Cited by:[§III\-A](https://arxiv.org/html/2608.29262#S3.SS1.p2.1)\.
- \[37\]A\. Karthikeyan, N\. Jain, N\. Natarajan, and P\. Jain\(2022\)Learning accurate decision trees with bandit feedback via quantized gradient descent\.Transactions on Machine Learning Research\.Cited by:[§III\-A](https://arxiv.org/html/2608.29262#S3.SS1.p2.1)\.
- \[38\]E\. Burman, P\. Hansbo, and M\. G\. Larson\(2023\)The augmented lagrangian method as a framework for stabilised methods in computational mechanics\.Archives of Computational Methods in Engineering30\(4\),pp\. 2579–2604\.Cited by:[§IV\-B](https://arxiv.org/html/2608.29262#S4.SS2.p1.1)\.
- \[39\]A\. Chao, N\. J\. Gotelli, T\. C\. Hsieh, E\. L\. Sander, K\. H\. Ma, R\. K\. Colwell, and A\. M\. Ellison\(2014\)Rarefaction and extrapolation with hill numbers: a framework for sampling and estimation in species diversity studies\.Ecological Monographs84\(1\),pp\. 45–67\.Cited by:[§IV\-B](https://arxiv.org/html/2608.29262#S4.SS2.p2.1)\.
- \[40\]T\. Chen and C\. Guestrin\(2016\)XGBoost: a scalable tree boosting system\.InProceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining,pp\. 785–794\.External Links:[Document](https://dx.doi.org/10.1145/2939672.2939785)Cited by:[§V\-A](https://arxiv.org/html/2608.29262#S5.SS1.SSS0.Px1.p1.1)\.

### \-AMBNDT training details

#### Restart policy

Final refits use five random restarts\. Stage 1 is run from each initialization and the restart checkpoint is selected using validation loss; stage 2 resumes from the selected checkpoint and early\-stops on validation balanced accuracy\. During the final refit, the corresponding limits are40/840/8and500/25500/25epochs/patience\.

#### Leaf\-budget update

For the selected leaf budgetKK, the experiments use the log\-scale violationv=\[log⁡Lsoft−log⁡K\]\+v=\[\\log L\_\{\\mathrm\{soft\}\}\-\\log K\]\_\{\+\}\. The minibatch objective addsμ​v\+\(ρ/2\)​v2\\mu v\+\(\\rho/2\)v^\{2\}with fixedρ=0\.03\\rho=0\.03and initial multiplierμ0=0\\mu\_\{0\}=0\. After each epoch, the mean training\-set violation is accumulated with an exponential moving average using coefficient0\.50\.5, and the multiplier is updated by projected dual ascent,μ←max⁡\{0,μ\+0\.003​v^\}\\mu\\leftarrow\\max\\\{0,\\mu\+0\.003\\,\\widehat\{v\}\\\}, wherev^\\widehat\{v\}denotes the smoothed violation\.

### \-BDataset characteristics and preprocessing detail

TABLE VII:Dataset characteristics for the 21 OpenML binary\-classification benchmarks\.Table[VII](https://arxiv.org/html/2608.29262#A0.T7)summarizes the publicly available OpenML datasets used in the main experiments\. All preprocessing is leakage\-safe, fitted on training data only\. Numeric features are median\-imputed\. Categorical features with more than ten unique values use leave\-one\-out \(LOO\) target encoding, and lower\-cardinality features use one\-hot encoding; LOO transforms use labels only when fitting on training data, while validation and test transforms are label\-free, with unseen or missing categories mapped to a default “unknown” value\. For the differentiable methods \(MBNDT, GradTree\), heavy\-tailed numeric features optionally receive alog⁡\(1\+x\)\\log\(1\+x\)transform followed by rank–Gaussian normalization to𝒩⁡\(0,1\)\\mathcal\{N\}\(0,1\), which stabilizes differentiable threshold learning; these transforms are method\-appropriate and do not alter the split structure available to the other tree baselines\. For SPLIT, numeric features are quantile\-discretized and categoricals one\-hot encoded, as required by its binarized solver\.

### \-CModel\-wise hyperparameter search space

TABLE VIII:MBNDT hyperparameter search space\.TABLE IX:GradTree hyperparameter search space\.TABLE X:SPLIT hyperparameter search space\.TABLE XI:CART hyperparameter search space\.TABLE XII:XGBoost hyperparameter search space\.