Application of machine learning to monster level prediction in tabletop RPG game design

# Application of machine learning to monster level prediction in tabletop RPG game design
Source: [https://arxiv.org/html/2607.09196](https://arxiv.org/html/2607.09196)
Jakub Adamczyk222Both authors contributed equally to this work\.[jadamczy@agh\.edu\.pl](https://arxiv.org/html/2607.09196v1/mailto:[email protected])

###### Abstract

Designing balanced adversaries is a central but labor\-intensive task in tabletop role\-playing game \(TTRPG\) development\. In systems such as Pathfinder, each monster is described by many numerical attributes that jointly determine its power, summarized as an ordinal level\. We investigate whether machine learning can support designers by predicting this level from a monster’s attributes, framing the task as tabular ordinal regression\. We introduce what is, to our knowledge, the first dataset built specifically for TTRPG monster\-level prediction, derived from publicly available Pathfinder Second Edition data\. Using it, we compare classical regression models with rounding schemes, dedicated tabular ordinal regression algorithms, and neural networks with ordinal\-aware losses\. To mirror real design workflows, we evaluate all models under chronological and expanding\-window protocols with several complementary metrics\. Results show that tree\-based ensembles outperform linear models and neural approaches, achieving near\-perfect ordinal ranking and high predictive accuracy\. Explainable AI analyses, such as feature importance and error distributions, show that the model is aligned with human intuition and follows patterns grounded in game rules\. Together, these results show that machine learning can reliably approximate designer judgments and serve as an effective computer\-aided tool for monster balancing and broader TTRPG system design\.

###### keywords:

machine learning , ordinal regression , computer\-aided design , game design

###### MSC:

62\-04 , 62J02 , 62P25 , 68\-04 , 68T05 , 68T35 , 68U07

††journal:Expert Systems with Applications\\affiliation

\[agh\]organization=Faculty of Computer Science, AGH University of Krakow, city=Cracow, country=Poland

label1label1footnotetext:ORCID 0009\-0004\-1889\-5655label2label2footnotetext:ORCID 0000\-0003\-4336\-4288## 1Introduction

Artificial intelligence \(AI\) and machine learning \(ML\) methods have been used to play games since the inception of these fields, with breakthroughs such as alpha\-beta pruning\[[42](https://arxiv.org/html/2607.09196#bib.bib1)\], Monte Carlo Tree Search \(MCTS\)\[[14](https://arxiv.org/html/2607.09196#bib.bib3)\], AlphaGo\[[47](https://arxiv.org/html/2607.09196#bib.bib4)\], and AlphaZero\[[48](https://arxiv.org/html/2607.09196#bib.bib5)\]\. These applications are player\-centered and assume an existing game environment together with its rules\. In other words, they presuppose that the game itself has already been created and designed, which allows an abstract mathematical representation of its rules and states\. In this work, we instead focus on using machine learning forgame design, an area that has attracted comparatively less interest yet is of high importance to the industry, as many steps of game creation are slow, labor\-intensive, and expensive\.

Specifically, we study role\-playing games \(RPGs\), and in particular pen & paper RPGs, also known astabletop RPGs \(TTRPGs\)\. These are distinct from, and predate, computer games and computer RPGs \(cRPGs\)\. In this type of game, participants take on the roles of fictional characters within a story narrated by a Game Master \(GM\)\. Players gather around a table, physically or virtually, to create and develop characters within an evolving story\. Gameplay is driven by narration and by the resolution of conflicts through a chosenrule system\(or simply “system”\), typically using multi\-sided dice rolls to introduce randomness\. The game design task considered here centers purely on the rules of the game itself, rather than on generating story, levels, or music, which are left to the GM and human creativity\. TTRPGs are also frequently adopted as rulesets for computer games, with famous examples such as the Baldur’s Gate series using Dungeons & Dragons \(D&D\) rules, and Pathfinder: Wrath of the Righteous using the Pathfinder First Edition system\.

The RPG market has been growing steadily for a long time\. In 2023, it was valued at approximately1\.921\.92billion USD, with projections estimating that it will reach5\.275\.27billion USD by 2033\[[9](https://arxiv.org/html/2607.09196#bib.bib6)\]\. This rapid expansion highlights the increasing popularity of TTRPGs and the evolving demands of players\. TTRPGs offer advantages over computer RPGs, which have a fixed narrative and leave players limited freedom, compared with the virtually infinite possibilities of TTRPGs\. However, to remain competitive, publishers, miniature manufacturers, and other industry players must continuously innovate\. Beyond creative marketing strategies and engaging gameplay demonstrations, companies place a strong emphasis on R&D and technological advancement to improve the gaming experience\. There is growing interest in applying AI and ML methods in this area, and some aspects are particularly well suited for them\.

One of the most important components of adventures is combat with adversaries, commonly known as enemies ormonsters, introduced by the GM, such as goblins or dragons\. Like player characters, monsters are represented in the rule system by numerous numericalattributesthat define their strength, abilities, and difficulty to defeat\. Examples include strength, the amount of damage dealt by each attack, and armor\. In many systems, the overall power of a monster is indicated by a value called itslevel, typically a non\-negative integer, with higher levels indicating more powerful monsters\. The level is directly and causally derived from the attribute values\. Selecting monsters of an appropriate level is essential for preparing encounters that are challenging yet winnable for players\.

An important task for an RPG publisher is creating whole books of example monsters that a GM can use in their own narrative\. This requires properlyestimating the level of each monster from its statistics\. However, in many systems, such as D&D or Pathfinder, monsters are described by dozens of attributes, which makes this process hard for human designers\. In practice, such estimation is done either through expert human judgment, which is error\-prone, or through many hours of gameplay, which is labor\-intensive and typically requires 4\-5 people \(a GM and players\) at a time\.

Selling books of monsters constitutes an important source of income for these companies, and a single compendium can contain over 100 monsters\. This creates a business need to quickly and automatically estimate a monster’s level from its numerical statistics, obtaining early validation of potential designs\. ML\-based solutions lend themselves naturally to this problem, as they can readily incorporate high\-dimensional numerical inputs and non\-linear relationships between attributes and level\. Such tools can help game designers accelerate the process by providing feedback on monster power during creation\. Furthermore, GMs can use these algorithms to design unique monsters, enhancing their own gameplay experience\.

Key contributionsof this work include:

- 1\.Creation of what is, to our knowledge, the first dataset for TTRPG game design, specifically for monster level estimation in the Pathfinder Second Edition game system\. It is publicly available under a permissive open source license, to foster further research in this area\.
- 2\.Formalization of monster level prediction as a tabular ordinal regression problem\. We provide feature engineering examples and perform a comprehensive benchmark of 16 models for this task, including regression with rounding, dedicated tabular ordinal regression algorithms, and neural networks with ordinal loss functions\.
- 3\.Domain\-specific evaluation schemes for a realistic quantification of model generalization, including a time\-based expanding\-window train\-test split and metrics such as macro\-averaged MAE for imbalanced regression\.
- 4\.Application of data analysis and explainable AI \(XAI\), demonstrating notable robustness of the models, alignment with human intuition, and real\-world applicability\.

## 2Literature review

AI and ML have been widely applied in games, but primarily for playing them, particularly through search techniques and reinforcement learning\. The area addressed in this work, game design, has attracted much less research interest\.

Much of the existing literature is concerned more with humanities aspects than with computer science, for example storytelling\[[16](https://arxiv.org/html/2607.09196#bib.bib15)\]or the integration of technology to improve traditional gameplay experiences\[[53](https://arxiv.org/html/2607.09196#bib.bib26)\]\. Applications of generative ML have been explored in procedural world design and creative content generation, such as map generation\[[44](https://arxiv.org/html/2607.09196#bib.bib16),[19](https://arxiv.org/html/2607.09196#bib.bib17),[56](https://arxiv.org/html/2607.09196#bib.bib18),[29](https://arxiv.org/html/2607.09196#bib.bib19),[50](https://arxiv.org/html/2607.09196#bib.bib20)\], music generation\[[37](https://arxiv.org/html/2607.09196#bib.bib21),[38](https://arxiv.org/html/2607.09196#bib.bib22)\], and non\-player character \(NPC\) text and conversation generation\[[43](https://arxiv.org/html/2607.09196#bib.bib23),[54](https://arxiv.org/html/2607.09196#bib.bib24),[4](https://arxiv.org/html/2607.09196#bib.bib25)\]\.

Predictive AI and ML methods, in particular for gameplay balance, have been much less explored\. The closest work to ours is adaptive gameplay balancing\[[21](https://arxiv.org/html/2607.09196#bib.bib8)\], which dynamically adjusts the game world, such as enemy statistics, based on player behavior\. However, these techniques adaptively control player adversaries, which means that they apply only to computer games, operate on an already existing game system rather than being part of game design, and require deep integration into the software\. As such, they fall outside the scope of this work, which is concerned with predicting monster level as part of game design, before gameplay begins\.

Several reviews of AI and ML applications in games have been published in recent years\[[17](https://arxiv.org/html/2607.09196#bib.bib9),[51](https://arxiv.org/html/2607.09196#bib.bib10),[23](https://arxiv.org/html/2607.09196#bib.bib11),[20](https://arxiv.org/html/2607.09196#bib.bib12),[57](https://arxiv.org/html/2607.09196#bib.bib13),[1](https://arxiv.org/html/2607.09196#bib.bib14)\]\. However, they focus heavily on the AI\-based content generation and ML\-based interactive gameplay described above, and none even mention the possibility of using predictive ML for enemy power prediction or statistic balancing\. To our knowledge, this work is therefore the first to propose such an application\.

The monster level in RPG games is typically an integer starting from 0 or 1\. Consequently, when predictive ML methods are applied, the dependent variable is both discrete and ordered, which constitutes anordinal regressionproblem, also known as ordinal classification\[[55](https://arxiv.org/html/2607.09196#bib.bib29),[8](https://arxiv.org/html/2607.09196#bib.bib30),[52](https://arxiv.org/html/2607.09196#bib.bib33)\]\. Various algorithms have been proposed for this problem, which arises in many ML applications, for example recommender systems \(user ratings such as 1\-5 stars\)\[[22](https://arxiv.org/html/2607.09196#bib.bib32)\], age estimation \(predicting the age of a subject in an image\)\[[33](https://arxiv.org/html/2607.09196#bib.bib50)\], and strength of opinion in the social sciences and psychology \(a value on a Likert scale\)\[[8](https://arxiv.org/html/2607.09196#bib.bib30)\]\. We could not find any prior application of ordinal regression to games or game design; this too constitutes a novel contribution of our work\.

One group of methods is based on standard regression, with the results rounded as a post\-processing step to ensure discrete outputs\[[26](https://arxiv.org/html/2607.09196#bib.bib31),[46](https://arxiv.org/html/2607.09196#bib.bib39)\]\. Various rounding schemes can be used in place of classical mathematical rounding to improve results\. However, this approach does not incorporate the information of discrete classes into the learning process\. Alternatively, many classical tabular ML models have been adapted for ordinal regression, including ordinal logistic regression\[[6](https://arxiv.org/html/2607.09196#bib.bib41)\], Ordered Random Forest \(ORF\)\[[27](https://arxiv.org/html/2607.09196#bib.bib46)\], and Gaussian Process Ordinal Regression \(GPOR\)\[[12](https://arxiv.org/html/2607.09196#bib.bib44)\]\. For neural networks, dedicated layers and loss functions for ordinal regression have been proposed, such as CORN\[[45](https://arxiv.org/html/2607.09196#bib.bib52)\], CORAL\[[10](https://arxiv.org/html/2607.09196#bib.bib51)\], and CONDOR\[[24](https://arxiv.org/html/2607.09196#bib.bib53)\]\. Detailed descriptions of these algorithms are presented in the Methods section\.

## 3Dataset and feature engineering

In this section, we describe the dataset construction and its basic characteristics\. We also explain the approach taken to feature engineering, in order to obtain the tabular dataset used for downstream model training\.

### 3\.1Dataset curation

In this work, we focus on the Pathfinder Roleplaying Second Edition game system \(referred to asPathfinder 2efor short\) for three reasons\. First, it is highly popular, consistently ranking among the most popular TTRPGs, which makes ML solutions of high interest to many potential users\. Second, it is highly mathematical: monsters are described by roughly a dozen basic attributes and many additional ones, which makes estimating their level challenging for humans yet provides ample opportunity for feature engineering\. Third, it is published under permissive open source licenses \(older works under the Open Game License 1\.0A\[[34](https://arxiv.org/html/2607.09196#bib.bib27)\], newer ones under the Open RPG Creative \(ORC\) License\[[35](https://arxiv.org/html/2607.09196#bib.bib28)\]\), which enables open science principles and allows us to share the data for reproducibility and further research\.

We created a dataset of monsters based on a publicly available database of Pathfinder 2e books, amounting to 6007 monster entries in total from 302 sourcebooks, adventures, and scenarios, the newest of which was published on 26\.04\.2026\. The full list of source books used is available in the GitHub repository\. Creatures come from books published by Paizo; each book provides either many monsters, in the case of monster compendia, or only a few unique ones, in the case of single adventure scenarios\. We note the strong temporal characteristics of monster design: newer monsters are designed based on prior ones\. For example, a novel yet similar variant of an existing monster should have a level comparable to that of the existing one\.

A monster entry provides descriptions and characteristics as a highly complex JSON object\. A rendered display of a monster statistics block, from the Archives of Nethys web page \(officially supported by the publisher Paizo\)\[[3](https://arxiv.org/html/2607.09196#bib.bib2)\], is provided in Figure[1](https://arxiv.org/html/2607.09196#S3.F1)\.

![Refer to caption](https://arxiv.org/html/2607.09196v1/img/monster.png)Figure 1:A monster example from Pathfinder 2e\[[2](https://arxiv.org/html/2607.09196#bib.bib7)\]\.In particular, each entry contains the monster level, which we treat as the ground truth\. This level was obtained through expert estimation and extensive practical gameplay\. In the Pathfinder 2e system, the level is an integer ranging from \-1 to 25\. Because very few monsters exceed level 20, with only 89 in total across levels 21\-25, we aggregate them into a single category denoting “level over 20”; for convenience, we mark it simply as 21\. The distribution of this dependent variable is shown in Figure[2](https://arxiv.org/html/2607.09196#S3.F2)\. This is animbalanced regressionproblem, with comparatively few monsters having very low levels \(\-1 or 0\) and very high levels \(19\-21\)\.

Level is an ordinal variable, being both ordered and discrete, but it does not lie on a ratio scale\. For example, a level 4 monster is not necessarily twice as powerful as a level 2 monster, as higher\-level monsters often possess unique abilities or powerful spells\. The problem therefore requires learning this ordered relation, which is more sophisticated than merely regressing on integers\.

![Refer to caption](https://arxiv.org/html/2607.09196v1/x1.png)Figure 2:Distribution of monsters’ levels\.We also include an important metadata variable: the source book in which each monster was originally published\. By checking the publication date of each book, we can incorporate chronological information about the order in which monsters were designed\. As we show in the Experiments and results section, this is crucial for a realistic evaluation of ML models\.

### 3\.2Feature engineering

The raw input data is complex and contains a significant amount of information that is not directly relevant to predicting the monster level, and in particular information that would not be expected to be available from users at prediction time\. We therefore focus our dataset curation and feature engineering on attributes known to domain experts to be directly and causally related to the level\. For example, a monster with very high armor must have a high level, since otherwise it would be impossible for low\-level players to defeat\. This yields a tabular ML problem in which each monster is described by a vector of features\. We provide three feature sets extracted from the raw statistics: basic, extended, and full\.

Thebasicset consists of 8 fundamental statistics shared by all monsters, as well as by players, which influence almost every aspect of the Pathfinder system\. The first 6 are the basic attributes, from which many other characteristics are derived: Strength \(Str\), Dexterity \(Dex\), Constitution \(Con\), Intelligence \(Int\), Wisdom \(Wis\), and Charisma \(Cha\)\. We also include Hit Points \(HP\), which measure the health or durability of the monster, and Armor Class \(AC\), which indicates how hard the monster is to hit with an attack\. Both directly influence the survivability of the adversary and are therefore closely related to the level\.

Theextendedset adds 12 further characteristics: 4 related to defensive abilities, 4 to offensive, and 4 determining its magical power\.

Perception measures how easily a monster detects players, and also captures some special abilities\. We further include three so\-called saving throws, which determine resistance to spells, special abilities, and attacks: Fortitude, Reflex, and Will\. This set also includes attack features, as nearly every monster possesses at least one melee and one ranged attack, since their main role is to fight players\. In the Pathfinder 2e system, an attack involves rolling a 20\-sided die \(d20\) and modifying the result with a so\-called attack bonus\. If the value is at least equal to the target’s Armor Class \(AC\), the attack succeeds and damage is calculated\. Damage uses one or more multi\-sided dice plus a damage modifier\. Monsters may possess several attacks of each type, but most often use the one with the highest attack bonus\. For each monster, we therefore extracted the highest available attack bonus for both melee and ranged attacks, and computed the expected damage of the corresponding attack, which resulted in 4 features\.

For example, for an attack described as“2d8 \+ 1d4 \+ 3”, the expected damage is the sum of the expected values of each die roll plus the modifier\. Hered8denotes an 8\-sided die, which forms a discrete uniform distribution over the values\[1,2,…,8\]\[1,2,\\dots,8\], i\.e\.,𝒰​\(1,8\)\\mathcal\{U\}\(1,8\)\. With a slight abuse of expected\-value notation, the expected damage is:

𝔼​\[2​d​8\+1​d​4\+3\]=2⋅𝔼​\[𝒰​\(1,8\)\]\+𝔼​\[𝒰​\(1,4\)\]\+3=14\.5\\mathbb\{E\}\[2d8\+1d4\+3\]=2\\cdot\\mathbb\{E\}\[\\mathcal\{U\}\(1,8\)\]\+\\mathbb\{E\}\[\\mathcal\{U\}\(1,4\)\]\+3=14\.5
Some monsters are mages, able to cast spells according to their power\. The Pathfinder system divides spells into 9 spell levels based on their power\. We include the highest available spell level as a feature, because characters \(including players\) gain access to higher\-level spells as they advance, so there is a strong correlation between available spell level and monster level\. Furthermore, mages of the same spell level can differ in power, which is captured by two characteristics: the spell attack modifier, used to hit targets with spells, and the spell Difficulty Class \(DC\), which determines how challenging it is for a target to resist a spell cast by the monster\. We therefore include both of these features as well\. There is also a type of spells that require special points, called Focus\. Even classes that are not typical spellcasters can use them, and because of that, Focus was the last feature included as part of this set\.

Thefullset adds 13 more complex characteristics, in particular several derived from the basic features through complex, non\-linear formulas\. Many of these features are optional; for example, most monsters cannot fly and therefore have a fly speed of zero\. We include features related to movement and to immunities against damage and magic spells\.

First, we add 3 speed features: land, fly, and swim speed\. These also provide implicit information about the availability of powerful movement types; for example,f​l​y\>0fly\>0means that the monster can fly\.

Next, we add the total number of a monster’s immunities and resistances to certain damage types, such as a fire dragon being immune to fire\. Because immunity types are numerous \(over a dozen\) and very sparse \(most monsters have none, or at most one or two\), we include the total number of immunities as a single feature\. Initial experiments showed that this approach yields performance similar to or better than using separate features\.

In addition to the spell\-related features above, we include the number of spells available at each level as 9 separate features\. This is a more detailed information than just the highest spell level\. Introducing both improved the results in initial experiments\.

In principle, models could learn the maximum spell level implicitly from the available spells at each level\. However, most likely because of the limited amount of data, explicitly including the maximum spell level alongside the 9 per\-level spell\-count features further improved results in initial experiments\.

We summarize the three feature sets in Table[1](https://arxiv.org/html/2607.09196#S3.T1)\. Gains from additional features, together with feature importance analyses, are detailed in the Experiments and results section\.

Table 1:Summary of the three feature sets\.

## 4Methods

We frame the problem of predicting the monster level as tabular ordinal regression\. In ordinal regression, also known as ordinal classification, the dependent variable is discrete and ordered, with target values \(classes\) typically represented as integers\. However, the intervals between categories are not necessarily equal\. Moreover, evaluation must take the error magnitude into account, since predicting class 2 instead of class 1 is less problematic than predicting class 3\. These characteristics make ordinal regression similar to both regression and classification, yet distinct in its own right\.

The simplest approach is to treat the problem as standard regression and round the predictions as a post\-processing step\. This assumes a ratio scale and introduces rounding error, but it is simple and allows the use of arbitrary regression models\. Alternatively, many ML models have been modified to incorporate inductive biases specific to ordinal regression\. An important such bias is a cumulative\-probability \(rank\-consistency\) guarantee: the model should predict monotonically non\-increasing scores for higher classes, i\.e\., a level 4 monster is at least as likely to be “at least levelkk” as a level 3 monster\. Similar ideas can also be used to construct loss functions for neural networks\.

### 4\.1Notation

To keep the presentation uniform, we use the following notation throughout this section\. The training set is\{\(𝒙i,yi\)\}i=1N\\\{\(\\bm\{x\}\_\{i\},y\_\{i\}\)\\\}\_\{i=1\}^\{N\}ofNNmonsters, where𝒙i∈ℝd\\bm\{x\}\_\{i\}\\in\\mathbb\{R\}^\{d\}is the feature vector of theii\-th monster andyi∈\{1,…,K\}y\_\{i\}\\in\\\{1,\\dots,K\\\}is its ordinal level, one ofKKordered classes\. The indexiialways denotes a*sample*, and the indexkkalways denotes a*class*or, equivalently, one of theK−1K\-1ordinal*thresholds*that separate consecutive classes\.

Many methods reduce ordinal regression toK−1K\-1binary subproblems, in which thekk\-th subproblem asks whetheryi\>ky\_\{i\}\>k; we writefk​\(𝒙\)f\_\{k\}\(\\bm\{x\}\)for the \(probabilistic\) output of thekk\-th binary sub\-model andf​\(𝒙\)f\(\\bm\{x\}\)for a single scalar score\. Thresholds \(also called cutpoints\) are writtenθ1<θ2<⋯<θK−1\\theta\_\{1\}<\\theta\_\{2\}<\\dots<\\theta\_\{K\-1\}, with the conventionθ0=−∞\\theta\_\{0\}=\-\\inftyandθK=\+∞\\theta\_\{K\}=\+\\infty\.

We denote by𝒘\\bm\{w\}a vector of linear weights, byσ​\(⋅\)\\sigma\(\\cdot\)the logistic sigmoid, and byΦ​\(⋅\)\\Phi\(\\cdot\)the standard normal cumulative distribution function \(the probit function\)\. Estimated quantities carry a hat, for exampley^i\\hat\{y\}\_\{i\}for the predicted level andP^​\(yi=k\)\\hat\{P\}\(y\_\{i\}=k\)for a predicted class probability\. Neural network parameters are collectively denotedΘ\\Theta\.

### 4\.2Regression models with rounding

Tabular regression models, both linear and non\-linear, can be applied directly to monster level prediction\. These include linear regression with ridge \(L2\) regularization \(Ridge\), support vector machines \(SVM\)\[[13](https://arxiv.org/html/2607.09196#bib.bib37)\], Random Forest\[[7](https://arxiv.org/html/2607.09196#bib.bib35)\], and boosting methods such as LightGBM\[[25](https://arxiv.org/html/2607.09196#bib.bib34)\]\.

An approach based on k\-nearest neighbors \(kNN\) is implicitly described in the Pathfinder 2e rulebook, which mentions estimating a monster’s level by comparison with the most similar monsters in official books\. kNN is therefore an important baseline that reflects human\-level performance\.

After applying such a method, the continuous output is transformed into a discrete ordinal label byrounding\. Many options exist here, from mathematical rounding \(a threshold of 0\.5 between levels\) to sophisticated stacking using an additional classifier model\[[40](https://arxiv.org/html/2607.09196#bib.bib38)\]\. In this work, we use mathematical rounding, which maps continuous predictions up or down based on a fixed threshold of 0\.5\. It introduces no additional parameters, albeit it may give suboptimal results for imbalanced regression\. For example, if the model consistently predicts values that are too small, it could be beneficial to use a lower threshold, such as rounding up at a threshold of 0\.3\. We leave the exploration of this idea for future work\.

### 4\.3Tabular ordinal regression models

We selected several dedicated tabular ordinal regression models for evaluation\. In particular, we focused on modified variants of the regression models above, for example Ordered Random Forest\[[27](https://arxiv.org/html/2607.09196#bib.bib46)\], to compare with Random Forest regression with rounding\. This lets us fairly evaluate the gains from these problem\-specific modifications\. Selection also relied on the availability of easy\-to\-use, open source implementations in Python\.

#### 4\.3\.1Simple Approach to Ordinal Classification \(SAOC\)

Frank and Hall\[[18](https://arxiv.org/html/2607.09196#bib.bib40)\]proposed a simple, universal way to adapt an arbitrary binary classifier to ordinal regression, which we call the Simple Approach to Ordinal Classification \(SAOC\)\. The original problem withKKordered classes is decomposed intoK−1K\-1binary classification problems, indexed byk∈\{1,…,K−1\}k\\in\\\{1,\\dots,K\-1\\\}, and one binary classifier is trained for each\. Thekk\-th classifier estimates the probability that the level exceedskk, i\.e\.,fk​\(𝒙\)≈P​\(y\>k∣𝒙\)f\_\{k\}\(\\bm\{x\}\)\\approx P\(y\>k\\mid\\bm\{x\}\)\. For thekk\-th problem, the label of sampleiiis transformed according to Equation[1](https://arxiv.org/html/2607.09196#S4.E1)\.

yi\(k\)=\{1if​yi\>k0if​yi≤ky\_\{i\}^\{\(k\)\}=\\begin\{cases\}1&\\text\{if \}y\_\{i\}\>k\\\\ 0&\\text\{if \}y\_\{i\}\\leq k\\\\ \\end\{cases\}\(1\)
In particular, for the first classifier \(k=1k=1\), every sample whose true level is 1 is relabeled 0, and every sample of a higher level is relabeled 1\. For the last classifier \(k=K−1k=K\-1\), only samples of the highest levelKKare relabeled 1, and all others 0\. Because of this construction, no classifier with indexKKis needed\.

This yieldsK−1K\-1datasets with the same features but different transformed labels, on each of which we train a standard binary classifier\. To make a prediction for a new sample𝒙\\bm\{x\}, we first obtainfk​\(𝒙\)=P^​\(y\>k∣𝒙\)f\_\{k\}\(\\bm\{x\}\)=\\hat\{P\}\(y\>k\\mid\\bm\{x\}\)from each classifier and then recombine them into class probabilities following Equation[2](https://arxiv.org/html/2607.09196#S4.E2)\.

P^​\(y=k∣𝒙\)=\{1−f1​\(𝒙\)if​k=1fk−1​\(𝒙\)​\(1−fk​\(𝒙\)\)if​1<k<KfK−1​\(𝒙\)if​k=K\\hat\{P\}\(y=k\\mid\\bm\{x\}\)=\\begin\{cases\}1\-f\_\{1\}\(\\bm\{x\}\)&\\text\{if \}k=1\\\\ f\_\{k\-1\}\(\\bm\{x\}\)\\,\\bigl\(1\-f\_\{k\}\(\\bm\{x\}\)\\bigr\)&\\text\{if \}1<k<K\\\\ f\_\{K\-1\}\(\\bm\{x\}\)&\\text\{if \}k=K\\\\ \\end\{cases\}\(2\)
We then predict the class with the highest probability\. Because the binary classifiers are trained independently, they need not beconsistent: a model may assignfk​\(𝒙\)\>fk−1​\(𝒙\)f\_\{k\}\(\\bm\{x\}\)\>f\_\{k\-1\}\(\\bm\{x\}\), i\.e\., a higher probability toy\>ky\>kthan toy\>k−1y\>k\-1\. This should not happen for a well\-trained model with enough data, but it is not formally enforced\.

In our experiments, SAOC uses a Random Forest as the underlying binary classifier\. It gave stable results in initial experiments, is fast to train and parallelizable \(which is highly relevant when trainingK−1=21K\-1=21underlying models\), and has low sensitivity to hyperparameter values\[[39](https://arxiv.org/html/2607.09196#bib.bib36)\]\.

#### 4\.3\.2Ordered Logistic Regression \(ORD\)

Ordered Logistic Regression \(ORD\)\[[6](https://arxiv.org/html/2607.09196#bib.bib41)\]adapts logistic regression to ordinal labels\. It remains a linear model, but replaces the standard odds with cumulative odds, modeling the probability that the level is at mostkk\. This has the considerable advantage of guaranteeing consistent predictions by incorporating the ordering into the learning process\. Writingpk=P​\(y=k\)p\_\{k\}=P\(y=k\), the cumulative probability of classkkor below is given by Equation[3](https://arxiv.org/html/2607.09196#S4.E3), the corresponding odds by Equation[4](https://arxiv.org/html/2607.09196#S4.E4), and the logit by Equation[5](https://arxiv.org/html/2607.09196#S4.E5)\.

P​\(y≤k\)=p1\+⋯\+pkP\(y\\leq k\)=p\_\{1\}\+\\dots\+p\_\{k\}\(3\)
odds​\(y≤k\)=P​\(y≤k\)P​\(y\>k\)=p1\+⋯\+pkpk\+1\+⋯\+pK\\mathrm\{odds\}\(y\\leq k\)=\\frac\{P\(y\\leq k\)\}\{P\(y\>k\)\}=\\frac\{p\_\{1\}\+\\dots\+p\_\{k\}\}\{p\_\{k\+1\}\+\\dots\+p\_\{K\}\}\(4\)
logit​\(y≤k\)=ln⁡P​\(y≤k\)1−P​\(y≤k\),k=1,…,K−1\\mathrm\{logit\}\(y\\leq k\)=\\ln\\frac\{P\(y\\leq k\)\}\{1\-P\(y\\leq k\)\},\\quad k=1,\\dots,K\-1\(5\)
This model is based on the proportional\-odds assumption, which states that the effect of the predictors is constant across all thresholds\. In other words, if a particular feature increases the probability of belonging to a lower class, it does so consistently across all boundaries between classes\. The weight vector𝒘\\bm\{w\}is shared across thresholds, while only the intercepts \(which play the role of the cutpointsθk\\theta\_\{k\}\) vary\. Under this assumption, the cumulative logit for thresholdkkis:

logit​\(y≤k\)=θk−𝒘⊤​𝒙,k=1,…,K−1\\mathrm\{logit\}\(y\\leq k\)=\\theta\_\{k\}\-\\bm\{w\}^\{\\top\}\\bm\{x\},\\quad k=1,\\dots,K\-1\(6\)
where𝒙\\bm\{x\}is the feature vector,𝒘\\bm\{w\}the shared regression coefficients, andθk\\theta\_\{k\}the intercept \(cutpoint\) of thekk\-th cumulative probability\. The coefficients𝒘\\bm\{w\}and the cutpointsθ1,…,θK−1\\theta\_\{1\},\\dots,\\theta\_\{K\-1\}are estimated jointly by maximum likelihood\.

#### 4\.3\.3Threshold\-based linear models

Threshold\-based models\[[41](https://arxiv.org/html/2607.09196#bib.bib42)\]were proposed to generalize the loss functions of linear classifiers to multiple ordered categories\. The framework generalizes the single\-threshold setting of binary classification by introducingK−1K\-1thresholdsθ1<θ2<⋯<θK−1\\theta\_\{1\}<\\theta\_\{2\}<\\dots<\\theta\_\{K\-1\}forKKclasses, together withθ0=−∞\\theta\_\{0\}=\-\\inftyandθK=\+∞\\theta\_\{K\}=\+\\infty\. These thresholds partition the real line intoKKsegments, and a scalar model outputf​\(𝒙\)f\(\\bm\{x\}\)that falls in\(θk−1,θk\)\(\\theta\_\{k\-1\},\\theta\_\{k\}\)assigns the instance to classkk\.

Two loss variants are used: Immediate\-Threshold \(IT\) and All\-Threshold \(AT\)\. Both generalize an arbitrary binary lossℓ\\ell, and we use the hinge loss for its robustness to outliers\. It can be shown\[[36](https://arxiv.org/html/2607.09196#bib.bib45)\]that IT is a surrogate for the 0\-1 loss, while AT is a surrogate for the absolute loss\.

The Immediate\-Threshold \(IT\) variant penalizes only the two thresholds that bound the correct class\. For a sample\(𝒙i,yi\)\(\\bm\{x\}\_\{i\},y\_\{i\}\)withyi=ky\_\{i\}=k, the pair\(θk−1,θk\)\(\\theta\_\{k\-1\},\\theta\_\{k\}\)delimits the range in which the output is considered correct, and predictions outside it are penalized, as shown in Equation[7](https://arxiv.org/html/2607.09196#S4.E7)\.

LIT​\(f​\(𝒙i\),yi\)=ℓ​\(f​\(𝒙i\)−θyi−1\)\+ℓ​\(θyi−f​\(𝒙i\)\)L\_\{\\mathrm\{IT\}\}\\bigl\(f\(\\bm\{x\}\_\{i\}\),y\_\{i\}\\bigr\)=\\ell\\bigl\(f\(\\bm\{x\}\_\{i\}\)\-\\theta\_\{y\_\{i\}\-1\}\\bigr\)\+\\ell\\bigl\(\\theta\_\{y\_\{i\}\}\-f\(\\bm\{x\}\_\{i\}\)\\bigr\)\(7\)
The All\-Threshold \(AT\) variant sums penalties over*all*thresholds, as shown in Equation[8](https://arxiv.org/html/2607.09196#S4.E8)\. This not only encourages correct classification but also more heavily penalizes predictions far from the correct label\. The penalty grows linearly with the distance between the predicted and the correct class\.

LAT​\(f​\(𝒙i\),yi\)=∑k=1K−1\{ℓ​\(θk−f​\(𝒙i\)\)if​k≥yiℓ​\(f​\(𝒙i\)−θk\)if​k<yiL\_\{\\mathrm\{AT\}\}\\bigl\(f\(\\bm\{x\}\_\{i\}\),y\_\{i\}\\bigr\)=\\sum\_\{k=1\}^\{K\-1\}\\begin\{cases\}\\ell\\bigl\(\\theta\_\{k\}\-f\(\\bm\{x\}\_\{i\}\)\\bigr\)&\\text\{if \}k\\geq y\_\{i\}\\\\ \\ell\\bigl\(f\(\\bm\{x\}\_\{i\}\)\-\\theta\_\{k\}\\bigr\)&\\text\{if \}k<y\_\{i\}\\end\{cases\}\(8\)
In our initial experiments, the AT variant performed consistently slightly worse than the IT variant\. Thus, we report results only for the latter\.

#### 4\.3\.4Ordered Random Forest

The Ordered Random Forest \(ORF\)\[[27](https://arxiv.org/html/2607.09196#bib.bib46),[28](https://arxiv.org/html/2607.09196#bib.bib47)\]has been proposed as an adaptation of the Random Forest to ordinal regression\. It estimates the conditional class probabilitiesP​\(yi=k∣𝒙i\)P\(y\_\{i\}=k\\mid\\bm\{x\}\_\{i\}\)from cumulative, nested binary indicators𝟏​\{yi≤k\}\\mathbf\{1\}\\\{y\_\{i\}\\leq k\\\}fork∈\{1,…,K−1\}k\\in\\\{1,\\dots,K\-1\\\}\. This decomposes the problem intoK−1K\-1binary classification problems, whose forests produce cumulative estimates \(Equation[9](https://arxiv.org/html/2607.09196#S4.E9)\)\.

F^k​\(𝒙i\)=P^​\(yi≤k∣𝒙i\)\\hat\{F\}\_\{k\}\(\\bm\{x\}\_\{i\}\)=\\hat\{P\}\(y\_\{i\}\\leq k\\mid\\bm\{x\}\_\{i\}\)\(9\)
The class probabilities are then obtained as successive differences of the cumulative estimates \(Equation[10](https://arxiv.org/html/2607.09196#S4.E10)\)

P^​\(yi=k\)=\{F^1​\(𝒙i\)if​k=1F^k​\(𝒙i\)−F^k−1​\(𝒙i\)if​1<k<K1−F^K−1​\(𝒙i\)if​k=K\\hat\{P\}\(y\_\{i\}=k\)=\\begin\{cases\}\\hat\{F\}\_\{1\}\(\\bm\{x\}\_\{i\}\)&\\text\{if \}k=1\\\\ \\hat\{F\}\_\{k\}\(\\bm\{x\}\_\{i\}\)\-\\hat\{F\}\_\{k\-1\}\(\\bm\{x\}\_\{i\}\)&\\text\{if \}1<k<K\\\\ 1\-\\hat\{F\}\_\{K\-1\}\(\\bm\{x\}\_\{i\}\)&\\text\{if \}k=K\\end\{cases\}\(10\)
Because the binary forests are trained independently, a difference can occasionally be negative for imbalanced data with sparsely populated classes\. Therefore, we clip each estimate at zero,P^​\(yi=k\)←max⁡\(0,P^​\(yi=k\)\)\\hat\{P\}\(y\_\{i\}=k\)\\leftarrow\\max\\bigl\(0,\\hat\{P\}\(y\_\{i\}=k\)\\bigr\), and renormalize so that the class probabilities sum to one \(Equation[11](https://arxiv.org/html/2607.09196#S4.E11)\)\. In practice this correction is rarely needed for well\-performing models\.

P^​\(yi=k\)=P^​\(yi=k\)∑k′=1KP^​\(yi=k′\),1≤k≤K\\hat\{P\}\(y\_\{i\}=k\)=\\frac\{\\hat\{P\}\(y\_\{i\}=k\)\}\{\\sum\_\{k^\{\\prime\}=1\}^\{K\}\\hat\{P\}\(y\_\{i\}=k^\{\\prime\}\)\},\\quad 1\\leq k\\leq K\(11\)

#### 4\.3\.5Gaussian Processes for Ordinal Regression

Gaussian Processes for Ordinal Regression \(GPOR\)\[[12](https://arxiv.org/html/2607.09196#bib.bib44)\]have been proposed to adapt the Gaussian processes \(GPs\) for ordinal regression tasks\. They provide a flexible, nonparametric, kernel\-based probabilistic model with built\-in uncertainty estimates, which can be an additional benefit for game designers when balancing monsters\. In this method, the latent functionffis modeled by a Gaussian process, and the likelihood is defined by a threshold model that generalizes the probit function to ordinal targets, interpreting each ordinal label as an interval of the latent space delimited by the thresholds\.

Given a latent valuef=f​\(𝒙\)f=f\(\\bm\{x\}\), thresholdsθ0<θ1<⋯<θK\\theta\_\{0\}<\\theta\_\{1\}<\\dots<\\theta\_\{K\}\(withθ0=−∞\\theta\_\{0\}=\-\\infty,θK=\+∞\\theta\_\{K\}=\+\\infty\), and a scale \(noise\) parameterss, the probability of classkkis expressed through the standard normal CDFΦ​\(⋅\)\\Phi\(\\cdot\)\(Equation[12](https://arxiv.org/html/2607.09196#S4.E12)\)\.

P​\(y=k∣f\)=Φ​\(θk−fs\)−Φ​\(θk−1−fs\),k=1,…,KP\(y=k\\mid f\)=\\Phi\\\!\\left\(\\frac\{\\theta\_\{k\}\-f\}\{s\}\\right\)\-\\Phi\\\!\\left\(\\frac\{\\theta\_\{k\-1\}\-f\}\{s\}\\right\),\\quad k=1,\\dots,K\(12\)
The thresholdsθk\\theta\_\{k\}and the scalessare learned by maximizing the posterior given the observed dataDD\(Equation[13](https://arxiv.org/html/2607.09196#S4.E13)\), whereP​\(D∣θ,s\)P\(D\\mid\\theta,s\)is the data likelihood\. A prior may be specified from domain knowledge, chosen to be uninformative, or omitted\. The last option reduces the objective to log marginal likelihood maximization \(MLE\) rather than maximum a posteriori \(MAP\) estimation\.

P​\(θ,s∣D\)∝P​\(D∣θ,s\)​P​\(θ\)P\(\\theta,s\\mid D\)\\propto P\(D\\mid\\theta,s\)\\,P\(\\theta\)\(13\)
This model combines the flexibility of Gaussian processes with an explicit ordinal likelihood\. A known limitation is its poor scalability in the number of training samples, on the order ofO​\(N3\)O\(N^\{3\}\)\. For Pathfinder this is not particularly problematic, as the number of monsters is relatively small\. We use the RBF \(squared exponential\) kernel and set no prior, maximizing the pure log marginal likelihood, since this combination is highly flexible and data\-driven\. GPflow\[[31](https://arxiv.org/html/2607.09196#bib.bib48)\]is used for the implementation, with a Variational Gaussian Process \(VGP\) as we have a non\-Gaussian likelihood, and parameters are optimized using L\-BFGS\-B\.

### 4\.4Neural networks for ordinal regression

The models above introduce modifications that assume a particular functional form of the underlying model\. Adapting neural networks to ordinal regression requires more general, differentiable modifications, many of which have been proposed in the literature\. Typically, ordinal regression is decomposed into a series of binary classification tasks, with a problem\-specific inductive bias introduced through the loss function and/or the final layers of a multilayer perceptron \(MLP\) classifier\.

Because our data is tabular, we focus exclusively on the MLP model\. We keep a similar number of layers and parameters across architectures\. Our base network has 2 hidden layers \(128 and 64 neurons, respectively\) with ReLU activation, followed by a model\-dependent output layer whose number of outputs, activation, and specific modifications depend on the architecture, as described below\. These choices were made based on initial experiments, which showed no significant gains from larger or more heavily regularized models\. In what follows,Θ\\Thetadenotes the network parameters\.

#### 4\.4\.1Cumulative link model

The cumulative link model \(CLM\)\[[32](https://arxiv.org/html/2607.09196#bib.bib43),[36](https://arxiv.org/html/2607.09196#bib.bib45)\]takes a probabilistic view of the problem: instead of scoring each class directly, it models the*cumulative*class probabilities through a link function\. It belongs to the same family of threshold\-based surrogates, like IT and AT losses described above, but replaces the margin loss with a negative log\-likelihood\. In our case the scalar scoref​\(𝒙\)f\(\\bm\{x\}\)is produced by an MLP with parametersΘ\\Theta\.

Given the scoref​\(𝒙\)f\(\\bm\{x\}\), ordered cutpoints \(thresholds\)θ1≤θ2≤⋯≤θK−1\\theta\_\{1\}\\leq\\theta\_\{2\}\\leq\\dots\\leq\\theta\_\{K\-1\}, and a link functionσ\\sigma, the CLM sets the cumulative probability of classkkor below to:

σ​\(θk−f​\(𝒙\)\)=P^​\(y≤k∣𝒙\),k=1,…,K−1\\sigma\\bigl\(\\theta\_\{k\}\-f\(\\bm\{x\}\)\\bigr\)=\\hat\{P\}\(y\\leq k\\mid\\bm\{x\}\),\\qquad k=1,\\dots,K\-1\(14\)
Using the logistic sigmoid asσ\\sigmayields the proportional\-odds \(cumulative\-logit\) model, while using the standard normal CDFΦ\\Phiinstead recovers the probit link employed by GPOR\. The per\-class probabilities are the successive differences of these cumulative probabilities:

P^​\(y=k\)=\{σ​\(θ1−f​\(𝒙\)\)if​k=1σ​\(θk−f​\(𝒙\)\)−σ​\(θk−1−f​\(𝒙\)\)if​1<k<K1−σ​\(θK−1−f​\(𝒙\)\)if​k=K\\hat\{P\}\(y=k\)=\\begin\{cases\}\\sigma\\bigl\(\\theta\_\{1\}\-f\(\\bm\{x\}\)\\bigr\)&\\text\{if \}k=1\\\\ \\sigma\\bigl\(\\theta\_\{k\}\-f\(\\bm\{x\}\)\\bigr\)\-\\sigma\\bigl\(\\theta\_\{k\-1\}\-f\(\\bm\{x\}\)\\bigr\)&\\text\{if \}1<k<K\\\\ 1\-\\sigma\\bigl\(\\theta\_\{K\-1\}\-f\(\\bm\{x\}\)\\bigr\)&\\text\{if \}k=K\\end\{cases\}\(15\)
The ordering constraintθ1≤⋯≤θK−1\\theta\_\{1\}\\leq\\dots\\leq\\theta\_\{K\-1\}ensures that these differences are non\-negative, so that Equation[15](https://arxiv.org/html/2607.09196#S4.E15)defines a valid, rank\-consistent probability distribution\. ForK=2K=2it reduces to standard binary logistic regression\.

The network parametersΘ\\Thetaand the cutpoints𝜽=\(θ1,…,θK−1\)\\bm\{\\theta\}=\(\\theta\_\{1\},\\dots,\\theta\_\{K\-1\}\)are learned jointly by minimizing the cumulative\-link negative log\-likelihood surrogate\[[36](https://arxiv.org/html/2607.09196#bib.bib45)\]over the training set:

L​\(Θ,𝜽\)=−∑i=1Nlog⁡P^​\(y=yi∣𝒙i\)L\(\\Theta,\\bm\{\\theta\}\)=\-\\sum\_\{i=1\}^\{N\}\\log\\hat\{P\}\(y=y\_\{i\}\\mid\\bm\{x\}\_\{i\}\)\(16\)
whereP^​\(y=yi∣𝒙i\)\\hat\{P\}\(y=y\_\{i\}\\mid\\bm\{x\}\_\{i\}\)is given by Equation[15](https://arxiv.org/html/2607.09196#S4.E15)\. Because the cutpoints are learned directly from the data, the model adapts to non\-uniform label distributions and spacings\. At inference time, we predict the most probable class,y^=arg⁡maxk⁡P^​\(y=k\)\\hat\{y\}=\\arg\\max\_\{k\}\\hat\{P\}\(y=k\)\.

#### 4\.4\.2NNRank

NNRank\[[11](https://arxiv.org/html/2607.09196#bib.bib49)\]recasts ordinal regression as multi\-output binary classification\. Each level is encoded as a binary target vector of lengthKK: for sampleii, thekk\-th entry is 1 wheneverk≤yik\\leq y\_\{i\}\(Equation[17](https://arxiv.org/html/2607.09196#S4.E17)\)\.

ti,k=𝟏​\{k≤yi\},k=1,…,Kt\_\{i,k\}=\\mathbf\{1\}\\\{k\\leq y\_\{i\}\\\},\\quad k=1,\\dots,K\(17\)
The output layer hasKKneurons with sigmoid activations, sofk​\(𝒙i\)≈P​\(yi≥k\)f\_\{k\}\(\\bm\{x\}\_\{i\}\)\\approx P\(y\_\{i\}\\geq k\), and the loss is averaged over allKKoutputs\. This, together with the encoding, provides the ordinal inductive bias\. Following the original work\[[11](https://arxiv.org/html/2607.09196#bib.bib49)\], we use the squared error as the base loss\.

At inference time, each probabilityfk​\(𝒙i\)f\_\{k\}\(\\bm\{x\}\_\{i\}\)is thresholded at0\.50\.5, and the predicted level is the number of leading 1s \(the count of positive outputs up to the first negative one\)\. This does not guarantee consistency; for example, a pattern such as\[1,1,0,1,1,1,0,…\]\[1,1,0,1,1,1,0,\\dots\]is possible\. However, it arises rarely for well\-performing models and did not appear in our initial experiments\.

#### 4\.4\.3OR\-CNN

The Ordinal Regression CNN \(OR\-CNN\)\[[33](https://arxiv.org/html/2607.09196#bib.bib50)\]was originally proposed for age estimation\. However, it uses a dedicated loss that can be used for any imbalanced ordinal regression problem\. It transforms theKK\-class problem intoK−1K\-1binary tasks with the same encoding as NNRank, wherefk​\(𝒙i\)≈P​\(yi\>k\)f\_\{k\}\(\\bm\{x\}\_\{i\}\)\\approx P\(y\_\{i\}\>k\)\. Unlike NNRank, the prediction is always the sum of the binary decisions of all tasks \(Equation[18](https://arxiv.org/html/2607.09196#S4.E18)\)\.

y^i=1\+∑k=1K−1𝟏​\{fk​\(𝒙i\)\>0\.5\}\\hat\{y\}\_\{i\}=1\+\\sum\_\{k=1\}^\{K\-1\}\\mathbf\{1\}\\\{f\_\{k\}\(\\bm\{x\}\_\{i\}\)\>0\.5\\\}\(18\)
No consistency constraint is enforced between tasks, which simplifies training\. To improve results under class imbalance, OR\-CNN uses a weighted binary cross\-entropy \(rather than the squared error of NNRank\), where thekk\-th task carries a weightλk\\lambda\_\{k\}that is inversely related toNkN\_\{k\}, the number of training samples for that task \(Equation[19](https://arxiv.org/html/2607.09196#S4.E19)\)\.

λk=Nk∑k′=1KNk′\\lambda\_\{k\}=\\frac\{\\sqrt\{N\_\{k\}\}\}\{\\sum\_\{k^\{\\prime\}=1\}^\{K\}\\sqrt\{N\_\{k^\{\\prime\}\}\}\}\(19\)
This gives higher weight to less populated tasks, and the authors show\[[33](https://arxiv.org/html/2607.09196#bib.bib50)\]that it improves overall results\. The square root smooths the weights, avoiding excessively large weights for tasks with very few samples, which could cause the model to overfit them\.

#### 4\.4\.4CORAL

The COnsistent RAnk Logits \(CORAL\) model\[[10](https://arxiv.org/html/2607.09196#bib.bib51)\]enforces prediction consistency, in contrast to the approaches above\. Like OR\-CNN, it decomposes the problem intoK−1K\-1binary tasks and predicts the level by summing the binary decisionsfk​\(𝒙i\)f\_\{k\}\(\\bm\{x\}\_\{i\}\)\. CORAL enforces consistency by sharing the weight vector𝒘\\bm\{w\}of the output layer across all tasks, so that only the biasesbkb\_\{k\}are task\-specific\.

Formally, consistency is expressed as rank monotonicity,f1​\(𝒙i\)≥f2​\(𝒙i\)≥⋯≥fK−1​\(𝒙i\)f\_\{1\}\(\\bm\{x\}\_\{i\}\)\\geq f\_\{2\}\(\\bm\{x\}\_\{i\}\)\\geq\\dots\\geq f\_\{K\-1\}\(\\bm\{x\}\_\{i\}\)\. The authors prove that minimizing the average binary cross\-entropy over theK−1K\-1tasks yields non\-increasing biasesb1≥b2≥⋯≥bK−1b\_\{1\}\\geq b\_\{2\}\\geq\\dots\\geq b\_\{K\-1\}\. Since𝒘\\bm\{w\}is shared, the outputs are then non\-increasing as well, giving rank\-monotonic and consistent predictions\.

#### 4\.4\.5CORN

Conditional Ordinal Regression for Neural networks \(CORN\)\[[45](https://arxiv.org/html/2607.09196#bib.bib52)\]guarantees rank consistency without the weight\-sharing constraint of CORAL, which can limit model expressiveness\. Instead, CORN uses a training scheme based on conditional training subsets and the chain rule of probability\. As in CORAL, the problem is decomposed intoK−1K\-1binary tasks, but CORN estimates a series of*conditional*probabilities over the nested events\{yi\>k\}⊆\{yi\>k−1\}\\\{y\_\{i\}\>k\\\}\\subseteq\\\{y\_\{i\}\>k\-1\\\}\. The output of thekk\-th task is defined in Equation[20](https://arxiv.org/html/2607.09196#S4.E20), and the unconditional probabilities are recovered by the chain rule in Equation[21](https://arxiv.org/html/2607.09196#S4.E21)\.

fk​\(𝒙i\)=\{P​\(yi\>k​∣yi\>​k−1\)if​k\>1P​\(yi\>1\)if​k=1f\_\{k\}\(\\bm\{x\}\_\{i\}\)=\\begin\{cases\}P\(y\_\{i\}\>k\\mid y\_\{i\}\>k\-1\)&\\text\{if \}k\>1\\\\ P\(y\_\{i\}\>1\)&\\text\{if \}k=1\\end\{cases\}\(20\)
P​\(yi\>k\)=∏j=1kfj​\(𝒙i\)P\(y\_\{i\}\>k\)=\\prod\_\{j=1\}^\{k\}f\_\{j\}\(\\bm\{x\}\_\{i\}\)\(21\)
Because0≤fj​\(𝒙i\)≤10\\leq f\_\{j\}\(\\bm\{x\}\_\{i\}\)\\leq 1for alljj, the unconditional probabilities are non\-increasing, which guarantees rank consistency \(Equation[22](https://arxiv.org/html/2607.09196#S4.E22)\)\.

P​\(yi\>1\)≥P​\(yi\>2\)≥⋯≥P​\(yi\>K−1\)P\(y\_\{i\}\>1\)\\geq P\(y\_\{i\}\>2\)\\geq\\dots\\geq P\(y\_\{i\}\>K\-1\)\(22\)
Training uses conditional subsets, one per binary task\. The subsetSkS\_\{k\}is used to trainP​\(yi\>k​∣yi\>​k−1\)P\(y\_\{i\}\>k\\mid y\_\{i\}\>k\-1\)and contains only the samples that passed the previous threshold:

The loss minimized during backpropagation is

L​\(Θ\)=−∑k=1K−1∑i:\(𝒙i,yi\)∈Sk\[log⁡fk​\(𝒙i\)​1​\{yi\>k\}\+log⁡\(1−fk​\(𝒙i\)\)​1​\{yi≤k\}\]∑k=1K−1\|Sk\|L\(\\Theta\)=\-\\frac\{\\sum\_\{k=1\}^\{K\-1\}\\sum\_\{i\\,:\\,\(\\bm\{x\}\_\{i\},y\_\{i\}\)\\in S\_\{k\}\}\\Bigl\[\\log f\_\{k\}\(\\bm\{x\}\_\{i\}\)\\,\\mathbf\{1\}\\\{y\_\{i\}\>k\\\}\+\\log\\bigl\(1\-f\_\{k\}\(\\bm\{x\}\_\{i\}\)\\bigr\)\\,\\mathbf\{1\}\\\{y\_\{i\}\\leq k\\\}\\Bigr\]\}\{\\sum\_\{k=1\}^\{K\-1\}\|S\_\{k\}\|\}\(23\)
For numerical stability, the authors also give an equivalent formulation in terms of logitszi,kz\_\{i,k\}, wherelog⁡fk​\(𝒙i\)=log⁡σ​\(zi,k\)\\log f\_\{k\}\(\\bm\{x\}\_\{i\}\)=\\log\\sigma\(z\_\{i,k\}\):

L​\(Θ\)=−∑k=1K−1∑i:\(𝒙i,yi\)∈Sk\[log⁡σ​\(zi,k\)​1​\{yi\>k\}\+\(log⁡σ​\(zi,k\)−zi,k\)​1​\{yi≤k\}\]∑k=1K−1\|Sk\|L\(\\Theta\)=\-\\frac\{\\sum\_\{k=1\}^\{K\-1\}\\sum\_\{i\\,:\\,\(\\bm\{x\}\_\{i\},y\_\{i\}\)\\in S\_\{k\}\}\\Bigl\[\\log\\sigma\(z\_\{i,k\}\)\\,\\mathbf\{1\}\\\{y\_\{i\}\>k\\\}\+\\bigl\(\\log\\sigma\(z\_\{i,k\}\)\-z\_\{i,k\}\\bigr\)\\,\\mathbf\{1\}\\\{y\_\{i\}\\leq k\\\}\\Bigr\]\}\{\\sum\_\{k=1\}^\{K\-1\}\|S\_\{k\}\|\}\(24\)
Prediction is as in CORAL: the predicted level is the count of unconditional probabilities of at least0\.50\.5\. In CORN, however, rank consistency is guaranteed by construction of the training scheme\.

#### 4\.4\.6CONDOR

CONDitionals for Ordinal Regression \(CONDOR\)\[[24](https://arxiv.org/html/2607.09196#bib.bib53)\]combines several of the advances above, providing built\-in rank consistency without weight sharing\. Like CORN, it transforms the task intoK−1K\-1binary problems, and like NNRank it uses the binary encodingti,kt\_\{i,k\}of Equation[17](https://arxiv.org/html/2607.09196#S4.E17)\. Predictions are obtained as in OR\-CNN, by thresholding the marginal probabilities at0\.50\.5and summing them \(Equation[18](https://arxiv.org/html/2607.09196#S4.E18)\)\. As in CORN, CONDOR estimates the conditional probabilitiesqk​\(𝒙;Θ\)q\_\{k\}\(\\bm\{x\};\\Theta\)defined in Equation[25](https://arxiv.org/html/2607.09196#S4.E25)\.

qk​\(𝒙;Θ\)=P​\(y\>k​∣y\>​k−1,𝒙;Θ\)q\_\{k\}\(\\bm\{x\};\\Theta\)=P\\bigl\(y\>k\\mid y\>k\-1,\\bm\{x\};\\Theta\\bigr\)\(25\)
The boundary condition “y\>0y\>0” is set to unit probability\. The marginal probabilities follow from the product rule, as in CORN \(Equation[21](https://arxiv.org/html/2607.09196#S4.E21)\)\.

pk​\(𝒙;Θ\)=∏j=1kqj​\(𝒙;Θ\)p\_\{k\}\(\\bm\{x\};\\Theta\)=\\prod\_\{j=1\}^\{k\}q\_\{j\}\(\\bm\{x\};\\Theta\)\(26\)
Each output node represents oneqk​\(𝒙;Θ\)q\_\{k\}\(\\bm\{x\};\\Theta\)and uses a sigmoid activation, soqk​\(𝒙;Θ\)∈\(0,1\)q\_\{k\}\(\\bm\{x\};\\Theta\)\\in\(0,1\)and the marginals of Equation[26](https://arxiv.org/html/2607.09196#S4.E26)are guaranteed to be non\-increasing\. Rank consistency thus holds by construction, for any upstream architecture\. CONDOR is trained with a maximum likelihood loss\. Lettingti,kt\_\{i,k\}be the binary target for taskkk\(Equation[17](https://arxiv.org/html/2607.09196#S4.E17)\) andBCE​\(t,q\)=−\[t​log⁡q\+\(1−t\)​log⁡\(1−q\)\]\\mathrm\{BCE\}\(t,q\)=\-\\bigl\[t\\log q\+\(1\-t\)\\log\(1\-q\)\\bigr\]the binary cross\-entropy, the loss is defined in Equation[27](https://arxiv.org/html/2607.09196#S4.E27)\.

L​\(Θ\)=∑i=1N∑k=1K−1ti,k−1⋅BCE​\(ti,k,qk​\(𝒙i;Θ\)\)L\(\\Theta\)=\\sum\_\{i=1\}^\{N\}\\sum\_\{k=1\}^\{K\-1\}t\_\{i,k\-1\}\\cdot\\mathrm\{BCE\}\\bigl\(t\_\{i,k\},\\,q\_\{k\}\(\\bm\{x\}\_\{i\};\\Theta\)\\bigr\)\(27\)
The factorti,k−1t\_\{i,k\-1\}ensures that only the conditional probabilities on the “active” path \(those withti,k−1=1t\_\{i,k\-1\}=1\) contribute to the loss, mirroring the conditional subsets of CORN\. The authors prove that minimizing this loss yields the maximum\-likelihood estimate ofΘ\\Theta\. The key advantage of CONDOR is its universality: it guarantees rank\-consistent outputs regardless of the upstream architecture, provided the final layer follows the CONDOR formulation\. This makes it broadly applicable and theoretically robust compared with methods such as CORAL that require weight sharing\.

### 4\.5Train\-test protocols

To measure the generalization capabilities of the models, we need realistic evaluation protocols\. A commonly used random split does not account for the structure or temporal dependencies in TTRPG data, as it does not capture the fact that monster designers often base new creations on existing ones\. As a result, random splitting can introduce data leakage, where information from similar or related instances appears in both the training and test sets\. This leads to an overly optimistic estimate of generalization performance, which undermines the reliability of the evaluation\.

Instead, this setting more closely resembles time series data, where the newest samples should be used for evaluation\. We therefore propose achronological split, which assigns older data to the training set and the newest monsters to the test set, based on the publication date of each book\. The oldest monsters were published on 01\.08\.2019 and the newest ones included were published on 26\.04\.2026\. The test set constitutes approximately20%20\\%of the data, and the oldest monsters in it were published on 01\.08\.2025\. This approach reflects the natural emergence of new data over time and more accurately models how game designers would use the resulting models\. A single chronological train\-test split, however, assumes that we are interested in long\-term predictions over large batches of monsters, since the test set covers samples far into the future relative to the training set\.

We further propose a more refined strategy based on the domain’s characteristics\. Pathfinder monsters are naturally partitioned into chronologically ordered subsets, namely the books that contain them\. During the design process, game designers focus on the next book or the few books scheduled for publication in the near future\. Once those are published, the data can be added to the training set and the ML models retrained\. This is equivalent to theexpanding windowevaluation strategy from the time series forecasting literature\. We first train on the initial portion of the data, test on the next batch, then add that batch to the training data, retrain the model, and repeat until all available samples have been used\. The reported score is the average metric over all test sets\. Hyperparameters are tuned separately each time using cross\-validation on the training data\. While this procedure has a considerable computational cost, it yields a robust evaluation, reflecting real\-world practice\.

In our case, the first subset includes all monsters published at the premiere of Pathfinder 2e: 7 sourcebooks with 232 monsters\. We then test on subsequent books, ordered by publication date, ensuring that each test set contains at least 100 monsters, which lowers the variance of the estimate\. To ensure temporal consistency, all monsters published on the same day were always assigned to the same subset\. We visualize this in Figure[3](https://arxiv.org/html/2607.09196#S4.F3)\.

![Refer to caption](https://arxiv.org/html/2607.09196v1/img/expanding_window_train_test.png)Figure 3:Expanding window evaluation\.
### 4\.6Evaluation metrics

The ordinal regression task combines aspects of both classification and standard regression\. We therefore use a combination of metrics from both domains to comprehensively assess model performance, following best practices recommended in prior work\[[5](https://arxiv.org/html/2607.09196#bib.bib54)\]\. We keep the notation of the previous subsections:yiy\_\{i\}is the true level of sampleii,y^i\\hat\{y\}\_\{i\}the prediction,KKthe number of classes, andNkN\_\{k\}the number of test samples of true classkk\.

To capture the magnitude of prediction errors, we use the standard RMSE and MAE\. Because the dataset is imbalanced, with far more monsters at low levels than at high levels, these plain averages may not reflect prediction quality well\. We therefore also use their macro\-averaged versionsMAEM\\mathrm\{MAE\}^\{M\}andRMSEM\\mathrm\{RMSE\}^\{M\}\[[5](https://arxiv.org/html/2607.09196#bib.bib54)\], which have been proposed specifically for imbalanced ordinal regression\. They compute the metric separately for each class and then average over classes \(Equations[28](https://arxiv.org/html/2607.09196#S4.E28)and[29](https://arxiv.org/html/2607.09196#S4.E29)\)\.

RMSEM=1K​∑k=1K1Nk​∑i:yi=k\(yi−y^i\)2\\mathrm\{RMSE\}^\{M\}=\\frac\{1\}\{K\}\\sum\_\{k=1\}^\{K\}\\sqrt\{\\frac\{1\}\{N\_\{k\}\}\\sum\_\{i\\,:\\,y\_\{i\}=k\}\\left\(y\_\{i\}\-\\hat\{y\}\_\{i\}\\right\)^\{2\}\}\(28\)
MAEM=1K​∑k=1K1Nk​∑i:yi=k\|yi−y^i\|\\mathrm\{MAE\}^\{M\}=\\frac\{1\}\{K\}\\sum\_\{k=1\}^\{K\}\\frac\{1\}\{N\_\{k\}\}\\sum\_\{i\\,:\\,y\_\{i\}=k\}\\left\|y\_\{i\}\-\\hat\{y\}\_\{i\}\\right\|\(29\)
Classification metrics assess the precision of the label predictions\. Plain accuracy \(a​c​cacc\) measures how often the prediction exactly matches the true label\. It ignores error magnitude and is a harsh metric in ordinal regression\. To contextualize performance, we also use accuracy\-at\-1 \(a​c​c​@​1acc@1\), which counts a prediction one level above or below the ground truth as correct\. It is always at least as large as plain accuracy\.

To further assess the ordinal agreement between predicted and true labels, we use Somers’ D rank correlation coefficient\[[49](https://arxiv.org/html/2607.09196#bib.bib55)\]\. It evaluates how often the predicted ordering of a pair of samples agrees with the ordering of their true labels: a pair is concordant if the two orderings match and discordant otherwise\. LetCCandDDbe the numbers of concordant and discordant pairs, andTTthe number of pairs tied in the predicted scores\. Somers’ D is given in Equation[30](https://arxiv.org/html/2607.09196#S4.E30); it lies in\[−1,1\]\[\-1,1\], with11indicating perfect agreement,0no ordinal association, and−1\-1perfectly reversed ordering\.

Somers’​D​\(y,y^\)=C−DC\+D\+T\\text\{Somers' \}D\(y,\\hat\{y\}\)=\\frac\{C\-D\}\{C\+D\+T\}\(30\)

## 5Experiments and results

In this section, we present the experiments, results, and discussion\. We start by analyzing the dataset and feature sets, and then validate the proposed evaluation scheme, namely chronological split and random split\. Then, we present experiments with different rounding schemes, and finally a full comparison of regression with rounding, ordinal regression, and ordinal neural network models\. Lastly, we present explainable AI analyses for the best performing model, as well as selected use cases\.

### 5\.1Experimental setup

As a first step, the features were normalized to the\[0,1\]\[0,1\]range using min\-max scaling on the whole dataset\. Unless stated otherwise, each experiment uses the chronological split\.

In every experiment, hyperparameter optimization was performed using 5\-fold cross\-validation with MAE as the optimization criterion\. Grid search was used for all models except LightGBM, which used the Tree\-structured Parzen Estimator \(TPE\) due to its large number of hyperparameters\. See Appendix[C](https://arxiv.org/html/2607.09196#A3)for hyperparameter grids\.

Neural network models were trained with the AdamW optimizer for 40 epochs\. Initial experiments indicated that this is enough for both training and validation losses to converge\.

### 5\.2Dataset analysis

For initial data inspection and visualization, we use principal component analysis \(PCA\) and t\-SNE \(default hyperparameters\) using Full set of features, see Figure[4](https://arxiv.org/html/2607.09196#S5.F4)\. Points have been colored based on their level, with lighter points indicating higher\-level monsters\. The dataset clearly shows a smooth transition of monster power, indicating well\-engineered feature space\. This also means that ML methods in the original space should have relatively good performance, as the feature\-label relation should not be overly complex\.

![Refer to caption](https://arxiv.org/html/2607.09196v1/x2.png)Figure 4:Dataset visualization after dimensionality reduction\.Subsequently, we computed the Pearson correlation coefficients among all features using the Full feature set, see Figure[5](https://arxiv.org/html/2607.09196#S5.F5)\. The resulting correlation matrix reveals several clusters of strongly intercorrelated variables\. As anticipated, one such cluster consists of features characterizing essential spell\-related attributes \(e\.g\., spell attack bonus, maximum spell level, and spell save DC\), which also exhibit strong correlations with more granular magic\-related features, such as the number of spells per level\. An additional set of highly correlated features is associated with defensive capabilities, including armor class \(AC\), hit points \(HP\), saving throws, and perception\.

![Refer to caption](https://arxiv.org/html/2607.09196v1/x3.png)Figure 5:Pearson correlation between features\.In order to evaluate the usefulness of the proposed chronological split, we compare its results with random split on selected models\. Interestingly, results in Figure[6](https://arxiv.org/html/2607.09196#S5.F6)show that random split often results in more pessimistic evaluation, i\.e\., higher macroaveraged MAE\. This can be explained by the fact that designers often base new monsters on existing ones, which may may place similar samples in both the train and test sets\. However, the expanding window strategy results in much more pessimistic evaluation in all cases, which may be caused by data distribution shifts, for example\. As the chronological split and expanding window are more aligned with the design process, we focus on them in subsequent experiments\.

![Refer to caption](https://arxiv.org/html/2607.09196v1/x4.png)Figure 6:Performance of different evaluation strategies\.Next, we evaluate the gain from incorporating more features, with results summarized in Figure[7](https://arxiv.org/html/2607.09196#S5.F7)\. Adding more features generally results in a performance gain, particularly going from basic to expanded feature set\. In particular, the best result with Random Forest has been obtained with the full feature set\. This positively validates our choices in terms of feature engineering, and all subsequent experiments use the Full set of features\.

![Refer to caption](https://arxiv.org/html/2607.09196v1/x5.png)Figure 7:Performance of different feature sets\.
### 5\.3Models performance comparison

We present primary results of ML models for chronological split in Table[2](https://arxiv.org/html/2607.09196#S5.T2), and for expanding window in Table[3](https://arxiv.org/html/2607.09196#S5.T3)\. In all cases, we highlight the best model in bold, and second best in italics based onM​A​EMMAE^\{M\}\. Based on those results and observations, we can draw several conclusions\. They remain consistent to both chronological split and expanding window evaluation strategies\.

Conclusion 1: ML outperforms human\-inspired baselines\.We included kNN models as baselines to model the human design process, which is based on comparison with existing monsters, as also recommended by the Pathfinder books\. Their results are markedly inferior to those of all ML models, including even simple ridge regression\. This difference is evident across all metrics, particularly in accuracy\. This indicates the strong potential of ML to help human designers, and particularly amateur GMs, to create home\-made monsters\.

Conclusion 2: Monster level prediction requires strong non\-linear models\.The four best\-performing models are tree\-based ensembles: RF, LightGBM, ORF, and SAOC \(with an underlying Random Forest\)\. They strongly outperform linear models such as ridge regression, ORD, and Immediate\-Threshold \(IT\)\. This indicates that monster level prediction is highly non\-linear and requires sufficiently expressive models\. Tree\-based models also outperform the other non\-linear models, including kernel SVM, GPOR, and neural networks\. This can be explained by the limited data availability, which favors robust models that reduce error through ensembling\.

Conclusion 3: Neural networks for ordinal regression are not the best models for tabular data\.While all neural networks outperformed the baseline, none of those models matched the tree\-based ensembles\. They also showed higher standard deviations under expanding\-window evaluation compared with other models, indicating relatively low stability and high variance\. This can be attributed to the relatively small amount of data and aligns with the literature, which shows the superiority of tree\-based ensembles over MLPs for tabular data in most cases\. This holds despite the ordinal\-regression\-specific inductive biases incorporated into these neural models\.

Conclusion 4: Top models are strong and reliable\.The best\-performing models \- Random Forest, LightGBM, SAOC and ORF \- achieved near\-perfect Somers’ D and acc@1\. This means that they can almost always properly sort monsters by predicted power and their predictions nearly always lie within just 1 level of ground truth\. Further, very lowM​A​EMMAE^\{M\},R​M​S​EMRMSE^\{M\}, and high accuracy indicate that predictions are exactly equal to the ground truth most of the time, and very close otherwise\. The standard deviation of these models in case of expanding window is also relatively low across all metrics compared with the other models, meaning that they are stable and robust\.

While there is still room for improvement, particularly in accuracy andR​M​S​EMRMSE^\{M\}, the created ML models are reliable tools for Pathfinder monster level prediction\. The results achieved are good enough to warrant considering their deployment in practical applications supporting TTRPG game design\.

Model𝐌𝐀𝐄𝐌\\mathbf\{MAE\}^\{\\mathbf\{M\}\}𝐑𝐌𝐒𝐄𝐌\\mathbf\{RMSE\}^\{\\mathbf\{M\}\}Somers’ DAccAcc@1kNN0\.520\.860\.9659%94%Ridge0\.380\.660\.9769%98%SVM0\.280\.590\.9878%97%Random Forest0\.130\.390\.9988%99%LightGBM0\.140\.400\.9987%99%SAOC0\.160\.440\.9887%99%ORD0\.350\.630\.9772%98%IT0\.310\.600\.9775%98%ORF0\.160\.450\.9887%99%GPOR0\.330\.620\.9773%98%CLM0\.440\.720\.9766%97%NNRank0\.450\.730\.9862%97%OR\-CNN0\.410\.700\.9767%98%CORAL0\.270\.550\.9878%98%CORN0\.330\.610\.9772%98%CONDOR0\.360\.630\.9772%99%Table 2:Chronological split results\. The best result marked in bold, second best in italics\.Model𝐌𝐀𝐄𝐌\\mathbf\{MAE\}^\{\\mathbf\{M\}\}𝐑𝐌𝐒𝐄𝐌\\mathbf\{RMSE\}^\{\\mathbf\{M\}\}Somers’ DAccAcc@1kNN0\.63±\\pm0\.201\.01±\\pm0\.250\.94±\\pm0\.0154±\\pm8%91±\\pm5%Ridge0\.42±\\pm0\.110\.77±\\pm0\.180\.96±\\pm0\.0168±\\pm5%97±\\pm2%SVM0\.35±\\pm0\.140\.69±\\pm0\.230\.97±\\pm0\.0174±\\pm7%97±\\pm2%Random Forest0\.25±\\pm0\.120\.57±\\pm0\.180\.97±\\pm0\.0180±\\pm7%98±\\pm2%LightGBM0\.27±\\pm0\.120\.59±\\pm0\.180\.97±\\pm0\.0179±\\pm7%98±\\pm1%SAOC0\.24±\\bm\{\\pm\}0\.090\.55±\\bm\{\\pm\}0\.110\.97±\\pm0\.0181±\\bm\{\\pm\}7%98±\\pm2%ORD0\.37±\\pm0\.100\.71±\\pm0\.150\.96±\\pm0\.0171±\\pm5%97±\\pm2%IT0\.34±\\pm0\.100\.68±\\pm0\.150\.96±\\pm0\.0173±\\pm4%97±\\pm2%ORF0\.26±\\pm0\.110\.56±\\pm0\.150\.97±\\pm0\.0179±\\pm8%98±\\pm2%GPOR0\.36±\\pm0\.130\.70±\\pm0\.210\.96±\\pm0\.0172±\\pm5%97±\\pm2%CLM0\.77±\\pm1\.071\.18±\\pm1\.270\.91±\\pm0\.1456±\\pm15%90±\\pm18%NNRank0\.54±\\pm0\.250\.89±\\pm0\.310\.96±\\pm0\.0161±\\pm13%94±\\pm6%OR\-CNN0\.52±\\pm0\.200\.87±\\pm0\.270\.96±\\pm0\.0164±\\pm9%94±\\pm5%CORAL0\.48±\\pm0\.250\.83±\\pm0\.310\.96±\\pm0\.0166±\\pm10%95±\\pm5%CORN0\.50±\\pm0\.240\.86±\\pm0\.290\.96±\\pm0\.0166±\\pm11%95±\\pm6%CONDOR0\.62±\\pm0\.341\.00±\\pm0\.410\.95±\\pm0\.0260±\\pm14%92±\\pm8%Table 3:Expanding window results\. Mean and standard deviation over all test windows are reported\. The best result \(mean value\) marked in bold, second best in italics\.See Appendix[A](https://arxiv.org/html/2607.09196#A1)for comparison of regular and macroaveraged metrics\. In Appendix[B](https://arxiv.org/html/2607.09196#A2), we also provide the detailed results for all testing windows under expanding window evaluation for the Random Forest\.

### 5\.4Best model analysis

The results in the previous section show that the models perform well on average\. Here, we analyze the top\-performing model, Random Forest, in more depth to inspect its behavior\. We examine its error distribution, the relationship between error and out\-of\-distribution data, feature importances, and a few illustrative case studies\. We use the chronological split throughout this section\.

Figure[8](https://arxiv.org/html/2607.09196#S5.F8)shows the distribution of prediction signed errors, i\.e\.,y−y^y\-\\hat\{y\}, with logarithmic scale for readability\. Predicting11or22levels too high \(prediction error−1\-1or−2\-2\) is slightly more common, but this aligns with the imbalanced labels distribution in the dataset \(see Figure[2](https://arxiv.org/html/2607.09196#S3.F2)\)\. Most of the time, the model makes no mistake at all, or at most a single level, with just under2020cases having an error of22or33levels\. As such, we can trust its predictions during the monster design process\.

![Refer to caption](https://arxiv.org/html/2607.09196v1/x6.png)Figure 8:Errors histogram for Random Forest, logarithmic scale\.The confusion matrix in Figure[9](https://arxiv.org/html/2607.09196#S5.F9)confirms this: nearly all errors fall within a single level of the ground truth, and no systematic misclassification pattern emerges\. Cases where the model makes a mistake may correspond to atypical monsters, for example those whose difficulty stems from unique special abilities rather than from their core statistics\. Expanding the model to incorporate them is left as future work\.

![Refer to caption](https://arxiv.org/html/2607.09196v1/x7.png)Figure 9:Confusion matrix for Random Forest\.Figure[10](https://arxiv.org/html/2607.09196#S5.F10)plots each test monster’s absolute error against its cosine distance to the nearest training example\. We observe no clear relationship between the two, and almost all distances are small\. This suggests that new monsters rarely fall far from the training distribution and that the model is therefore not exposed to strongly out\-of\-distribution inputs\.

![Refer to caption](https://arxiv.org/html/2607.09196v1/x8.png)Figure 10:Distance to training data vs prediction error\.The analysis of feature importances, based on mean absolute SHAP values\[[30](https://arxiv.org/html/2607.09196#bib.bib56),[15](https://arxiv.org/html/2607.09196#bib.bib57)\], is presented in Figure[11](https://arxiv.org/html/2607.09196#S5.F11)\. It indicates that the most influential predictors are primarily defensive statistics: Armor Class \(AC\), Hit Points \(HP\), Perception, and saving throws \(Will, Fortitude, Reflex\)\. Collectively, these variables capture a creature’s ability to avoid or withstand damage, to detect threats early, and to influence the initiative order in combat\.

Among offensive capabilities, the most important features are associated with melee combat, specifically the melee attack bonus and average melee damage\. This aligns with the fact that melee is the most frequently used attack modality in the dataset\. Spell\-related statistics rank next in importance, reflecting their role as an alternative but still substantial source of offensive potential for magic\-oriented creatures\.

A notable finding is that features related to ranged attacks have comparatively low importance\. They do not appear among top 15 most influential features\. This also aligns with the fact that they are less common than melee and spell abilities among monsters in the dataset\.

![Refer to caption](https://arxiv.org/html/2607.09196v1/x9.png)Figure 11:Mean absolute SHAP feature importances, top 15 features\.We conducted a case analysis of monsters with high prediction error in the test set\. Most such cases correspond to entities designed for highly specific scenarios, frequently implemented as modified variants of well\-known monsters\. They possess the “Unique” tag and are highly customized for specific use cases, making them naturally challenging for ML\. Another subset consists of creatures that are not independent monsters in the strict sense, but rather derivative or semi\-autonomous creatures generated by another monster during combat\. Although these derivatives are typically assigned the same difficulty level as their progenitor, they possess reduced statistics, which in practice leads to less challenging encounters\. The only regular monster with a high prediction error is the Snowy Owl, whose statistics are more consistent with lower\-level monsters\. Its sole distinctive feature is flight, implemented as a special ability rather than as an inherent flying speed, which is not captured by the model\. Extending featurization to include special abilities is left as future work\.

In addition, we analyzed the model’s performance for two common monster families, well represented both in training and test data: Trolls and Dragons\. For the Troll family, the model showed no substantial difficulty in predicting the correct levels\. The distance between the true class and the continuous output of the RF model tended to be slightly higher for cross\-breed variants such as the Trollhund \(a Troll\-dog hybrid\), yet the predictions remained close to the correct label, with an absolute error below0\.40\.4\. Note that this family is represented in the test set by monsters with levels between33and1515\.

The Dragon family is larger and predominantly composed of higher\-level monsters\. Individual dragons are typically represented by three age\-based variants \(Young, Adult, Ancient\) and two role\-based variants \(standard and spellcaster\)\. Most levels within this family were predicted correctly\. We did not observe a consistent pattern in which the prediction error increases with the level \(or age\) of a given variant\. No misclassifications occurred for Young dragons\. In a few isolated instances an incorrect level was assigned\. However, a misprediction for an Adult variant did not lead to an incorrect prediction for its corresponding Ancient variant\. For the standard versus spellcaster forms, errors either occurred for both variants simultaneously or for neither\.

## 6Conclusions

This work investigated the application of machine learning methods to support game design in tabletop role\-playing games, focusing on the prediction of monster levels in the Pathfinder Second Edition system\. We created an openly available dataset, formalized the problem as ordinal regression, and validated our feature engineering process through data analysis\. We conducted a comprehensive experimental study comparing a wide range of models, including classical regression methods with several rounding schemes, tabular ordinal regression models, and neural networks with appropriate inductive biases\. The models were evaluated using multiple metrics and validation strategies, including chronological train\-test splits and expanding\-window evaluation, to ensure a robust and realistic assessment\.

The results demonstrate that machine learning methods substantially outperform human\-inspired baselines\. This confirms the strong potential of data\-driven approaches to assist both professional designers and amateur game masters in creating balanced monsters\. Among the evaluated models, tree\-based ensemble methods, particularly Random Forest, LightGBM, ORF, and SAOC, achieved the best overall performance\. These models effectively capture the highly non\-linear relationships present in the data and consistently produced accurate and stable predictions\. In contrast, neural network approaches, including architectures specifically designed for ordinal regression, did not outperform simpler models and showed higher variance, which is likely related to the limited size and tabular nature of the dataset\.

The best\-performing Random Forest model achieved near\-perfect ranking quality and high accuracy, with prediction errors typically within one level of the true value\. Deeper error analysis further supports the robustness of the proposed solution, and the resulting models align with human intuition regarding which features are most relevant to monster level prediction\. These results indicate that machine learning models can serve as reliable tools for assisting the design of monsters in the Pathfinder 2e system and, potentially, in similar rule\-based games\.

We plan to extend this research in several directions\. First, designers may be interested not only in predicting a monster’s level from its statistics, but also in actionable insights into how to change those statistics to better match a target power\. We plan to use counterfactual generation methods to propose minimal changes to a monster’s statistics that bring it to the desired level, providing data\-driven expert\-system support for game design\. Second, since the target users are game designers without a technical background, we plan to conduct user studies to practically evaluate explainable AI \(XAI\) methods for this problem, in order to understand the user experience \(UX\) aspects of post\-hoc explanations of ordinal regression models\.

## References

- \[1\]M\. M\. Al\-Surayhi, M\. R\. Al\-Qahtani, S\. N\. Al\-Marshadi, A\. A\. Q\. Al\-Ghamdi, and M\. M\. Al\-Surayhi\(2025\)Generative artificial intelligence in game design: a narrative review\.Journal of Ecohumanism4\(4\),pp\. 1702–1712\.Cited by:[§2](https://arxiv.org/html/2607.09196#S2.p4.1)\.
- \[2\]Archives of NethysBig Boss Goblin\(Website\)External Links:[Link](https://2e.aonprd.com/NPCs.aspx?ID=3643)Cited by:[Figure 1](https://arxiv.org/html/2607.09196#S3.F1),[Figure 1](https://arxiv.org/html/2607.09196#S3.F1.3.2)\.
- \[3\]Archives of Nethys\(2026\)Monsters — Archives of Nethys: Pathfinder 2nd Edition Database\.Note:[https://2e\.aonprd\.com/Monsters\.aspx](https://2e.aonprd.com/Monsters.aspx)Published by Rose\-Winds LLC \(Blake Davis\)\. Accessed 2026\-07\-01Cited by:[§3\.1](https://arxiv.org/html/2607.09196#S3.SS1.p3.1)\.
- \[4\]T\. Ashby, B\. K\. Webb, G\. Knapp, J\. Searle, and N\. Fulda\(2023\)Personalized quest and dialogue generation in role\-playing games: a knowledge graph\- and language model\-based approach\.InProceedings of the 2023 CHI Conference on Human Factors in Computing Systems,pp\. 1–20\.Cited by:[§2](https://arxiv.org/html/2607.09196#S2.p2.1)\.
- \[5\]S\. Baccianella, A\. Esuli, and F\. Sebastiani\(2009\)Evaluation measures for ordinal regression\.In2009 Ninth International Conference on Intelligent Systems Design and Applications \(ISDA\),pp\. 283–287\.Cited by:[§4\.6](https://arxiv.org/html/2607.09196#S4.SS6.p1.6),[§4\.6](https://arxiv.org/html/2607.09196#S4.SS6.p2.2)\.
- \[6\]R\. Bender and U\. Grouven\(1997\)Ordinal logistic regression in medical research\.Journal of the Royal College of Physicians of London31\(5\),pp\. 546–551\.Cited by:[§2](https://arxiv.org/html/2607.09196#S2.p6.1),[§4\.3\.2](https://arxiv.org/html/2607.09196#S4.SS3.SSS2.p1.3)\.
- \[7\]L\. Breiman\(2001\)Random forests\.Machine Learning45\(1\),pp\. 5–32\.Cited by:[§4\.2](https://arxiv.org/html/2607.09196#S4.SS2.p1.1)\.
- \[8\]P\. Bürkner and M\. Vuorre\(2019\)Ordinal regression models in psychology: a tutorial\.Advances in Methods and Practices in Psychological Science2\(1\),pp\. 77–101\.Cited by:[§2](https://arxiv.org/html/2607.09196#S2.p5.1)\.
- \[9\]Business Research Insights\(2025\)Tabletop Role\-Playing Game \(TTRPG\) Market Size, Share, Growth, and Industry Analysis, Regional Insights and Forecast From 2025 to 2033\(Website\)External Links:[Link](https://www.businessresearchinsights.com/market-reports/tabletop-role-playing-game-ttrpg-market-110856)Cited by:[§1](https://arxiv.org/html/2607.09196#S1.p3.2)\.
- \[10\]W\. Cao, V\. Mirjalili, and S\. Raschka\(2020\)Rank consistent ordinal regression for neural networks with application to age estimation\.Pattern Recognition Letters140,pp\. 325–331\.External Links:ISSN 0167\-8655,[Document](https://dx.doi.org/10.1016/j.patrec.2020.11.008)Cited by:[§2](https://arxiv.org/html/2607.09196#S2.p6.1),[§4\.4\.4](https://arxiv.org/html/2607.09196#S4.SS4.SSS4.p1.4)\.
- \[11\]J\. Cheng, Z\. Wang, and G\. Pollastri\(2008\)A neural network approach to ordinal regression\.In2008 IEEE International Joint Conference on Neural Networks \(IJCNN\),pp\. 1279–1284\.Cited by:[§4\.4\.2](https://arxiv.org/html/2607.09196#S4.SS4.SSS2.p1.4),[§4\.4\.2](https://arxiv.org/html/2607.09196#S4.SS4.SSS2.p3.3)\.
- \[12\]W\. Chu and Z\. Ghahramani\(2005\)Gaussian processes for ordinal regression\.Journal of Machine Learning Research6,pp\. 1019–1041\.Cited by:[§2](https://arxiv.org/html/2607.09196#S2.p6.1),[§4\.3\.5](https://arxiv.org/html/2607.09196#S4.SS3.SSS5.p1.1)\.
- \[13\]C\. Cortes and V\. Vapnik\(1995\)Support\-vector networks\.Machine Learning20\(3\),pp\. 273–297\.Cited by:[§4\.2](https://arxiv.org/html/2607.09196#S4.SS2.p1.1)\.
- \[14\]R\. Coulom\(2006\)Efficient selectivity and backup operators in Monte\-Carlo tree search\.InInternational Conference on Computers and Games \(CG 2006\),pp\. 72–83\.Cited by:[§1](https://arxiv.org/html/2607.09196#S1.p1.1)\.
- \[15\]I\. Covert, S\. M\. Lundberg, and S\. Lee\(2020\)Understanding global feature contributions with additive importance measures\.Advances in Neural Information Processing Systems33,pp\. 17212–17223\.Cited by:[§5\.4](https://arxiv.org/html/2607.09196#S5.SS4.p5.1)\.
- \[16\]A\. Drachen, M\. Hitchens, A\. Jhala, and G\. Yannakakis\(2009\)Towards data\-driven drama management: issues in data collection and annotation\.InProceedings of the 2009 DiGRA International Conference: Breaking New Ground: Innovation in Games, Play, Practice and Theory,Cited by:[§2](https://arxiv.org/html/2607.09196#S2.p2.1)\.
- \[17\]G\. Edwards, N\. Subianto, D\. Englund, J\. W\. Goh, N\. Coughran, Z\. Milton, N\. Mirnateghi, and S\. A\. A\. Shah\(2021\)The role of machine learning in game development domain: a review of current trends and future directions\.In2021 Digital Image Computing: Techniques and Applications \(DICTA\),pp\. 1–7\.Cited by:[§2](https://arxiv.org/html/2607.09196#S2.p4.1)\.
- \[18\]E\. Frank and M\. Hall\(2001\)A simple approach to ordinal classification\.InMachine Learning: ECML 2001,pp\. 145–156\.Cited by:[§4\.3\.1](https://arxiv.org/html/2607.09196#S4.SS3.SSS1.p1.8)\.
- \[19\]M\. Guzdial and M\. Riedl\(2018\)Automated game design via conceptual expansion\.InProceedings of the AAAI Conference on Artificial Intelligence and Interactive Digital Entertainment \(AIIDE\),Vol\.14,pp\. 31–37\.Cited by:[§2](https://arxiv.org/html/2607.09196#S2.p2.1)\.
- \[20\]D\. Hooshyar, M\. Yousefi, and H\. Lim\(2018\)Data\-driven approaches to game player modeling: a systematic literature review\.ACM Computing Surveys \(CSUR\)50\(6\),pp\. 1–19\.Cited by:[§2](https://arxiv.org/html/2607.09196#S2.p4.1)\.
- \[21\]M\. M\. Hossan, M\. M\. Fouda, and F\. Z\. Eishita\(2024\)Adaptive game design using machine learning techniques: a survey\.In2024 IEEE International Conference on Internet of Things and Intelligence Systems \(IoTaIS\),pp\. 144–150\.Cited by:[§2](https://arxiv.org/html/2607.09196#S2.p3.1)\.
- \[22\]J\. Hu and P\. Li\(2018\)Collaborative filtering via additive ordinal regression\.InProceedings of the Eleventh ACM International Conference on Web Search and Data Mining \(WSDM\),pp\. 243–251\.Cited by:[§2](https://arxiv.org/html/2607.09196#S2.p5.1)\.
- \[23\]Z\. Hu, Y\. Ding, R\. Wu, L\. Li, R\. Zhang, Y\. Hu, F\. Qiu, Z\. Zhang, K\. Wang, S\. Zhao,et al\.\(2023\)Deep learning applications in games: a survey from a data perspective\.Applied Intelligence53\(24\),pp\. 31129–31164\.Cited by:[§2](https://arxiv.org/html/2607.09196#S2.p4.1)\.
- \[24\]G\. Jenkinson, G\. R\. Oliver, K\. Khezeli, J\. Kalantari, and E\. W\. Klee\(2021\)Universally rank consistent ordinal regression in neural networks\.Note:arXiv preprint arXiv:2110\.07470Cited by:[§2](https://arxiv.org/html/2607.09196#S2.p6.1),[§4\.4\.6](https://arxiv.org/html/2607.09196#S4.SS4.SSS6.p1.4)\.
- \[25\]G\. Ke, Q\. Meng, T\. Finley, T\. Wang, W\. Chen, W\. Ma, Q\. Ye, and T\. Liu\(2017\)LightGBM: a highly efficient gradient boosting decision tree\.Advances in Neural Information Processing Systems30\.Cited by:[§4\.2](https://arxiv.org/html/2607.09196#S4.SS2.p1.1)\.
- \[26\]S\. Kramer, G\. Widmer, B\. Pfahringer, and M\. De Groeve\(2001\)Prediction of ordinal classes using regression trees\.Fundamenta Informaticae47\(1\-2\),pp\. 1–13\.Cited by:[§2](https://arxiv.org/html/2607.09196#S2.p6.1)\.
- \[27\]M\. Lechner and G\. Okasa\(2019\)ORF: Ordered Random Forests\.Note:R package version 0\.1\.3Cited by:[§2](https://arxiv.org/html/2607.09196#S2.p6.1),[§4\.3\.4](https://arxiv.org/html/2607.09196#S4.SS3.SSS4.p1.4),[§4\.3](https://arxiv.org/html/2607.09196#S4.SS3.p1.1)\.
- \[28\]M\. Lechner and G\. Okasa\(2025\)Random forest estimation of the ordered choice model\.Empirical Economics68\(1\),pp\. 1–106\.Cited by:[§4\.3\.4](https://arxiv.org/html/2607.09196#S4.SS3.SSS4.p1.4)\.
- \[29\]J\. Liu, S\. Snodgrass, A\. Khalifa, S\. Risi, G\. N\. Yannakakis, and J\. Togelius\(2021\)Deep learning for procedural content generation\.Neural Computing and Applications33\(1\),pp\. 19–37\.Cited by:[§2](https://arxiv.org/html/2607.09196#S2.p2.1)\.
- \[30\]S\. M\. Lundberg and S\. Lee\(2017\)A unified approach to interpreting model predictions\.Advances in Neural Information Processing Systems30\.Cited by:[§5\.4](https://arxiv.org/html/2607.09196#S5.SS4.p5.1)\.
- \[31\]A\. G\. d\. G\. Matthews, M\. van der Wilk, T\. Nickson, K\. Fujii, A\. Boukouvalas, P\. León\-Villagrá, Z\. Ghahramani, and J\. Hensman\(2017\)GPflow: a Gaussian process library using TensorFlow\.Journal of Machine Learning Research18\(40\),pp\. 1–6\.Cited by:[§4\.3\.5](https://arxiv.org/html/2607.09196#S4.SS3.SSS5.p6.1)\.
- \[32\]P\. McCullagh\(1980\)Regression models for ordinal data\.Journal of the Royal Statistical Society: Series B \(Methodological\)42\(2\),pp\. 109–127\.Cited by:[§4\.4\.1](https://arxiv.org/html/2607.09196#S4.SS4.SSS1.p1.2)\.
- \[33\]Z\. Niu, M\. Zhou, L\. Wang, X\. Gao, and G\. Hua\(2016\)Ordinal regression with multiple output CNN for age estimation\.InProceedings of the IEEE Conference on Computer Vision and Pattern Recognition \(CVPR\),pp\. 4920–4928\.Cited by:[§2](https://arxiv.org/html/2607.09196#S2.p5.1),[§4\.4\.3](https://arxiv.org/html/2607.09196#S4.SS4.SSS3.p1.3),[§4\.4\.3](https://arxiv.org/html/2607.09196#S4.SS4.SSS3.p5.1)\.
- \[34\]Paizo Inc\.Paizo Inc\. Pathfinder® Second Edition Compatibility License\(Website\)External Links:[Link](https://paizo.com/pathfinder/compatibility)Cited by:[§3\.1](https://arxiv.org/html/2607.09196#S3.SS1.p1.1)\.
- \[35\]Paizo Inc\.Pathfinder Infinite and the ORC License\(Website\)External Links:[Link](https://paizo.com/community/blog/v5748dyo6sig4?Pathfinder-Infinite-and-the-ORC-License)Cited by:[§3\.1](https://arxiv.org/html/2607.09196#S3.SS1.p1.1)\.
- \[36\]F\. Pedregosa, F\. Bach, and A\. Gramfort\(2017\)On the consistency of ordinal regression methods\.Journal of Machine Learning Research18\(55\),pp\. 1–35\.External Links:[Link](http://jmlr.org/papers/v18/15-495.html)Cited by:[§4\.3\.3](https://arxiv.org/html/2607.09196#S4.SS3.SSS3.p2.1),[§4\.4\.1](https://arxiv.org/html/2607.09196#S4.SS4.SSS1.p1.2),[§4\.4\.1](https://arxiv.org/html/2607.09196#S4.SS4.SSS1.p7.2)\.
- \[37\]D\. Plans and D\. Morelli\(2012\)Experience\-driven procedural music generation for games\.IEEE Transactions on Computational Intelligence and AI in Games4\(3\),pp\. 192–198\.Cited by:[§2](https://arxiv.org/html/2607.09196#S2.p2.1)\.
- \[38\]C\. Plut and P\. Pasquier\(2020\)Generative music in video games: state of the art, challenges, and prospects\.Entertainment Computing33,pp\. 100337\.Cited by:[§2](https://arxiv.org/html/2607.09196#S2.p2.1)\.
- \[39\]P\. Probst, M\. N\. Wright, and A\. Boulesteix\(2019\)Hyperparameters and tuning strategies for random forest\.WIREs Data Mining and Knowledge Discovery9\(3\),pp\. e1301\.Cited by:[§4\.3\.1](https://arxiv.org/html/2607.09196#S4.SS3.SSS1.p7.1)\.
- \[40\]G\. K\. Rajbahadur, S\. Wang, Y\. Kamei, and A\. E\. Hassan\(2017\)The impact of using regression models to build defect classifiers\.In2017 IEEE/ACM 14th International Conference on Mining Software Repositories \(MSR\),pp\. 135–145\.External Links:[Document](https://dx.doi.org/10.1109/MSR.2017.4)Cited by:[§4\.2](https://arxiv.org/html/2607.09196#S4.SS2.p3.1)\.
- \[41\]J\. D\. M\. Rennie and N\. Srebro\(2005\)Loss functions for preference levels: regression with discrete ordered labels\.InProceedings of the IJCAI Multidisciplinary Workshop on Advances in Preference Handling,Vol\.1,pp\. 1–6\.Cited by:[§4\.3\.3](https://arxiv.org/html/2607.09196#S4.SS3.SSS3.p1.9)\.
- \[42\]S\. J\. Russell and P\. Norvig\(2016\)Artificial Intelligence: A Modern Approach\.Third edition,Pearson Education,Harlow, England\.External Links:ISBN 9781292153964Cited by:[§1](https://arxiv.org/html/2607.09196#S1.p1.1)\.
- \[43\]M\. Sakura, K\. Tasaka, M\. Numao, and S\. Nishide\(2025\)NPC dialogue generation in a custom\-built game using generative AI\.In2025 IEEE 14th Global Conference on Consumer Electronics \(GCCE\),pp\. 416–420\.Cited by:[§2](https://arxiv.org/html/2607.09196#S2.p2.1)\.
- \[44\]A\. Sarkar and S\. Cooper\(2020\)Towards game design via creative machine learning \(GDCML\)\.In2020 IEEE Conference on Games \(CoG\),pp\. 744–751\.Cited by:[§2](https://arxiv.org/html/2607.09196#S2.p2.1)\.
- \[45\]X\. Shi, W\. Cao, and S\. Raschka\(2023\)Deep neural networks for rank\-consistent ordinal regression based on conditional probabilities\.Pattern Analysis and Applications26\(3\),pp\. 941–955\.External Links:[Document](https://dx.doi.org/10.1007/s10044-023-01181-9)Cited by:[§2](https://arxiv.org/html/2607.09196#S2.p6.1),[§4\.4\.5](https://arxiv.org/html/2607.09196#S4.SS4.SSS5.p1.3)\.
- \[46\]N\. Shin, S\. Lee, and C\. Kim\(2022\)Moving window regression: a novel approach to ordinal regression\.InProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition \(CVPR\),pp\. 18760–18769\.Cited by:[§2](https://arxiv.org/html/2607.09196#S2.p6.1)\.
- \[47\]D\. Silver, A\. Huang, C\. J\. Maddison, A\. Guez, L\. Sifre, G\. van den Driessche, J\. Schrittwieser, I\. Antonoglou, V\. Panneershelvam, M\. Lanctot,et al\.\(2016\)Mastering the game of Go with deep neural networks and tree search\.Nature529\(7587\),pp\. 484–489\.Cited by:[§1](https://arxiv.org/html/2607.09196#S1.p1.1)\.
- \[48\]D\. Silver, T\. Hubert, J\. Schrittwieser, I\. Antonoglou, M\. Lai, A\. Guez, M\. Lanctot, L\. Sifre, D\. Kumaran, T\. Graepel, T\. Lillicrap, K\. Simonyan, and D\. Hassabis\(2018\)A general reinforcement learning algorithm that masters chess, shogi, and Go through self\-play\.Science362\(6419\),pp\. 1140–1144\.External Links:[Document](https://dx.doi.org/10.1126/science.aar6404)Cited by:[§1](https://arxiv.org/html/2607.09196#S1.p1.1)\.
- \[49\]R\. H\. Somers\(1962\)A new asymmetric measure of association for ordinal variables\.American Sociological Review27\(6\),pp\. 799–811\.Cited by:[§4\.6](https://arxiv.org/html/2607.09196#S4.SS6.p6.7)\.
- \[50\]A\. Summerville, S\. Snodgrass, M\. Guzdial, C\. Holmgård, A\. K\. Hoover, A\. Isaksen, A\. Nealen, and J\. Togelius\(2018\)Procedural content generation via machine learning \(PCGML\)\.IEEE Transactions on Games10\(3\),pp\. 257–270\.Cited by:[§2](https://arxiv.org/html/2607.09196#S2.p2.1)\.
- \[51\]M\. Treanor, A\. Zook, M\. P\. Eladhari, J\. Togelius, G\. Smith, M\. Cook, T\. Thompson, B\. Magerko, J\. Levine, and A\. Smith\(2015\)AI\-based game design patterns\.InProceedings of the 10th International Conference on the Foundations of Digital Games \(FDG 2015\),Cited by:[§2](https://arxiv.org/html/2607.09196#S2.p4.1)\.
- \[52\]G\. Tutz\(2022\)Ordinal regression: a review and a taxonomy of models\.WIREs Computational Statistics14\(2\),pp\. e1545\.Cited by:[§2](https://arxiv.org/html/2607.09196#S2.p5.1)\.
- \[53\]A\. Tychsen, N\. Pagden,et al\.\(2006\)Making magic\! the effects of enhancing PnP RPGs with computing technology\.InSeminar on Playing Roles,Cited by:[§2](https://arxiv.org/html/2607.09196#S2.p2.1)\.
- \[54\]J\. van Stegeren and J\. Myśliwiec\(2021\)Fine\-tuning GPT\-2 on annotated RPG quests for NPC dialogue generation\.InProceedings of the 16th International Conference on the Foundations of Digital Games \(FDG\),pp\. 1–8\.Cited by:[§2](https://arxiv.org/html/2607.09196#S2.p2.1)\.
- \[55\]C\. Winship and R\. D\. Mare\(1984\)Regression models with ordinal variables\.American Sociological Review49\(4\),pp\. 512–525\.Cited by:[§2](https://arxiv.org/html/2607.09196#S2.p5.1)\.
- \[56\]Z\. Wu, Y\. Mao, and Q\. Li\(2021\)Procedural game map generation using multi\-leveled cellular automata by machine learning\.InProceedings of the 2nd International Symposium on Artificial Intelligence for Medicine Sciences,pp\. 168–172\.Cited by:[§2](https://arxiv.org/html/2607.09196#S2.p2.1)\.
- \[57\]M\. Zhu and A\. I\. Wang\(2019\)Model\-driven game development: a literature review\.ACM Computing Surveys \(CSUR\)52\(6\),pp\. 1–32\.Cited by:[§2](https://arxiv.org/html/2607.09196#S2.p4.1)\.

## Appendix ARegular vs macroaveraged metrics

In Tables[4](https://arxiv.org/html/2607.09196#A1.T4)and[5](https://arxiv.org/html/2607.09196#A1.T5), we present the regular MAE and RMSE, as well as their macroaveraged variants\. Macroaveraging always results in higher error values, often substantially\. This follows the literature, confirming their usefulness for imbalanced ordinal regression\.

Table 4:Unweighted and macroaveraged metrics, chronological split\.ModelMAE𝐌𝐀𝐄𝐌\\mathbf\{MAE\}^\{\\mathbf\{M\}\}RMSE𝐑𝐌𝐒𝐄𝐌\\mathbf\{RMSE\}^\{\\mathbf\{M\}\}kNN0\.58±\\pm0\.160\.63±\\pm0\.200\.93±\\pm0\.211\.01±\\pm0\.25Ridge0\.37±\\pm0\.080\.42±\\pm0\.110\.71±\\pm0\.160\.77±\\pm0\.18SVM0\.30±\\pm0\.110\.35±\\pm0\.140\.63±\\pm0\.180\.69±\\pm0\.23Random Forest0\.22±\\pm0\.090\.25±\\pm0\.120\.52±\\bm\{\\pm\}0\.140\.57±\\pm0\.18LightGBM0\.23±\\pm0\.090\.27±\\pm0\.120\.53±\\pm0\.150\.59±\\pm0\.18SAOC0\.22±\\bm\{\\pm\}0\.080\.24±\\bm\{\\pm\}0\.090\.53±\\pm0\.110\.55±\\bm\{\\pm\}0\.11ORD0\.34±\\pm0\.070\.37±\\pm0\.100\.68±\\pm0\.130\.71±\\pm0\.15IT0\.31±\\pm0\.060\.34±\\pm0\.100\.65±\\pm0\.120\.68±\\pm0\.15ORF0\.23±\\pm0\.100\.26±\\pm0\.110\.54±\\pm0\.130\.56±\\pm0\.15GPOR0\.32±\\pm0\.080\.36±\\pm0\.130\.65±\\pm0\.140\.70±\\pm0\.21CLM0\.70±\\pm0\.880\.77±\\pm1\.071\.08±\\pm1\.051\.18±\\pm1\.27NNRank0\.47±\\pm0\.210\.54±\\pm0\.250\.81±\\pm0\.260\.89±\\pm0\.31OR\-CNN0\.44±\\pm0\.170\.52±\\pm0\.200\.79±\\pm0\.250\.87±\\pm0\.27CORAL0\.41±\\pm0\.180\.48±\\pm0\.250\.76±\\pm0\.240\.83±\\pm0\.31CORN0\.41±\\pm0\.190\.50±\\pm0\.240\.76±\\pm0\.240\.86±\\pm0\.29CONDOR0\.51±\\pm0\.270\.62±\\pm0\.340\.87±\\pm0\.311\.00±\\pm0\.41Table 5:Unweighted and macroaveraged metrics, expanding window\.
## Appendix BDetailed expanding window results

In Table[6](https://arxiv.org/html/2607.09196#A2.T6), we provide the detailed results over all testing windows under the expanding window evaluation for Random Forest\. Following the protocol explained in the main body, we obtained 21 train\-test runs \(windows\)\. The metrics remain generally consistent after a few runs, with highly noisy, but overall accurate results\. This can be explained by data distribution shifts as novel monsters are introduced\.

Table 6:Detailed expanding window results for Random Forest\.
## Appendix CHyperparameter grids

Here, we present the hyperparameter grids evaluated in our experiments\. Regression and tabular ordinal regression models’ grids are summarized in Table[7](https://arxiv.org/html/2607.09196#A3.T7)\.dddenotes the number of input features\. All neural networks used the same grid: learning rate\{10−3,10−2,10−1\}\\\{10^\{\-3\},10^\{\-2\},10^\{\-1\}\\\}, weight decay\{10−3,10−2,10−1,1\}\\\{10^\{\-3\},10^\{\-2\},10^\{\-1\},1\\\}\.

ModelHyperparameterGrid / valueRidgeα\\alpha10 000 values, linearly spaced in\[10−3,1\]\[10^\{\-3\},1\]SVM \(RBF kernel\)CC100 values, linearly spaced in\[1,10\]\[1,10\]kNNn\_neighbors\{1,3\}\\\{1,3\\\}weights\{\\\{uniform, distance\}\\\}metric\{\\\{Minkowski, Manhattan, Euclidean\}\\\}leaf\_size\{50,60,70,80,90\}\\\{50,60,70,80,90\\\}Random Forestmax\_features\{d,0\.3​d\}\\\{\\sqrt\{d\},\\ 0\.3d\\\}n\_estimators\{100,200,500\}\\\{100,200,500\\\}criterion\{\\\{squared error, absolute error, Friedman MSE\}\\\}ORFmax\_features\{0\.3​d\}\\\{0\.3d\\\}min\_samples\_leaf\{2,3,4,5,6,7\}\\\{2,3,4,5,6,7\\\}n\_estimators\{100,200,500\}\\\{100,200,500\\\}honestyFalseITα\\alpha100 values, linearly spaced in\[0,10−3\]\[0,10^\{\-3\}\]ORDoffset11 values, linearly spaced in\[0\.25,1\.25\]\[0\.25,1\.25\]SAOC \(Random Forest\)max\_features\{d,0\.3​d\}\\\{\\sqrt\{d\},\\ 0\.3d\\\}n\_estimators\{100,200,500\}\\\{100,200,500\\\}criterion\{\\\{Gini, entropy\}\\\}GPORkernelArcCosinemax\_iter100Table 7:Hyperparameter grids for regression and tabular ordinal regression models\.