Comparing Architectures for Supervised Political Scaling

# Comparing Architectures for Supervised Political Scaling
Source: [https://arxiv.org/html/2607.01464](https://arxiv.org/html/2607.01464)
Sebastian Padó Institute for Natural Language Processing \(IMS\) University of Stuttgart, Germany \{anna\.golub,sebastian\.pado\}@ims\.uni\-stuttgart\.de

###### Abstract

Text scaling, the task of positioning political actors on an ideological scale, is a fundamental task in political analysis\. To ease the need for manual analysis, various NLP methods have been proposed for this task, including classification\- and regression\-based approaches, showing successes as well as limitations\. The goal of our paper is to consolidate the state of the art in this area\. We ask two questions: \(a\) Can the performance of scaling methods be improved by predicting scales not individually but jointly? \(b\) Is there a middle ground between classification and regression?

Comparing Architectures for Supervised Political Scaling

Anna Golub and Sebastian PadóInstitute for Natural Language Processing \(IMS\)University of Stuttgart, Germany\{anna\.golub,sebastian\.pado\}@ims\.uni\-stuttgart\.de

## 1Introduction

Text scaling, the task of extracting the stances of political actors from written documents or speeches and mapping them to scores on an ideological axis, is a fundamental task in political text analysis\(Laveret al\.,[2003](https://arxiv.org/html/2607.01464#bib.bib15)\)\. Representing party positions in this way allows to quantitatively measure the differences between them, which is instrumental in understanding voters’ behavior during an election as well as the parties’ strategies once in office\(Benoit and Laver,[2006](https://arxiv.org/html/2607.01464#bib.bib7)\)\.

Determining a gold standard for party positions is challenging\. Probably one of the best\-known sources, based directly on querying experts, is the Chapel Hill Expert Survey\(Rovnyet al\.,[2025](https://arxiv.org/html/2607.01464#bib.bib6)\)\. An alternative approach was chosen by the Manifesto Research on Political Representation \(MARPOR\) project111Previously known as the Comparative Manifesto Project \(CMP\),[https://manifestoproject\.wzb\.eu/](https://manifestoproject.wzb.eu/)\. Grounding party positioning directly in texts, MARPOR has collected over 3200 manifestos \(regarded as the most comprehensive source on party policy\) and annotated political statements in them according to a fine\-grained ontology\. Overall party positions can then be obtained by aggregating category frequencies \(see Section 2 for details\)\. The most well\-known scale arising from this work isRILE, or the Standard Right\-Left Scale, reflecting mainly positions on economic policy\(Volkenset al\.,[2013](https://arxiv.org/html/2607.01464#bib.bib9); Budge,[2013](https://arxiv.org/html/2607.01464#bib.bib8)\)\.

To alleviate the cost of gold standard creation, the scores can be estimated automatically with the help of NLP methods\. Early word frequency\-based statistical methods\(Laveret al\.,[2003](https://arxiv.org/html/2607.01464#bib.bib15); Slapin and Proksch,[2008](https://arxiv.org/html/2607.01464#bib.bib16)\)already revealed the potential of computerized approaches for estimating party positions\. Later, those results were extended and improved upon by methods from distributional semantics\(Glavašet al\.,[2017](https://arxiv.org/html/2607.01464#bib.bib22); Rheault and Cochrane,[2020](https://arxiv.org/html/2607.01464#bib.bib23)\), Transformer\-based sentence embeddingsCeronet al\.\([2022](https://arxiv.org/html/2607.01464#bib.bib25)\); Nikolaevet al\.\([2023](https://arxiv.org/html/2607.01464#bib.bib14)\)and large language models \(LLMs\)Benoitet al\.\([2025](https://arxiv.org/html/2607.01464#bib.bib63)\)\.

The results of these studies are encouraging but still far from perfect, unveiling that the task of political scaling, as approached generally, suffers from two interrelated problems\. The first isdocument length: Political documents, such as election manifestos, tend to comprise thousands of sentences\. Thus, they are too long for current Transformer models to process in one and need to be subdivided in some manner\. The second issingle\-scale prediction: Despite the broad consensus in political science that multiple scales are necessary to adequately capture party positions\(Koedamet al\.,[2025](https://arxiv.org/html/2607.01464#bib.bib5)\), most computational work focuses on positioning on single scales \(mostly RILE\)\. As a result, models arguably need to learn to ignore a lot of information in the input instead of using it to their advantage\.

To address these shortcomings, we systematically compare the performance of different approaches, focusing on two research questions:

RQ1: Does joint prediction improve scaling?TheGAL\-TAN, ranging from the Green\-Alternative\-Liberal to the Traditional\-Authoritarian\-Nationalist extreme\(Marks and Steenbergen,[2004](https://arxiv.org/html/2607.01464#bib.bib54)\), captures socio\-cultural views that complement RILE’s economic perspective\. Still, to our knowledge, there is no work on predicting GAL\-TAN positions\. What is more, despite being orthogonal in theory, the empirical RILE and GAL\-TAN scores are often found to be interdependent\. Left\-wing parties gravitate towardGALsocial policies, and right\-wing views tend to co\-occurTANones, whereas the remaining two combinations are much less frequent\(Jahn,[2011](https://arxiv.org/html/2607.01464#bib.bib57); Brigevich and Smith,[2017](https://arxiv.org/html/2607.01464#bib.bib1); Wagneret al\.,[2021](https://arxiv.org/html/2607.01464#bib.bib52)\)\. On this basis, we investigate whether joint models for predicting RILE and GAL\-TAN together can improve results\.

![Refer to caption](https://arxiv.org/html/2607.01464v1/images/rqs_diagram_updated.jpg)Figure 1:Experimental setup: Computational approaches to scaling and research questions\.RQ2: How to deal with long input documents?Traditional studies followed the MARPOR methodology in building a pipeline that first classifies individual sentences and then aggregates class frequencies into scale positions \(label aggregation\)\.Nikolaevet al\.\([2023](https://arxiv.org/html/2607.01464#bib.bib14)\)also trained transformers to directly predict scale positions with a regression head, but still had to divide the input into smaller chunks for processing \(chunk\-level regression\)\. The trade\-off between the two approaches remains underexplored\. Notably, thechunk sizeis a crucial \(hyper\-\)parameter for this approach which has not been investigated\.

Figure[1](https://arxiv.org/html/2607.01464#S1.F1)shows the resulting experimental design\. We find for RQ1 that GAL\-TAN can be predicted about as well as RILE, but that joint prediction does not improve performance\. For RQ2, we establish that chunk size, somewhat surprisingly, matters rather little as far as performance is concerned, creating a continuum between regression and classification\. We will make our code available\.

## 2Methods

### 2\.1Label Aggregation

Mimicking a simplified version of the MARPOR coding scheme, label aggregationNikolaevet al\.\([2023](https://arxiv.org/html/2607.01464#bib.bib14)\)predicts the RILE score via sentence classification\. A pre\-trained Transformer encoder generates a sentence embedding to be fed to a classification head, which outputs one of the labelsRight,LeftorOther\. The model is trained with cross\-entropy loss\. The gold standard categories are derived from fine\-grained MARPOR sentence annotations in accordance with the literature\(Marks and Steenbergen,[2004](https://arxiv.org/html/2607.01464#bib.bib54); Volkenset al\.,[2013](https://arxiv.org/html/2607.01464#bib.bib9)\)\. Using Eq\. \([1](https://arxiv.org/html/2607.01464#S2.E1)\), the sentence\-level predictions are aggregated into manifesto\-level positions between \-1 \(extreme left\) and \+1 \(extreme right\):

R​I​L​E=R−LR\+L\+ORILE=\\frac\{R\-L\}\{R\+L\+O\}\(1\)whereRR,LLandOOare the number of sentences with the categoriesRight,LeftandOther, respectively\.

This approach can be extended to the GAL\-TAN score estimation by replacing the RILE\-specific categories with\(G\)AL,\(T\)ANand\(O\)ther, and aggregating the predictions using the equation \([2](https://arxiv.org/html/2607.01464#S2.E2)\)\.

G​A​L−T​A​N=G−TG\+T\+OGAL\-TAN=\\frac\{G\-T\}\{G\+T\+O\}\(2\)We refer to this method aslabel aggregation individual prediction, as it estimates RILE and GAL\-TAN independently from each other\.

### 2\.2Chunk\-Level Regression

Alternatively, the prediction tasks can be operationalized as regression, where a model maps a manifesto directly to a score in the range\[−1,1\]\[\-1,1\]Nikolaevet al\.\([2023](https://arxiv.org/html/2607.01464#bib.bib14)\)\. While there is continuous research on robustly processing long inputs\(Alva Principeet al\.,[2025](https://arxiv.org/html/2607.01464#bib.bib4)\), the context window of the modern state\-of\-the\-art Transformer encoders is nowhere near large enough to fit a whole manifesto\. Nonetheless, some encoders allow inputs of up to 8192 tokensWarneret al\.\([2025](https://arxiv.org/html/2607.01464#bib.bib18)\), or over 650 MARPOR sentences, and therefore can approximate direct prediction with chunk\-level processing\. Specifically, the given manifesto is split into chunks of the maximum allowed length, after which a pre\-trained encoder generates chunk embeddings, and a regression head trained on top outputs per\-chunk scores\. The model is trained with MSE loss, with the gold standard scores for a chunk calculated by aggregating the sentence labels \(Eq\.[1](https://arxiv.org/html/2607.01464#S2.E1)\) within the given chunk\. For inference, the score of a whole manifesto is estimated as the average of the scores of its constituent chunks\. Similarly to label aggregation,chunk\-level regression individual predictioncan be employed equally to predict RILE and GAL\-TAN scores\.

### 2\.3Joint Modeling

We experiment with two strategies to move from individual to joint prediction \(RQ1\)\.

#### Multitask Training

Among the multitask optimization methods such as multi\-objective optimization, adversarial learning or neural architecture search, the one used most widely isscalarization, where the joint loss is a linear combination of the losses for the individual tasks\(Yuet al\.,[2025](https://arxiv.org/html/2607.01464#bib.bib10)\)\. To apply scalarization to label aggregation and chunk\-level regression, each pre\-trained encoder is fine\-tuned with two classification or regression heads on top, a RILE and a GAL\-TAN specific one\. The joint prediction loss function is the sum of the individual cross\-entropy or MSE functions\. The RILE and the GAL\-TAN term are assigned equal weights because there is no implicit superiority of one over the other, and they are represented by the same number of data points\. Because the RILE and GAL\-TAN scores are correlated, multitask training is expected to improve over individual prediction, since the gradient of the encoder weights shows the direction where both the loss terms are minimized\.

#### Contrastive Training

As an alternative to multitask training, we jointly train the label aggregation encoders with a contrastive objective\. As our object function, we adopttriplet loss\(Schroffet al\.,[2015](https://arxiv.org/html/2607.01464#bib.bib11)\), which aims to push similar instances close together in the embedding space while driving dissimilar instances apart\. This method is used, for example, to tune the SBERT \(sentence BERT\) architecture\(Reimers and Gurevych,[2019](https://arxiv.org/html/2607.01464#bib.bib20)\)\.

The training data is processed in triplets of ananchor conceptaa, apositive conceptppfrom the same class asaa, and anegative conceptnnfrom a different class\. The triplet loss aims to reduce the distance betweenaaandppto zero while increasing the distance betweenaaandnnto at leastϵ\\epsilon:

m​a​x​\(‖Sa−Sp‖−‖Sa−Sn‖\+ϵ,0\)max\(\|\|S\_\{a\}\-S\_\{p\}\|\|\-\|\|S\_\{a\}\-S\_\{n\}\|\|\+\\epsilon,0\)\(3\)whereSa,Sp,SnS\_\{a\},S\_\{p\},S\_\{n\}are the vector representations ofa,p,na,p,n;\|\|⋅\|\|\|\|\\cdot\|\|is the distance metric\. We employ the following two triplet mining strategies:

1. 1\.RILE and GAL\-TAN categories: The data points are selected randomly such thataaandppbelong to the same RILE and the same GAL\-TAN class whilenndiffers in at least one of the labels\. This sampling strategy perpetuates the idea that the RILE and GAL\-TAN categories are interdependent and each combination of them should occupy an isolated cluster in the sentence embedding space\.
2. 2\.Party: The triplets are chosen such that the anchoraaand the positive exampleppare sentences from manifestos by the same party while the negative examplennis authored by a different one\. When fine\-tuned on the MARPOR sentences with party\-based triplet loss, SBERT showed the best performance on pairwise party similarity estimation\(Ceronet al\.,[2022](https://arxiv.org/html/2607.01464#bib.bib25)\)\. Overall, party\-based triplets allow the encoder to learn the general nature of manifesto text, which is helpful for positioning parties on a political axis\.

After contrastive tuning, the weights of the encoder are frozen and two classification heads are trained on top to predict the RILE and GAL\-TAN categories\. In addition, before the embeddings are fed to the classification heads, they are normalized using thewhitening transformation\(Suet al\.,[2021](https://arxiv.org/html/2607.01464#bib.bib28)\), which is known to reduce theanisotropyof the embedding and improve performance on various NLP tasks\(Suet al\.,[2021](https://arxiv.org/html/2607.01464#bib.bib28)\), including pairwise party similarity estimationCeronet al\.\([2022](https://arxiv.org/html/2607.01464#bib.bib25)\)\.

### 2\.4LLM Baseline

Recent workLe Mens and Gallego \([2025](https://arxiv.org/html/2607.01464#bib.bib65)\); Ornsteinet al\.\([2025](https://arxiv.org/html/2607.01464#bib.bib64)\); Benoitet al\.\([2025](https://arxiv.org/html/2607.01464#bib.bib63)\)found that LLMs perform comparable to embedding\-based approaches on political scaling\. We therefore include an LLM in our experiment that implements the label aggregation approach\. We prompt an LLM to annotate the given sentence with one of the labelsRight,LeftandOther, and estimate the manifesto RILE scores using Eq\. \([1](https://arxiv.org/html/2607.01464#S2.E1)\)\. GAL\-TAN individual prediction is approached analogously\. We evaluate joint prediction capabilities by asking to assign both a RILE and a GAL\-TAN category in one prompt\. We call this approach a ’baseline’ because we do not optimize the prompt\.

## 3Experimental Setup

### 3\.1Data

For our experiments, we adopt theX\-time\(old\-vs\.\-new\)setting fromNikolaevet al\.\([2023](https://arxiv.org/html/2607.01464#bib.bib14)\)\. This dataset primarily tests the generalizability of scaling models over time, from current to future election cycles, in countries seen during training\. All models are trained on 1005 MARPOR manifestos for the years 2000–2018 \(over 1M sentences\)\. For each individual training setup, 10% of this data is selected randomly and held out for validation\. The available data for the years 2019–2023 \(147 manifestos, 163K sentences\), are used for testing\. Data from before 2000 is excluded because of inconsistencies in the annotation\.

The model inputs are the translations of the original manifesto text created byNikolaevet al\.\([2023](https://arxiv.org/html/2607.01464#bib.bib14)\)222[https://osf\.io/aypxd/overview](https://osf.io/aypxd/overview)with Opus\-MT models available through EasyNMT\.333[https://github\.com/UKPLab/EasyNMT](https://github.com/UKPLab/EasyNMT)Nikolaevet al\.\([2023](https://arxiv.org/html/2607.01464#bib.bib14)\)compared the use of MT with multilingual models and found almost identical results\.

### 3\.2Label Aggregation

FollowingNikolaevet al\.\([2023](https://arxiv.org/html/2607.01464#bib.bib14)\), we use a text encoder with a classification head on top\. We compare two encoders: SBERT and ModernBERT\. The joint versions have two structually identical heads, one per scale \(cf\. Section[2\.3](https://arxiv.org/html/2607.01464#S2.SS3)\)\.

#### Classification head\.

The classification head is a multi\-layer perceptron \(MLP\) consisting of two layers with the hidden size of 1024 and a tanh activation after the first layer\. The output vector is passed through the softmax function to obtain a probability distribution\.

#### SBERT

Sentence TransformersReimers and Gurevych \([2019](https://arxiv.org/html/2607.01464#bib.bib20)\), or SBERT, is a class of siamese Transformer models optimized for semantic textual similarity tasks\. They have been shown to perform well on political party positioning, with SBERT label aggregation fine\-tuned on the MARPOR data scoring the highest on RILE score predictionNikolaevet al\.\([2023](https://arxiv.org/html/2607.01464#bib.bib14)\)\. Moreover,Ceronet al\.\([2022](https://arxiv.org/html/2607.01464#bib.bib25)\)trained SBERT on MARPOR sentences for pairwise party similarity estimation, and after the whitening transformation, SBERT embeddings achieved the strongest correlation with the ground truth\. For label aggregation, we useall\-mpnet\-base\-v2, an SBERT model based on MPNetSonget al\.\([2020](https://arxiv.org/html/2607.01464#bib.bib31)\)that is recommended as the general purpose Sentence Transformer with the highest embedding quality\.444[https://sbert\.net/docs/sentence\_transformer/pretrained\_models\.html\#original\-models](https://sbert.net/docs/sentence_transformer/pretrained_models.html#original-models)However, due to its small context size, it is not suitable as encoder for chunk\-level regression\.

#### ModernBERT

ModernBERTWarneret al\.\([2025](https://arxiv.org/html/2607.01464#bib.bib18)\)is a recent update on the original BERT model\(Devlinet al\.,[2019](https://arxiv.org/html/2607.01464#bib.bib12)\)\. It incorporates rotary positional embeddings \(RoPE\), pre\-normalization blocks and an updated activation function, all established improvements on the original Transformer architecture\(Warneret al\.,[2025](https://arxiv.org/html/2607.01464#bib.bib18)\)\. The self\-attention mechanism at the core of the Transformer architecture is associated with a quadratic computational cost, which ModernBERT addresses by only usingglobal self\-attentionin every third layer\. The rest arelocal attentionlayers, where each input token only attends to the tokens close to it\. ModernBERT outperforms BERT variants of a similar size and is competitive with bigger, slower variants such as GTE\-en\-MLM\(Zhanget al\.,[2024](https://arxiv.org/html/2607.01464#bib.bib40)\)and DeBERTa\-v3\-large\(Heet al\.,[2023](https://arxiv.org/html/2607.01464#bib.bib39)\)\. We use ModernBERT\-base in our experiments\.555[https://huggingface\.co/answerdotai/ModernBERT\-base](https://huggingface.co/answerdotai/ModernBERT-base)

Table 1:Rank correlation with the gold standard manifesto scores\. The best performance on RILE and on GAL\-TAN each is highlighted in bold\. When multiple values are boldfaced for a given target, the difference between them is not statistically significant\. The joint contrastive models are SBERT\-RILE\-GAL\-TANwhitenand ModernBERT\-RILE\-GAL\-TAN\. N23 isNikolaevet al\.\([2023](https://arxiv.org/html/2607.01464#bib.bib14)\)\.

### 3\.3Chunk\-Level Regression

For chunk\-level regression, we combine, again, a text encoder \(BigBird and ModernBERT\) with a regression head\. As before, the joint versions have two structurally identical heads\.

#### Regression head\.

The regression is the same MLP described above, except that the final softmax layer is replaced with a single tanh\-activated unit to obtain predictions in the−1​…​1\-1\\dots 1range\.

#### BigBird

Zaheeret al\.\([2020](https://arxiv.org/html/2607.01464#bib.bib19)\)tackled the complexity of the self\-attention computation by approximating it withsparse attention, which prunes the set of possible attention links to achieve a speed\-up\. The resulting model can deal with a context window of 4096 tokens\. BigBird was initialized from RoBERTa\(Zhuanget al\.,[2021](https://arxiv.org/html/2607.01464#bib.bib38)\)and further trained on a large web corpus\. When evaluated on long\-input question answering and long document classification, BigBird set the new state of the art on several datasets and otherwise demonstrated competitive performance\. InNikolaevet al\.\([2023](https://arxiv.org/html/2607.01464#bib.bib14)\), BigBird was the best out of the evaluated long\-input encoder but only scored moderately well compared to the label aggregation setup\. BigBird chunks consist of an average of 173 sentences\. This means that the mean manifesto length is 6–7 chunks\. We use BigBird\-base in our experiments\.666[https://huggingface\.co/google/bigbird\-roberta\-base](https://huggingface.co/google/bigbird-roberta-base)

#### ModernBERT

ModernBERT employs Flash Attention\(Daoet al\.,[2022](https://arxiv.org/html/2607.01464#bib.bib44)\)to extend the original BERT context window of 512 tokens to 8192, which makes it suitable also for long input regression\. On average, one chunk of 8192 tokens fits a maximum of 315 MARPOR sentences, and the mean manifesto length is 3\-4 chunks\. Hence, ModernBERT is expected to perform well as the encoder in both label aggregation and chunk\-level regression, allowing for a direct comparison between the approaches on a conceptual level\.

### 3\.4LLM Baseline

For LLM\-based prediction, we useOlmo\-3\-7B\-InstructEttingeret al\.\([2026](https://arxiv.org/html/2607.01464#bib.bib66)\), one of the few LLMs with a completely open training procedure\(Liesenfeld and Dingemanse,[2024](https://arxiv.org/html/2607.01464#bib.bib3)\)\. We employ it in a zero\-shot setting, see Section[2\.4](https://arxiv.org/html/2607.01464#S2.SS4)and Appendix[A](https://arxiv.org/html/2607.01464#A1)for the prompts\.

### 3\.5Evaluation

To make our results comparable toNikolaevet al\.\([2023](https://arxiv.org/html/2607.01464#bib.bib14)\), we measure the RILE and GAL\-TAN score prediction quality as Spearman rank correlation coefficient between the predicted and the gold standard scores at the manifesto level\. This metric evaluates the ability of the models to correctly rank parties on the scales, rather than absolute positions\.

We train each supervised model with 5 random seeds and report the averages of the per\-seed evaluation metrics\. For the LLM baseline, we sample 5 responses per data point and average the evaluation metrics over the LLM answers\. For all pairs of models with≤\\leq15 percentage points of difference in performance, we test the statistical significance of the difference by running the bootstrap resampling test with10,00010,000resamples and a95%95\\%confidence interval\(Efron and Tibshirani,[1994](https://arxiv.org/html/2607.01464#bib.bib62)\)\.

## 4Results

![Refer to caption](https://arxiv.org/html/2607.01464v1/images/rile_sbert_ind_pred_lr1e-05_bs256_ep3_percent.png)![Refer to caption](https://arxiv.org/html/2607.01464v1/images/bigbird_rile_chunk_vs_manifesto.png)
Figure 2:Left: Confusion matrix heatmap: Label aggregation RILE individual prediction with SBERT\.Right: Predicted vs\. true scatterplot: Chunk\-level RILE regression with BigBird \(test set, random seed 7\)\.### 4\.1Individual Prediction

The main results are reported in Table[1](https://arxiv.org/html/2607.01464#S3.T1)\. Label aggregation individual prediction with SBERT achieves a very strong correlation ofρ=0\.88\\rho=0\.88on RILE\. Our result on RILE is a very close replication of the result byNikolaevet al\.\([2023](https://arxiv.org/html/2607.01464#bib.bib14)\)\.

The GAL\-TAN scale is slightly more difficult to predict, with best results aroundρ=0\.83\\rho=0\.83to0\.840\.84, even if not by a large margin\. This may be related to the more complex nature of cultural, as opposed to economic, stances\(Kurella and Rapp,[2026](https://arxiv.org/html/2607.01464#bib.bib2)\), as well as the training data imbalance that is more severe for GAL\-TAN than for RILE\.

Comparing the different implementations of label aggregation, we note that ModernBERT performs on a par with SBERT on both scales\. Zero\-shot prediction with Olmo 3 is much less robust than the supervised methods withρ=0\.67/0\.53\\rho=0\.67/0\.53\. While it might be possible to improve on these results with more careful prompt engineering, we note thatBenoitet al\.\([2025](https://arxiv.org/html/2607.01464#bib.bib63)\)report \(linear\) correlation scores of a similar magnitude in an experiment with MARPOR data \(r=0\.57r=0\.57onTaxes vs\. Spendingandr=0\.68r=0\.68on theSocialaxis\)\.777From the supplementary materials ofBenoitet al\.\([2025](https://arxiv.org/html/2607.01464#bib.bib63)\)\.

For chunk\-level regression, prediction with BigBird is on a par with label aggregation on GAL\-TAN and slightly behind on RILE withρ=0\.84\\rho=0\.84on both\. This is a new qualitative finding, as chunk\-level regression with BigBird was reported byNikolaevet al\.\([2023](https://arxiv.org/html/2607.01464#bib.bib14)\)to fall behind label aggregation\. ModernBERT underperforms BigBird somewhat, however this may be a consequence of our choice to use the maximum chunk size \(twice as high for ModernBERT as for BigBird\), cf\. Section[4\.3](https://arxiv.org/html/2607.01464#S4.SS3)\.

#### Error Analysis\.

While the manifesto\-level rank correlation is strong, there is still space for improvement at the level of predicted labels\. Due to the skew towards the labelOtherin the training data, label aggregation misclassifies over30%30\\%of the sentences markedRight,Left,GALandTAN\(see Figure[2](https://arxiv.org/html/2607.01464#S4.F2), left\)\. The aggregation of the sentence\-level predictions according to Eq\. \([1](https://arxiv.org/html/2607.01464#S2.E1)\) and \([2](https://arxiv.org/html/2607.01464#S2.E2)\) respectively appears to smooth out the errors, but the resulting manifesto scores suffer from regression to the mean \(i\.e\. zero\): They are correct in sign but too small in magnitude\. Figure[2](https://arxiv.org/html/2607.01464#S4.F2)\(right\) shows that chunk\-level regression shows the same effects, which can be interpreted as low model confidence\.

### 4\.2Joint Prediction \(RQ1\)

Contrary to our expectations from RQ 1, joint multitask training does not improve over individual prediction in almost all settings\. In the label aggregation setting, joint multitask learning does not have any major effect on the embedding\-based model\. Prompting Olmo to output labels for both scales jointly even has a clear negative effect\.

Contrastive tuning also impedes the quality of the predictions throughout\. As shown in App\.[B](https://arxiv.org/html/2607.01464#A2), performance drops further when selecting triplets based on party and while in line withCeronet al\.\([2022](https://arxiv.org/html/2607.01464#bib.bib25)\), the whitening transformation has an overall positive effect on SBERT, it is detrimental to ModernBERT\.

Finally, in the chunk regression setting, joint multitask training leads to slightly worse results for BigBird\. While ModernBERT improves slightly with joint multitask training, it still scores lower than BigBird due to its lower starting point\.

![Refer to caption](https://arxiv.org/html/2607.01464v1/images/loss_curve_sbert_ind_sum_vs_joint_small.png)Figure 3:SBERT learning curves: joint multitask learning vs\. the sum of the RILE and GAL\-TAN losses during individual optimization \(random seed 7\)\.#### Training dynamics\.

To better understand this negative result, we analyzed the behavior of the RILE and GAL\-TAN train and validation losses during training\. As Figure[3](https://arxiv.org/html/2607.01464#S4.F3)shows, the losses behave in the same way as the sum of the losses computed during individual optimization\. Therefore, the two objectives appear to neither sabotage nor support each other and are effectively learned separately\. We observe very similar behavior for chunk\-level regression\.

One underlying reason for this might be that the correlation between RILE and GAL\-TAN in the training data is too weak for the models to pick up and to capitalize on\. Another explanation, however, is a ceiling effect, i\.e\., the individual predictions are already bounded by the reliability of the data so that better modeling mechanisms cannot further improve the results\. Indeed, a MARPOR coder reliability studyMikhaylovet al\.\([2012](https://arxiv.org/html/2607.01464#bib.bib68)\)reports numbers that correspond to a macro\-F1 score of0\.660\.66between two human annotators that classify sentences into the RILE categoriesRight,LeftandOther\. This is coincidentally the exact performance achieved by SBERT on the classification task \(cf\. Appendix[B](https://arxiv.org/html/2607.01464#A2)\)\. On GAL\-TAN, the model’s macro\-F1 is0\.670\.67whereas the human estimate is0\.610\.61\. These correspondences make an interpretation as a ceiling effect plausible, even though the limited data available on inter\-rater agreement does not allow us to draw strong conclusions\.

### 4\.3Long Input Documents \(RQ2\)

Our ModernBERT encoder is suitable for both label aggregation\-based and for regression\-based prediction of party positions \(cf\. Section[3](https://arxiv.org/html/2607.01464#S3)\)\. However, a notable result from Table[1](https://arxiv.org/html/2607.01464#S3.T1)is that ModernBERT performs substantially better for label aggregation by a margin of 5\-8 points in rank correlation\. This may be due to the model becoming less robust with longer inputs, or due to the difference between classification and regression as tasks\. To investigate the trade\-offs between the two task formulations, we vary the chunk sizes for joint ModernBERT\-based chunk\-level regression on a logarithmic scale fromnn=1 to aroundn≈300n\\approx 300sentences on average, covering the full range of the possible input length \(see App\.[A](https://arxiv.org/html/2607.01464#A1)for details\)\. Each model is trained once with the random seed 7\.

![Refer to caption](https://arxiv.org/html/2607.01464v1/images/ModernBERT_chunk_size_rho_upd.png)Figure 4:Joint chunk\-level regression with ModernBERT on various chunk sizes \(test set, random seed 7\)\. The dots are connected for readability only\. The stars mark the performance of joint label aggregation with ModernBERT\.Figure[4](https://arxiv.org/html/2607.01464#S4.F4)presents the results\.888The results in Tab\.[1](https://arxiv.org/html/2607.01464#S3.T1)adopt the largest chunk size \(nn=315\)\.Strikingly, the models score in the same rangeregardless of the chunk size, namelyρ\\rho=0\.82–0\.87 on RILE andρ\\rho=0\.79–0\.84 on GAL\-TAN\. The bootstrap test shows that all the models are statistically on a par\.

![Refer to caption](https://arxiv.org/html/2607.01464v1/images/modernbert_from_clf_to_rgr_rile_scatterplot_manifesto.png)Figure 5:Classification\-regression continuum: RILE true vs\. predicted manifesto scores scatterplot \(test set, random seed 7\)\.nnis the number of input sentences;s​l​pslpis the slope of the regression line \(red\)\. Lowers​l​pslpindicates stronger regression to the mean\.#### Classification\-Regression Continuum\.

Note that for the chunk sizen=1n=1sentence, the regression model is trained to predict−1\-1,0, or−1\-1, depending on the class of the sentence\. This is very similar to the training task of the label aggregation setup, only with a regression instead of a classification head\. Thus, ModernBERT can be seen as supporting a continuum between classification and regression\. This does not mean that the results are exactly the same, though: Regression with chunk sizenn=1 is numerically on a par with label aggregation on RILE \(0\.87 vs\. 0\.86\) but slightly worse on GAL\-TAN \(0\.81 vs\. 0\.83\)\. Indeed, the regression is less precise on justnn=1 sentence per chunk than on max\.nn=100 \(97 on average\): MSE=0\.23/0\.15=0\.23/0\.15\(RILE/GAL\-TAN\) fornn=1 vs\. MSE=0\.022/0\.016=0\.022/0\.016fornn=100 at the chunk level\. As discussed in Section[4\.1](https://arxiv.org/html/2607.01464#S4.SS1), performance levels out once the chunk estimates are averaged to represent full manifestos, producing MSE=0\.009/0\.007=0\.009/0\.007fornn=1 and MSE=0\.006/0\.006=0\.006/0\.006fornn=100 at the manifesto level\.

Another drawback of small chunk sizes is a strong regression to the mean: For most manifestos, their distance from the neutral value is underestimated\. Figure[5](https://arxiv.org/html/2607.01464#S4.F5)shows this by plotting predicted against gold manifesto RILE positions \(the picture is the same for GAL\-TAN\)\. In these scatterplots, ideal predictions would lie along a regression line with slope 1; slopes below 1 indicate underestimation\. As the Figure shows, both classification and regression suffer from underestimation\. For the regression models, a medium chunk size ofnn=100 obtains the overall highest slope of 0\.78, indicating that this model more accurately approximates not only the gold standard ranking but also the absolute values of positions\. For very large chunk sizes, both correlation and slope decrease, indicating that models still struggle to extract information reliably from very large contexts\(Wuet al\.,[2025](https://arxiv.org/html/2607.01464#bib.bib42)\)\.

Taken together, these results indicate that regression\-based direct prediction of positions on political scales provides an alternative to label aggregation\. While the performance is surprisingly constant across chunk sizes, it degrades somewhat for the largest chunk sizes, indicating that current models are still not ideally equipped to analyze very large contexts\. At the same time, very small chunk sizes suffer from regression to the mean, indicating that medium context sizes \(20–100 sentences per chunk\) are the most robust choice\.

## 5Related Work

Early work on automating political scaling relied on word frequencies for supervisedLaveret al\.\([2003](https://arxiv.org/html/2607.01464#bib.bib15)\)and unsupervisedSlapin and Proksch \([2008](https://arxiv.org/html/2607.01464#bib.bib16)\)estimation of party positions\. Once NLP shifted to distributional semantics,Glavašet al\.\([2017](https://arxiv.org/html/2607.01464#bib.bib22)\)employed word embeddings for unsupervised analysis of multilingual data, computing pairwise party similarities and rescaling them to obtain positions on an axis\. This line of work was continued byRheault and Cochrane \([2020](https://arxiv.org/html/2607.01464#bib.bib23)\)andNanniet al\.\([2022](https://arxiv.org/html/2607.01464#bib.bib24)\)\.

More recently, encoder\-only Transformer models have been used for pairwise party similarity estimation, overallCeronet al\.\([2022](https://arxiv.org/html/2607.01464#bib.bib25)\)and within policy domains with the aid of a sentence domain classifierCeronet al\.\([2023](https://arxiv.org/html/2607.01464#bib.bib17)\)\.Dayaniket al\.\([2022](https://arxiv.org/html/2607.01464#bib.bib41)\)fine\-tuned BERT to predict fine\-grained MARPOR sentence labels, improving the performance on infrequent classes with hierarchical classification\.

The success of LLMs on various NLP tasksMinaeeet al\.\([2025](https://arxiv.org/html/2607.01464#bib.bib21)\)has called for research on their applicability to political scaling\.Le Mens and Gallego \([2025](https://arxiv.org/html/2607.01464#bib.bib65)\)queried state\-of\-the\-art closed\-source LLMs to map a given sentence to a position on a economic or social axis, achieving strong correlation with the expert and crowd\-sourced gold standard\.Benoitet al\.\([2025](https://arxiv.org/html/2607.01464#bib.bib63)\)ensembled the predictions of several zero\- and few\-shot closed\-source LLMs which were prompted to summarize long inputs first and then scale them\. The results show strong correlation with expert positions but moderate correlation with the MARPOR ground truth\.

## 6Conclusions

In this paper, we have systematically evaluated embedding\-based approaches to party positioning based on election manifestos\. Our study was carried out on a sample from the MARPOR corpus corresponding to generalization from past to future election cycles\. We considered two approaches, label aggregation and chunk\-level regression, and focused on comparing them, establishing two main findings:

First, joint prediction of party positions on the RILE and GAL\-TAN scales does not improve prediction quality\. While this is at first glance a disappointing result, we found that the performance of our non\-joint models already approaches a plausible ceiling arising from inter\-annotator disagreement\. It would be worthwhile, in future work, to test joint modeling approaches in more challenging scenarios, such as generalization to new countries\(Nikolaevet al\.,[2023](https://arxiv.org/html/2607.01464#bib.bib14)\)or party position prediction based on less data – modeling situations where approaches based on domain knowledge tend to help\(Dayaniket al\.,[2022](https://arxiv.org/html/2607.01464#bib.bib41)\)\.

Second, ModernBERT\-based chunk regression offers a viable alternative to label aggregation in political positioning, in particular when medium chunk sizes \(20\-100 sentences\) are chosen\. This result is at the same time encouraging and disappointing: while it is conceptually more elegant to directly predict positions without an intermediate labeling step, and the regression approach models the actual score distribution better than the classification approach, the decreasing performance for the longest chunks indicates that manifestos are still too long for current transformer LMs to obtain good end\-to\-end learning results\. An interesting direction for future work follows from the realization that chunk\-level regression models do not require gold standard annotation at the sentence level for training\. This raises the question of whether the detailed MARPOR annotation could be replaced by a continuous position annotation at the chunk level, possibly framed as a ranking task\(Carlson and Montgomery,[2017](https://arxiv.org/html/2607.01464#bib.bib43)\)\.

## Limitations

Our study only investigated a single, comparatively simple setup for political party positioning: generalizing from previous to future election cycles\. We also relied on the quality of the MT system used byNikolaevet al\.\([2023](https://arxiv.org/html/2607.01464#bib.bib14)\)and did not experiment with multilingual models999A verified English translation of the MARPOR data has since become available\.[https://manifesto\-project\.wzb\.eu/information/documents/translation](https://manifesto-project.wzb.eu/information/documents/translation)\. Due to limited resources, we used smaller versions of the pre\-trained models, and the hyperparameter search was performed manually\.

The LLM approach that we included in our experiments was not optimized regarding its prompt, nor did we set up a few\-shot variant; in this sense, it can be considered an unsupervised \(or semi\-supervised\) point of comparison for the fully supervised models that we focused on\.

## References

- R\. Alva Principe, N\. Chiarini, and M\. Viviani \(2025\)Long document classification in the transformer era: a survey on challenges, advances, and open issues\.WIREs Data Mining and Knowledge Discovery15\(2\),pp\. e70019\.Note:e70019 DMKD\-00676\.R1External Links:[Document](https://dx.doi.org/https%3A//doi.org/10.1002/widm.70019),[Link](https://wires.onlinelibrary.wiley.com/doi/abs/10.1002/widm.70019),https://wires\.onlinelibrary\.wiley\.com/doi/pdf/10\.1002/widm\.70019Cited by:[§2\.2](https://arxiv.org/html/2607.01464#S2.SS2.p1.1)\.
- K\. Benoit, S\. De Marchi, C\. Laver, M\. Laver, and J\. Ma \(2025\)Using large language models to analyze political texts through natural language understanding\.American Journal of Political Science\.Cited by:[§1](https://arxiv.org/html/2607.01464#S1.p3.1),[§2\.4](https://arxiv.org/html/2607.01464#S2.SS4.p1.1),[§4\.1](https://arxiv.org/html/2607.01464#S4.SS1.p3.3),[§5](https://arxiv.org/html/2607.01464#S5.p3.1),[footnote 7](https://arxiv.org/html/2607.01464#footnote7)\.
- K\. Benoit and M\. Laver \(2006\)Party policy in modern democracies\.Routledge,London\.External Links:ISBN 9780203028179,[Document](https://dx.doi.org/10.4324/9780203028179)Cited by:[§1](https://arxiv.org/html/2607.01464#S1.p1.1)\.
- A\. Brigevich and W\. B\. Smith \(2017\)Unpacking the social \(GAL/TAN\) dimension of party politics: Euroscepticism and party positioning on Europe’s “other”\.InProceedings of the European Union Studies Association Biannual Conference,Miami\.External Links:[Link](https://api.semanticscholar.org/CorpusID:211235760)Cited by:[item RQ1: Does joint prediction improve scaling?](https://arxiv.org/html/2607.01464#S1.I1.ix1.p1.1)\.
- I\. Budge \(2013\)The standard left\-right scale\.Technical reportComparative Manifesto Project\.External Links:[Link](https://manifesto-project.wzb.eu/down/papers/budge_right-left-scale.pdf)Cited by:[§1](https://arxiv.org/html/2607.01464#S1.p2.1)\.
- D\. Carlson and J\. M\. Montgomery \(2017\)A pairwise comparison framework for fast, flexible, and reliable human coding of political texts\.American Political Science Review111\(4\),pp\. 835–843\.External Links:[Document](https://dx.doi.org/10.1017/S0003055417000302)Cited by:[§6](https://arxiv.org/html/2607.01464#S6.p3.1)\.
- T\. Ceron, N\. Blokker, and S\. Padó \(2022\)Optimizing text representations to capture \(dis\)similarity between political parties\.InProceedings of the 26th Conference on Computational Natural Language Learning \(CoNLL\),A\. Fokkens and V\. Srikumar \(Eds\.\),Abu Dhabi, United Arab Emirates \(Hybrid\),pp\. 325–338\.External Links:[Link](https://aclanthology.org/2022.conll-1.22/),[Document](https://dx.doi.org/10.18653/v1/2022.conll-1.22)Cited by:[§A\.1](https://arxiv.org/html/2607.01464#A1.SS1.p1.3),[§1](https://arxiv.org/html/2607.01464#S1.p3.1),[item 2](https://arxiv.org/html/2607.01464#S2.I1.i2.p1.3),[§2\.3](https://arxiv.org/html/2607.01464#S2.SS3.SSS0.Px2.p3.1),[§3\.2](https://arxiv.org/html/2607.01464#S3.SS2.SSS0.Px2.p1.1),[§4\.2](https://arxiv.org/html/2607.01464#S4.SS2.p2.1),[§5](https://arxiv.org/html/2607.01464#S5.p2.1)\.
- T\. Ceron, D\. Nikolaev, and S\. Padó \(2023\)Additive manifesto decomposition: a policy domain aware method for understanding party positioning\.InFindings of the Association for Computational Linguistics: ACL 2023,A\. Rogers, J\. Boyd\-Graber, and N\. Okazaki \(Eds\.\),Toronto, Canada,pp\. 7874–7890\.External Links:[Link](https://aclanthology.org/2023.findings-acl.499/),[Document](https://dx.doi.org/10.18653/v1/2023.findings-acl.499)Cited by:[§5](https://arxiv.org/html/2607.01464#S5.p2.1)\.
- T\. Dao, D\. Fu, S\. Ermon, A\. Rudra, and C\. Ré \(2022\)Flashattention: fast and memory\-efficient exact attention with io\-awareness\.Advances in neural information processing systems35,pp\. 16344–16359\.Cited by:[§3\.3](https://arxiv.org/html/2607.01464#S3.SS3.SSS0.Px3.p1.1)\.
- E\. Dayanik, A\. Blessing, N\. Blokker, S\. Haunss, J\. Kuhn, G\. Lapesa, and S\. Pado \(2022\)Improving neural political statement classification with class hierarchical information\.InFindings of the Association for Computational Linguistics: ACL 2022,S\. Muresan, P\. Nakov, and A\. Villavicencio \(Eds\.\),Dublin, Ireland,pp\. 2367–2382\.External Links:[Link](https://aclanthology.org/2022.findings-acl.186/),[Document](https://dx.doi.org/10.18653/v1/2022.findings-acl.186)Cited by:[§5](https://arxiv.org/html/2607.01464#S5.p2.1),[§6](https://arxiv.org/html/2607.01464#S6.p2.1)\.
- J\. Devlin, M\. Chang, K\. Lee, and K\. Toutanova \(2019\)BERT: pre\-training of deep bidirectional transformers for language understanding\.InProceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 \(Long and Short Papers\),J\. Burstein, C\. Doran, and T\. Solorio \(Eds\.\),Minneapolis, Minnesota,pp\. 4171–4186\.External Links:[Link](https://aclanthology.org/N19-1423/),[Document](https://dx.doi.org/10.18653/v1/N19-1423)Cited by:[§3\.2](https://arxiv.org/html/2607.01464#S3.SS2.SSS0.Px3.p1.1)\.
- B\. Efron and R\. J\. Tibshirani \(1994\)An introduction to the bootstrap\.Chapman and Hall/CRC\.Cited by:[§A\.4](https://arxiv.org/html/2607.01464#A1.SS4.p1.2),[§3\.5](https://arxiv.org/html/2607.01464#S3.SS5.p2.3)\.
- A\. Ettinger, A\. Bertsch, B\. Kuehl, D\. Graham, D\. Heineman, D\. Groeneveld, F\. Brahman, F\. Timbers, H\. Ivison, J\. Morrison, J\. Poznanski, K\. Lo, L\. Soldaini, M\. Jordan, M\. Chen, M\. Noukhovitch, N\. Lambert, P\. Walsh, P\. Dasigi, R\. Berry, S\. Malik, S\. Shah, S\. Geng, S\. Arora, S\. Gupta, T\. Anderson, T\. Xiao, T\. Murray, T\. Romero, V\. Graf, A\. Asai, A\. Bhagia, A\. Wettig, A\. Liu, A\. Rangapur, C\. Anastasiades, C\. Huang, D\. Schwenk, H\. Trivedi, I\. Magnusson, J\. Lochner, J\. Liu, L\. J\. V\. Miranda, M\. Sap, M\. Morgan, M\. Schmitz, M\. Guerquin, M\. Wilson, R\. Huff, R\. L\. Bras, R\. Xin, R\. Shao, S\. Skjonsberg, S\. Z\. Shen, S\. S\. Li, T\. Wilde, V\. Pyatkin, W\. Merrill, Y\. Chang, Y\. Gu, Z\. Zeng, A\. Sabharwal, L\. Zettlemoyer, P\. W\. Koh, A\. Farhadi, N\. A\. Smith, and H\. Hajishirzi \(2026\)Olmo 3\.External Links:2512\.13961,[Link](https://arxiv.org/abs/2512.13961)Cited by:[§3\.4](https://arxiv.org/html/2607.01464#S3.SS4.p1.1)\.
- G\. Glavaš, F\. Nanni, and S\. P\. Ponzetto \(2017\)Unsupervised cross\-lingual scaling of political texts\.InProceedings of the 15th Conference of the European Chapter of the Association for Computational Linguistics: Volume 2, Short Papers,M\. Lapata, P\. Blunsom, and A\. Koller \(Eds\.\),Valencia, Spain,pp\. 688–693\.External Links:[Link](https://aclanthology.org/E17-2109/)Cited by:[§1](https://arxiv.org/html/2607.01464#S1.p3.1),[§5](https://arxiv.org/html/2607.01464#S5.p1.1)\.
- P\. He, J\. Gao, and W\. Chen \(2023\)DeBERTav3: improving deBERTa using ELECTRA\-style pre\-training with gradient\-disentangled embedding sharing\.InThe Eleventh International Conference on Learning Representations,External Links:[Link](https://openreview.net/forum?id=sE7-XhLxHA)Cited by:[§3\.2](https://arxiv.org/html/2607.01464#S3.SS2.SSS0.Px3.p1.1)\.
- D\. Jahn \(2011\)Conceptualizing left and right in comparative politics: towards a deductive approach\.Party Politics17\(6\),pp\. 745–765\.Cited by:[item RQ1: Does joint prediction improve scaling?](https://arxiv.org/html/2607.01464#S1.I1.ix1.p1.1)\.
- J\. Koedam, G\. Binding, and M\. R\. Steenbergen \(2025\)Multidimensional party polarization in Europe: cross\-cutting divides and effective dimensionality\.British Journal of Political Science55,pp\. e24\.External Links:[Document](https://dx.doi.org/10.1017/S0007123424000474),[Link](https://doi.org/)Cited by:[§1](https://arxiv.org/html/2607.01464#S1.p4.1)\.
- A\. Kurella and M\. Rapp \(2026\)Unfolding GAL\-TAN: the multi\-dimensional nature of public opinion in western europe\.West European Politics49\(3\),pp\. 841–866\.External Links:[Document](https://dx.doi.org/10.1080/01402382.2025.2466117)Cited by:[§4\.1](https://arxiv.org/html/2607.01464#S4.SS1.p2.2)\.
- M\. Laver, K\. Benoit, and J\. Garry \(2003\)Extracting policy positions from political texts using words as data\.American political science review97\(2\),pp\. 311–331\.Cited by:[§1](https://arxiv.org/html/2607.01464#S1.p1.1),[§1](https://arxiv.org/html/2607.01464#S1.p3.1),[§5](https://arxiv.org/html/2607.01464#S5.p1.1)\.
- G\. Le Mens and A\. Gallego \(2025\)Positioning political texts with large language models by asking and averaging\.Political Analysis33\(3\),pp\. 274–282\.Cited by:[§2\.4](https://arxiv.org/html/2607.01464#S2.SS4.p1.1),[§5](https://arxiv.org/html/2607.01464#S5.p3.1)\.
- A\. Liesenfeld and M\. Dingemanse \(2024\)Rethinking open source generative ai: open\-washing and the eu ai act\.InProceedings of the ACM Conference on Fairness, Accountability, and Transparency,Rio de Janeiro, Brazil,pp\. 1774–1787\.External Links:ISBN 9798400704505,[Link](https://doi.org/10.1145/3630106.3659005),[Document](https://dx.doi.org/10.1145/3630106.3659005)Cited by:[§3\.4](https://arxiv.org/html/2607.01464#S3.SS4.p1.1)\.
- I\. Loshchilov and F\. Hutter \(2019\)Decoupled weight decay regularization\.InInternational Conference on Learning Representations,External Links:[Link](https://openreview.net/forum?id=Bkg6RiCqY7)Cited by:[§A\.1](https://arxiv.org/html/2607.01464#A1.SS1.p1.3)\.
- G\. Marks and M\. R\. Steenbergen \(2004\)European integration and political conflict\.Cambridge University Press\.Cited by:[item RQ1: Does joint prediction improve scaling?](https://arxiv.org/html/2607.01464#S1.I1.ix1.p1.1),[§2\.1](https://arxiv.org/html/2607.01464#S2.SS1.p1.4)\.
- S\. Mikhaylov, M\. Laver, and K\. R\. Benoit \(2012\)Coder reliability and misclassification in the human coding of party manifestos\.Political analysis20\(1\),pp\. 78–91\.Cited by:[§4\.2](https://arxiv.org/html/2607.01464#S4.SS2.SSS0.Px1.p2.3)\.
- S\. Minaee, T\. Mikolov, N\. Nikzad, M\. Chenaghlu, R\. Socher, X\. Amatriain, and J\. Gao \(2025\)Large Language Models: A Survey\.External Links:2402\.06196,[Link](https://arxiv.org/abs/2402.06196)Cited by:[§5](https://arxiv.org/html/2607.01464#S5.p3.1)\.
- F\. Nanni, G\. Glavaš, I\. Rehbein, S\. P\. Ponzetto, and H\. Stuckenschmidt \(2022\)Political text scaling meets computational semantics\.ACM/IMS Trans\. Data Sci\.2\(4\)\.External Links:ISSN 2691\-1922,[Link](https://doi.org/10.1145/3485666),[Document](https://dx.doi.org/10.1145/3485666)Cited by:[§5](https://arxiv.org/html/2607.01464#S5.p1.1)\.
- D\. Nikolaev, T\. Ceron, and S\. Padó \(2023\)Multilingual estimation of political\-party positioning: from label aggregation to long\-input transformers\.InProceedings of the 2023 Conference on Empirical Methods in Natural Language Processing,H\. Bouamor, J\. Pino, and K\. Bali \(Eds\.\),Singapore,pp\. 9497–9511\.External Links:[Link](https://aclanthology.org/2023.emnlp-main.591/),[Document](https://dx.doi.org/10.18653/v1/2023.emnlp-main.591)Cited by:[§A\.1](https://arxiv.org/html/2607.01464#A1.SS1.p1.3),[item RQ2: How to deal with long input documents?](https://arxiv.org/html/2607.01464#S1.I1.ix2.p1.1),[§1](https://arxiv.org/html/2607.01464#S1.p3.1),[§2\.1](https://arxiv.org/html/2607.01464#S2.SS1.p1.4),[§2\.2](https://arxiv.org/html/2607.01464#S2.SS2.p1.1),[§3\.1](https://arxiv.org/html/2607.01464#S3.SS1.p1.1),[§3\.1](https://arxiv.org/html/2607.01464#S3.SS1.p2.1),[§3\.2](https://arxiv.org/html/2607.01464#S3.SS2.SSS0.Px2.p1.1),[§3\.2](https://arxiv.org/html/2607.01464#S3.SS2.p1.1),[§3\.3](https://arxiv.org/html/2607.01464#S3.SS3.SSS0.Px2.p1.1),[§3\.5](https://arxiv.org/html/2607.01464#S3.SS5.p1.1),[Table 1](https://arxiv.org/html/2607.01464#S3.T1),[§4\.1](https://arxiv.org/html/2607.01464#S4.SS1.p1.1),[§4\.1](https://arxiv.org/html/2607.01464#S4.SS1.p4.1),[§6](https://arxiv.org/html/2607.01464#S6.p2.1),[Limitations](https://arxiv.org/html/2607.01464#Sx1.p1.1)\.
- J\. T\. Ornstein, E\. N\. Blasingame, and J\. S\. Truscott \(2025\)How to train your stochastic parrot: large language models for political texts\.Political Science Research and Methods13\(2\),pp\. 264–281\.Cited by:[§2\.4](https://arxiv.org/html/2607.01464#S2.SS4.p1.1)\.
- N\. Reimers and I\. Gurevych \(2019\)Sentence\-BERT: sentence embeddings using Siamese BERT\-networks\.InProceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing \(EMNLP\-IJCNLP\),K\. Inui, J\. Jiang, V\. Ng, and X\. Wan \(Eds\.\),Hong Kong, China,pp\. 3982–3992\.External Links:[Link](https://aclanthology.org/D19-1410/),[Document](https://dx.doi.org/10.18653/v1/D19-1410)Cited by:[§2\.3](https://arxiv.org/html/2607.01464#S2.SS3.SSS0.Px2.p1.1),[§3\.2](https://arxiv.org/html/2607.01464#S3.SS2.SSS0.Px2.p1.1)\.
- L\. Rheault and C\. Cochrane \(2020\)Word embeddings for the analysis of ideological placement in parliamentary corpora\.Political Analysis28\(1\),pp\. 112–133\.External Links:[Document](https://dx.doi.org/10.1017/pan.2019.26)Cited by:[§1](https://arxiv.org/html/2607.01464#S1.p3.1),[§5](https://arxiv.org/html/2607.01464#S5.p1.1)\.
- J\. Rovny, J\. Polk, R\. Bakker, L\. Hooghe, S\. Jolly, G\. Marks, M\. Steenbergen, and M\. A\. Vachudova \(2025\)The 2024 chapel hill expert survey on political party positioning in europe: twenty\-five years of party positional data\.Electoral Studies97,pp\. 102981\.External Links:ISSN 0261\-3794,[Document](https://dx.doi.org/https%3A//doi.org/10.1016/j.electstud.2025.102981),[Link](https://www.sciencedirect.com/science/article/pii/S0261379425000873)Cited by:[§1](https://arxiv.org/html/2607.01464#S1.p2.1)\.
- F\. Schroff, D\. Kalenichenko, and J\. Philbin \(2015\)Facenet: a unified embedding for face recognition and clustering\.InProceedings of the IEEE conference on computer vision and pattern recognition,pp\. 815–823\.Cited by:[§2\.3](https://arxiv.org/html/2607.01464#S2.SS3.SSS0.Px2.p1.1)\.
- J\. B\. Slapin and S\. Proksch \(2008\)A scaling model for estimating time\-series party positions from texts\.American Journal of Political Science52\(3\),pp\. 705–722\.Cited by:[§1](https://arxiv.org/html/2607.01464#S1.p3.1),[§5](https://arxiv.org/html/2607.01464#S5.p1.1)\.
- K\. Song, X\. Tan, T\. Qin, J\. Lu, and T\. Liu \(2020\)MPNet: masked and permuted pre\-training for language understanding\.InAdvances in Neural Information Processing Systems,H\. Larochelle, M\. Ranzato, R\. Hadsell, M\.F\. Balcan, and H\. Lin \(Eds\.\),Vol\.33,pp\. 16857–16867\.External Links:[Link](https://proceedings.neurips.cc/paper_files/paper/2020/file/c3a690be93aa602ee2dc0ccab5b7b67e-Paper.pdf)Cited by:[§3\.2](https://arxiv.org/html/2607.01464#S3.SS2.SSS0.Px2.p1.1)\.
- J\. Su, J\. Cao, W\. Liu, and Y\. Ou \(2021\)Whitening sentence representations for better semantics and faster retrieval\.External Links:2103\.15316,[Link](https://arxiv.org/abs/2103.15316)Cited by:[§2\.3](https://arxiv.org/html/2607.01464#S2.SS3.SSS0.Px2.p3.1)\.
- A\. Volkens, J\. Bara, I\. Budge, M\. D\. McDonald, R\. Best, and S\. Franzmann \(2013\)Understanding and validating the left\-right scale \(rile\)\.InMapping Policy Preferences From Texts: Statistical Solutions for Manifesto Analysts,External Links:ISBN 9780199640041,[Document](https://dx.doi.org/10.1093/acprof%3Aoso/9780199640041.003.0006),[Link](https://doi.org/10.1093/acprof:oso/9780199640041.003.0006),https://academic\.oup\.com/book/0/chapter/197751501/chapter\-ag\-pdf/44604447/book\_27649\_section\_197751501\.ag\.pdfCited by:[§1](https://arxiv.org/html/2607.01464#S1.p2.1),[§2\.1](https://arxiv.org/html/2607.01464#S2.SS1.p1.4)\.
- W\. Wagner, L\. Pelaez, T\. Raunio, and M\. van de Koppel \(2021\)The party politics of the eu’s relations with the usa: evidence from the european parliament\.European security30\(3\),pp\. 418–438\.Cited by:[item RQ1: Does joint prediction improve scaling?](https://arxiv.org/html/2607.01464#S1.I1.ix1.p1.1)\.
- B\. Warner, A\. Chaffin, B\. Clavié, O\. Weller, O\. Hallström, S\. Taghadouini, A\. Gallagher, R\. Biswas, F\. Ladhak, T\. Aarsen, G\. T\. Adams, J\. Howard, and I\. Poli \(2025\)Smarter, better, faster, longer: a modern bidirectional encoder for fast, memory efficient, and long context finetuning and inference\.InProceedings of the 63rd Annual Meeting of the Association for Computational Linguistics \(Volume 1: Long Papers\),W\. Che, J\. Nabende, E\. Shutova, and M\. T\. Pilehvar \(Eds\.\),Vienna, Austria,pp\. 2526–2547\.External Links:[Link](https://aclanthology.org/2025.acl-long.127/),[Document](https://dx.doi.org/10.18653/v1/2025.acl-long.127),ISBN 979\-8\-89176\-251\-0Cited by:[§2\.2](https://arxiv.org/html/2607.01464#S2.SS2.p1.1),[§3\.2](https://arxiv.org/html/2607.01464#S3.SS2.SSS0.Px3.p1.1)\.
- X\. Wu, Y\. Wang, S\. Jegelka, and A\. Jadbabaie \(2025\)On the emergence of position bias in transformers\.InProceedings of the 42nd International Conference on Machine Learning,ICML’25\.Cited by:[§4\.3](https://arxiv.org/html/2607.01464#S4.SS3.SSS0.Px1.p2.1)\.
- J\. Yu, X\. Liu, C\. Luo, R\. Zhou, Y\. Liu, J\. Hu, J\. Chen, K\. Zhang, D\. Zhang, Y\. Shen, E\. Adhikarla, Y\. Dai, K\. Zhang, Z\. Kong, W\. Ye, Y\. Yin, V\. Namboodiri, B\. Davison, J\. Moore, and Y\. Chen \(2025\)Multitask Learning 1997–2024: Part II Regularization and Optimization\.Harvard Data Science Review7\(3\)\.Note:https://hdsr\.mitpress\.mit\.edu/pub/7fcc3jhvCited by:[§2\.3](https://arxiv.org/html/2607.01464#S2.SS3.SSS0.Px1.p1.1)\.
- M\. Zaheer, G\. Guruganesh, K\. A\. Dubey, J\. Ainslie, C\. Alberti, S\. Ontanon, P\. Pham, A\. Ravula, Q\. Wang, L\. Yang,et al\.\(2020\)Big bird: transformers for longer sequences\.Advances in neural information processing systems33,pp\. 17283–17297\.Cited by:[§3\.3](https://arxiv.org/html/2607.01464#S3.SS3.SSS0.Px2.p1.1)\.
- X\. Zhang, Y\. Zhang, D\. Long, W\. Xie, Z\. Dai, J\. Tang, H\. Lin, B\. Yang, P\. Xie, F\. Huang, M\. Zhang, W\. Li, and M\. Zhang \(2024\)mGTE: generalized long\-context text representation and reranking models for multilingual text retrieval\.InProceedings of the 2024 Conference on Empirical Methods in Natural Language Processing: Industry Track,F\. Dernoncourt, D\. Preoţiuc\-Pietro, and A\. Shimorina \(Eds\.\),Miami, Florida, US,pp\. 1393–1412\.External Links:[Link](https://aclanthology.org/2024.emnlp-industry.103/),[Document](https://dx.doi.org/10.18653/v1/2024.emnlp-industry.103)Cited by:[§3\.2](https://arxiv.org/html/2607.01464#S3.SS2.SSS0.Px3.p1.1)\.
- L\. Zhuang, L\. Wayne, S\. Ya, and Z\. Jun \(2021\)A robustly optimized BERT pre\-training approach with post\-training\.InProceedings of the 20th Chinese National Conference on Computational Linguistics,S\. Li, M\. Sun, Y\. Liu, H\. Wu, K\. Liu, W\. Che, S\. He, and G\. Rao \(Eds\.\),Huhhot, China,pp\. 1218–1227\(eng\)\.External Links:[Link](https://aclanthology.org/2021.ccl-1.108/)Cited by:[§3\.3](https://arxiv.org/html/2607.01464#S3.SS3.SSS0.Px2.p1.1)\.

## Appendix AExperimental Setup

### A\.1Training Hyperparameters

FollowingNikolaevet al\.\([2023](https://arxiv.org/html/2607.01464#bib.bib14)\), all supervised models are trained for 5 epochs using the AdamW optimizer\(Loshchilov and Hutter,[2019](https://arxiv.org/html/2607.01464#bib.bib29)\)with early stopping\. The learning rate is10−510^\{\-5\}for individual and joint multitask prediction, and5⋅10−55\\cdot 10^\{\-5\}for training the classification heads after joint contrastive tuning\. The margin hyperparameter in the triplet loss function \([3](https://arxiv.org/html/2607.01464#S2.E3)\) is set to11based on preliminary experiments\. FollowingCeronet al\.\([2022](https://arxiv.org/html/2607.01464#bib.bib25)\), we use Euclidean distance as the distance metric and employ a linear learning rate schedule with 100 warmup steps\. The mini\-batch size is 256 for label aggregation, except for contrastive tuning where it is set to 16, and 4 for chunk\-level regression\.

### A\.2LLM Prompts

#### RILE

Question: What political position is expressed in this statement? Statement: <sentence\> Option A: Right\-wing Option B: Left\-wing Option C: Neutral Keep your response short \(up to 10 words\) by choosing exactly one option\! Correct option:

#### GAL\-TAN

Question: What political position is expressed in this statement? Statement: <sentence\> Option A: Green\-Alternative\-Liberal Option B: Traditional\-Authoritarian\-Nationalist Option C: Neutral Keep your response short \(up to 10 words\) by choosing exactly one option\! Correct option:

#### Joint Prompting

Question: What political position is expressed in this statement? Statement: <sentence\> Choose exactly one option from each of the two lists below\. List 1 \(economic policy\): Option A: Right\-wing Option B: Left\-wing Option C: Neutral List 2 \(socio\-cultural policy\): Option D: Green\-Alternative\-Liberal Option E: Traditional\-Authoritarian\-Nationalist Option F: Neutral Keep your response short \(up to 10 words\)\! Correct options:

### A\.3Chunk Sizes

Table[2](https://arxiv.org/html/2607.01464#A1.T2)gives an overview of the chunk sizes evaluated in the chunk size experiment on ModernBERT joint regression\.

Train time /epoch\(hh:mm\)101010On a GPU server Nvidia RTX 6000 Ada, 48 GB\.11735233202:405515681131601:15202033514511600:40504942971117800:401009750562206400:4020018981734281200:5067631581927153401:10Table 2:Chunk length statistics in the sentence\-chunk smoothing experiment with ModernBERT\. The values delimiting the chunk length are highlighted in bold\.
### A\.4Statistical Significance Testing

Whenever comparing models with less than 15 percentage points of gap in performance, we run thebootstrap resampling testto see if the difference is statistically significant\(Efron and Tibshirani,[1994](https://arxiv.org/html/2607.01464#bib.bib62)\)\. This test is non\-parametric and thus applicable to any metric, including the rank correlation\. Given two modelsm1m\_\{1\}andm2m\_\{2\}, the procedure is run as follows\.

1. 1\.The test set manifestos are sampled with replacement, the size of each sample the same as that of the original test set\. The predictions ofm1m\_\{1\}andm2m\_\{2\}for those manifestos as well as the ground truth scores are compiled accordingly\.
2. 2\.For each model and random seed, the rank correlation of the predictions with the ground truth is calculated\. The resulting values are averaged over the random seeds, producingρ1\\rho\_\{1\}andρ2\\rho\_\{2\}to representm1m\_\{1\}andm2m\_\{2\}, respectively\. Their differenceρ1−ρ2\\rho\_\{1\}\-\\rho\_\{2\}is the variable of interest\.
3. 3\.The operations in \(1\) and \(2\) are repeatedn=10,000n=10,000times, creating a distribution of\{ρ1−ρ2\}\\\{\\rho\_\{1\}\-\\rho\_\{2\}\\\}\.
4. 4\.The2,52,5and the97,597,5percentile of that distribution are calculated, producing a95%95\\%confidence interval\[a,b\]\[a,b\]\. It is interpreted as follows: - •0∈\[a,b\]0\\in\[a,b\]—m1m\_\{1\}andm2m\_\{2\}perform on a par - •0<a0<a—m1m\_\{1\}performs better thanm2m\_\{2\} - •0\>b0\>b—m1m\_\{1\}performs worse thanm2m\_\{2\}

## Appendix BExtended Results

### B\.1Label Aggregation

Table[3](https://arxiv.org/html/2607.01464#A2.T3)presents a detailed evaluation of the label aggregation models, additionally reporting accuracy, weighted F1\-score and macro F1\-score computed at the sentence classification level\. The macro\-F1 values are consistently lower than the accuracy and weighted F1 \(0\.75/0\.820\.75/0\.82vs0\.66/0\.670\.66/0\.67\)\. Yet, among the classification\-level metrics, macro\-F1 is the most reliable predictor for the manifesto\-level rank correlationρ\\rho, which highlights the importance of evaluating all classes equally for a fair view of the model quality\.

The SBERT and ModernBERT baselines have the encoder weights frozen and only the classification heads trained during fine\-tuning\. The baselines reveal that ModernBERT has a weaker starting point but catches up to SBERT when trained for individual or joint prediction\. Notably, the SBERT baseline is more robust than the LLM baseline on both RILE and GAL\-TAN\.

The majority prediction baseline always outputs the labelOther, as it is the most prevalent in the training data for both RILE and GAL\-TAN\. For this method, the rank correlation cannot be calculated, since that requires dividing by the covariance which equals zero for a constant series \(all of the manifesto\-level estimates equal 1; cf\. eq\. \([1](https://arxiv.org/html/2607.01464#S2.E1),[2](https://arxiv.org/html/2607.01464#S2.E2)\)\)\.

Table 3:The performance of label aggregation \(primary overview\)\.Joint contrast\. RLGTrefers to joint contrastive tuning with RILE and GAL\-TAN based triplets;whitenindicates the whitening transformation\.w\. F1andm\. F1are the weighted F1 and macro F1, respectively\.
### B\.2Joint Contrastive Training

Table[4](https://arxiv.org/html/2607.01464#A2.T4)allows a more detailed look into joint contrastive training\. It achieves the rank correlation of at mostρ=0\.83\\rho=0\.83andρ=0\.77\\rho=0\.77on RILE and GAL\-TAN, respectively\. The most robust way to select triplets for training is based on the RILE and GAL\-TAN category labels\. SBERT–RILE–GAL\-TANwhitenand ModernBERT–RILE–GAL\-TAN perform on a par in that setting\. In particular, layering in the whitening transformation allows SBERT to get a small but statistically significant boost on RILE \(ρ=0\.83\\rho=0\.83vs0\.810\.81\), whereas on GAL\-TAN the difference inρ\\rhois insignificant\. In contrast, the whitening transformation dramatically impairs the downstream classification quality of ModernBERT–RILE–GAL\-TAN\.

Mining triplets based on party makes for inferior RILE and GAL\-TAN classifiers\. SBERT–Party, even with the aid of the whitening transformation, scores significantly lower even than the baseline SBERT where only the classification heads were trained\. With slightly more success, ModernBERT–Party improves over the ModernBERT baseline on RILE and scores on a par on GAL\-TAN\. Again, whitening the embeddings causes a decrease in the ModernBERT scores\.

Table 4:Performance of label aggregation with joint contrastive tuning\. Apart from the baselines, the second column states the triplet mining strategy;whitenis the whitening transformation\.
### B\.3Chunk\-Level Regression

The chunk\-level regression models are additionally assessed on the MSE value at the manifesto level\. Its dynamics mostly correspond to those of the rank correlation score\.

RILEGAL\-TANMSE𝝆\\bm\{\\rho\}MSE𝝆\\bm\{\\rho\}BigBirdBaseline0\.0180\.610\.0150\.58Individual prediction0\.0080\.840\.0060\.84Joint multitask0\.0090\.840\.0070\.82ModernBERTBaseline0\.0240\.450\.0150\.33Individual prediction0\.0090\.790\.0070\.78Joint multitask0\.0130\.830\.0080\.79Table 5:Performance of chunk\-level regression\.