Making Political Text Scaling Comparable: Infrastructure and Hyperparameter Sensitivity for 17 Algorithms

# Infrastructure and Hyperparameter Sensitivity for 17 Algorithms
Source: [https://arxiv.org/html/2609.17602](https://arxiv.org/html/2609.17602)
## Making Political Text Scaling Comparable: Infrastructure and Hyperparameter Sensitivity for 17 Algorithms

Patrick ParschanAffiliation:Department of Media and CommunicationAffiliation:LMU MunichEmail:[p\.parschan@lmu\.de](mailto:)

###### Abstract

Computational text\-based ideal point estimation \(CT\-IPE\) methods are usually compared as named algorithms, yet applying them involves numerous researcher choices that configure how political text is turned into position estimates\. This paper argues that CT\-IPE methods are better understood as configurable measurement pipelines than as fixed estimators\. Building on a large\-scale comparative experiment spanning 17 CT\-IPE algorithms, 5,537 experimental runs, and approximately 4\.25 million left–right position estimates, I describe the shared infrastructure that makes these heterogeneous methods jointly executable and quantify how sensitive their estimates are to alternative hyperparameter choices\. Variance\-partitioning and SHAP\-based sensitivity analyses show that, for most algorithms, hyperparameter profiles explain little residual variance through a shared shift: 13 of the 17 algorithms exhibit ICC values below \.10\. Where this profile\-level sensitivity is present, it is concentrated in a small number of consequential researcher choices, most notably the selection of the underlying language or embedding model, the seed keyword lists that anchor the construct, and the number of topics\.

## 1Introduction

Computational text\-based ideal point estimation \(CT\-IPE\), also known as political text scaling, estimates numerical political positions from text, most commonly on dimensions such as the left–right axis\([Carroll, 2023](https://arxiv.org/html/2609.17602#bib.bib36);[Bafumi et al\., 2005](https://arxiv.org/html/2609.17602#bib.bib35)\)\. Over the past three decades, the field has expanded from classical word\-frequency models such as Wordscores\([Laver et al\., 2003](https://arxiv.org/html/2609.17602#bib.bib33)\)and Wordfish\([Slapin and Proksch, 2008](https://arxiv.org/html/2609.17602#bib.bib20)\)to topic models\([Vafa et al\., 2020](https://arxiv.org/html/2609.17602#bib.bib40), e\.g\.,\), embedding\-based approaches\([Rheault and Cochrane, 2020](https://arxiv.org/html/2609.17602#bib.bib41), e\.g\.,\), and, more recently, large language model \(LLM\)\-based methods\([Le Mens and Gallego, 2025](https://arxiv.org/html/2609.17602#bib.bib13), e\.g\.,\)\. As this landscape has grown, methodological research has primarily compared algorithms with respect to agreement with humans\([Bruinsma and Gemenis, 2019](https://arxiv.org/html/2609.17602#bib.bib31);[Hjorth et al\., 2015](https://arxiv.org/html/2609.17602#bib.bib29)\)or – more limitedly – with respect to agreement with other algorithms\([Egerod and Klemmensen, 2020](https://arxiv.org/html/2609.17602#bib.bib23)\)\. The named algorithm has consequently become the central unit of methodological investigation\.

However, applying a CT\-IPE method involves considerably more than selecting an algorithm\. Researchers must also decide how political constructs are operationalized, how texts are represented, how computational models are configured, how external information is incorporated, and how political positions are estimated\. Although these choices are documented within individual algorithms, they are rarely conceptualized as recurring methodological decisions that span otherwise heterogeneous CT\-IPE approaches\. As a result, conceptually similar researcher choices remain difficult to identify and compare across algorithms\. This raises two research questions:

#### RQ1:

How can heterogeneous CT\-IPE algorithms be made jointly executable and systematically comparable within a shared experimental infrastructure?

#### RQ2:

What patterns of researcher choice emerge from a systematic comparison of heterogeneous CT\-IPE algorithms?

Throughout,*researcher choices*refers to the configurable decisions involved in applying a CT\-IPE method to data—hyperparameters, construct anchors, model selection, and related pipeline settings—not to the author’s selection of which algorithms to include in the study\.

To address these questions, this paper argues that CT\-IPE methods are better understood as configurable measurement pipelines than as fixed estimators\. Rather than comparing algorithms directly, I compare the researcher choices that configure them\. I build on a large\-scale computational experiment that executes 17 CT\-IPE algorithms under a shared design, describe the software infrastructure that makes these heterogeneous methods jointly runnable, and quantify how strongly alternative hyperparameter settings influence the resulting position estimates, using variance partitioning and SHAP\-based \(SHapley Additive exPlanations, see[Lundberg and Lee \(2017\)](https://arxiv.org/html/2609.17602#bib.bib1)\) sensitivity analysis\.

The results show that, for most CT\-IPE algorithms, alternative hyperparameter profiles explain little of the remaining variance through a shared shift within the evaluated specification space, and that where this profile\-level sensitivity is present, it is concentrated in a small number of consequential researcher choices\.

## 2Researcher Choices in CT\-IPE

CT\-IPE methods are usually discussed as named algorithms: Wordscores, Wordfish, Party Embeddings, TBIP, LaMP, or other approaches\.111For readability, I use only abbreviations \(rather than full algorithm names\) throughout the text\. A complete list of all algorithms with their full names, corresponding abbreviations, and relevant papers is provided in Appendix[A](https://arxiv.org/html/2609.17602#A1)\.This algorithm\-centered vocabulary is useful, but it hides an important methodological feature of applied CT\-IPE: algorithms are configured through researcher choices\. These choices determine how political text is transformed into position estimates\.

I use the term researcher choice or researcher decision to refer to a theoretically or methodologically justifiable decision that may alter estimated political positions\. This definition is broader than the notion of a software hyperparameter\. A single researcher choice may be implemented through multiple parameters within one algorithm and through different numbers or combinations of parameters in other algorithms\. Conversely, some implementation parameters are purely technical and do not constitute substantive researcher choices\. This perspective is consistent with machine learning \(ML\) and Natural Language Processing \(NLP\) research showing that model behavior depends not only on the learning algorithm itself but also on configurable choices throughout the modeling pipeline\([Bouthillier et al\., 2021](https://arxiv.org/html/2609.17602#bib.bib15);[Yang and Shami, 2020](https://arxiv.org/html/2609.17602#bib.bib46), e\.g\.,\)\. The analyses below therefore treat hyperparameters as part of the measurement procedure rather than as technical implementation details, and examine them at the level of individual researcher choices\.

## 3Methods

### 3\.1Comparative Experiment

To investigate researcher choices empirically, I build on a large\-scale comparative experiment that evaluates CT\-IPE algorithms under a shared design\. The experiment comprises 17 unsupervised and semi\-supervised CT\-IPE algorithms spanning four broad methodological types identified in the systematic review of[Parschan and Jakob \(2025\)](https://arxiv.org/html/2609.17602#bib.bib11): word frequency\-based methods \(Type I\), topic modeling\-based methods \(Type II\), word embedding\-based methods \(Type III\), and LLM\-based methods \(Type IV\)\. The included algorithms are the unsupervised and semi\-supervised CT\-IPE methods covered by that review \(see Appendix[A](https://arxiv.org/html/2609.17602#A1)\)\. Network\-based approaches that infer ideology from non\-textual signals \(e\.g\.[Barberá et al\. \(2015\)](https://arxiv.org/html/2609.17602#bib.bib47)\) fall outside this text\-based scope\.

The algorithms are applied to estimate left–right positions in six corpora consisting of party manifestos, parliamentary speeches, and political tweets from Germany and the United States \(see Table[1](https://arxiv.org/html/2609.17602#S3.T1)\)\. These textual genres are those for which CT\-IPE methods have most frequently been developed and applied\([Parschan and Jakob, 2025](https://arxiv.org/html/2609.17602#bib.bib11)\)\.

Table 1:Datasets overview\. Number of parties and documents per corpus\.Section[3\.2](https://arxiv.org/html/2609.17602#S3.SS2)describes how the shared implementation pipeline makes these heterogeneous methods jointly executable under one comparative design\.

The resulting experiment comprises 5,537 attempted runs, of which 4,565 completed successfully, producing approximately 4\.25 million political position estimates\. These provide the empirical basis for the analyses presented below\. Summary statistics for the runs can be found in Appendix[B](https://arxiv.org/html/2609.17602#A2)\. Run counts differ considerably across algorithms because hyperparameter grids differ in size and because runtime costs limited the number of evaluated configurations for computationally expensive methods\. Each run corresponds to a configuration of implementation parameter values drawn from the experimental grid, which together instantiate a specific configuration of researcher choices\.

The evaluated specification space was designed to be as comprehensive as possible: implementation parameters and their candidate values were systematically derived from the CT\-IPE literature review of[Parschan and Jakob \(2025\)](https://arxiv.org/html/2609.17602#bib.bib11)\. Different experimental designs or additional researcher choices may nevertheless yield different sensitivity profiles\.

### 3\.2Code Implementation

Making 17 heterogeneous CT\-IPE methods executable under a shared comparative design required substantial software engineering\. The methods were originally developed as standalone tools with corpus\-specific assumptions, heterogeneous APIs, and incompatible output formats\. The project therefore executes all compared algorithms through algorithm\-specific Python wrappers that connect their existing implementations to the shared corpora, metadata, and hyperparameter grid\. The wrappers are not intended to reimplement the algorithms or standardize their internal estimation logic; their purpose is to make heterogeneous methods jointly runnable while preserving native implementations\.

Native algorithm outputs are retained as produced during inference\. Only during analysis are document\-level estimates aggregated to party\-\-source cells where necessary and estimates robustly standardized to a common scale\. The sensitivity analysis below uses these analysis\-time transforms but leaves each run’s native polarity unchanged\.222Polarity is the orientation of a left–right scale: which end corresponds to left and which to right\. Several unsupervised estimators identify the axis only up to sign, so otherwise similar runs can place left parties at opposite ends\. Native polarity is this sign as the algorithm emitted it\.A robustness check additionally aligns polarity per run using fixed anchor parties, so that the right\-anchor party receives the larger value \(Section[4\.2](https://arxiv.org/html/2609.17602#S4.SS2)\)\. This keeps the original method outputs distinct from the quantities that enter the variance decomposition\.

A fully literal replication of every original implementation was not always possible: some methods were developed for specific corpora, model versions, APIs, preprocessing choices, or hardware environments\. Where exact replication would have made the broader comparison infeasible, I introduced bounded modifications of the original setup\.333These modifications are documented inMODIFICATIONS\.mdin the online repository \(Appendix[F](https://arxiv.org/html/2609.17602#A6)\)\.

Three engineering lessons proved especially consequential for comparative CT\-IPE work\. First, a thin wrapper around each native estimator is more maintainable than a forced common API: methods differ too sharply in inputs \(document–term matrices, embeddings, prompts, pairwise rankings\) for a single interface, yet they can still share corpora, metadata, logging, and result storage\. Second, configuration validation must be treated as part of the experimental record\. Unknown keys in the grid used to be dropped before execution instead of failing; configured and executed conditions could therefore silently diverge\. Unknown keys are now rejected at planning time so that the written grid and the runs stay aligned \(see the Limitations section\)\. Third, resource heterogeneity is itself a researcher constraint: runtimes range from seconds to tens of minutes per configuration, so equal\-sized grids are often infeasible and uneven executed samples are an expected property of a joint design rather than a defect of it \(Appendix[B](https://arxiv.org/html/2609.17602#A2)\)\.

The CPU system used for running and developing the code was equipped with 32 vCPUs \(AMD EPYC 7H12\) and 125 GB RAM\. The GPU system used for inference was equipped with an RTX PRO 6000 with 96 GB VRAM and the same CPU and RAM settings\. The machines were rented via[https://runpod\.io](https://runpod.io/)\. Costs for development and inference amounted to $1,125\.

Taken together, this shared execution layer answers RQ1: heterogeneous CT\-IPE algorithms become jointly executable and systematically comparable through algorithm\-specific wrappers around preserved native implementations, a shared configuration grid, and analysis\-time standardization that is kept separate from native outputs\.

## 4Results

Section[4](https://arxiv.org/html/2609.17602#S4)addresses RQ2\. The empirical analysis proceeds in two steps: a variance decomposition that quantifies profile\-level \(common\-direction\) sensitivity, and a SHAP\-based attribution of that shared shift to individual researcher choices\.

### 4\.1Estimand and Estimation

Estimates are analyzed in a defined experimental context \(see Table[1](https://arxiv.org/html/2609.17602#S3.T1)\)\. Context comprises country, text genre \(data source\), party, result granularity,444Whether the algorithm estimates positions at the document level or directly at the actor \(party\) level\.and time\-period;555Some semi\-supervised algorithms take time\-slicing as an input\.where an algorithm was run with multiple random number seeds, the seed is likewise treated as a contextual factor\. Document\- and actor\-level scores are aggregated to the party in each context and, within each algorithm, robustly standardized using the median and interquartile range\. The unit of analysis is then one standardized left–right score for a party in a given context under one hyperparameter configuration\.

The analysis is conducted separately for each algorithm\. The quantity of interest is how strongly alternative researcher choices move party scores once the experimental setting is held constant\. I group those choices into cleaned hyperparameter profiles: the substantive decisions that define a configuration, after setting aside random seeds that capture only estimation noise and parameters that merely restate the corpus context \(for example, country\-specific prompt prefixes\)\. Letyi​p​cy\_\{ipc\}denote the standardized left–right score for partyppunder profileiiin contextcc\. I estimate

yi​p​c=𝐱c⊤​𝜷\+ui\+εi​p​c,y\_\{ipc\}=\\mathbf\{x\}\_\{c\}^\{\\top\}\\boldsymbol\{\\beta\}\+u\_\{i\}\+\\varepsilon\_\{ipc\},\(1\)where𝐱c\\mathbf\{x\}\_\{c\}accounts for the context factors that vary for that algorithm,uiu\_\{i\}is a shared shift associated with profileii\(varianceτ2\\tau^\{2\}\), andεi​p​c\\varepsilon\_\{ipc\}is leftover variation within profiles \(varianceσ2\\sigma^\{2\}\)\. Hyperparameter sensitivity is the share of this remaining variation that lies between profiles,

ρ=τ2τ2\+σ2,\\rho=\\frac\{\\tau^\{2\}\}\{\\tau^\{2\}\+\\sigma^\{2\}\},\(2\)the intraclass correlation \(ICC\)\. Values near zero mean that, in the same setting, differences between hyperparameter profiles account for little of the remaining variance through a shared shift; values near one mean that those differences account for much of the remaining variance after contextual adjustment\. Becauseuiu\_\{i\}is a common shift, the ICC captures researcher\-choice effects that move parties in the same direction\. Effects that reorder parties or reverse the scale’s polarity are absorbed into the residual; they are examined separately in the polarity\-alignment robustness check \(Section[4\.2](https://arxiv.org/html/2609.17602#S4.SS2)\) and the party landscapes \(Section[4\.3](https://arxiv.org/html/2609.17602#S4.SS3)\)\. A low ICC therefore does not imply that researcher choices cannot rearrange relative party positions\.

The ICC describes the executed specification space\. It does not identify the causal effect of any single parameter, and algorithms with few configurations yield noisier estimates\. The ICC therefore asks how much of the remaining variation, after the experimental setting is held constant, is associated with hyperparameter profiles as a whole\. To unpack that bundle into individual researcher choices, I fit an ANOVA\-style linear model\. The dependent variable is that same context\-adjusted party score; the independent variables are the individual hyperparameters that define the cleaned profiles, not every implementation knob in the grid\. That linear model uses hyperparameter main effects rather than party\-by\-hyperparameter interactions, so the attribution likewise concerns common\-direction movement rather than party\-specific reordering\. SHAP values then show how much each of those parameters contributes\. Uncertainty is reported as 95% intervals from 1,000 bootstrap resamples of hyperparameter profiles\.

### 4\.2Profile\-Level Sensitivity

Figure[1](https://arxiv.org/html/2609.17602#S4.F1)summarizes profile\-level \(common\-direction\) sensitivity: the ICC share of remaining variance associated with a shared shift under each cleaned hyperparameter profile\. It is not a measure of every way researcher choices can change estimates, because party\-specific reordering is absorbed into the residual\.

![Refer to caption](https://arxiv.org/html/2609.17602v1/fig_01_hyperparam_lollipop.png)Figure 1:Profile\-level \(common\-direction\) hyperparameter sensitivity across the 17 CT\-IPE algorithms\. Points show the ICC: the proportion of variation in context\-residualized political position estimates attributable to a shared shift associated with alternative cleaned hyperparameter profiles; horizontal lines and bracketed values give 95% percentile intervals from a cluster bootstrap over cleaned profiles \(1,000 replicates\)\. Party\-specific reordering is not captured by this quantity\.The results reveal that this shared\-shift variance is small for most algorithms\. Thirteen of the 17 methods exhibit ICC values below \.10; for seven of them, the estimated share is essentially zero\. For these algorithms, alternative researcher choices within the evaluated design explain little residual variance through a common intercept\. Four algorithms show non\-negligible profile\-level sensitivity: LSS \(\.19\), Semantic Scaling \(\.28\), and TBIP \(\.34\) occupy an intermediate range, while CGCoT \(\.81\) is the only algorithm for which differences between hyperparameter settings dominate the remaining variation after contextual adjustment\. As discussed in Section[4\.3](https://arxiv.org/html/2609.17602#S4.SS3), the CGCoT estimate rests on very few executed configurations and should be interpreted with particular caution\.

Low ICC values are not an artifact of empty or trivial grids\. Several methods with near\-zero ICCs still vary many cleaned profiles and substantive parameters: Wordscores \(ICC=\.00=\.00\) has 136 cleaned profiles across seven retained parameters; Wordfish \(\.00\.00\) has 72 profiles and six parameters; Correspondence Analysis \(\.00\.00\) has 64 profiles; and Party Embeddings \(\.00\.00\) has 113 profiles across six parameters \(Appendix[B](https://arxiv.org/html/2609.17602#A2)\)\. By contrast, CGCoT’s high ICC rests on only four cleaned profiles\. A near\-zero ICC in this design therefore means that profile differences explain little of the context\-adjusted variance through a shared shift despite nontrivial researcher\-choice variation, not that little was varied\.

The bootstrap intervals in Figure[1](https://arxiv.org/html/2609.17602#S4.F1)support this reading while making its uncertainty explicit\. The intervals of the four algorithms with non\-negligible ICCs exclude zero, but they are wide where the executed grids are small: CGCoT’s interval spans \[\.27, \.87\] with only four cleaned profiles, and the interval of[Benoit et al\. \(2026\)](https://arxiv.org/html/2609.17602#bib.bib3)\(\[\.00, \.28\], six profiles\) is compatible with both negligible and moderate profile\-level sensitivity\. For most algorithms with near\-zero point estimates, the intervals are tight around zero\. Wordscores is the exception \(\[\.00, \.74\]\): its point estimate is zero, but a small share of configurations produces extremely unstable estimates \(some fully degenerate reference\-rescaling runs were already excluded during data preparation\), so resamples that overweight these configurations yield large variance shares\. Its low point estimate should therefore be read as “hyperparameters explain little of a noisy signal”; its profile\-level sensitivity cannot be characterized precisely because of these unstable configurations\.

A second robustness concern is polarity\. For several algorithms, left and right can swap from run to run\. The ICC only counts a shared shift that moves parties in the same direction, so those flips are not registered as profile\-level sensitivity\. They instead add leftover scatter, which can shrink the ICC and hide a genuine common\-direction effect\. Re\-estimating all ICCs after applying an anchor\-based polarity alignment, which sign\-flips every run whose fixed anchor parties appear in reversed order, leaves the substantive picture intact \(Appendix[C](https://arxiv.org/html/2609.17602#A3)\): the near\-zero algorithms remain at essentially zero although between 19% and 53% of their runs are mirrored, so their low sensitivity is not an artifact of sign indeterminacy\. Two shifts are informative\. SemScale rises from \.06 to \.16, indicating that polarity flips of its embedding\-based scale masked genuine profile\-driven variance; its headline value understates its sensitivity\. Conversely, CGCoT falls from \.81 to \.36 and LSS from \.19 to \.10: their polarity is substantively defined by prompts and seed keywords, so part of their measured sensitivity consists of configurations that reverse the direction of the scale outright rather than shifting positions along it\.666Seed\-keyword lists are inconfigs/experiment\_grid\.yamlon the online repository; LLM prompt files are in each algorithm’sprompt\_configsfolder\.

However, Figure[1](https://arxiv.org/html/2609.17602#S4.F1)should not be interpreted as a ranking of methodological quality\. High profile\-level sensitivity does not imply that an algorithm is inferior, nor does a near\-zero ICC imply greater validity\. Instead, the results quantify the extent to which alternative, theoretically or methodologically justifiable researcher choices produce a shared shift in political position estimates within the evaluated comparison framework\. Sensitivity to construct\-defining hyperparameters, such as seed keywords, can even be desirable: those choices are how researchers steer which variation in the text is treated as the relevant signal for the construct\.

A first answer to RQ2 is therefore that, for most of the sampled named algorithms, hyperparameter profiles explain little residual variance through a shared shift, and that only a small subset exhibits substantial profile\-level sensitivity\. The following section examines which individual researcher choices these differences arise from\.

### 4\.3Which Researcher Choices Drive Sensitivity?

The profile\-level ICC does not identify which choices produce the shared shift\. To answer that question, I use the ANOVA\-style linear model described above and SHAP values to attribute leftover variation to the implementation parameters that varied within the executed specification space\. Because that model uses hyperparameter main effects rather than party\-by\-hyperparameter interactions, the shares likewise attribute common\-direction sensitivity\. The resulting importances are normalized within each algorithm and provide an exploratory attribution of hyperparameter\-driven variation to individual parameters\. Figure[2](https://arxiv.org/html/2609.17602#S4.F2)shows these within\-algorithm importance shares for the eight algorithms with a profile\-level ICC of at least \.05; Appendix[D](https://arxiv.org/html/2609.17602#A4)reports the corresponding profiles for all 17 algorithms\. Two diagnostics bound what the shares can support, and both are reported in the facet headers\. First, the shares are proportions of each algorithm’s own – possibly negligible – shared\-shift variance, so they must be read together with the ICC\. Second, the adjustedR2R^\{2\}of this linear model indicates how much of the context\-adjusted variation the attribution model itself captures: it is substantial only for CGCoT \(\.79\) and TBIP \(\.29\), whereas for LSS and Semantic Scaling \(≈\.06\\approx\.06\) and the remaining algorithms the shares rank candidate parameters within a weakly explained outcome rather than partitioning it precisely\.

![Refer to caption](https://arxiv.org/html/2609.17602v1/fig_02_shap_per_algorithm_sensitive.png)Figure 2:Within\-algorithm distribution of profile\-level hyperparameter sensitivity for the eight algorithms with an ICC of at least \.05\. Bars show the share of total mean absolute SHAP value attributable to each varied implementation parameter; whiskers give 95% percentile intervals from the cluster bootstrap over cleaned profiles \(1,000 replicates\); facets are ordered by the ICC in Figure[1](https://arxiv.org/html/2609.17602#S4.F1)\. Facet headers also report the adjustedR2R^\{2\}of that linear model; where it is near zero, shares are exploratory rankings rather than precise partitions of shared\-shift variance\.A first observation is that, where profile\-level sensitivity exists, it is highly concentrated: for every algorithm in Figure[2](https://arxiv.org/html/2609.17602#S4.F2), one or two parameters account for the bulk of the attributed importance\. For CGCoT and the approach of[Benoit et al\. \(2026\)](https://arxiv.org/html/2609.17602#bib.bib3), the choice of the underlying LLM is the only consequential parameter; it was also the only parameter varied in the experiment \(see Appendix[B](https://arxiv.org/html/2609.17602#A2)\)\. Given the prompt insensitivity reported by[Benoit et al\. \(2026\)](https://arxiv.org/html/2609.17602#bib.bib3)and the resource demand of these two algorithms, I opted not to make prompt variation a major focus of the experiment for CGCoT and[Benoit et al\. \(2026\)](https://arxiv.org/html/2609.17602#bib.bib3)\. Prompt variation was included for LaMP and Asking & Averaging, which expose the prompt as a hyperparameter\. Once prompt wording is set aside, LLM model choice is the remaining parameter that the attribution flags as consequential for those algorithms\.777Apart from decoding knobs such as temperature and top\-ppsampling\.For Semantic Scaling, the underlying NLI model and the natural\-language definition of the scaling dimension \(the entailment hypotheses\) dominate, although the bootstrap interval for the latter is wide \(\[\.20, \.62\]\)\. For TBIP, sensitivity derives from the number of topics and the training batch size, and for LSS from the seed keyword lists that define the ideological poles; for LSS, the dominance of the seed keywords is corroborated independently by the landscape analysis below\. Across the evaluated specification space, the choices flagged as consequential thus reduce to a short list: which model is used, which seed keywords anchor the construct, and how many topics are estimated – read as a robust ranking where that linear model fits weakly rather than as an exact variance decomposition\. Conversely, the preprocessing parameters often emphasized for classical frequency\-based methods, such as term\- and document\-frequency thresholds, show no measurable influence on context\-residualized estimates within the evaluated grids: Wordscores, Wordfish, Correspondence Analysis, and Wordshoal all exhibit ICC values of zero\.

Figure[3](https://arxiv.org/html/2609.17602#S4.F3)makes the practical consequence of such concentrated sensitivity concrete\. Holding the texts, algorithm, and pipeline fixed, it maps the six German parties under five alternative LSS seed\-keyword choices\. With the bare German anchorslinks/rechts, the AfD is placed far right and the SPD far left; with the Fightin’ Words\([Monroe et al\., 2008](https://arxiv.org/html/2609.17602#bib.bib42)\)dictionary the AfD flips to the far left; with the Wordfish\-derived\([Slapin and Proksch, 2008](https://arxiv.org/html/2609.17602#bib.bib20)\)dictionary it is again extreme right, while the Laver–Garry\([Laver et al\., 2003](https://arxiv.org/html/2609.17602#bib.bib33), see\)and[Jiang et al\. \(2024\)](https://arxiv.org/html/2609.17602#bib.bib12)dictionaries compress nearly all parties toward the center\. The figure is an illustration for one algorithm and corpus, not a comparative ranking of dictionaries; in the executed LSS grid the keyword family also determines whether auto\-seeding from anchors is used, so the rows show the joint effect of those coupled choices\. Appendix[E](https://arxiv.org/html/2609.17602#A5)reports the same presentation for Semantic Scaling and TBIP, with Wordfish as a low\-sensitivity contrast\.

![Refer to caption](https://arxiv.org/html/2609.17602v1/fig_03_party_landscape.png)Figure 3:Party landscapes under alternative LSS seed\-keyword choices \(German manifestos\)\. Points are mean scaled left–right estimates per party within each keyword family; whiskers span the range of specification means within that family\. Rows differ only in the seed keyword lists \(and the auto\-seeding mode coupled to them in the executed grid\)\. A pooled row of additional generic anchor pairs is omitted for readability\.LLM\-based approaches require somewhat different interpretation\. Semantic Scaling relies on natural language inference \(NLI\) rather than prompting generatively trained LLMs, so construct specification is operationally distinct from prompt engineering\([Laurer et al\., 2023](https://arxiv.org/html/2609.17602#bib.bib44);[Halterman and Keith, 2026](https://arxiv.org/html/2609.17602#bib.bib10)\)\. Within the evaluated specification space, Semantic Scaling exhibits greater profile\-level sensitivity \(\.28\) than the generative approaches LaMP \(\.08\), Asking & Averaging \(\.09\), and[Benoit et al\. \(2026\)](https://arxiv.org/html/2609.17602#bib.bib3)\(\.02\), driven by the underlying NLI model and the natural\-language definition of the scaling dimension\. For the generative LLM methods, even where prompt\-related parameters were included in the experiment and accounted for a recognizable share of the within\-algorithm profile \(LaMP and A&A\), they partition a profile\-level sensitivity that is small in absolute terms\. This accords with the prototyping analysis in the supplementary material of[Benoit et al\. \(2026\)](https://arxiv.org/html/2609.17602#bib.bib3), which finds that systematic small changes in scoring\-prompt wording rarely alter position scores and are “not critical in this application\.”

The primary exception within this group is CGCoT, whose comparatively high sensitivity should be interpreted cautiously\. Owing to its substantially higher computational cost, fewer configurations could be evaluated than for most other algorithms \(Appendix[B](https://arxiv.org/html/2609.17602#A2)\)\. Moreover, CGCoT was developed for short political texts; manifesto corpora were therefore excluded from its executed grid, and the algorithm was evaluated on speeches and tweets only\. Its pairwise ranking procedure also requires more than two political actors, rendering it inapplicable to the two\-party U\.S\. datasets\. Consequently, only 8 of the 16 attempted specifications produced valid results, and the estimated sensitivity rests on this very small executed grid\.

Similar caution applies to other algorithms with small executed grids or low run counts \(Appendix[B](https://arxiv.org/html/2609.17602#A2)\); comparisons involving them should be read descriptively rather than causally\. SemScale \(\.06\) and SemScore \(\.04\) are not in that thin\-grid group: their low ICCs rest on substantial executed samples, including in\-domain embedding training \(Appendix[B](https://arxiv.org/html/2609.17602#A2), see also the Limitations section\)\.

## 5Conclusion

This paper argued that CT\-IPE methods are better understood as configurable measurement pipelines than as fixed algorithms, and compared the researcher choices that configure 17 methods in a shared experiment\. Within the evaluated specification space, hyperparameter profiles account for less than 10% of the context\-adjusted variance through a shared shift for 13 of the 17 algorithms; the broader pattern remains after anchor\-based polarity alignment\. Where this profile\-level sensitivity is present, it concentrates in a few decisions—the underlying language or embedding model, the seed keywords that anchor the construct, and the number of topics—rather than spreading across the pipeline\.

The named algorithm therefore remains a reasonable unit of comparison where those shared shifts are small\. A few construct\-defining choices can still steer the estimates, and those choices deserve explicit reporting\. Party\-specific reordering lies outside the ICC estimand\.

## Limitations

Several limitations should be acknowledged\. The reported analysis characterizes the researcher choices varied in the present comparative experiment rather than every conceivable implementation choice, and results may change as additional algorithms, corpora, or methodological decisions are incorporated\.

The ICC reported here is a profile\-level shared\-shift quantity\. Party\-specific reordering is absorbed into the residual and is illustrated only for selected algorithms and corpora, so a near\-zero ICC should not be read as immunity of the party landscape to researcher choices\.

The experimental grids are also uneven across algorithms\. That imbalance is largely unavoidable: the methods expose different numbers of hyperparameters, and resource constraints limited how many configurations could be run for the more expensive ones\. Cross\-algorithm contrasts of sensitivity should therefore be read descriptively, especially where executed samples are thin\.

In addition, a post\-hoc audit of the execution pipeline found that configuration validation originally dropped unknown YAML keys instead of failing\. A written grid and the executed runs could therefore silently diverge\. Unknown keys are now rejected at planning time\. The sensitivity estimates reported here are for the executed specification space\. Where pretrained embeddings are used, the accompanying training knobs \(window size, epochs, and the like\) are placeholders and do not constitute additional trained variation\.

Future work should therefore broaden the evaluated specification space, investigate additional forms of researcher choice, and examine how hyperparameter sensitivity relates to substantive conclusions drawn from political text scaling\.

## References

- J\. Bafumi, A\. Gelman, D\. K\. Park, and N\. KaplanPractical Issues in Implementing and Understanding Bayesian Ideal Point Estimation\.Political Analysis13\(2\),pp\. 171–187\(en\)\.External Links:ISSN 1047\-1987, 1476\-4989,[Link](https://www.cambridge.org/core/product/identifier/S1047198700001054/type/journal_article),[Document](https://dx.doi.org/10.1093/pan/mpi010)Cited by:[§1](https://arxiv.org/html/2609.17602#S1.p1.1)\.
- Balasubramanianet al\.\(2024\)A\. Balasubramanian, V\. Zou, H\. Narayana, C\. You, L\. Luceri, and E\. FerraraA Public Dataset Tracking Social Media Discourse about the 2024 U\.S\. Presidential Election on Twitter/X\.arXiv\.Note:arXiv:2411\.00376 \[cs\]External Links:[Link](http://arxiv.org/abs/2411.00376),[Document](https://dx.doi.org/10.48550/arXiv.2411.00376)Cited by:[Table 1](https://arxiv.org/html/2609.17602#S3.T1.2.4.7.1.1)\.
- Barberáet al\.\(2015\)P\. Barberá, J\. T\. Jost, J\. Nagler, J\. A\. Tucker, and R\. BonneauTweeting from left to right: is online political communication more than an echo chamber?\.Psychological Science\.External Links:[Document](https://dx.doi.org/10.1177/0956797615594620)Cited by:[§3\.1](https://arxiv.org/html/2609.17602#S3.SS1.p1.1)\.
- Beauchamp \(2012\)N\. BeauchampUsing Text to Scale Legislatures with Uninformative Voting\.Note:Accessed 2025\-03\-06External Links:[Link](https://nickbeauchamp.com/work/Beauchamp_scaling_current.pdf)Cited by:[Appendix A](https://arxiv.org/html/2609.17602#A1.p1.p1.1.2.1.1.1)\.
- Benoitet al\.\(2026\)K\. Benoit, S\. De Marchi, C\. Laver, M\. Laver, and J\. MaUsing large language models to analyze political texts through natural language understanding\.American Journal of Political Science,pp\. ajps\.70050\(en\)\.External Links:ISSN 0092\-5853, 1540\-5907,[Link](https://onlinelibrary.wiley.com/doi/10.1111/ajps.70050),[Document](https://dx.doi.org/10.1111/ajps.70050)Cited by:[Appendix A](https://arxiv.org/html/2609.17602#A1.p1.p1.1.18.1.1.1),[§4\.2](https://arxiv.org/html/2609.17602#S4.SS2.p4.1),[§4\.3](https://arxiv.org/html/2609.17602#S4.SS3.p2.1),[§4\.3](https://arxiv.org/html/2609.17602#S4.SS3.p4.1)\.
- Bouthillieret al\.\(2021\)X\. Bouthillier, P\. Delaunay, M\. Bronzi, A\. Trofimov, B\. Nichyporuk, J\. Szeto, N\. Mohammadi Sepahvand, E\. Raff, K\. Madan, V\. Voleti, S\. Ebrahimi Kahou, V\. Michalski, T\. Arbel, C\. Pal, G\. Varoquaux, and P\. VincentAccounting for Variance in Machine Learning Benchmarks\.InProceedings of Machine Learning and Systems,A\. Smola, A\. Dimakis, and I\. Stoica \(Eds\.\),Vol\.3,pp\. 747–769\.External Links:[Link](https://proceedings.mlsys.org/paper_files/paper/2021/file/0184b0cd3cfb185989f858a1d9f5c1eb-Paper.pdf)Cited by:[§2](https://arxiv.org/html/2609.17602#S2.p2.1)\.
- Bruinsma and Gemenis \(2019\)B\. Bruinsma and K\. GemenisValidating Wordscores: The Promises and Pitfalls of Computational Text Scaling\.Communication Methods and Measures13\(3\),pp\. 212–227\(en\)\.External Links:ISSN 1931\-2458, 1931\-2466,[Link](https://www.tandfonline.com/doi/full/10.1080/19312458.2019.1594741),[Document](https://dx.doi.org/10.1080/19312458.2019.1594741)Cited by:[§1](https://arxiv.org/html/2609.17602#S1.p1.1)\.
- Burnham \(2024\)M\. BurnhamSemantic Scaling: Bayesian Ideal Point Estimates with Large Language Models\.arXiv\.External Links:[Link](https://arxiv.org/abs/2405.02472),[Document](https://dx.doi.org/10.48550/ARXIV.2405.02472)Cited by:[Appendix A](https://arxiv.org/html/2609.17602#A1.p1.p1.1.16.1.1.1)\.
- Carroll \(2023\)R\. CarrollIdeal Point Estimation\.\(en\)\.Note:https://doi\.org/10\.1093/obo/9780199756223\-0360External Links:[Link](https://doi.org/10.1093/obo/9780199756223-0360)Cited by:[§1](https://arxiv.org/html/2609.17602#S1.p1.1)\.
- Egerod and Klemmensen \(2020\)B\. C\.K\. Egerod and R\. KlemmensenScaling Political Positions from Text: Assumptions, Methods and Pitfalls\.InThe SAGE Handbook of Research Methods in Political Science and International Relations,L\. Curini and R\. Franzese \(Eds\.\),pp\. 498–521\(en\)\.External Links:ISBN 978\-1\-5264\-5993\-0 978\-1\-5264\-8638\-7,[Link](https://sk.sagepub.com/reference/the-sage-handbook-of-research-methods-in-political-science-and-ir/i4547.xml),[Document](https://dx.doi.org/10.4135/9781526486387.n30)Cited by:[§1](https://arxiv.org/html/2609.17602#S1.p1.1)\.
- Fanget al\.\(2012\)Y\. Fang, L\. Si, N\. Somasundaram, and Z\. YuMining contrastive opinions on political texts using cross\-perspective topic model\.InProceedings of the fifth ACM international conference on Web search and data mining,Seattle Washington USA,pp\. 63–72\(en\)\.External Links:ISBN 978\-1\-4503\-0747\-5,[Link](https://dl.acm.org/doi/10.1145/2124295.2124306),[Document](https://dx.doi.org/10.1145/2124295.2124306)Cited by:[Appendix A](https://arxiv.org/html/2609.17602#A1.p1.p1.1.10.1.1.1),[Appendix A](https://arxiv.org/html/2609.17602#A1.p1.p1.1.10.3.1.1)\.
- Gabel and Huber \(2000\)M\. J\. Gabel and J\. D\. HuberPutting Parties in Their Place: Inferring Party Left\-Right Ideological Positions from Party Manifestos Data\.American Journal of Political Science44\(1\),pp\. 94–103\.External Links:ISSN 0092\-5853,[Link](https://www.jstor.org/stable/2669295),[Document](https://dx.doi.org/10.2307/2669295)Cited by:[Appendix A](https://arxiv.org/html/2609.17602#A1.p1.p1.1.5.1.1.1)\.
- Gentzkowet al\.\(2019\)M\. Gentzkow, J\. M\. Shapiro, and M\. TaddyMeasuring Group Differences in High‐Dimensional Choices: Method and Application to Congressional Speech\.Econometrica87\(4\),pp\. 1307–1340\(en\)\.External Links:ISSN 0012\-9682,[Link](https://www.econometricsociety.org/doi/10.3982/ECTA16566),[Document](https://dx.doi.org/10.3982/ECTA16566)Cited by:[Appendix A](https://arxiv.org/html/2609.17602#A1.p1.p1.1.9.1.1.1),[Appendix A](https://arxiv.org/html/2609.17602#A1.p1.p1.1.9.3.1.1)\.
- Glavašet al\.\(2017\)G\. Glavaš, F\. Nanni, and S\. P\. PonzettoUnsupervised Cross\-Lingual Scaling of Political Texts\.InProceedings of the 15th Conference of the European Chapter of the Association for Computational Linguistics: Volume 2, Short Papers,Valencia, Spain,pp\. 688–693\(en\)\.External Links:[Link](http://aclweb.org/anthology/E17-2109),[Document](https://dx.doi.org/10.18653/v1/E17-2109)Cited by:[Appendix A](https://arxiv.org/html/2609.17602#A1.p1.p1.1.12.1.1.1),[Appendix A](https://arxiv.org/html/2609.17602#A1.p1.p1.1.13.1.1.1)\.
- Halterman and Keith \(2026\)A\. Halterman and K\. A\. KeithCodebook LLMs: evaluating LLMs as measurement tools for political science concepts\.Political Analysis34,pp\. 188–204\.External Links:ISSN 1047\-1987, 1476\-4989,[Link](https://www.cambridge.org/core/product/identifier/S1047198725100174/type/journal_article),[Document](https://dx.doi.org/10.1017/pan.2025.10017)Cited by:[§4\.3](https://arxiv.org/html/2609.17602#S4.SS3.p4.1)\.
- Hjorthet al\.\(2015\)F\. Hjorth, R\. Klemmensen, S\. Hobolt, M\. E\. Hansen, and P\. Kurrild\-KlitgaardComputers, coders, and voters: Comparing automated methods for estimating party positions\.Research & Politics2\(2\),pp\. 2053168015580476\(en\)\.External Links:ISSN 2053\-1680, 2053\-1680,[Link](https://journals.sagepub.com/doi/10.1177/2053168015580476),[Document](https://dx.doi.org/10.1177/2053168015580476)Cited by:[§1](https://arxiv.org/html/2609.17602#S1.p1.1)\.
- Hofmarcheret al\.\(2025\)P\. Hofmarcher, J\. Vávra, S\. Adhikari, and B\. GrünRevisiting Group Differences in High\-Dimensional Choices: Method and Application to Congressional Speech\.arXiv\.External Links:[Link](https://arxiv.org/abs/2206.10877),[Document](https://dx.doi.org/10.48550/ARXIV.2206.10877)Cited by:[Appendix A](https://arxiv.org/html/2609.17602#A1.p1.p1.1.9.1.1.1),[Appendix A](https://arxiv.org/html/2609.17602#A1.p1.p1.1.9.3.1.1)\.
- Jentschet al\.\(2020\)C\. Jentsch, E\. R\. Lee, and E\. MammenTime\-dependent Poisson reduced rank models for political text data analysis\.Computational Statistics & Data Analysis142\(en\)\.External Links:ISSN 01679473,[Link](https://linkinghub.elsevier.com/retrieve/pii/S0167947319301586),[Document](https://dx.doi.org/10.1016/j.csda.2019.106813)Cited by:[Appendix A](https://arxiv.org/html/2609.17602#A1.p1.p1.1.6.1.1.1),[Appendix A](https://arxiv.org/html/2609.17602#A1.p1.p1.1.6.3.1.1)\.
- Jentschet al\.\(2021\)C\. Jentsch, E\. R\. Lee, and E\. MammenPoisson reduced\-rank models with an application to political text data\.Biometrika108\(2\),pp\. 455–468\(en\)\.External Links:ISSN 0006\-3444, 1464\-3510,[Link](https://academic.oup.com/biomet/article/108/2/455/5879273),[Document](https://dx.doi.org/10.1093/biomet/asaa063)Cited by:[Appendix A](https://arxiv.org/html/2609.17602#A1.p1.p1.1.6.1.1.1),[Appendix A](https://arxiv.org/html/2609.17602#A1.p1.p1.1.6.3.1.1)\.
- Jianget al\.\(2024\)H\. Jiang, D\. Beeferman, W\. Brannon, A\. Heyward, and D\. RoyBridging Dictionary: AI\-Generated Dictionary of Partisan Language Use\.InCompanion Publication of the 2024 Conference on Computer\-Supported Cooperative Work and Social Computing,San Jose Costa Rica,pp\. 79–82\(en\)\.External Links:ISBN 979\-8\-4007\-1114\-5,[Link](https://dl.acm.org/doi/10.1145/3678884.3681820),[Document](https://dx.doi.org/10.1145/3678884.3681820)Cited by:[§4\.3](https://arxiv.org/html/2609.17602#S4.SS3.p3.1)\.
- Juddet al\.\(2019\)N\. Judd, D\. Drinkard, J\. Carbaugh, and L\. YoungCongressional\-record: A parser for the Congressional Record\.Note:https://huggingface\.co/datasets/mesut/congress\_speechesExternal Links:[Link](https://github.com/unitedstates/congressional-record)Cited by:[Table 1](https://arxiv.org/html/2609.17602#S3.T1.2.3.7.1.1)\.
- Kratzke \(2017\)N\. KratzkeThe \#BTW17 Twitter Dataset \- Recorded Tweets of the Federal Election Campaigns of 2017 for the 19th German Bundestag\.Zenodo\(de\)\.External Links:[Link](https://zenodo.org/record/835734),[Document](https://dx.doi.org/10.5281/ZENODO.835734)Cited by:[Table 1](https://arxiv.org/html/2609.17602#S3.T1.2.7.7.1.1)\.
- Lange and Jentsch \(2023\)K\. Lange and C\. JentschSpeakGer: A meta\-data enriched speech corpus of German state and federal parliaments\.InProceedings of the 3rd Workshop on Computational Linguistics for the Political and Social Sciences,C\. Klamm, G\. Lapesa, V\. Gold, T\. Gessler, and S\. P\. Ponzetto \(Eds\.\),Ingolstadt, Germany,pp\. 19–28\.External Links:[Link](https://aclanthology.org/2023.cpss-1.3/)Cited by:[Table 1](https://arxiv.org/html/2609.17602#S3.T1.2.6.7.1.1)\.
- Lauderdale and Herzog \(2016\)B\. E\. Lauderdale and A\. HerzogMeasuring Political Positions from Legislative Speech\.Political Analysis24\(3\),pp\. 374–394\(en\)\.External Links:ISSN 1047\-1987, 1476\-4989,[Link](https://www.cambridge.org/core/product/identifier/S1047198700014091/type/journal_article),[Document](https://dx.doi.org/10.1093/pan/mpw017)Cited by:[Appendix A](https://arxiv.org/html/2609.17602#A1.p1.p1.1.4.1.1.1)\.
- Laureret al\.\(2023\)M\. Laurer, W\. v\. Atteveldt, A\. Casas, and K\. WelbersLess Annotating, More Classifying: Addressing the Data Scarcity Issue of Supervised Machine Learning with Deep Transfer Learning and BERT\-NLI\.Political Analysis,pp\. 1–17\(en\)\.External Links:ISSN 1047\-1987, 1476\-4989,[Link](https://www.cambridge.org/core/journals/political-analysis/article/less-annotating-more-classifying-addressing-the-data-scarcity-issue-of-supervised-machine-learning-with-deep-transfer-learning-and-bertnli/05BB05555241762889825B080E097C27#article-tab),[Document](https://dx.doi.org/10.1017/pan.2023.20)Cited by:[§4\.3](https://arxiv.org/html/2609.17602#S4.SS3.p4.1)\.
- Laveret al\.\(2003\)M\. Laver, K\. Benoit, and J\. GarryExtracting Policy Positions from Political Texts Using Words as Data\.American Political Science Review97\(en\)\.External Links:ISSN 0003\-0554,[Link](https://doi.org/10.1017/S0003055403000698),[Document](https://dx.doi.org/10.1017/S0003055403000698)Cited by:[Appendix A](https://arxiv.org/html/2609.17602#A1.p1.p1.1.2.1.1.1),[§1](https://arxiv.org/html/2609.17602#S1.p1.1),[§4\.3](https://arxiv.org/html/2609.17602#S4.SS3.p3.1)\.
- Le Mens and Gallego \(2025\)G\. Le Mens and A\. GallegoPositioning Political Texts with Large Language Models by Asking and Averaging\.Political Analysis33\(3\),pp\. 274–282\(en\)\.External Links:ISSN 1047\-1987, 1476\-4989,[Link](https://www.cambridge.org/core/journals/political-analysis/article/positioning-political-texts-with-large-language-models-by-asking-and-averaging/BB17F58E7329B53BC4B5F4065C7E00FF),[Document](https://dx.doi.org/10.1017/pan.2024.29)Cited by:[Appendix A](https://arxiv.org/html/2609.17602#A1.p1.p1.1.17.1.1.1),[§1](https://arxiv.org/html/2609.17602#S1.p1.1)\.
- Lehmannet al\.\(2025\)P\. Lehmann, S\. Franzmann, D\. Al\-Gaddooa, T\. Burst, C\. Ivanusch, S\. Regel, F\. Riethmüller, A\. Volkens, B\. Weßels, L\. Zehnter, Wissenschaftszentrum Berlin für Sozialforschung \(WZB\), and Institut für Demokratieforschung Göttingen \(IfDem\)Manifesto Project Dataset\.Manifesto Project\(en\)\.External Links:[Link](https://manifesto-project.wzb.eu/datasets/MPDS2025a),[Document](https://dx.doi.org/10.25522/MANIFESTO.MPDS.2025A)Cited by:[Table 1](https://arxiv.org/html/2609.17602#S3.T1.2.2.7.1.1),[Table 1](https://arxiv.org/html/2609.17602#S3.T1.2.5.7.1.1)\.
- Lowe \(2008\)W\. LoweUnderstanding Wordscores\.Political Analysis16\(4\),pp\. 356–371\(en\)\.External Links:ISSN 1047\-1987, 1476\-4989,[Link](https://www.cambridge.org/core/product/identifier/S104719870000228X/type/journal_article),[Document](https://dx.doi.org/10.1093/pan/mpn004)Cited by:[Appendix A](https://arxiv.org/html/2609.17602#A1.p1.p1.1.2.1.1.1)\.
- Lowe \(2016\)W\. LoweScaling things we can count\.\(en\)\.Note:Unpublished manuscriptExternal Links:[Link](https://dl.conjugateprior.org/preprints/all-on-the-line.pdf)Cited by:[Appendix A](https://arxiv.org/html/2609.17602#A1.p1.p1.1.5.1.1.1)\.
- Lundberg and Lee \(2017\)S\. M\. Lundberg and S\. LeeA unified approach to interpreting model predictions\.InProceedings of the 31st International Conference on Neural Information Processing Systems,NIPS’17,Red Hook, NY, USA,pp\. 4768–4777\.External Links:ISBN 978\-1\-5108\-6096\-4,[Link](https://dl.acm.org/doi/10.5555/3295222.3295230)Cited by:[§1](https://arxiv.org/html/2609.17602#S1.SS0.SSS0.Px2.p3.1)\.
- Martin and Vanberg \(2008\)L\. W\. Martin and G\. VanbergA Robust Transformation Procedure for Interpreting Political Text\.Political Analysis16\(1\),pp\. 93–100\(en\)\.External Links:ISSN 1047\-1987, 1476\-4989,[Link](https://www.cambridge.org/core/product/identifier/S1047198700006707/type/journal_article),[Document](https://dx.doi.org/10.1093/pan/mpm010)Cited by:[Appendix A](https://arxiv.org/html/2609.17602#A1.p1.p1.1.2.1.1.1)\.
- Monroeet al\.\(2008\)B\. L\. Monroe, M\. P\. Colaresi, and K\. M\. QuinnFightin’ Words: Lexical Feature Selection and Evaluation for Identifying the Content of Political Conflict\.Political Analysis16\(4\),pp\. 372–403\(en\)\.External Links:ISSN 1047\-1987, 1476\-4989,[Link](https://www.cambridge.org/core/journals/political-analysis/article/fightin-words-lexical-feature-selection-and-evaluation-for-identifying-the-content-of-political-conflict/81B3703230D21620B81EB6E2266C7A66),[Document](https://dx.doi.org/10.1093/pan/mpn018)Cited by:[§4\.3](https://arxiv.org/html/2609.17602#S4.SS3.p3.1)\.
- Nanniet al\.\(2022\)F\. Nanni, G\. Glavaš, I\. Rehbein, S\. P\. Ponzetto, and H\. StuckenschmidtPolitical Text Scaling Meets Computational Semantics\.ACM/IMS Transactions on Data Science2\(4\),pp\. 29:1–29:27\.External Links:ISSN 2691\-1922,[Link](https://dl.acm.org/doi/10.1145/3485666),[Document](https://dx.doi.org/10.1145/3485666)Cited by:[Appendix A](https://arxiv.org/html/2609.17602#A1.p1.p1.1.12.1.1.1),[Appendix A](https://arxiv.org/html/2609.17602#A1.p1.p1.1.13.1.1.1)\.
- Parschan and Jakob \(2025\)P\. Parschan and C\. JakobComputational measurement of political positions: a review of text\-based ideal point estimation algorithms\.Quality & Quantity60\(2\),pp\. 5663–5690\(en\)\.External Links:ISSN 1573\-7845,[Link](https://link.springer.com/10.1007/s11135-025-02500-4),[Document](https://dx.doi.org/10.1007/s11135-025-02500-4)Cited by:[§3\.1](https://arxiv.org/html/2609.17602#S3.SS1.p1.1),[§3\.1](https://arxiv.org/html/2609.17602#S3.SS1.p2.1),[§3\.1](https://arxiv.org/html/2609.17602#S3.SS1.p5.1)\.
- Rheault and Cochrane \(2020\)L\. Rheault and C\. CochraneWord Embeddings for the Analysis of Ideological Placement in Parliamentary Corpora\.Political Analysis28\(1\),pp\. 112–133\(en\)\.External Links:ISSN 1047\-1987, 1476\-4989,[Link](https://www.cambridge.org/core/product/identifier/S1047198719000263/type/journal_article),[Document](https://dx.doi.org/10.1017/pan.2019.26)Cited by:[Appendix A](https://arxiv.org/html/2609.17602#A1.p1.p1.1.11.1.1.1),[§1](https://arxiv.org/html/2609.17602#S1.p1.1)\.
- Riesch \(2025\)B\. RieschWordkrill: Extending Wordfish into the multidimensional political space\.arXiv\.Note:arXiv:2506\.20275 \[stat\]External Links:[Link](http://arxiv.org/abs/2506.20275),[Document](https://dx.doi.org/10.48550/arXiv.2506.20275)Cited by:[Appendix A](https://arxiv.org/html/2609.17602#A1.p1.p1.1.3.1.1.1),[Appendix A](https://arxiv.org/html/2609.17602#A1.p1.p1.1.3.3.1.1)\.
- Sältzer \(2022\)M\. SältzerFinding the bird’s wings: Dimensions of factional conflict on Twitter\.Party Politics28\(1\),pp\. 61–70\(en\)\.External Links:ISSN 1354\-0688, 1460\-3683,[Link](https://journals.sagepub.com/doi/10.1177/1354068820957960),[Document](https://dx.doi.org/10.1177/1354068820957960)Cited by:[Appendix A](https://arxiv.org/html/2609.17602#A1.p1.p1.1.5.1.1.1),[Appendix A](https://arxiv.org/html/2609.17602#A1.p1.p1.1.5.3.1.1)\.
- Slapin and Proksch \(2008\)J\. B\. Slapin and S\. ProkschA Scaling Model for Estimating Time\-Series Party Positions from Texts\.American Journal of Political Science52\(3\),pp\. 705–722\(en\)\.External Links:ISSN 00925853, 15405907,[Link](https://onlinelibrary.wiley.com/doi/10.1111/j.1540-5907.2008.00338.x),[Document](https://dx.doi.org/10.1111/j.1540-5907.2008.00338.x)Cited by:[Appendix A](https://arxiv.org/html/2609.17602#A1.p1.p1.1.3.1.1.1),[§1](https://arxiv.org/html/2609.17602#S1.p1.1),[§4\.3](https://arxiv.org/html/2609.17602#S4.SS3.p3.1)\.
- Vafaet al\.\(2020\)K\. Vafa, S\. Naidu, and D\. BleiText\-Based Ideal Points\.InProceedings of the 58th Annual Meeting of the Association for Computational Linguistics,Online,pp\. 5345–5357\(en\)\.External Links:[Link](https://www.aclweb.org/anthology/2020.acl-main.475),[Document](https://dx.doi.org/10.18653/v1/2020.acl-main.475)Cited by:[Appendix A](https://arxiv.org/html/2609.17602#A1.p1.p1.1.8.1.1.1),[Appendix A](https://arxiv.org/html/2609.17602#A1.p1.p1.1.9.1.1.1),[Appendix A](https://arxiv.org/html/2609.17602#A1.p1.p1.1.9.3.1.1),[§1](https://arxiv.org/html/2609.17602#S1.p1.1)\.
- Van Der Zwaanet al\.\(2016\)M\. J\. Van Der Zwaan, M\. Maarten, and K\. JaapValidating Cross\-Perspective Topic Modeling for Extracting Political Parties’ Positions from Parliamentary Proceedings\.InFrontiers in Artificial Intelligence and Applications,External Links:[Link](https://www.medra.org/servlet/aliasResolver?alias=iospressISBN&isbn=978-1-61499-671-2&spage=28&doi=10.3233/978-1-61499-672-9-28),[Document](https://dx.doi.org/10.3233/978-1-61499-672-9-28)Cited by:[Appendix A](https://arxiv.org/html/2609.17602#A1.p1.p1.1.10.1.1.1),[Appendix A](https://arxiv.org/html/2609.17602#A1.p1.p1.1.10.3.1.1)\.
- Warode \(2025\)L\. WarodeMapping left\-right associations: a framework using open\-ended survey responses and political positions\.Humanities and Social Sciences Communications12\(1\),pp\. 1318\(en\)\.External Links:ISSN 2662\-9992,[Link](https://www.nature.com/articles/s41599-025-05679-x),[Document](https://dx.doi.org/10.1057/s41599-025-05679-x)Cited by:[Appendix A](https://arxiv.org/html/2609.17602#A1.p1.p1.1.7.1.1.1),[Appendix A](https://arxiv.org/html/2609.17602#A1.p1.p1.1.7.3.1.1)\.
- Watanabe \(2015\)K\. WatanabeLeft\-right policy position dictionary\.\(en\-US\)\.External Links:[Link](https://blog.koheiw.net/?p=155)Cited by:[Appendix A](https://arxiv.org/html/2609.17602#A1.p1.p1.1.7.1.1.1),[Appendix A](https://arxiv.org/html/2609.17602#A1.p1.p1.1.7.3.1.1)\.
- Watanabe \(2021\)K\. WatanabeLatent Semantic Scaling: A Semisupervised Text Analysis Technique for New Domains and Languages\.Communication Methods and Measures15\(2\),pp\. 81–102\.Note:\_eprint: https://doi\.org/10\.1080/19312458\.2020\.1832976External Links:ISSN 1931\-2458,[Link](https://doi.org/10.1080/19312458.2020.1832976),[Document](https://dx.doi.org/10.1080/19312458.2020.1832976)Cited by:[Appendix A](https://arxiv.org/html/2609.17602#A1.p1.p1.1.7.1.1.1)\.
- Wuet al\.\(2023\)P\. Y\. Wu, J\. Nagler, J\. A\. Tucker, and S\. MessingLarge Language Models Can Be Used to Estimate the Latent Positions of Politicians\.arXiv\.Note:Version Number: 4External Links:[Link](https://arxiv.org/abs/2303.12057),[Document](https://dx.doi.org/10.48550/ARXIV.2303.12057)Cited by:[Appendix A](https://arxiv.org/html/2609.17602#A1.p1.p1.1.14.1.1.1),[Appendix A](https://arxiv.org/html/2609.17602#A1.p1.p1.1.15.3.1.1)\.
- Wuet al\.\(2024\)P\. Y\. Wu, J\. Nagler, J\. A\. Tucker, and S\. MessingConcept\-Guided Chain\-of\-Thought Prompting for Pairwise Comparison Scoring of Texts with Large Language Models\.In2024 IEEE International Conference on Big Data \(BigData\),Washington, DC, USA,pp\. 7232–7241\.External Links:ISBN 979\-8\-3503\-6248\-0,[Link](https://ieeexplore.ieee.org/document/10825235/),[Document](https://dx.doi.org/10.1109/BigData62323.2024.10825235)Cited by:[Appendix A](https://arxiv.org/html/2609.17602#A1.p1.p1.1.15.1.1.1)\.
- Yang and Shami \(2020\)L\. Yang and A\. ShamiOn hyperparameter optimization of machine learning algorithms: Theory and practice\.Neurocomputing415,pp\. 295–316\.External Links:ISSN 0925\-2312,[Link](https://www.sciencedirect.com/science/article/pii/S0925231220311693),[Document](https://dx.doi.org/10.1016/j.neucom.2020.07.061)Cited by:[§2](https://arxiv.org/html/2609.17602#S2.p2.1)\.

## Appendix AIncluded Algorithms and Relevant Papers

Table 2:Overview of all reviewed CT\-IPE algorithms\. The main paper relevant for the algorithm and the comparison implemented in this study is always mentioned first\.

## Appendix BOverview of Algorithm Runs

Table 3:Per\-algorithm overview: number of attempted runs, successful runs, mean runtime, peak RAM, and the number of cleaned hyperparameter profiles and retained substantive parameters entering the sensitivity analysis \(Section[4\.1](https://arxiv.org/html/2609.17602#S4.SS1)\)\. Runs fail or are skipped when an algorithm is not applicable to a corpus or a configuration does not complete\. Total position estimates across all algorithms: 4,252,357\. Total attempted runs: 5,537; successful runs: 4,565\.

## Appendix CPolarity\-Alignment Robustness Check

Table[4](https://arxiv.org/html/2609.17602#A3.T4)reports the cleaned\-profile ICC of Figure[1](https://arxiv.org/html/2609.17602#S4.F1)after per\-run anchor\-based polarity alignment: every run whose fixed anchor parties \(Grüne vs\. AfD for Germany, Democrat vs\. Republican for the US\) appear in reversed order is sign\-flipped before the per\-algorithm robust standardization and the variance decomposition are recomputed\. Runs for which no anchor order can be established \(missing anchor party or tie\) are excluded\. The flipped share is a descriptive polarity\-instability diagnostic; it mixes arbitrary sign indeterminacy \(unsupervised scales\) with substantive choice\-induced reversal \(where prompts, seed keywords, or entailment hypotheses define the scale’s direction\)\. Alignment therefore removes a nuisance for the former group but part of the measured researcher\-choice effect for the latter\.

Table 4:Cleaned\-profile ICC before and after per\-run anchor\-based polarity alignment, share of run groups sign\-flipped by the alignment \(among decidable groups\), and share of run groups excluded as unalignable\. For SS and CPTM, part of the aligned\-vs\-headline difference stems from restricting to alignable runs: on that subset, the unaligned ICCs are \.36 and \.06, respectively\.

## Appendix DHyperparameter Sensitivity Profiles for All Algorithms

![Refer to caption](https://arxiv.org/html/2609.17602v1/fig_02_shap_per_algorithm_all.png)Figure 4:Within\-algorithm distribution of hyperparameter sensitivity for all 17 algorithms \(share of total mean absolute SHAP value per varied implementation parameter; whiskers give 95% percentile intervals from the cluster bootstrap over cleaned profiles\)\. Facets are ordered by the ICC in Figure[1](https://arxiv.org/html/2609.17602#S4.F1)and additionally report the adjustedR2R^\{2\}of that linear model\. For algorithms with an ICC near zero or a near\-zero model fit, the shares partition a negligible or weakly explained amount of variance and should not be substantively interpreted\.
## Appendix EParty Landscapes under Alternative Researcher Choices

Figure[5](https://arxiv.org/html/2609.17602#A5.F5)extends the landscape presentation of Figure[3](https://arxiv.org/html/2609.17602#S4.F3)to the remaining manifesto\-compatible algorithms with non\-negligible profile\-level sensitivity, plus Wordfish as a contrast case with ICC of zero\. For Semantic Scaling, rows correspond to the underlying NLI model \(the dominant SHAP driver\); for TBIP, to the number of topics; for Wordfish, to the two hyperparameters with the largest SHAP shares in Figure[4](https://arxiv.org/html/2609.17602#A4.F4)\(min\_termfreqandmin\_docfreq\)\. All panels use German manifestos\. CGCoT is omitted because manifesto corpora were excluded from its executed grid; it was evaluated on speeches and tweets only\. Semantic Scaling likewise lacks SPD and FDP in the executed German\-manifesto runs\.

The three panels illustrate different forms of sensitivity\. Semantic Scaling’s NLI\-model choice can reorder parties and reverse the AfD’s polarity across models\. TBIP’s topic\-count choice, despite a pooled ICC of \.34, leaves the German\-manifesto party order largely intact in this cell: high profile\-level sensitivity need not imply landscape reordering in every corpus\. Wordfish’s landscapes under alternative minimum term\- and document\-frequency thresholds are nearly identical, consistent with its ICC of zero\.

![Refer to caption](https://arxiv.org/html/2609.17602v1/fig_03_party_landscape_appendix.png)Figure 5:Party landscapes under alternative researcher choices for Semantic Scaling, TBIP, and Wordfish \(German manifestos\)\. Presentation as in Figure[3](https://arxiv.org/html/2609.17602#S4.F3); row definitions differ by algorithm \(NLI model; number of topics;min\_termfreq×\\timesmin\_docfreq\)\. Whiskers span remaining within\-row specification means\.
## Appendix FOnline Supplementary Material

The following zip files are available on the online repository:

- •experiment\_pipeline\_replication\_data\_and\_code\.zip: the experimental pipeline, replication data, and code used in the experiments\.
- •cpss\_at\_konvens\_harvard\_dataverse\.zip: the data and code for the analysis of this paper at Konvens 2026\.

## Acknowledgments

The author received financial support via the Graduate Center for Doctoral Researchers at the Bavarian Research Institute for Digital Transformation \(bidt\)\.