跨语言及文本粒度的 CEFR 等级评估

# CEFR Level Assessment across Languages and Text Granularities
Source: [https://arxiv.org/html/2610.41553](https://arxiv.org/html/2610.41553)
###### Abstract

Automatic assessment of textual CEFR levels is an important task in educational natural language processing\. However, accurate CEFR level prediction remains challenging due to the naturally imbalanced distribution of CEFR levels and the ambiguous boundaries between adjacent levels\. To address these challenges, we propose a unified training objective that combines focal loss with supervised contrastive learning\. Focal loss mitigates class imbalance by emphasizing underrepresented and difficult examples, while supervised contrastive learning learns more discriminative representations by encouraging intra\-class compactness and inter\-class separation\. We evaluate the proposed method on sentence\- and document\-level CEFR assessment in both monolingual and multilingual settings\. Experimental results demonstrate that our method consistently outperforms strong baselines across all settings, establishing new state\-of\-the\-art performance on multiple CEFR assessment benchmarks\.

## 1Introduction

Automatic text difficulty assessment has long been an active research topic in natural language processing, with applications in educational technologies such as automated essay scoring and text simplification\([Li et al\., 2025](https://arxiv.org/html/2610.41553#bib.bib4)\)\. The Common European Framework of Reference \(CEFR\)111[https://www\.coe\.int/en/web/common\-european\-framework\-reference\-languages](https://www.coe.int/en/web/common-european-framework-reference-languages)is a widely adopted framework for describing language proficiency, defining six levels ranging from A1 \(beginner\) to C2 \(proficient\)\.

Early approaches to predicting CEFR levels of texts relied on manually engineered linguistic features\([Vajjala and Meurers, 2012](https://arxiv.org/html/2610.41553#bib.bib10)\), while more recent work\([Imperial et al\., 2025](https://arxiv.org/html/2610.41553#bib.bib2)\)has substantially improved performance using pretrained language models such as BERT[Devlin et al\. \(2019\)](https://arxiv.org/html/2610.41553#bib.bib8)\. Despite these advances, two fundamental challenges remain\. First, the distribution of CEFR levels is inherently imbalanced, with A1 and C2 texts occurring much less frequently than texts at intermediate levels\. Second, CEFR levels form an ordinal scale, and the boundaries between adjacent levels are often ambiguous, making neighboring levels difficult to distinguish consistently\. Although[Arase et al\. \(2022\)](https://arxiv.org/html/2610.41553#bib.bib1)addressed class imbalance through metric learning, their approach does not explicitly account for the ambiguity of adjacent CEFR levels\.

To address these challenges, we propose a CEFR level assessment method that jointly leverages focal loss\([Lin et al\., 2017](https://arxiv.org/html/2610.41553#bib.bib5)\)and supervised contrastive learning\([Gao et al\., 2021](https://arxiv.org/html/2610.41553#bib.bib9)\)\. Focal loss mitigates the effect of class imbalance by emphasizing underrepresented and difficult examples, while supervised contrastive learning learns more discriminative representations by pulling together examples from the same CEFR level while increasing the separation between different CEFR levels in the embedding space\. Together, these objectives improve CEFR level prediction under both class imbalance and ambiguous class boundaries\.

We evaluate the proposed method on sentence\- and document\-level CEFR assessment in both monolingual \(English\) and multilingual settings\. Experimental results demonstrate that it consistently outperforms strong state\-of\-the\-art baselines across all evaluation settings\.

The contributions of this paper are threefold: \(i\) we propose a unified training objective combining focal loss and supervised contrastive learning for CEFR level assessment, addressing both naturally imbalanced CEFR distributions and ambiguity between adjacent levels; \(ii\) we demonstrate its effectiveness on sentence\- and document\-level CEFR level assessment in both monolingual and multilingual settings; and \(iii\) we achieve state\-of\-the\-art on multiple CEFR level assessment datasets\.

## 2Proposed Method

Following[Arase et al\. \(2022\)](https://arxiv.org/html/2610.41553#bib.bib1), we formulate CEFR level prediction as a classification task: predict a class labely∈\{1,…,K\}​\(K=6\)y\\in\\\{1,\\dots,K\\\}\(K=6\)given an input textX=\(x1,…,xT\)X=\(x\_\{1\},\\dots,x\_\{T\}\)\. We employ a pretrained masked language model \(PMLM\) for encoding the input text\. The hidden representation𝐳∈ℝd\\mathbf\{z\}\\in\\mathbb\{R\}^\{d\}is passed to a classification head, which is a single linear layer mapping𝐳\\mathbf\{z\}toKKlogits, followed by a softmax that yields the class posterior\.

We conduct two\-stage training; as in[Khosla et al\. \(2020\)](https://arxiv.org/html/2610.41553#bib.bib3), preliminary experiments confirmed this separation performed better than joint training\. In the first stage, the PMLM parameters are fine\-tuned with the supervised contrastive learning \(Section[2\.2](https://arxiv.org/html/2610.41553#S2.SS2)\)\. In the second stage, the first eight layers of the embedder are frozen, and the classification head is trained on top of it with the focal loss \(Section[2\.3](https://arxiv.org/html/2610.41553#S2.SS3)\)\.

### 2\.1Encoding

First, the input textXXis encoded byPMLM\\operatorname\{PMLM\}:𝐡1,𝐡2,…,𝐡T=PMLM⁡\(X\)\\mathbf\{h\}\_\{1\},\\mathbf\{h\}\_\{2\},\\ldots,\\mathbf\{h\}\_\{T\}=\\operatorname\{PMLM\}\(X\), where𝐡t∈ℝd\\mathbf\{h\}\_\{t\}\\in\\mathbb\{R\}^\{d\}is the final layer hidden representation\. We then obtain a fixed\-size sentence representation,𝐳\\mathbf\{z\}, by mean pooling over the sequence dimension:𝐳=MeanPool⁡\(𝐡1,𝐡2,…,𝐡T\)\.\\mathbf\{z\}=\\operatorname\{MeanPool\}\(\\mathbf\{h\}\_\{1\},\\mathbf\{h\}\_\{2\},\\ldots,\\mathbf\{h\}\_\{T\}\)\.

### 2\.2Supervised Contrastive Learning

Supervised Contrastive Learning[Khosla et al\. \(2020\)](https://arxiv.org/html/2610.41553#bib.bib3)regards samples sharing the same class label as positive pairs, while all remaining samples in the batch as negatives\. Formally, given anii\-th sample, the probability assigned to a positive pairppis

pcon​\(i,p\)\\displaystyle p\_\{\\mathrm\{con\}\}\(i,p\)=exp⁡\(sim⁡\(𝐳i,𝐳p\)/τ\)∑a∈A⁡\(i\)exp⁡\(sim⁡\(𝐳i,𝐳a\)/τ\),\\displaystyle=\\frac\{\\exp\(\\operatorname\{sim\}\(\\mathbf\{z\}\_\{i\},\\mathbf\{z\}\_\{p\}\)/\\tau\)\}\{\\sum\_\{a\\in A\(i\)\}\\exp\(\\operatorname\{sim\}\(\\mathbf\{z\}\_\{i\},\\mathbf\{z\}\_\{a\}\)/\\tau\)\},\(1\)A⁡\(i\)\\displaystyle A\(i\)=\{1,2,…,2​N\}∖\{i\}\\displaystyle=\\\{1,2,\\ldots,2N\\\}\\setminus\\\{i\\\}\(2\)wheresim⁡\(⋅,⋅\)\\operatorname\{sim\}\(\\cdot,\\cdot\)is cosine similarity measuring the closeness of two embeddings,τ\\tauis a temperature hyperparameter of the distribution, andNNis the number of training samples\. We augment the samples by encoding each input text using the PMLM twice\. The randomness of the dropout layers yields two slightly different embeddings of the same sentence, which serves as the augmented positive pair[Gao et al\. \(2021\)](https://arxiv.org/html/2610.41553#bib.bib9)\. Finally, the loss function is defined as:

ℒSupCon\\displaystyle\\mathcal\{L\}\_\{\\text\{SupCon\}\}=∑i=12​N−1\|P⁡\(i\)\|​∑p∈P⁡\(i\)log⁡pcon​\(i,p\),\\displaystyle=\\sum\_\{i=1\}^\{2N\}\\frac\{\-1\}\{\|P\(i\)\|\}\\sum\_\{p\\in P\(i\)\}\\log p\_\{\\mathrm\{con\}\}\(i,p\),\(3\)P⁡\(i\)\\displaystyle P\(i\)=\{p∈A⁡\(i\)∣yp=yi\}\.\\displaystyle=\\\{p\\in A\(i\)\\mid y\_\{p\}=y\_\{i\}\\\}\.\(4\)

### 2\.3Focal Loss

The focal loss[Lin et al\. \(2017\)](https://arxiv.org/html/2610.41553#bib.bib5)is a generalization of the cross\-entropy loss that introduces a modulating factor to adjust the contribution of each sample to the total loss\. For a training example with ground\-truth classyy, it is defined as

ℒfocal=−\(1−py\)γ​log⁡py,\\mathcal\{L\}\_\{\\mathrm\{focal\}\}=\-\(1\-p\_\{y\}\)^\{\\gamma\}\\log p\_\{y\},\(5\)wherepyp\_\{y\}denotes the predicted probability of the ground\-truth class andγ≥0\\gamma\\geq 0is the focusing parameter\. Forγ=0\\gamma=0, the focal loss reduces to the standard cross\-entropy loss and forγ\>0\\gamma\>0, it down\-weights well\-classified examples and shifts training towards hard ones\.

Focal loss adaptively reweights individual training examples based on their prediction confidence\. To additionally account for the global class distribution, we combine it with the class\-level weighting scheme[Arase et al\. \(2022\)](https://arxiv.org/html/2610.41553#bib.bib1), which weights the loss for each CEFR level according to its frequency in the training data:wi=qiα∑j=1Kqjαw\_\{i\}=\\frac\{q\_\{i\}^\{\\alpha\}\}\{\\sum\_\{j=1\}^\{K\}q\_\{j\}^\{\\alpha\}\}, whereqiq\_\{i\}is the frequency of leveliiin the training set andα∈\[0,1\]\\alpha\\in\[0,1\]controls the weight strength\.

Table 1:Number of training, development, and test samples in CEFR\-SP and UniversalCEFR\.Table 2:Per\-level F1 and macro\-F1 on the CEFR\-SP and UniversalCEFR \(sentence\- and document\-level\) test sets\. For CEFR\-SP, we report mean±\\pmstd over1212seeds, discarding the best\- and worst\-performing seed by macro\-F1 \(i\.e\., averaging over the remaining1010\); for UniversalCEFR, mean±\\pmstd over33seeds\. Thebest Macro\-F1 scoresare indicated by bold fonts, while thebest per\-level F1 scoresare indicated by underlines\.

## 3Experiments

We evaluate the performance of the proposed method on multi\-granular and multilingual settings\.

### 3\.1Datasets

We use two datasets, CEFR\-SP[Arase et al\. \(2022\)](https://arxiv.org/html/2610.41553#bib.bib1), an English sentence\-level corpus, and UniversalCEFR[Imperial et al\. \(2025\)](https://arxiv.org/html/2610.41553#bib.bib2), a multilingual collection spanning multiple text granularities\. Table[1](https://arxiv.org/html/2610.41553#S2.T1)shows the number of samples in each dataset\. For the label distributions of CEFR levels, please refer to Table[4](https://arxiv.org/html/2610.41553#A2.T4)in Appendix[B](https://arxiv.org/html/2610.41553#A2)\.

CEFR\-SP provides labels assigned by two annotators\. We expand our contrastive learning setting to regard both labels as correct CEFR levels \(see Appendix[A](https://arxiv.org/html/2610.41553#A1)for more details\)\.

UniversalCEFR covers1313languages; we pool all languages and conduct separate experiments at the sentence and document level \(denoted as “UCEFR\-sent” and “UCEFR\-doc”, respectively\)\. Because C2 samples are scarce on UniversalCEFR, we merge C1 and C2 into a single C class to discuss based on stable statistics\.

![Refer to caption](https://arxiv.org/html/2610.41553v1/fig_confusion_annotators.png)Figure 1:Confusion matrices of our model on the CEFR\-SP test set against each annotator’s labels\. Cells show row\-normalised recall, averaged over1212training seeds discarding the best\- and worst\-performing seed by macro\-F1\.
### 3\.2Evaluation Metrics

Since the label distributions of all CEFR datasets are highly imbalanced, we report macro\-F1, which weights all levels equally and thus prevents the scarce levels from being dominated by the well\-represented ones\. For CEFR\-SP, where each sentence carries two annotator labels, we count a prediction as correct if it matches either label[Arase et al\. \(2022\)](https://arxiv.org/html/2610.41553#bib.bib1)\. We report the average scores of over1212random seeds for CEFR\-SP and33seeds per configuration for UniversalCEFR\.

### 3\.3Implementation & Baselines

We compare our method against two baselines: \(i\) the same architecture as our method yet trained end\-to\-end with a plain cross\-entropy loss, and \(ii\) the state\-of\-the\-art on CEFR\-SP[Arase et al\. \(2022\)](https://arxiv.org/html/2610.41553#bib.bib1), denoted as “Prototype”\. We employbert\-base\-uncasedon CEFR\-SP andxlm\-roberta\-base[Conneau et al\. \(2020\)](https://arxiv.org/html/2610.41553#bib.bib7)on UniversalCEFR as the common PMLM\. Training and hyperparameter settings are in Appendix[C](https://arxiv.org/html/2610.41553#A3)\.

### 3\.4Results and Discussions

Table[2](https://arxiv.org/html/2610.41553#S2.T2)reports per\-level F1 and macro\-F1 scores on test sets of CEFR\-SP and UniversalCEFR\. Overall,the proposed method performs best across sentence and document level assessment and monolingual and multilingual settings\.On CEFR\-SP, plain cross\-entropy training struggles on the scarce classes A1 and C2, where it exhibits both low scores and high variance across seeds\. In contrast, the prototype classifier[Arase et al\. \(2022\)](https://arxiv.org/html/2610.41553#bib.bib1)and our method both mitigate the class imbalance, improving macro\-F1 by more than1212points while remaining on par with cross\-entropy on the well\-represented intermediate levels\. On UniversalCEFR, the prototype classifier in turn falls behind cross\-entropy at almost all levels, on both granularities\. The proposed method consistently achieves the best macro\-F1 scores on all settings\.

Table 3:Silhouette scores of the test\-set embedding spaces with respect to the gold CEFR levels, averaged over the same seeds as Table[2](https://arxiv.org/html/2610.41553#S2.T2)\. For CEFR\-SP, the score is the mean over the two annotators’ label sets\.Moreover,supervised contrastive learning produces a more class\-separable embedding space\.Table[3](https://arxiv.org/html/2610.41553#S3.T3)reports the silhouette scores[Rousseeuw \(1987\)](https://arxiv.org/html/2610.41553#bib.bib6)of the final models’ test\-set embeddings \(visualized in Appendix[D](https://arxiv.org/html/2610.41553#A4)\)\. The proposed method scores highest on CEFR\-SP and at the document level UniversalCEFR, and is on par with the prototype classifier at the sentence level\.

Our analysis found thatalmost all misclassifications occur between adjacent CEFR levels\.Figure[1](https://arxiv.org/html/2610.41553#S3.F1)shows the confusion matrices of our proposed method, evaluated on CEFR\-SP against each annotator’s labels\. Errors are concentrated almost exclusively in neighboring classes:99\.1%99\.1\\%of test sentences are predicted within one level of the target for Annotator A, and97\.3%97\.3\\%for Annotator B\. These results indicate that the proposed method effectively separates distant CEFR levels than baselines but still struggles to distinguish adjacent levels, whose boundaries are inherently ambiguous\. This observation is consistent with previous findings[Arase et al\. \(2022\)](https://arxiv.org/html/2610.41553#bib.bib1);[Imperial et al\. \(2025\)](https://arxiv.org/html/2610.41553#bib.bib2)\. Although the proposed training objective improves class discrimination, it does not explicitly model the ordinal relationships among CEFR levels\. Incorporating ordinal\-aware objectives that preserve the inherent similarity between neighboring levels while improving their discriminability is a promising direction for future work\.

## 4Conclusion

In this paper, we presented a unified training objective for CEFR level assessment that combines focal loss and supervised contrastive learning to address two fundamental challenges of the task: the naturally imbalanced distribution of CEFR levels and the ambiguity between adjacent levels\. Extensive experiments on sentence\- and document\-level CEFR assessment in both monolingual and multilingual settings demonstrated that the proposed method consistently outperforms strong baselines\.

The results suggest that jointly addressing class imbalance and adjacent\-level ambiguity leads to more discriminative representations for CEFR level prediction and generalizes well across languages and text granularities\. As future work, we plan to investigate training objectives that explicitly exploit the ordinal structure of CEFR levels and to extend our approach to other educational NLP tasks involving ordinal difficulty level prediction\.

## Limitations

Although the proposed training objective consistently improves CEFR level assessment across multiple datasets and evaluation settings, several limitations remain\. First, the method treats CEFR level prediction as a classification task and does not explicitly model the ordinal relationships among levels\. While supervised contrastive learning learns more discriminative representations, it does not distinguish between adjacent and distant CEFR levels in the optimization objective\.

In addition, although we evaluate the proposed method in both monolingual and multilingual settings, we do not investigate its generalization to domains beyond the available benchmarks\. Evaluating the method on a broader range of educational resources, particularly expert domains such as law and medical, remains an important direction for future work\.

## References

- Araseet al\.\(2022\)Y\. Arase, S\. Uchida, and T\. KajiwaraCEFR\-based sentence difficulty annotation and assessment\.InProceedings of the 2022 Conference on Empirical Methods in Natural Language Processing,Y\. Goldberg, Z\. Kozareva, and Y\. Zhang \(Eds\.\),Abu Dhabi, United Arab Emirates,pp\. 6206–6219\.External Links:[Link](https://aclanthology.org/2022.emnlp-main.416/),[Document](https://dx.doi.org/10.18653/v1/2022.emnlp-main.416)Cited by:[§1](https://arxiv.org/html/2610.41553#S1.p2.1),[§2\.3](https://arxiv.org/html/2610.41553#S2.SS3.p2.1),[Table 2](https://arxiv.org/html/2610.41553#S2.T2.2.2.4),[§2](https://arxiv.org/html/2610.41553#S2.p1.1),[§3\.1](https://arxiv.org/html/2610.41553#S3.SS1.p1.1),[§3\.2](https://arxiv.org/html/2610.41553#S3.SS2.p1.1),[§3\.3](https://arxiv.org/html/2610.41553#S3.SS3.p1.1),[§3\.4](https://arxiv.org/html/2610.41553#S3.SS4.p1.1),[§3\.4](https://arxiv.org/html/2610.41553#S3.SS4.p3.1)\.
- Conneauet al\.\(2020\)A\. Conneau, K\. Khandelwal, N\. Goyal, V\. Chaudhary, G\. Wenzek, F\. Guzmán, E\. Grave, M\. Ott, L\. Zettlemoyer, and V\. StoyanovUnsupervised cross\-lingual representation learning at scale\.InProceedings of the 58th Annual Meeting of the Association for Computational Linguistics,D\. Jurafsky, J\. Chai, N\. Schluter, and J\. Tetreault \(Eds\.\),Online,pp\. 8440–8451\.External Links:[Link](https://aclanthology.org/2020.acl-main.747/),[Document](https://dx.doi.org/10.18653/v1/2020.acl-main.747)Cited by:[§3\.3](https://arxiv.org/html/2610.41553#S3.SS3.p1.1)\.
- Devlinet al\.\(2019\)J\. Devlin, M\. Chang, K\. Lee, and K\. ToutanovaBERT: pre\-training of deep bidirectional transformers for language understanding\.InProceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 \(Long and Short Papers\),J\. Burstein, C\. Doran, and T\. Solorio \(Eds\.\),Minneapolis, Minnesota,pp\. 4171–4186\.External Links:[Link](https://aclanthology.org/N19-1423/),[Document](https://dx.doi.org/10.18653/v1/N19-1423)Cited by:[§1](https://arxiv.org/html/2610.41553#S1.p2.1)\.
- Gaoet al\.\(2021\)T\. Gao, X\. Yao, and D\. ChenSimCSE: simple contrastive learning of sentence embeddings\.InProceedings of the 2021 Conference on Empirical Methods in Natural Language Processing,M\. Moens, X\. Huang, L\. Specia, and S\. W\. Yih \(Eds\.\),Online and Punta Cana, Dominican Republic,pp\. 6894–6910\.External Links:[Link](https://aclanthology.org/2021.emnlp-main.552/),[Document](https://dx.doi.org/10.18653/v1/2021.emnlp-main.552)Cited by:[§1](https://arxiv.org/html/2610.41553#S1.p3.1),[§2\.2](https://arxiv.org/html/2610.41553#S2.SS2.p1.2)\.
- Imperialet al\.\(2025\)J\. M\. Imperial, A\. Barayan, R\. Stodden, R\. Wilkens, R\. Muñoz Sánchez, L\. Gao, M\. Torgbi, D\. Knight, G\. Forey, R\. R\. Jablonkai, E\. Kochmar, R\. J\. Reynolds, E\. Ribeiro, H\. Saggion, E\. Volodina, S\. Vajjala, T\. François, F\. Alva\-Manchego, and H\. Tayyar MadabushiUniversalCEFR: enabling open multilingual research on language proficiency assessment\.InProceedings of the 2025 Conference on Empirical Methods in Natural Language Processing,C\. Christodoulopoulos, T\. Chakraborty, C\. Rose, and V\. Peng \(Eds\.\),Suzhou, China,pp\. 9703–9755\.External Links:[Link](https://aclanthology.org/2025.emnlp-main.491/),[Document](https://dx.doi.org/10.18653/v1/2025.emnlp-main.491),ISBN 979\-8\-89176\-332\-6Cited by:[§1](https://arxiv.org/html/2610.41553#S1.p2.1),[§3\.1](https://arxiv.org/html/2610.41553#S3.SS1.p1.1),[§3\.4](https://arxiv.org/html/2610.41553#S3.SS4.p3.1)\.
- Khoslaet al\.\(2020\)P\. Khosla, P\. Teterwak, C\. Wang, A\. Sarna, Y\. Tian, P\. Isola, A\. Maschinot, C\. Liu, and D\. KrishnanSupervised contrastive learning\.InAdvances in Neural Information Processing Systems,H\. Larochelle, M\. Ranzato, R\. Hadsell, M\.F\. Balcan, and H\. Lin \(Eds\.\),Vol\.33,pp\. 18661–18673\.External Links:[Link](https://proceedings.neurips.cc/paper_files/paper/2020/file/d89a66c7c80a29b1bdbab0f2a1a94af8-Paper.pdf)Cited by:[§2\.2](https://arxiv.org/html/2610.41553#S2.SS2.p1.1),[§2](https://arxiv.org/html/2610.41553#S2.p2.1)\.
- Liet al\.\(2025\)G\. Li, Y\. Arase, and N\. CrespiAligning sentence simplification with ESL learner’s proficiency for language acquisition\.InProceedings of the 2025 Conference of the Nations of the Americas Chapter of the Association for Computational Linguistics: Human Language Technologies \(Volume 1: Long Papers\),L\. Chiruzzo, A\. Ritter, and L\. Wang \(Eds\.\),Albuquerque, New Mexico,pp\. 492–507\.External Links:[Link](https://aclanthology.org/2025.naacl-long.21/),[Document](https://dx.doi.org/10.18653/v1/2025.naacl-long.21),ISBN 979\-8\-89176\-189\-6Cited by:[§1](https://arxiv.org/html/2610.41553#S1.p1.1)\.
- Linet al\.\(2017\)T\. Lin, P\. Goyal, R\. Girshick, K\. He, and P\. DollarFocal loss for dense object detection\.InProceedings of the IEEE International Conference on Computer Vision \(ICCV\),Cited by:[§1](https://arxiv.org/html/2610.41553#S1.p3.1),[§2\.3](https://arxiv.org/html/2610.41553#S2.SS3.p1.1)\.
- Loshchilov and Hutter \(2019\)I\. Loshchilov and F\. HutterDecoupled weight decay regularization\.InInternational Conference on Learning Representations,External Links:[Link](https://openreview.net/forum?id=Bkg6RiCqY7)Cited by:[Appendix C](https://arxiv.org/html/2610.41553#A3.p1.1)\.
- Rousseeuw \(1987\)P\. J\. RousseeuwSilhouettes: a graphical aid to the interpretation and validation of cluster analysis\.Journal of Computational and Applied Mathematics20,pp\. 53–65\.External Links:ISSN 0377\-0427,[Document](https://dx.doi.org/https%3A//doi.org/10.1016/0377-0427%2887%2990125-7),[Link](https://www.sciencedirect.com/science/article/pii/0377042787901257)Cited by:[Appendix C](https://arxiv.org/html/2610.41553#A3.p1.1),[§3\.4](https://arxiv.org/html/2610.41553#S3.SS4.p2.1)\.
- Vajjala and Meurers \(2012\)S\. Vajjala and D\. MeurersOn improving the accuracy of readability classification using insights from second language acquisition\.InProceedings of the Seventh Workshop on Building Educational Applications Using NLP,J\. Tetreault, J\. Burstein, and C\. Leacock \(Eds\.\),Montréal, Canada,pp\. 163–173\.External Links:[Link](https://aclanthology.org/W12-2019/)Cited by:[§1](https://arxiv.org/html/2610.41553#S1.p2.1)\.
- van der Maaten and Hinton \(2008\)L\. van der Maaten and G\. HintonVisualizing data using t\-sne\.Journal of Machine Learning Research9\(86\),pp\. 2579–2605\.External Links:[Link](http://jmlr.org/papers/v9/vandermaaten08a.html)Cited by:[Appendix D](https://arxiv.org/html/2610.41553#A4.p1.1)\.

## Appendix ADual\-Label Handling

Unlike standard classification datasets, each sentence in CEFR\-SP is annotated by two annotators, resulting in two labelsyi\(a\)y\_\{i\}^\{\(a\)\}andyi\(b\)y\_\{i\}^\{\(b\)\}per sampleii\. To adapt supervised contrastive learning to this dual\-label setting, we redefine the set of positivesP⁡\(i\)P\(i\)such that a sampleppis positive if any of its annotator labels matches any ofii’s annotator labels:

P⁡\(i\)=\{p∈A⁡\(i\)\|yi\(a\)=yp\(a\)∨yi\(a\)=yp\(b\)∨yi\(b\)=yp\(a\)∨yi\(b\)=yp\(b\)\}\\displaystyle P\(i\)=\\left\\\{p\\in A\(i\)\\;\\middle\|\\;\\begin\{aligned\} &y\_\{i\}^\{\(a\)\}=y\_\{p\}^\{\(a\)\}\\;\\vee\\\\ &y\_\{i\}^\{\(a\)\}=y\_\{p\}^\{\(b\)\}\\;\\vee\\\\ &y\_\{i\}^\{\(b\)\}=y\_\{p\}^\{\(a\)\}\\;\\vee\\\\ &y\_\{i\}^\{\(b\)\}=y\_\{p\}^\{\(b\)\}\\end\{aligned\}\\right\\\}\(6\)In other words,iiandppform a positive pair if they agree on the CEFR level under at least one combination of their respective annotators’ labels\.

## Appendix BDataset Statistics

Table[4](https://arxiv.org/html/2610.41553#A2.T4)reports the number of texts per CEFR level for all datasets and splits used in our experiments\.

Table 4:Number of texts per CEFR level\. Every CEFR\-SP sentence is labeled by two annotators\. For UniversalCEFR the C1 and C2 classes are merged into a single C class, the merged count spans the C1/C2 columns\.
## Appendix CImplementation Details

All models are optimized using AdamW[Loshchilov and Hutter \(2019\)](https://arxiv.org/html/2610.41553#bib.bib11), and training was terminated by early stopping \(patience of1010epochs\)\. The batch size was128128\. For our two\-stage approach, the stopping criterion was the silhouette score[Rousseeuw \(1987\)](https://arxiv.org/html/2610.41553#bib.bib6)of the resulting embedding space during the first stage, and macro\-F1 on the development set during the second stage\. Hyperparameters were tuned to maximize the macro\-F1 scores on the development set \(Table[5](https://arxiv.org/html/2610.41553#A3.T5)\)\.

Table 5:Hyperparameters of the proposed method, cross\-entropy \(CE\) and prototype baselines\. For the proposed method, the two learning rates \(LR\) are for the first and second stages, respectively\.
## Appendix DEmbedding Space Visualization

Figure[2](https://arxiv.org/html/2610.41553#A4.F2)shows t\-SNE projections[van der Maaten and Hinton \(2008\)](https://arxiv.org/html/2610.41553#bib.bib12)of the test\-set embedding spaces, for one representative training seed per panel\. On CEFR\-SP, the proposed method arranges the levels along a largely ordinal progression\. On UniversalCEFR, it instead separates the levels into multiple clusters, some dominated by a single level and others exhibiting the same ordinal progression\. In both datasets, overlapping levels are almost exclusively adjacent, in line with our observation that misclassifications concentrate on neighboring levels\.

\(a\)CEFR\-SP\. Both rows show the same embeddings, colored by annotator A’s and annotator B’s labels, respectively\.\(b\)UniversalCEFR, sentence level \(top\) and document level \(bottom\), with C1 and C2 merged\.
Figure 2:t\-SNE projections of the test\-set embeddings for each method \(columns\), colored by gold CEFR level\. Each panel shows a single training seed: the66th\-best of1212seeds by test macro\-F1 for CEFR\-SP and the22nd\-best of33for UniversalCEFR\. Panels are independent projections, so distances are not comparable across panels\.