Dream at SemEval-2026 Task 13: SALSA for Single-Pass Machine-Generated Code Detection

# SALSA for Single-Pass Machine-Generated Code Detection
Source: [https://arxiv.org/html/2606.25102](https://arxiv.org/html/2606.25102)
## Dream at SemEval\-2026 Task 13: SALSA for Single\-Pass Machine\-Generated Code Detection

Shai Nahum\-Gefen Dream Security Ltd\. Tel Aviv, Israel shai@dreamgroup\.comElad Ben\-Zaken Dream Security Ltd\. Tel Aviv, Israel elad@dreamgroup\.com

###### Abstract

Large language models have transformed code generation, raising concerns around authorship, assessment integrity, and software trust\. SemEval\-2026 Task 13 Subtask A operationalizes detection as binary classification over code snippets, with a particular emphasis on out\-of\-distribution \(OOD\) generalization across unseen programming languages and application domains\. We propose a SALSA\-style formulation, Single\-pass Autoregressive LLM Structured Classification, that maps each class to a dedicated output token and trains the model to emit a*single\-token*label in a structured response\. Rather than engineering hand\-crafted features or decision rules, this formulation*delegates*the authorship decision to the model\. To improve OOD robustness, we combine balanced sampling across languages with parameter\-efficient fine\-tuning and conservative training \(low learning rate, single epoch\) to avoid overfitting to the training domain\. Our best system achieves OODF1=0\.789F\_\{1\}=0\.789on the official leaderboard, substantially outperforming the CodeBERT baseline \(F1=0\.305F\_\{1\}=0\.305\)\.

Dream at SemEval\-2026 Task 13: SALSA for Single\-Pass Machine\-Generated Code Detection

Ruslan BerdichevskyDream Security Ltd\.Tel Aviv, Israelruslan@dreamgroup\.comShai Nahum\-GefenDream Security Ltd\.Tel Aviv, Israelshai@dreamgroup\.comElad Ben\-ZakenDream Security Ltd\.Tel Aviv, Israelelad@dreamgroup\.com

## 1Introduction

![Refer to caption](https://arxiv.org/html/2606.25102v1/eaa15075-8e24-40ae-9982-f660f40faf8c.png)Figure 1:Overview of the SALSA classification pipeline: the structured prompt maps each class to a single token; a single LLM forward pass produces logits for the target token position; the class\-token logits are extracted and passed through a softmax to obtain the predicted label\.Large language models \(LLMs\) have revolutionized code generation, but this has important consequences for programming skills, ethics, and assessment integrity, making the detection of LLM\-generated code essential for maintaining accountability and standardsOrelet al\.\([2026b](https://arxiv.org/html/2606.25102#bib.bib18)\)\. While there is prior work on machine\-generated code detection, it often lacks domain coverage and robustness, and typically supports only a small number of programming languagesOrelet al\.\([2025b](https://arxiv.org/html/2606.25102#bib.bib15),[a](https://arxiv.org/html/2606.25102#bib.bib16),[2026a](https://arxiv.org/html/2606.25102#bib.bib19)\)\.

SemEval\-2026 Task 13 Subtask A provides an important testbed by requiring a binary label indicating whether a code snippet is machine\-generated\. A central challenge is generalization to out\-of\-distribution \(OOD\) settings, since practical detectors must handle unseen scenarios beyond the training distribution\.

Rather than solving the problem via hand\-designed features or decision rules, we*delegate*the authorship decision to the model by specifying the task as a natural language instruction, since language models are strong general\-purpose learners when prompted with task descriptionsBrownet al\.\([2020](https://arxiv.org/html/2606.25102#bib.bib17)\)\. However, using LLMs as classifiers this way is often brittle: the model may emit verbose explanations, inconsistent label formats, or multi\-token labels\. SALSA111Our code is publicly available at[https://github\.com/dreamgroupai\-ai/SALSA](https://github.com/dreamgroupai-ai/SALSA)\., Single\-pass Autoregressive LLM Structured Classification\(Berdichevskyet al\.,[2025](https://arxiv.org/html/2606.25102#bib.bib2)\), addresses this by mapping each class to a distinct output token and structuring the prompt with clear delimiters and a direct authorship question, turning classification into a reliable constrained next\-token prediction\.

During fine\-tuning, the model simultaneously learns task alignment \(following the classification instruction\) and in\-domain expertise \(adapting to the training distribution\)\. Since our primary goal is out\-of\-distribution \(OOD\) performance, we bias training toward preserving task alignment and general capabilities rather than over\-fitting to the source domain\. To this end, we run only one epoch and select the best checkpoint based on validation; prior work suggests that earlier training tends to improve general\-purpose features, while later stages may over\-specialize to the source domain and degrade OOD performanceLiu and Gurevych \([2024](https://arxiv.org/html/2606.25102#bib.bib13)\)\. We also use a low learning rate, as we empirically observe that larger learning rates tend to reduce OOD performance while improving in\-distribution metrics\. Together, these choices keep the fine\-tuned model closer to the base model, which tends to improve generalization\.

SALSA has shown consistent results across diverse classification settings, including the GLUE benchmarkWanget al\.\([2019](https://arxiv.org/html/2606.25102#bib.bib24)\)and content moderationSorensenet al\.\([2025](https://arxiv.org/html/2606.25102#bib.bib25)\); Berdichevsky \([2025](https://arxiv.org/html/2606.25102#bib.bib6)\); here we apply it to machine\-generated code detection and report strong OOD performance \(F1=0\.789F\_\{1\}=0\.789\)\.

## 2Background

Subtask A is a binary classification setting: given an input code snippet, the system predicts0\(human\-written\) or1\(machine\-generated\)\. The task is designed to stress\-test generalization across both programming languages and application scenarios: training data usesC\+\+,Python, andJavafrom an algorithmic domain, while evaluation includes \(i\) seen vs\. unseen languages and \(ii\) seen vs\. unseen domains \(research/production\), yielding four settings in total\.

We use the official dataset distribution released by the task organizers\. The dataset provides the code snippet \(code\), the binary label \(label\), and metadata including programming language \(language\)\. The released sizes for Subtask A are 500K training samples \(238K human\-written and 262K machine\-generated\) and 100K validation samples, and a private test set of 500K samples for which no labels or metadata are provided\.

## 3System Overview

### 3\.1SALSA

SALSA reframes classification as single\-token generation over an autoregressive LLM\. Each class is mapped to a unique output token, and the prompt is structured to elicit exactly that token at a fixed position\. At inference time, a single forward pass suffices: the output logits are projected onto the class tokens only, yielding an efficient classifier with minimal decoding overhead\. Figure[1](https://arxiv.org/html/2606.25102#S1.F1)illustrates the end\-to\-end pipeline, and Figure[2](https://arxiv.org/html/2606.25102#S3.F2)shows an example prompt for our task\.

#### Class\-to\-token mapping\.

For Subtask A we use label tokens"0"and"1"\(a single token each under the model tokenizer\)\.

#### Structured prompting\.

We wrap the code snippet in delimiters and ask the model whether the code is machine generated\. We require the answer in the form<ANSWER\>\#</ANSWER\>where\#is0or1\. Figure[2](https://arxiv.org/html/2606.25102#S3.F2)shows the prompt structure used for the task\. The/no\_thinkflag and an empty<think\>block suppress chain\-of\-thought reasoning in Qwen3, keeping the output to a single label tokenQwen Team \([2025](https://arxiv.org/html/2606.25102#bib.bib21)\)\.

<\|begin\_of\_text\|\>
<\|im\_start\|\> user /no\_think
Given the code: <CODE\>
s = input\(\)
pal = s \+ s\[::\-1\]
print\(pal\)
</ CODE\>
is the code machine generated?
provide answer in format:
<ANSWER\>\#Number</ANSWER\>
where the number is one of the following:
0 \- No
1 \- Yes
<\|im\_end\|\>
<\|im\_start\|\> assistant
<think\> </think\>
<ANSWER\>\#</ANSWER\>
<\|im\_end\|\>BOS tokenTask \+ inputClass mappingAnswer formatwith masked tokenFigure 2:Example SALSA\-structured prompt for machine\-generated code detection \(Subtask A\)\.

### 3\.2Implementation Details

#### Training objective

We implement SALSA by supervising only the label token position in the assistant response\. Concretely, the model is trained with a causal language modeling objective where the loss is evaluated exclusively at the final occurrence of the label token, so gradients update the model primarily to improve the next\-token distribution over the target classes\.

#### Parameter\-efficient fine\-tuning

We fine\-tune with LoRA, Low\-Rank AdaptationHuet al\.\([2022](https://arxiv.org/html/2606.25102#bib.bib1)\), to reduce overfitting risk and speed up training\.

#### Conservative fine\-tuning\.

We use a low learning rate \(5×10−65\{\\times\}10^\{\-6\}\), train for one epoch, and select the best checkpoint on validation\. We empirically observe that more aggressive training reduces OOD performance while improving in\-distribution metrics\. In particular, training beyond a single epoch consistently degrades OOD generalization, suggesting that extended exposure to the source domain causes the model to over\-specialize at the expense of its pretrained representations\.

#### Dataset splits\.

Table[1](https://arxiv.org/html/2606.25102#S3.T1)shows the per\-language and per\-label breakdown of the training \(500K\) and validation \(100K\) splits\. The dataset is heavily skewed towardPython, which accounts for 91\.5% of all samples, whileC\+\+andJavatogether comprise less than 9%\. The label distribution is roughly balanced overall \(47\.7% human vs\. 52\.3% machine\-generated\), and this balance is preserved within each language; the validation split closely mirrors the training proportions\.

Table 1:Label distribution by split and programming language\.
#### Balanced sampling

As shown in Table[1](https://arxiv.org/html/2606.25102#S3.T1), the training set is highly imbalanced across programming languages\. To prevent majority\-language dominance, we balance the training data by sampling an equal number of examples from each\(language, label\)group\. The number of samples drawn per group is determined by the size of the smallest group; samples are drawn without repetition, so each training example is used at most once\. This discards a substantial portion of Python training data, a deliberate trade\-off: preserving Python’s dominance would steer fine\-tuning toward in\-domain expertise on Python\-specific patterns at the expense of the task alignment carried over from the pretrained starting point\. In preliminary experiments, the unbalanced distribution yielded weaker task alignment \(lower OODF1F\_\{1\}\), consistent with this view\.

#### Inference

Our inference follows the SALSA principle of restricted logit projection: we generate a single token and read the model’s log\-probabilities for the class tokens only\. The predicted label is the argmax over the two target token logits\. We achieve high\-throughput inference by merging the LoRA adapter into the base model and using vLLMKwonet al\.\([2023](https://arxiv.org/html/2606.25102#bib.bib22)\)to compute logprobs efficiently\.

## 4Experimental Setup

#### Configuration\.

We fine\-tune Qwen3\-8B, Qwen3\-32B and Qwen2\.5\-72B\-InstructQwen Team \([2024](https://arxiv.org/html/2606.25102#bib.bib20)\)with LoRA adapters using the SALSA structured prompt described in Section[3](https://arxiv.org/html/2606.25102#S3)\. Full hyperparameter details are provided in Appendix[A](https://arxiv.org/html/2606.25102#A1)\. We train for up to 1 epoch, selecting the checkpoint with the best validationF1F\_\{1\}, and use left truncation at 8192 tokens to preserve the label token at the end of the prompt\.

#### Metric\.

We report macro\-F1F\_\{1\}\. For in\-distribution performance, we evaluate on the official Kaggle validation split as described in the challenge \(F1=0\.996F\_\{1\}=0\.996in our experiment\)\. For out\-of\-distribution performance, we report the competition OOD testF1F\_\{1\}\(0\.789\) on the Kaggle leaderboard\.

## 5Results

Table[2](https://arxiv.org/html/2606.25102#S5.T2)summarizes our results\. The official CodeBERT baseline scores 0\.305 OODF1F\_\{1\}\. Our best submission, Qwen2\.5\-72B\-Instruct fine\-tuned with SALSA, achieves 0\.789 OODF1F\_\{1\}, a substantial improvement over both the baseline and the zero\-shot upper bound for that model \(0\.449\)\.

Table 2:Results on SemEval\-2026 Task 13 Subtask A\. CodeBERT is the organizers’ baseline\. TestF1F\_\{1\}denotes the leaderboard OOD score\. Zero\-shot uses the SALSA prompt without fine\-tuning\.†*Note:*The official Kaggle submission achieved a test \(OOD\)F1F\_\{1\}of 0\.73 due to suboptimal checkpoint selection; a post\-competition ablation improved this to 0\.76\.#### Zero\-shot prediction bias\.

The precision/recall breakdown in Table[3](https://arxiv.org/html/2606.25102#S5.T3)reveals that the aggregateF1F\_\{1\}scores in Table[2](https://arxiv.org/html/2606.25102#S5.T2)mask qualitatively different failure modes\. Qwen3\-Next\-80B\-A3B\-Instruct is the only zero\-shot model with balanced precision and recall \(0\.707 / 0\.638\), while Qwen3\-32B and Qwen2\.5\-72B\-Instruct are strongly biased toward predicting human\-written code, achieving recall of only 0\.242 and 0\.035 respectively despite high precision\. This near\-total failure to detect machine\-generated code explains the val/OOD inversion for Qwen2\.5\-72B\-Instruct \(valF1F\_\{1\}=0\.359 vs\. OODF1F\_\{1\}=0\.449\): the OOD distribution is apparently less dominated by machine\-generated examples, so the model’s conservative bias is less penalized\. The breakdown also directly motivates our balanced sampling strategy: a model that defaults to predicting “human” will score well on a skewed distribution but generalize poorly\. The per\-language breakdown in Appendix[C](https://arxiv.org/html/2606.25102#A3)shows no strong language\-level skew in the validation set\.

Table 3:Zero\-shot validation performance\. Prec\./Rec\. denote precision and recall for the machine\-generated class;F1F\_\{1\}is macro\-averaged\. The high\-precision/low\-recall asymmetry indicates a bias toward predicting human\-written code; per\-language breakdown in Appendix[C](https://arxiv.org/html/2606.25102#A3)\.
#### Validation results\.

All fine\-tuned models achieve near\-perfect validationF1F\_\{1\}: 0\.991 \(8B\), 0\.994 \(32B\), and 0\.996 \(72B\), essentially matching their training\-splitF1F\_\{1\}\. The absence of a train–val gap indicates in\-distribution fitting without overfitting\. While the differences are small in absolute terms, the scores increase monotonically with model size, consistent with the OOD trend\. We use the validation score for checkpoint selection within training, but do not draw strong conclusions from cross\-model comparisons given the narrow range\. Extended validation error analysis is provided in Appendix[D](https://arxiv.org/html/2606.25102#A4)\.

#### Effect of fine\-tuning\.

Notably, all zero\-shot models already outperform the CodeBERT baseline \(0\.305\), with scores of 0\.359–0\.672, suggesting that general\-purpose LLMs carry useful prior knowledge for this task even without task\-specific training\. Fine\-tuning with SALSA yields substantial further gains for larger models, with both Qwen2\.5\-72B\-Instruct and Qwen3\-32B improving markedly over their zero\-shot counterparts\. For Qwen3\-8B, the gain is negligible, indicating that fine\-tuning alone is insufficient for small models in this setting\.

## 6Conclusion

We applied SALSA to SemEval\-2026 Task 13 Subtask A, achieving an OODF1F\_\{1\}of 0\.789 with Qwen2\.5\-72B\-Instruct, placing among the top scores on the official leaderboard\. SALSA is a natural fit for this task: by reducing the output space to two tokens and delegating the authorship decision to the model via structured prompting and tuning, it yields stable, efficient classification\. OOD performance improves monotonically with model scale, which we attribute to larger models retaining more of their pretrained knowledge after conservative fine\-tuning rather than over\-specializing to the training domain\. Interestingly, Qwen2\.5\-72B\-Instruct leads after fine\-tuning, yet its zero\-shot performance is clearly weaker than Qwen3 models, suggesting that Qwen3’s stronger instruction alignment is beneficial in the zero\-shot regime while Qwen2\.5’s broader pretraining knowledge transfers better under fine\-tuning\. The MoE model achieves the strongest zero\-shot alignment; since our current implementation does not support MoE fine\-tuning, exploring that direction is left for future work\. Future work should focus on improving task instruction alignment without inducing training\-domain specialization\.

## References

- R\. Berdichevsky, S\. Nahum\-Gefen, and E\. Ben Zaken \(2025\)SALSA: single\-pass autoregressive LLM structured classification\.Note:arXiv preprintarXiv:2510\.22691External Links:[Link](https://arxiv.org/abs/2510.22691)Cited by:[§1](https://arxiv.org/html/2606.25102#S1.p3.1)\.
- R\. Berdichevsky \(2025\)4th place solution \(jigsaw agile community rules\) — salsa\-based writeup reference\.Note:Kaggle WriteupExternal Links:[Link](https://www.kaggle.com/competitions/jigsaw-agile-community-rules/writeups/4th-place-solution)Cited by:[§1](https://arxiv.org/html/2606.25102#S1.p5.1)\.
- T\. B\. Brown, B\. Mann, N\. Ryder, M\. Subbiah, J\. Kaplan, P\. Dhariwal, A\. Neelakantan, P\. Shyam, G\. Sastry, A\. Askell, S\. Agarwal, A\. Herbert\-Voss, G\. Krueger, T\. Henighan, R\. Child, A\. Ramesh, D\. M\. Ziegler, J\. Wu, C\. Winter, C\. Hesse, M\. Chen, E\. Sigler, M\. Litwin, S\. Gray, B\. Chess, J\. Clark, C\. Berner, S\. McCandlish, A\. Radford, I\. Sutskever, and D\. Amodei \(2020\)Language models are few\-shot learners\.InAdvances in Neural Information Processing Systems,External Links:[Link](https://arxiv.org/abs/2005.14165)Cited by:[§1](https://arxiv.org/html/2606.25102#S1.p3.1)\.
- E\. J\. Hu, Y\. Shen, P\. Wallis, Z\. Allen\-Zhu, Y\. Li, S\. Wang, L\. Wang, and W\. Chen \(2022\)LoRA: low\-rank adaptation of large language models\.External Links:2106\.09685,[Link](https://arxiv.org/abs/2106.09685)Cited by:[§3\.2](https://arxiv.org/html/2606.25102#S3.SS2.SSS0.Px2.p1.1)\.
- W\. Kwon, Z\. Li, S\. Zhuang, Y\. Sheng, L\. Zheng, C\. H\. Yu, J\. E\. Gonzalez, H\. Zhang, and I\. Stoica \(2023\)Efficient memory management for large language model serving with pagedattention\.InProceedings of the 29th Symposium on Operating Systems Principles,Cited by:[Appendix B](https://arxiv.org/html/2606.25102#A2.p1.1),[§3\.2](https://arxiv.org/html/2606.25102#S3.SS2.SSS0.Px6.p1.1)\.
- C\. C\. Liu and I\. Gurevych \(2024\)Early period of training impacts adaptation for out\-of\-distribution generalization: an empirical study\.Note:arXiv preprintarXiv:2403\.15210External Links:[Link](https://arxiv.org/pdf/2403.15210)Cited by:[§1](https://arxiv.org/html/2606.25102#S1.p4.1)\.
- D\. Orel, D\. Azizov, and P\. Nakov \(2025a\)CoDet\-m4: detecting machine\-generated code in multi\-lingual, multi\-generator and multi\-domain settings\.InFindings of the Association for Computational Linguistics: ACL 2025,W\. Che, J\. Nabende, E\. Shutova, and M\. T\. Pilehvar \(Eds\.\),Vienna, Austria,pp\. 10570–10593\.External Links:[Link](https://aclanthology.org/2025.findings-acl.550/),ISBN 979\-8\-89176\-256\-5Cited by:[§1](https://arxiv.org/html/2606.25102#S1.p1.1)\.
- D\. Orel, D\. Azizov, I\. Paul, Y\. Wang, I\. Gurevych, and P\. Nakov \(2026a\)AICD bench: a challenging benchmark for AI\-generated code detection\.InProceedings of the 19th Conference of the European Chapter of the Association for Computational Linguistics \(Volume 1: Long Papers\),Rabat, Morocco\.External Links:[Link](https://arxiv.org/abs/2602.02079)Cited by:[§1](https://arxiv.org/html/2606.25102#S1.p1.1)\.
- D\. Orel, D\. Azizov, I\. Paul, Y\. Wang, I\. Gurevych, and P\. Nakov \(2026b\)SemEval\-2026 task 13: detecting machine\-generated code with multiple programming languages, generators, and application scenarios\.InProceedings of the 20th International Workshop on Semantic Evaluation \(SemEval\-2026\),San Diego, USA\.Cited by:[§1](https://arxiv.org/html/2606.25102#S1.p1.1)\.
- D\. Orel, I\. Paul, I\. Gurevych, and P\. Nakov \(2025b\)Droid: a resource suite for ai\-generated code detection\.InProceedings of the 2025 Conference on Empirical Methods in Natural Language Processing,pp\. 31251–31277\.Cited by:[§1](https://arxiv.org/html/2606.25102#S1.p1.1)\.
- Qwen Team \(2024\)Qwen2\.5 technical report\.External Links:2412\.15115,[Link](https://arxiv.org/abs/2412.15115)Cited by:[§4](https://arxiv.org/html/2606.25102#S4.SS0.SSS0.Px1.p1.1)\.
- Qwen Team \(2025\)Qwen3 technical report\.External Links:2505\.09388,[Link](https://arxiv.org/abs/2505.09388)Cited by:[§3\.1](https://arxiv.org/html/2606.25102#S3.SS1.SSS0.Px2.p1.1)\.
- J\. Rasley, S\. Rajbhandari, O\. Ruwase, and Y\. He \(2020\)DeepSpeed: system optimizations enable training deep learning models with over 100 billion parameters\.InProceedings of the 26th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining,pp\. 3505–3514\.Cited by:[Appendix A](https://arxiv.org/html/2606.25102#A1.p1.4)\.
- J\. Sorensen, L\. Dos Santos, L\. Vasserman, M\. Cruz, T\. Acosta, and W\. Reade \(2025\)Jigsaw — agile community rules classification\.Note:[https://kaggle\.com/competitions/jigsaw\-agile\-community\-rules](https://kaggle.com/competitions/jigsaw-agile-community-rules)KaggleCited by:[§1](https://arxiv.org/html/2606.25102#S1.p5.1)\.
- A\. Wang, A\. Singh, J\. Michael, F\. Hill, O\. Levy, and S\. R\. Bowman \(2019\)GLUE: a multi\-task benchmark and analysis platform for natural language understanding\.InProceedings of the 7th International Conference on Learning Representations,External Links:[Link](https://openreview.net/forum?id=rJ4km2R5t7)Cited by:[§1](https://arxiv.org/html/2606.25102#S1.p5.1)\.

Table 4:Zero\-shot per\-language results on the validation set\. TP/TN/FP/FN are counts for the machine\-generated class \(label=1\)\. The human\-prediction bias of Qwen2\.5\-72B\-Instruct and Qwen3\-32B is consistent across all languages\.## Appendix AHardware and Hyperparameters

All models were fine\-tuned on 2 NVIDIA B200 GPUs using DeepSpeedRasleyet al\.\([2020](https://arxiv.org/html/2606.25102#bib.bib23)\)ZeRO stage 0 and for more than 70B stage 3\. We trained for at most one epoch with a learning rate of5×10−65\{\\times\}10^\{\-6\}, a per\-GPU batch size of 1, and gradient accumulation over 32 steps \(effective batch size 64\)\. Input sequences were left\-truncated to 8,192 tokens to ensure the label token is always retained at the end of the prompt\. LoRA adapters were configured with rankr=16r\{=\}16, scaling factorα=32\\alpha\{=\}32, and dropout0\.050\.05\.

## Appendix BInference Runtime

Table[5](https://arxiv.org/html/2606.25102#A2.T5)reports wall\-clock inference times for a full evaluation run over the 500K test records, measured on 2 B200 GPUs using vLLMKwonet al\.\([2023](https://arxiv.org/html/2606.25102#bib.bib22)\)\.

Table 5:Wall\-clock inference time for 500K records on 2 B200 GPUs using vLLM\.
## Appendix CZero\-Shot Per\-Language Breakdown

Table[4](https://arxiv.org/html/2606.25102#A0.T4)reports precision, recall, and macro\-F1F\_\{1\}for each zero\-shot model broken down by programming language on the validation set\. The human\-prediction bias of Qwen2\.5\-72B\-Instruct and Qwen3\-32B is consistent across all three languages, confirming it is a model\-level property rather than a language\-specific artifact\. Qwen3\-Next\-80B\-A3B\-Instruct maintains the most balanced recall across languages, though recall drops notably forC\+\+andJavacompared toPython\.

## Appendix DValidation Error Analysis

We analyze the prediction errors of our lead Qwen3\-32B checkpoint on the 100K validation split\. The model reaches a macro\-F1F\_\{1\}of0\.9940\.994, with 601 misclassified samples\.

#### Confusion Matrix\.

A slight bias toward the human label can be observed in Table[6](https://arxiv.org/html/2606.25102#A4.T6)\. However, because the overall separation is nearly perfect, this effect is too small to support a strong conclusion\.

Table 6:Validation confusion matrix for Qwen3\-32B\.
#### Per\-language breakdown\.

Despite balanced sampling across \(language,label\) groups,Pythonretains a noticeably lower error rate thanC\+\+andJava\(Table[7](https://arxiv.org/html/2606.25102#A4.T7)\), likely reflecting broader Python coverage in Qwen3’s pretraining\.

Table 7:Per\-language validation performance for fine\-tuned Qwen3\-32B\. “Err\.” is the number of misclassified samples; precision, recall, andF1F\_\{1\}are macro\-averaged\.
#### Per\-generator error rate\.

Table[8](https://arxiv.org/html/2606.25102#A4.T8)reports the validation error rate for each AI source model\. The hardest generators to detect are from the IBM Granite family, followed byCodeLlama\-34b\-Instructanddeepseek\-coder\-6\.7b\-base\. The table suggests that*base*variants are often harder to detect than*instruct*variants, though this trend is not uniform\. This may indicate that instruction tuning can produce more stylized, template\-like outputs that are easier to separate from human code\.

Table 8:Per\-generator validation error rate for fine\-tuned Qwen3\-32B\.