Mapping CVEs to MITRE ATT&CK Techniques: A Curated Gold-Set Classifier and the Limits of LLM-Assisted Label Expansion

Hugging Face Daily Papers Papers
cve-mapping mitre-attack llm-labeling vulnerability-analysis multi-label-classification cybersecurity label-quality

Summary

This paper presents a reproducible pipeline for mapping CVEs to MITRE ATT&CK techniques using a curated gold dataset, and investigates the limits of using LLM-assisted labeling to expand training data, finding that LLM-generated labels do not reliably improve performance due to evaluation noise and label quality issues.

We present a reproducible pipeline for mapping Common Vulnerabilities and Exposures (CVEs) to MITRE ATT&CK Enterprise techniques from free-text vulnerability descriptions. Rather than relying on the CWE->CAPEC->ATT&CK derivation chain, whose table-expansion artifacts we quantify, we train a multi-label classifier on a curated gold dataset of 1,207 CVEs from expert MITRE Center for Threat-Informed Defense mappings. The resulting model approximately doubles recall@5 compared with a zero-shot embedding-similarity baseline and improves every ranking metric. We then investigate whether LLM-assisted labeling can extend the gold dataset. Initial experiments suggest contradictory conclusions: a single run indicates degraded performance, while averaging over five random seeds suggests a small gain. However, an independent replication and an expansion-size study (100--984 additional CVEs) show that the apparent improvement is an evaluation artifact. LLM-generated labels, with approximately 0.39 agreement with expert annotations, provide no reliable improvement at any expansion size and reduce rare-technique coverage at around 1,000 added CVEs (macro-F1 decreases by 0.04). The root cause is evaluation noise. Selecting checkpoints on a small test split effectively maximizes over many noisy evaluations, producing recall@5 differences of up to 0.05 between otherwise identical runs. Using a corrected protocol based on validation-split checkpoint selection, the gold-only model achieves recall@5 of (0.673 \pm 0.019), and repeating the decisive experiment confirms the null result for LLM expansion. A final scaling study shows that additional expert-curated data consistently improves performance, whereas LLM-labeled data does not, indicating that the classifier is limited by label quality rather than dataset size. All datasets, models, code, and training logs are publicly released.
Original Article
View Cached Full Text

Cached at: 07/29/26, 07:51 AM

Paper page - Mapping CVEs to MITRE ATT&CK Techniques: A Curated Gold-Set Classifier and the Limits of LLM-Assisted Label Expansion

Source: https://huggingface.co/papers/2607.25572 Published on Jul 28

·

Submitted byhttps://huggingface.co/cedricbonhomme

Cédricon Jul 29

Abstract

WepresentareproduciblepipelineformappingCommonVulnerabilitiesandExposures(CVEs)toMITREATT&CKEnterprisetechniquesfromfree-textvulnerabilitydescriptions.RatherthanrelyingontheCWE->CAPEC->ATT&CKderivationchain,whosetable-expansionartifactswequantify,wetrainamulti-labelclassifieronacuratedgolddatasetof1,207CVEsfromexpertMITRECenterforThreat-InformedDefensemappings.Theresultingmodelapproximatelydoublesrecall@5comparedwithazero-shotembedding-similaritybaselineandimproveseveryrankingmetric.WetheninvestigatewhetherLLM-assistedlabelingcanextendthegolddataset.Initialexperimentssuggestcontradictoryconclusions:asinglerunindicatesdegradedperformance,whileaveragingoverfiverandomseedssuggestsasmallgain.However,anindependentreplicationandanexpansion-sizestudy(100--984additionalCVEs)showthattheapparentimprovementisanevaluationartifact.LLM-generatedlabels,withapproximately0.39agreementwithexpertannotations,providenoreliableimprovementatanyexpansionsizeandreducerare-techniquecoverageataround1,000addedCVEs(macro-F1decreasesby0.04).Therootcauseisevaluationnoise.Selectingcheckpointsonasmalltestspliteffectivelymaximizesovermanynoisyevaluations,producingrecall@5differencesofupto0.05betweenotherwiseidenticalruns.Usingacorrectedprotocolbasedonvalidation-splitcheckpointselection,thegold-onlymodelachievesrecall@5of(0.673\pm0.019),andrepeatingthedecisiveexperimentconfirmsthenullresultforLLMexpansion.Afinalscalingstudyshowsthatadditionalexpert-curateddataconsistentlyimprovesperformance,whereasLLM-labeleddatadoesnot,indicatingthattheclassifierislimitedbylabelqualityratherthandatasetsize.Alldatasets,models,code,andtraininglogsarepubliclyreleased.

View arXiv pageView PDFProject pageGitHub3Add to collection

Get this paper in your agent:

hf papers read 2607\.25572

Don’t have the latest CLI?curl \-LsSf https://hf\.co/cli/install\.sh \| bash

Models citing this paper2

#### CIRCL/vulnerability-attack-technique-classification-roberta-base Text Classification• 0.1B• Updatedabout 2 hours ago • 125 • 1 #### CIRCL/vulnerability-attack-technique-classification-roberta-base-llm-expanded Text Classification• 0.1B• Updatedabout 2 hours ago • 44

Datasets citing this paper2

#### CIRCL/vulnerability-attack-techniques Viewer• Updatedabout 2 hours ago • 1.21k • 191 #### CIRCL/vulnerability-attack-techniques-llm-scaling Viewer• Updatedabout 2 hours ago • 984 • 72 • 1

Spaces citing this paper0

No Space linking this paper

Cite arxiv.org/abs/2607.25572 in a Space README.md to link it from this page.

Collections including this paper0

No Collection including this paper

Add this paper to acollectionto link it from this page.

Similar Articles

Gate AI: LLM Security Benchmark Evaluation Methodology and Results

arXiv cs.LG

This paper presents an evaluation methodology for LLM security detectors that addresses systematic weaknesses like per-dataset threshold tuning and undisclosed operating points. The framework uses cross-validation across 16 benchmarks, selects a single global operating point, and includes multiple diagnostics for generalization.

Measuring LLMs' impact on N-day exploits (18 minute read)

TLDR AI

This article from Anthropic evaluates how large language models like Claude Mythos Preview can accelerate the development of exploits for N-day vulnerabilities. Across tests on Firefox and Windows kernel patches, the model autonomously built working exploit chains, highlighting increased risks in the patch gap.

TADDLE: A Tool-Augmented Agent for Detecting Deficient LLM-Generated Peer Reviews

arXiv cs.AI

Introduces TADDLE, a tool-augmented agent for detecting deficient LLM-generated peer reviews, along with an expert-annotated benchmark of 1,800 reviews on 50 ICLR 2025 papers. The system decomposes detection into four specialized analysis tools and uses two-stage semi-supervised learning for binary and multi-label classification.