SkillTrace: Multi-Trace Provenance Auditing for LLM-Agent Skill Reuse
Summary
Presents SkillTrace, a multi-trace provenance auditing framework for LLM-agent skill reuse that extracts expression, implementation, and operational traces, achieving strong accuracy on a benchmark and enabling large-scale wild audits.
View Cached Full Text
Cached at: 08/07/26, 07:45 AM
# SkillTrace: Multi-Trace Provenance Auditing for LLM-Agent Skill Reuse
Source: [https://arxiv.org/html/2608.05204](https://arxiv.org/html/2608.05204)
Jialuo Chen12, Minghe Wang2, Lingqi Jiang2, Jianan Ma31, Xinhao Deng14, Xiaohu Du1, Ruixiao Lin2, Yunhao Feng5, Linkang Du6, Jingyi Wang2
###### Abstract
LLM\-agent ecosystems are rapidly growing around reusable skills: mixed\-modality packages of metadata, natural\-language instructions, code, tools, references, and operational workflows\. As skills become marketplace artifacts, auditing their reuse is no longer the same problem as ordinary code clone detection\. Existing detectors target single\-modality source code or whole\-package similarity, yet skill reuse evidence is distributed across authored text, implementation fragments, and operational structure\. As a result, they can miss reuse that preserves only one part of a skill\. We presentSkillTrace, a multi\-trace provenance auditing framework for LLM\-agent skill reuse\.SkillTraceextracts three provenance traces: Expression, Implementation, and Operational\. It represents the Operational Trace as a Skill Operational Graph \(SOG\) that captures activation, procedure, and resource\-flow structure\. An LLM assists only the Operational\-trace extraction, once at ingestion; at audit timeSkillTracecompares cached traces deterministically, calibrates each trace against same\-function strict negatives, and reports which trace supports a reuse decision\. OnSkillTrace\-Bench, with 820 transformed reuse positives over 100 marketplace anchors and 751 negative controls,SkillTraceachieves AUROC 0\.938 and F1 0\.898\. A 36,446\-skill wild audit further shows that trace\-attributed evidence surfaces actionable reuse review queues beyond repository\-level baselines\.
## IIntroduction
LLM\-agent ecosystems are rapidly growing around*skills*: reusable packages that let an agent acquire a capability by loading natural\-language instructions, executable snippets, tool interfaces, reference files, and examples into its context\[[5](https://arxiv.org/html/2608.05204#bib.bib65),[4](https://arxiv.org/html/2608.05204#bib.bib25),[3](https://arxiv.org/html/2608.05204#bib.bib24)\]\. Unlike a traditional code artifact, a skill is mixed\-modality and agent\-facing: its behavior is shaped not only by code, but also by instructions, activation cues, tool\-use rules, and workflow descriptions\. As public registries and marketplaces make skills easy to publish, adapt, and redistribute\[[18](https://arxiv.org/html/2608.05204#bib.bib68)\], they are becoming software\-like components whose reuse must be understood and governed\.
Reuse is not inherently harmful\. Developers may fork a skill, adapt it to another host, extract a useful script, port a workflow across registries, or instantiate many skills from a common template\. The problem is provenance visibility: current catalogs mix independent implementations, template\-generated variants, near\-duplicates, and transformed derivatives that all appear as separate marketplace entries\. Recent ecosystem studies report substantial reuse and clone\-like relations\[[15](https://arxiv.org/html/2608.05204#bib.bib51)\], and show that exploitable flaws can propagate when vulnerable skills are copied, forked, or repackaged\[[21](https://arxiv.org/html/2608.05204#bib.bib58)\]\. Without provenance\-aware reuse analysis, it becomes difficult to measure ecosystem growth, construct de\-duplicated benchmarks, remediate propagated flaws, or maintain quality\-aware registries\. The underlying task is provenance auditing: given a reference skill and a candidate, decide whether the candidate inherits provenance\-bearing artifacts from the reference rather than merely serving the same purpose\.
Existing reuse and clone detectors fit this setting only partially\. Source\-code clone detectors such as MOSS\[[32](https://arxiv.org/html/2608.05204#bib.bib41)\], JPlag\[[27](https://arxiv.org/html/2608.05204#bib.bib42)\], Deckard\[[12](https://arxiv.org/html/2608.05204#bib.bib43)\], and SourcererCC\[[31](https://arxiv.org/html/2608.05204#bib.bib46)\]are designed for relatively uniform code substrates\. Repository\-level fingerprinting methods such as sdhash\[[29](https://arxiv.org/html/2608.05204#bib.bib53)\]and ssdeep\[[16](https://arxiv.org/html/2608.05204#bib.bib67)\]compare whole\-package byte or token content, and flat\-text similarity collapses a skill into a single document representation\. These techniques are useful for broad near\-duplicate discovery, but skill reuse evidence may be distributed across authored text, implementation fragments, and operational design\. A repository\-level score cannot explain whether a candidate inherited a script, a workflow, or only a common scaffold\.
Figure 1:SkillTraceat a glance\. The auditor extracts Expression, Implementation, and Operational traces from reference and candidate skill packages\. The Operational Trace is represented as a Skill Operational Graph \(SOG\) with Activation, Procedure, and Resource\-Flow views\. At audit time, cached traces are compared deterministically andSkillTraceemits an audit report\.The central challenge is therefore not whether two skills resemble each other globally, but which provenance survives transformation, and this is hard because a skill’s reusable value is unevenly distributed\. The value often concentrates in a few artifacts, such as a debugged support script, or a carefully designed tool\-orchestration workflow, rather than in the surrounding boilerplate\. A derivative can preserve these high\-value artifacts while reorganizing files, rewriting documentation, renaming identifiers, porting the package to another host, or padding it with unrelated content\. Under such transformations global resemblance may be too weak to flag the relation even though concrete artifacts are inherited\. Conversely, two independently developed skills that solve the same task may resemble each other globally since they share templates, APIs, or generic workflows\. Skill reuse auditing must therefore recover transformed inheritance while controlling false attribution on same\-function independent skills\.
Our key insight is that skill reuse leaves complementary traces\. Expression evidence may vanish when descriptions, examples, or trigger prose are paraphrased, and implementation evidence may vanish when scripts, commands, or API calls are rewritten; yet operational evidence often remains: a derivative that preserves the skill’s functionality must still retain much of its activation logic, task procedure, and resource flow\. Rewriting all three at once is possible, but it approaches the cost of building a new skill from scratch and so erodes the very value that motivated the reuse\. A clean\-room implementation of the same task, by contrast, may share high\-level task semantics yet should not consistently preserve the same expression choices, implementation fragments, and operational structure\. This motivates trace\-attributed auditing: rather than producing one global resemblance score, an auditor should ask which provenance trace, if any, survives in the candidate\.
Based on this insight, we presentSkillTrace, a trace\-attributed framework for auditing reuse provenance in LLM\-agent skills\. As shown in Fig\.[1](https://arxiv.org/html/2608.05204#S1.F1),SkillTracehas two stages\. During trace extraction, it extracts three provenance traces from each skill package:*Expression*, capturing authored natural\-language content;*Implementation*, capturing executable realization such as scripts, commands, and API\-use patterns; and*Operational*, capturing host\-aware execution design\. The Operational Trace is represented as a Skill Operational Graph \(SOG\) with Activation, Procedure, and Resource\-Flow views\. During reuse detection,SkillTracecompares cached traces deterministically, calibrates each trace against same\-function strict negatives, and emits a trace\-attributed evidence report for human review\. Expression and Implementation are computed deterministically, while an LLM is used only once at ingestion to normalize each skill’s mixed\-modality content into its Operational Graph; audits then run over cached traces with no further LLM calls\.
To evaluateSkillTrace, we constructSkillTrace\-Bench, a trace\-preservation benchmark over real marketplace anchors and same\-function independent controls\. The benchmark includes transformed reuse positives covering repackaging, porting or relocation across hosts, and partial replacement; same\-function strict negatives mined from the real corpus; and metadata\-only clean\-room controls that probe the expected boundary of artifact\-based auditing\. We further audit a 36,446\-skill public corpus to test whetherSkillTracecan build actionable reuse review queues beyond controlled benchmarks\.
Our contributions include:
- •We formulate post\-hoc skill reuse provenance auditing as a software engineering problem over heterogeneous LLM\-agent skill packages, separating inherited provenance\-bearing traces from same\-function independent similarity\.
- •We presentSkillTrace, a trace\-attributed audit framework that extracts Expression, Implementation, and Operational traces\. Its Operational Trace is represented as a Skill Operational Graph, enabling reviewable evidence about which part of a skill survived a transformation\.
- •We constructSkillTrace\-Bench, a benchmark with 820 transformed reuse positives over 100 real marketplace anchors and 751 negative controls\.SkillTracereaches AUROC 0\.938 and F1 0\.898\.
- •We conduct a 36,446\-skill marketplace audit showing that trace\-attributed evidence surfaces actionable reuse review queues, including trace\-specific candidates that repository\-level similarity under\-prioritizes\.
## IIBackground
### II\-ASoftware Reuse Detection
Software reuse detection has long studied how to recognize copied, adapted, or independently reimplemented code\[[30](https://arxiv.org/html/2608.05204#bib.bib39)\]\. Classical clone detectors compare normalized token streams \(MOSS\[[32](https://arxiv.org/html/2608.05204#bib.bib41)\], JPlag\[[27](https://arxiv.org/html/2608.05204#bib.bib42)\], CCFinder\[[14](https://arxiv.org/html/2608.05204#bib.bib52)\], SourcererCC\[[31](https://arxiv.org/html/2608.05204#bib.bib46)\]\), while tree\- and dependence\-graph methods compare structure to resist local edits \(Deckard\[[12](https://arxiv.org/html/2608.05204#bib.bib43)\], GPLAG\[[19](https://arxiv.org/html/2608.05204#bib.bib54)\]\)\. Fuzzy digests and resemblance measures compare larger artifacts \(sdhash\[[29](https://arxiv.org/html/2608.05204#bib.bib53)\], Broder\[[6](https://arxiv.org/html/2608.05204#bib.bib56)\]\), and neural representations provide semantic code or text similarity signals\[[9](https://arxiv.org/html/2608.05204#bib.bib47),[35](https://arxiv.org/html/2608.05204#bib.bib48),[28](https://arxiv.org/html/2608.05204#bib.bib49)\]\. A related idea is the*software birthmark*: an intrinsic property that tends to survive renaming, reformatting, and local refactoring because removing it would change the program’s behavior\[[33](https://arxiv.org/html/2608.05204#bib.bib55)\]\. Birthmarks are useful because they focus on behavior\-bearing structure, not only surface text\. Skills need an analogous but broader artifact model\. A skill may preserve authored prose, executable fragments, or host\-facing operational design even when the rest of the package is rewritten\.SkillTracefollows this lineage by treating Operational Trace as a skill\-level operational birthmark, while keeping Expression and Implementation traces for textual and executable reuse evidence\.
### II\-BLLM\-Agent Skills and Tool Ecosystems
LLM\-agent capabilities are increasingly packaged as reusable artifacts\[[34](https://arxiv.org/html/2608.05204#bib.bib23),[41](https://arxiv.org/html/2608.05204#bib.bib72),[13](https://arxiv.org/html/2608.05204#bib.bib73)\]\. OpenAI’s GPTs package custom instructions and knowledge for distribution through the GPT Store\[[24](https://arxiv.org/html/2608.05204#bib.bib30),[25](https://arxiv.org/html/2608.05204#bib.bib31)\]; Anthropic’s Agent Skills define filesystem packages centered on aSKILL\.mdmanifest with optional scripts and references progressively disclosed to the agent\[[3](https://arxiv.org/html/2608.05204#bib.bib24),[4](https://arxiv.org/html/2608.05204#bib.bib25)\]; and the Model Context Protocol \(MCP\) standardizes tool and data connections across agent platforms\[[2](https://arxiv.org/html/2608.05204#bib.bib26),[23](https://arxiv.org/html/2608.05204#bib.bib27),[10](https://arxiv.org/html/2608.05204#bib.bib28),[40](https://arxiv.org/html/2608.05204#bib.bib29)\]\. Recent studies of agent skills and tool ecosystems document the same shift toward reusable, tool\-mediated agent execution\[[17](https://arxiv.org/html/2608.05204#bib.bib36),[36](https://arxiv.org/html/2608.05204#bib.bib37),[21](https://arxiv.org/html/2608.05204#bib.bib58),[20](https://arxiv.org/html/2608.05204#bib.bib38)\]\. This packaging is now reflected in public and paid skill marketplaces\. Directories such as ClawHub111[https://clawhub\.ai/](https://clawhub.ai/)and SkillHub222[https://skillhub\.club/](https://skillhub.club/)index reusable skills with creator and listing metadata, while paid marketplaces such as Agensi333[https://www\.agensi\.io/](https://www.agensi.io/)support buying and selling skill packages\. This does not make reuse inherently wrongful, but it makes provenance visibility operationally important for registry maintenance, creator\-facing review, and marketplace governance\.
The closest measurement work is Kimet al\.’s study of tool cloning in agentic\-AI ecosystems\[[15](https://arxiv.org/html/2608.05204#bib.bib51)\], which measures repository\-level Jaccard and ssdeep similarity across MCP and Skills repositories and manually validates high\-similarity buckets\. We use this view as a deployable baseline:*RepoClone*denotes our implementation of Kimet al\.’s repository\-level Jaccard/ssdeep similarity over normalized skill packages\. Our audit question is different: whether a candidate skill preserves provenance\-bearing traces under transformed reuse, even when whole\-package similarity is low\.
### II\-CPrompt and Agent Artifact Protection
Prompt\-level protection is related but narrower\. Watermarking schemes such as PromptCARE\[[39](https://arxiv.org/html/2608.05204#bib.bib2)\]and PromptCOS\[[37](https://arxiv.org/html/2608.05204#bib.bib3)\]protect prompts before release, while prompt leaking, stealing, and injection work shows that proprietary instructions can be exposed or abused in practice\[[11](https://arxiv.org/html/2608.05204#bib.bib6),[38](https://arxiv.org/html/2608.05204#bib.bib5),[8](https://arxiv.org/html/2608.05204#bib.bib7),[22](https://arxiv.org/html/2608.05204#bib.bib8),[26](https://arxiv.org/html/2608.05204#bib.bib71)\]\. Agent artifacts extend this problem beyond flat prompt text: skills bundle instructions with code, tools, references, and operational procedures\. Prior protection mechanisms therefore complement rather than replace post\-hoc reuse auditing\.SkillTraceassumes two published packages are available and asks whether observable provenance traces survive marketplace transformations, without modifying or watermarking either artifact\.
## IIISkillTrace Design
### III\-AProblem Definition
We study*post\-hoc skill reuse auditing*\. Given a reference skillRRand a candidate skillCC, the goal is to decide whetherCCis derived fromRR\(a reuse candidate\) or is an independently developed skill\. A skill differs from a flat prompt or a code repository: it may contain natural\-language instructions, executable code, reference documents and embedded workflows\. Reuse may therefore preserve only part of the reference skill\. A candidate may inherit the reference wording, executable implementation, or operational design while rewriting the rest of the package\. Conversely, two independently developed skills may solve the same task and share APIs or templates while preserving no provenance\-bearing details\.
We call\(R,C\)\(R,C\)a*reuse candidate*whenCCpreserves at least one concrete provenance trace ofRRbeyond the similarity expected from same\-function independent skills\.SkillTraceis not a legal infringement detector; it produces calibrated, trace\-attributed evidence for review\.
Algorithm 1SkillTraceaudit procedure1:Reference skill
RR, candidate skill
CC
2:Reuse score and evidence report
3:
\(τE,τI,τO\)←LoadCalibratedThresholds\(\)\(\\tau\_\{E\},\\tau\_\{I\},\\tau\_\{O\}\)\\leftarrow\\textsc\{LoadCalibratedThresholds\}\(\)
4:
β←LoadDecisionBound\(\)\\beta\\leftarrow\\textsc\{LoadDecisionBound\}\(\)⊳\\trianglerightdefaultβ=1\.0\\beta=1\.0
5:
Tr←ExtractTraces\(R\)T\_\{r\}\\leftarrow\\textsc\{ExtractTraces\}\(R\)
6:
Tc←ExtractTraces\(C\)T\_\{c\}\\leftarrow\\textsc\{ExtractTraces\}\(C\)⊳\\trianglerightT=\(TE,TI,TO\)T=\(T\_\{E\},T\_\{I\},T\_\{O\}\)
7:for
k∈\{E,I,O\}k\\in\\\{E,I,O\\\}do
8:
sk←TraceSimilarity\(Trk,Tck\)s\_\{k\}\\leftarrow\\textsc\{TraceSimilarity\}\(T\_\{r\}^\{k\},T\_\{c\}^\{k\}\)
9:
zk←sk/τkz\_\{k\}\\leftarrow s\_\{k\}/\\tau\_\{k\}
10:endfor
11:
z←max\(zE,zI,zO\)z\\leftarrow\\max\(z\_\{E\},z\_\{I\},z\_\{O\}\)
12:
ℰ←CollectEvidence\(Tr,Tc,\{k:zk≥β\}\)\\mathcal\{E\}\\leftarrow\\textsc\{CollectEvidence\}\(T\_\{r\},T\_\{c\},\\\{k:z\_\{k\}\\geq\\beta\\\}\)
13:if
z≥βz\\geq\\betathen
14:return
Surfaced\(z,ℰ\)\\textsc\{Surfaced\}\(z,\\mathcal\{E\}\)
15:else
16:return
NotSurfaced\(z,ℰ\)\\textsc\{NotSurfaced\}\(z,\\mathcal\{E\}\)
17:endif
### III\-BFramework Overview
SkillTracetreats skill reuse as*trace preservation*\. Instead of collapsing a skill pair into one whole\-package similarity score, it extracts three provenance traces and asks which trace, if any, remains observable in the candidate\.
The pipeline has two stages, shown in Fig\.[1](https://arxiv.org/html/2608.05204#S1.F1)\. First, during1\) Trace Extraction,SkillTraceparses each skill package once and extracts an*Expression Trace*for authored textual choices, an*Implementation Trace*for executable realization, and an*Operational Trace*for host\-aware execution design\. The Operational Trace is represented as a Skill Operational Graph \(SOG\)\. The extracted traces are cached and versioned, so repeated audits over the same skill do not require re\-extraction\. Second, during2\) Reuse Detection,SkillTracecompares the cached traces ofRRandCC\. Each trace produces a raw similarity score and has its own threshold calibrated on same\-function strict negatives\. A pair is surfaced when the calibrated MaxFusion score meets the decision bound\. The output is an evidence report that records the firing trace, raw and calibrated scores, and matched evidence pointers\. Algorithm[1](https://arxiv.org/html/2608.05204#alg1)summarizes the audit procedure\.
### III\-CMulti\-Trace Extraction
SkillTraceextracts three provenance traces from each skill package: Expression, Implementation, and Operational Trace \(Table[I](https://arxiv.org/html/2608.05204#S3.T1)\)\. They correspond to three ways in which reuse can survive rewriting\. A candidate may preserve the authored wording, the executable realization, or the operational design of the reference skill\. Keeping the traces separate letsSkillTracereport which part of the reference remains observable instead of relying on a single whole\-package score\.
#### III\-C1Expression Trace
Expression Trace captures*how a skill is written*\. It is extracted from natural\-language substrates: metadata, descriptions, when\-to\-use fields, section prose, instructions, reference files, and assets\. The extractor tokenizes these fields deterministically\. Common stopwords, markdown boilerplate, and generic skill\-scaffold terms are removed\.
Expression evidence is useful when a derivative keeps documentation, examples, or language\. Its limitation is also clear: prose can often be paraphrased without changing the underlying skill behavior\.
#### III\-C2Implementation Trace
Implementation Trace captures*how a skill is realized in executable form*\. It is extracted from inline code, support scripts, shell commands, API calls, configuration schemas, and command\-line interfaces\. Following token\-based code clone detection\[[14](https://arxiv.org/html/2608.05204#bib.bib52),[31](https://arxiv.org/html/2608.05204#bib.bib46)\],SkillTraceuses a language\-agnostic token representation as the default implementation trace\. This choice matches skill packages, which often mix Python, JavaScript, shell snippets, and natural\-language command recipes\.
Implementation evidence is useful when a derivative keeps code templates, scripts, API call patterns, or command recipes while rewriting the surrounding documentation\. It weakens when the code is removed or rewritten in another language\.
TABLE I:Three provenance traces extracted bySkillTrace\. Expression and Implementation adapt traditional text/code reuse signals to heterogeneous skill packages, while Operational Trace is represented as a Skill Operational Graph \(SOG\), a skill\-level operational birthmark\.TracePrimary substrateCapturesExpressionNL textAuthored wording, examples, trigger proseImplementationCode / commandsScripts, API tokens, command interfacesOperationalSkill Operational GraphActivation, procedure, tool/resource flow
#### III\-C3Operational Trace
Operational Trace captures*how a skill works when hosted by an agent*\. This trace is specific to skills: a skill is not only text or code, but also a package that must be discovered, activated, assembled into context, and executed through tools and resources\.SkillTracerepresents this trace as a typed*Skill Operational Graph*\(SOG\)\.
### III\-DSkill Operational Graph
Traditional software reuse detection often relies on code birthmarks: structural properties that tend to survive renaming, formatting changes, and local refactoring\. Agent skills need a higher\-level analogue because their behavior is distributed across instructions, code snippets, tool descriptions, reference files, activation rules, and host\-specific loading behavior\. A derivative may therefore preserve the way the skill operates even when its prose, file layout, or code identifiers have changed\.
SkillTracerepresents this higher\-level identity as a*Skill Operational Graph*\. The SOG is a skill\-level operational birthmark: it abstracts a skill into the actions it performs, the conditions under which it is activated, and the tools, resources, and artifacts that flow through its execution\. It is inspired by control\-flow and data/dependence graphs\[[1](https://arxiv.org/html/2608.05204#bib.bib69),[7](https://arxiv.org/html/2608.05204#bib.bib70)\], but it is defined over agent\-skill behavior rather than program statements, variables, or code basic blocks\.
Given a skill packageSS, the SOG is a typed graph
GO\(S\)=\(BS,AS,PS,RS\)\.G\_\{O\}\(S\)=\(B\_\{S\},A\_\{S\},P\_\{S\},R\_\{S\}\)\.\(1\)
whereBSB\_\{S\}is the set of normalized operational blocks,ASA\_\{S\}is the activation signature that describes how the skill is entered by an agent host,PSP\_\{S\}is the procedure structure relating blocks through ordering, dependency, branch, and fallback relations, andRSR\_\{S\}is the resource\-flow structure relating blocks to tools, resources, and artifacts they consume or produce\. These four parts answer four audit questions: what operations exist, how the skill is entered, how the operations are organized, and what tools or artifacts flow through them\.
Operational Block\.An operational block is a normalized skill\-level action unit extracted from instructions, code, tool specifications, or reference files\. It describes one coherent operation in the skill, such as validating an input, retrieving an external resource, transforming data, invoking a tool, or producing an output\. A basic block is a unit of program execution; an operational block is a unit of skill operation\.
Each operational block is represented by five normalized fields as:
\(action,object\_role,tool\_role,inputs,outputs\)\.\(\\textsf\{action\},\\textsf\{object\\\_role\},\\textsf\{tool\\\_role\},\\textsf\{inputs\},\\textsf\{outputs\}\)\.These fields capture the minimal reusable semantics of a skill operation: what action is performed, what role of object it acts on, what kind of tool is used, and which artifacts flow into and out of the operation\. Concrete names such as file paths, function names, library identifiers, and example values are normalized away when possible, because they are easy to rewrite; the operational roles are harder to remove without changing the skill’s behavior\.
The SOG schema uses closed vocabularies for actions, object roles, tool roles, activation contexts, and relation types\. We derive these vocabularies from corpus\-level inspection and freeze them before extraction, merging rare lexical variants into coarser roles\. This keeps the Operational Trace abstract enough to survive paraphrase and host relocation, but concrete enough to avoid collapsing all same\-function skills into a single task intent\.
SOG views\.The three SOG views expose different relations over the same operational blocks\. Activation describes how the graph is entered, Procedure describes how blocks are ordered and conditioned, and Resource\-Flow describes how tools, resources, and artifacts connect to the blocks\. Table[II](https://arxiv.org/html/2608.05204#S3.T2)summarizes these views, and Fig\.[2](https://arxiv.org/html/2608.05204#S3.F2)illustrates them using a PDF form\-filling skill as an example\.
TABLE II:Internal views of the Skill Operational Graph\.ViewCapturesExample relationActivationEntry context and start blockcomplete\_form→\\rightarrowb1\.inspectProcedureOrder, branch, fallbackb1\.inspect→\\rightarrowb2\.extractResource\-FlowTool and artifact dependenciespdf\_reader\+blank\_form→\\rightarrowb2\.extract
*1\) Activation View:*captures when and why the skill becomes active: the user intent, required context, host\-facing trigger conditions, and the first operational blocks needed for that context\. It is analogous to a route\-to\-handler mapping but for agent skills\.
*2\) Procedure View:*it captures how operational blocks are sequenced: ordering, branches, fallback paths, failure handling, and tool\-call order\. It is similar in spirit to a control\-flow graph, but its nodes are skill\-level operations rather than code basic blocks\.
*3\) Resource\-Flow View:*it captures which tools, resources, and artifacts flow through the procedure: support files, external services, APIs, intermediate outputs, generated reports, and validation artifacts\. It is related to data\-flow and dependence graphs, but it operates at the level of agent resources and skill artifacts rather than program variables\.
These views are not separate detectors\. They are projections of one Operational Trace\. Together, they explain how a skill can remain operationally similar after a derivative rewrites wording, changes file layout, or renames code identifiers\.
Figure 2:Example Skill Operational Graph \(SOG\) for a PDF form\-filling skill \(simplified\)\.SOG Extraction\. Operational traces are extracted once at ingestion time using an LLM constrained to emit the SOG schema above\. The extractor uses a frozen schema and is instructed not to copy distinctive phrases from the source package\. The extracted graph is normalized, cached, and tagged with the prompt and schema versions\. Pairwise reuse detection does not call an LLM: it compares cached SOG traces deterministically\.
### III\-EReuse Detection
Once the traces have been extracted and cached, reuse detection becomes a pairwise trace\-comparison problem\. Given a reference skillRRand a candidate skillCC,SkillTracecomputes one similarity score for each provenance trace:
sE\(R,C\),sI\(R,C\),sO\(R,C\)\.s\_\{E\}\(R,C\),\\quad s\_\{I\}\(R,C\),\\quad s\_\{O\}\(R,C\)\.for Expression, Implementation, and Operational Trace respectively\. The scores are kept separate until calibration so that the final report can identify which trace supports the reuse finding\.
#### III\-E1Expression Similarity
Expression similarity compares authored textual features\. LetTE\(S\)T\_\{E\}\(S\)be the normalized token set extracted from the natural\-language substrates of skillSS\.SkillTraceuses token Jaccard:
sE\(R,C\)=\|TE\(R\)∩TE\(C\)\|\|TE\(R\)∪TE\(C\)\|\.s\_\{E\}\(R,C\)=\\frac\{\|T\_\{E\}\(R\)\\cap T\_\{E\}\(C\)\|\}\{\|T\_\{E\}\(R\)\\cup T\_\{E\}\(C\)\|\}\.\(2\)Matched tokens are retained as evidence pointers for review\.
#### III\-E2Implementation Similarity
Implementation similarity compares executable and command\-like features\. Following token\-based code clone detection such as CCFinder and SourcererCC,SkillTraceextracts normalized tokens including inline code, support scripts and shell commands\. LetTI\(S\)T\_\{I\}\(S\)be this implementation\-token set\. When the trace is applicable on both sides,SkillTracecomputes:
sI\(R,C\)=\|TI\(R\)∩TI\(C\)\|\|TI\(R\)∪TI\(C\)\|\.s\_\{I\}\(R,C\)=\\frac\{\|T\_\{I\}\(R\)\\cap T\_\{I\}\(C\)\|\}\{\|T\_\{I\}\(R\)\\cup T\_\{I\}\(C\)\|\}\.\(3\)We use token\-level similarity because skills often mix Python, JavaScript, shell snippets, JSON schemas, API names, and natural\-language command recipes\. A single AST representation would not cover this heterogeneous setting\. If either side lacks sufficient implementation substrate, the Implementation Trace is marked as not applicable rather than assigned a zero score\.
#### III\-E3Operational Similarity
Operational similarity compares the Skill Operational Graphs of two skills\. It asks four questions: do the skills contain similar operational units; are those units entered through similar activation conditions; are they organized by similar procedure structure; and do they connect to similar tools, resources, and artifacts? As shown in \([1](https://arxiv.org/html/2608.05204#S3.E1)\), these four questions correspond to the four parts of the SOG:
GO\(R\)\\displaystyle G\_\{O\}\(R\)=\(BR,AR,PR,RR\),\\displaystyle=\(B\_\{R\},A\_\{R\},P\_\{R\},R\_\{R\}\),GO\(C\)\\displaystyle G\_\{O\}\(C\)=\(BC,AC,PC,RC\)\.\\displaystyle=\(B\_\{C\},A\_\{C\},P\_\{C\},R\_\{C\}\)\.SkillTracecomputes a deterministic similarity for each part and combines them into one Operational score:
sO\(R,C\)=\\displaystyle s\_\{O\}\(R,C\)=λBBlockSim\(BR,BC\)\\displaystyle\\lambda\_\{B\}\\,\\mathrm\{BlockSim\}\(B\_\{R\},B\_\{C\}\)\(4\)\+λAActSim\(AR,AC\)\\displaystyle\+\\lambda\_\{A\}\\,\\mathrm\{ActSim\}\(A\_\{R\},A\_\{C\}\)\+λPProcSim\(PR,PC\)\\displaystyle\+\\lambda\_\{P\}\\,\\mathrm\{ProcSim\}\(P\_\{R\},P\_\{C\}\)\+λRResSim\(RR,RC\)\.\\displaystyle\+\\lambda\_\{R\}\\,\\mathrm\{ResSim\}\(R\_\{R\},R\_\{C\}\)\.We use fixed weights
\(λB,λA,λP,λR\)=\(0\.30,0\.10,0\.40,0\.20\),\(\\lambda\_\{B\},\\lambda\_\{A\},\\lambda\_\{P\},\\lambda\_\{R\}\)=\(0\.30,0\.10,0\.40,0\.20\),where the largest weight is assigned to procedure structure\. These four terms are internal to Operational Trace; they are not separate provenance traces\.
*BlockSim*captures whether two skills contain similar operational units\. Because a derivative may split, merge, rename, or relocate operations, blocks are not compared position\-by\-position\. Instead,SkillTraceuses maximum one\-to\-one matching between normalized operational blocks, with a weak longest\-common\-subsequence term over the normalized block sequence to capture coarse order preservation\.*ActSim*compares compact activation signatures by normalized role overlap; it is useful as host\-facing corroboration, but is down\-weighted because same\-function independent skills may share generic trigger conditions\.*ProcSim*combines typed procedure\-edge Jaccard with sequence alignment over normalized block order, capturing dependencies, branches, fallback paths, and execution order\.*ResSim*compares directed typed block\-resource dependency edges after node\-label normalization; resource flow is graph\-structured but not globally ordered, because tools, support files, intermediate artifacts, and outputs may branch, merge, or be reused by multiple blocks\.
This is a decomposed operational\-graph match rather than full graph edit distance\. It keeps the graph information most relevant to skill reuse while remaining deterministic, scalable, and easy to expose in an evidence report\.
#### III\-E4Calibrated Detection
Raw trace scores are not directly comparable: Expression, Implementation, and Operational Trace have different feature spaces and different background collision rates\.SkillTracetherefore calibrates one threshold per trace from same\-function strict negatives:
τE,τI,τO\.\\tau\_\{E\},\\quad\\tau\_\{I\},\\quad\\tau\_\{O\}\.In our experiments, eachτ\\tauis the 95th percentile of the corresponding trace score on the strict\-negative set, giving a per\-trace 5% false\-positive calibration without using positive examples\. In deployment, the same mechanism can be tightened or relaxed to match a marketplace’s review budget\. Automatic threshold selection is left as future work\.
Let𝒜\(R,C\)\\mathcal\{A\}\(R,C\)be the set of applicable traces for the pair \(e\.g\., Implementation is excluded when either side lacks enough code/command substrate\)\. For each applicable tracei∈\{E,I,O\}i\\in\\\{E,I,O\\\},SkillTracecomputes:
SkillTrace\(R,C\)=maxi∈𝒜\(R,C\)si\(R,C\)τi\.\\mathrm\{SkillTrace\}\(R,C\)=\\max\_\{i\\in\\mathcal\{A\}\(R,C\)\}\\frac\{s\_\{i\}\(R,C\)\}\{\\tau\_\{i\}\}\.\(5\)The pair is surfaced for review when this calibrated score meets or exceeds a decision boundβ\\beta\. We useβ=1\.0\\beta=1\.0in all experiments, so one trace meeting its N1\-calibrated threshold is sufficient to enter the review queue\. In deployment, operators can raiseβ\\betafor a smaller, higher\-confidence queue or lower it for exploratory triage\. The maximum rule matches the forensic setting: strong evidence in any one trace can justify review even when other traces have been rewritten\. A weighted average would dilute such localized evidence and introduce tunable cross\-trace weights; calibrated max keeps the decision rule transparent\.
### III\-FInterpretability
SkillTraceis designed to support human review\. For every surfaced pair, the report includes the firing trace, raw and calibrated scores, and trace\-specific evidence pointers: matched textual features for Expression, matched code/API/command features for Implementation, and matched SOG evidence for Operational Trace, including aligned blocks, activation entries, procedure edges, and resource\-flow edges\. This makesSkillTracedifferent from a monolithic similarity classifier\. It does not merely say that two skills are similar; it explains where the provenance evidence appears and which trace survived the reuse transformation\.
## IVEvaluation
### IV\-AResearch Questions
We evaluateSkillTracealong four research questions\.
- •RQ1: Detection effectiveness\.CanSkillTracedetect skill reuse under realistic rewrites while maintaining low false positives on same\-function independent skills?
- •RQ2: Trace complementarity\.Do Expression, Implementation, and Operational Trace capture different reuse patterns, and does the SOG\-based Operational Trace add evidence beyond text/code similarity?
- •RQ3: Ecosystem discovery\.CanSkillTracesurface actionable reuse candidates in a real public skill corpus, including cases that repository\-level baselines miss?
- •RQ4: Cost and scalability\.CanSkillTraceamortize Operational extraction at ingestion time and run repeated audits deterministically at marketplace scale?
### IV\-BBenchmark Construction
We constructSkillTrace\-Benchfrom 100 public skill anchors selected from high\-activity skill marketplaces and registries\. The anchors cover diverse task categories, package sizes, and implementation styles, while limiting duplicates from the same owner and provenance cluster\.
##### Positive reuse pairs
SkillTrace\-Benchcontains 820 positive reuse pairs from these anchors\. The positive set combines observable ecosystem reuse with controlled transformations\. When public provenance is available, we include real forks, versioned variants, and cross\-registry repackages verified by repository metadata, file paths, names, and package structure\. Because such provenance is sparse and uneven, we also generate controlled derivatives that model realistic reuse motives\. The resulting families are grouped into the rows reported in Table[III](https://arxiv.org/html/2608.05204#S4.T3)\.
- •R: Repackagingmodels low\-effort republication where the derivative keeps most skill content but changes marketplace\-facing metadata, section layout, or file organization\. It contains R1 metadata rewrite and R2 structural reorganization\.
- •P: Porting and rewritingmodels reuse that preserves the same skill capability while changing deployment context, host format, or surface realization\. It contains P1 cross\-modal port, P2 registry fork, and P3 full LLM rewrite\.
- •L: Partial lifting and replacementmodels higher\-effort reuse where one valuable part of a skill is preserved while another is rewritten or replaced\. It contains L1 implementation lift, L2 code refactor, and L3 documentation reuse\. These three regimes span near\-literal reuse, operational migration, and selective reuse of documentation or implementation\.
TABLE III:Per\-source coverage onSkillTrace\-Bench\. Cells show mean raw similarity with detection rate in parentheses; for each positive row, the best detection rate is bolded\. Source columns useτλ\\tau\_\{\\lambda\}, the 95th percentile on N1 strict negatives\. MaxFusion reports the calibrated ratiomaxisi/τi\\max\_\{i\}s\_\{i\}/\\tau\_\{i\}and fires at decision boundβ=1\.0\\beta=1\.0\. The Positive avg\. row is a micro\-average over all 820 positive pairs\. Implementation rates use only applicable pairs where both sides contain sufficient code\.BaselineSkillTraceSubjectRepo\-JacRepo\-ssdeepExpressionImplementationOperationalMaxFusionPositiveR1 metadata rewrite0\.988 \(1\.00\)0\.934 \(1\.00\)0\.982 \(1\.00\)0\.999 \(1\.00\)0\.950 \(1\.00\)7\.174 \(1\.00\)R2 structural reorg\.0\.947 \(1\.00\)0\.427 \(0\.77\)0\.949 \(1\.00\)0\.871 \(0\.99\)0\.924 \(1\.00\)6\.578 \(1\.00\)P1 cross\-modal port0\.203 \(0\.45\)0\.002 \(0\.00\)0\.234 \(0\.62\)0\.100 \(0\.23\)0\.421 \(0\.43\)1\.572 \(0\.72\)P2 registry fork0\.215 \(0\.48\)0\.000 \(0\.00\)0\.257 \(0\.69\)0\.084 \(0\.16\)0\.498 \(0\.63\)1\.797 \(0\.86\)P3 full LLM rewrite0\.156 \(0\.32\)0\.009 \(0\.02\)0\.149 \(0\.31\)0\.169 \(0\.24\)0\.681 \(0\.73\)2\.724 \(0\.78\)L1 implementation lift0\.480 \(0\.92\)0\.263 \(0\.47\)0\.160 \(0\.42\)0\.835 \(1\.00\)0\.936 \(1\.00\)6\.016 \(1\.00\)L2 code refactor0\.886 \(1\.00\)0\.744 \(0\.97\)0\.999 \(1\.00\)0\.806 \(1\.00\)0\.982 \(1\.00\)6\.176 \(1\.00\)L3 documentation reuse0\.389 \(0\.74\)0\.088 \(0\.16\)0\.468 \(0\.88\)0\.196 \(0\.33\)0\.676 \(0\.82\)3\.125 \(0\.98\)Positive avg\.0\.510 \(0\.69\)0\.278 \(0\.37\)0\.517 \(0\.75\)0\.559 \(0\.65\)0\.694 \(0\.76\)3\.997 \(0\.88\)Neg\.N1 strict negative0\.106 \(0\.05\)0\.000 \(0\.00\)0\.092 \(0\.05\)0\.062 \(0\.05\)0\.273 \(0\.05\)0\.786 \(0\.10\)N2 clean\-room synthesis0\.096 \(0\.07\)0\.000 \(0\.00\)0\.103 \(0\.07\)0\.046 \(0\.00\)0\.333 \(0\.10\)0\.856 \(0\.12\)τλ\\tau\_\{\\lambda\}/β\\beta0\.1820\.0000\.1720\.1390\.4061\.000
##### Negative controls
The negative side contains 751 controls and separates reuse from same\-function similarity\. N1 contains same\-function strict negatives mined from the real corpus: pairs that solve similar tasks but are independently implemented\. We exclude mirrors, same\-owner reuse, same\-provenance clusters, and shared\-template\-only cases; admission requires cross\-model agreement that the pair is same\-function but independent\. N1 is the main false\-positive regime for provenance auditing\. N2 contains clean\-room skills generated from task metadata by LLM without access to the original skill body\. N2 is not used for threshold calibration; it is a boundary control for cases where no body\-level trace should be preserved\.
##### Validation
All benchmark pairs are checked before inclusion\. Observable ecosystem reuse is verified from repository metadata and package structure\. For LLM\-generated positives, a cross\-model judge whose generator and judge come from different model families whenever possible is used as an initial filtering gate\. After this filtering step, we manually check the admitted positive and negative pairs to ensure that the benchmark labels are reliable\. The manual check agrees with the judge on the include/exclude decision for each pair\.
### IV\-CExperimental Setup
#### IV\-C1Baselines
We compare against repository\-level baselines\[[15](https://arxiv.org/html/2608.05204#bib.bib51)\], RepoClone\-Jaccard and RepoClone\-ssdeep, following the measurement style of prior tool\-cloning studies\. RepoClone treats each skill as a whole package: it flattens repository contents and computes either token Jaccard similarity or fuzzy\-hash similarity with ssdeep\[[16](https://arxiv.org/html/2608.05204#bib.bib67)\]\. It is therefore a package\-level similarity baseline rather than a trace\-attributed reuse detector\. We report the threeSkillTracetrace scores separately: Expression, Implementation, and Operational\. The headline method,SkillTrace\-MaxFusion, surfaces a pair when any calibrated trace meets the decision bound\.
#### IV\-C2Evaluation metrics
For controlled benchmark evaluation, we report AUROC, precision, recall, F1, per\-family mean scores, and detection rates at thresholds calibrated only on N1 strict negatives\. For the wild\-corpus study, which lacks complete ground truth, we do not report AUROC; instead we report ecosystem fingdings, disagreement between RepoClone andSkillTrace, and manual validation on sampled candidates\.
### IV\-DRQ1: Detection Effectiveness
Table[III](https://arxiv.org/html/2608.05204#S4.T3)reports per\-trace raw similarity and the thresholdedSkillTrace\-MaxFusion decision score atβ=1\.0\\beta=1\.0, broken down by attack family\. The thresholdsτλ\\tau\_\{\\lambda\}are calibrated only on the N1 strict same\-function negatives\. Over the combined negative pool \(N1\+N2\),SkillTrace\-MaxFusion reaches AUROC 0\.938, precision 0\.914, recall 0\.883, and F1 0\.898, outperforming RepoClone\-Max \(AUROC 0\.841, F1 0\.783\) and the strongest single trace, Operational \(AUROC 0\.878, F1 0\.836\)\. Each trace column reports raw similarity: Jaccard for Expression and Implementation, and aligned SOG\-view coverage for Operational\. RepoClone\-Max takes the larger calibrated score of RepoClone\-Jaccard and RepoClone\-ssdeep\.
Repo\-JacRepo\-ssdeepExpressionImplementationOperational0\.250\.51\.0R1 \(repackage\)P3 \(rewrite\)L1 \(lift\)N1 \(Negative\)Figure 3:Coverage radar for four representative subjects over the five raw audit metrics in Table[III](https://arxiv.org/html/2608.05204#S4.T3)\.Observations\.Three patterns follow from Table[III](https://arxiv.org/html/2608.05204#S4.T3)\. First, R\-class repackaging is saturated: near\-literal metadata and structural repackages remain easy for both RepoClone andSkillTrace, which serves as a sanity check for the pipeline\. Second, P\-class porting/rewriting is where repository\-level similarity becomes unreliable\. In P3 full LLM rewrite, RepoClone\-Jaccard drops to 0\.156 with only 0\.32 detection rate, while Operational remains the strongest single trace at 0\.681 mean similarity and 0\.73 detection rate; MaxFusion raises the P3 detection rate to 0\.78\. This reflects the key skill\-specific signal: even when wording and code are rewritten, a functional rewrite tends to preserve the operational procedure needed for the skill to work\. Third, L\-class partial lifting/replacement transformations show that the firing trace rotates with the reuser’s preservation choice\. In L1 implementation lift, Expression collapses to 0\.160, but Implementation rises to 0\.835 and Operational to 0\.936; in L3 documentation reuse, Expression becomes the dominant surface trace\. Across L1–L3, MaxFusion fires on at least 0\.96 of positives\.
Figure[3](https://arxiv.org/html/2608.05204#S4.F3)visualizes the same pattern\. R1 fills the outer ring because all traces are preserved\. P3 collapses on repository and lexical axes but retains the Operational trace\. L1 suppresses Expression while preserving Implementation and Operational trace\. N1 strict negatives remain near the center, showing that the calibrated thresholds are not simply detecting same\-function similarity\.
Answer to RQ1:SkillTraceimproves over repository\-level baselines and single\-trace variants on realistic skill reuse, reaching AUROC 0\.938 and F1 0\.898\.
### IV\-ERQ2: Trace Complementarity
A natural concern is whether the three traces inSkillTraceare merely different names for the same surface similarity\. We do not claim that the traces are statistically independent features: near\-literal repackages should make all traces rise together\. Instead, RQ2 asks whether the traces provide complementary forensic paths when a reuse transformation rewrites one part of a skill while preserving another\. We examine this at three levels: score correlation, thresholded firing patterns, and fusion behavior\.
Trace\-level complementarity\.Table[IV](https://arxiv.org/html/2608.05204#S4.T4)reports Pearson correlations between Expression, Implementation, and Operational similarities\. The useful signal is not the all\-positive average, which mixes easy repackages with harder rewrites, but the family\-wise split\. In partial\-lift reuses, Expression–Implementation correlation falls tor=0\.15r\{=\}0\.15: some derivatives preserve executable material while rewriting documentation, while others preserve documentation and replace code\. In porting and rewriting reuses, Expression–Operational and Implementation–Operational correlations drop tor=0\.28r\{=\}0\.28andr=0\.36r\{=\}0\.36\. This is the regime where Operational Trace matters most: words and code can be rewritten, but a useful derivative often preserves the procedure, tool roles, and execution structure needed for the skill to work\.
TABLE IV:Pearson correlation between trace similarities by reuse family\.Reuse FamilyE–IE–OI–OAll positives0\.760\.620\.67R repackage0\.410\.130\.43P port / rewrite0\.430\.280\.36L partial lift0\.150\.200\.72Decision\-level complementarity\.Thresholded decisions show the same effect\. Under the defaultβ=1\.0\\beta=1\.0bound, a non\-trivial set of positives fires through exactly one trace: about one hundred Expression\-only cases, about seventy Operational\-only cases, and only a few Implementation\-only cases\. The Operational\-only cases are especially important: they are mostly cross\-modal ports and full LLM rewrites where repository overlap, authored wording, and code tokens fall below threshold, but the operational procedure remains detectable\. Implementation\-only fires are rare because reused implementation usually also carries the procedure that invokes it\. The point is therefore complementarity rather than orthogonality: each trace provides a different path by which preserved provenance can surface\.
TABLE V:Global detection effectiveness against the negative pool\. Scores are trace\-normalized by N1\-calibrated thresholds and evaluated at decision boundβ=1\.0\\beta=1\.0; linear fusion averages applicable normalized traces\. Implementation\-only is evaluated on its applicable subset\.VariantAUROCPrecisionRecallF1RepoClone\-Max \(baseline\)0\.8410\.9240\.6790\.783Expression only0\.8660\.9390\.7500\.834Implementation only0\.7440\.9350\.6470\.765Operational only0\.8780\.9340\.7560\.836Linear fusion \(equal E/I/O\)0\.9090\.9730\.7440\.843SkillTrace\-MaxFusion0\.9380\.9140\.8830\.898
Decision\-rule ablation\.Table[V](https://arxiv.org/html/2608.05204#S4.T5)compares single\-trace variants, linear fusion, andSkillTrace\-MaxFusion against the combined N1\+N2 negative pool\. Linear fusion over normalized E/I/O traces is conservative: it reaches the highest precision \(0\.973\) but lower recall \(0\.744\), because a strong trace is averaged against weaker or rewritten traces\. MaxFusion instead surfaces a pair when any calibrated trace fires, giving the best AUROC, recall, and F1 while keeping precision high \(0\.914\)\. This supports the forensic design choice behind MaxFusion: localized provenance evidence should be sufficient to route a pair for review, while the report still identifies which trace fired\.
Answer to RQ2:The three traces are complementary forensic paths rather than redundant aliases\. Expression, Implementation, and Operational traces split under different reuse scenarios, and MaxFusion preserves localized evidence that linear averaging would dilute\.
### IV\-FRQ3: Ecosystem Discovery
We audit a public corpus of 36,446 real\-world skills collected from SkillHub and ClawHub as a deployment study\. The corpus has no complete provenance ground truth, so RQ3 is not a supervised benchmark: we do not report AUROC on wild pairs and we do not make legal infringement claims\. Instead, we ask whetherSkillTracecan build actionable, trace\-attributed reuse review queues at marketplace scale\. We run two complementary audits: an anchor\-driven scan over popular skills and a global audit over the full corpus\.
Top\-50 hot\-skill exposure\.We first audit from the perspective of popular skill authors by taking the top 50 skills from SkillHub as anchors and searching the full corpus for conservative reuse candidates\. The result shows that reuse around popular skills is widespread but uneven: 49 of the 50 hot anchors have at least one jointly flagged reuse candidate where both RepoClone andSkillTraceevidence fire, and the scan surfaces 357 such candidates in total\. Reuse is concentrated rather than uniformly distributed: 7 anchors have more than ten reuse candidates, withself\-improvementalone surfacing 41\. Software\-engineering skills account for the largest volume, with 211 reuse candidates across 30 anchors\.
The anchor scan also exposes the regime that motivates trace attribution\. StaticSkillTracesurfaces 18SkillTrace\-only candidates across 8 anchors: repository\-level similarity stays below the conservative threshold, butSkillTraceevidence is strong enough to justify human review\. Thus, broad full\-skill reuse is the dominant hot\-anchor pattern, while source\-specific partial reuse is rarer but visible and would be easy to under\-prioritize with package\-level similarity alone\.
Global reuse audit\.We next scale from hot anchors to the full corpus\. A naive all\-pair audit would require more than6\.6×1086\.6\\times 10^\{8\}comparisons, soSkillTraceuses source\-indexed candidate generation: it builds indexes over repository tokens, Expression tokens, and Implementation tokens, enumerates pairs that share enough static evidence to plausibly pass a conservative review threshold, unions candidates across sources, and then exact\-scores the resulting root\-pair set\. RepoClone serves as the package\-level baseline, whileSkillTracerecords which static trace fired\. We deliberately keep Operational Trace out of the global routing rule: it is useful as corroborating evidence, but generic same\-function skills can collide on common operational schemas such as API wrappers or document\-conversion pipelines\.
The global pass exact\-scores204,386204\{,\}386deduplicated candidate pairs\. On this pool, RepoClone and staticSkillTraceevidence jointly flag43,58143\{,\}581pairs \(21\.3%21\.3\\%\), the broad\-reuse region where package\-level and trace\-level evidence agree\.SkillTraceadditionally surfaces3,0303\{,\}030static\-only candidates \(1\.5%1\.5\\%\):1,5291\{,\}529Expression\-only,1,3381\{,\}338Implementation\-only, and163163where both static traces fire while RepoClone remains below threshold\. These pairs are the main ecosystem contribution ofSkillTrace: they preserve inspectable source\-level evidence without enough whole\-package overlap to dominate a repository\-level queue\. The reverse region is much smaller:439439pairs \(0\.2%0\.2\\%\) are RepoClone\-only, a mixed bucket of scaffolds, templates, and borderline reuse\. Overall, near\-full reuse is more common in the wild, but partial source\-level reuse appears at nontrivial scale\.
Manual validation\.The automatic buckets are triage queues, not provenance verdicts: they decide what a marketplace auditor should inspect, not whether infringement occurred\. We therefore blindly reviewed 200 stratified wild pairs, including 100 pairs surfaced bySkillTraceand 100 low\-priority controls\. TheSkillTrace\-flagged pairs consistently contain inspectable reuse evidence aligned with the firing trace: reused documentation, examples, support scripts, or implementation fragments\. The low\-priority controls are overwhelmingly independent or unclear and lack comparable provenance evidence\. This manual check supports the central wild\-corpus claim:SkillTrace’s source\-attributed queues correspond to evidence a human reviewer can inspect, while the tool itself stops short of legal adjudication\.
Answer to RQ3:In a large public corpus,SkillTracebuilds actionable reuse review queues beyond package\-level similarity: hot\-skill reuse is widespread, and global auditing surfaces a substantial set of trace\-attributed candidates that repository\-level baselines would under\-prioritize\.
### IV\-GRQ4: Cost and Scalability
SkillTraceseparates per\-skill registration from repeated pairwise audits\. At ingestion time, Expression and Implementation traces are extracted deterministically, while the Operational trace is extracted once with an LLM and cached with its prompt version\. In our configuration, cold Operational extraction costs about 26 seconds and $0\.01 per skill \(roughly 10K input and 1K output tokens\)\. Registering the 36,446\-skill wild corpus therefore costs about $360 and 263 single\-worker hours, or 5\.3 hours with 50 parallel workers\.
After registration, audit\-time scoring uses only cached traces and makes no LLM calls\. On a cached pairwise timing sample, fullSkillTrace\-MaxFusion scoring takes 105 ms per pair on average\. Even using this full\-score timing as a conservative bound, the 204,386 global candidate pairs in RQ3 require about 6\.0 single\-worker hours, or roughly 22 minutes with 16 workers, and zero token cost\. ThusSkillTrace’s cost is paid once when a skill enters the registry; repeated reuse audits are deterministic and inexpensive\.
Answer to RQ4:SkillTraceamortizes LLM use at ingestion time\. After traces are cached, marketplace\-scale audits run deterministically with zero token cost, and minutes\-scale latency under modest parallelism\.
### IV\-HThreats to validity\.
The external threats to validity mainly come from benchmark construction and corpus coverage\. Some positive pairs are generated by LLM\-assisted transformations, which may introduce model\-specific style\. We reduce this risk through cross\-model generation and judging, per\-trace reporting, and manual spot checks, but the benchmark cannot cover every future marketplace reuse pattern\. The wild audit also covers public registries rather than all commercial or private skill ecosystems; we therefore report it as a deployment study and review\-queue analysis, not as a prevalence estimate for all skill markets\.
Internal threats concern trace extraction and scoring correctness\.SkillTraceuses LLM assistance only for Operational trace extraction at ingestion time; audit\-time scoring is deterministic over cached traces\. The extractor runs under an auditor\-controlled, temperature\-zero, versioned prompt, but extracted SOGs can still be noisy or too coarse\. This matters especially for generic same\-function skills\. For this reason, Operational evidence is calibrated against strict negatives and used cautiously in wild\-corpus routing\. Similarly, Implementation Trace is applicable only when both skills contain enough code or command material; code\-light skills should not be treated as implementation mismatches\.
Construct threats concern what a trace match means\.SkillTracemeasures provenance evidence, not legal infringement, intent, license compatibility, or ownership\. Shared templates, common API wrappers, and multi\-origin merges can produce genuine similarity without a clean one\-to\-one reuse relation\. We therefore surface trace\-attributed review candidates rather than automatic verdicts\. Reuse that deliberately targets the detector is likewise outside our current benchmark; as skill ecosystems evolve, benchmark families, strict negatives, and human\-review protocols will need to evolve with them\.
## VDiscussion
### V\-ARegistry deployment and governance
SkillTraceis designed for marketplace\-scale deployment: skills pay a one\-time registration cost for trace extraction, while repeated audits operate over cached substrates with no audit\-time LLM call \(§[IV\-G](https://arxiv.org/html/2608.05204#S4.SS7)\)\. This separation lets registries reuse the same traces for dispute review, duplicate management, and quality ranking\. A registry can use source\-attributed reuse evidence to cluster related skills, surface a more complete or maintained variant, link alternatives, and warn users about smaller or outdated derivatives\. The trace labels make this more actionable than a monolithic score: Expression points to reused prose or examples, Implementation to lifted scripts or command/API patterns, and Operational to preserved agent\-facing behavior\.
### V\-BProtectability and legal boundary
We use protectability only as a review\-prioritization concept\. Skills with substantial authored instructions, reusable implementation, distinctive tool/resource organization, or nontrivial operational design are more worth auditing because these properties reflect author effort and marketplace value\.SkillTracedoes not decide legal protectability; it identifies where provenance evidence survives and what deserves human review\. More broadly,SkillTraceproduces provenance evidence, not legal verdicts\. It does not infer intent, ownership, license compatibility, or infringement; those depend on jurisdiction, license terms, fair use, and marketplace policy\. Accordingly, wild\-corpus reports use anonymized cases and describe surfaced pairs as review candidates\. The tool’s role is forensic triage: which traces survived, where to inspect, and whether follow\-up is warranted\.
### V\-COperational trace and skill value
Operational Trace is the most skill\-specific trace inSkillTrace: it acts as a skill\-level operational birthmark\. Yet Operational similarity alone is not a reuse verdict\. Same\-function skills may collide on generic operational schemas because functional requirements can impose similar high\-level procedures\. This is an information\-content issue rather than a failure of the trace: for complex skills, a richer SOG carries more distinctive design decisions and can serve as a stronger birthmark; generic, low\-information SOG patterns should not be treated as reuse evidence by themselves\. We therefore treat generic schema matches as an ambiguous review zone rather than a positive finding\. Operational evidence is strongest when corroborated by Expression, Implementation, distinctive resources, or propagation patterns\. Beyond provenance auditing, the SOG may also provide a future signal for skill\-value assessment, since richer operational structure can indicate more substantial agent\-facing design\.
## VIConclusion
In this work, we proposeSkillTrace, a multi\-trace provenance auditing framework for agent skills\. To capture provenance signals under realistic skill reuse,SkillTraceadopts a two\-stage design\. The trace extraction stage extracts Expression, Implementation, and Operational traces, with the Operational Trace instantiated as a Skill Operational Graph\. The reuse detection stage compares these traces separately with calibrated thresholds, allowingSkillTraceto surface not only whether a candidate is suspicious but also which trace supports the decision\. Experiments onSkillTrace\-Benchand a large public skill corpus demonstrate the effectiveness, interpretability, and deployment practicality ofSkillTracein reuse auditing\. Compared with baselines,SkillTraceachieves stronger detection performance while surfacing trace\-attributed review evidence that global package similarity misses\. A promising direction for future work is to reuse these trace representations for adjacent marketplace tasks, such as skill quality assessment and trace\-aware skill search\.
## References
- \[1\]\(1970\)Control flow analysis\.ACM Sigplan Notices5\(7\),pp\. 1–19\.Cited by:[§III\-D](https://arxiv.org/html/2608.05204#S3.SS4.p2.1)\.
- \[2\]Anthropic\(2024\)Introducing the model context protocol\.Note:[https://www\.anthropic\.com/news/model\-context\-protocol](https://www.anthropic.com/news/model-context-protocol)Accessed Jun\. 11, 2026Cited by:[§II\-B](https://arxiv.org/html/2608.05204#S2.SS2.p1.1)\.
- \[3\]Anthropic\(2025\)Equipping agents for the real world with agent skills\.Note:[https://www\.anthropic\.com/engineering/equipping\-agents\-for\-the\-real\-world\-with\-agent\-skills](https://www.anthropic.com/engineering/equipping-agents-for-the-real-world-with-agent-skills)Accessed Jun\. 11, 2026Cited by:[§I](https://arxiv.org/html/2608.05204#S1.p1.1),[§II\-B](https://arxiv.org/html/2608.05204#S2.SS2.p1.1)\.
- \[4\]Anthropic\(2026\)Agent skills\.Note:[https://platform\.claude\.com/docs/en/agents\-and\-tools/agent\-skills/overview](https://platform.claude.com/docs/en/agents-and-tools/agent-skills/overview)Accessed Jun\. 11, 2026Cited by:[§I](https://arxiv.org/html/2608.05204#S1.p1.1),[§II\-B](https://arxiv.org/html/2608.05204#S2.SS2.p1.1)\.
- \[5\]Anthropic\(2026\)Extend claude with skills\.Note:[https://code\.claude\.com/docs/en/skills](https://code.claude.com/docs/en/skills)Claude Code documentationCited by:[§I](https://arxiv.org/html/2608.05204#S1.p1.1)\.
- \[6\]A\. Z\. Broder\(1997\)On the resemblance and containment of documents\.InProc\. Compression and Complexity of SEQUENCES 1997 \(Cat\. No\. 97TB100171\),pp\. 21–29\.Cited by:[§II\-A](https://arxiv.org/html/2608.05204#S2.SS1.p1.1)\.
- \[7\]J\. Ferrante, K\. J\. Ottenstein, and J\. D\. Warren\(1987\)The program dependence graph and its use in optimization\.ACM Transactions on Programming Languages and Systems \(TOPLAS\)9\(3\),pp\. 319–349\.Cited by:[§III\-D](https://arxiv.org/html/2608.05204#S3.SS4.p2.1)\.
- \[8\]K\. Greshake, S\. Abdelnabi, S\. Mishra, C\. Endres, T\. Holz, and M\. Fritz\(2023\)Not what you’ve signed up for: compromising real\-world LLM\-integrated applications with indirect prompt injection\.InProc\. ACM Workshop Artif\. Intell\. Secur\. \(AISec\),Cited by:[§II\-C](https://arxiv.org/html/2608.05204#S2.SS3.p1.1)\.
- \[9\]D\. Guo, S\. Ren, S\. Lu, Z\. Feng, D\. Tang, S\. Liu, L\. Zhou, N\. Duan, A\. Svyatkovskiy, S\. Fu, M\. Tufano, S\. K\. Deng, C\. Clement, D\. Drain, N\. Sundaresan, J\. Yin, D\. Jiang, and M\. Zhou\(2021\)GraphCodeBERT: pre\-training code representations with data flow\.InProc\. Int\. Conf\. Learn\. Represent\. \(ICLR\),Cited by:[§II\-A](https://arxiv.org/html/2608.05204#S2.SS1.p1.1)\.
- \[10\]M\. M\. Hasan, H\. Li, E\. Fallahzadeh, G\. K\. Rajbahadur, B\. Adams, and A\. E\. Hassan\(2025\)Model context protocol \(mcp\) at first glance: studying the security and maintainability of mcp servers\.ACM Trans\. Softw\. Eng\. Methodol\.\.Cited by:[§II\-B](https://arxiv.org/html/2608.05204#S2.SS2.p1.1)\.
- \[11\]B\. Hui, H\. Yuan, N\. Z\. Gong, P\. Burlina, and Y\. Cao\(2024\)PLeak: prompt leaking attacks against large language model applications\.InProc\. ACM Conf\. Comput\. Commun\. Secur\. \(CCS\),Cited by:[§II\-C](https://arxiv.org/html/2608.05204#S2.SS3.p1.1)\.
- \[12\]L\. Jiang, G\. Misherghi, Z\. Su, and S\. Glondu\(2007\)Deckard: scalable and accurate tree\-based detection of code clones\.InProc\. 29th Int\. Conf\. Softw\. Eng\. \(ICSE’07\),pp\. 96–105\.Cited by:[§I](https://arxiv.org/html/2608.05204#S1.p3.1),[§II\-A](https://arxiv.org/html/2608.05204#S2.SS1.p1.1)\.
- \[13\]Y\. Jiang, D\. Li, H\. Deng, B\. Ma, X\. Wang, Q\. Wang, and G\. Yu\(2026\)SoK: agentic skills–beyond tool use in llm agents\.arXiv preprint arXiv:2602\.20867\.Cited by:[§II\-B](https://arxiv.org/html/2608.05204#S2.SS2.p1.1)\.
- \[14\]T\. Kamiya, S\. Kusumoto, and K\. Inoue\(2002\)CCFinder: a multilinguistic token\-based code clone detection system for large scale source code\.IEEE Trans\. Softw\. Eng\.28\(7\),pp\. 654–670\.Cited by:[§II\-A](https://arxiv.org/html/2608.05204#S2.SS1.p1.1),[§III\-C2](https://arxiv.org/html/2608.05204#S3.SS3.SSS2.p1.1)\.
- \[15\]T\. Kim, D\. Jiang, Y\. Hu, Y\. Jia, and N\. Gong\(2026\)Evaluating tool cloning in agentic\-ai ecosystems\.Note:arXiv preprint arXiv:2605\.09817Cited by:[§I](https://arxiv.org/html/2608.05204#S1.p2.1),[§II\-B](https://arxiv.org/html/2608.05204#S2.SS2.p2.1),[§IV\-C1](https://arxiv.org/html/2608.05204#S4.SS3.SSS1.p1.1)\.
- \[16\]J\. Kornblum\(2006\)Identifying almost identical files using context triggered piecewise hashing\.Digital investigation3,pp\. 91–97\.Cited by:[§I](https://arxiv.org/html/2608.05204#S1.p3.1),[§IV\-C1](https://arxiv.org/html/2608.05204#S4.SS3.SSS1.p1.1)\.
- \[17\]G\. Ling, S\. Zhong, and R\. Huang\(2026\)Agent skills: a data\-driven analysis of claude skills for extending large language model functionality\.Note:arXiv preprint arXiv:2602\.08004Cited by:[§II\-B](https://arxiv.org/html/2608.05204#S2.SS2.p1.1)\.
- \[18\]G\. Ling, S\. Zhong, and R\. Huang\(2026\)Agent skills: a data\-driven analysis of claude skills for extending large language model functionality\.External Links:2602\.08004,[Link](https://arxiv.org/abs/2602.08004)Cited by:[§I](https://arxiv.org/html/2608.05204#S1.p1.1)\.
- \[19\]C\. Liu, C\. Chen, J\. Han, and P\. S\. Yu\(2006\)GPLAG: detection of software plagiarism by program dependence graph analysis\.InProc\. 12th ACM SIGKDD Int\. Conf\. Knowl\. Discov\. Data Min\.,pp\. 872–881\.Cited by:[§II\-A](https://arxiv.org/html/2608.05204#S2.SS1.p1.1)\.
- \[20\]Y\. Liu, Z\. Chen, Y\. Zhang, G\. Deng, Y\. Li, J\. Ning, and L\. Y\. Zhang\(2026\)Malicious agent skills in the wild: a large\-scale security empirical study\.Note:arXiv preprint arXiv:2602\.06547Cited by:[§II\-B](https://arxiv.org/html/2608.05204#S2.SS2.p1.1)\.
- \[21\]Y\. Liu, W\. Wang, R\. Feng, Y\. Zhang, G\. Xu, G\. Deng, Y\. Li, and L\. Zhang\(2026\)Agent skills in the wild: an empirical study of security vulnerabilities at scale\.Note:arXiv preprint arXiv:2601\.10338Cited by:[§I](https://arxiv.org/html/2608.05204#S1.p2.1),[§II\-B](https://arxiv.org/html/2608.05204#S2.SS2.p1.1)\.
- \[22\]Y\. Liu, G\. Deng, Y\. Li, K\. Wang, Z\. Wang, X\. Wang, T\. Zhang, Y\. Liu, H\. Wang, Y\. Zheng, L\. Y\. Zhang, and Y\. Liu\(2023\)Prompt injection attack against LLM\-integrated applications\.Note:arXiv preprint arXiv:2306\.05499Cited by:[§II\-C](https://arxiv.org/html/2608.05204#S2.SS3.p1.1)\.
- \[23\]Model Context Protocol\(2025\)Model context protocol specification\.Note:[https://modelcontextprotocol\.io/specification/2025\-06\-18](https://modelcontextprotocol.io/specification/2025-06-18)Accessed Jun\. 11, 2026Cited by:[§II\-B](https://arxiv.org/html/2608.05204#S2.SS2.p1.1)\.
- \[24\]OpenAI\(2023\)Introducing GPTs\.Note:[https://openai\.com/index/introducing\-gpts/](https://openai.com/index/introducing-gpts/)Accessed Jun\. 11, 2026Cited by:[§II\-B](https://arxiv.org/html/2608.05204#S2.SS2.p1.1)\.
- \[25\]OpenAI\(2024\)Introducing the GPT store\.Note:[https://openai\.com/index/introducing\-the\-gpt\-store/](https://openai.com/index/introducing-the-gpt-store/)Accessed Jun\. 11, 2026Cited by:[§II\-B](https://arxiv.org/html/2608.05204#S2.SS2.p1.1)\.
- \[26\]F\. Perez and I\. Ribeiro\(2022\)Ignore previous prompt: attack techniques for language models\.Note:arXiv preprint arXiv:2211\.09527Cited by:[§II\-C](https://arxiv.org/html/2608.05204#S2.SS3.p1.1)\.
- \[27\]L\. Prechelt, G\. Malpohl, and M\. Philippsen\(2002\)Finding plagiarisms among a set of programs with JPlag\.InJ\. Universal Comput\. Sci\.,Cited by:[§I](https://arxiv.org/html/2608.05204#S1.p3.1),[§II\-A](https://arxiv.org/html/2608.05204#S2.SS1.p1.1)\.
- \[28\]N\. Reimers and I\. Gurevych\(2019\)Sentence\-BERT: sentence embeddings using siamese BERT\-networks\.InProc\. Conf\. Empir\. Methods Nat\. Lang\. Process\. \(EMNLP\),pp\. 3982–3992\.Cited by:[§II\-A](https://arxiv.org/html/2608.05204#S2.SS1.p1.1)\.
- \[29\]V\. Roussev\(2010\)Data fingerprinting with similarity digests\.InProc\. IFIP Digital Forensics,Cited by:[§I](https://arxiv.org/html/2608.05204#S1.p3.1),[§II\-A](https://arxiv.org/html/2608.05204#S2.SS1.p1.1)\.
- \[30\]C\. K\. Roy and J\. R\. Cordy\(2007\)A survey on software clone detection research\.Queen’s School of computing TR541\(115\),pp\. 64–68\.Cited by:[§II\-A](https://arxiv.org/html/2608.05204#S2.SS1.p1.1)\.
- \[31\]H\. Sajnani, V\. Saini, J\. Svajlenko, C\. K\. Roy, and C\. V\. Lopes\(2016\)Sourcerercc: scaling code clone detection to big\-code\.InProc\. 38th Int\. Conf\. Softw\. Eng\. \(ICSE\),pp\. 1157–1168\.Cited by:[§I](https://arxiv.org/html/2608.05204#S1.p3.1),[§II\-A](https://arxiv.org/html/2608.05204#S2.SS1.p1.1),[§III\-C2](https://arxiv.org/html/2608.05204#S3.SS3.SSS2.p1.1)\.
- \[32\]S\. Schleimer, D\. S\. Wilkerson, and A\. Aiken\(2003\)Winnowing: local algorithms for document fingerprinting\.InProc\. ACM SIGMOD Int\. Conf\. Manage\. Data,pp\. 76–85\.Cited by:[§I](https://arxiv.org/html/2608.05204#S1.p3.1),[§II\-A](https://arxiv.org/html/2608.05204#S2.SS1.p1.1)\.
- \[33\]D\. Schuler, V\. Dallmeier, and C\. Lindig\(2007\)A dynamic birthmark for java\.InProc\. 22nd IEEE/ACM Int\. Conf\. Autom\. Softw\. Eng\.,pp\. 274–283\.Cited by:[§II\-A](https://arxiv.org/html/2608.05204#S2.SS1.p1.1)\.
- \[34\]L\. Wanget al\.\(2024\)A survey on large language model based autonomous agents\.Front\. Comput\. Sci\.\.Cited by:[§II\-B](https://arxiv.org/html/2608.05204#S2.SS2.p1.1)\.
- \[35\]Y\. Wang, W\. Wang, S\. Joty, and S\. C\. H\. Hoi\(2021\)CodeT5: identifier\-aware unified pre\-trained encoder\-decoder models for code understanding and generation\.InProc\. Conf\. Empir\. Methods Nat\. Lang\. Process\. \(EMNLP\),pp\. 8696–8708\.Cited by:[§II\-A](https://arxiv.org/html/2608.05204#S2.SS1.p1.1)\.
- \[36\]R\. Xu and Y\. Yan\(2026\)Agent skills for large language models: architecture, acquisition, security, and the path forward\.Note:arXiv preprint arXiv:2602\.12430Cited by:[§II\-B](https://arxiv.org/html/2608.05204#S2.SS2.p1.1)\.
- \[37\]Y\. Yang, Y\. Li, H\. Yao, E\. Huang, S\. Shao, Y\. Wang, Z\. Wang, D\. Tao, and Z\. Qin\(2026\)PromptCOS: towards content\-only system prompt copyright auditing for LLMs\.InProc\. IEEE Symp\. Secur\. Priv\. \(S&P\),pp\. 4301–4319\.External Links:[Document](https://dx.doi.org/10.1109/SP63933.2026.00142)Cited by:[§II\-C](https://arxiv.org/html/2608.05204#S2.SS3.p1.1)\.
- \[38\]Y\. Yang, C\. Li, Q\. Li, O\. Ma, H\. Wang, Z\. Wang, Y\. Gao, W\. Chen, and S\. Ji\(2025\)PRSA: prompt stealing attacks against Real\-World prompt services\.InProc\. 34th USENIX Security Symp\. \(USENIX Security 25\),pp\. 2283–2302\.Cited by:[§II\-C](https://arxiv.org/html/2608.05204#S2.SS3.p1.1)\.
- \[39\]H\. Yao, J\. Lou, K\. Ren, and Z\. Qin\(2024\)PromptCARE: prompt copyright protection by watermark injection and verification\.InProc\. IEEE Symp\. Secur\. Priv\. \(S&P\),Cited by:[§II\-C](https://arxiv.org/html/2608.05204#S2.SS3.p1.1)\.
- \[40\]W\. Zhao, J\. Liu, B\. Ruan, S\. Li, and Z\. Liang\(2025\)When MCP servers attack: taxonomy, feasibility, and mitigation\.Note:arXiv preprint arXiv:2509\.24272Cited by:[§II\-B](https://arxiv.org/html/2608.05204#S2.SS2.p1.1)\.
- \[41\]Y\. Zhou, Z\. Zhang, Z\. Cheng, S\. Zhang, Q\. Lan, Z\. Chen, Z\. Yang, R\. Chen, H\. Wang, S\. Hu,et al\.\(2026\)Skillgenbench: benchmarking skill generation pipelines for llm agents\.arXiv preprint arXiv:2605\.18693\.Cited by:[§II\-B](https://arxiv.org/html/2608.05204#S2.SS2.p1.1)\.Similar Articles
SkillAudit: Ground-Truth-Free Skill Evolution via Paired Trajectory Auditing
SkillAudit introduces a framework for evolving LLM agent skills without ground-truth feedback by using paired trajectory auditing and contrastive evaluation. It achieves 73.9% average task reward across 89 tasks, outperforming baseline methods.
SkillMaster: Toward Autonomous Skill Mastery in LLM Agents
This paper introduces SkillMaster, a training framework that enables LLM agents to autonomously create, refine, and select skills through trajectory-informed review and counterfactual utility evaluation.
Safeguarding LLM Agents from Misalignment through Provenance Analysis
This paper proposes a provenance-based framework and multi-stage pipeline, \tool, to detect misalignment in LLM agents' tool invocations before execution, reducing error rates significantly compared to LLM-as-a-judge baselines.
TRACE: Trajectory Reasoning through Adaptive Cross-Step Evidence Aggregation for LLM Agents
TRACE is a monitoring framework for long-horizon LLM agent trajectories that uses a Triage-Inspect-Judge loop to connect evidence across temporally distant actions, achieving high recall and F1 on evasive sabotage detection tasks.
SkillCorpus: Consolidating and Evaluating the Open Skill Ecosystem for Real-World LLM Agents
SkillCorpus presents a framework to consolidate, curate, and evaluate the open skill ecosystem for LLM agents, demonstrating consistent performance gains across multiple benchmarks through retrieval-augmented skill integration.