LLM-FACETS: A Privacy-Preserving Framework for Evaluating LLM Transparency and Accountability

# LLM-FACETS: A Privacy-Preserving Framework for Evaluating LLM Transparency and Accountability
Source: [https://arxiv.org/html/2605.31167](https://arxiv.org/html/2605.31167)
\(2026\)

###### Abstract\.

Assessing whether Large Language Models outputs are factually grounded, epistemically calibrated, and methodologically reproducible is a prerequisite for responsible AI deployment\. Yet the practice of auditing LLMs remains difficult to access for non\-technical practitioners: existing tools require programming expertise, non\-trivial environment configuration, and lack systematic methodological approaches to multi\-dimensional evaluation, while cloud\-hosted platforms transmit evaluation data to external services, creating barriers for domain experts and compliance officers who bear legal and ethical responsibility for AI oversight\. We introduceLLM\-FACETS\(LLMFActualityCross\-EvaluaTionSystem\): an open\-source evaluation framework, accompanied by a tool offering a browser\-accessible interface and a plugin architecture, designed to help practitioners make informed, transparent decisions about LLM quality\. The framework structures evaluation around three practitioner profiles \(technical experts, domain experts, and compliance officers\), inspired by the stakeholder categories identified in the EU AI Act and the NIST AI Risk Management Framework as responsible for human oversight of AI systems\. The tool’s architecture makes data flows explicit: deterministic metrics \(BLEU, ROUGE, BERTScore\) run entirely within the self\-hosted server process with no outbound data transmission; LLM\-judge metrics contact external APIs explicitly, with users retaining full control over credentials and responsible for applicable data protection agreements\. The framework operationalizes transparency through three auditing mechanisms: token\-level log\-probability visualization for epistemic uncertainty \(model confidence\) assessment, multi\-judge consensus evaluation to mitigate judge bias, and RAG Triad metrics \(Faithfulness, Answer Relevance, Context Relevance\) to detect and localize hallucinations\. A plugin architecture allows any new metric or dataset to be integrated without modifying the evaluation pipeline\. The open\-source implementation enables cross\-checking across multiple metrics targeting the same property, ensuring the reproducibility required by open science principles, and decoupling AI accountability from the teams that built the systems being assessed\. We verify the framework through cross\-validation of 18 metric implementations against canonical reference libraries\.

LLM evaluation, AI transparency, LLM\-as\-a\-Judge, Retrieval Augmented Generation, data flow transparency, responsible AI, hallucination detection, open\-source, evaluation framework, reproducible evaluation, open science, practitioner\-oriented transparency, plugin architecture

††copyright:cc††journalyear:2026††doi:XXXXXXX\.XXXXXXX††journal:JRC††journalvolume:0††journalnumber:0††article:0††publicationmonth:0††ccs:Computing methodologies Machine learning††ccs:Social and professional topics Accountability††ccs:Computing methodologies Natural language processing††ccs:Software and its engineering Software creation and management††ccs:General and reference Open source software## 1\.Introduction

Auditing Large Language Models \(LLMs\)—assessing whether their outputs are factually grounded, epistemically calibrated, and methodologically reproducible—has become a prerequisite for responsible deployment across high\-stakes domains, from clinical decision support to legal document analysis\(European Parliament and Council of the European Union,[2024](https://arxiv.org/html/2605.31167#bib.bib15)\)\. Yet this practice remains largely inaccessible to the practitioners who need it most\. The tools required demand programming expertise, non\-trivial environment configuration, and data transmission to external APIs\. As a result, the actors most responsible for accountability, e\.g\. domain experts and compliance officers, are often excluded from evaluation processes they are legally and ethically required to perform\. Regulatory frameworks make this gap difficult to ignore\. The EU Artificial Intelligence Act \(AI Act\)\(European Parliament and Council of the European Union,[2024](https://arxiv.org/html/2605.31167#bib.bib15)\)requires that human overseers of high\-risk AI systems be capable of interpreting model outputs and certifying compliance, which is, in practice, a functional requirement for evaluation interfaces that should be accessible also for non\-engineers\. The General Data Protection Regulation \(GDPR\)\(European Parliament and Council of the European Union,[2016](https://arxiv.org/html/2605.31167#bib.bib16)\)requires a formal Data Processing Agreement whenever personal data is transmitted to a third\-party processor, raising compliance questions for cloud\-hosted evaluation platforms applied to sensitive datasets\. Existing evaluation ecosystems reinforce this fragmentation\. Programmatic evaluation libraries such as DeepEval\(Confident AI,[2024](https://arxiv.org/html/2605.31167#bib.bib19)\)and Ragas\(Eset al\.,[2024](https://arxiv.org/html/2605.31167#bib.bib6)\)offer powerful evaluation but require programming expertise, dependency management, and scripting—Barrier 1: programming barrier; Barrier 2: environment configuration\. Together, these two barriers result in the systematic exclusion of non\-technical practitioners from evaluation workflows\. On the hosted\-service side, platforms such as Arize Phoenix\(Arize AI,[2024](https://arxiv.org/html/2605.31167#bib.bib30)\)and Langfuse\(Langfuse,[2024](https://arxiv.org/html/2605.31167#bib.bib31)\)focus on monitoring deployed LLM applications in live production environments rather than collaborative metric exploration\. Academic visual analytics tools\(Kahnget al\.,[2024](https://arxiv.org/html/2605.31167#bib.bib14)\)provide sophisticated visualizations but remain isolated prototypes\. None of these solutions addresses data sovereignty: evaluating sensitive datasets \(medical records, legal documents, proprietary corpora\) through third\-party APIs creates irreconcilable conflicts with data protection regulations—Lack of data sovereignty assurance\. This paper presentsLLM\-FACETS, an open\-source evaluation framework that directly addresses these barriers\. The framework is structured around three practitioner profiles \(technical experts, domain experts, and compliance officers\)\. This mirrors the functional roles identified in the EU AI Act Article 14\(European Parliament and Council of the European Union,[2024](https://arxiv.org/html/2605.31167#bib.bib15)\)and the NIST AI Risk Management Framework\(National Institute of Standards and Technology,[2023](https://arxiv.org/html/2605.31167#bib.bib17)\)as responsible for human oversight of AI systems\. The framework makes three primary contributions:

1. \(1\)Amethodological frameworkconnecting transparency goals to concrete evaluation practices, structured around the aforementioned practitioner profiles\. Each profile has distinct transparency needs, accountability roles, and recommended metric configurations\.
2. \(2\)Aunified suite of transparency auditing mechanismsspanning epistemic uncertainty \(log\-probability visualization\), factual grounding \(RAG Triad\), and process fairness \(multi\-judge consensus\), delivered through a browser\-accessible interface that requires no programming expertise\.
3. \(3\)Anopen\-source toolimplementing the framework through 18 metric variants in TypeScript, cross\-validated against canonical Python reference implementations, with a plugin architecture in which adding a new metric or dataset propagates automatically to the navigation, benchmarking dashboard, and REST API without further configuration111Source code:[https://github\.com/Scriptor\-Group/AIMVi](https://github.com/Scriptor-Group/AIMVi)\.—enabling reproducible and auditable evaluation independent of proprietary tooling \(Section[6\.1](https://arxiv.org/html/2605.31167#S6.SS1)\)\.

The remainder of this paper is structured as follows\. Section[2](https://arxiv.org/html/2605.31167#S2)identifies the challenges inherent to LLM evaluation, including the need for transparency across multiple dimensions\. Section[3](https://arxiv.org/html/2605.31167#S3)defines the three practitioner profiles, formalizes the framework, and presents the evaluation workflow\. Section[4](https://arxiv.org/html/2605.31167#S4)describes the LLM\-FACETS tool architecture\. Section[5](https://arxiv.org/html/2605.31167#S5)details how the implemented metrics operationalize the transparency dimensions\. Section[6](https://arxiv.org/html/2605.31167#S6)provides cross\-validation evidence and performance analysis\. Section[7](https://arxiv.org/html/2605.31167#S7)validates the proposed framework through multi\-stakeholder use cases\. Section[8](https://arxiv.org/html/2605.31167#S8)discusses limitations and broader impact\. Section[9](https://arxiv.org/html/2605.31167#S9)concludes\.

## 2\.Challenges

This section identifies the five challenges that motivate this work: the fragmented landscape of existing evaluation tools \(§[2\.1](https://arxiv.org/html/2605.31167#S2.SS1)\), the design requirements for accessible AI explanation interfaces \(§[2\.2](https://arxiv.org/html/2605.31167#S2.SS2)\), the legal and regulatory constraints on evaluation practices \(§[2\.3](https://arxiv.org/html/2605.31167#S2.SS3)\), the reproducibility crisis in LLM benchmarking \(§[2\.4](https://arxiv.org/html/2605.31167#S2.SS4)\), and the multi\-dimensional nature of transparency itself \(§[2\.5](https://arxiv.org/html/2605.31167#S2.SS5)\)\. Together, these challenges establish the concrete requirements that the framework must satisfy\.

### 2\.1\.C1 — Onboarding

The rapid adoption of LLMs has spurred a proliferation of evaluation tools, each addressing a subset of the evaluation challenge—broadly defined as the problem of systematically measuring model output quality in a reproducible, interpretable, and scalable manner\(Gehrmannet al\.,[2023](https://arxiv.org/html/2605.31167#bib.bib18)\)\. Frameworks addressing factual accuracy and targeted evaluation exist\. For instance, Ragas\(Eset al\.,[2024](https://arxiv.org/html/2605.31167#bib.bib6)\)provides a programmatic framework for evaluating Retrieval Augmented Generation \(RAG\) pipelines through metrics such as Faithfulness, Answer Relevance, and Context Relevance\. DeepEval\(Confident AI,[2024](https://arxiv.org/html/2605.31167#bib.bib19)\)extends this to a broader suite of over 14 metrics including hallucination detection\(Liet al\.,[2023](https://arxiv.org/html/2605.31167#bib.bib41)\), bias assessment\(Zhenget al\.,[2023](https://arxiv.org/html/2605.31167#bib.bib7)\), and toxicity scoring, each grounded in peer\-reviewed evaluation research\. Both frameworks are programmatic: they require Python environments, dependency management, and scripting to execute evaluations\. On the hosted\-service side, LangSmith\(LangChain, Inc\.,[2023](https://arxiv.org/html/2605.31167#bib.bib21)\)offers a cloud\-hosted observability platform focused on tracing and monitoring deployed LLM applications in live production environments, offering custom LLM\-as\-a\-Judge templates rather than a standardized metric suite\. TruLens\(TruEra, Inc\.,[2024](https://arxiv.org/html/2605.31167#bib.bib20)\)provides feedback functions for evaluating RAG applications with a visualization dashboard\. Arize Phoenix and Langfuse operate in a similar space, focusing on production monitoring and trace analysis\. These tools share two structural limitations\. First, they present atechnical barrier: all require programming expertise and development environment configuration, making participation difficult for domain experts and compliance officers\. Second, cloud\-hosted platforms such as LangSmith transmit evaluation data—which may include sensitive documents—to external servers, creating data sovereignty questions\. Even nominally open\-source tools that rely on cloud\-based LLM judge calls transmit the texts being evaluated to third\-party providers\. This dual limitation—technical complexity and data exposure—motivates the design of a tool that is both browser\-accessible and explicit about where data flows\. A complementary approach is taken by ARES\(Saad\-Falconet al\.,[2024](https://arxiv.org/html/2605.31167#bib.bib34)\), which trains lightweight discriminative judges \(DeBERTa variants\) specifically for RAG evaluation tasks\. By fine\-tuning small models on domain\-relevant data, ARES reduces dependence on frontier model APIs while achieving comparable precision at a fraction of the cost—a significant advantage for high\-volume or cost\-constrained evaluation\. However, ARES remains a Python research framework without a visual interface or data sovereignty guarantees\. Academic visual analytics tools have begun to address the accessibility dimension\. LLM Comparator\(Kahnget al\.,[2024](https://arxiv.org/html/2605.31167#bib.bib14)\)provides side\-by\-side visual evaluation of LLM outputs for qualitative comparison\. However, such tools remain isolated prototypes that do not integrate quantitative metrics, privacy guarantees, or multi\-practitioner workflows into a unified evaluation framework\. The lack of such unified tools represents a major onboarding and accessibility barrier for institutions\.

### 2\.2\.C2 — Accessibility

A growing body of HCI research demonstrates that the design of AI explanation interfaces critically determines whether practitioners can effectively exercise oversight\. Liao et al\.\(Liaoet al\.,[2020](https://arxiv.org/html/2605.31167#bib.bib22)\)establish that AI explanation tools must be designed around the questions that practitioners actually ask—not around the technical artifacts that engineers find convenient to produce\. Their question\-driven design framework reveals a persistent mismatch: most explanation tools answer “how does the model work?” when users need to know “can I trust this specific output?” Ehsan et al\.\(Ehsanet al\.,[2021](https://arxiv.org/html/2605.31167#bib.bib23)\)extend this insight to the concept ofsocial transparency: making the reasoning processes and social contexts behind AI decisions accessible to non\-technical practitioners\. Their work demonstrates that transparency is not achieved by exposing model internals, but by presenting evaluation processes in forms that enable meaningful participation by diverse actors\. For LLM evaluation tooling, the implication is concrete: presenting results as raw numerical scores in terminal outputs or Jupyter notebooks fails the accessibility test that this body of research identifies as essential for genuine human oversight\(Liaoet al\.,[2020](https://arxiv.org/html/2605.31167#bib.bib22); Ehsanet al\.,[2021](https://arxiv.org/html/2605.31167#bib.bib23)\)\. An evaluation framework that aims to support multi\-practitioner auditing must therefore translate scores into interactive representations structured around the questions that non\-technical auditors actually ask: “Is this output factually grounded?”, “Can I trust this specific claim?”, “Is the evaluation methodology itself reliable?”\(Liaoet al\.,[2020](https://arxiv.org/html/2605.31167#bib.bib22)\)\. Producing tools capable of empowering domain experts without sacrificing technical depth is a critical challenge\.

### 2\.3\.C3 — Compliance

The global regulatory landscape increasingly mandates evaluation capabilities that existing tools do not provide\. While these requirements take different forms across jurisdictions \(such as the proposed US Algorithmic Accountability Act\), the European legal framework provides explicit, broadly applicable examples of these duties\. The EU AI Act\(European Parliament and Council of the European Union,[2024](https://arxiv.org/html/2605.31167#bib.bib15)\)imposes multiple evaluation\-relevant obligations on providers and deployers of high\-risk AI systems\. Articles 9 and 11 require continuous risk assessment and detailed technical documentation of evaluation metrics and their results\. Article 13 mandates that system outputs be interpretable by their intended users\. Article 14 requires human oversight by persons capable of understanding the capabilities and limitations of the system—a functional requirement for accessible evaluation interfaces\. The GDPR\(European Parliament and Council of the European Union,[2016](https://arxiv.org/html/2605.31167#bib.bib16)\)introduces a distinct but equally binding constraint\. Article 28 requires a formal Data Processing Agreement \(DPA\) whenever personal data is transmitted to a third party for processing\. Evaluating LLM outputs on datasets containing personal information—patient records, employee communications, legal case files—through cloud\-hosted evaluation APIs constitutes data processing under this definition\. In the absence of a DPA with the evaluation platform provider, such evaluation is unlawful under European data protection law\. ISO/IEC 42001:2023\(International Organization for Standardization,[2023](https://arxiv.org/html/2605.31167#bib.bib24)\)provides the first international management system standard for AI, requiring organizations to establish documented processes for AI risk assessment, continuous monitoring, and human control\. The NIST AI Risk Management Framework\(National Institute of Standards and Technology,[2023](https://arxiv.org/html/2605.31167#bib.bib17)\)similarly recommends measurable evaluation practices accessible to diverse practitioners\. Taken together, these three instruments establish distinct but complementary requirements: the EU AI Act mandates accessibility \(Art\. 14\) and technical documentation \(Arts\. 9, 11, 13\); GDPR constrains data transmission to third parties \(Art\. 28\); ISO 42001 requires documented, continuous monitoring processes\. No existing evaluation tool satisfies all three requirements simultaneously—a gap that motivates the architectural choices described in Section[4](https://arxiv.org/html/2605.31167#S4)\. Commercial governance platforms such as Credo AI\(Credo AI,[2024](https://arxiv.org/html/2605.31167#bib.bib38)\)and Fiddler AI\(Fiddler AI,[2024](https://arxiv.org/html/2605.31167#bib.bib39)\)address the compliance reporting dimension by translating technical evaluation evidence into regulatory artifacts mapped to the EU AI Act and ISO 42001\. However, these platforms require cloud data transmission and do not address the data sovereignty constraint: organizations subject to GDPR cannot use them to evaluate datasets containing personal information without a Data Processing Agreement\. This leaves a structural gap between compliance tooling and privacy\-preserving evaluation\.

### 2\.4\.C4 — Reproducibility

The reliability of LLM evaluation itself is increasingly questioned\. Gehrmann et al\.\(Gehrmannet al\.,[2023](https://arxiv.org/html/2605.31167#bib.bib18)\)conduct a comprehensive survey documenting evaluation obstacles including inconsistent metric implementations, unreported hyperparameters, and lack of standardization—concluding that the evaluation foundation of NLP \(Natural Language Processing\) exhibits systemic vulnerabilities that require methodological repair\. Post\(Post,[2018](https://arxiv.org/html/2605.31167#bib.bib26)\)demonstrates that reported BLEU scores—a widely used lexical overlap metric measuringnn\-gram precision between a generated text and reference translations—are not comparable across publications because implementations differ in tokenization, smoothing, and case normalization, rendering cross\-paper comparisons unreliable\. Questioning the evaluation paradigm itself, Bender et al\.\(Benderet al\.,[2021](https://arxiv.org/html/2605.31167#bib.bib25)\)argue that the benchmarking paradigm creates a false sense of progress: language model outputs are optimized for benchmark performance without adequate evaluation of real\-world behavior, and the scale of modern language models makes meaningful evaluation increasingly difficult\. This critique gains urgency in the LLM\-as\-a\-Judge paradigm, where Zheng et al\.\(Zhenget al\.,[2023](https://arxiv.org/html/2605.31167#bib.bib7)\)quantify position bias at 5–15 percentage points and demonstrate that single\-judge evaluation produces unstable rankings\. Three requirements for credible LLM evaluation follow from these findings:

1. \(1\)Deterministic metric implementations must be cross\-validated to ensure comparability across tools and studies\.
2. \(2\)LLM judge evaluations must report variance and bias quantification alongside primary scores\.
3. \(3\)Evaluation methodology must be transparent and reproducible so that third parties can verify published results\.

Building tools that enforce these requirements systematically, rather than leaving them to ad\-hoc scripts, is essential to repair the evaluation foundation\.

### 2\.5\.C5 — Transparency

Transparency in AI is a multidimensional concept that is frequently conflated with a single technical property \(e\.g\., explainability or interpretability\)\. A more fundamental observation, however, is that transparency is not a static property that a system either has or lacks: it is an activerelationbetween information and its audience\(Liaoet al\.,[2020](https://arxiv.org/html/2605.31167#bib.bib22); Ehsanet al\.,[2021](https://arxiv.org/html/2605.31167#bib.bib23)\)\. The same model output that is transparent to a machine learning engineer reading raw log\-probabilities in a terminal is entirely opaque to a compliance officer who needs to certify whether the model fabricates medical claims\. Transparency, in other words, is always transparencyfor someone, shaped by that person’s epistemic needs, accountability role, and available tools\. This relational framing has a direct architectural consequence: a framework that genuinely engineers transparency cannot offer a single view of model output quality and reliability\. It must provide distinct mechanisms oriented toward distinct purposes—each designed around the questions that a specific audience needs to answer\. For the purposes of LLM evaluation, we distinguish four operationalizable dimensions that correspond directly to the framework’s feature set\.

#### 2\.5\.1\.Epistemic Transparency: Knowing What the Model Doesn’t Know

Epistemic transparency concerns the model’s ability to communicate its own uncertainty\. A model that confidently generates a plausible\-sounding but factually incorrect statement—a hallucination—fails epistemically not because it errs, but because it does not signal that it may err\. Token\-level log\-probabilities \(LogProbs\) provide a direct window into this dimension\. Atokenis a sub\-word unit \(e\.g\., a word or morpheme\) produced sequentially by the model’s decoding process\. For each generated tokentit\_\{i\}, the model assigns an output probabilityp​\(ti∣t<i\)p\(t\_\{i\}\\mid t\_\{<i\}\): the conditional probability of that token given all preceding tokens\. Low values indicate regions of uncertainty where hallucination risk is elevated\. The sequence\-level confidence can be summarized as the mean of per\-token log\-probabilities:1n​∑i=1nlog⁡p​\(ti∣t<i\)\\frac\{1\}\{n\}\\sum\_\{i=1\}^\{n\}\\log p\(t\_\{i\}\\mid t\_\{<i\}\)\. Prior academic work has addressed LogProb visualization for ML researchers: the LM Transparency Tool\(Tufanovet al\.,[2024](https://arxiv.org/html/2605.31167#bib.bib36)\)provides interactive projections of layer\-wise attention contributions and token probability distributions, enabling mechanistic analysis of Transformer internals\. However, such tools target researchers studying model behavior, not non\-technical auditors assessing the reliability of specific outputs in a production context\. Despite their diagnostic value, LogProbs remain underutilized in practice because, to our knowledge, no standard web interface exposes them in an accessible, visual format for non\-technical practitioners\. Creating such interfaces is a critical challenge\.

#### 2\.5\.2\.Factual Transparency: Tracing Outputs to Sources

Factual transparency is central to RAG systems, which are increasingly deployed to ground LLM outputs in retrieved documents\. When such a system generates a claim, a responsible evaluator must ask:Is this claim supported by the retrieved context? Is the retrieved context actually relevant to the question? Is the generated answer actually addressing the question asked?The RAG Triad—Faithfulness, Answer Relevance, and Context Relevance\(Eset al\.,[2024](https://arxiv.org/html/2605.31167#bib.bib6)\)—operationalizes these three questions as measurable scores\. Faithfulness decomposes the generated response into atomic claims and verifies each against the retrieved context\. Answer Relevance uses reverse question generation to assess whether the response actually addresses the original query\. Context Relevance classifies retrieved sentences as signal or noise relative to the query\. Together, these metrics provide a traceable audit trail linking the model output back to its evidential basis\.

#### 2\.5\.3\.Process Transparency: Auditing the Evaluators

A less\-discussed but critical form of transparency concerns the evaluation process itself\. LLM\-as\-a\-Judge methods—where a powerful LLM scores the outputs of another model—have become the dominant paradigm for automated quality assessment\(Zhenget al\.,[2023](https://arxiv.org/html/2605.31167#bib.bib7); Liuet al\.,[2023](https://arxiv.org/html/2605.31167#bib.bib5)\)\. However, these judges are subject to well\-documented biases: position bias \(favoring responses presented first in the prompt context, meaning the comparisonAAvsBBmay yield a different outcome thanBBvsAAsolely due to their spatial ordering in the LLM’s context window\), verbosity bias \(rewarding longer responses\), and self\-enhancement bias \(favoring outputs from the same model family\)\. Process transparency requires that the evaluation methodology itself be auditable\. To enable practitioners to assess not justwhatthe evaluation result is, buthow reliable the evaluation process itself is, tools must mitigate known biases like position\-bias and verbosity bias across comparisons, and ensure the variance of results is exposed\. Without mechanisms like multi\-judge consensus or prompt permutations transparently exposed, automated evaluation remains a black box for non\-technical practitioners\.

#### 2\.5\.4\.Baseline Grounding: Anchoring Evaluation in Deterministic References

A fourth dimension of transparency concerns the availability of deterministic, reproducible reference points against which LLM\-based evaluations can be calibrated\. Traditional n\-gram metrics \(BLEU, ROUGE, METEOR\) and neural similarity measures \(BERTScore\) do not themselves implement epistemic, factual, or process transparency in the senses defined above\. Instead, they providebaseline grounding: well\-understood, externally validated measurements whose outputs are fully determined by the input—without stochastic LLM calls—and whose implementations can be cross\-validated against canonical reference libraries\. By anchoring the evaluation in these deterministic baselines, auditors can detect systematic divergences between traditional scores and LLM judge scores, which may reveal prompt engineering artifacts, model self\-bias, or genuine quality differentials that word\-overlap metrics cannot capture\. Baseline grounding thus serves as a calibration layer that strengthens the credibility of the other three transparency dimensions\.

## 3\.Audience, terminology, and workflow

The value of an evaluation framework is realized only when its methods are genuinely usable by the practitioners who need them\. This section defines the methodological contribution that structures how practitioners apply evaluation features purposefully to produce accountable evidence\. It first characterizes the three practitioner profiles, then formalizes the framework as a mathematical structure, and finally presents the evaluation workflow\.

### 3\.1\.Audience

To address the diverse accountability demands of LLM evaluation, the framework is built around three practitioner profiles\. These profiles map specific responsibilities to actionable transparency goals:

Table 1\.Practitioner Profiles and their corresponding transparency needsThe selection of these three profiles is grounded in the functional responsibilities that regulatory instruments assign to distinct categories of human overseers\. Article 14 of the EU AI Act\(European Parliament and Council of the European Union,[2024](https://arxiv.org/html/2605.31167#bib.bib15)\)requires that persons exercising oversight of high\-risk AI systems possess the ability to “correctly interpret the high\-risk AI system’s output,” a competence that differs structurally depending on whether the overseer is a technical implementer, a subject\-matter authority, or a governance officer\. The NIST AI Risk Management Framework\(National Institute of Standards and Technology,[2023](https://arxiv.org/html/2605.31167#bib.bib17)\)similarly distinguishes between AI actors whodesign and developAI systems, those whodeploy and usethem within a domain of expertise, and those responsible forgovernance and compliance—three functional categories that map directly onto the profiles in Table[1](https://arxiv.org/html/2605.31167#S3.T1)\. Table[1](https://arxiv.org/html/2605.31167#S3.T1)summarizes how each profile’s accountability role determines a distinct transparency need and, consequently, a distinct subset of the framework’s metrics\. TheTechnical Expert\(e\.g\., an ML engineer debugging a RAG pipeline\) requires epistemic and factual transparency: token\-level log\-probabilities to diagnose model uncertainty, and the RAG Triad to trace hallucinations to retrieval failures, producing per\-claim diagnostic breakdowns\. TheDomain Expert\(e\.g\., a clinical pharmacologist validating drug\-interaction answers\) requires factual and process transparency: Faithfulness scores to verify that outputs are grounded in authoritative sources, and the Jury module to assess whether evaluation judgments are stable across multiple independent judges, producing structured reports suitable for communication with technical teams\. TheCompliance Officer\(e\.g\., a data protection officer certifying a patient triage system\) requires process transparency above all: batch\-level RAG Triad scores and Faithfulness audits that can be exported as JSON/CSV audit trails for inclusion in regulatory documentation required under Articles 9 and 11 of the EU AI Act\. These profiles are not mutually exclusive: a single evaluation session may serve multiple roles simultaneously\. They are, however, distinct in thepurposethey assign to transparency—diagnostic, deliberative, or compliance\-oriented—and the framework’s design ensures that each purpose maps to specific, usable evaluation mechanisms\.

### 3\.2\.Terminology and Framework Definition

###### Definition 0\.

The*LLM\-FACETS*framework is a four\-tuple

ℱ=\(P,M,W,E\)\\mathcal\{F\}=\(P,\\;M,\\;W,\\;E\)where:

- •P=\{peng,pdom,pcmp\}P=\\\{p\_\{\\mathrm\{eng\}\},\\,p\_\{\\mathrm\{dom\}\},\\,p\_\{\\mathrm\{cmp\}\}\\\}is the finite set of*practitioner profiles*—technical expert, domain expert, and compliance officer—each characterized by a transparency goalg​\(p\)∈\{baseline​\_​grounding,epistemic,factual,process\}g\(p\)\\in\\\{\\mathrm\{baseline\\\_grounding\},\\,\\mathrm\{epistemic\},\\,\\mathrm\{factual\},\\,\\mathrm\{process\}\\\}, a recommended metric subsetMp⊆MM\_\{p\}\\subseteq M, and an expected output formo​\(p\)o\(p\);
- •M=Mlex∪Mneu∪Mjudge∪MragM=M\_\{\\mathrm\{lex\}\}\\cup M\_\{\\mathrm\{neu\}\}\\cup M\_\{\\mathrm\{judge\}\}\\cup M\_\{\\mathrm\{rag\}\}is the*set of metrics*\(Table[5](https://arxiv.org/html/2605.31167#S5.T5)\), where each metricm∈Mm\\in Mis annotated with a transparency dimensiond​\(m\)∈\{baseline​\_​grounding,epistemic,factual,process\}d\(m\)\\in\\\{\\mathrm\{baseline\\\_grounding\},\\,\\mathrm\{epistemic\},\\,\\mathrm\{factual\},\\,\\mathrm\{process\}\\\}and an execution type \(deterministic, provider\-native, or LLM\-based\);
- •W=\(w1,…,w5\)W=\(w\_\{1\},\\ldots,w\_\{5\}\)is the*workflow*\(presented in the next subsection\), mapping a profilep∈Pp\\in Pand a datasetDDto a scored evidence setℰ\\mathcal\{E\};
- •EEis the*generated evidence*, producing structured artifacts containing per\-sample scores, metric configurations, and full reasoning chains, enabling independent verification by third parties\.

The four components interact as follows\. A practitioner selects a profilep∈Pp\\in P\. The workflowWWguides the practitioner through five ordered steps—from goal definition to evidence export—enforcing the cross\-checking property that at least two metrics targeting the same dimension are used\. The audit interfaceEEpackages the resulting evidence into a portable, independently verifiable artifact\.

### 3\.3\.Evaluation Workflow

The evaluation workflow is defined independently of any specific tool implementation\. It structures how any practitioner—regardless of technical background—should proceed to obtain accountable evidence that allows them to make informative decisions:

1. \(1\)Define the transparency goal\(Algorithm[1](https://arxiv.org/html/2605.31167#algorithm1), Lines 1–11\)\. Identify which dimension is at stake—epistemic \(model confidence\), factual \(output grounding\), or process \(evaluation reliability\)—using the taxonomy in Section[2\.5](https://arxiv.org/html/2605.31167#S2.SS5)as a guide\.
2. \(2\)Select relevant metrics\(Algorithm[1](https://arxiv.org/html/2605.31167#algorithm1), Lines 12–14\)\. Consult Table[5](https://arxiv.org/html/2605.31167#S5.T5)to identify the metrics that operationalize the target dimension\. For robust conclusions, select at least two metrics targeting the same property to enable cross\-checking\.
3. \(3\)Load or configure a dataset\(Algorithm[1](https://arxiv.org/html/2605.31167#algorithm1), Lines 15–16\)\. Use an integrated benchmark \(SQuAD v2, SelfAware, HaluEval\) for standardized comparability, or upload a proprietary dataset\. The architecture \(Section[4](https://arxiv.org/html/2605.31167#S4)\) ensures that deterministic metrics require no outbound data transmission regardless of dataset sensitivity\.
4. \(4\)Cross\-check results\(Algorithm[1](https://arxiv.org/html/2605.31167#algorithm1), Lines 17–25\)\. Interpret scores from multiple metrics targeting the same property\. Convergence provides stronger evidence; divergence reveals where metrics disagree and warrants deeper investigation\. When metrics targeting the same dimension diverge \(r<0\.5r<0\.5in the correlation heatmap\), examine intermediate outputs: for Faithfulness vs\. BERTScore divergence, inspect the claim\-decomposition reasoning chain for errors\. The benchmark analysis dashboard \(Section[4](https://arxiv.org/html/2605.31167#S4)\) provides correlation heatmaps and distribution plots for this purpose\.
5. \(5\)Package evidence\(Algorithm[1](https://arxiv.org/html/2605.31167#algorithm1), Lines 26–27\)\. When evaluation is complete, results can be exported as structured JSON or CSV as a portable audit trail\. The exported file contains per\-sample scores, metric configurations, and—for LLM\-as\-a\-Judge metrics—the full reasoning chains, enabling independent verification by third parties\.

Algorithm[1](https://arxiv.org/html/2605.31167#algorithm1)formalizes this workflow, making explicit the inputs, outputs, and decision points of each step\.

Input:Practitioner profile

p∈Pp\\in P;

dataset

DD\(benchmark or proprietary corpus\)

Output:Evidence set

ℰ\\mathcal\{E\}: per\-sample scores, configurations, and reasoning chains

1

21exStep 1 — Choose an LLM

l←SelectModel​\(\)l\\leftarrow\\textsc\{SelectModel\}\(\)

//Select provider and model

3

41exStep 2 — Choose if anonymisation or not

a←SelectAnonymisation​\(\)a\\leftarrow\\textsc\{SelectAnonymisation\}\(\)

//Local NER / Presidio / None

5

61exStep 3 — Define transparency goal

d←g​\(p\)d\\leftarrow g\(p\)

//epistemic / factual / process \(Section[2\.5](https://arxiv.org/html/2605.31167#S2.SS5)\)

7

81exStep 4 — Select metric subset

9

Mp←\{m∈M∣d​\(m\)=d\}M\_\{p\}\\leftarrow\\\{m\\in M\\mid d\(m\)=d\\\}
10if*\|Mp\|<2\|M\_\{p\}\|<2*thenwarn: cross\-checking requires

≥2\{\\geq\}\\,2metrics per dimension

11

121exStep 5 — Load and configure dataset

13if*DDis a benchmark*thenload via Dataset Registry \(Section[4](https://arxiv.org/html/2605.31167#S4)\)

14elseupload CSV/Parquet; query in\-process via DuckDB \(data sovereignty preserved\)

15if*DDcontains Personally Identifiable Information \(PII\)*thenapply anonymization pipeline before any outbound API call

16

171exStep 6 — Cross\-check results

18foreach*m∈Mpm\\in M\_\{p\},s∈Ds\\in D*do

19

scorem​\(s\)←Evaluate​\(m,s\)\\mathrm\{score\}\_\{m\}\(s\)\\leftarrow\\textsc\{Evaluate\}\(m,s\)
20

21end foreach

22Compute inter\-metric Pearson correlation matrix

RRover

\{scorem\}m∈Mp\\\{\\mathrm\{score\}\_\{m\}\\\}\_\{m\\in M\_\{p\}\}
23if*∃mi,mj∈Mp:ri​j<0\.5\\exists\\,m\_\{i\},m\_\{j\}\\in M\_\{p\}:\\ r\_\{ij\}<0\.5*thenflag divergence; inspect intermediate outputs

24

251exStep 7 — Package evidence \(optional\)

ℰ←Export​\(\{scorem\}m∈Mp,configurations,reasoning chains\)\\mathcal\{E\}\\leftarrow\\textsc\{Export\}\(\\\{\\mathrm\{score\}\_\{m\}\\\}\_\{m\\in M\_\{p\}\},\\;\\text\{configurations\},\\;\\text\{reasoning chains\}\)

//JSON / CSV

return

ℰ\\mathcal\{E\}

Algorithm 1LLM\-FACETS Evaluation Workflow

## 4\.LLM\-FACETS Tool

![Refer to caption](https://arxiv.org/html/2605.31167v1/landing.png)Figure 1\.Landing page: initializing an LLM evaluation workflow\.Screenshot of the LLM\-FACETS landing page showing the initialization of an LLM evaluation workflow\.LLM\-FACETS is a Next\.js application comprising two layers: a browser client layer \(UI, IndexedDB storage, client\-side computation\) and a self\-hosted, stateless server process \(API routes, in\-process evaluation, BYOK proxy\), as illustrated in Figure[3](https://arxiv.org/html/2605.31167#S4.F3)\. The browser client \(blue zone\) manages the user interface, API key storage via IndexedDB, and evaluation results; the server process \(green zone\) hosts the REST API routes, the BYOK credential proxy, in\-process metric computation \(BERTScore via Transformers\.js, DuckDB for dataset queries\), and the models cache; the external LLM APIs \(orange zone\) are contacted only when the user explicitly selects LLM\-based metrics\. The server never persists user data or credentials between requests\. Figure[1](https://arxiv.org/html/2605.31167#S4.F1)shows the landing page from which a practitioner initializes an evaluation workflow, and Figure[2](https://arxiv.org/html/2605.31167#S4.F2)shows the configuration interface for defining practitioner profiles and metric strategies\. The architectural decisions are driven by the challenges established in Section[2](https://arxiv.org/html/2605.31167#S2): the interface must be accessible without programming expertise \(C1, C2\), data flows must be explicit and controllable \(C3\), and metric implementations must be cross\-validated and auditable \(C4\)\. This section describes the tool’s architecture and the design choices that address these challenges; Section[5](https://arxiv.org/html/2605.31167#S5)details how the implemented metrics operationalize the transparency dimensions \(C5\); Section[6](https://arxiv.org/html/2605.31167#S6)provides the implementation verification and performance evidence\. Two properties are targeted as architectural guarantees of the frameworkℱ\\mathcal\{F\}—intended to hold independently of configuration or deployment context\.Data sovereignty\.For deterministic metrics—those whose output is fully determined by the input without stochastic LLM calls—\(MlexM\_\{\\mathrm\{lex\}\},MneuM\_\{\\mathrm\{neu\}\}\), all computation executes either client\-side via WebAssembly or within the self\-hosted Next\.js server process; no evaluation data leaves the practitioner’s infrastructure\. For LLM\-judge metrics that require external API calls, the practitioner explicitly supplies credentials via per\-request credential injection \(the BYOK model detailed in Section[4\.1](https://arxiv.org/html/2605.31167#S4.SS1)\), and neither credentials nor evaluation data are persisted server\-side between requests\. Table[2](https://arxiv.org/html/2605.31167#S5.T2)in Section[5](https://arxiv.org/html/2605.31167#S5)documents the execution boundary for each metric\.Implementation correctness\.All deterministic metric implementations are cross\-validated against canonical reference libraries \(Section[6\.1](https://arxiv.org/html/2605.31167#S6.SS1)\), ensuring numerical reproducibility\.Methodological auditability\.LLM\-judge metrics expose full reasoning chains in every exported artifact, enabling independent reproduction of any element ofℰ\\mathcal\{E\}\.

![Refer to caption](https://arxiv.org/html/2605.31167v1/strategy.png)Figure 2\.Configuration UI for defining practitioner profiles and metric strategies\.Screenshot of the LLM\-FACETS configuration user interface for defining practitioner profiles and selecting metric strategies\.![Refer to caption](https://arxiv.org/html/2605.31167v1/architecture.png)Figure 3\.Architecture of LLM\-FACETS\.Blue zone: browser client \(IndexedDB stores API keys and results locally\)\.Green zone: self\-hosted Next\.js server process; all computation stays within user infrastructure—BLEU, ROUGE, METEOR, and BERTScore produce no outbound calls\.Orange zone: external LLM APIs, contacted only for LLM\-based metrics \(Jury, RAG Triad, G\-Eval, LogProbs\); data exposure is mitigated by the anonymization pipeline \(Section[4](https://arxiv.org/html/2605.31167#S4)\)\.Architecture diagram of LLM\-FACETS showing three zones: the browser client layer containing the React UI and IndexedDB storage for API keys and results; the self\-hosted Next\.js server layer containing the REST API routes, BYOK proxy, BERTScore via Transformers\.js, DuckDB in\-process, and Models Cache; and the external LLM APIs zone contacted only for judge\-based metrics\.### 4\.1\.The BYOK Privacy Model

The Bring Your Own Key \(BYOK\) model is the cornerstone of the framework’s privacy guarantees, directly addressing the compliance challenge \(C3\)\. LLM API keys are stored exclusively in the user’s browser via IndexedDB—a client\-side storage API with origin isolation enforced by the browser security model\. Keys are transmitted to LLM\-FACETS’ server routes using theX\-Provider\-KeysHTTP header and are used exclusively to instantiate provider objects for the duration of a single request\. They are never written to server\-side logs, databases, environment files, or any persistent storage\. This design means that even a fully compromised server cannot retroactively expose user credentials or reconstruct which keys were used for which evaluations\. For shared\-instance deployments \(e\.g\., organizational intranet servers with a single shared API key\), the BYOK model falls back to server\-side environment variables, maintaining the zero\-logging guarantee while enabling centralized key management\.

### 4\.2\.In\-Process Computation for Data Sovereignty

Beyond key management, the framework minimizes server\-side data retention through two mechanisms, extending the compliance guarantees \(C3\) to the computation level:In\-process semantic evaluation\.BERTScore computation uses contextual embeddings \(dense vector representations capturing semantic meaning\) generated via Transformers\.js\(Xenova,[2024](https://arxiv.org/html/2605.31167#bib.bib10)\)within the self\-hosted Next\.js server process\. The embedding model \(all\-MiniLM\-L6\-v2\) is loaded once as a server\-side singleton and processes all inputs locally, without transmitting text or embeddings to any external service\. This enables organizations to perform neural similarity evaluation\(Zhanget al\.,[2020](https://arxiv.org/html/2605.31167#bib.bib4)\)on sensitive documents \(e\.g\., clinical notes, legal briefs\) while keeping all data within their own infrastructure\.In\-process dataset analysis\.Dataset management uses DuckDB\(Raasveldt and Mühleisen,[2019](https://arxiv.org/html/2605.31167#bib.bib9)\), an in\-process analytical database engine that reads Apache Parquet files\(Apache Software Foundation,[2013](https://arxiv.org/html/2605.31167#bib.bib11)\)directly without requiring a server process\. Datasets are stored locally under a configurable path\. All queries \(schema introspection, random sampling, SQL filtering\) execute within the Next\.js server process and are never transmitted to external services\. This architecture enables SQL\-level analysis of large evaluation corpora while preserving full data sovereignty \(C3\) and enabling reproducible dataset exploration without external tooling \(C1\)\.

### 4\.3\.Privacy\-Preserving Input Anonymization

To evaluate LLM outputs on corpora containing personally identifiable information \(PII\)—a prerequisite for auditing in healthcare, legal, and financial domains—LLM\-FACETS provides a configurable anonymization pipeline that redacts sensitive entities before any text is transmitted to an external LLM judge API\. This extends the data sovereignty guarantee of the BYOK architecture to the content level: identifying information is eliminated from the evaluated text prior to any outbound network call\. The pipeline operates in two sequential stages\. A first stage deterministically applies regular expressions to identify four high\-precision structural entity types: email addresses, telephone numbers \(using standardized regex patterns\), IPv4 addresses, and credit card numbers\. A second, pluggable stage handles linguistic entities through Named Entity Recognition \(NER\), with three configurable strategies:Local NER \(compromise\.js\(Kelly,[2016](https://arxiv.org/html/2605.31167#bib.bib27)\)\)\.A server\-side JavaScript NLP library, integrated as a dependency within the Next\.js server process \(green zone in Figure[3](https://arxiv.org/html/2605.31167#S4.F3)\), that detects PERSON, LOCATION, and ORGANIZATION entities using grammatical rules, with no machine learning model and no network call\. This strategy operates fully offline with negligible latency \(<10<10ms per 1K words\) and is natively available within the Node\.js environment without external service dependencies\.Presidio\(Microsoft Corporation,[2018](https://arxiv.org/html/2605.31167#bib.bib29)\)\.A standalone self\-hosted NER service supporting over seven entity types across more than 20 languages with configurable confidence thresholds\. When deployed as an adjacent microservice on an organizational intranet, it provides high\-precision multilingual detection without transmitting data to any third party, satisfying the same GDPR Article 28 guarantees as the BYOK model\.Model\-based NER\(Google DeepMind,[2024](https://arxiv.org/html/2605.31167#bib.bib28)\)\.A Gemini LLM configured at temperature=0=0\(a parameter controlling output randomness; setting it to zero produces the most deterministic, reproducible responses\) returns a structured JSON array of detected entities, providing context\-aware detection of complex co\-references and domain\-specific identifiers that rule\-based approaches may miss\. Across all strategies, the internalEntityTrackercomponent automatically applies to enforceco\-reference consistency: each unique entity is assigned a typed, numbered placeholder upon first occurrence \(e\.g\.,\[PERSON\_1\],\[LOCATION\_2\]\), and the same placeholder is reused for all subsequent occurrences within the document\. This preserves the referential structure of the text—semantically meaningful for evaluating factual coherence—while eliminating identifying information\. The active strategy is selectable per\-session through the LLM\-FACETS interface; administrators may enforce a fixed strategy at the server level via environment variable, enabling policy\-compliant deployment in regulated environments\. It is important to note that metrics are computed over the anonymized text, not the original\. Lexical overlap metrics \(BLEU, ROUGE, METEOR\) will show reduced scores when evaluated against non\-anonymized reference texts, since placeholders replace named entities\. This trade\-off is expected: the anonymization pipeline prioritizes data protection at the cost of absolute metric fidelity\. Practitioners should ensure reference texts are anonymized consistently when using the anonymization pipeline for metric\-grounded evaluation\.

### 4\.4\.Plugin Architecture: Adding Metrics, Providers, and Datasets

The framework is built on Next\.js 16 \(App Router\), which provides server\-side rendering and a RESTful API layer\. Four design patterns underpin its extensibility\. They map directly to the Next\.js server components shown in Figure[3](https://arxiv.org/html/2605.31167#S4.F3)\. Together they ensure that adding a new capability to LLM\-FACETS never requires modifying the core evaluation pipeline\.Metric Registry\.Located in the API Routes layer of the server process \(green zone in Figure[3](https://arxiv.org/html/2605.31167#S4.F3)\), all metrics implement a commonMetricinterface exposing acompute\(sample\)function that returns a uniformly normalized score in\[0,1\]\[0,1\]\(where 1 consistently indicates the optimal or most\-reliable result regardless of the underlying metric semantics, meaning a low score always signifies poor quality or high hallucination risk\), together with optional reasoning chains and intermediate outputs\. Adding a new metric requires three steps: \(1\) implement theMetricinterface; \(2\) register it with metadata \(name, description, category, transparency dimension\) in the central registry; \(3\) no further changes are needed—the metric is automatically available in the*Navigation*and*React UI Components*\(Figure[3](https://arxiv.org/html/2605.31167#S4.F3)\), benchmarking dashboard, REST API, and export artifacts\. This means any metric proposed in future research can be integrated without modifying the evaluation pipeline\.Dataset Registry\.Implemented within the Dataset Service \(Figure[3](https://arxiv.org/html/2605.31167#S4.F3)\), datasets extend an abstractDatasetBaseclass that exposes a singlenormalize\(row\)method mapping dataset\-specific field names to the uniformDatasetSampleinterface\. Adding a new dataset requires implementingnormalize\(row\)and registering the dataset metadata; it then becomes available in the dataset explorer \(Figure[6](https://arxiv.org/html/2605.31167#S4.F6)\), benchmarking pipeline, and sample viewer\. Four datasets are currently implemented: SQuAD v2\(Rajpurkaret al\.,[2018](https://arxiv.org/html/2605.31167#bib.bib8)\)and PsiloQA\(Rykovet al\.,[2025](https://arxiv.org/html/2605.31167#bib.bib12)\)for reading\-comprehension Question Answering \(QA\), SelfAware\(Yinet al\.,[2023](https://arxiv.org/html/2605.31167#bib.bib40)\)for knowledge\-boundary and refusal evaluation, and HaluEval\(Liet al\.,[2023](https://arxiv.org/html/2605.31167#bib.bib41)\)for faithfulness and correctness evaluation with explicit hallucination labels\.Provider Factory Pattern\.Located in the Provider Factory component \(Figure[3](https://arxiv.org/html/2605.31167#S4.F3)\), all LLM interactions are mediated through an abstractLLMProviderbase class\. A factory map associates provider names \(OpenAI, Anthropic, Google Gemini, DeepSeek, Groq, Mistral, xAI, MiniMax, Devana, Z\.ai\) with constructor functions\. Each provider implementscall\(\)for text generation andgetAvailableModels\(\)for dynamic model listing\. These providers are only contacted when the user explicitly selects an LLM\-judge metric and supplies their own API key; deterministic metrics require no provider configuration\.Server\-side caching\.AModelsCachesingleton \(Figure[3](https://arxiv.org/html/2605.31167#S4.F3)\) with a configurable TTL \(default: 1 hour\) caches available model lists per provider, reducing redundant API calls and improving responsiveness\.

### 4\.5\.Statistical Benchmark Analysis Dashboard \[Process\]

Beyond per\-sample metric evaluation, LLM\-FACETS includes a multi\-tab analysis dashboard that derives statistical insights from complete benchmark runs \(Figure[4](https://arxiv.org/html/2605.31167#S4.F4)\)\. Results are persisted in the browser’s IndexedDB store and loaded directly into the analysis view without any server\-side storage, preserving the zero\-transmission guarantee\. Four analytical views are provided, addressing the accessibility \(C2\) and reproducibility \(C4\) challenges, with metrics filterable by category \(Lexical, Neural, RAG, Jury introduced in Section[5](https://arxiv.org/html/2605.31167#S5)\): A radar chart aggregates median values across nine primary metrics \(BLEU, ROUGE\-L, METEOR, BERTScore, Faithfulness, Answer Relevance, Context Relevance, RAG Triad composite, and Jury Score\), providing a compact multi\-dimensional quality signature for a benchmark run\. Box plots display the full quartile distribution \(Q1Q\_\{1\}, median,Q3Q\_\{3\}, whiskers atmin/max\\mathrm\{min\}/\\mathrm\{max\}, and a mean marker\) for each metric, exposing score spread and outliers that aggregate statistics conceal\. A dedicated hallucination histogram bins faithfulness scores into ten equal intervals and flags bins below 0\.7 as elevated\-risk distributions, providing at\-a\-glance triage of potential hallucination prevalence\. A Pearson correlation heatmap \(visible in the Correlation tab of the dashboard, not shown in Figure[4](https://arxiv.org/html/2605.31167#S4.F4)\) displays theN×NN\\times Ninter\-metric correlation matrix, computed as:

rX​Y=∑i\(xi−x¯\)​\(yi−y¯\)∑i\(xi−x¯\)2⋅∑i\(yi−y¯\)2r\_\{XY\}=\\frac\{\\displaystyle\\sum\_\{i\}\(x\_\{i\}\-\\bar\{x\}\)\(y\_\{i\}\-\\bar\{y\}\)\}\{\\sqrt\{\\displaystyle\\sum\_\{i\}\(x\_\{i\}\-\\bar\{x\}\)^\{2\}\\cdot\\displaystyle\\sum\_\{i\}\(y\_\{i\}\-\\bar\{y\}\)^\{2\}\}\}whererX​Yr\_\{XY\}is the correlation coefficient between two metricsXXandYY,xix\_\{i\}andyiy\_\{i\}are the individual scores for theii\-th sample, andx¯,y¯\\bar\{x\},\\bar\{y\}are the respective mean scores\. Cell color intensity encodes correlation strength \(blue for positive, red for negative\)\. Three pairwise RAG scatter plots \(Faithfulness vs\. Answer Relevance, Faithfulness vs\. Context Relevance, and Answer Relevance vs\. Context Relevance\) expose structural dependencies between the RAG Triad components\. A jury agreement histogram shows the distribution of inter\-judge consensus across samples, providing an aggregate view of evaluator stability\.Deep Dive\.A paginated per\-sample heatmap renders a color\-coded grid of all metric values across samples \(30 per page\), enabling rapid visual identification of underperforming individual outputs\. A BERTScore precision\-versus\-recall scatter plot, with points sized by F1 score, diagnoses whether semantic similarity shortfalls originate from precision deficits \(hallucinated content\) or recall deficits \(missing information\)\. An execution timeline tracks up to five metric trajectories across the sample execution order, exposing performance drift over the course of a benchmark run\.Multi\-Run Comparison\.A comparative radar chart and a grouped bar chart overlay up to five benchmark runs simultaneously on the same axes, enabling direct comparison of model versions, prompt strategies, or provider configurations evaluated on identical corpora\. This dashboard transforms raw benchmark data into auditable statistical evidence legible to non\-technical practitioners without recourse to scripting or external analytical tooling, operationalizing the process transparency requirement articulated in Section[2\.5](https://arxiv.org/html/2605.31167#S2.SS5)\.

![Refer to caption](https://arxiv.org/html/2605.31167v1/analyse.png)Figure 4\.Benchmark analysis dashboard \(Overview tab\): radar chart aggregating nine primary metrics, box plots showing per\-metric score distributions, and hallucination risk histogram\. All visualizations are generated client\-side from IndexDB\-stored results without any server\-side data transmission\.Screenshot of the benchmark analysis overview tab in LLM\-FACETS\. A radar chart in the top\-left displays nine metric axes\. Box plots on the right show score distributions for each metric with quartile markers\. A hallucination histogram at the bottom bins faithfulness scores and highlights bins below 0\.7 in red\.![Refer to caption](https://arxiv.org/html/2605.31167v1/dashboard.png)Figure 5\.Main dashboard showing the metrics selection grid\. Users can navigate between categories \(Traditional, Neural, LLM\-as\-a\-Judge, RAG\) and launch evaluations directly\.Screenshot of the LLM\-FACETS dashboard showing a grid of metric cards organized in four categories: Traditional \(BLEU, ROUGE, METEOR\), Neural \(BERTScore, LogProbs\), LLM\-as\-a\-Judge \(G\-Eval, Jury, Evaluation Topologies\), and RAG \(Faithfulness, Answer Relevance, Context Relevance\)\.![Refer to caption](https://arxiv.org/html/2605.31167v1/dataset.png)Figure 6\.Dataset explorer interface showing available datasets \(SQuAD v2, PsiloQA\), download status, split selection, and live row sampling with DuckDB results\.Screenshot of the dataset selector modal in LLM\-FACETS, showing two dataset cards for SQuAD v2 and PsiloQA with download progress indicators, split selectors, and a table of sampled rows\.

## 5\.Transparency Evaluation Metrics

The tool architecture described in Section[4](https://arxiv.org/html/2605.31167#S4)provides the infrastructure for privacy\-preserving, extensible evaluation\. This section details how the implemented metrics operationalize the four transparency dimensions identified in Section[2\.5](https://arxiv.org/html/2605.31167#S2.SS5)—baseline grounding, epistemic, factual, and process—each addressing a distinct practitioner question and directly responding to the transparency challenge \(C5\)\. Table[5](https://arxiv.org/html/2605.31167#S5.T5)provides a complete inventory of all implemented metrics with their categories, execution types, and transparency dimensions\. Table[2](https://arxiv.org/html/2605.31167#S5.T2)clarifies where each metric executes, making explicit the data sovereignty boundary \(C3\) for organizations operating under GDPR, HIPAA, or similar data protection regimes:Next\.js server\-localmeans the computation runs within the self\-hosted server process; no text or embeddings are transmitted to any external service\.

Table 2\.Execution location for each metric category\. “Next\.js server\-local” means the computation is confined to the self\-hosted server process\. Only Jury and RAG Triad metrics require outbound calls to external LLM APIs\.### 5\.1\.Auditing Hallucinations: The RAG Triad \[Factual\]

The RAG Triad \(Faithfulness, Answer Relevance, Context Relevance, detailed in Table[3](https://arxiv.org/html/2605.31167#S5.T3)\) implements factual transparency checks for RAG pipelines, operationalizing the three questions introduced in Section[2\.5](https://arxiv.org/html/2605.31167#S2.SS5):Is the answer grounded in the retrieved context? Does the answer address the query? Is the retrieved context relevant to the query?

Table 3\.RAG Triad ComponentsBy presenting all three scores simultaneously with per\-claim breakdowns \(Figure[8](https://arxiv.org/html/2605.31167#S7.F8)\), the interface allows a domain expert without programming skills to trace a hallucination back to its source in the retrieval pipeline, directly addressing the onboarding barrier \(C1\)\.

### 5\.2\.Visualizing Model Confidence: Token\-Level Log\-Probabilities \[Epistemic\]

LogProb visualization surfaces epistemic transparency\. For providers that return per\-token log\-probabilities \(OpenAI, DeepSeek, xAI\), LLM\-FACETS renders the generated text with each token color\-coded across five confidence tiers \(described in Table[4](https://arxiv.org/html/2605.31167#S5.T4)\), translating the mathematical concept of output probabilityp​\(ti∣t<i\)p\(t\_\{i\}\\mid t\_\{<i\}\)into an interface accessible to non\-technical auditors:

Table 4\.Confidence tiers and corresponding log\-probability thresholdsHovering over any token reveals its exact log\-probability and the top\-kkalternative tokens the model considered\. This visualization serves as an accessibility layer for uncertainty quantification: auditors who are not familiar with log\-probability mathematics can immediately identify when a model is “guessing” versus when it expresses strong confidence\. In combination with the RAG Triad, this allows correlation between low\-confidence tokens and unfaithful claims—a powerful diagnostic for systematic hallucination analysis\.

### 5\.3\.Mitigating Judge Bias: Multi\-Judge Consensus \[Process\]

The Jury module implements process transparency by exposing the unreliability of single\-judge LLM evaluation \(Figure[7](https://arxiv.org/html/2605.31167#S5.F7)\)\. A configurable set ofnnjudges \(potentially from different model providers and families\) independently score the same input\. The tool reports:

- •Individual judge scores and their reasoning chains\.
- •Consensus score \(mean or median, configurable\)\.
- •Inter\-judge agreement \(σ\\sigmaof scores\); high variance signals contested assessments\.
- •Statistical outlier detection to identify rogue judges\.

![Refer to caption](https://arxiv.org/html/2605.31167v1/llm-judge.png)Figure 7\.Jury module: three judges from different provider families \(OpenAI, Google, Alibaba\) independently score the same input using pointwise evaluation with median aggregation\. Individual scores, reasoning chains, inter\-judge agreement \(σ\\sigma\), and outlier flags are reported simultaneously\.Screenshot of the Jury evaluation interface in LLM\-FACETS\. Three judge panels show individual scores and reasoning from gpt\-4o\-mini, gemini\-flash, and qwen3\-32b\. A summary panel displays the median consensus score, standard deviation, and agreement index\.The Evaluation Topologies module extends this with pairwise comparative evaluation \(O​\(N2\)O\(N^\{2\}\)or tournament\-style\) where position\-bias mitigation is achieved by running each comparison in both orderings \(A vs\. B and B vs\. A\) and averaging\. This directly addresses the single largest known bias in LLM\-as\-a\-Judge evaluation\(Zhenget al\.,[2023](https://arxiv.org/html/2605.31167#bib.bib7)\)\. This multi\-judge design is empirically grounded in recent work\. The Panel of LLM Evaluators \(PoLL\)\(Vergaet al\.,[2024](https://arxiv.org/html/2605.31167#bib.bib35)\)demonstrates that a diverse jury of smaller, heterogeneous models outperforms a single frontier judge across multiple evaluation datasets while reducing cost by up to7×7\\times—a finding that validates the Jury module’s default configuration using models from different provider families\. At the aggregation level, CARE\(Zhaoet al\.,[2026](https://arxiv.org/html/2605.31167#bib.bib37)\)models inter\-judge correlations to isolate shared confounders—response length, stylistic similarity—from true quality signals, providing a statistically principled estimator\. LLM\-FACETS’ trimmed\-mean and weighted aggregation strategies address the same problem at lower computational cost, prioritizing deployment accessibility over statistical optimality while exposing inter\-judge variance as a first\-class reported statistic\.

### 5\.4\.Baseline Grounding: Traditional and Neural Metrics \[Baseline Grounding\]

Traditional n\-gram metrics \(BLEU\(Papineniet al\.,[2002](https://arxiv.org/html/2605.31167#bib.bib1)\), ROUGE\(Lin,[2004](https://arxiv.org/html/2605.31167#bib.bib2)\), METEOR\(Banerjee and Lavie,[2005](https://arxiv.org/html/2605.31167#bib.bib3)\)\) and neural similarity \(BERTScore\(Zhanget al\.,[2020](https://arxiv.org/html/2605.31167#bib.bib4)\)\) provide deterministic, reproducible baselines that anchor LLM\-FACETS in well\-understood, externally validated methods\. These baselines serve a specific role in baseline grounding transparency \(Section[2\.5](https://arxiv.org/html/2605.31167#S2.SS5)\): they provide deterministic, cross\-validated reference measurements that allow auditors to detect systematic divergences between traditional metric scores and LLM judge scores, which may indicate prompt engineering artifacts, model self\-bias, or genuine quality differentials that word\-overlap metrics cannot capture\. Table[5](https://arxiv.org/html/2605.31167#S5.T5)summarizes all implemented metrics across the four categories \(Traditional, Neural, LLM\-as\-a\-Judge, RAG\)\. Note that the 18 metricvariantsreported in Section[9](https://arxiv.org/html/2605.31167#S9)arise from multi\-variant metrics: ROUGE alone yields six variants \(ROUGE\-1, ROUGE\-2, ROUGE\-L, ROUGE\-W, ROUGE\-S, ROUGE\-SU\), and BERTScore yields three \(Precision, Recall, F1\)\.

Table 5\.Summary of implemented evaluation metrics in LLM\-FACETS\.

## 6\.Implementation Verification and Performance

Credible evaluation requires verified implementations\. The reproducibility challenge \(C4\) identified in Section[2](https://arxiv.org/html/2605.31167#S2)demands that metric implementations be cross\-validated against established reference libraries and that performance characteristics be documented\. This section provides that evidence\.

### 6\.1\.Cross\-Validation Against Reference Implementations

To ensure the reproducibility of our implementations, all metric implementations in LLM\-FACETS undergo rigorous cross\-validation against established Python reference libraries \(e\.g\., NLTK, Hugging Faceevaluate\)\. We executed a comprehensive test suite comprising 267 validation cases\. Deterministic metrics—including BLEU, ROUGE variants, METEOR, and BERTScore—passed all tests, matching their Python reference implementations with an absolute error below10−510^\{\-5\}\. Table[6](https://arxiv.org/html/2605.31167#S6.T6)provides evidence of a sample validation, and Table[7](https://arxiv.org/html/2605.31167#S6.T7)reports maximum absolute errors measured over 100 randomly sampled test cases from the SQuAD v2 validation set\. For non\-deterministic LLM\-based metrics \(RAG Triad, G\-Eval, Jury\), we evaluate score consistency by computing the standard deviation \(σ\\sigma\) across repeated runs \(n=5n=5\)\. In our validation suite using thegpt\-4o\-minimodel, all LLM\-based metrics maintainedσ<0\.05\\sigma<0\.05across evaluation runs, confirming sufficient score stability for production use\.

Table 6\.Validation of TypeScript BLEU implementation against Pythonnltk\.translate\.bleu\_scorereference on a standardized Wikipedia excerpt\.Table 7\.Cross\-validation of deterministic metric implementations against Python reference libraries\. Maximum absolute error over 100 test cases\.
### 6\.2\.Performance Analysis

A critical challenge in LLM evaluation is the computational overhead of metric calculation, particularly over large datasets\. To address transparency and streamline the process, the tool provides a comprehensive dashboard grid \(illustrated in Figure[5](https://arxiv.org/html/2605.31167#S4.F5)\) from which evaluations are launched\. To evaluate the computational efficiency of the LLM\-FACETS tool, we conducted a benchmark over the SelfAware\(Yinet al\.,[2023](https://arxiv.org/html/2605.31167#bib.bib40)\)\(1,047 samples\) and HaluEval\(Liet al\.,[2023](https://arxiv.org/html/2605.31167#bib.bib41)\)\(12,322 samples\) datasets on a high\-performance server \(Intel Xeon Platinum 8480\+ with 28 vCPUs, 251GB RAM, and a single NVIDIA H100 80GB GPU, running Ubuntu 22\.04 LTS\)\. Processing a batch of 13,369 samples from these datasets to compute generations and traditional metrics \(BLEU, ROUGE, METEOR, BERTScore\) took a total of 24\.7 seconds\. Out of this total, text generation using the Mistral\-Small\-3\.2\-24B\-Instruct model \(via vLLM\) accounted for 17\.9 seconds \(1\.3 ms/sample\), while the actual computation of the traditional metrics executed in merely 6\.7 seconds \(0\.5 ms/sample/metric\)\. This provides evidence of the efficiency of our in\-process metric computation engine\.

## 7\.Running Examples

### 7\.1\.Example 1: The Developer – RAG Pipeline Debugging

A machine learning engineer \(acting as a Technical Expert\) is developing a RAG pipeline for a legal document summarization system\. They suspect that the retriever is returning partially irrelevant context and that the generator occasionally introduces unsupported claims\. Using LLM\-FACETS, the engineer:

1. \(1\)Setup inputs: They submit a question, retrieved context passage, and generated answer to the RAG Triad interface\.
2. \(2\)Observe scores: They note that the Faithfulness score is 0\.6 and Context Relevance is 0\.4\. All scores are normalized to\[0,1\]\[0,1\], where 1 indicates the best result \(Section[4](https://arxiv.org/html/2605.31167#S4)\): a Faithfulness of 0\.6 means 3 of 5 generated claims are unsupported by the context, while a Context Relevance of 0\.4 indicates that the retrieval is noisy\.
3. \(3\)Analyze breakdowns: They use the per\-claim breakdown \(visible in Figure[8](https://arxiv.org/html/2605.31167#S7.F8)\) to identify which specific generated sentence introduces the hallucination\.
4. \(4\)Cross\-reference metrics: They correlate the finding with the LogProbs visualization, noticing the hallucinated phrase corresponds to low\-confidence tokens \(p<0\.4p<0\.4, orange tier\), confirming model uncertainty \(Figure[9](https://arxiv.org/html/2605.31167#S7.F9)\)\.
5. \(5\)Iterate: They iterate on the retrieval strategy and re\-evaluate without leaving the browser, repeating until Faithfulness exceeds the target threshold \(e\.g\., 0\.85\) or until unsupported claims are eliminated\.

![Refer to caption](https://arxiv.org/html/2605.31167v1/ragtriad.png)Figure 8\.RAG Triad evaluation interface showing a complete example: the user\-provided question, retrieved context, and generated answer, alongside the three computed scores\.Screenshot of the RAG Triad interface in LLM\-FACETS\. The left panel shows the input fields for question, context, and generated answer\. The right panel displays three circular score gauges for Faithfulness \(0\.6\), Answer Relevance \(0\.85\), and Context Relevance \(0\.4\), with a per\-claim breakdown below\.![Refer to caption](https://arxiv.org/html/2605.31167v1/logprobs.png)Figure 9\.Token\-level log\-probability visualization: a generated response with per\-token confidence color coding \(green: high, yellow: moderate, red: low\)\. Low\-confidence tokens indicate hallucination\-prone regions\.Screenshot of the LogProbs visualization interface\. A generated text response is shown with individual tokens highlighted in colors ranging from green to red based on their log\-probability scores\. A legend explains the color scale\.This workflow compresses a debugging loop that previously required scripting, library installation, and manual log analysis into a single browser session, reducing iteration time from hours to minutes\.

### 7\.2\.Example 2: The Compliance Officer – Auditing Without Coding

A legal compliance officer \(acting as a Compliance Officer\) at a healthcare organization needs to certify that an LLM\-based patient triage assistant does not fabricate medical information\. The officer has no programming background\. Following the workflow presented in Section[3\.3](https://arxiv.org/html/2605.31167#S3.SS3):

1. \(1\)Define transparency goal: Factual grounding \(are recommendations supported by clinical protocol?\)\.
2. \(2\)Select metrics: Faithfulness and Context Relevance \(RAG Triad\), targeting the factual dimension\.
3. \(3\)Load dataset: The officer uploads a local CSV of patient queries and model responses\. To preserve privacy, they leverage the tool’s zero\-transmission architecture \(C3\), which ensures data query operations are executed entirely in\-process via DuckDB while the API key remains stored strictly in the browser’s IndexedDB\.
4. \(4\)Cross\-check results: The Faithfulness score \(Figure[8](https://arxiv.org/html/2605.31167#S7.F8)\) reveals which triage recommendations are unsupported by the retrieved clinical protocol; the per\-claim breakdown identifies the specific sentences introducing unsupported claims\.
5. \(5\)Package evidence: The officer exports a structured JSON artifact containing per\-claim scores and reasoning chains for inclusion in the regulatory compliance report\.

Unlike the developer’s iterative debugging loop \(Example 1\), the compliance officer’s workflow is a single\-pass audit: the goal is to produce a complete evidence artifact certifying whether the system meets the required Faithfulness threshold, not to iteratively improve the system\. If the audit reveals unacceptable hallucination rates, the officer documents the finding and refers the system back to the development team for remediation\. In this scenario the compliance officer performs an independent, structured evaluation of the AI system without reliance on the development team, using the same methodology but without requiring engineering assistance\. This is the operational meaning of responsible AI governance: accountability that does not depend on the goodwill of the system’s developers\.

### 7\.3\.Example 3: The Domain Expert – Validating Specialized Knowledge

A clinical pharmacologist \(acting as a Domain Expert\) needs to evaluate whether a new open\-weight LLM can accurately answer complex drug\-interaction questions\. The pharmacologist has deep medical expertise but no coding skills\. After navigating to the LLM\-FACETS instance in their browser \(e\.g\.,http://localhost:3000\), the expert:

1. \(1\)Setup inputs: They load a tailored benchmark dataset of clinical questions and reference answers into the dataset explorer \(Figure[6](https://arxiv.org/html/2605.31167#S4.F6)\) via a simple file upload\.
2. \(2\)Select metrics: They choose G\-Eval \(with a custom rubric emphasizing medical accuracy\) and Jury to mitigate the evaluation biases of a single model\.
3. \(3\)Analyze outputs: They identify specific responses where the model diverges from the reference answers by sorting the evaluation results by their G\-Eval scores in the benchmark analysis dashboard \(Figure[4](https://arxiv.org/html/2605.31167#S4.F4)\)\.
4. \(4\)Cross\-check divergence: They check the Jury consensus \(Figure[7](https://arxiv.org/html/2605.31167#S5.F7)\)\. If judges strongly disagree on a score, the pharmacologist can manually review the reasoning chains to see if the model’s answer is a valid alternative or a hallucinated contradiction\.
5. \(5\)Communicate findings: They compile the identified failure cases into a structured report for the technical team, documenting the specific clinical edge cases where the LLM fails\. This step concludes the evaluation: unlike the developer’s iterative workflow, the domain expert’s output is a definitive assessment shared with the engineering team for action\.

Taken together, these cases illustrate that transparency is not a fixed property of an evaluation tool, but a relation that must be shaped to the specific epistemic needs and accountability roles of each practitioner\. The developer requires low\-level diagnostic precision; the compliance officer requires evidence portable into a regulatory report; the domain expert requires the ability to interrogate the semantic validity of the model\. The same framework serves all three because its mechanisms are oriented toward distinct purposes rather than toward a single technical artifact\. This workflow makes the framework’s methodological contribution explicit: it is not sufficient to provide tools; the framework must guide practitioners in applying those tools purposefully to produce evidence that is both scientifically credible and practically actionable\.

## 8\.Discussion

In this section, we reflect on the implications of LLM\-FACETS for the broader LLM evaluation ecosystem\. We discuss how its extensibility fosters community\-driven infrastructure, clarify its stance on data flow transparency, and connect its methodological approach to the goals of open science and responsible computing\.

### 8\.1\.Extensibility as Community Infrastructure

LLM\-FACETS is not an evaluation tool with a fixed metric set: it is a plugin infrastructure on which the research community can build\. The Metric Registry and Dataset Registry \(Section[4](https://arxiv.org/html/2605.31167#S4)\) mean that any metric proposed in a future paper can be integrated into LLM\-FACETS by implementing a single interface, without modifying the evaluation pipeline or the dashboards that consume it\. This architectural choice directly addresses the onboarding barrier \(C1\) and the reproducibility challenge \(C4\): it prevents every team from reimplementing the same evaluation scaffolding from scratch, and it enables cumulative progress—a new faithfulness metric published in 2027 can slot into the same workflow, visualization, and export format as the metrics described in this paper\. This extensibility is also what makes the framework genuinely useful for long\-term governance contexts\. AI regulation evolves: new transparency obligations will create new metric requirements\. A fixed tool becomes obsolete; a plugin architecture grows with the field\. The same property applies to data: as new benchmarks for bias, fairness, or multimodal evaluation emerge, the Dataset Registry provides an integration path without architectural overhaul\. We view the plugin architecture as the primary technical contribution of this work, distinct from the specific metrics currently implemented\. The current metric suite represents one concrete realization of the framework—a specific set of metrics, datasets, and providers assembled to address the transparency challenges identified in Section[2](https://arxiv.org/html/2605.31167#S2)\. The framework itself, however, provides the foundational components \(Metric Registry, Dataset Registry, Provider Factory, workflow engine\) into which future metric suites can be integrated—for instance, fairness auditing or multimodal evaluation—without modifying the evaluation pipeline, dashboards, or export format\.

### 8\.2\.Data Flow Transparency and the BYOK Distinction

LLM\-FACETS does not claim to eliminate data exposure from all LLM evaluation—doing so would be architecturally impossible for any tool that relies on proprietary external LLM\-as\-a\-Judge APIs\. However, it addresses the compliance challenge \(C3\) by completely structurally separating local computation from external API calls\. While other programmatic tools like Ragas also support Bring Your Own Key \(BYOK\) approaches, they operate within opaque data\-processing environments or rely on Python scripts that can easily blend local and remote processing without explicit boundaries\. In contrast, LLM\-FACETS enforces data sovereignty by design: deterministic metrics \(BLEU, ROUGE, METEOR, BERTScore\) run entirely in\-process on the self\-hosted Next\.js server\. No evaluation data leaves the practitioner’s infrastructure for these tasks\. For LLM\-judge metrics \(Jury, RAG Triad, G\-Eval, LogProbs\), practitioners supply their own API keys via the browser’s IndexedDB\. The tool acts exclusively as a stateless proxy; it does not log, store, or intercept the payload en route to the provider\. This guarantees that practitioners remain the sole data controllers, assuming full responsibility for their proprietary data interactions without an intermediary platform capturing their evaluation traces\.

### 8\.3\.Future Outlook and Open Science as Strategy

As the LLM evaluation toolscape matures, we anticipate a convergence between technical evaluation capabilities and automated compliance reporting\. LLM\-FACETS is designed to anticipate this shift by positioning multi\-stakeholder transparency \(C5\) not as a downstream reporting task, but as an integral part of the evaluation lifecycle\. The need for tools that decouple AI accountability from the development teams engineering those systems—addressing the accessibility challenge \(C2\)—will only grow as AI regulation enforcement deepens\(European Parliament and Council of the European Union,[2024](https://arxiv.org/html/2605.31167#bib.bib15)\)\. Open science remains a foundational strategy for this future\. Reproducibility \(C4\) ensures that published results—whether in scientific literature or regulatory audits—can be independently verified\. By open\-sourcing the exact methods for evaluating factual grounding, uncertainty, and model bias, LLM\-FACETS offers a transparent alternative to proprietary evaluation APIs whose criteria often remain opaque\. The codebase and benchmark datasets will be publicly available, inviting community scrutiny and enabling any team with the data \(and the API key\) to replicate the tests within their own infrastructure\.

### 8\.4\.Limitations

Several limitations warrant acknowledgment\. These concern the scope of the metric definitions, the maturity of the tool’s evaluation, and potential threats to the validity of the claims made in this paper\. They are organized by their context:\(Metric definition\) LLM\-as\-a\-Judge biases\.Despite the Jury module’s bias mitigation, LLM judges remain susceptible to systematic biases correlated with their training data and instruction tuning\. No amount of averaging across judges eliminates shared biases common to models of the same generation or provider\.\(Tool\) LogProb calibration\.Log\-probability scores are not universally calibrated across providers or quantization levels\. A low\-probability token in GPT\-4o does not mean the same thing as a low\-probability token in a smaller open\-weight model\. Cross\-model comparisons of LogProb\-based confidence should be made cautiously\.\(Process\) RAG Triad semantic drift\.The Faithfulness and Answer Relevance metrics depend on LLM\-generated intermediate outputs \(claim decomposition, reverse question generation\)\. Errors in these intermediate steps propagate to the final scores\. The tool mitigates this by displaying intermediate reasoning chains, but does not eliminate the risk\.\(Metric definition\) Metric coverage\.The current suite does not include bias detection metrics, fairness metrics, toxicity assessment, or multimodal evaluation\. These represent important dimensions of responsible AI that LLM\-FACETS does not yet address\.\(Validation\) Absence of formal stakeholder validation\.The practitioner\-centered design choices presented in this paper are grounded in prior HCI research\(Liaoet al\.,[2020](https://arxiv.org/html/2605.31167#bib.bib22); Ehsanet al\.,[2021](https://arxiv.org/html/2605.31167#bib.bib23)\)and illustrated through representative use cases, but have not yet been validated through formal user studies with domain experts or compliance officers\. Empirical evaluation of whether LLM\-FACETS genuinely enables independent oversight in practice remains an important direction for future work\.\(Tool\) LogProb availability\.The log\-probability visualization feature depends on commercial providers that expose per\-token log\-probabilities in their API responses\. Open\-weight models served locally via inference frameworks such as Ollama are not yet integrated, limiting the feature to providers that support it natively\. Future work will address this gap through local model integration\.\(Tool\) Limited performance evaluation\.The computational performance analysis presented in Section[6\.2](https://arxiv.org/html/2605.31167#S6.SS2)covers only deterministic metrics on a single hardware configuration\. Comprehensive benchmarking of LLM\-as\-a\-Judge metrics over massive datasets, as well as scalability analysis across larger distributed clusters, remains for future work\.

## 9\.Conclusion

Engineering transparency in artificial intelligence cannot remain an exclusive exercise for technical experts and software developers\. As AI systems permeate society, their evaluation must become a collaborative, multi\-stakeholder endeavor\. In this paper, we presented LLM\-FACETS, an open\-source framework that lowers the barrier to the auditing of LLMs\. By abstracting the complexities of diverse evaluation paradigms—from traditional n\-gram metrics to RAG\-specific hallucination detection and probabilistic confidence visualization—into an accessible web interface, the framework aims to enable domain experts and compliance officers to actively participate in AI validation alongside technical teams\. LLM\-FACETS’ plugin architecture ensures that the framework can grow with the field: any metric proposed in future research can be integrated by implementing a single TypeScript interface, without modifying the evaluation pipeline, dashboards, or export format\. Its architecture makes data flows explicit—deterministic metrics run within the self\-hosted server; LLM\-judge metrics contact external APIs transparently, with users retaining full control\. Validation confirms that this accessibility does not come at the expense of scientific rigor: deterministic implementations maintain numerical parity with canonical Python reference implementations across 18 metric variants \(Section[6\.1](https://arxiv.org/html/2605.31167#S6.SS1)\)\. Future work will focus on: \(1\) expanding alignment with the EU AI Act transparency requirements, including automated compliance reporting templates; \(2\) incorporating explainable AI \(XAI\) techniques to provide deeper insights into LLM\-as\-a\-Judge decision processes; \(3\) integrating local model inference for fully offline evaluation; \(4\) developing shared evaluation campaign infrastructure for coordinated multi\-institutional auditing; and \(5\) building a community metric registry enabling researchers to publish and share metric implementations as reusable plugins\. Ultimately, true transparency in AI requires both transparent models and transparent, accessible methods of evaluation\. LLM\-FACETS represents a practical step toward building that collaborative evaluation ecosystem\.

###### Acknowledgements\.

The authors thank the Scriptor Artis team for infrastructure support during development\. This work was conducted as part of a research collaboration between Scriptor Artis SAS and the Luxembourg Institute of Science and Technology \(LIST\), funded by Scriptor Artis\.

## Competing Interests

Tom Lucas and Barbara Delacroix are affiliated with Scriptor Artis SAS, which develops and maintains the Devana platform \(devana\.ai\); Devana is integrated as an optional LLM provider in LLM\-FACETS\. The evaluation framework and its methodology are independent of any commercial product\. The other authors declare no competing financial interests or personal relationships that could have influenced this work\.

## Funding

This research was funded by Scriptor Artis SAS \(France\) through a research collaboration agreement with the Luxembourg Institute of Science and Technology \(LIST\)\.

## CRediT Authorship Contribution Statement

Tom Lucas:Conceptualization, Methodology, Software, Validation, Writing – original draft\.Alessio Buscemi:Methodology, Validation, Writing – review & editing\.Alfredo Capozucca:Methodology, Writing – review & editing, Supervision\.German Castignani:Project administration, Writing – review & editing, Supervision\.Barbara Delacroix:Supervision\.

## Declaration of Generative AI Use

During the preparation of this work the authors used GitHub Copilot and Google Gemini to assist with code development and documentation\. After using these tools, the authors reviewed and edited the content as needed and take full responsibility for the content of the published article\.

## Data Availability

The source code and datasets have been anonymized for peer review and are provided as supplementary material\. Upon acceptance, the repository will be made publicly available under the MIT License with Commons Clause\.

## References

- Apache Software Foundation \(2013\)Apache parquet: columnar storage format\.Note:[https://parquet\.apache\.org](https://parquet.apache.org/)Cited by:[§4\.2](https://arxiv.org/html/2605.31167#S4.SS2.p1.1)\.
- Arize AI \(2024\)Phoenix: open\-source AI observability platform\.Note:\[software\]\.[https://phoenix\.arize\.com](https://phoenix.arize.com/)Cited by:[§1](https://arxiv.org/html/2605.31167#S1.p1.1)\.
- S\. Banerjee and A\. Lavie \(2005\)METEOR: an automatic metric for MT evaluation with improved correlation with human judgments\.InProceedings of the ACL Workshop on Intrinsic and Extrinsic Evaluation Measures for Machine Translation and/or Summarization,Ann Arbor, Michigan, USA,pp\. 65–72\.Cited by:[§5\.4](https://arxiv.org/html/2605.31167#S5.SS4.p1.1),[Table 5](https://arxiv.org/html/2605.31167#S5.T5.1.4.3.1)\.
- E\. M\. Bender, T\. Gebru, A\. McMillan\-Major, and S\. Shmitchell \(2021\)On the dangers of stochastic parrots: can language models be too big?\.InProceedings of the 2021 ACM Conference on Fairness, Accountability, and Transparency \(FAccT\),Virtual Event, Canada,pp\. 610–623\.External Links:[Document](https://dx.doi.org/10.1145/3442188.3445922)Cited by:[§2\.4](https://arxiv.org/html/2605.31167#S2.SS4.p1.1)\.
- Confident AI \(2024\)DeepEval: open\-source LLM evaluation framework\.Note:Version 1\.0 \[software\]\.[https://github\.com/confident\-ai/deepeval](https://github.com/confident-ai/deepeval)Cited by:[§1](https://arxiv.org/html/2605.31167#S1.p1.1),[§2\.1](https://arxiv.org/html/2605.31167#S2.SS1.p1.1)\.
- Credo AI \(2024\)Credo AI: AI governance platform\.Note:[https://www\.credo\.ai](https://www.credo.ai/)Cited by:[§2\.3](https://arxiv.org/html/2605.31167#S2.SS3.p1.1)\.
- U\. Ehsan, Q\. V\. Liao, M\. Muller, M\. O\. Riedl, and J\. D\. Weisz \(2021\)Expanding explainability: towards social transparency in AI systems\.InProceedings of the 2021 CHI Conference on Human Factors in Computing Systems,Yokohama, Japan,pp\. 1–19\.External Links:[Document](https://dx.doi.org/10.1145/3411764.3445188)Cited by:[§2\.2](https://arxiv.org/html/2605.31167#S2.SS2.p1.1),[§2\.5](https://arxiv.org/html/2605.31167#S2.SS5.p1.1),[§8\.4](https://arxiv.org/html/2605.31167#S8.SS4.p1.1)\.
- S\. Es, J\. James, L\. Espinosa\-Anke, and S\. Schockaert \(2024\)RAGAS: automated evaluation of retrieval augmented generation\.InProceedings of the 18th Conference of the European Chapter of the Association for Computational Linguistics \(EACL\),Malta,pp\. 150–158\.External Links:[Document](https://dx.doi.org/10.18653/v1/2024.eacl-demo.16)Cited by:[§1](https://arxiv.org/html/2605.31167#S1.p1.1),[§2\.1](https://arxiv.org/html/2605.31167#S2.SS1.p1.1),[§2\.5\.2](https://arxiv.org/html/2605.31167#S2.SS5.SSS2.p1.1),[Table 5](https://arxiv.org/html/2605.31167#S5.T5.1.10.9.1),[Table 5](https://arxiv.org/html/2605.31167#S5.T5.1.11.10.1),[Table 5](https://arxiv.org/html/2605.31167#S5.T5.1.12.11.1)\.
- European Parliament and Council of the European Union \(2016\)Regulation \(EU\) 2016/679 of the european parliament and of the council on the protection of natural persons with regard to the processing of personal data \(general data protection regulation\)\.Note:Official Journal of the European Union L 119/1\.[https://eur\-lex\.europa\.eu/legal\-content/EN/TXT/?uri=CELEX:32016R0679](https://eur-lex.europa.eu/legal-content/EN/TXT/?uri=CELEX:32016R0679)Cited by:[§1](https://arxiv.org/html/2605.31167#S1.p1.1),[§2\.3](https://arxiv.org/html/2605.31167#S2.SS3.p1.1)\.
- European Parliament and Council of the European Union \(2024\)Regulation \(EU\) 2024/1689 of the european parliament and of the council – artificial intelligence act\.Note:Official Journal of the European Union\.[https://eur\-lex\.europa\.eu/legal\-content/EN/TXT/?uri=CELEX:32024R1689](https://eur-lex.europa.eu/legal-content/EN/TXT/?uri=CELEX:32024R1689)Cited by:[§1](https://arxiv.org/html/2605.31167#S1.p1.1),[§2\.3](https://arxiv.org/html/2605.31167#S2.SS3.p1.1),[§3\.1](https://arxiv.org/html/2605.31167#S3.SS1.p2.1),[§8\.3](https://arxiv.org/html/2605.31167#S8.SS3.p1.1)\.
- Fiddler AI \(2024\)Fiddler AI: model performance management platform\.Note:[https://www\.fiddler\.ai](https://www.fiddler.ai/)Cited by:[§2\.3](https://arxiv.org/html/2605.31167#S2.SS3.p1.1)\.
- S\. Gehrmann, E\. Clark, and T\. Sellam \(2023\)Repairing the cracked foundation: a survey of obstacles in evaluation practices for generated text\.Journal of Artificial Intelligence Research77,pp\. 103–166\.External Links:[Document](https://dx.doi.org/10.1613/jair.1.13715)Cited by:[§2\.1](https://arxiv.org/html/2605.31167#S2.SS1.p1.1),[§2\.4](https://arxiv.org/html/2605.31167#S2.SS4.p1.1)\.
- Google DeepMind \(2024\)Gemini: a family of highly capable multimodal models\.Note:[https://deepmind\.google/technologies/gemini/](https://deepmind.google/technologies/gemini/)Cited by:[§4\.3](https://arxiv.org/html/2605.31167#S4.SS3.p1.2.3)\.
- International Organization for Standardization \(2023\)ISO/IEC 42001:2023 — information technology — artificial intelligence — management system\.Note:[https://www\.iso\.org/standard/81230\.html](https://www.iso.org/standard/81230.html)Cited by:[§2\.3](https://arxiv.org/html/2605.31167#S2.SS3.p1.1)\.
- M\. Kahng, I\. Tenney, M\. Neumann, J\. Wexler, F\. Viégas, and M\. Wattenberg \(2024\)LLM comparator: visual analytics for side\-by\-side evaluation of large language models\.InExtended Abstracts of the CHI Conference on Human Factors in Computing Systems,Honolulu, Hawaii, USA,pp\. 1–7\.External Links:[Document](https://dx.doi.org/10.1145/3613905.3650755)Cited by:[§1](https://arxiv.org/html/2605.31167#S1.p1.1),[§2\.1](https://arxiv.org/html/2605.31167#S2.SS1.p1.1)\.
- S\. Kelly \(2016\)Compromise: modest natural language processing for JavaScript\.Note:\[software\]\.[https://github\.com/spencermountain/compromise](https://github.com/spencermountain/compromise)Cited by:[§4\.3](https://arxiv.org/html/2605.31167#S4.SS3.p1.2.1)\.
- LangChain, Inc\. \(2023\)LangSmith: platform for building production\-grade LLM applications\.Note:[https://smith\.langchain\.com](https://smith.langchain.com/)Cited by:[§2\.1](https://arxiv.org/html/2605.31167#S2.SS1.p1.1)\.
- Langfuse \(2024\)Langfuse: open source LLM engineering platform\.Note:\[software\]\.[https://langfuse\.com](https://langfuse.com/)Cited by:[§1](https://arxiv.org/html/2605.31167#S1.p1.1)\.
- J\. Li, X\. Cheng, W\. X\. Zhao, J\. Nie, and J\. Wen \(2023\)HaluEval: a large\-scale hallucination evaluation benchmark for large language models\.InProceedings of the 2023 Conference on Empirical Methods in Natural Language Processing,Singapore,pp\. 6449–6464\.External Links:[Document](https://dx.doi.org/10.18653/v1/2023.emnlp-main.397)Cited by:[§2\.1](https://arxiv.org/html/2605.31167#S2.SS1.p1.1),[§4\.4](https://arxiv.org/html/2605.31167#S4.SS4.p1.1),[§6\.2](https://arxiv.org/html/2605.31167#S6.SS2.p1.1)\.
- Q\. V\. Liao, D\. Gruen, and S\. Miller \(2020\)Questioning the AI: informing design practices for explainable AI user experiences\.InProceedings of the 2020 CHI Conference on Human Factors in Computing Systems,Honolulu, Hawaii, USA,pp\. 1–15\.External Links:[Document](https://dx.doi.org/10.1145/3313831.3376590)Cited by:[§2\.2](https://arxiv.org/html/2605.31167#S2.SS2.p1.1),[§2\.5](https://arxiv.org/html/2605.31167#S2.SS5.p1.1),[§8\.4](https://arxiv.org/html/2605.31167#S8.SS4.p1.1)\.
- C\. Lin \(2004\)ROUGE: a package for automatic evaluation of summaries\.InText Summarization Branches Out,Barcelona, Spain,pp\. 74–81\.External Links:[Link](https://aclanthology.org/W04-1013)Cited by:[§5\.4](https://arxiv.org/html/2605.31167#S5.SS4.p1.1),[Table 5](https://arxiv.org/html/2605.31167#S5.T5.1.3.2.1)\.
- Y\. Liu, D\. Iter, Y\. Xu, S\. Wang, R\. Xu, and C\. Zhu \(2023\)G\-Eval: NLG evaluation using GPT\-4 with better human alignment\.InProceedings of the 2023 Conference on Empirical Methods in Natural Language Processing \(EMNLP\),Singapore,pp\. 2511–2522\.External Links:[Document](https://dx.doi.org/10.18653/v1/2023.emnlp-main.153)Cited by:[§2\.5\.3](https://arxiv.org/html/2605.31167#S2.SS5.SSS3.p1.4),[Table 5](https://arxiv.org/html/2605.31167#S5.T5.1.7.6.1)\.
- Microsoft Corporation \(2018\)Presidio: context\-aware, pluggable and customizable data protection and anonymization service for text and images\.Note:\[software\]\.[https://microsoft\.github\.io/presidio/](https://microsoft.github.io/presidio/)Cited by:[§4\.3](https://arxiv.org/html/2605.31167#S4.SS3.p1.2.2)\.
- National Institute of Standards and Technology \(2023\)Artificial intelligence risk management framework \(AI RMF 1\.0\)\.Technical reportTechnical ReportNIST AI 100\-1,National Institute of Standards and Technology,Gaithersburg, Maryland, USA\.External Links:[Document](https://dx.doi.org/10.6028/NIST.AI.100-1)Cited by:[§1](https://arxiv.org/html/2605.31167#S1.p1.1),[§2\.3](https://arxiv.org/html/2605.31167#S2.SS3.p1.1),[§3\.1](https://arxiv.org/html/2605.31167#S3.SS1.p2.1)\.
- K\. Papineni, S\. Roukos, T\. Ward, and W\. Zhu \(2002\)BLEU: a method for automatic evaluation of machine translation\.InProceedings of the 40th Annual Meeting of the Association for Computational Linguistics \(ACL\),Philadelphia, Pennsylvania, USA,pp\. 311–318\.External Links:[Document](https://dx.doi.org/10.3115/1073083.1073135)Cited by:[§5\.4](https://arxiv.org/html/2605.31167#S5.SS4.p1.1),[Table 5](https://arxiv.org/html/2605.31167#S5.T5.1.2.1.1)\.
- M\. Post \(2018\)A call for clarity in reporting BLEU scores\.InProceedings of the Third Conference on Machine Translation \(WMT\),Brussels, Belgium,pp\. 186–191\.External Links:[Document](https://dx.doi.org/10.18653/v1/W18-6319)Cited by:[§2\.4](https://arxiv.org/html/2605.31167#S2.SS4.p1.1)\.
- M\. Raasveldt and H\. Mühleisen \(2019\)DuckDB: an embeddable analytical database\.InProceedings of the 2019 ACM SIGMOD International Conference on Management of Data,Amsterdam, The Netherlands,pp\. 1981–1984\.External Links:[Document](https://dx.doi.org/10.1145/3299869.3320212)Cited by:[§4\.2](https://arxiv.org/html/2605.31167#S4.SS2.p1.1)\.
- P\. Rajpurkar, R\. Jia, and P\. Liang \(2018\)Know what you don’t know: unanswerable questions for SQuAD\.InProceedings of the 56th Annual Meeting of the Association for Computational Linguistics \(ACL\),Melbourne, Australia,pp\. 784–789\.External Links:[Document](https://dx.doi.org/10.18653/v1/P18-2124)Cited by:[§4\.4](https://arxiv.org/html/2605.31167#S4.SS4.p1.1)\.
- E\. Rykov, K\. Petrushina, M\. Savkin, V\. Olisov, A\. Vazhentsev, K\. Titova, A\. Panchenko, V\. Konovalov, and J\. Belikova \(2025\)When models lie, we learn: multilingual span\-level hallucination detection with PsiloQA\.Note:HuggingFace Datasets\.[https://huggingface\.co/datasets/s\-nlp/PsiloQA](https://huggingface.co/datasets/s-nlp/PsiloQA)External Links:[Document](https://dx.doi.org/10.48550/arXiv.2510.04849)Cited by:[§4\.4](https://arxiv.org/html/2605.31167#S4.SS4.p1.1)\.
- J\. Saad\-Falcon, O\. Khattab, C\. Potts, and M\. Zaharia \(2024\)ARES: an automated evaluation framework for retrieval\-augmented generation systems\.InProceedings of the 2024 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies \(NAACL\),Mexico City, Mexico,pp\. 338–354\.External Links:[Document](https://dx.doi.org/10.18653/v1/2024.naacl-long.20)Cited by:[§2\.1](https://arxiv.org/html/2605.31167#S2.SS1.p1.1)\.
- TruEra, Inc\. \(2024\)TruLens: evaluation and tracking for LLM experiments\.Note:\[software\]\.[https://github\.com/truera/trulens](https://github.com/truera/trulens)Cited by:[§2\.1](https://arxiv.org/html/2605.31167#S2.SS1.p1.1)\.
- I\. Tufanov, K\. Hambardzumyan, J\. Ferrando, and E\. Voita \(2024\)LM transparency tool: interactive tool for analyzing transformer language models\.InProceedings of the 62nd Annual Meeting of the Association for Computational Linguistics \(ACL\), Volume 3: System Demonstrations,Bangkok, Thailand,pp\. 29–41\.External Links:[Document](https://dx.doi.org/10.18653/v1/2024.acl-demos.6)Cited by:[§2\.5\.1](https://arxiv.org/html/2605.31167#S2.SS5.SSS1.p1.3)\.
- P\. Verga, S\. Hofstatter, S\. Althammer, Y\. Su, A\. Piktus, A\. Arkhangorodsky, M\. Xu, N\. White, and P\. Lewis \(2024\)Replacing judges with juries: evaluating LLM generations with a panel of diverse models\.Note:arXiv preprint arXiv:2404\.18796External Links:[Document](https://dx.doi.org/10.48550/arXiv.2404.18796)Cited by:[§5\.3](https://arxiv.org/html/2605.31167#S5.SS3.p2.2)\.
- Xenova \(2024\)Transformers\.js: state\-of\-the\-art machine learning for the web\.Note:Version 2\.17\.2 \[software\]\.[https://github\.com/xenova/transformers\.js](https://github.com/xenova/transformers.js)Cited by:[§4\.2](https://arxiv.org/html/2605.31167#S4.SS2.p1.1)\.
- Z\. Yin, Q\. Sun, Q\. Guo, J\. Wu, X\. Qiu, and X\. Huang \(2023\)Do large language models know what they don’t know?\.InFindings of the Association for Computational Linguistics: ACL 2023,Toronto, Canada,pp\. 8653–8665\.External Links:[Document](https://dx.doi.org/10.18653/v1/2023.findings-acl.551)Cited by:[§4\.4](https://arxiv.org/html/2605.31167#S4.SS4.p1.1),[§6\.2](https://arxiv.org/html/2605.31167#S6.SS2.p1.1)\.
- T\. Zhang, V\. Kishore, F\. Wu, K\. Q\. Weinberger, and Y\. Artzi \(2020\)BERTScore: evaluating text generation with BERT\.InInternational Conference on Learning Representations \(ICLR\),Addis Ababa, Ethiopia,pp\. 1–15\.External Links:[Link](https://openreview.net/forum?id=SkeHuCVFDr)Cited by:[§4\.2](https://arxiv.org/html/2605.31167#S4.SS2.p1.1),[§5\.4](https://arxiv.org/html/2605.31167#S5.SS4.p1.1),[Table 5](https://arxiv.org/html/2605.31167#S5.T5.1.5.4.1)\.
- J\. Zhao, C\. Shin, T\. Huang, S\. S\. S\. N\. GNVV, and F\. Sala \(2026\)CARE: confounder\-aware aggregation for reliable LLM evaluation\.Note:arXiv preprint arXiv:2603\.00039External Links:[Document](https://dx.doi.org/10.48550/arXiv.2603.00039)Cited by:[§5\.3](https://arxiv.org/html/2605.31167#S5.SS3.p2.2)\.
- L\. Zheng, W\. Chiang, Y\. Sheng, S\. Zhuang, Z\. Wu, Y\. Zhuang, Z\. Lin, Z\. Li, D\. Li, E\. P\. Xing, H\. Zhang, J\. E\. Gonzalez, and I\. Stoica \(2023\)Judging LLM\-as\-a\-Judge with MT\-Bench and chatbot arena\.InAdvances in Neural Information Processing Systems \(NeurIPS\),New Orleans, Louisiana, USA,pp\. 46595–46623\.External Links:[Link](https://proceedings.neurips.cc/paper_files/paper/2023/hash/91f18a1287b398d378ef22505ea41832-Abstract-Datasets-and-Benchmarks.html)Cited by:[§2\.1](https://arxiv.org/html/2605.31167#S2.SS1.p1.1),[§2\.4](https://arxiv.org/html/2605.31167#S2.SS4.p1.1),[§2\.5\.3](https://arxiv.org/html/2605.31167#S2.SS5.SSS3.p1.4),[§5\.3](https://arxiv.org/html/2605.31167#S5.SS3.p2.2)\.