G-MARK: Grounded Multi-Agent Reasoning for Cooperative Driving via Knowledge Graphs

# G-MARK: Grounded Multi-Agent Reasoning for Cooperative Driving via Knowledge Graphs
Source: [https://arxiv.org/html/2608.19964](https://arxiv.org/html/2608.19964)
Bhavya Gupta1, Onat Gungor2, Tajana Rosing1Affiliation:1University of California, San Diego, CA, USA 2West Virginia University, Morgantown, WV, USA \{b5gupta, tajana\}@ucsd\.edu, onat\.gungor@wvu\.edu

###### Abstract

Autonomous driving systems must operate under partial observability, where safety\-critical objects may be occluded or visible only to neighboring connected vehicles\. Vehicle\-to\-vehicle cooperation can reduce this uncertainty, but existing cooperative driving methods often compress multi\-agent evidence into latent features or hidden multimodal states\. As a result, they obscure which agent observed each object, whether the object is visible to the ego vehicle, and how conflicting evidence affects downstream decisions\. We proposeG\-MARK, a grounded multi\-agent reasoning framework that converts cooperative object\-centric observations into explicit provenance\-aware knowledge graphs \(KGs\)\. The resulting KGs preserve object hypotheses together with their source attribution, ego\-versus\-partner visibility, uncertainty, conflicts, spatial relations, and planning\-relevant context\.G\-MARKthen derives a shared feature representation from these KGs, enabling lightweight task heads to support object reasoning, motion prediction, control selection, and trajectory forecasting\. Compared with the state\-of\-the\-art baseline,G\-MARKimproves occlusion reasoning accuracy by 42\.2%, reduces control\-selection error by 13\.1%, and achieves comparable trajectory\-planning accuracy with a 25\.6×\\timessmaller structured communication payload\. Our code is available at[this repository](https://github.com/bhavyagupta98/g-mark)\.

###### Index Terms:

Cooperative Driving, Multi\-Agent Knowledge Graphs, Vehicle\-to\-Vehicle Communication, Motion Planning

## IIntroduction

Autonomous driving systems must make safety\-critical decisions from incomplete and viewpoint\-dependent observations\[[25](https://arxiv.org/html/2608.19964#bib.bib19)\]\. Objects that are crucial for planning may be occluded, outside the ego vehicle’s field of view, or observable only by a connected partner vehicle\[[24](https://arxiv.org/html/2608.19964#bib.bib6)\]\. Vehicle\-to\-vehicle cooperation can mitigate this partial observability by sharing complementary scene evidence across agents\[[19](https://arxiv.org/html/2608.19964#bib.bib5)\]\. However, when the ego vehicle must reason about an object it has not directly observed, the system should retain more than the object hypothesis itself\. It should also preserve its provenance: which agent observed the object, how reliable the observation is, and why the object matters for downstream planning\[[9](https://arxiv.org/html/2608.19964#bib.bib4)\]\.

Existing cooperative perception methods improve autonomous driving perception through early fusion, late fusion, and feature\-level fusion\[[7](https://arxiv.org/html/2608.19964#bib.bib13),[17](https://arxiv.org/html/2608.19964#bib.bib9),[18](https://arxiv.org/html/2608.19964#bib.bib8)\]\. These methods primarily aim to aggregate multi\-agent observations into more accurate perception outputs, often by compressing shared evidence into latent feature maps or final detections\. However, this compression can discard the evidence structure needed for downstream reasoning\. Hence, downstream modules cannot infer which agent supported an object hypothesis, whether the ego vehicle directly observed the object, or how much uncertainty remains in the object estimate\.

Recent language\-based and vision\-language driving systems study high\-level driving reasoning through visual question answering, explanation generation, behavior planning, and language\-conditioned control\[[13](https://arxiv.org/html/2608.19964#bib.bib10),[20](https://arxiv.org/html/2608.19964#bib.bib16),[12](https://arxiv.org/html/2608.19964#bib.bib11),[14](https://arxiv.org/html/2608.19964#bib.bib12)\]\. Cooperative driving systems such as V2V\-LLM\[[3](https://arxiv.org/html/2608.19964#bib.bib3)\]and V2V\-GoT\[[2](https://arxiv.org/html/2608.19964#bib.bib1)\]extend this paradigm to multi\-agent reasoning\. Yet, their intermediate states are often encoded as prompts, generated text, free\-form rationales, or hidden multimodal features rather than structured object\-level cooperative evidence\. This limits traceability for safety\-critical decisions that depend on objects outside the ego vehicle’s direct view but observed by a connected partner\.

TABLE I:Comparison of cooperative evidence exposed by related method families\.CapabilityCooperative Perception\[[17](https://arxiv.org/html/2608.19964#bib.bib9)\]/ V2X Fusion\[[18](https://arxiv.org/html/2608.19964#bib.bib8)\]Language / VLM Driving Reasoning\[[13](https://arxiv.org/html/2608.19964#bib.bib10)\]Structured Scene and KG\-based Reasoning\[[15](https://arxiv.org/html/2608.19964#bib.bib2)\]Cooperative Language\-Based Reasoning\[[2](https://arxiv.org/html/2608.19964#bib.bib1)\]G\-MARK\(Ours\)Multi\-agent evidence✓△\\triangle—✓✓Object provenance——△\\triangle△\\triangle✓Ego\-versus\-partner visibility—△\\triangle—△\\triangle✓Uncertainty / conflict—△\\triangle△\\triangle△\\triangle✓Planning relevance△\\triangle✓△\\triangle✓✓Reusable across tasks—✓△\\triangle✓✓Comm\. efficiency△\\triangle△\\triangle—△\\triangle✓
✓Explicitly modeled

△\\trianglePartially exposed

—Not explicitly exposed

We proposeG\-MARK, a grounded multi\-agent reasoning framework that represents cooperative driving scenes as provenance\-aware knowledge graphs\. Operating above the perception layer,G\-MARKconsumes processed multi\-agent observations, associates compatible observations into shared object hypotheses, and preserves weak or unmatched observations as uncertain candidates rather than discarding them\. The resulting graph explicitly stores source\-agent support, ego\-versus\-partner visibility, uncertainty, cross\-agent disagreement, spatial relations, motion cues, and planning context\. KG\-based task heads then query this shared KG to support object grounding, occlusion reasoning, hidden\-object discovery, motion prediction, control selection, and trajectory forecasting\.

We evaluateG\-MARKon V2V\-GoT\-QA, a cooperative driving benchmark spanning perception, prediction, and planning tasks\. Compared with V2V\-GoT\[[2](https://arxiv.org/html/2608.19964#bib.bib1)\],G\-MARKimproves occlusion reasoning by42\.2%42\.2\\%, hidden\-object discovery by12\.3%12\.3\\%, and control\-selection error by13\.1%13\.1\\%\. For future trajectory forecasting,G\-MARKreaches near\-parity in average L2 error while using approximately25\.6×25\.6\\timeslower structured communication\. Our contributions are summarized as follows:

- •We introduce a provenance\-aware cooperative KG that preserves object hypotheses, source\-agent support, visibility, uncertainty, disagreement, spatial relations, motion cues, and planning context\.
- •We propose task\-conditioned delayed evidence fusion, which retains weak partner\-only observations so downstream tasks can decide which evidence is relevant\.
- •We show that our approach can support object reasoning, motion prediction, control selection, and future trajectory forecasting with compact structured communication\.

## IIRelated Work

Prior work relevant toG\-MARKspans cooperative perception and V2X fusion\[[7](https://arxiv.org/html/2608.19964#bib.bib13),[18](https://arxiv.org/html/2608.19964#bib.bib8),[17](https://arxiv.org/html/2608.19964#bib.bib9)\], language\- and VLM\-based driving reasoning\[[13](https://arxiv.org/html/2608.19964#bib.bib10),[14](https://arxiv.org/html/2608.19964#bib.bib12),[12](https://arxiv.org/html/2608.19964#bib.bib11),[20](https://arxiv.org/html/2608.19964#bib.bib16)\], structured scene and KG\-based reasoning\[[6](https://arxiv.org/html/2608.19964#bib.bib22),[15](https://arxiv.org/html/2608.19964#bib.bib2)\], and cooperative language\-based reasoning\[[3](https://arxiv.org/html/2608.19964#bib.bib3),[2](https://arxiv.org/html/2608.19964#bib.bib1),[4](https://arxiv.org/html/2608.19964#bib.bib14),[11](https://arxiv.org/html/2608.19964#bib.bib15)\]\.G\-MARKtargets the gap between these research families by constructing an explicit cooperative evidence representation from perception outputs\. This representation preserves source support, ego\-versus\-partner visibility, uncertainty, disagreement, and planning context before downstream task inference\. These properties are important because cooperative driving decisions depend not only on which objects are present, but also on who observed them, whether they are visible to the ego vehicle, and why they affect planning\[[9](https://arxiv.org/html/2608.19964#bib.bib4)\]\. Table[I](https://arxiv.org/html/2608.19964#S1.T1)compares how existing method families expose these evidence variables for downstream reasoning\.

Cooperative Perception and V2X Fusion\.Cooperative perception improves scene understanding beyond a single ego viewpoint by sharing raw observations, object predictions, learned features, BEV representations, or sparse spatial messages across agents\. Representative systems include Where2Comm\[[7](https://arxiv.org/html/2608.19964#bib.bib13)\], V2X\-ViT\[[18](https://arxiv.org/html/2608.19964#bib.bib8)\], and CoBEVT\[[17](https://arxiv.org/html/2608.19964#bib.bib9)\]\. These methods provide strong perception backbones, but their fused outputs rarely expose source support, ego\-versus\-partner visibility, uncertainty, or disagreement as reusable object\-level evidence\.

Language and VLM\-based Driving Reasoning\.Language, vision\-language, and multimodal large language models have recently been used for driving question answering, explanation, planning, and control\[[13](https://arxiv.org/html/2608.19964#bib.bib10),[20](https://arxiv.org/html/2608.19964#bib.bib16),[12](https://arxiv.org/html/2608.19964#bib.bib11),[14](https://arxiv.org/html/2608.19964#bib.bib12)\]\. These methods broaden autonomous\-driving evaluation from perception accuracy to reasoning\-oriented tasks, including object grounding, scene understanding, decision explanation, and planning\. However, most are ego\-centric or non\-cooperative, motivating a layer that preserves source\-agent support and cross\-agent disagreement\.

Structured Scene and KG\-based Reasoning\.Structured scene representations and knowledge graphs encode entities, attributes, and relations in an explicit form for visual reasoning and decision making\[[6](https://arxiv.org/html/2608.19964#bib.bib22)\]\. In autonomous driving, systems such as DriveLM\[[13](https://arxiv.org/html/2608.19964#bib.bib10)\]and KLDrive\[[15](https://arxiv.org/html/2608.19964#bib.bib2)\]show that structured scene facts can support interpretable, planning\-oriented reasoning\. However, existing driving KG approaches primarily focus on ego\-centric scenes, language\-grounded QA, or high\-level facts rather than cooperative V2V evidence\.G\-MARKextends this direction with a per\-scene cooperative KG that stores object hypotheses, source\-agent support, visibility, uncertainty, disagreement, and planning relevance\.

Cooperative Language\-Based Reasoning\.Recent cooperative language\-based methods extend language reasoning to multi\-agent driving\. LangCoop uses language for compact collaborative communication\[[4](https://arxiv.org/html/2608.19964#bib.bib14)\], CoLMDriver studies LLM\-based negotiation\[[11](https://arxiv.org/html/2608.19964#bib.bib15)\], V2V\-LLM formulates vehicle\-to\-vehicle reasoning as multimodal QA\[[3](https://arxiv.org/html/2608.19964#bib.bib3)\], and V2V\-GoT organizes perception, prediction, and planning questions through graph\-of\-thought reasoning\[[2](https://arxiv.org/html/2608.19964#bib.bib1)\]\. These systems are closest toG\-MARKin task scope, but their intermediate states are typically language messages, generated answers, reasoning traces, or hidden multimodal activations rather than explicit object\-level evidence\.G\-MARKinstead uses a provenance\-aware KG, allowing downstream predictions to be traced to source agents, visibility facts, uncertainty, and supporting observations\.

## IIIG\-MARK Framework

![Refer to caption](https://arxiv.org/html/2608.19964v1/gmark_arch.png)Fig\. 1:Overview of theG\-MARKcooperative KG reasoning framework\.Fig\.[1](https://arxiv.org/html/2608.19964#S3.F1)provides an overview ofG\-MARK, a cooperative reasoning layer that operates on top of a cooperative perception pipeline\. Given processed multi\-agent scene artifacts, including object detections or tracks, agent poses, confidence scores, and planning context,G\-MARKconstructs a provenance\-aware KG for downstream reasoning\. Importantly,G\-MARKdoes not consume raw camera or LiDAR streams directly; instead, it abstracts already\-processed cooperative perception evidence into a structured graph representation\. The central design principle ofG\-MARKis to postpone irreversible fusion until the downstream reasoning task is specified\. Rather than merging all agent observations into a single fused object list,G\-MARKrepresents observing agents, local observations, and object hypotheses as linked graph entities\. This structure allows different reasoning heads, such as occlusion reasoning and motion prediction, to query the same cooperative evidence while emphasizing different task\-relevant factors\.

G\-MARKproceeds in four stages\. First, each CAV’s processed perception output is instantiated as a local evidence KG containing an agent node, observation nodes, and object\-hypothesis nodes before cross\-agent association\. Second, ego\-side conservative association links compatible local evidence across CAVs into shared object hypotheses, attaches observation nodes as support, and retains weak unmatched candidates\. Third, each hypothesis is enriched with provenance, agent\-relative visibility, uncertainty, disagreement, spatial relations, motion cues, and planning relevance\. Finally, KG\-based task heads query the enriched cooperative KG for object\-level reasoning, motion prediction, and trajectory planning\.

### III\-ACooperative KG Generation

G\-MARKrepresents each CAV’s processed perception output, including local observations and V2V\-communicated object hypotheses, as local KG evidence and assembles it into an initial cooperative KG, instead of collapsing it into a flat list of fused objects\[[6](https://arxiv.org/html/2608.19964#bib.bib22),[15](https://arxiv.org/html/2608.19964#bib.bib2)\]\. A flat object list can indicate that an object exists, but it does not preserve who observed it, whether it is ego\-visible or partner\-only, how reliable the supporting evidence is, or whether the object should influence planning\. These factors are especially important under occlusion and partial observability, where safety\-relevant evidence may be available only through another connected agent\[[16](https://arxiv.org/html/2608.19964#bib.bib7),[10](https://arxiv.org/html/2608.19964#bib.bib23),[18](https://arxiv.org/html/2608.19964#bib.bib8)\]\.

To preserve the evidence structure behind cooperative perception outputs, each local evidence graph separatesagent nodes,observation nodes, andobject\-hypothesis nodes\. Anagent nodestores the identity, pose, timestamp, and ego\-versus\-partner role of the CAV that produced the local evidence\. Anobservation noderepresents one agent\-local object observationot,j\(i\)o^\{\(i\)\}\_\{t,j\}, while anobject\-hypothesis noderepresents an object candidate from communicated CAV evidence or a shared hypothesis formed after cross\-agent association\. This separation preserves the source vehicle, supporting observation, and ego\-versus\-partner visibility of each hypothesis\. For example, if a partner observes a pedestrian hidden from the ego vehicle, the agent node identifies the partner, the observation node stores the local detection, and the hypothesis node represents the shared belief that the pedestrian exists\.

Let𝒜t=\{a1,…,aNt\}\\mathcal\{A\}\_\{t\}=\\\{a\_\{1\},\\ldots,a\_\{N\_\{t\}\}\\\}denote the set of connected agents available at timestamptt, whereNtN\_\{t\}is the number of agents at that timestamp\. Each agentai∈𝒜ta\_\{i\}\\in\\mathcal\{A\}\_\{t\}reports a set ofMi,tM\_\{i,t\}local object observations:

𝒪t\(i\)=\{ot,1\(i\),…,ot,Mi,t\(i\)\}\.\\mathcal\{O\}^\{\(i\)\}\_\{t\}=\\\{o^\{\(i\)\}\_\{t,1\},\\ldots,o^\{\(i\)\}\_\{t,M\_\{i,t\}\}\\\}\.Here,Mi,tM\_\{i,t\}is the number of local observations from agentaia\_\{i\}at timett, andj=1,…,Mi,tj=1,\\ldots,M\_\{i,t\}indexes them\. Each observation is represented as

ot,j\(i\)=\(𝐱t,j\(i\),τt,j\(i\),ct,j\(i\),ai,t,𝜼t,j\(i\)\),o^\{\(i\)\}\_\{t,j\}=\\left\(\\mathbf\{x\}^\{\(i\)\}\_\{t,j\},\\tau^\{\(i\)\}\_\{t,j\},c^\{\(i\)\}\_\{t,j\},a\_\{i\},t,\\boldsymbol\{\\eta\}^\{\(i\)\}\_\{t,j\}\\right\),wherejjdenotes an individual observation,𝐱t,j\(i\)\\mathbf\{x\}^\{\(i\)\}\_\{t,j\}is the object position or bounding box in a shared coordinate frame,τt,j\(i\)\\tau^\{\(i\)\}\_\{t,j\}is the semantic object type, such as car, truck, or pedestrian,ct,j\(i\)c^\{\(i\)\}\_\{t,j\}is the detection confidence, andaia\_\{i\}records the source agent\. The auxiliary vector𝜼t,j\(i\)\\boldsymbol\{\\eta\}^\{\(i\)\}\_\{t,j\}stores optional observation\-level attributes, such as velocity, track identity, source flags, or visibility labels\. Graph\-level quantities such as hypothesis\-level provenance, uncertainty, disagreement, and planning relevance are finalized after association\. Each local observationot,j\(i\)o^\{\(i\)\}\_\{t,j\}is instantiated as an observation nodevt,i,jO∈𝒱tOv^\{O\}\_\{t,i,j\}\\in\\mathcal\{V\}^\{O\}\_\{t\}\. One agent nodevt,iA∈𝒱tAv^\{A\}\_\{t,i\}\\in\\mathcal\{V\}^\{A\}\_\{t\}is created for each connected vehicleai∈𝒜ta\_\{i\}\\in\\mathcal\{A\}\_\{t\}\. Each observation nodevt,i,jOv^\{O\}\_\{t,i,j\}is linked to the agent nodevt,iAv^\{A\}\_\{t,i\}that generated it, making the source of every observation explicit in the KG\.

The CAV\-local evidence is then assembled into an initial cooperative KG:

𝒢t=\(𝒱t,ℰt\),\\mathcal\{G\}\_\{t\}=\(\\mathcal\{V\}\_\{t\},\\mathcal\{E\}\_\{t\}\),with node set

𝒱t=𝒱tA∪𝒱tO∪𝒱tH\.\\mathcal\{V\}\_\{t\}=\\mathcal\{V\}^\{A\}\_\{t\}\\cup\\mathcal\{V\}^\{O\}\_\{t\}\\cup\\mathcal\{V\}^\{H\}\_\{t\}\.Here,𝒱tA\\mathcal\{V\}^\{A\}\_\{t\},𝒱tO\\mathcal\{V\}^\{O\}\_\{t\}, and𝒱tH\\mathcal\{V\}^\{H\}\_\{t\}denote agent nodes, observation nodes, and object\-hypothesI removedis nodes, respectively\.

The relation setℰt\\mathcal\{E\}\_\{t\}uses typed relations for source attribution, hypothesis support, spatial context, planning context, and evidence consistency\. The source relation connects an agent node to each observation node it produces, while the support relation connects an observation node to the object\-hypothesis node it contributes to\. Spatial relations, such asfront\-of,behind,left\-of,right\-of, andnear, are computed in the ego coordinate frame\. Planning\-context relations, such asnear\-trajectoryandpath\-relevant, use distances to the ego vehicle’s planned or predicted trajectory\. Evidence\-consistency relations includecooperatively\-supported, indicating support from multiple agents or from a partner agent, andlow\-conflict, indicating geometrically consistent support\. This typed relation design follows standard entity–relation KG representations\[[6](https://arxiv.org/html/2608.19964#bib.bib22)\], while provenance is stored explicitly through the support set and provenance record\[[5](https://arxiv.org/html/2608.19964#bib.bib24)\]\.

An object\-hypothesis nodeh∈𝒱tHh\\in\\mathcal\{V\}^\{H\}\_\{t\}is associated with a support set of compatible observation nodes,

𝒮⁡\(h\)⊆𝒱tO\.\\mathcal\{S\}\(h\)\\subseteq\\mathcal\{V\}^\{O\}\_\{t\}\.Thus,𝒮⁡\(h\)\\mathcal\{S\}\(h\)contains observation nodes, each corresponding to an original local observationot,j\(i\)o^\{\(i\)\}\_\{t,j\}\. It is represented as

h=\(𝐱h,τh,ch,πh,𝒮⁡\(h\),uh,γh,rh\),h=\(\\mathbf\{x\}\_\{h\},\\tau\_\{h\},c\_\{h\},\\pi\_\{h\},\\mathcal\{S\}\(h\),u\_\{h\},\\gamma\_\{h\},r\_\{h\}\),where\(𝐱h,τh,ch\)\(\\mathbf\{x\}\_\{h\},\\tau\_\{h\},c\_\{h\}\)denote fused location, semantic type, and confidence;\(πh,𝒮⁡\(h\)\)\(\\pi\_\{h\},\\mathcal\{S\}\(h\)\)store supporting agents and observations; and\(uh,γh,rh\)\(u\_\{h\},\\gamma\_\{h\},r\_\{h\}\)capture uncertainty, disagreement, and planning relevance\.

The provenance record is

πh=\{\(ai,vt,i,jO\):vt,i,jO∈𝒮\(h\),vt,i,jOis produced byai\}\.\\pi\_\{h\}=\\\{\(a\_\{i\},v^\{O\}\_\{t,i,j\}\):v^\{O\}\_\{t,i,j\}\\in\\mathcal\{S\}\(h\),\\;v^\{O\}\_\{t,i,j\}\\text\{ is produced by \}a\_\{i\}\\\}\.This record keeps each object hypothesis traceable to its supporting agents and observations\[[5](https://arxiv.org/html/2608.19964#bib.bib24)\]\. The KG exposes cooperative evidence variables that are often lost after immediate fusion\. Source support is computed from the number and identity of agents inπh\\pi\_\{h\}\. Ego\-relative visibility is represented as an agent\-specific relation,

vis⁡\(h,ai\)∈\{visible,occluded,uncertain\},\\mathrm\{vis\}\(h,a\_\{i\}\)\\in\\\{\\mathrm\{visible\},\\mathrm\{occluded\},\\mathrm\{uncertain\}\\\},rather than a global label\. Downstream heads use this relation to distinguish ego\-visible hypotheses from partner\-only or occluded hypotheses, especially for invisible\-object retrieval, occlusion reasoning, and planning\-aware object selection\. Uncertaintyuhu\_\{h\}captures weak confidence, candidate status, and limited support; disagreementγh\\gamma\_\{h\}captures inconsistent observations; and planning relevancerhr\_\{h\}measures proximity to the ego vehicle’s future path\.

The output of this stage is an initial cooperative KG𝒢t\\mathcal\{G\}\_\{t\}containing agent nodes, observation nodes, object\-hypothesis nodes, typed relations, and traceable evidence variables\. The following stages resolve compatible candidate hypotheses through conservative association and enrich each hypothesis with provenance, visibility, uncertainty, disagreement, and planning relevance\.

### III\-BConservative Association

After local evidence is assembled into the cooperative KG,G\-MARKassociates compatible candidate hypotheses across CAVs into shared cooperative object hypotheses\. Observation nodes are attached as hypothesis support, while unmatched observations are retained as weak candidates\. This avoids over\-merging nearby but distinct objects while preserving partner\-only evidence for occlusion and planning\.

Following standard tracking and data\-association practice, we use a deterministic gating rule to decide which local evidence pairs are eligible for association\[[1](https://arxiv.org/html/2608.19964#bib.bib17)\]\. For observation\-level support attachment, letvpO,vqO∈𝒱tOv\_\{p\}^\{O\},v\_\{q\}^\{O\}\\in\\mathcal\{V\}^\{O\}\_\{t\}denote two observation nodes, with semantic typesτp,τq\\tau\_\{p\},\\tau\_\{q\}and positions𝐱p,𝐱q\\mathbf\{x\}\_\{p\},\\mathbf\{x\}\_\{q\}inherited from their corresponding local observations\. We define the association gate as

δt\(vpO,vqO\)=𝟏\[type\_match\(τp,τq\)∧d\(𝐱p,𝐱q\)<ϵτp,τq\],\\delta\_\{t\}\(v\_\{p\}^\{O\},v\_\{q\}^\{O\}\)=\\mathbf\{1\}\\left\[\\mathrm\{type\\\_match\}\(\\tau\_\{p\},\\tau\_\{q\}\)\\land d\(\\mathbf\{x\}\_\{p\},\\mathbf\{x\}\_\{q\}\)<\\epsilon\_\{\\tau\_\{p\},\\tau\_\{q\}\}\\right\],whered⁡\(⋅,⋅\)d\(\\cdot,\\cdot\)is the distance between observation centers andϵτp,τq\\epsilon\_\{\\tau\_\{p\},\\tau\_\{q\}\}is a class\-dependent association threshold\. The functiontype​\_​match​\(τp,τq\)\\mathrm\{type\\\_match\}\(\\tau\_\{p\},\\tau\_\{q\}\)returns true only when the two observations have the same semantic class\. We use exact semantic\-class matching as a conservative design choice because false cross\-class merges can create unsafe object hypotheses\. For example, nearby vehicle observations can be associated, but a pedestrian and a vehicle remain separate even if they are geometrically close\.

For gated pairs,G\-MARKassigns an interpretable distance\-based affinity score,

αt,p​q=\{1−d⁡\(𝐱p,𝐱q\)ϵτp,τq,if​δt​\(vpO,vqO\)=1,0,otherwise\.\\alpha\_\{t,pq\}=\\begin\{cases\}1\-\\dfrac\{d\(\\mathbf\{x\}\_\{p\},\\mathbf\{x\}\_\{q\}\)\}\{\\epsilon\_\{\\tau\_\{p\},\\tau\_\{q\}\}\},&\\text\{if \}\\delta\_\{t\}\(v\_\{p\}^\{O\},v\_\{q\}^\{O\}\)=1,\\\\ 0,&\\text\{otherwise\.\}\\end\{cases\}The affinity score ranks gated pairs by spatial agreement, with higher values assigned to closer pairs\. The gating criterion follows standard data\-association practice\[[1](https://arxiv.org/html/2608.19964#bib.bib17)\], while the normalized distance score is an interpretable heuristic used inG\-MARK\. When an unmatched observation node is promoted to a candidate hypothesis, its representative state inherits the observation’s semantic type and position, so the same semantic and geometric compatibility rule is used for later candidate\-hypothesis merging\. The association graph induced byδt​\(vpO,vqO\)=1\\delta\_\{t\}\(v\_\{p\}^\{O\},v\_\{q\}^\{O\}\)=1determines which observation nodes attach as support evidence\. For candidate hypotheses, the same semantic and geometric rule is applied to their representative states to identify candidates eligible for merging\. The fused position𝐱h\\mathbf\{x\}\_\{h\}is computed as a confidence\-weighted average of the observation positions in𝒮⁡\(h\)\\mathcal\{S\}\(h\)\.

For each object hypothesis,G\-MARKsummarizes confidence using a bounded noisy\-or\-style aggregation\[[8](https://arxiv.org/html/2608.19964#bib.bib25)\]:

ch=1−∏vO∈𝒮⁡\(h\)\(1−c⁡\(vO\)\),c\_\{h\}=1\-\\prod\_\{v^\{O\}\\in\\mathcal\{S\}\(h\)\}\(1\-c\(v^\{O\}\)\),wherec⁡\(vO\)c\(v^\{O\}\)is the confidence stored by observation nodevOv^\{O\}\. We usechc\_\{h\}as a bounded confidence summary rather than as a calibrated probabilistic fusion model\. Unlike immediate fusion, the association output is not only a fused object location\. For every hypothesis,G\-MARKstores the support set𝒮⁡\(h\)\\mathcal\{S\}\(h\), provenance recordπh\\pi\_\{h\}, confidence summarychc\_\{h\}, candidate status, uncertainty, and disagreement attributes\. If an observation node does not pass the association gate with any other observation node,G\-MARKretains it as a candidate hypothesis with lower support and higher uncertainty\. Candidate retention is a safety\-oriented design choice: partner\-only or partially observed objects remain available for downstream inference, but are not treated as fully reliable detections\.

This module outputs well\-supported hypotheses with attached observation evidence, merged compatible candidates, and retained weak candidates\. The next module enriches them with provenance, visibility, uncertainty, disagreement, and planning relevance\.

### III\-CCooperative Context Enrichment

After hypothesis formation,G\-MARKderives context features for each object hypothesis from support, visibility, relative geometry, and ego motion context\. Unlike association, this stage describes how the belief was formed, who can observe it, how reliable it is, whether agents disagree, and whether it is relevant to ego motion\. We group these enrichments into five categories, summarized in Table[II](https://arxiv.org/html/2608.19964#S3.T2)\. Together, these enrichments let downstream modules reason over object state, supporting evidence, and ego\-motion relevance\[[13](https://arxiv.org/html/2608.19964#bib.bib10)\]\.

TABLE II:Cooperative context enrichment categories inG\-MARK\.These enrichments are needed because the same object can matter differently across tasks: partner\-only objects support hidden\-object discovery, uncertainty affects prediction confidence, and path proximity affects control and forecasting\. Formally, for each object hypothesishh,G\-MARKaugments the node with an enrichment vector

ϕ⁡\(h\)=\[ϕprov​\(h\),ϕvis​\(h\),ϕunc​\(h\),ϕdis​\(h\),ϕplan​\(h\)\],\\phi\(h\)=\[\\phi\_\{\\mathrm\{prov\}\}\(h\),\\phi\_\{\\mathrm\{vis\}\}\(h\),\\phi\_\{\\mathrm\{unc\}\}\(h\),\\phi\_\{\\mathrm\{dis\}\}\(h\),\\phi\_\{\\mathrm\{plan\}\}\(h\)\],whereϕprov​\(h\)\\phi\_\{\\mathrm\{prov\}\}\(h\),ϕunc​\(h\)\\phi\_\{\\mathrm\{unc\}\}\(h\), andϕdis​\(h\)\\phi\_\{\\mathrm\{dis\}\}\(h\)summarize support, provenance, confidence, and association consistency from the conservative association stage\. The componentsϕvis​\(h\)\\phi\_\{\\mathrm\{vis\}\}\(h\)andϕplan​\(h\)\\phi\_\{\\mathrm\{plan\}\}\(h\)encode ego\-relative visibility and distance to the ego vehicle’s planned or predicted trajectory\. The final KG\-derived feature vectorψ⁡\(h\)\\psi\(h\)concatenates object\-state attributes, such as position, semantic type, confidence, and motion cues, with the enrichment vectorϕ⁡\(h\)\\phi\(h\)\.

The output is an enriched cooperative KG in which downstream modules can query each hypothesis throughψ⁡\(h\)\\psi\(h\)without recomputing evidence from raw observations\.

TABLE III:KG\-based task\-head families inG\-MARK\.TABLE IV:Main results on V2V\-GoT\-QA\.G\-MARKimproves most strongly on tasks that require explicit cooperative evidence, including occlusion reasoning and hidden\-object discovery\.
### III\-DUnified KG Task Heads

The enriched KG provides a shared evidence state for task heads with different output formats\. As shown in Table[III](https://arxiv.org/html/2608.19964#S3.T3), we group them into object selection and visibility reasoning, motion and interaction prediction, and control and trajectory planning, following the standard perception\-prediction\-planning decomposition\[[23](https://arxiv.org/html/2608.19964#bib.bib18)\]\. Each head operates on KG\-derived featuresψ⁡\(h\)\\psi\(h\)that combine object state, provenance, visibility, uncertainty, disagreement, spatial relations, and planning\-context attributes\.

For a task familykk, we write the inference head as

yk=Rk​\(𝒢t,\{ψ⁡\(h\):h∈𝒱tH\}\),y\_\{k\}=R\_\{k\}\\\!\\left\(\\mathcal\{G\}\_\{t\},\\\{\\psi\(h\):h\\in\\mathcal\{V\}^\{H\}\_\{t\}\\\}\\right\),where𝒢t\\mathcal\{G\}\_\{t\}is the enriched cooperative KG,ψ⁡\(h\)\\psi\(h\)is the KG\-derived feature vector for hypothesishh,RkR\_\{k\}is the task head, andyky\_\{k\}is the structured task output\. Depending on the task,RkR\_\{k\}is implemented as either a deterministic retrieval/ranking function or a lightweight learned predictor overψ⁡\(h\)\\psi\(h\)\. Learned heads support motion regression, scene\-level prediction, control selection, and trajectory forecasting without heavy multimodal inference, while preserving evidence traceability\. Table[III](https://arxiv.org/html/2608.19964#S3.T3)summarizes how the same enriched KG supports different task families through a common feature interface\. This design decouples representation construction from task inference:G\-MARKbuilds one provenance\-aware cooperative KG per scene, and each head selects the evidence needed for its output\. In our benchmark instantiation, these heads cover object grounding, visibility and occlusion reasoning, planning\-aware object selection, motion prediction, control selection, and future trajectory forecasting\.

## IVExperimental Analysis

### IV\-AExperimental Setup

Benchmarks\.V2V4Real provides synchronized real\-world V2V scenes with connected vehicles, object annotations, agent poses, timestamps, and processed perception artifacts\. V2V\-GoT\-QA extends this setting with typed object\-selection, visibility\-reasoning, motion\-prediction, control\-selection, and trajectory\-forecasting tasks\. We use the official splits, with approximately110110K training and3131K validation questions, and evaluate on the validation split using the same task definitions and reference answers as V2V\-GoT\[[2](https://arxiv.org/html/2608.19964#bib.bib1)\]\.

Evaluation metrics\.Object selection and visibility reasoning tasks use localization\-aware precision, recall, and F1\-score, where predictions must match ground\-truth coordinates within the benchmark threshold\. Motion and trajectory outputs use L2 error, agent motion uses binary accuracy, and control selection uses normalized action error\. Higher is better for precision, recall, F1, and accuracy; lower is better for L2 and action error\.

Baselines\.Our primary baseline is V2V\-GoT\[[2](https://arxiv.org/html/2608.19964#bib.bib1)\], the closest recent cooperative reasoning method for this benchmark\. V2V\-GoT extends V2V\-LLM\-style cooperative multimodal QA\[[3](https://arxiv.org/html/2608.19964#bib.bib3)\]by incorporating graph\-of\-thought reasoning for perception, prediction, and planning questions\. It reports strong performance on future trajectory forecasting and other high\-level cooperative driving tasks\. We therefore compareG\-MARKwith the reported V2V\-GoT results across all benchmark tasks, including object reasoning, motion prediction, control selection, and trajectory forecasting\.

Communication efficiency is an important requirement for cooperative driving, where agents must share sufficient scene evidence without incurring excessive bandwidth overhead\. For the communication–accuracy analysis, we compareG\-MARKwith the future\-trajectory forecasting results of No Fusion, AttFuse\[[19](https://arxiv.org/html/2608.19964#bib.bib5)\], V2X\-ViT\[[18](https://arxiv.org/html/2608.19964#bib.bib8)\], CoBEVT\[[17](https://arxiv.org/html/2608.19964#bib.bib9)\], V2V\-LLM\[[3](https://arxiv.org/html/2608.19964#bib.bib3)\], and V2V\-GoT\[[2](https://arxiv.org/html/2608.19964#bib.bib1)\]\. We restrict this analysis to future trajectory forecasting because this task directly links cooperative communication cost to downstream planning accuracy\.

These comparisons evaluate whether explicit cooperative evidence improves visibility\-sensitive reasoning, generalizes across multiple task families, and offers a communication\-efficient alternative to language\-mediated cooperative reasoning\.

Implementation details\.All experiments use the KG construction pipeline over processed cooperative perception artifacts, including object boxes, confidence scores, agent poses, trajectory context, and scene metadata\. After KG construction,G\-MARKexports a shared KG\-derived feature bank with three typed views: object retrieval for Q1–Q4, motion regression for Q5/Q7/Q9, and scene\-action prediction for Q6/Q8\.

Object retrieval uses a shared logistic regression ranker with task one\-hot features and thresholds selected from the training split\. Motion prediction uses regularized regression heads, with a shared Q5/Q7 object\-motion head and a separate Q9 trajectory head\. Action prediction uses lightweight classifiers for Q6 and separate speed and steering heads for Q8\. All heads are trained on the official training split, with validation labels used only for evaluation; experiments run on a Linux machine with 8 CPU cores and 24 GB RAM\.

### IV\-BResults

Explicit cooperative evidence improves visibility\-sensitive reasoning\.Table[IV](https://arxiv.org/html/2608.19964#S3.T4)comparesG\-MARKwith V2V\-GoT\[[2](https://arxiv.org/html/2608.19964#bib.bib1)\], a state\-of\-the\-art cooperative reasoning baseline for V2V\-GoT\-QA, across all task categories\. We report Object Motion Q5 and Q7 separately because they are distinct benchmark entries, although they share the same output family\. The strong gains on these entries likely reflect the structure of the output: object\-motion prediction is a numeric regression problem, andG\-MARKdirectly exposes object state, recent motion cues, and path\-relative geometry as compact KG\-derived features\. This representation is well suited to motion regression becauseψ⁡\(h\)\\psi\(h\)directly encodes numeric object state, recent motion cues, and path\-relative geometry\. Language\-mediated reasoning baselines must infer these quantities less directly before producing a numeric output\.

G\-MARKimproves over V2V\-GoT on eight of the nine benchmark entries, with the largest gains on tasks that require explicit cooperative evidence\. Occlusion reasoning improves by42\.2%42\.2\\%, hidden\-object discovery by12\.3%12\.3\\%, and object motion prediction by49\.8%49\.8\\%\. These gains support task\-conditioned delayed evidence fusion: preserving provenance, visibility, uncertainty, disagreement, and candidate status helps each task head query relevant evidence rather than relying on a single collapsed object state\. The shared KG also supports prediction and control, with object motion benefiting from KG\-derived motion cues and path\-relative geometry, and control\-selection error reduced by13\.1%13\.1\\%\. Future trajectory forecasting shows a different pattern:G\-MARKreaches near\-parity with V2V\-GoT, with an average L2 error of2\.7102\.710m compared with2\.6202\.620m\. This suggests that the KG\-conditioned representation captures useful planning context, but longer\-horizon forecasting remains sensitive to accumulated speed, steering, and curvature errors\.

Fig\. 2:Planning accuracy versus communication cost\.G\-MARKapproaches planning accuracy with substantially lower communication\.We use future trajectory forecasting because it tests whether communication preserves planning\-relevant evidence, including nearby objects, occluded or partner\-only hazards, and path\-relative interactions\. Trajectory error therefore measures useful scene information retained per unit of communication\. Fig\.[2](https://arxiv.org/html/2608.19964#S4.F2)comparesG\-MARKwith three categories of cooperative planning baselines: a non\-cooperative baseline \(No Fusion\), intermediate\-fusion baselines including AttFuse\[[19](https://arxiv.org/html/2608.19964#bib.bib5)\], V2X\-ViT\[[18](https://arxiv.org/html/2608.19964#bib.bib8)\], and CoBEVT\[[17](https://arxiv.org/html/2608.19964#bib.bib9)\], and language\-mediated cooperative reasoning baselines including V2V\-LLM\[[3](https://arxiv.org/html/2608.19964#bib.bib3)\]and V2V\-GoT\[[2](https://arxiv.org/html/2608.19964#bib.bib1)\]\. No Fusion has high trajectory error, while intermediate\-fusion and language\-mediated methods improve accuracy at roughly0\.40080\.4008–0\.40680\.4068MB per sample\.G\-MARKachieves a different operating point: it uses only0\.01590\.0159MB per sample, giving about25\.2×25\.2\\timeslower communication than intermediate\-fusion methods and25\.6×25\.6\\timeslower communication than V2V\-GoT\. This reduction comes from transmitting compact KG evidence rather than dense feature\-level or language\-mediated payloads\. Although V2V\-GoT achieves the lowest trajectory error,G\-MARKattains comparable forecasting accuracy with far lower communication and explicit evidence traceability\.

Fig\. 3:End\-to\-end latency breakdown ofG\-MARKby task\.Fig\. 4:Diagnostic ablations on evidence\-sensitive tasks\.Structured KG reasoning adds minimal CPU\-only overhead\.Fig\.[3](https://arxiv.org/html/2608.19964#S4.F3)shows representative per\-sample CPU runtimes after upstream perception has produced processed scene artifacts\. We group runtime into scene loading, KG/task solving, and other overhead for graph construction and feature extraction\.

The reasoning head is lightweight: task\-solver time remains below1\.41\.4ms per sample and below11ms for six of the eight task types\. Current\-frame tasks require10\.310\.3–14\.214\.2ms total CPU latency, while temporal motion tasks require32\.432\.4–33\.433\.4ms due to previous\-frame context loading\. Most runtime comes from data access and scene preparation rather than KG reasoning, indicating that the proposed reasoning layer is not the primary bottleneck\. Recent VLM\-based cooperative driving systems report end\-to\-end latencies on the order of hundreds of milliseconds\[[22](https://arxiv.org/html/2608.19964#bib.bib20),[21](https://arxiv.org/html/2608.19964#bib.bib21)\]\. Although not a direct system\-level comparison,G\-MARKrequires only about44–6%6\\%of these reported latencies for current\-frame tasks, while preserving millisecond\-scale CPU reasoning\.

### IV\-CAblation Study

We conduct construction\-level ablations to evaluate which KG evidence components contribute to downstream reasoning\. The task heads are kept fixed, while the KG evidence is selectively modified before inference\. Fig\.[4](https://arxiv.org/html/2608.19964#S4.F4)summarizes the results for hidden\-object discovery and control selection\.

Partner evidence is essential for hidden\-object discovery\.As shown in Fig\.[4](https://arxiv.org/html/2608.19964#S4.F4), removing partner observations drops Invisible Objects F1 from0\.4940\.494to0\.0000\.000\. This confirms that hidden\-object discovery depends on cooperative evidence outside the ego vehicle’s direct view; an ego\-only representation cannot recover objects visible only to connected vehicles\.

Provenance affects both reasoning and control\.Removing provenance reduces Invisible Objects F1 from0\.4940\.494to0\.3960\.396and increases Control Settings Action L1 from0\.0760\.076to0\.1520\.152, showing that source\-agent support is useful for downstream decisions\.

KG\-derived structure helps beyond object geometry\.Replacing the KG with an unstructured object\-level representation reduces Invisible Objects F1 from0\.4940\.494to0\.4430\.443and increases Control Settings Action L1 from0\.0760\.076to0\.0890\.089\. These results suggest that visibility, uncertainty, disagreement, and path\-relevance features provide useful structure for evidence\-sensitive cooperative reasoning\.

## VConclusion

We presentedG\-MARK, a grounded multi\-agent reasoning framework that maps processed cooperative perception evidence into a provenance\-aware KG for object reasoning, prediction, control selection, and trajectory forecasting\. On V2V\-GoT\-QA,G\-MARKimproves evidence\-sensitive tasks such as occlusion reasoning and hidden\-object discovery, while achieving near\-parity in future trajectory forecasting with25\.6×25\.6\\timeslower structured communication and millisecond\-scale CPU overhead\. These results demonstrate the potential of provenance\-aware cooperative KGs as compact, traceable, and communication\-efficient reasoning layers for cooperative driving\.

## Acknowledgements

This work has been funded in part by NSF, with award numbers \#2112665, \#2112167, \#2003279, \#2120019, \#2211386, \#2052809, \#1911095 and in part by PRISM and CoCoSys, centers in JUMP 2\.0, an SRC program sponsored by DARPA\.

## References

- \[1\]\(1988\)Tracking and data association\.Academic Press\.Cited by:[§III\-B](https://arxiv.org/html/2608.19964#S3.SS2.p2.1),[§III\-B](https://arxiv.org/html/2608.19964#S3.SS2.p3.2)\.
- \[2\]H\. Chiuet al\.\(2025\)V2V\-GoT: vehicle\-to\-vehicle cooperative autonomous driving with multimodal large language models and graph\-of\-thoughts\.Note:Accepted to ICRA 2026External Links:2509\.18053Cited by:[TABLE I](https://arxiv.org/html/2608.19964#S1.T1.2.1.5.1.1.1),[§I](https://arxiv.org/html/2608.19964#S1.p3.1),[§I](https://arxiv.org/html/2608.19964#S1.p5.1),[§II](https://arxiv.org/html/2608.19964#S2.p1.1),[§II](https://arxiv.org/html/2608.19964#S2.p5.1),[TABLE IV](https://arxiv.org/html/2608.19964#S3.T4.4.1.5.1),[§IV\-A](https://arxiv.org/html/2608.19964#S4.SS1.p1.1),[§IV\-A](https://arxiv.org/html/2608.19964#S4.SS1.p3.1),[§IV\-A](https://arxiv.org/html/2608.19964#S4.SS1.p4.1),[§IV\-B](https://arxiv.org/html/2608.19964#S4.SS2.p1.1),[§IV\-B](https://arxiv.org/html/2608.19964#S4.SS2.p3.1)\.
- \[3\]H\. Chiuet al\.\(2025\)V2V\-LLM: vehicle\-to\-vehicle cooperative autonomous driving with multi\-modal large language models\.Note:Accepted to ICRA 2026External Links:2502\.09980Cited by:[§I](https://arxiv.org/html/2608.19964#S1.p3.1),[§II](https://arxiv.org/html/2608.19964#S2.p1.1),[§II](https://arxiv.org/html/2608.19964#S2.p5.1),[§IV\-A](https://arxiv.org/html/2608.19964#S4.SS1.p3.1),[§IV\-A](https://arxiv.org/html/2608.19964#S4.SS1.p4.1),[§IV\-B](https://arxiv.org/html/2608.19964#S4.SS2.p3.1)\.
- \[4\]X\. Gaoet al\.\(2025\)LangCoop: collaborative driving with language\.InProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops,Cited by:[§II](https://arxiv.org/html/2608.19964#S2.p1.1),[§II](https://arxiv.org/html/2608.19964#S2.p5.1)\.
- \[5\]P\. Groth and L\. Moreau\(2013\)PROV\-Overview: An Overview of the PROV Family of Documents\.Note:W3C Working Group NoteWorld Wide Web Consortium \(W3C\)External Links:[Link](https://www.w3.org/TR/prov-overview/)Cited by:[§III\-A](https://arxiv.org/html/2608.19964#S3.SS1.p5.1),[§III\-A](https://arxiv.org/html/2608.19964#S3.SS1.p7.2)\.
- \[6\]A\. Hoganet al\.\(2021\)Knowledge graphs\.ACM Computing Surveys54\(4\),pp\. 1–37\.External Links:[Document](https://dx.doi.org/10.1145/3447772)Cited by:[§II](https://arxiv.org/html/2608.19964#S2.p1.1),[§II](https://arxiv.org/html/2608.19964#S2.p4.1),[§III\-A](https://arxiv.org/html/2608.19964#S3.SS1.p1.1),[§III\-A](https://arxiv.org/html/2608.19964#S3.SS1.p5.1)\.
- \[7\]Y\. Huet al\.\(2022\)Where2comm: communication\-efficient collaborative perception via spatial confidence maps\.InAdvances in Neural Information Processing Systems,Vol\.35,pp\. 4874–4886\.Cited by:[§I](https://arxiv.org/html/2608.19964#S1.p2.1),[§II](https://arxiv.org/html/2608.19964#S2.p1.1),[§II](https://arxiv.org/html/2608.19964#S2.p2.1)\.
- \[8\]D\. Koller and N\. Friedman\(2009\)Probabilistic graphical models: principles and techniques\.MIT Press\.Cited by:[§III\-B](https://arxiv.org/html/2608.19964#S3.SS2.p4.1)\.
- \[9\]A\. Kuznietsovet al\.\(2024\)Explainable ai for safe and trustworthy autonomous driving: a systematic review\.IEEE Transactions on Intelligent Transportation Systems25\(12\),pp\. 19342–19364\.Cited by:[§I](https://arxiv.org/html/2608.19964#S1.p1.1),[§II](https://arxiv.org/html/2608.19964#S2.p1.1)\.
- \[10\]Y\. Li, D\. Ma, Z\. An, Z\. Wang, Y\. Zhong, S\. Chen, and C\. Feng\(2022\)V2X\-Sim: multi\-agent collaborative perception dataset and benchmark for autonomous driving\.IEEE Robotics and Automation Letters7\(4\),pp\. 10914–10921\.External Links:[Document](https://dx.doi.org/10.1109/LRA.2022.3192802)Cited by:[§III\-A](https://arxiv.org/html/2608.19964#S3.SS1.p1.1)\.
- \[11\]C\. Liuet al\.\(2025\)CoLMDriver: llm\-based negotiation benefits cooperative autonomous driving\.InProceedings of the IEEE/CVF International Conference on Computer Vision,Cited by:[§II](https://arxiv.org/html/2608.19964#S2.p1.1),[§II](https://arxiv.org/html/2608.19964#S2.p5.1)\.
- \[12\]H\. Shaoet al\.\(2024\)LMDrive: closed\-loop end\-to\-end driving with large language models\.InProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition \(CVPR\),Cited by:[§I](https://arxiv.org/html/2608.19964#S1.p3.1),[§II](https://arxiv.org/html/2608.19964#S2.p1.1),[§II](https://arxiv.org/html/2608.19964#S2.p3.1)\.
- \[13\]C\. Sima, K\. Renz, K\. Chitta, L\. Chen, H\. Zhang, C\. Xie, J\. Beisswenger, P\. Luo, A\. Geiger, and H\. Li\(2024\)DriveLM: driving with graph visual question answering\.InProceedings of the European Conference on Computer Vision \(ECCV\),pp\. 256–274\.Cited by:[TABLE I](https://arxiv.org/html/2608.19964#S1.T1.2.1.3.1.1.1),[§I](https://arxiv.org/html/2608.19964#S1.p3.1),[§II](https://arxiv.org/html/2608.19964#S2.p1.1),[§II](https://arxiv.org/html/2608.19964#S2.p3.1),[§II](https://arxiv.org/html/2608.19964#S2.p4.1),[§III\-C](https://arxiv.org/html/2608.19964#S3.SS3.p1.1)\.
- \[14\]X\. Tian, J\. Gu, B\. Li, Y\. Liu, Y\. Wang, Z\. Zhao, K\. Zhan, P\. Jia, X\. Lang, and H\. Zhao\(2024\)DriveVLM: the convergence of autonomous driving and large vision\-language models\.InProceedings of the Conference on Robot Learning \(CoRL\),Cited by:[§I](https://arxiv.org/html/2608.19964#S1.p3.1),[§II](https://arxiv.org/html/2608.19964#S2.p1.1),[§II](https://arxiv.org/html/2608.19964#S2.p3.1)\.
- \[15\]Y\. Tian, J\. Zhang, Z\. Wang, X\. Ren, X\. Yu, O\. Gungor, and T\. Rosing\(2026\)KLDrive: fine\-grained 3d scene reasoning for autonomous driving based on knowledge graph\.arXiv preprint arXiv:2603\.21029\.Cited by:[TABLE I](https://arxiv.org/html/2608.19964#S1.T1.2.1.4.1.1.1),[§II](https://arxiv.org/html/2608.19964#S2.p1.1),[§II](https://arxiv.org/html/2608.19964#S2.p4.1),[§III\-A](https://arxiv.org/html/2608.19964#S3.SS1.p1.1)\.
- \[16\]R\. Xuet al\.\(2023\)V2V4Real: a real\-world large\-scale dataset for vehicle\-to\-vehicle cooperative perception\.InProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition,pp\. 13712–13722\.Cited by:[§III\-A](https://arxiv.org/html/2608.19964#S3.SS1.p1.1)\.
- \[17\]R\. Xu, Z\. Tu, H\. Xiang, W\. Shao, B\. Zhou, and J\. Ma\(2023\)CoBEVT: cooperative bird’s eye view semantic segmentation with sparse transformers\.InConference on Robot Learning,pp\. 989–1000\.Cited by:[TABLE I](https://arxiv.org/html/2608.19964#S1.T1.2.1.2.1.1.1),[§I](https://arxiv.org/html/2608.19964#S1.p2.1),[§II](https://arxiv.org/html/2608.19964#S2.p1.1),[§II](https://arxiv.org/html/2608.19964#S2.p2.1),[§IV\-A](https://arxiv.org/html/2608.19964#S4.SS1.p4.1),[§IV\-B](https://arxiv.org/html/2608.19964#S4.SS2.p3.1)\.
- \[18\]R\. Xu, H\. Xiang, Z\. Tu, X\. Xia, M\. Yang, and J\. Ma\(2022\)V2X\-vit: vehicle\-to\-everything cooperative perception with vision transformer\.InEuropean Conference on Computer Vision,pp\. 107–124\.Cited by:[TABLE I](https://arxiv.org/html/2608.19964#S1.T1.2.1.2.1.1.1),[§I](https://arxiv.org/html/2608.19964#S1.p2.1),[§II](https://arxiv.org/html/2608.19964#S2.p1.1),[§II](https://arxiv.org/html/2608.19964#S2.p2.1),[§III\-A](https://arxiv.org/html/2608.19964#S3.SS1.p1.1),[§IV\-A](https://arxiv.org/html/2608.19964#S4.SS1.p4.1),[§IV\-B](https://arxiv.org/html/2608.19964#S4.SS2.p3.1)\.
- \[19\]R\. Xu, H\. Xiang, X\. Xia, X\. Han, J\. Li, and J\. Ma\(2022\)Opv2v: an open benchmark dataset and fusion pipeline for perception with vehicle\-to\-vehicle communication\.In2022 International Conference on Robotics and Automation \(ICRA\),pp\. 2583–2589\.Cited by:[§I](https://arxiv.org/html/2608.19964#S1.p1.1),[§IV\-A](https://arxiv.org/html/2608.19964#S4.SS1.p4.1),[§IV\-B](https://arxiv.org/html/2608.19964#S4.SS2.p3.1)\.
- \[20\]Z\. Xu, Y\. Zhang, E\. Xie, Z\. Zhao, Y\. Guo, K\. K\. Wong, Z\. Li, and H\. Zhao\(2024\)DriveGPT4: interpretable end\-to\-end autonomous driving via large language model\.IEEE Robotics and Automation Letters\.Note:Also available as arXiv:2310\.01412Cited by:[§I](https://arxiv.org/html/2608.19964#S1.p3.1),[§II](https://arxiv.org/html/2608.19964#S2.p1.1),[§II](https://arxiv.org/html/2608.19964#S2.p3.1)\.
- \[21\]J\. You, P\. Li, Z\. Jiang, Z\. Huang, R\. Gan, H\. Shi, and B\. Ran\(2025\)V2X\-realm: vision\-language model\-based robust end\-to\-end cooperative autonomous driving with adaptive long\-tail modeling\.arXiv preprint arXiv:2506\.21041\.Cited by:[§IV\-B](https://arxiv.org/html/2608.19964#S4.SS2.p5.1)\.
- \[22\]J\. Youet al\.\(2024\)V2X\-vlm: end\-to\-end v2x cooperative autonomous driving through large vision\-language models\.arXiv preprint arXiv:2408\.09251\.Cited by:[§IV\-B](https://arxiv.org/html/2608.19964#S4.SS2.p5.1)\.
- \[23\]E\. Yurtsever, J\. Lambert, A\. Carballo, and K\. Takeda\(2020\)A survey of autonomous driving: common practices and emerging technologies\.IEEE Access8,pp\. 58443–58469\.Cited by:[§III\-D](https://arxiv.org/html/2608.19964#S3.SS4.p1.1)\.
- \[24\]C\. Zhanget al\.\(2023\)Occlusion\-aware planning for autonomous driving with vehicle\-to\-everything communication\.IEEE Transactions on Intelligent Vehicles9\(1\),pp\. 1229–1242\.Cited by:[§I](https://arxiv.org/html/2608.19964#S1.p1.1)\.
- \[25\]Z\. Zhang and J\. F\. Fisac\(2021\)Safe occlusion\-aware autonomous driving via game\-theoretic active perception\.arXiv preprint arXiv:2105\.08169\.Cited by:[§I](https://arxiv.org/html/2608.19964#S1.p1.1)\.