与我对话，Jarvis：一个面向自主赛车的开源可边缘部署语音助手框架

# Talk to Me, Jarvis: An Open-Source Edge-Deployable Voice Assistant Framework for Autonomous Racecars
Source: [https://arxiv.org/html/2609.21109](https://arxiv.org/html/2609.21109)
Frederik WernerAlexander LangmannJohannes Betz††thanks:D\. Henel, A\. Langmann and J\. Betz are with the Professorship of Autonomous Vehicle Systems, TUM School of Engineering and Design, Technical University of Munich, 85748 Garching, Germany; Munich Institute of Robotics and Machine Intelligence \(MIRMI\)††thanks:F\. Werner is with the Institute of Automotive Technology, TUM School of Engineering and Design, Technical University of Munich, 85748 Garching, Germany; Munich Institute of Robotics and Machine Intelligence \(MIRMI\)\. This work has been accepted for publication in the Proceedings of the IEEE Intelligent Transportation Systems Conference \(ITSC 2026\)

###### Abstract

Recent advances in large language models have improved their effectiveness as back\-end components for voice assistants, particularly in intent understanding and context\-aware input classification\. However, online\-hosted models introduce network dependency and variable inference latency, limiting their suitability for time\-critical autonomous driving applications\.

In this work, we address these issues by developing Jarvis, an offline voice assistant for high\-level behavioral commands of autonomous vehicles\. Its architecture integrates speech recognition and synthesis with natural language command classification into a lightweight, local framework\.

Jarvis’ core component is a text\-to\-command classifier, built using a domain\-specific fine\-tuning of the Mistral 7B model, demonstrating low\-latency inference\. Our experimental evaluation demonstrates that our solution outperforms larger online\-hosted models, achieving 97\.63 % intent recognition accuracy with an average processing latency of 1\.39 s, making it well\-suited for operations requiring quick response times\. To support further research and fine\-tuning, we provide an open\-source implementation\.

Index Terms—voice assistant, autonomous driving, intent recognition, large language model

## IINTRODUCTION

In racing, fractions of a second can decide between victory and defeat\. Autonomous racing vehicles must make split\-second decisions to outperform competitors, placing high demands on their software systems\[[1](https://arxiv.org/html/2609.21109#bib.bib2)\]\. This requirement extends to the AI models that support these vehicles, which must be optimized to ensure both speed and efficiency while maintaining reliable performance\.

Despite a high degree of autonomy, human input is still needed in certain situations in autonomous racing, e\.g\., issuing high\-level commands like start or stop, or reacting to unexpected situations on the track that require sending the car to the pit lane\. Traditional interfaces for these commands are usually provided through a graphical user interface \(GUI\) or a command line, which require visual attention and increase reaction time\.

To address this problem, voice assistants provide an alternative solution, allowing the human operator to issue commands without looking away from the live telemetry\. However, developing such voice assistants is not straightforward\. Challenges like latency, command detection reliability, and network dependency need to be addressed to design a useful voice assistant system\.

Another key challenge arises from the inherent mismatch between the unbounded space of natural language expressions and the limited set of executable commands\. For example, an operator may issue a“stop”command in many different ways \-“halt the car”,“bring it to a stop”, or simply“stop”\- all of which must be correctly recognized and mapped to the same underlying action\. This variability motivates the use of Large\-Language\-Models \(LLMs\) to capture semantic equivalence across phrasings\.

### I\-AContributions

This work makes the following contributions:

- •A comparison of online\-hosted and locally deployed language models for voice command classification, evaluating both inference latency and intent recognition accuracy\.
- •A fine\-tuning and deployment pipeline for a lightweight local model that achieves superior intent recognition accuracy while reducing inference latency compared to online alternatives\.
- •An open\-source, offline voice assistant architecture integrating wake\-word detection, speech transcription, and command classification, demonstrated on resource\-constrained mobile hardware \([github\.com/TUM\-AVS/jarvis\_voice\_assistant](https://github.com/TUM-AVS/jarvis_voice_assistant)\)\.

Fig\. 1:System Architecture: Jarvis is a voice assistant for human operators used to control a autonomous racecar via voice commands\.Fig\. 2:Autonomous driving software modules\.
### I\-BRelated Work

LLMs are increasingly used in various components of autonomous driving, such as scene understanding and multimodal perception\[[2](https://arxiv.org/html/2609.21109#bib.bib20),[3](https://arxiv.org/html/2609.21109#bib.bib33)\], planning and decision making\[[4](https://arxiv.org/html/2609.21109#bib.bib34),[5](https://arxiv.org/html/2609.21109#bib.bib35)\], explainability and human\-machine interaction\[[6](https://arxiv.org/html/2609.21109#bib.bib36),[7](https://arxiv.org/html/2609.21109#bib.bib37)\], commonsense and traffic rule reasoning\[[8](https://arxiv.org/html/2609.21109#bib.bib38),[9](https://arxiv.org/html/2609.21109#bib.bib39)\]\.

Many researchers investigate the use of LLMs in intent recognition and command classification\[[10](https://arxiv.org/html/2609.21109#bib.bib21),[11](https://arxiv.org/html/2609.21109#bib.bib22),[12](https://arxiv.org/html/2609.21109#bib.bib23)\], focusing on translating natural language or contextual cues into actionable driving behaviors\. Cui et al\.\[[13](https://arxiv.org/html/2609.21109#bib.bib7)\]propose LLMs for Autonomous Driving \(LLM4AD\), which conceptualize the LLM as a cognitive brain for autonomous vehicles, capable of reasoning and intent understanding\. By leveraging the broad knowledge of large\-scale models like GPT\-4\[[14](https://arxiv.org/html/2609.21109#bib.bib24)\], LLM4AD aims to interpret abstract human inputs, such as adjusting driving style e\.g\. because a passenger is in a hurry, and translate them into high\-level control strategies\. However, while LLM4AD provides a benchmark for general passenger comfort, it relies on cloud\-based Application Programming Interfaces \(APIs\) that introduce notable inference latencies and depend on a stable network connection\. Furthermore, the framework is primarily designed for civilian vehicles with occupants\. In contrast, our approach targets autonomous racing, shifting the focus to the interface between a race engineer and the autonomous driving system operating the vehicle\. This scope requires a responsive architecture that is more focused on accurate command understanding and low latency and less on general human\-centric interaction\.

Domain\-specific adaptation of lightweight Small Language Models \(SLMs\)\[[15](https://arxiv.org/html/2609.21109#bib.bib25)\]has proven effective across a variety of specialized tasks\[[16](https://arxiv.org/html/2609.21109#bib.bib29),[17](https://arxiv.org/html/2609.21109#bib.bib28),[18](https://arxiv.org/html/2609.21109#bib.bib27)\]\. For instance, Pal et al\.\[[19](https://arxiv.org/html/2609.21109#bib.bib8)\]demonstrated that supervised fine\-tuning combined with Quantized Low\-Rank Adaptation \(QLoRA\)\[[20](https://arxiv.org/html/2609.21109#bib.bib10)\]on the Phi\-3\.5 Mini Instruct model\[[21](https://arxiv.org/html/2609.21109#bib.bib26)\]significantly improves intent recognition in vehicle\-related tasks\. This architecture addresses critical challenges such as computational constraints and latency by employing 4\-bit quantization to enable efficient execution on edge hardware\. Our work adopts a similar technical approach, described in detail in Section[III\-C](https://arxiv.org/html/2609.21109#S3.SS3)\.

Beyond LLM\-based intent recognition, the operational viability of voice\-enabled assistants in time\-critical applications is fundamentally tied to the underlying speech\-to\-text \(STT\) architecture\[[22](https://arxiv.org/html/2609.21109#bib.bib30),[23](https://arxiv.org/html/2609.21109#bib.bib11)\]\. Di Leo et al\.\[[24](https://arxiv.org/html/2609.21109#bib.bib9)\]propose a real\-time STT framework specifically designed for edge computing environments requiring ultra\-low latency and local execution\. Their prototype utilizes the lightweight Vosk engine\[[25](https://arxiv.org/html/2609.21109#bib.bib31)\]for offline automatic speech recognition \(ASR\)\[[23](https://arxiv.org/html/2609.21109#bib.bib11)\], focusing on data privacy and operational independence by eliminating reliance on external network bandwidth\. The authors highlight that their modular and extensible design is intended to accommodate the future integration of more advanced neural STT engines, such as OpenAI’s Whisper\[[26](https://arxiv.org/html/2609.21109#bib.bib6)\]\. Our work adopts this local\-first philosophy but directly utilizes the Whisper architecture, as described in detail in Section[II\-B](https://arxiv.org/html/2609.21109#S2.SS2)\.

## IIMethod

We integrate ourJarvisvoice assistant with a software stack for autonomous motorsport applications\[[27](https://arxiv.org/html/2609.21109#bib.bib1)\]\. We describe the integration in Section[II\-A](https://arxiv.org/html/2609.21109#S2.SS1)and theJarvisarchitecture itself in Section[II\-B](https://arxiv.org/html/2609.21109#S2.SS2)\.

### II\-ASystem Architecture

The system is centered around a base station \(Figure[1](https://arxiv.org/html/2609.21109#S1.F1)\), which acts as the interface between humans and the onboard vehicle software\. Moreover, it receives information about the status of the racing car and all associated software modules \(Figure[2](https://arxiv.org/html/2609.21109#S1.F2)\), providing telemetry, and diagnostics\. The software modules include, among others, perception, planning, detection, which communicate via the Robot Operating System 2 \(ROS2\)\[[28](https://arxiv.org/html/2609.21109#bib.bib3)\]\. OurJarvisvoice assistant is implemented as a stand\-alone module and extends the base station by enabling a human operator to issue high\-level commands using natural speech\. Speech inputs are processed by the voice assistant pipeline, as described in Section[II\-B](https://arxiv.org/html/2609.21109#S2.SS2), where automatic speech recognition and intent interpretation generate predefined behavioral commands, such as speed targets, start/stop commands or requests to exit or return to the pit lane\. These commands are transmitted to the base station using ROS2, where they undergo validation and safety checks before being accepted\. Once validated, the base station forwards the commands to the racing car over a mobile network connection to the vehicle\. To ensure transparency and operator awareness, every voice\-enabled command is visualized in the base station interface, allowing the human controller to verify correct command detection and execution\.

### II\-BVoice Assistant Architecture

Fig\. 3:System architecture and process flow of Jarvis\. After wake\-word detection, the system performs local speech\-to\-text transcription, LLM\-based command classification, and text\-to\-speech feedback\. Confirmed commands are transmitted via ROS2 to the base station, where safety checks precede transmission on the autonomous racing vehicle\.The architecture ofJarvisis visualized in Figure[3](https://arxiv.org/html/2609.21109#S2.F3)\. The system is initiated by wake\-word detection based on the openWakeWord framework111[https://github\.com/dscripka/openWakeWord](https://github.com/dscripka/openWakeWord), which continuously monitors the audio stream for the trigger phrase “Hey Jarvis\!”, using keyword spotting techniques\[[29](https://arxiv.org/html/2609.21109#bib.bib5)\]\. Upon activation, the assistant acknowledges the human operator with a short synthesized response "Hi\!" and transitions into a command\-listening state\.

Speech\-to\-text conversion is performed locally using OpenAI’s Whisper model, specifically the lightweight English\-only “base\.en” variant222[https://github\.com/openai/whisper](https://github.com/openai/whisper), enabling low\-latency, network\-independent transcription\. The resulting text is forwarded to the core LLM\-based text\-to\-command classifier, which interprets the operator’s intent and maps the natural\-language input to a predefined behavioral command\. We compared online\-hosted and locally deployed open\-weight models for command classification\. While cloud models provide strong zero\-shot performance, network\-based inference introduces higher latency\. Fine\-tuned local models achieved the best trade\-off between latency and accuracy as described in Section[III](https://arxiv.org/html/2609.21109#S3)\.

Following command interpretation, auditory feedback is generated for the operator using a text\-to\-speech \(TTS\) module based on the Coqui TTS framework333[https://github\.com/coqui\-ai/TTS](https://github.com/coqui-ai/TTS)and a VITS\-based neural speech synthesis model\[[30](https://arxiv.org/html/2609.21109#bib.bib4)\]\. This feedback confirms successful command recognition or indicates failure\. Frequently used responses are pre\-recorded to reduce latency and played back directly\.

In the final step, the human operator is asked to confirm the execution of a recognized command, and only confirmed commands are transmitted to the base station via ROS2, where additional safety checks and state validations are performed before forwarding the commands to the autonomous racing car\.

## IIIRESULTS

We conducted a multi\-stage evaluation\. First, we compared multiple LLMs in default configurations\. We then applied two successive fine\-tuning stages, resulting in the final selected model\. Our primary benchmarking criteria were:

- •Intent Classification Accuracydefined as the ratio of correctly classified commands to the total number of test samples\.
- •Inference timemeasured as the end\-to\-end processing time required for the model to classify a command\. The measurement interval began the moment the tokenized input was fed into the model and ended once the final "end\-of\-sequence" token was generated\.

### III\-AInitial benchmarking of LLMs

We first report benchmark results for multiple LLMs evaluated without task\-specific fine\-tuning comprising of several online\-hosted and locally deployed models in terms of inference time and initial classification accuracy\. The benchmark was conducted using an expert\-curated dataset of predefined high\-level racing commands\. The dataset of 85 samples was designed for preliminary evaluation and was paired with a prompt that described all commands and enforced a structured output format to ensure consistent command interpretation\.

We evaluated OpenAI GPT\-4, GPT\-3\.5, and o3\-mini variants, listed in Table[I](https://arxiv.org/html/2609.21109#S3.T1)\. All models were accessed via the OpenAI444[https://developers\.openai\.com/api/reference/overview/](https://developers.openai.com/api/reference/overview/)API under a paid account plan, using default settings, including the temperature equal 1\.0\.

Additionally, we benchmarked locally deployed, open\-weight SLMs from the Gamma, DeepSeek, Llama, and Mistral families\. These experiments were run on an NVIDIA GeForce RTX 4090 GPU\.

The results, summarized in Table[I](https://arxiv.org/html/2609.21109#S3.T1), reveal a trade\-off between accuracy and latency\. Online models generally achieved higher classification accuracy but often exhibited prohibitively long response times, making them unsuitable for our target application\. In contrast, locally deployed models showed lower accuracy but significantly reduced inference latency\. Given the quick response time demand of the target application, latency was prioritized over accuracy\. Consequently,Mistral 7B\[[31](https://arxiv.org/html/2609.21109#bib.bib15)\],LLaMA\-3\.1 8B, andLLaMA\-3\.2 3B\[[32](https://arxiv.org/html/2609.21109#bib.bib32)\], the three models with the lowest prompt processing times, were selected for further fine\-tuning\. Initial accuracy deficiencies were considered secondary, as classification performance was expected to improve through fine\-tuning on a domain\-specific dataset\.

TABLE I:Initial benchmarking of online and local Models\. Highlighted models were selected for further fine\-tuning\.ModelAccuracy \[%\]Avg\. Latency \[s\]Online ModelsGPT\-4o\-2024\-11\-2076\.473\.38GPT\-4o\-2024\-08\-0674\.124\.16o3\-mini\-2025\-01\-3185\.8821\.08GPT\-4\-turbo\-2024\-04\-0981\.1815\.73GPT\-3\.5\-turbo\-012557\.655\.12Locally Deployed ModelsLlama3\.1\-8B8\.240\.61Llama3\.2\-3B2\.350\.44Deepseek\-R1\-7B5\.884\.07Deepseek\-R1\-14B16\.479\.86Deepseek\-R1\-32B21\.1834\.96Gemma2\-9B8\.240\.82Gemma2\-27B27\.061\.82Gemma3\-12B18\.821\.14Mistral\-7B27\.060\.64Mistral\-small\-22B16\.471\.31Mistral\-small\-24B24\.711\.15
### III\-BDataset Preparation for Model Fine\-Tuning

The small initial dataset used for benchmarking, described in Subsection[III\-A](https://arxiv.org/html/2609.21109#S3.SS1)was sufficient for preliminary evaluation\. However, it was too small for effective fine\-tuning of language models\. To address this limitation, we extended the dataset\. The augmentation process was performed using the GPT\-4 LLM, which demonstrated strong semantic understanding in the initial benchmarking experiments\. It was prompted to use various augmentation techniques\[[33](https://arxiv.org/html/2609.21109#bib.bib19),[34](https://arxiv.org/html/2609.21109#bib.bib18),[35](https://arxiv.org/html/2609.21109#bib.bib17),[36](https://arxiv.org/html/2609.21109#bib.bib16)\], including synonym replacement, paraphrasing, word reordering, and minor contextual additions\. These techniques increased linguistic diversity without introducing new command semantics, enabling effective fine\-tuning\. Beyond predefined commands, we integrated an out\-of\-scope \(OOS\) class to handle unrelated inputs\. This enhances the training of the classifier to increase system robustness through more reliable rejection of irrelevant prompts\. The resulting extended dataset, comprising 17 command classes and 1,645 samples, provides increased linguistic coverage by incorporating a broader range of vocabulary and syntactic structures\. It was partitioned into an 80/20 train\-test split, providing a training set of 1,308 samples\.

### III\-CFine\-Tuning and Model Optimization

The three models selected in Subsection[III\-A](https://arxiv.org/html/2609.21109#S3.SS1)for further fine\-tuning, were:Mistral 7B,LLaMA\-3\.1 8B, andLLaMA\-3\.2 3B\.

Training was implemented using theUnsloth555[https://unsloth\.ai](https://unsloth.ai/)framework to apply QLoRA, targeting all linear modules\[[37](https://arxiv.org/html/2609.21109#bib.bib13)\]\. Specifically, adaptations were applied to the attention projection layers \(q,k,v,oq,k,v,o\)\[[38](https://arxiv.org/html/2609.21109#bib.bib12)\]and the MLP layers \(g​a​t​e,u​p,d​o​w​ngate,up,down\)\[[39](https://arxiv.org/html/2609.21109#bib.bib14)\]\.

To identify the optimal configuration, we conducted a grid search over the following discrete hyperparameter space:

- •Learning Rates:1×10−51\\times 10^\{\-5\}and5×10−55\\times 10^\{\-5\}
- •Epochs:2 and 3
- •Gradient Accumulation Steps:4 and 8
- •LoRA Rank \(rr\):16 and 32 \(withα\\alphascaled accordingly\)

Fine\-tuning was performed in two stages\.

### III\-DFirst Stage Fine\-tuning

In the first phase, we evaluated the receptiveness of all three models towards fine\-tuning\. Given the large number of hyperparameter combinations, and to reduce computational cost, the initial fine\-tuning was conducted on a subset of five out of seventeen command classes extracted from the full training dataset \( Section[III\-B](https://arxiv.org/html/2609.21109#S3.SS2)\)\. Accordingly, the evaluation was performed on the corresponding subset of the test dataset\. All experiments in this phase were performed on an NVIDIA GeForce RTX 4090 GPU\. The results are presented in Table[II](https://arxiv.org/html/2609.21109#S3.T2)and identifiedMistral 7Bas the best\-performing model, achieving the highest accuracy of 76\.74 %\.

TABLE II:Initial Fine\-Tuning for Basic Commands
Best Configurations for Selected ModelsModelAccuracy \[%\]Avg\. Inference \[s\]EpochsGrad Acc StepsralphaLocally Deployed ModelsMistral\-7B76\.740\.92243232Llama3\.1\-8B75\.580\.88343232Llama3\.2\-3B73\.260\.67343232
### III\-ESecond Stage Fine\-Tuning

Mistral 7Bwas selected for full fine\-tuning on the complete training dataset, using the best\-performing hyperparameter configuration\. The full fine\-tuning and evaluation was conducted on an NVIDIA GeForce RTX 3060 Laptop GPU using the full train and test data as described in Section[III\-B](https://arxiv.org/html/2609.21109#S3.SS2)\. The best\-performing fine\-tuned variant of Mistral\-7B, configured as specified in Table[III](https://arxiv.org/html/2609.21109#S3.T3), achieved a 97\.63 % classification accuracy, which exceeded the performance of every evaluated cloud\-based model \(Table[I](https://arxiv.org/html/2609.21109#S3.T1)\)\. Moreover, with an average inference latency of 1\.39 s, it delivered faster response times, demonstrating that locally fine\-tuned open\-weight models can surpass the online\-based ones in both accuracy and efficiency\.

TABLE III:Best model results and hyperparameter after second stage fine\-tuningParameterValueModel NameMistral\-7B\-v0\.3\-bnb\-4bitAccuracy \[%\]97\.63Avg\. Inference \[s\]1\.39Learning Rate5×10−55\\times 10^\{\-5\}Epochs2LoRA Rank \(rr\)32LoRA Alpha \(α\\alpha\)32Gradient Accumulation Steps4OptimizerAdamW 8\-bit

## IVDISCUSSION

The experimental results demonstrate that, for narrowly scoped command classification tasks, domain\-specific fine\-tuning of lightweight local language models can outperform larger online\-hosted systems\. While cloud\-based models such as GPT\-4 variants provide strong zero\-shot intent and contextual understanding, they introduce an inference burden in a constrained command\-classification scenario\. Additionally, cloud\-based models introduce network dependency, making system performance sensitive to connectivity and external service conditions\. In contrast, our locally fine\-tuned Mistral\-7B model operates entirely on\-device and is optimized for domain\-specific phrasing patterns, resulting in more consistent behavior under controlled inference settings\.

Our findings further highlight the effectiveness of QLoRA\-based fine\-tuning for edge deployment scenarios\. Fine\-tuning 7B\-scale models using 4\-bit quantization enables high task\-specific performance on resource\-constrained hardware, such as a mobile laptop workstation equipped with an NVIDIA GeForce RTX 3060 GPU\. Our fine\-tuning approach improves the accuracy of the final candidate model from 27\.06 % to 97\.64 %\.

Despite these strengths, several limitations remain\. First, although the dataset was expanded through GPT\-4\-based augmentation to increase linguistic diversity, it remains partially synthetic and text\-based\. While the augmentation process introduced paraphrastic variation, it may not capture the full distribution of possible inputs, including incomplete sentences, hesitations, or unconventional formulations\. Consequently, future work should incorporate real\-world operator transcripts collected under authentic racing conditions to better reflect natural language usage and improve generalization\.

Moreover, the current system operates within a strictly predefined command space\. Although this constraint enhances safety and interpretability, it limits conversational flexibility\. Extending the system toward more adaptive dialogue capabilities would require hierarchical command validation and formally defined safety constraints to prevent unintended behavior\.

## VCONCLUSION & OUTLOOK

In this work, we presentedJarvis, an offline voice assistant architecture for high\-level control of autonomous racing vehicles\. By integrating wake\-word detection, local speech\-to\-text transcription, and a fine\-tuned lightweight language model for intent classification, we developed a system with quick response times and network\-independent operation, suitable for use as a voice assistant in high\-speed racing applications\.

The presented architecture provides an open\-source framework for an interactive voice assistant using speech\-to\-command classification\. Beyond racing, the approach is transferable to other robotic systems where deterministic performance, low latency, and network independence are essential\.

Future work includes multi\-turn dialogue and bidirectional telemetry querying\.

## References

- \[1\]J\. Betz, H\. Zheng, A\. Liniger, U\. Rosolia, P\. Karle, M\. Behl, V\. Krovi, and R\. Mangharam\(2022\)Autonomous vehicles on the edge: a survey on autonomous vehicle racing\.IEEE Open Journal of Intelligent Transportation Systems3\(\),pp\. 458–488\.External Links:[Document](https://dx.doi.org/10.1109/OJITS.2022.3181510)Cited by:[§I](https://arxiv.org/html/2609.21109#S1.p1.1)\.
- \[2\]H\. Liu, C\. Li, Q\. Wu, and Y\. J\. Lee\(2023\)Visual instruction tuning\.InAdvances in Neural Information Processing Systems,A\. Oh, T\. Naumann, A\. Globerson, K\. Saenko, M\. Hardt, and S\. Levine \(Eds\.\),pp\. 34892–34916\.Cited by:[§I\-B](https://arxiv.org/html/2609.21109#S1.SS2.p1.1)\.
- \[3\]C\. Cui, Y\. Ma, X\. Cao, W\. Ye, Y\. Zhou, K\. Liang, J\. Chen, J\. Lu, Z\. Yang, K\. Liao, T\. Gao, E\. Li, K\. Tang, Z\. Cao, T\. Zhou, A\. Liu, X\. Yan, S\. Mei, J\. Cao, Z\. Wang, and C\. Zheng\(2024\)A survey on multimodal large language models for autonomous driving\.InProceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision \(WACV\) Workshops,pp\. 958–979\.Cited by:[§I\-B](https://arxiv.org/html/2609.21109#S1.SS2.p1.1)\.
- \[4\]J\. Mao, Y\. Qian, H\. Zhao, and Y\. Wang\(2023\)GPT\-driver: learning to drive with gpt\.arXiv preprint arXiv:2310\.01415\.Cited by:[§I\-B](https://arxiv.org/html/2609.21109#S1.SS2.p1.1)\.
- \[5\]H\. Sha, Y\. Mu, Y\. Jiang, L\. Chen, C\. Xu, P\. Luo, S\. E\. Li, M\. Tomizuka, W\. Zhan, and M\. Ding\(2023\)LanguageMPC: large language models as decision makers for autonomous driving\.arXiv preprint arXiv:2310\.03026\.Cited by:[§I\-B](https://arxiv.org/html/2609.21109#S1.SS2.p1.1)\.
- \[6\]Z\. Xu, Y\. Zhang, E\. Xie, Z\. Zhao, Y\. Guo, K\. K\. Wong, Z\. Li, and H\. Zhao\(2024\)DriveGPT4: interpretable end\-to\-end autonomous driving via large language model\.IEEE Robotics and Automation Letters9\(10\),pp\. 8186–8193\.External Links:[Document](https://dx.doi.org/10.1109/LRA.2024.3440097)Cited by:[§I\-B](https://arxiv.org/html/2609.21109#S1.SS2.p1.1)\.
- \[7\]C\. Sima, K\. Renz, K\. Chitta, L\. Chen, H\. Zhang, C\. Xie, P\. Luo, A\. Geiger, and H\. Li\(2024\)DriveLM: driving with graph visual question answering\.InProceedings of the European Conference on Computer Vision \(ECCV\),pp\. 256–274\.Cited by:[§I\-B](https://arxiv.org/html/2609.21109#S1.SS2.p1.1)\.
- \[8\]L\. Chen, O\. Sinavski, J\. Hünermann, A\. Karnsund, A\. J\. Willmott, D\. Birch, D\. Maund, and J\. Shotton\(2024\)Driving with LLMs: fusing object\-level vector modality for explainable autonomous driving\.InProceedings of the IEEE International Conference on Robotics and Automation \(ICRA\),pp\. 14093–14100\.Cited by:[§I\-B](https://arxiv.org/html/2609.21109#S1.SS2.p1.1)\.
- \[9\]B\. Li, Y\. Wang, J\. Mao, B\. Ivanovic, S\. Veer, K\. Leung, and M\. Pavone\(2024\)Driving everywhere with large language model policy adaptation\.InProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition \(CVPR\),pp\. 14948–14957\.Cited by:[§I\-B](https://arxiv.org/html/2609.21109#S1.SS2.p1.1)\.
- \[10\]Y\. Yang, Q\. Zhang, C\. Li, D\. S\. Marta, N\. Batool, and J\. Folkesson\(2024\)Human‑centric autonomous systems with llms for user command reasoning\.InProceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision \(WACV\) Workshops,pp\. 988–994\.Cited by:[§I\-B](https://arxiv.org/html/2609.21109#S1.SS2.p2.1)\.
- \[11\]J\. He, S\. Chen, F\. Zhang, and Z\. Yang\(2024\)From words to actions: unveiling the theoretical underpinnings of llm‑driven autonomous systems\.InProceedings of the 41st International Conference on Machine Learning,pp\. 17807–17841\.External Links:[Link](https://proceedings.mlr.press/v235/he24a.html)Cited by:[§I\-B](https://arxiv.org/html/2609.21109#S1.SS2.p2.1)\.
- \[12\]M\. Seegert, K\. Moller, and J\. Betz\(2026\)Modular autonomy with conversational interaction: an llm‑driven framework for decision making in autonomous driving\.Note:arXiv:2601\.05806Cited by:[§I\-B](https://arxiv.org/html/2609.21109#S1.SS2.p2.1)\.
- \[13\]C\. Cui, Y\. Ma, S\. Park, Z\. Yang, Y\. Zhou, P\. Liu, J\. Lu, J\. Peng, J\. Zhang, R\. Zhang, L\. Li, Y\. Chen, J\. H\. Panchal, A\. Abdelraouf, R\. Gupta, K\. Han, and Z\. Wang\(2026\)LLM4AD: large language models for autonomous driving—concept, review, benchmark, experiments, and future trends\.Proceedings of the IEEE\(\),pp\. 1–36\.External Links:[Document](https://dx.doi.org/10.1109/JPROC.2026.3678829)Cited by:[§I\-B](https://arxiv.org/html/2609.21109#S1.SS2.p2.1)\.
- \[14\]OpenAI, J\. Achiam, S\. Adler, and et al\.\(2024\)GPT\-4 technical report\.External Links:2303\.08774Cited by:[§I\-B](https://arxiv.org/html/2609.21109#S1.SS2.p2.1)\.
- \[15\]Z\. Lu, X\. Li, D\. Cai, R\. Yi, F\. Liu, X\. Zhang, N\. D\. Lane, and M\. Xu\(2025\)Small language models: survey, measurements, and insights\.Note:arXiv:2409\.15790External Links:2409\.15790Cited by:[§I\-B](https://arxiv.org/html/2609.21109#S1.SS2.p3.1)\.
- \[16\]Y\. Gu, L\. Dong, F\. Wei, and M\. Huang\(2026\)MiniLLM: on\-policy distillation of large language models\.Note:arXiv:2306\.08543External Links:2306\.08543Cited by:[§I\-B](https://arxiv.org/html/2609.21109#S1.SS2.p3.1)\.
- \[17\]C\. Hsieh, C\. Li, C\. Yeh, H\. Nakhost, Y\. Fujii, A\. Ratner, R\. Krishna, C\. Lee, and T\. Pfister\(2023\)Distilling step\-by\-step\! outperforming larger language models with less training data and smaller model sizes\.InFindings of the Association for Computational Linguistics: ACL 2023,A\. Rogers, J\. Boyd\-Graber, and N\. Okazaki \(Eds\.\),pp\. 8003–8017\.Cited by:[§I\-B](https://arxiv.org/html/2609.21109#S1.SS2.p3.1)\.
- \[18\]S\. Wang, Y\. Fu, and J\. Kim\(2026\)Toward construction\-specialized, small language models: the interplay of domain adaptation, model scale and data volume\.Advanced Engineering Informatics69,pp\. 104035\.External Links:ISSN 1474\-0346,[Document](https://dx.doi.org/https%3A//doi.org/10.1016/j.aei.2025.104035)Cited by:[§I\-B](https://arxiv.org/html/2609.21109#S1.SS2.p3.1)\.
- \[19\]V\. Pal, M\. Kour, and G\. Sethi\(2025\)AI\-powered iot assistant for electric vehicles: fine\-tuning a small language model for intent recognition and response generation\.In2025 IEEE International Transportation Electrification Conference \(ITEC\-India\),Vol\.,pp\. 1–6\.External Links:[Document](https://dx.doi.org/10.1109/ITEC-India67626.2025.11439142)Cited by:[§I\-B](https://arxiv.org/html/2609.21109#S1.SS2.p3.1)\.
- \[20\]T\. Dettmers, A\. Pagnoni, A\. Holtzman, and L\. Zettlemoyer\(2023\)QLoRA: efficient finetuning of quantized llms\.Advances in Neural Information Processing Systems37\.Cited by:[§I\-B](https://arxiv.org/html/2609.21109#S1.SS2.p3.1)\.
- \[21\]M\. Abdin, J\. Aneja, H\. Awadalla, A\. Awadallah, A\. A\. Awan, and et al\.\(2024\)Phi\-3 technical report: a highly capable language model locally on your phone\.Note:arXiv:2404\.14219External Links:2404\.14219Cited by:[§I\-B](https://arxiv.org/html/2609.21109#S1.SS2.p3.1)\.
- \[22\]A\. Rista and A\. Kadriu\(2020\)Automatic speech recognition: a comprehensive survey\.SEEU Review15\(2\),pp\. 86–112\.External Links:[Document](https://dx.doi.org/10.2478/seeur-2020-0019),[Link](https://doi.org/10.2478/seeur-2020-0019)Cited by:[§I\-B](https://arxiv.org/html/2609.21109#S1.SS2.p4.1)\.
- \[23\]Md\. Nayeem, M\. S\. Tabrej, K\. J\. Deb, S\. Goswami, and Md\. A\. Hakim\(2025\)Automatic speech recognition in the modern era: architectures, training, and evaluation\.Note:arXiv:2510\.12827External Links:2510\.12827Cited by:[§I\-B](https://arxiv.org/html/2609.21109#S1.SS2.p4.1)\.
- \[24\]S\. Di Leo, L\. De Cicco, and S\. Mascolo\(2025\)Real\-time speech\-to\-text on edge: a prototype system for ultra\-low latency communication with ai\-powered nlp\.Information16\(8\),pp\. 685\.External Links:[Document](https://dx.doi.org/10.3390/info16080685)Cited by:[§I\-B](https://arxiv.org/html/2609.21109#S1.SS2.p4.1)\.
- \[25\]A\. A\. Soni\(2025\)Improving speech recognition accuracy using custom language models with the vosk toolkit\.Note:arXiv:2503\.21025External Links:2503\.21025Cited by:[§I\-B](https://arxiv.org/html/2609.21109#S1.SS2.p4.1)\.
- \[26\]A\. Radford, J\. W\. Kim, T\. Xu, G\. Brockman, C\. McLeavey, and I\. Sutskever\(2023\)Robust speech recognition via large\-scale weak supervision\.InProceedings of the 40th International Conference on Machine Learning,ICML’23\.Cited by:[§I\-B](https://arxiv.org/html/2609.21109#S1.SS2.p4.1)\.
- \[27\]S\. Hoffmann, S\. Sagmeister, T\. Betz, J\. Bongard, S\. Büttner, D\. Ebner, D\. Esser, G\. Jank, S\. Goblirsch, A\. Langmann, M\. Leitenstern, L\. Ögretmen, P\. Pitschi, A\. Schwehn, C\. Schröder, M\. Weinmann, F\. Werner, B\. Lohmann, J\. Betz, and M\. Lienkamp\(2026\)Head\-to\-head autonomous racing at the limits of handling in the a2rl challenge\.External Links:2602\.08571,[Link](https://arxiv.org/abs/2602.08571)Cited by:[§II](https://arxiv.org/html/2609.21109#S2.p1.1)\.
- \[28\]M\. Quigley, K\. Conley, B\. Gerkey, J\. Faust, T\. Foote, J\. Leibs, R\. Wheeler, and A\. Ng\(2009\)ROS: an open\-source robot operating system\.InICRA Workshop on Open Source Software,Cited by:[§II\-A](https://arxiv.org/html/2609.21109#S2.SS1.p1.1)\.
- \[29\]G\. Chen, C\. Parada, and G\. Heigold\(2014\)Small\-footprint keyword spotting using deep neural networks\.In2014 IEEE International Conference on Acoustics, Speech and Signal Processing \(ICASSP\),Vol\.,pp\. 4087–4091\.External Links:[Document](https://dx.doi.org/10.1109/ICASSP.2014.6854370)Cited by:[§II\-B](https://arxiv.org/html/2609.21109#S2.SS2.p1.1)\.
- \[30\]J\. Kim, J\. Kong, and J\. Son\(2021\)Conditional variational autoencoder with adversarial learning for end\-to\-end text\-to\-speech\.InProceedings of the 38th International Conference on Machine Learning,Vol\.139,pp\. 5530–5540\.Cited by:[§II\-B](https://arxiv.org/html/2609.21109#S2.SS2.p3.1)\.
- \[31\]A\. Q\. Jiang, A\. Sablayrolles, A\. Mensch, C\. Bamford, D\. S\. Chaplot, D\. de las Casas, F\. Bressand, G\. Lengyel, G\. Lample, L\. Saulnier, L\. R\. Lavaud, M\. Lachaux, P\. Stock, T\. L\. Scao, T\. Lavril, T\. Wang, T\. Lacroix, and W\. E\. Sayed\(2023\)Mistral 7b\.Note:arXiv:2310\.06825External Links:2310\.06825Cited by:[§III\-A](https://arxiv.org/html/2609.21109#S3.SS1.p4.1)\.
- \[32\]A\. Grattafiori, A\. Dubey, A\. Jauhri, A\. Pandey, A\. Kadian, and et al\.\(2024\)The llama 3 herd of models\.Note:arXiv:2407\.21783External Links:2407\.21783Cited by:[§III\-A](https://arxiv.org/html/2609.21109#S3.SS1.p4.1)\.
- \[33\]Z\. Wang, P\. Wang, K\. Liu, P\. Wang, Y\. Fu, C\. Lu, C\. C\. Aggarwal, J\. Pei, and Y\. Zhou\(2026\)A comprehensive survey on data augmentation\.IEEE Transactions on Knowledge and Data Engineering38\(1\),pp\. 47–66\.External Links:[Document](https://dx.doi.org/10.1109/TKDE.2025.3622600)Cited by:[§III\-B](https://arxiv.org/html/2609.21109#S3.SS2.p1.1)\.
- \[34\]F\. Gao, J\. Zhu, L\. Wu, Y\. Xia, T\. Qin, X\. Cheng, W\. Zhou, and T\. Liu\(2019\)Soft contextual data augmentation for neural machine translation\.InProceedings of the 57th Annual Meeting of the Association for Computational Linguistics,pp\. 5539–5544\.External Links:[Document](https://dx.doi.org/10.18653/v1/P19-1555)Cited by:[§III\-B](https://arxiv.org/html/2609.21109#S3.SS2.p1.1)\.
- \[35\]A\. Karimi, L\. Rossi, and A\. Prati\(2021\)AEDA: an easier data augmentation technique for text classification\.InFindings of the Association for Computational Linguistics: EMNLP 2021,pp\. 2748–2754\.External Links:[Document](https://dx.doi.org/10.18653/v1/2021.findings-emnlp.234)Cited by:[§III\-B](https://arxiv.org/html/2609.21109#S3.SS2.p1.1)\.
- \[36\]J\. Wei and K\. Zou\(2019\)EDA: easy data augmentation techniques for boosting performance on text classification tasks\.InProceedings of the 2019 Conference on Empirical Methods in Natural Language Processing,pp\. 6382–6388\.External Links:[Document](https://dx.doi.org/10.18653/v1/D19-1670)Cited by:[§III\-B](https://arxiv.org/html/2609.21109#S3.SS2.p1.1)\.
- \[37\]E\. J\. Hu, Y\. Shen, P\. Wallis, Z\. Allen\-Zhu, Y\. Li, S\. Wang, L\. Wang, and W\. Chen\(2021\)LoRA: low\-rank adaptation of large language models\.Note:arXiv:2106\.09685Cited by:[§III\-C](https://arxiv.org/html/2609.21109#S3.SS3.p2.1)\.
- \[38\]A\. Vaswani, N\. Shazeer, N\. Parmar, J\. Uszkoreit, L\. Jones, A\. N\. Gomez, Ł\. Kaiser, and I\. Polosukhin\(2017\)Attention is all you need\.InAdvances in Neural Information Processing Systems,Vol\.30,pp\.\.Cited by:[§III\-C](https://arxiv.org/html/2609.21109#S3.SS3.p2.1)\.
- \[39\]N\. Shazeer\(2020\)GLU variants improve transformer\.Note:arXiv:2002\.05202External Links:2002\.05202Cited by:[§III\-C](https://arxiv.org/html/2609.21109#S3.SS3.p2.1)\.