Large Language Models for Enhancing Human-Machine Interaction in Autonomous Vehicles: A Comprehensive Survey

Abstract

Effective human–machine interaction is essential for ensuring the safety, reliability, and public acceptance of autonomous vehicles (AVs). Although large language models (LLMs) are increasingly influencing AV systems through their advanced natural language understanding and reasoning capabilities, their specific contributions to HMI have not yet been systematically examined. To address this gap, this paper presents the first comprehensive survey of LLM applications in automotive HMI. We structure the discussion around three functional domains: Human Understanding and Modelling (MODEL), in which LLMs interpret human states and intentions; Context-Aware Reasoning and Explanation (REASON), in which they support safety-critical reasoning and interaction-relevant explanation; and Adaptive Interaction Execution (ACT), in which they enable bidirectional natural-language communication between users and vehicles. Across these domains, we show that LLMs are shifting automotive HMI from static, rule-based interfaces toward semantically grounded, adaptive, and context-aware interaction. We further discuss the human-factor implications of this transition, including behavioral adaptation, trust calibration, and over-reliance. Critical challenges identified include interaction robustness, safety validation gaps, and domain-specific technical constraints. Taken together, this review establishes a conceptual framework to guide researchers and practitioners in advancing next-generation HMI systems that promote safer and more user-centered autonomous driving.

Links

DOI: 10.1016/j.trc.2026.105819 · Literature repository: GitHub