Transmode-LLM Achieves 12.50% Improvement in Travel Mode Prediction Accuracy

Accurately modelling travel behaviour remains a crucial challenge for effective transport planning. Meijing Zhang from Singapore University of Technology and Design, alongside Ying Xu, present TransMode-LLM, a novel framework combining statistical analysis with large language models to predict travel mode choice from survey data. This research is significant because it demonstrates how LLMs can achieve competitive accuracy compared to traditional methods, and importantly, reveals that carefully crafted ‘domain-enhanced prompting’ can boost performance by up to 42.9% in certain models , although the benefits vary depending on the LLM’s architecture. The findings offer valuable insights for future academic research and could substantially improve the development of data-driven transport policies.

LLMs predict travel choices via statistical encoding

Scientists have unveiled TransMode-LLM, a groundbreaking framework integrating statistical analysis with large language models (LLMs) to accurately predict travel mode choices from survey data. This innovative approach addresses limitations in traditional transportation planning methods, which often struggle to capture the complex contextual factors influencing individual decisions. The research team developed a three-phase system, beginning with statistical analysis to pinpoint key behavioural features, followed by natural language encoding to transform structured data into contextual descriptions, and culminating in LLM adaptation for travel mode prediction using zero-shot, one-shot, and few-shot learning paradigms. This allows the models to not only identify patterns but also reason about travel choices in a more human-like manner, potentially revolutionising how transportation systems are planned and optimised.
The core innovation lies in translating numerical travel data into natural language, enabling LLMs to leverage their pre-trained knowledge of human decision-making processes. Researchers meticulously identified crucial behavioural features through literature review and feature importance analysis, then converted these into descriptive narratives for LLM processing. Experiments were conducted using both general-purpose LLMs, GPT-4o and GPT-4o-mini, and reasoning-focused models, o3-mini and o4-mini, with varying sample sizes on real-world travel survey data to rigorously test the framework’s performance. This systematic evaluation ensures the robustness and generalizability of the findings, demonstrating the potential for widespread application in diverse transportation contexts.

Extensive experimental results demonstrate that the LLM-based approach achieves competitive accuracy when compared to state-of-the-art baseline classifier models. Significantly, few-shot learning dramatically improves prediction accuracy, with the o3-mini model exhibiting consistent improvements of up to 42.9% when provided with just five examples. This highlights the LLMs’ capacity to rapidly learn and adapt from limited data, a crucial advantage in dynamic transportation environments. Furthermore, the study explored domain-enhanced prompting, revealing divergent effects across different LLM architectures, with GPT-4o achieving improvements ranging from 2.27% to 12.50%.

However, the research also established that domain knowledge enhancement doesn’t universally improve performance for reasoning-oriented models like o3-mini and o4-mini, indicating a nuanced relationship between model architecture and the effectiveness of specific prompting strategies. This detailed analysis provides valuable insights for optimising LLM performance in travel behaviour modelling, paving the way for more accurate and adaptable transportation policy-making and academic research in the future. The team’s hierarchical evaluation system, complemented by F1-Macro and F1-Weighted scores, further confirms the effectiveness of few-shot learning in addressing class imbalance, reducing the accuracy-F1-macro gap by up to 71%.

Statistical-LLM Integration for Travel Mode Prediction improves accuracy

Scientists developed TransMode-LLM, a novel framework integrating statistical methods with large language models (LLMs) to predict travel modes from survey data. The research team first employed statistical analysis, guided by existing literature, to pinpoint key behavioural features influencing travel choices, subsequently assessing feature importance to refine the selection process. These identified variables were then transformed into contextual descriptions using natural language encoding, effectively bridging the gap between structured data and LLM processing capabilities. This innovative approach allows the LLMs to leverage contextual information often overlooked by traditional modelling techniques.

Experiments utilized both general-purpose LLMs, GPT-4o and GPT-4o-mini, alongside reasoning-focused models, o3-mini and o4-mini, to evaluate the framework’s performance across diverse architectures. The study systematically varied sample sizes of travel survey data to assess the robustness of the predictions under different data availability scenarios. Researchers then implemented multiple learning paradigms, including zero-shot, one-shot, and few-shot learning, to optimise the LLMs’ predictive accuracy without extensive retraining. Notably, few-shot learning demonstrated significant improvements, with the o3-mini model achieving up to a 42.9% increase in accuracy when provided with just 5 example cases.

Further methodological innovation involved domain-enhanced prompting, where the LLMs received transportation-specific knowledge to refine their predictions. The team observed divergent effects of this prompting technique; GPT-4o, a general-purpose model, benefited from improvements ranging from 2.27% to 12.50%, while reasoning-oriented models, o3-mini and o4-mini, did not consistently improve with the addition of domain knowledge. This nuanced finding highlights the importance of tailoring prompting strategies to specific LLM architectures. The work demonstrates that LLM-based approaches can achieve competitive accuracy compared to state-of-the-art baseline classifiers, advancing the application of LLMs in travel behaviour modelling and offering valuable insights for future transportation policy-making.

LLM predicts travel modes with high accuracy

Scientists have developed TransMode-LLM, a novel framework integrating statistical methods with Large Language Models (LLMs) to predict travel modes from survey data. The research team meticulously analysed travel survey data, identifying key behavioural features as a crucial first step in the process. Subsequently, structured data was transformed into contextual descriptions using natural language encoding, enabling LLMs to interpret complex travel patterns. Experiments revealed that this LLM-based approach achieves competitive accuracy when compared to state-of-the-art baseline classifier models, marking a significant advancement in transportation planning.

The team measured prediction accuracy using various LLM architectures, including GPT-4o, GPT-4o-mini, o3-mini, and o4-mini, across differing sample sizes. Results demonstrate that few-shot learning substantially improves prediction performance, with the o3-mini model consistently exhibiting improvements of up to 42.9% when provided with just 5 examples. This highlights the LLM’s capacity to learn and adapt quickly from limited data, offering a practical advantage for real-world applications. Data shows that the framework effectively captures nuanced travel behaviours, potentially leading to more accurate transportation models.

Further investigation focused on domain-enhanced prompting, revealing divergent effects across different LLM architectures. Specifically, general-purpose models like GPT-4o benefited from domain knowledge, achieving performance improvements ranging from 2.27% to 12.50%. However, reasoning-oriented models (o3-mini and o4-mini) did not consistently improve with the addition of domain-specific information, suggesting that their inherent reasoning capabilities may already be sufficient for accurate predictions. Measurements confirm that the framework’s adaptability allows for tailored implementation based on the chosen LLM.

The study also incorporated a hierarchical evaluation system, complemented by F1-Macro and F1-Weighted scores, to assess performance comprehensively. F1 score analysis revealed that few-shot learning effectively addresses class imbalance, reducing the gap between accuracy and F1-Macro scores by up to 71%. This is a critical finding, as real-world travel data often exhibits uneven distribution across different modes. The breakthrough delivers a promising methodology for both academic research and future transportation policy-making, offering valuable insights into traveller behaviour.

LLM Adaptation Boosts Travel Mode Prediction accuracy significantly

Scientists have developed TransMode-LLM, a new framework integrating statistical methods with large language models (LLMs) to predict travel mode choice from survey data. The framework functions by first identifying key behavioural features statistically, then encoding this structured data into contextual descriptions, and finally utilising LLM adaptation for travel mode prediction through various prompting techniques. Researchers evaluated TransMode-LLM using general-purpose and reasoning-focused LLMs with differing sample sizes, analysing real-world travel survey data. Extensive experiments revealed that the LLM-based approach achieves accuracy comparable to existing state-of-the-art classifiers.

Furthermore, one/few-shot learning significantly improved prediction accuracy, with some models showing up to a 42.9% improvement using only five examples. Domain-enhanced prompting yielded varied results; it improved performance in general-purpose models like GPT-4o (by 2.27% to 12.50%), but did not consistently benefit reasoning-oriented models. The authors acknowledge that domain knowledge integration should be adaptive and model-aware, as more capable models may already possess relevant transportation knowledge. This study demonstrates the potential of bridging traditional quantitative modelling with emerging LLM techniques for travel mode prediction, suggesting a shift from training-based to prompt-based prediction in transportation.

Practically, the findings indicate LLMs can serve as both predictive tools and interpretable frameworks offering insights into complex travel decisions. The primary benefit of few-shot learning appears to be addressing class imbalance in travel survey data, reducing the gap between accuracy and F1-macro scores by up to 71%. Future research could focus on refining domain knowledge integration strategies to optimise performance across different LLM architectures and exploring the interpretability of LLM-driven travel behaviour insights.