LLM-Powered System Answers HVAC Questions For Non-Expert Users

JARVIS, a novel question-answering framework utilising large language models, enhances interaction with heating, ventilation, and air conditioning systems. It translates user queries into executable instructions, retrieving and processing sensor data via SQL. Evaluations using real-world data demonstrate improved accuracy and response quality compared to existing methods.
The increasing complexity of building management systems necessitates intuitive interfaces for accessing crucial operational data, particularly for those without specialist knowledge. Researchers are now exploring the potential of large language models (LLMs) to provide precisely this functionality, enabling natural language interaction with sensor networks. Sungmin Lee, Minju Kang, and colleagues, from Yonsei University and LG Electronics, alongside Pei Zhang from the University of Michigan, detail their work in a paper entitled ‘LLM-based Question-Answer Framework for Sensor-driven HVAC System Interaction’. They present JARVIS, a two-stage framework designed to translate user queries into actionable instructions for retrieving and processing data from heating, ventilation, and air conditioning (HVAC) systems, ultimately delivering accurate and readily understandable responses.

The increasing sophistication of building management systems generates substantial data streams from numerous sensors, particularly within Heating, Ventilation, and Air Conditioning (HVAC) systems, offering significant potential for optimising energy consumption, proactively identifying faults, and enhancing occupant comfort. However, a gap exists between the data generated and the ability of non-expert users to effectively utilise it for informed decision-making, hindering widespread adoption of data-driven building management strategies. The projected growth of the sensor-driven smart building sector, estimated at $8.31 billion by 2029, underscores the demand for more accessible and user-friendly interfaces capable of translating complex data into actionable insights.

Traditional methods of interacting with these systems rely heavily on manual data querying and expert analysis, proving costly and difficult to scale for large deployments, while automated rule-based systems offer a degree of scalability but struggle with the nuances of natural language and the complexity of real-world HVAC operations. Recent advances in large language models (LLMs) present a potential solution by combining the scalability of automation with improved natural language understanding, enabling more intuitive and efficient interaction with building management systems. HVAC systems generate data that is constantly updated, rendering static databases unsuitable for providing accurate and timely responses, and LLMs, trained on fixed datasets, cannot directly access or process these dynamic streams, necessitating alternative methods for data injection. This requires frameworks capable of retrieving, processing, and integrating real-time sensor data into the LLM’s reasoning process, ensuring the information provided is current and relevant. Effective data retrieval requires navigating structured databases using database-oriented approaches, such as SQL, and raw sensor data often requires domain-specific preprocessing, including statistical summarisation and temporal filtering, to derive meaningful insights.

JARVIS: A Question-Answering Framework for HVAC Systems

JARVIS, a novel question-answering framework leveraging large language models, addresses the growing need for intuitive interaction with complex HVAC systems, particularly for users lacking specialised expertise, moving beyond simple data retrieval to enable coherent, context-aware responses to user queries regarding HVAC performance and status. The system achieves this through a two-stage process, beginning with an ‘Expert-LLM’ that translates natural language questions into structured execution instructions, ensuring the LLM understands the intent behind the question and can formulate a plan to retrieve the necessary information. The second stage involves an ‘Agent’, responsible for executing those instructions by querying the HVAC system’s database using SQL, performing statistical analysis on the retrieved data, and ultimately generating a comprehensive and understandable response for the user.

A key innovation of JARVIS lies in its ability to handle the unique challenges posed by real-time sensor data and domain-specific knowledge, overcoming the limitations of traditional LLM-based QA systems that often struggle with frequently updating information and require extensive retraining to incorporate new knowledge. JARVIS overcomes this limitation through an adaptive context injection strategy, dynamically incorporating relevant HVAC system information, deployment specifics, and recent sensor data into the LLM’s context window, ensuring it has access to the most up-to-date and pertinent information. Furthermore, the system incorporates a parameterized SQL builder and executor to enhance the reliability of data access, automatically constructing queries based on the user’s question and the system’s knowledge, reducing the risk of errors and ensuring data integrity.

To ensure consistency and coherence across multi-stage responses, JARVIS employs a bottom-up planning scheme, breaking down complex questions into smaller, more manageable sub-questions, and then systematically addressing each sub-question to build a complete and logical answer. Evaluation of JARVIS using real-world data from a commercial HVAC system and a curated QA dataset by HVAC experts demonstrates its effectiveness, consistently outperforming baseline and ablation variants in both automated metrics and user-centred assessments.

Enhanced Accessibility and Future Development

The research presents JARVIS, a novel question-answering (QA) framework that leverages large language models (LLMs) to facilitate interaction with heating, ventilation, and air conditioning (HVAC) systems. This addresses the need for accessible, real-time insights from complex sensor data, particularly for users lacking specialist HVAC knowledge. JARVIS distinguishes itself through a two-stage architecture, employing an ‘Expert-LLM’ to convert natural language queries into structured instructions and an ‘Agent’ to execute these instructions via SQL-based data retrieval and processing, enabling coherent and context-aware responses crucial for effective HVAC system interaction. A key innovation lies in JARVIS’s adaptive context injection strategy, dynamically integrating relevant HVAC-specific and deployment-specific information, ensuring the LLM operates with the necessary background knowledge.

Furthermore, the framework incorporates a parameterized SQL builder and executor, enhancing the reliability of data access and minimising errors during data retrieval from time-series databases such as TimescaleDB (Timescale Inc., 2017) and Pandas (Reback et al., 2020), particularly important given the volume and complexity of sensor data generated by modern HVAC systems. The system’s bottom-up planning scheme ensures consistency across multi-stage response generation, breaking down complex queries into smaller, manageable steps, improving the accuracy and interpretability of the final response. The research builds upon existing work in LLM-based QA, including the development of instruction-following models (Ouyang et al., 2022) and chain-of-thought prompting (Wei et al., 2022), as well as advancements in text-to-SQL generation (Zhu et al., 2024).

Evaluation utilising real-world data from a commercial HVAC system and a curated QA dataset demonstrates JARVIS’s superior performance compared to baseline models and ablation studies, confirming the system’s ability to deliver accurate and interpretable responses across a diverse range of queries. Future work should focus on expanding the scope of JARVIS to encompass more complex HVAC system configurations and integrate with a wider range of sensor data sources. Investigating methods for automated knowledge base construction and refinement, leveraging techniques such as reinforcement learning, could further enhance the framework’s adaptability and reduce reliance on manually curated knowledge. Exploring the potential for proactive anomaly detection and predictive maintenance, based on insights derived from the QA interface, represents another promising avenue for future research, while research could investigate the framework’s scalability and robustness in handling large-scale deployments with numerous sensors and complex system interactions. Developing methods for explainable AI (XAI) to provide users with a clear understanding of the reasoning behind JARVIS’s responses would further enhance trust and usability.