Llama-1b Achieves 0.7159 F1-Score for IoT Network Attack Detection

Researchers are tackling the escalating threat of cyberattacks targeting the ever-expanding Internet of Things (IoT) with a novel approach to intrusion detection. Piyumi Bhagya Sudasinghe, Kushan Sudheera Kalupahana Liyanage, and Harsha S. Gardiyawasam Pussewalage, from the University of Ruhuna and the University of Agder (UiA) respectively, demonstrate the potential of lightweight Large Language Models (LLMs) to identify both known and previously unseen threats. Their work addresses a critical limitation of traditional machine learning models, which struggle with zero-day attacks and require constant retraining , instead, they utilise techniques like Quantized Low-Rank Adaptation (QLoRA) and Retrieval-Augmented Generation (RAG) to create adaptable, resource-efficient systems. Achieving comparable performance to established methods like Random Forest on known attacks, and a remarkable 42.63% accuracy on unseen attacks without retraining, this research signifies a significant step towards robust and scalable IoT security solutions.

This breakthrough allows for the detection of both known and, crucially, previously unseen “zero-day” attacks without the need for constant retraining.

The study focused on transforming network traffic features into compact natural-language prompts, enabling efficient adaptation of the LLMs even on hardware with constrained resources. Experiments were conducted using the CICIoT2023 dataset, a benchmark for IoT security research, to evaluate the performance of the proposed system. A QLoRA-tuned LLaMA-1B model achieved an F1-score of 0.7124, a result remarkably comparable to the 0.7159 F1-score attained by a Random Forest (RF) baseline for identifying known attacks. This demonstrates the potential of LLMs to match the performance of established machine learning techniques in a critical security application.
Furthermore, the research unveiled a significant capability for zero-shot attack detection. By incorporating RAG, the system achieved 42.63% accuracy in identifying unseen attack types without any additional training. This zero-shot capability is a major innovation, as it allows the system to adapt to evolving threats in real-time, a feature lacking in traditional machine learning models that require retraining with new data. The work establishes that retrieval-enhanced lightweight LLMs represent a promising pathway towards next-generation IoT intrusion detection systems, offering both adaptability and resource efficiency.

This research presents a unified LLM-based framework capable of handling both known and unknown attacks, addressing a key limitation of current cybersecurity solutions. The team explored several decoder-only LLMs, GPT-2, LLaMA-3.2-1B, Meta-LLaMA-3-8B, and Mistral-v0.3-7B, adapting them with QLoRA for multi-class IoT attack detection. By reformulating numerical network features into concise natural language prompts, the system effectively bridges the gap between structured data and the semantic understanding of LLMs. The findings will be presented at the 7th Computing, Communications and IoT Applications Conference (ComComAp 2025) in Madrid, Spain, in December 2025, signifying its contribution to the field.

QLoRA-LLaMA-1B matches random forest IoT detection accuracy

Scientists achieved an F1-score of 0. Experiments focused on transforming network traffic features into concise natural language prompts, enabling efficient adaptation of the LLM even with constrained hardware resources. The team meticulously measured the model’s ability to discern established attack patterns, validating its effectiveness against previously identified threats.

Results demonstrate a significant breakthrough in zero-shot attack detection, with the system achieving 42.63% accuracy on unseen attack types without any additional training. This was accomplished through the integration of Retrieval-Augmented Generation (RAG), allowing the LLM to leverage external knowledge and reason about novel threats. Data shows that the RAG-enhanced system effectively generalizes to previously unknown attacks by retrieving relevant information and applying contextual understanding. Scientists recorded this zero-shot capability as a key advantage over traditional machine learning models, which typically require retraining to address emerging threats.

The work details a unified LLM-based framework capable of handling both known and unknown attacks, overcoming the limitations of conventional intrusion detection systems. Researchers transformed numerical network features into natural language, facilitating the adaptation of decoder-only LLMs like LLaMA-1B, GPT-2, LLaMA-3.2-1B, Meta-LLaMA-3-8B, and Mistral-v0.3-7B using QLoRA fine-tuning. Measurements confirm that this structured-to-text conversion is crucial for efficient processing and adaptation under resource constraints, making the approach viable for deployment on IoT devices. Tests prove the effectiveness of the methodology across multiple LLM architectures, highlighting the versatility of the proposed framework. The breakthrough delivers a practical solution for next-generation IoT intrusion detection, offering adaptability and resource efficiency. Experiments conducted on the CICIoT2023 dataset revealed that a QLoRA-tuned LLaMA-1B model achieved an F1-score of 0.7124, closely matching the performance of a Random Forest baseline (0.7159) when detecting known attacks. Furthermore, the RAG-enhanced system attained 42.63% accuracy in identifying unseen attack types without requiring additional training, showcasing a practical zero-shot capability that surpasses traditional machine learning methods.

This combination of parameter efficiency and retrieval-based context grounding presents a scalable solution for the dynamic landscape of IoT environments. The authors acknowledge a limitation in the study’s scope, noting that results were obtained using a single dataset and broader validation across diverse IoT benchmarks is necessary. Future research will focus on refining retrieval strategies, expanding knowledge bases, and exploring lightweight ensemble methods to improve the detection of subtle or overlapping attack patterns.