Incremental Learning Reveals Declining Plasticity in Neural Networks for Traffic Classification

Ensuring cybersecurity and effective network governance presents a growing challenge as digital environments become ever more complex, and accurate network traffic classification is crucial for tasks ranging from optimising quality of service to detecting malicious activity. Tian Qin, Guang Cheng, and Zihan Chen, from Southeast University, alongside their colleagues, address a key limitation in current approaches: the tendency of deep learning models to lose their ability to adapt as new types of encrypted network traffic emerge. Their research introduces PRIME, a novel framework that monitors a model’s capacity to learn and dynamically adjusts its size to maintain performance, effectively overcoming the problem of declining plasticity. This innovation significantly improves the accuracy of encrypted traffic classification across diverse network conditions and evolving threat landscapes, offering a robust solution for maintaining network security in a dynamic world.

Network traffic classification analyses attributes such as application categories and malicious intent, supporting network management services like QoS optimisation, intrusion detection, and targeted billing. As the prevalence of traffic encryption increases, deep learning models are relied upon for content-agnostic analysis of packet sequences. However, the emergence of new services and attack variants often leads to incremental tasks for these models, and recent studies indicate that neural networks experience declining plasticity as the number of tasks increases.

Encrypted Traffic Classification via Continual Learning

This research focuses on encrypted traffic classification, the process of identifying the application or service generating network traffic even when it is encrypted. This is vital for network security, quality of service, and overall network management. Traditional methods, which rely on inspecting the contents of network packets, become ineffective when traffic is encrypted. The work explores continual learning, a technique that allows models to learn new information without forgetting previously acquired knowledge, crucial because network applications and traffic patterns are constantly evolving.

The research employs deep learning models, specifically neural networks, to classify network traffic. These models utilize convolutional neural networks for extracting relevant features and transformers, which excel at capturing long-range dependencies, to analyse traffic patterns. Generative models are also used to create synthetic data, helping to mitigate catastrophic forgetting, where new learning overwrites old knowledge. The process involves extracting features from network traffic, including packet header information, flow-based statistics, and payload-agnostic characteristics, and employing continual learning strategies like replay buffers and generative replay to retain previously learned information while adapting to new applications. The research aims to improve the accuracy and robustness of encrypted traffic classification, enabling models to adapt to evolving network conditions and new applications. By mitigating catastrophic forgetting, the models can maintain high accuracy across all learned tasks, even as the number of categories increases, contributing to a more secure and manageable network infrastructure with solutions for identifying malicious traffic, prioritising network resources, and enhancing intrusion detection systems.

Plasticity Monitoring Improves Network Traffic Classification

This research introduces PRIME, a new framework designed to enhance incremental learning for network traffic classification. As networks evolve and new applications emerge, the ability to continuously update traffic classification models without forgetting previously learned information is essential, yet challenging. PRIME addresses the issue of declining plasticity in neural networks by actively monitoring the model’s ability to learn, assessing plasticity through two key indicators: the effective rank of the model’s parameters and the diversity of neuron activation. By observing these metrics, the framework can determine when the model’s capacity is becoming limited and proactively increase its size, adding new parameters before performance degrades.

This expansion is achieved by carefully replicating and slightly modifying existing parameters, ensuring the new additions are active and contribute to learning without disrupting existing knowledge. The results demonstrate a significant improvement in performance compared to other incremental learning algorithms across multiple encrypted traffic datasets and various scenarios involving the addition of new traffic categories, achieved with only a minimal increase in the model’s overall size. A noteworthy aspect of PRIME is its ability to avoid catastrophic forgetting, where learning new tasks leads to a loss of accuracy on previously learned ones. By carefully monitoring plasticity and expanding the model’s capacity when needed, PRIME maintains high accuracy across all learned tasks, even as the number of categories increases, representing a significant advancement in incremental learning with a robust and scalable solution for adapting to the ever-changing landscape of network traffic.

Dynamic Plasticity Maintains Network Traffic Classification

This research presents PRIME, a new framework designed to address the issue of declining plasticity in neural networks used for incremental learning of network traffic classification. The team observed that as models learn to classify increasing numbers of traffic types, their ability to adapt diminishes, hindering performance on new tasks. PRIME tackles this by monitoring the effective rank of model parameters and the proportion of active neurons, dynamically adjusting the model’s scale to maintain sufficient plasticity. Experiments across multiple encrypted traffic datasets and varying incremental learning scenarios demonstrate that PRIME significantly outperforms existing incremental learning algorithms, achieving improved accuracy and reduced forgetting of previously learned tasks while minimizing increases in model complexity.

The researchers developed a formula combining parameter sensitivity and information density to accurately determine when to expand the model’s capacity. The authors acknowledge that the performance gains are dependent on the specific datasets and incremental scenarios tested. They also note that determining optimal expansion thresholds may require further tuning for different applications, and future work could explore adaptive strategies for determining these thresholds and investigate the application of PRIME to other continual learning problems beyond network traffic classification.