Smart Meter Data Achieves Behaviour-Centric Consumer Segmentation with CROCS Clustering

Demand-Side Management is gaining prominence as a cost-effective solution for balancing increasingly complex electricity systems, driven by renewable energy integration and electrification. Luke W. Yerbury, Ricardo J.G.B. Campello, G. C. Livingston Jr, and colleagues present a new framework, Clustered Representations Optimising Consumer Segmentation (CROCS), designed to improve consumer segmentation using smart meter data. The researchers address limitations in current clustering methods, which struggle with behavioural diversity, data anomalies, and scalability. CROCS employs a two-stage process, creating representative load sets and utilising a novel distance measure, to identify and interpret distinct consumer groups. This innovative approach demonstrates improved accuracy, robustness, and efficiency in analysing smart meter data, potentially enabling more effective and targeted Demand Response programs.

The researchers address limitations in current clustering methods, which struggle with behavioural diversity, data anomalies, and scalability. CROCS employs a two-stage process, creating representative load sets and utilising a novel distance measure, to identify and interpret distinct consumer groups. This innovative approach demonstrates improved accuracy, robustness, and efficiency in analysing smart meter data, potentially enabling more effective and targeted Demand Response programs. Some studies utilise a two-stage approach, beginning with pre-clustering or initial segmentation, followed by refinement. Dynamic Time Warping (DTW) is often used for comparing time series data, particularly when the timing of events varies between profiles. External validity measures, such as the Adjusted Rand Index and Normalized Mutual Information, alongside internal validation metrics like the Davies-Bouldin and Calinski-Harabasz indices, are used to evaluate cluster quality. Clustering finds application in load profiling, identifying customer segments, and demand response analysis. The research addresses limitations in existing methods by capturing the diversity of consumer behaviours with greater accuracy and robustness. Experiments utilising both synthetic and real Australian smart meter datasets demonstrate CROCS’ ability to represent intra-consumer variability, uncovering both synchronous and asynchronous behavioural similarities within consumption patterns. The team measured performance using metrics including Adjusted Mutual Information (AMI) and Adjusted Rand Index (ARI) to assess the quality of the resulting consumer clusters.

CROCS consistently outperformed existing methods in handling anomalies and missing data, while maintaining efficient scalability through natural parallelisation. The framework generates Representative Load Sets (RLS), compact summaries of typical daily consumption, and employs a novel set-to-set measure, the Weighted Sum of Minimum Distances (WSMD), to compare these RLSs. This approach accounts for both the prevalence and similarity of behaviours, offering a more nuanced understanding of consumer groups. Further analysis involved community detection on the WSMD-induced graph, revealing higher-order prototypes that embody shared diurnal behaviours.

Tests confirm that CROCS effectively captures the complexities of energy consumption, even in the presence of irregularities or incomplete data. The study leveraged datasets encompassing nearly 1.5 billion households globally, reflecting the widespread deployment of smart metering infrastructure. This work highlights CROCS as a robust and extensible framework for strengthening the analytical foundations of DSM and DR strategies. Measurements confirm that the CROCS framework provides insights at both consumer and system levels, extending to a wide range of energy applications. The research demonstrates the potential to optimise grid management by better understanding and responding to consumer behaviour, particularly as electricity grids transition towards increased renewable energy sources and electrification of transport and heating. CROCS addresses limitations in existing methods by utilising a representative load set to summarise individual consumption behaviours and employing a weighted sum of minimum distances to compare these sets, accounting for both prevalence and similarity of behaviours. Community detection further refines the resulting clusters, offering enhanced interpretability of consumer groups and their shared diurnal patterns. Extensive testing using both simulated and real-world Australian smart meter data demonstrates CROCS’s ability to capture variability within individual consumer behaviour, identify both synchronous and asynchronous similarities, and maintain robustness in the presence of data anomalies or missing information.

The framework also exhibits efficient scalability through parallel processing. Authors acknowledge that the number of initial clusters requires careful consideration, but demonstrate a robust approach of using a single, overestimated value for all consumers, mitigating the need for individual optimisation. Future work could explore the application of CROCS to diverse datasets and electricity markets, potentially refining the weighting mechanisms within the WSMD measure to further enhance segmentation accuracy and inform targeted energy programs.