Skeleton-Based Methods Advance Emotion Recognition from Full-Body Motion and Gait

Recognising human emotion from body language presents a compelling alternative to methods reliant on facial expressions or physiological data, offering increased privacy and broader applicability. Haifeng Lu, Jiuyi Chen, and Zhen Zhang, from Shenzhen MSU-BIT University, alongside their colleagues, comprehensively survey the rapidly developing field of skeleton-based emotion recognition. Their work systematically reviews techniques that interpret emotion from 3D skeletal movements, detailing how advances in pose estimation are driving progress. This survey not only categorises existing approaches, proposing a unified taxonomy of technical paradigms, but also highlights the potential of this technology for applications such as mental health assessment, including the detection of conditions like depression and autism, and outlines key challenges for future research.

Skeletal Data Recognizes Human Actions, Emotions

Research increasingly focuses on understanding human actions and emotions through the analysis of skeletal data, offering a new approach to recognizing affective states. This work utilizes data captured from tracking the human skeleton, using technologies like motion capture or video pose estimation, to identify patterns associated with specific actions, emotions, and even anomalies in behaviour. Researchers are exploring how these skeletal movements can reveal a person’s emotional state, offering a non-intrusive alternative to traditional methods. A significant portion of this research combines skeletal data with other information, such as facial expressions, speech, or physiological signals, to improve accuracy.</p

Deep learning techniques, including convolutional and recurrent neural networks, are extensively used to extract meaningful features and classify different actions or emotions. Researchers are also leveraging pre-trained models to enhance performance, particularly when limited labelled data is available, allowing systems to recognize actions or emotions even with minimal training examples. Beyond simple recognition, researchers are applying these techniques to a variety of real-world problems, including identifying movement patterns in individuals with Autism Spectrum Disorder, monitoring patient rehabilitation progress, detecting falls, assessing stress levels, and enhancing security surveillance. Graph neural networks, which represent the skeleton as a network of connected joints, are proving particularly effective at capturing complex relationships between body parts and modelling dynamic movements.</p

The field is also exploring advanced techniques like multimodal fusion, combining skeletal data with audio, video, and other sources to create a more comprehensive understanding of human behaviour. The integration of large language models and vision-language models is a recent trend, allowing systems to interpret actions and emotions in a more contextualized way. Self-supervised learning, where models learn from unlabeled data, is also gaining traction, reducing the need for extensive manual annotation.</p
D Skeleton Tracking for Emotion Recognition

Researchers are increasingly focused on recognizing human emotions through body movements as a privacy-preserving alternative to traditional methods. Current emotion recognition techniques often rely on analyzing facial expressions, audio signals, text, or physiological data, but these approaches can be intrusive or require close proximity. This work addresses these limitations by focusing on full-body motion as a source of emotional cues, offering a non-contact and readily accessible means of assessment. The methodology centres on leveraging advancements in 3D skeleton acquisition and pose estimation technologies, which allow for the accurate tracking of body movements without requiring direct physical contact.</p

These technologies capture the position and orientation of key body joints, creating a digital “skeleton” that represents a person’s posture and motion over time. By analysing changes in this skeletal data, researchers aim to identify patterns associated with different emotional states. The research systematically categorizes existing methods into those focused on static posture analysis and those that examine dynamic gait patterns, providing a comprehensive overview of the field. This categorization allows for a structured comparison of different methods, highlighting their strengths and weaknesses.</p

Researchers benchmarked these approaches across commonly used datasets, providing a quantitative assessment of their performance. Furthermore, the study extends beyond basic emotion recognition to explore applications in mental health assessment, specifically focusing on detecting conditions like depression and autism. This demonstrates the potential of skeleton-based emotion recognition to provide valuable insights into mental wellbeing and facilitate early intervention.</p

Body Movement Reveals Human Emotional States

Recent research demonstrates a growing ability to recognize human emotions through the analysis of full-body movements, offering a compelling alternative to traditional methods that rely on facial expressions, audio signals, or physiological data. This approach leverages advances in depth-sensing technologies and human pose estimation to accurately capture 3D skeleton data, providing a robust and privacy-preserving means of understanding emotional states. Unlike methods requiring contact-based sensors, skeleton-based emotion recognition offers a non-intrusive and comfortable experience for users. The core principle rests on the understanding that body movements are a rich, nonverbal channel for conveying emotions, with gestures and gait patterns providing valuable cues to underlying affective states.</p

Researchers categorize these approaches into posture-based recognition, which analyzes movements associated with specific actions, and gait-based recognition, which focuses on dynamic features during natural walking. The increasing volume of research in both areas indicates a growing recognition of the practical potential and effectiveness of these skeleton-based methods for enhancing human-computer interaction and affective computing. Current research builds upon established psychological models of emotion, including discrete emotions theory, multidimensional approaches, and componential models, to interpret the relationship between body movement and emotional expression. The field has seen a steady increase in publications focusing on both posture and gait analysis, demonstrating the growing interest in this area.</p

This progress is particularly significant as it offers a means of long-range emotion sensing, unlike methods limited by proximity or requiring direct contact with the individual. This emerging field holds substantial value for a wide range of applications, including mental health monitoring, educational technology, and security surveillance, offering a non-invasive and potentially more reliable means of assessing emotional states and improving the functionality of intelligent systems. By focusing on full-body movements, researchers are developing methods that are not only accurate but also address growing concerns about privacy and user comfort, paving the way for more seamless and intuitive human-computer interactions.</p

Skeletal Data Reveals Human Emotional States

This survey provides a comprehensive overview of recent advances in recognising emotion from human body movements, specifically using 3D skeletal data. By examining both posture and gait-based approaches, and categorising technical strategies from traditional methods to deep learning architectures, the research offers a unified perspective on the field’s development. Compared to methods relying on facial expressions or voice analysis, recognising emotion from skeletal data offers advantages such as resilience to environmental changes and enhanced privacy protection. The findings highlight the potential of this technology for real-world applications including healthcare monitoring, improved human-computer interaction, and public safety initiatives. While significant progress has been made, the authors acknowledge limitations including the need for more diverse datasets that accurately reflect real-world scenarios, and the challenge of ensuring consistent performance across different users and environments. Future research directions include integrating multiple data sources, such as voice and facial expressions, and developing more lightweight and explainable artificial intelligence models.</p