Cag-Avatar Achieves High-Fidelity 3D Head Reconstruction with Adaptive Gaussian Primitives

Researchers are tackling the persistent problem of creating realistic and responsive 3D head avatars for digital animation. Zhe Chang, Haodong Jin, and Yan Song, from the Department of Control Science and Engineering at the University of Shanghai for Science and Technology, alongside Hui Yu, present a novel framework , CAG-Avatar , that significantly improves upon existing methods. Current techniques often treat all facial features equally, resulting in blurred details and distortions, but CAG-Avatar uses a cross-attention mechanism to allow each element of the 3D model to respond uniquely to expressions. This “tailor-made” approach unlocks finer control over facial dynamics, delivering substantially higher reconstruction fidelity , especially in complex areas like teeth , without sacrificing the speed needed for real-time applications.

Adaptive Gaussian Avatars for Realistic Facial Animation offer

Scientists have achieved a breakthrough in creating realistic, real-time 3D head avatars for digital animation, addressing a core challenge in fields like virtual reality and telepresence. This simplistic method struggles to accurately represent the distinct dynamics of different facial regions, such as the deformable skin and rigid teeth, resulting in blurring and distortion artifacts. Experiments conducted by the researchers demonstrate a substantial improvement in reconstruction fidelity, particularly in challenging areas like teeth, while maintaining the real-time rendering performance essential for interactive applications. The research establishes a new approach to animating 3D head avatars by moving beyond global conditioning and embracing localized, attention-guided control.
This is a significant advancement over previous methods, which often struggle with these contrasting movements and produce visually unconvincing results. The team’s work opens exciting possibilities for creating more immersive and realistic virtual experiences, enhancing the quality of telepresence systems, and even improving the accuracy of facial analysis in areas like intelligent vehicles. Furthermore, the study unveils an efficient, high-fidelity head reconstruction framework that seamlessly integrates the cross-attention module with 3D-GS. This integration not only enhances the modeling of complex facial dynamics but also preserves the real-time rendering capabilities that make 3D-GS so appealing.

The researchers designed the system to address detail loss and ambiguity stemming from global conditioning, ensuring that even subtle expressions are accurately captured and reproduced. This achievement represents a major. Quantitative results in Table 1 show that the CAG-Avatar framework achieved an L1 loss of 0.0128, a PSNR of 26.53, an SSIM of 0.9329, and a LPIPS score of 0.0810. These values outperform the baseline FlashAvatar, which recorded an L1 loss of 0.0137, a PSNR of 26.51, an SSIM of 0.9256, and a LPIPS score of 0.0971. These measurements confirm the superior fidelity of the new framework in reconstructing facial details and reducing distortion.

Furthermore, tests prove the model’s ability to accurately replicate expression dynamics while preserving identity features like hairstyle and eyes, yielding natural results in expression reanimation. The framework was trained using the Adam optimizer with β = (0.9, 0.999), a learning rate of 1e−4 for the fusion module and offset MLP, and a perceptual loss weight, λlpips, increased from 0 to 0.05 after 15,000 iterations, with λmouth set to 40. All experiments were conducted on a single NVIDIA RTX 4090 GPU over 15,000 iterations to ensure fair comparison. The breakthrough delivers precise, decoupled control over expression and pose, enabling the synthesis of novel expressions and head poses as demonstrated in Fig0.4. This research advances the development of high-fidelity, drivable digital humans, with future work planned for full-body capture and optimized attention mechanisms for greater efficiency. This work was funded by UKRI (EP/Z000025/1) and the Horizon Europe Programme under the MSCA grant for AC-Mod (Grant No0.101130271).

Adaptive Fusion for Detailed Gaussian Avatar Animation enables

Current 3D-GS techniques often apply a uniform driving signal to all parts of a facial model, resulting in blurred details and distortions, particularly in areas like teeth and rigid facial structures. Quantitative evaluations, including PSNR, SSIM, and LPIPS metrics, demonstrate the framework’s superior performance in replicating expression and pose, while maintaining key identity features. The authors acknowledge a limitation in the current scope of their research, focusing primarily on head avatars. Future work will explore extending the framework to encompass full-body capture and optimising the attention mechanisms for increased computational efficiency. This research represents an advancement in the creation of high-fidelity, drivable digital humans, offering improved realism and control in digital animation applications.