Shows Sequence Diffusion Model Captures Uncertainty in Temporal Link Prediction

Researchers are tackling the challenge of predicting future connections within networks that change over time, a crucial problem for understanding real-world systems. Nguyen Minh Duc and Viet Cuong Ta, from VNU University of Engineering and Technology, Hanoi, alongside their colleagues, present a new approach that moves beyond simply predicting if a link will form, to modelling the probability of future interactions and their sequential nature. Their work introduces a Sequence Diffusion Model (SDG) which learns comprehensive interaction distributions by injecting noise into historical sequences and reconstructing them, offering a generative framework unlike existing discriminative models. This innovative method consistently outperforms current state-of-the-art techniques on standard temporal graph benchmarks, representing a significant advance in temporal link prediction.

Modelling future interactions through sequence-level denoising in dynamic graphs offers promising results

Scientists have developed a novel sequence-level diffusion framework, termed SDG, to address the fundamental challenge of temporal link prediction in dynamic graphs. The research unifies dynamic graph learning with generative denoising, moving beyond purely discriminative models that produce point estimates for future links.
SDG injects noise into the entire historical interaction sequence and reconstructs all interaction embeddings via a conditional denoising process, enabling a more comprehensive capture of interaction distributions. This breakthrough establishes a new approach to modelling temporal interactions by explicitly addressing the uncertainty and sequential structure inherent in dynamic graphs.

The team achieved this by framing temporal link prediction as a sequence-level denoising task, where noise is introduced to both historical interactions and future destinations. This encourages the model to learn richer node distributions, rather than solely focusing on final interactions, and facilitates reasoning about the temporal structure across multiple interactions.

The study unveils a scalable cross-attention denoising decoder that leverages preceding sequence information to guide the reconstruction of the destination sequence. This aligns the generative denoising process directly with the task of temporal link prediction, optimising the model in an end-to-end manner.

Experiments conducted on various temporal graph benchmarks demonstrate that SDG consistently achieves state-of-the-art performance in temporal link prediction. Researchers prove that by drawing inspiration from diffusion models, which excel at capturing uncertainty and generating diverse outputs, they can significantly improve the accuracy and robustness of temporal link prediction.

The work opens new avenues for applications in areas such as recommendation systems, anomaly detection, and knowledge graph completion, where predicting future connections is crucial. This innovative framework offers a probabilistic approach to dynamic graph learning, addressing limitations of existing methods and paving the way for more sophisticated temporal modelling.

Temporal Link Prediction via Sequence Denoising with Cross-Attention Decoding leverages contextual information effectively

Scientists developed Sequence Diffusion for Dynamic Graphs, or SDG, a novel framework uniting temporal graph learning with diffusion processes to address limitations in existing temporal link prediction models. The study pioneered a method injecting noise into both historical neighbour interactions and future destination nodes, transforming temporal link prediction into a sequence-level denoising task.

This approach encourages learning richer node distributions, moving beyond a focus solely on final interactions and enabling comprehensive interaction distribution capture. Researchers engineered a scalable cross-attention denoising decoder that conditions on encoded temporal interactions to reconstruct the destination sequence.

This decoder guides the denoising process, aligning generative modelling with the task of temporal link prediction and optimising the model end-to-end. Experiments employed multiple benchmark datasets to demonstrate SDG’s performance, consistently surpassing state-of-the-art temporal graph methods in temporal link prediction tasks.

The work introduces a diffusion framework for continuous-time dynamic graphs, injecting noise into historical neighbours and destination nodes to effectively learn richer node distributions. This innovative technique addresses the stochasticity and noise inherent in temporal interactions, a key limitation of previous discriminative models.

SDG’s methodology harnesses a cross-attention mechanism within the decoder, leveraging preceding sequence information to refine the denoising process and improve prediction accuracy. The team validated the approach through extensive experimentation, achieving consistently superior results and demonstrating the potential of generative modelling in dynamic graph analysis.

SDG achieves superior performance via conditional generation of node representations, leveraging message passing

Scientists developed a novel sequence-level diffusion framework, termed SDG, to address temporal link prediction in dynamic graphs. The research unifies dynamic graph learning with generative denoising, moving beyond purely discriminative models. Experiments involved injecting noise into historical interaction sequences and jointly reconstructing interaction embeddings, enabling the model to capture comprehensive interaction distributions.

Results demonstrate that SDG consistently achieves state-of-the-art performance in temporal link prediction benchmarks. The team measured the model’s ability to estimate the likelihood of future interactions between nodes, framing the task as a conditional generation problem. Specifically, the work reformulates temporal link prediction as generating the destination node’s representation conditioned on its historical interactions.

Measurements confirm that SDG employs a conditional diffusion model, parameterised using a denoising diffusion probabilistic model (DDPM). The process gradually adds Gaussian noise to the target sample, ultimately following a Gaussian distribution. The forward noise injection is defined by a Markov chain, q(xk | xk−1) = N(xk; p 1 −βkxk−1, βkI), where βk represents the variance of the noise added at each step.

Data shows the researchers derived a closed-form expression for the marginal distribution of xk conditioned on the original clean data x0, expressed as q(xk | x0) = N(xk; √ αkx0, (1 − αk)I). The team then trained a denoise model to invert the noising procedure, conditioned on the historical interaction sequence H(Su,t).

The model’s performance was evaluated using a mean-squared error (MSE) loss, L(θ) = Ek,x0 ||x0 −x0||2 2, to minimise the difference between predicted and ground truth clean samples. The breakthrough delivers a framework where SDG encodes the historical sequence into a latent space, Z(Su,t), which guides the reconstruction of the target sequence, Tu,t. This approach frames temporal representation learning as a denoising process applied to historical neighbour interactions, effectively capturing intricate temporal patterns and long-range dependencies within dynamic systems.

By injecting noise into the entire historical interaction sequence and reconstructing embeddings, SDG aims to model the inherent uncertainty present in these interactions. Extensive experimentation across various benchmark datasets demonstrates that SDG consistently surpasses the performance of existing state-of-the-art temporal graph models in future link prediction tasks, while maintaining computational efficiency for large-scale applications.

Analysis of the embedding space reveals that SDG generates a more diverse and structured representation compared to alternative methods, indicating a greater capacity to capture complex interaction dynamics. The authors acknowledge a limitation in the current scope of their work, specifically its application to transductive settings. Future research will focus on extending SDG using alternative diffusion frameworks and exploring its potential in inductive scenarios.