Researchers Unlock New Music Generation with Amadeus, Challenging Sequential Dependencies in Attribute Token Modelling

Symbolic music generation increasingly relies on models that predict sequences of musical notes and their characteristics, but current approaches often treat these characteristics as rigidly ordered, limiting their potential. Hongju Su, Ke Li, and Lan Yang, along with Honggang Zhang and Yi-Zhe Song, challenge this assumption by introducing Amadeus, a new framework that views musical attributes as a more flexible, concurrent set. Their work demonstrates that modelling attributes bidirectionally, rather than sequentially, significantly improves music generation quality and speed, outperforming existing state-of-the-art models across multiple benchmarks. Crucially, the team also introduces a substantial new open-source dataset, the Amadeus MIDI Dataset, to facilitate further research and development in this rapidly evolving field, and showcases the possibility of detailed, user-guided control over generated music.

Symbolic music generation models typically model music as a sequence of attributes, assuming a fixed dependency structure between them. However, observations suggest that musical attributes are essentially concurrent and unordered, rather than strictly sequential.

Large Language Models Generate Symbolic Music

Recent advances in symbolic music generation leverage large language models, originally designed for text, to create musical pieces. These models aim to generate music that is structurally sound, musically expressive, and controllable, allowing specification of genre, mood, or style. Several models have emerged, building on previous techniques. The Music Transformer was an early model applying the transformer architecture to music generation. MuseCoco generates symbolic music from text captions, while the Pop Music Transformer focuses on expressive pop piano compositions.

Other models, such as the Compound Word Transformer and Jazz Transformer, explore different approaches to modelling full songs and evaluating AI-composed music. Recent developments include Diffuse and Disperse, Megabyte, Byte Latent Transformer, Stable Audio Open, and Mupt, each contributing unique techniques to the field. Models like NotaGen, MelodyT5, Text2midi-InferAlign, and XMusic further advance the state-of-the-art, aiming for a generalized and controllable symbolic music generation framework. Current research focuses on improving musicality and expressiveness, enhancing controllability, scaling to longer sequences, and addressing the lack of high-quality data. Developing better metrics for evaluating AI-composed music remains a significant challenge, driving the need for continued innovation in this field.

Amadeus Generates Music with Enhanced Quality and Speed

Researchers have developed Amadeus, a novel framework for symbolic music generation that significantly outperforms existing models in both quality and speed. Challenging the conventional sequential approach, the team observed that musical note attributes are concurrent and unordered. This led to a two-level architecture combining an autoregressive model for note sequences with a bidirectional discrete diffusion model for attributes, fundamentally changing how symbolic music is generated. Amadeus incorporates two key innovations: the Music Latent Space Discriminability Enhancement Strategy and the Conditional Information Enhancement Module.

The former utilizes enhanced contrastive learning to amplify the discriminability of intermediate music representations, while the latter employs attention mechanisms to strengthen note latent vector representation, leading to more precise note decoding. Experiments demonstrate that Amadeus achieves at least a four-fold increase in speed compared to previous methods. The results show that Amadeus excels in music quality, condition adherence, attribute controllability, and inference speed, allowing flexible trade-offs between these factors. 9 million pre-training samples and 320,000 annotated fine-tuning samples. This dataset and the innovative Amadeus framework promise to unlock new possibilities in computer-aided composition and music generation.

Concurrent Diffusion Beats Sequential Music Generation

Amadeus presents a new approach to symbolic music generation, challenging the reliance on sequential modelling of musical attributes. The framework adopts a two-level architecture, combining an autoregressive model for note sequences with a bidirectional discrete diffusion model for attributes, treating these attributes as a concurrent, unordered set. Performance improvements are achieved through Music Latent Space Discriminability Enhancement Strategies and the Conditional Information Enhancement Module, which refine the representation of musical data. Experiments demonstrate that Amadeus significantly outperforms existing models in both unconditional and text-conditioned music generation tasks, while also achieving a substantial increase in generation speed. The authors acknowledge that further optimisation is possible to balance computational efficiency with generation quality, and future work will explore lossless acceleration techniques to enable real-time music generation.