Automated Algorithm Discovery Enables Signal Processing with 50,000+ Variable Frameworks

The quest to create efficient algorithms for reconstructing signals from incomplete or noisy data presents a significant challenge for researchers, often relying on intuition and extensive trial and error. Patrick Yubeaton and Sarthak Gupta, both from New York University, alongside M. Salman Asif from the University of California, Riverside, and Chinmay Hegde from New York University, demonstrate a groundbreaking approach to this problem by employing Neural Architecture Search, a technique commonly used in machine learning, to automatically discover signal processing algorithms. Their work successfully rediscovers key elements of established methods, such as the Iterative Shrinkage Thresholding Algorithm and its accelerated variant, within a vast search space of over 50,000 possibilities. This achievement not only validates the potential of automated algorithm design but also establishes a flexible framework applicable to a wide range of data types and algorithmic structures, promising to accelerate innovation in signal processing and related fields.

The research demonstrates the potential to automate the design of complex algorithms, a process traditionally reliant on expert knowledge and extensive trial and error. The model’s objective was to learn the optimal activation function for sparse recovery, and results demonstrate that the NAS framework successfully identified the shrinkage operator as the preferred choice. Analysis of the learned parameters revealed that the framework prioritized activation functions that minimized reconstruction error, effectively learning the optimal algorithm structure.

Further experiments explored methods to decrease NAS training time, revealing that a larger search space significantly increased computational demands. A “looped NAS model”, reusing a single NAS cell for all layers, dramatically reduced the parameter count and accelerated training, while still successfully identifying the shrinkage operation. This achievement demonstrates the feasibility of automating algorithm design, a process traditionally reliant on expert knowledge and extensive manual effort.

The method involves representing algorithms as recurrent neural networks and then employing NAS to learn the optimal network structure and weights, effectively ‘rebuilding’ the algorithm from data. Experiments confirm the framework’s ability to generalize beyond ISTA and FISTA, suggesting broad applicability to other signal processing tasks and algorithms. While the authors acknowledge the computational cost associated with the search process, they highlight the potential for significant efficiency gains in algorithm development.