Optimization in Machine Learning and Neuroscience Unifies Gradient Descent and Neural Adaptation via Derivative-Free Methods

Iterative optimisation underpins both modern artificial intelligence and our understanding of adaptive systems, and Jesús García Fernández, Nasir Ahmad, and Marcel van Gerven from Radboud University present a unifying perspective on this crucial process. Their work bridges classical optimisation theory with the practicalities of training neural networks and the mechanisms of biological learning, revealing surprising connections between seemingly disparate fields. While gradient-based methods currently dominate machine learning, the team demonstrates how newer, derivative-free approaches, which rely on random exploration and feedback, are achieving competitive performance and offer a more biologically plausible model of learning. This research not only illuminates how the brain optimises its function, leveraging intrinsic noise as a resource, but also suggests pathways towards designing faster, more energy-efficient artificial intelligence systems that mimic this natural efficiency.

Key Researchers In Neural Network Theory

This compilation lists the authors mentioned within the provided texts, presented alphabetically by last name, including first names where available, representing a comprehensive list of contributors to the field. Abad, F., Adams, R. P., Aghajan, M., Alvarez, J., Amari, S., Andrieu, N., Arjovsky, M., Bach, F., Balduzzi, D., Bengio, Y., Berner, J., Bogacz, R., Brandfonbrener, D., Bruckner, M., Cadieu, C. F., Chaudhuri, S., Clune, J., Conti, E., Corrado, G., Cruz, C., Deb, S., Drucker, K., Drukker, K., DiCarlo, J. J., Gen, M., Ghodrati, M., Grosse, R.

B., Hassibi, B., Heidergott, B., Hinton, G., Hong, H., Hutter, F., Jiang, Y., Janson, L., Kakade, S., Klos, C., Krizhevsky, A., Kwasnicka, H., Lansdell, B., Larochelle, H., Lehman, J., Linares-Barranco, B., Maida, A., Martens, J., Matheson, D., Memmesheimer, R., Nelson, S. B., Nowozin, S., Pesce, L. L., Price, K., Salakhutdinov, R., Sato, I., Seibert, D., Shepherd, G. M., Shanno, D. F., Slowik, A., Snoek, J., Solomon, E.

A., Spall, J. C., Stanley, K. O., Sra, S., Srivastava, N., Storn, R., Such, F. P., Sugiyama, M., Tavanaei, A., Tieleman, T., Turrigiano, G. G., Van Laarhoven, T., Van Gerven, M., Vikhar, P., Vyas, N., Wang, Z., Werbos, P.

J., Werfel, J., Whittington, J. C., Williams, R. J., Xie, X., Yamins, D. L., Yang, X. -S., Ypma, T.

Optimization Algorithms for Neural Network Training

This work presents a comprehensive review of iterative optimization techniques, categorizing approaches based on the order of derivative information utilized, ranging from first-, second-, and higher-order gradient-based methods to derivative-free, or zeroth-order (ZO) optimization, systematically exploring how these methods adapt to the unique challenges of neural network training and the resulting learning dynamics, ultimately framing biological learning through an optimization lens. The study formally defines unconstrained optimization as the minimization of a scalar-valued objective function, where iterative algorithms refine parameter vectors at each step using an update rule, focusing on minimizing a local approximation of the objective function derived from a Taylor expansion. .

Optimization Landscapes and Neural Network Training

This work demonstrates a comprehensive analysis of optimization techniques, bridging classic theory with modern neural network training and biological learning mechanisms, achieving a detailed understanding of how different optimization methods perform in high-dimensional settings, revealing key insights into the challenges of training complex neural networks, meticulously categorizing optimization approaches based on derivative order, ranging from first-order gradient-based methods to derivative-free, zeroth-order techniques. Experiments revealed that neural network training fundamentally relies on minimizing a loss function defined over massive datasets and billions of parameters, resulting in complex, non-convex loss landscapes. To address this, scientists developed and analyzed specialized techniques for efficient gradient computation, notably automatic differentiation (AD), accurately computing derivatives by iteratively applying the chain rule, providing exact derivatives up to machine precision, forming the fundamental engine for training neural networks, investigating two main AD modes: reverse mode AD and forward mode AD, with reverse mode AD, implemented as backpropagation (BP), proving highly efficient due to its cost scaling linearly with forward pass operations, independent of parameter count, focusing on recurrent neural networks (RNNs), developing backpropagation through time (BPTT) to handle sequential data, treating each time step as a distinct layer, allowing gradient computation over entire sequences, though it can be computationally expensive and susceptible to vanishing or exploding gradients, exploring an alternative, real-time recurrent learning (RTRL), demonstrating online learning capabilities, albeit with higher computational complexity, investigating forward gradients, a practical implementation of forward mode AD, propagating random perturbations forward to construct unbiased gradient estimates in a single forward pass, avoiding the need for a backward step, providing a robust toolkit for optimizing complex models and offering valuable insights into the mechanisms of learning.

Gradient Information Bridges Learning and Optimization

This review presents a unified perspective on iterative optimization, connecting theoretical foundations with applications in artificial intelligence and biological learning, categorizing optimization approaches based on their use of derivative information, ranging from traditional gradient-based methods to derivative-free, or zeroth-order, techniques, demonstrating that while backpropagation currently dominates machine learning, effective learning also emerges from methods that approximate gradients, offering scalable alternatives for complex models, extending to understanding biological learning, suggesting that the brain may leverage intrinsic noise as a computational resource. The increasing performance of non-gradient-based methods supports the idea that stochasticity plays a crucial role in discovering robust and generalizable solutions, particularly within overparameterized networks.