Kolmogorov Arnold Networks Achieve Superior Accuracy in Function Approximation and Time-Series Prediction

The challenge of building efficient and accurate neural networks continues to drive innovation in computational modelling. Aradhya Gaonkar, Nihal Jain, and Vignesh Chougule, from KLE Technological University, alongside colleagues, present a detailed comparison of Kolmogorov-Arnold Networks (KANs) and conventional Multi-Layer Perceptrons (MLPs). Their research demonstrates KANs consistently outperform MLPs across a range of benchmarks, including nonlinear function approximation, time-series prediction, and classification tasks. This improvement is achieved through a unique adaptive spline-based structure, resulting in higher predictive accuracy alongside a significant reduction in computational demands. The findings suggest KANs offer a paradigm shift in neural modelling, particularly for applications where resource limitations and real-time performance are critical.

This research establishes that KANs consistently outperform MLPs across a range of computational challenges, including nonlinear function approximation, time-series prediction, and multivariate classification. The team achieved this by leveraging the Kolmogorov representation theorem, implementing adaptive spline-based activation functions and grid-based structures within the KAN framework, offering a transformative approach to traditional neural networks. Utilizing datasets ranging from quadratic and cubic mathematical functions to real-world applications like daily temperature prediction and wine categorization, the study rigorously assessed model performance.

Experiments show that KANs reliably exceed MLPs in every benchmark, attaining higher predictive accuracy while simultaneously reducing computational costs. The research meticulously evaluated performance using metrics such as Mean Squared Error (MSE) for regression tasks and classification accuracy, alongside a detailed assessment of computational expense measured in Floating Point Operations (FLOPs). This outcome highlights the ability of KANs to balance computational efficiency with accuracy, making them particularly advantageous in resource-constrained and real-time operational settings. By elucidating the architectural and functional distinctions between KANs and MLPs, the work provides a systematic framework for selecting the most appropriate neural network for specific tasks.

The study unveils the capabilities of KANs in advancing intelligent systems, influencing their application in scenarios demanding both interpretability and computational efficiency. Rooted in the Kolmogorov-Arnold representation theorem, KANs decompose complex functions into simpler univariate components, effectively mitigating the curse of dimensionality and improving performance on compositional data. Scaling law tests reveal that KANs achieve a test loss reduction proportional to N−4, demonstrably outperforming MLPs and retaining prior knowledge during learning, ensuring adaptability in dynamic environments. This breakthrough establishes a clear advantage for KANs, particularly in scenarios where computational resources are limited or real-time processing is critical. The research not only provides a comparative analysis but also a foundational framework for future development in neural network design, paving the way for more efficient and interpretable intelligent systems. Researchers harnessed the principles of Kolmogorov’s representation theorem to construct KANs, employing adaptive spline-based activation functions and grid-based structures as a distinct alternative to conventional neural network architectures. This approach enabled the decomposition of complex, high-dimensional functions into simpler univariate components, directly addressing limitations associated with the curse of dimensionality. Experiments utilized a diverse range of datasets, progressing from mathematical function estimation, specifically quadratic and cubic functions, to real-world applications such as daily temperature prediction and wine categorization.

Model performance was comprehensively assessed using Mean Squared Error (MSE) for regression tasks and classification accuracy for categorical problems, providing a dual metric for evaluating both predictive power and practical utility. Computational expense was quantified through Floating Point Operations (FLOPs), offering insights into resource demands and suitability for constrained environments. The team engineered a system to systematically evaluate and compare KANs and MLPs, focusing on the theoretical foundations of the Kolmogorov-Arnold theorem and its implementation in deep architectures. A key innovation involved extending the theorem’s application by allowing for variable depths and widths, moving beyond the original depth-2, width-2n+1 configuration.

This allowed for the creation of learnable spline-based transformations, combining linear transformations and nonlinearities, unlike MLPs which separate these elements into weights and fixed activations. Furthermore, the study pioneered a grid extension technique to address the issue of input variables exceeding the initial grid range during training. This optimization problem, defined by equations relating initial grid size (G1) to enlarged grid size (G2) and spline degree (k), ensured that learning remained influenced by the splines, rather than solely by weight adjustments. The research rigorously tested both architectures on tasks including nonlinear function approximation, time-series prediction, and multivariate classification, utilising datasets from mathematical function estimation, quadratic and cubic functions, to real-world applications like daily temperature prediction and wine categorization. Experiments revealed that KANs consistently outperformed MLPs, demonstrating superior predictive accuracy alongside significantly reduced computational demands. This outcome underscores their potential in scenarios where computational resources are limited or real-time operation is critical.

Data shows KANs achieve a test loss reduction proportional to N−4, a scaling law test indicating a substantial improvement over MLPs. This enhanced performance stems from KANs’ ability to decompose complex functions into simpler, univariate components, effectively mitigating the challenges posed by the curse of dimensionality. The team measured KANs’ capacity for continual learning, demonstrating the model’s ability to retain prior knowledge while adapting to new tasks, ensuring robustness in evolving environments. Furthermore, tests prove KANs offer enhanced interpretability, providing comprehensible insights into decision-making processes.

Measurements confirm that KANs employ a grid-based structure with adaptive spline-based activation functions, allowing for a dynamic adjustment of grid size during training. Specifically, the research details an extension procedure defined by the equation c′j = argc′j min Ex∼p(x) G2+k−1X j=0 c′jBj(x′) − G1+k−1X j=0 cjBj(x), where G1 represents the initial grid size, G2 the expanded grid size, and k the degree of the B-splines. This grid extension accommodates inputs exceeding the original range, preserving the number of control points and spline form while enhancing precision and maintaining overall efficiency. The breakthrough delivers a systematic framework for selecting the most appropriate neural network for specific tasks, highlighting the benefits of KANs in applications requiring both interpretability and computational efficiency. Analysis of splines acquired during convolutional layers revealed context-dependent behaviour, suggesting a learning process not governed by standardized patterns, but rather influenced by the specific layer and position within the network. The findings demonstrate that KANs consistently outperform MLPs, achieving superior accuracy while requiring significantly fewer computational resources, as measured by both Mean Squared Error and Floating Point Operations. This advantage stems from KANs’ foundation in Kolmogorov’s representation theorem, enabling efficient function approximation through adaptive spline-based activations and grid structures. The demonstrated balance between accuracy and computational efficiency positions KANs as a promising alternative to MLPs, particularly in scenarios with limited resources or real-time processing demands. The study establishes a systematic framework for selecting the most appropriate neural network architecture based on specific task requirements, and suggests potential applications in areas such as financial forecasting, robotics, and biomedical signal analysis. The authors acknowledge limitations related to the scope of datasets used and suggest future work could focus on further optimising KANs through techniques like pruning, quantization, and implementation on Field Programmable Gate Arrays (FPGAs) to enhance performance still further.