P-adic Neural Networks Represent Hierarchies with Transparent Ultrametric Representation Learning

Hierarchical data, such as biological classifications, linguistic structures, and file systems, often prove challenging for conventional deep learning models that rely on Euclidean space. Gnankan Landry Regis N’guessan, affiliated with Σηigmα Research Group, the Nelson Mandela African Institution of Science and Technology, and the African Institute for Mathematical Sciences, leads a team that introduces van der Put Neural Networks (v-PuNNs), a novel architecture designed to represent hierarchical relationships with greater accuracy and interpretability. This research establishes a new principle, Transparent Ultrametric Representation Learning, where network weights are themselves structured to reflect the underlying hierarchy, enabling exact subtree semantics. The team demonstrates state-of-the-art performance on established benchmarks, achieving exceptional accuracy in classifying nouns, gene functions, and mammalian species, while also offering a pathway towards generating structural insights and controllable generative codes for complex data, effectively bridging the fields of number theory and deep learning.

P-adic Numbers and Neural Network Hierarchies

This research explores a new intersection between number theory and deep learning, investigating how p-adic numbers can improve neural networks’ ability to learn hierarchical structures within data. Conventional neural networks sometimes struggle to represent complex, tree-like relationships efficiently, but p-adic analysis offers a different mathematical framework naturally suited to these structures. The team hypothesized that leveraging p-adic numbers could lead to more effective hierarchical representation learning, crucial for tasks like image analysis, language processing, and understanding complex systems. The core innovation is HiPaN, or Hierarchical p-adic Neural Network, an architecture designed to exploit the unique properties of p-adic numbers and ultrametricity, a stronger notion of distance than standard metrics.

HiPaN organizes layers in a hierarchical structure, mirroring the data’s organization, and utilizes p-adic arithmetic for computations. This approach holds potential for applications in natural language processing, computer vision, bioinformatics, and data mining. Results demonstrate that HiPaN achieves better calibration, meaning its predicted probabilities more closely align with actual outcomes, and operates efficiently with reasonable computational cost. Recognizing the limitations of embedding hierarchies in standard Euclidean space, the team turned to p-adic systems and ultrametric spaces, mathematical frameworks naturally suited to representing tree-like structures. This involved constructing neural networks where neurons are defined by characteristic functions of p-adic balls, creating a system that mirrors the branching nature of hierarchies at a fundamental level. The team addressed the challenge of training these networks in a discrete space by developing Valuation-Adaptive Perturbation Optimization (VAPO), including fast and moment-based variants for efficient and stable training. Unlike conventional deep learning models that force hierarchical data into Euclidean space, v-PuNNs operate within a mathematical space called p-adic numbers, which naturally reflects the nested relationships within hierarchies. This approach avoids distortions and allows for a more accurate and interpretable representation of the data. In practical tests, these networks achieved state-of-the-art results on benchmark datasets, including 99. 96% accuracy in classifying over 52,000 nouns from WordNet and 100% accuracy on a gene ontology dataset with over 27,000 proteins.

When applied to classifying mammals, the learned relationships closely matched known taxonomic distances, demonstrating a strong correlation with the data. The learned relationships within the network are perfectly ultrametric, suggesting the network not only classifies data correctly but also understands the underlying relationships. Unlike conventional models that embed data in Euclidean space, v-PuNNs utilize p-adic numbers, aligning the model’s geometry with the inherent structure of hierarchical systems like taxonomies and knowledge graphs. A key theoretical result is the Finite Hierarchical Approximation Theorem, which demonstrates that a v-PuNN with a specific number of coefficients can universally represent any K-level tree, offering guarantees of expressiveness. These networks demonstrate state-of-the-art performance on benchmark datasets including WordNet, Gene Ontology, and NCBI Mammalia, achieving high accuracy with significantly fewer parameters than existing hyperbolic or transformer-based models.

Beyond classification, the research also introduces methods to derive structural invariants and generate controllable codes for tabular data, highlighting the versatility of the approach. The authors acknowledge that future work could explore mixed-radix primes for more complex trees and investigate p-adic information geometry to improve optimization. They also suggest integrating p-adic codes into other deep learning models, such as language models and graph neural networks, as a promising avenue for further research.