Researchers have achieved a 5.7% character error rate in recognizing text from PDFs using a surprising approach: a simulated fruit fly circuit. The system, built around the MaleCNS connectome containing 166,700 neurons and 25,582,938 connections, demonstrates an ability to process visual information and identify characters despite its unconventional architecture, the company says.
The circuit correctly identified 21 of 21 and 44 of 45 exact cells in two selected table crops, showcasing a particular strength in recognizing structured numeric data. This exploratory model, detailed in a new research report, uses a small decoder trained on the fly circuit’s activity, though the team emphasizes it is not evidence that a biological fly reads.
Fixed Connectome Achieves 5.7% Character Error on PDF Text
Achieving 87.6% accuracy on a 1,632-glyph, 68-class benchmark, the system demonstrated high performance, demonstrating a capacity for general character identification beyond simple PDF text extraction. This performance on the broader glyph set precedes the reported 5.7% character error rate achieved when processing text directly from PDF documents, indicating the circuit’s foundational ability to discern visual patterns. Researchers focused on an eight-line sample containing 176 characters to quantify the PDF recognition performance, a deliberately limited scope chosen to isolate the circuit’s core capabilities.
The MaleCNS connectome, serving as the foundation for this optical character recognition, processed input as a series of individual glyphs, resetting between each to maintain clarity for the decoder. Beyond isolated character recognition, the circuit exhibited a particular aptitude for structured data, correctly identifying 21 out of 21 and 44 out of 45 exact cells within two selected numeric table crops.
This success with tables contrasts with the 5.7% error rate on general PDF text, suggesting the circuit benefits from the inherent order and constraints of tabular data. The system’s architecture intentionally separates letter and numeric modes, a design choice reflected in the distinct performance metrics for each task. Preprocessing steps important to conventional OCR systems, such as baseline normalization and word gap detection, were deliberately omitted; the circuit operates directly on pixel data after initial thresholding and geometric reconstruction.
The research team utilized a Brian2 dynamics comparison to validate the small-network tests, ensuring the simulated neural activity aligns with expected biological behavior. Replay verification, a key component of the methodology, relies on saved readouts and crops, allowing researchers to reproduce class scores and recheck text segmentation without requiring access to the connectome or network itself.
According to the project repository, running the system locally on a Mac requires a series of initial steps, including syncing dependencies and downloading necessary data, before the OCR process can begin. “There is no automatic whole-document table discovery,” the documentation states, clarifying the current limitations of the system and outlining areas for future development. The letter alphabet is 0123456789,.-$ABCDEFGHIJKLMNOPQRSTUVWXYZabcdefghijklmnopqrstuvwxyz. Spaces come from geometry. Unsupported characters may be forced into known classes.
Character Recognition Performance on Glyphs and Tables
A focused subset of 1,248 letters yielded an even higher 85.0% accuracy, suggesting the circuit handles letterforms particularly well. While excelling at isolated glyph recognition, the system also demonstrated a surprising aptitude for interpreting structured data within tables, according to MaleCNS. The system’s developers acknowledge current limitations, noting that only one of eight PDF lines is typically rendered exactly correctly, and even a slight three-degree tilt can disrupt segmentation.
Despite this, the ability to reconstruct selected tables from PDF pixels, coupled with the visualization of character scans, eye samples, and neural activity, offers a unique window into the circuit’s decision-making process. The system was tested on an Apple M2 Pro with 32 GiB RAM, requiring several gigabytes for source and graph caches, and at least 10 GiB of free space for setup and experiments, demonstrating the computational resources needed to run the simulation locally.
Decoder Architecture: 64 Hidden Units & Frozen Weights
The decoder architecture powering this unconventional optical character recognition system utilizes 64 hidden units, a deliberately constrained design choice given the scale of the underlying neural network. This relatively small decoder, containing 266,628 trained parameters, learns to interpret character predictions derived from the activity within the simulated MaleCNS connectome, a circuit retaining 166,700 neurons and 25,582,938 connections. Researchers intentionally froze the weights of the core connectome, focusing training efforts solely on this downstream decoder to establish a clear separation between the fixed biological model and the learned recognition capability.
This approach contrasts with end-to-end training of larger neural networks, and allows for focused analysis of how effectively the fruit fly-inspired circuit can provide useful features for character identification. The decoder receives input from 1,024 downstream neurons after a 100 millisecond presentation of a character to the simulated circuit; these signals represent the fly connectome’s internal processing of the visual input.
Four bins are used to categorize these signals, providing a condensed representation for the decoder to interpret. The report details how table reconstruction uses geometric information to define rows and columns, supplementing the circuit’s character recognition capabilities. The project repository contains detailed information on input sampling, decoder choices, and comparisons with alternative approaches, allowing for further investigation and refinement of this unique system.
See today’s quantum computing news on Quantum Zeitgeist for the latest breakthroughs in qubits, hardware, algorithms, and industry deals.
