Since 1994, the Department of Energy has stored surplus plutonium in nested containers known as 3013 containers and actively monitors for potential cracking in the innermost layer. Destructive evaluation of a small number of the inner containers revealed cracking, prompting a project to develop a machine learning model capable of proactively identifying potential issues.
Researchers found a U-Net type of convolutional neural network outperformed previous methods in identifying potential cracking, evaluated by summing pixel values to assess accuracy. “The model predictions indicate ‘crack-like features,’ which require further human evaluation before being confirmed as cracks,” said Stephanie Gamble, along with Joe Cammarata, Anna d’Entremont, and Jason Bakos.
U-Net Model Detects Cracks in 3013 Plutonium Canisters
Destructive testing of a small number of these inner containers showed cracking after evaluation, prompting a focused effort to develop automated detection methods before potential failures compromise long-term storage safety. The project resulted in a U-Net convolutional neural network that outperformed earlier algorithms in identifying potential crack formations within the containers.
This advancement allows for more efficient analysis of the thousands of laser confocal microscope, wide-area three-dimensional measurement system, and scanning electron microscope scans collected for ongoing surveillance. Evaluating the performance of these machine learning models presented unique challenges, given the microscopic scale of the potential cracks and the resulting imbalanced data sets.
Traditional crack-detection metrics, precision, recall, and Fβ scores, are commonly used for images of larger-scale cracks in materials like concrete, but proved less effective with the high ratio of non-crack to crack pixels observed in the canister scans, approximately 1:30,000. Identifying these subtle flaws by visual inspection is significantly more difficult than detecting cracks in concrete, necessitating a model optimized to minimize false negatives, even at the cost of increased false positives.
This approach allows for a detailed comparison of different algorithms and their ability to identify potential cracking with minimal oversight, and also provided a framework for evaluating future iterations and refinements. The team is extending the system to analyze data from WAMS.
Challenges with Pixel-Based Scoring for Microscopic Cracks
Evaluating machine learning models designed to detect microscopic cracks within the Department of Energy’s 3013 containers, used since 1994, presented unique hurdles beyond those encountered in typical crack detection applications. Laser confocal microscope scans of the container interiors yield images with approximately one crack pixel for every 30,000 non-crack pixels, a ratio of 1:30,000 that skewed the performance of standard pixel-based scoring methods.
The goal was to highlight areas of interest for further scrutiny, rather than pinpointing every individual cracked pixel, a task even human labelers found inconsistent. Various labeling approaches, such as marking crack regions with lines of differing thicknesses, were investigated but ultimately deemed unsuitable for pixel-based scoring, and the team found that consistently defining the precise boundaries of microscopic cracks proved challenging, even for expert human annotators.
Instead of striving for perfect pixel alignment, the developed system utilizes a graphical user interface that displays numbered bounding boxes around detected crack features, alongside a measurement tool allowing manual length assessment. This approach, while not relying on a flawless metric, demonstrated strong alignment with the practical needs of scientists reviewing the data and is currently being developed using the same methods to analyze data from wide-area monitoring systems (WAMS). The resulting detection system, complete with an intuitive GUI, enables scientists to effectively analyze data, saving time and improving detection consistency.
Tile and Feature-Based Metrics for Crack Identification
While initial inspections relied on visual assessment, the subtle nature of these potential flaws prompted a shift towards machine learning techniques to proactively identify issues before they escalate. Instead, the researchers prioritized highlighting areas, acknowledging that further human review would be necessary to confirm actual flaws. Initial investigations explored tile-based metrics, assessing model performance by summing pixel values within identified crack regions and highlighting tiles exceeding a defined threshold; however, the team discovered that feature-based metrics offered a more robust approach.
These metrics considered characteristics like elongation, vertical pixel sum, and connected pixel count to define a crack feature, allowing the model to focus on identifying the overall presence of a flaw rather than pinpointing every individual pixel. A Bayesian optimization scheme then fine-tuned these feature definitions, maximizing model performance using an (F_{β}) score calculated from feature-based precision and recall. The team found that consistently identifying the number of labeled features proved particularly useful in assessing the system’s effectiveness, offering a practical measure of its ability to flag potential areas of concern within the 3013 containers.
Bayesian Optimization Refines Feature Detection Performance
This scheme didn’t solely focus on feature definitions; it also fine-tuned neural network hyperparameters, including the number of filters, class weights, and learning rate, to further enhance detection accuracy. Analyses revealed that laser confocal microscopy (LCM) offered higher resolution, 0.7 micrometers per pixel, compared to wide-angle microscopy systems (WAMS, at 3.7 micrometers per pixel), making it a superior candidate for precise crack feature detection, despite its higher cost in terms of both time and resources.
The detection program, nearing completion, incorporates a graphical user interface to facilitate data upload and model execution for scientists, while parallel work is in development using the same methods to adapt these methods to WAMS and scanning electron microscopy (SEM). SEM promises very high resolutions with significantly reduced processing times compared to LCM, offering a potential pathway to balance accuracy and efficiency.
Although the work is in the final year of the intended funding, the team is actively developing additional models for WAMS, demonstrating a commitment to a multi-platform approach to monitoring the integrity of these long-term storage containers.
See today’s quantum computing news on Quantum Zeitgeist for the latest breakthroughs in qubits, hardware, algorithms, and industry deals.
