Reinforcement Learning Automates Atomic Structure Creation with Scanning Tunneling Microscopy

Creating artificial structures with precisely controlled electronic properties represents a major goal in materials science, and researchers are now demonstrating a significant step towards automated fabrication. Ganesh Narasimha, Mykola Telychko, Wooin Yang, and colleagues at the Center for Nanophase Materials Sciences, Oak Ridge National Laboratory, have developed a new framework that uses artificial intelligence to build these structures at the atomic scale. The team’s method employs a scanning tunneling microscope, but overcomes the limitations of manual construction by using reinforcement learning to control the placement of carbon monoxide molecules on a copper surface. This automated workflow not only speeds up the fabrication process, but also enables the creation of larger, more complex lattices with designer electronic states, confirmed in this study by the successful construction of a lattice exhibiting a characteristic Dirac point, a key feature for advanced electronic applications.

Currently, fabricating these structures relies heavily on manual control, a process that is both time-consuming and limited in its ability to create complex designs. This restricts the exploration of a vast design space and hinders the realisation of materials with tailored electronic properties. Consequently, there is a significant need for automated methods capable of constructing artificial lattice structures with precisely defined electronic states. This research addresses this need by developing an automated construction technique, aiming to overcome the limitations of manual fabrication and accelerate the discovery of novel materials with designer quantum properties.

Reinforcement Learning Controls Scanning Tunneling Microscope Tip

Researchers have developed a sophisticated system that automates the manipulation of individual molecules on a surface using a scanning tunneling microscope. The core of this system is a deep reinforcement learning agent that learns to control the microscope tip, positioning molecules into a desired lattice structure. The agent receives information about the current and target positions of molecules, as well as the angle of the tip, and uses this data to optimise its actions. This approach moves beyond traditional manual control, which is slow and limited in its ability to create complex designs. The system identifies molecules using computer vision techniques and then plans a path for their movement.

The learning process begins with initial experimentation, gathering data that trains the agent to make increasingly efficient decisions. This training involves adding small amounts of noise to the data to improve the agent’s robustness and shuffling episodes to enhance learning. To address challenges like molecules sticking to the surface or the microscope drifting over time, the researchers implemented strategies such as slightly overshooting the target position, aligning the manipulation path with the surface’s atomic structure, and using an anchor molecule as a reference point to correct for drift.

Automated Atomic Construction via Reinforcement Learning

Researchers have developed a new automated method for constructing artificial structures at the atomic scale, offering a significant advance in the design of materials with tailored properties. This work centres on manipulating individual carbon monoxide molecules on a copper surface using a scanning tunneling microscope, but crucially, employs reinforcement learning to dramatically improve the process. Traditionally, building these structures requires painstaking manual control of the microscope tip and careful optimisation of numerous parameters. The team’s approach uses a machine learning agent to learn the optimal conditions for manipulating the molecules, effectively automating much of the construction process.

The system first identifies the locations of carbon monoxide molecules and then plans a path for their movement to designated target sites, all while dynamically adjusting parameters like voltage and speed. This learning process involves initial randomised experimentation to gather data, which then trains the agent to make increasingly efficient decisions, minimising errors and maximising the speed of construction. The result is a substantial reduction in human input and the ability to create larger, more complex structures than previously possible. Demonstrating this technique, the researchers successfully built an extended artificial lattice resembling graphene with a high degree of precision. Importantly, they confirmed the presence of a ‘Dirac point’ within the lattice’s electronic structure, a key characteristic indicating the material possesses unique and potentially valuable electronic properties. This confirms the automated construction not only creates the desired structure, but also that it functions as intended, paving the way for the design of materials with specific electronic behaviours.

Automated Molecular Manipulation Builds Graphene Lattice

This research presents a new methodology for automating the manipulation of individual molecules using a scanning tunneling microscope. The team developed a system that combines object detection to identify molecules on a surface with reinforcement learning to predict the optimal parameters for moving them into desired positions. This significantly reduces the need for manual control during the construction of artificial structures at the atomic scale. The system was successfully used to build an extended lattice resembling graphene and confirmed the presence of a characteristic Dirac point in its electronic structure, a feature important for understanding its electronic properties.

By automating the process, researchers can more easily create and study designer quantum states, which are difficult to observe in natural materials due to imperfections. The authors acknowledge that further challenges remain in scaling up this approach to create even more complex structures. Future work could also explore using inverse learning techniques to extract information about interactions within a material from its spectra, offering a unique way to test theoretical models. The code developed for this research is publicly available, allowing other scientists to reproduce and build upon these findings. This represents a significant step towards automating the creation of atomically precise structures and accelerating discoveries in areas like quantum materials science.