New AI Framework Learns from Human Feedback, Not Datasets Alone

Researchers at Duke University and the Army Research Laboratory have developed a new framework that enables artificial intelligence (AI) to learn through real-time human feedback, rather than relying on massive datasets and simulations.

Dubbed GUIDE, this platform allows humans to observe AI’s actions in real-time and provide nuanced feedback, similar to how a driving instructor would teach a student driver. According to Boyuan Chen, professor of mechanical engineering and materials science at Duke, existing training methods are limited by their reliance on pre-existing datasets and traditional feedback approaches.

GUIDE bridges this gap by incorporating continuous human feedback, enabling AI to learn complex tasks more like humans. In its debut study, GUIDE was used to teach an AI player how to play hide-and-seek, with a human trainer providing real-time feedback on the AI’s searching strategy. The results showed a significant improvement in the AI’s performance, with up to a 30% increase in success rates compared to current state-of-the-art methods.

The traditional approach to training artificial intelligence (AI) systems has been through massive datasets and extensive simulations. However, this method fails to teach AI to perform complex tasks requiring fast decision-making based on limited learning information. Researchers from Duke University and the Army Research Laboratory have developed a novel platform, nicknamed GUIDE, which enables AI to learn through real-time human feedback, paving the way for more responsive AI systems.

The Limitations of Traditional Training Methods

Existing training methods are often constrained by their reliance on extensive pre-existing datasets while also struggling with the limited adaptability of traditional feedback approaches. These limitations hinder AI’s ability to handle tasks that require fast decision-making based on limited learning information. Professor Boyuan Chen, director of the Duke General Robotics Lab, explains that the goal of GUIDE is to bridge this gap by incorporating real-time continuous human feedback.

The Power of Real-Time Human Feedback

GUIDE functions by allowing humans to observe AI’s actions in real-time and provide ongoing, nuanced feedback. This approach is reminiscent of how a skilled driving coach wouldn’t just shout “left” or “right,” but instead offer detailed guidance that fosters incremental improvements and deeper understanding. In its debut study, GUIDE helps AI learn how best to play hide-and-seek, demonstrating the effectiveness of this novel training strategy.

The game of hide-and-seek involves two beetle-shaped players, one red and one green, controlled by computers. The red player is working to advance its AI controller, while a human trainer provides feedback on its searching strategy. Unlike previous attempts at this sort of training strategy, GUIDE allows humans to hover a mouse cursor over a gradient scale to provide real-time feedback. This nuanced approach enables the AI to learn from constant, detailed human input.

The experiment involved 50 adult participants with no prior training or specialized knowledge, making it the largest-scale study of its kind. The researchers found that just 10 minutes of human feedback significantly improved the AI’s performance. GUIDE achieved up to a 30% increase in success rates compared to current state-of-the-art human-guided reinforcement learning methods. This strong quantitative and qualitative evidence highlights the effectiveness of the GUIDE approach, demonstrating its ability to boost adaptability and help AI independently navigate complex, dynamic environments.

The researchers also demonstrated that human trainers are only really needed for a short period of time. As participants provided feedback, the team created a simulated human trainer AI based on their insights within particular scenarios at particular points in time. This allows the seeker AI to continually train long after a human has grown weary of helping it learn. While training an AI “coach” that isn’t as good as the AI it’s coaching may seem counterintuitive, it’s actually a very human thing to do, as Professor Chen explains.