Tenet Achieves Compact Robot Policies from Language in a Single Instantiation

Researchers are tackling the challenge of creating robots that seamlessly understand and execute natural language commands. Ariyan Bighashdel from Utrecht University and Vrije Universiteit Amsterdam, alongside Kevin Sebastian Luck from Vrije Universiteit Amsterdam, present TeNet , a novel framework for building compact robot policies directly from textual descriptions. This innovative approach bypasses the limitations of both rigid, hand-designed interfaces and computationally expensive end-to-end models, offering a pathway to efficient, real-time robot control. By leveraging pretrained large language models only during policy creation, TeNet generates lightweight controllers that operate swiftly on minimal state information, demonstrating significantly reduced size and strong performance across various benchmarks , paving the way for practical, language-driven robotics in resource-constrained environments.

TeNet learns robot policies from language

Scientists have unveiled TeNet (Text-to-Network), a groundbreaking framework capable of instantiating compact, task-specific robot policies directly from natural language descriptions. The research establishes a novel approach to robot control, moving beyond traditional high-level planning interfaces or computationally expensive end-to-end models. TeNet uniquely conditions a hypernetwork on text embeddings generated by a pretrained large language model (LLM) to create a fully executable policy, operating solely on low-dimensional state inputs at high control frequencies. This innovative design allows the system to leverage the general knowledge and paraphrasing robustness of LLMs while maintaining a lightweight and efficient execution profile, crucial for real-time applications.

The team achieved this by utilising language only during policy instantiation, effectively decoupling language understanding from the control loop itself. By employing a hypernetwork, TeNet generates a complete policy based on the initial text input, enabling rapid and resource-efficient control. To further enhance generalisation, researchers optionally ground the language in behaviour during training, aligning text embeddings with demonstrated actions, crucially, no demonstrations are required during inference. Experiments conducted on both MuJoCo and Meta-World benchmarks demonstrate that TeNet produces policies significantly smaller, orders of magnitude smaller, than sequence-based baseline models.

These policies not only exhibit strong performance in multi-task and meta-learning scenarios but also support high-frequency control, a critical requirement for many robotic applications. The study reveals that text-conditioned hypernetworks offer a practical pathway to building compact, language-driven controllers for resource-constrained robot control tasks demanding real-time responsiveness. This breakthrough addresses a key gap in the field, bridging the divide between expressive language-conditioned systems and efficient, language-enabled policies. Furthermore, the work opens exciting possibilities for future research, particularly in integrating TeNet with vision-grounded systems to create truly intelligent and adaptable robots. The researchers highlight that while their initial focus is on low-dimensional, trajectory-based domains, the principles of language-enabled hypernetworks could be extended to more complex, perception-rich environments. This investigation represents a significant step towards creating robots that can seamlessly understand and execute natural language instructions, paving the way for more intuitive and versatile human-robot interaction.

TeNet framework for language-conditioned robot policy generation offers

Scientists pioneered TeNet (Text-to-Network), a novel framework designed to directly instantiate compact robot policies from natural language descriptions. The study addresses a critical gap between expressive language-conditioned systems and efficient, compact policies by leveraging a hypernetwork conditioned on text embeddings from a pretrained large language model (LLM). This innovative approach generates a fully executable policy that operates solely on low-dimensional state inputs at high control frequencies, enabling real-time robot control. By employing language only during policy instantiation, TeNet inherits the general knowledge and paraphrasing robustness of LLMs while maintaining a lightweight and efficient execution profile.

Researchers engineered a system where the LLM produces text embeddings which then serve as the conditioning signal for the hypernetwork, effectively translating language into a task-specific policy. The hypernetwork, a crucial component of TeNet, synthesizes the policy weights, allowing for the creation of compact controllers suitable for resource-constrained robots. To enhance generalisation capabilities, the team optionally grounded language in behaviour during training by aligning text embeddings with demonstrated actions, crucially, no demonstrations are required during inference. This alignment enriches linguistic representations with behavioural semantics, leading to improved performance in multi-task and meta-learning scenarios.

Experiments employed the MuJoCo and Meta-World benchmarks to rigorously evaluate TeNet’s performance. The study meticulously compared TeNet against sequence-based baseline models, demonstrating that the generated policies are orders of magnitude smaller in size. Performance was assessed across both multi-task and meta-learning settings, confirming TeNet’s ability to support high-frequency control. The team quantified the reduction in policy size, highlighting the practical benefits for deployment on robots with limited computational resources. Furthermore, the research pioneered a method for enriching language representations with behavioural semantics through alignment with expert trajectories.

This grounding technique, applied solely during training, allows task descriptions to capture not only linguistic intent but also the underlying behavioural requirements. The resulting policies exhibit stronger generalisation capabilities, demonstrating the effectiveness of this innovative approach. This work investigates a first approach into utilising large robotic foundation models for resource-constrained robots via language-enabled hypernetworks for compact policy synthesis, isolating the role of language in policy instantiation without addressing perception.

Compact robot policies from language descriptions

Scientists have developed TeNet (Text-to-Network), a novel framework for creating compact robot policies directly from natural language descriptions. The research team successfully instantiated task-specific policies using a hypernetwork conditioned on text embeddings generated by a pretrained large language model (LLM), enabling operation solely on low-dimensional state inputs at high control frequencies. Experiments demonstrate that TeNet produces policies significantly smaller, orders of magnitude, than sequence-based baseline models, while maintaining strong performance across both multi-task and meta-learning settings. This breakthrough delivers a practical approach to building language-driven controllers for robots with limited computational resources and real-time requirements.

The team measured substantial reductions in policy size compared to existing methods, achieving a level of compactness crucial for deployment on resource-constrained platforms. Results show that by utilising language only during policy instantiation, TeNet inherits the general knowledge and paraphrasing robustness of LLMs without incurring the computational cost at execution time. Furthermore, the researchers optionally grounded language in behaviour during training by aligning text embeddings with demonstrated actions, improving generalisation capabilities without requiring demonstrations during inference. Tests confirm that this grounding technique enriches linguistic representations with behavioural semantics, leading to enhanced performance in complex scenarios.

Scientists recorded strong performance on both MuJoCo and Meta-World benchmarks, showcasing the versatility of the TeNet framework. The breakthrough delivers high-frequency control capabilities, essential for dynamic robotic applications, while maintaining a lightweight policy architecture. Measurements confirm that the generated policies operate efficiently on low-dimensional state inputs, reducing computational demands and enabling real-time responsiveness. Data shows that the framework effectively translates natural language instructions into executable robot behaviours, opening possibilities for intuitive human-robot interaction.

Researchers achieved improved generalisation through the optional grounding of language in behaviour, aligning text embeddings with expert trajectories during training. This alignment captured both linguistic intent and behavioural semantics, resulting in stronger performance in multi-task and meta-learning settings. The study highlights that this grounding is only necessary during training, allowing policies to be instantiated from text alone at inference time, further streamlining the control process. The work investigates a novel approach to utilising large robotic foundation models for resource-constrained robots via language-enabled hypernetworks for compact policy synthesis, isolating the role of language in policy instantiation.

TeNet delivers compact, high-frequency robot control for demanding

Scientists have developed TeNet (Text-to-Network), a new framework for creating compact robot policies directly from natural language instructions. This system utilises a hypernetwork conditioned on text embeddings from a pretrained large language model to generate a fully executable policy, operating on low-dimensional state inputs at high frequencies. By employing language only during policy creation, TeNet leverages the knowledge and robustness of large language models while maintaining efficiency during operation. Experiments conducted on MuJoCo and Meta-World benchmarks demonstrate that TeNet generates policies significantly smaller than sequence-based alternatives, achieving strong performance in both multi-task and meta-learning scenarios, and enabling high-frequency control, exceeding 9kHz. These findings suggest that text-conditioned hypernetworks represent a viable pathway towards building compact, language-driven controllers for resource-constrained robotic tasks demanding real-time responses. The authors acknowledge limitations relating to the need for clean demonstrations and the current inability to process multimodal inputs like vision alongside language; future work will focus on addressing these challenges and exploring reinforcement learning for fine-tuning.