Multimodal / Agent / Robotics
ASPIRE: Agentic Skill Programming through Iterative Robot Exploration
** Runyu Lu, Yubo Wu, Ethan Kou, Letian Fu, Wenli Xiao, Ajay Mandlekar, Yinzhen Xu, Guanya Shi, Ken Goldberg, Ang Chen, Mosharaf Chowdhury, Yuke Zhu, Linxi "Jim" Fan, Guanzhi Wang
ASPIRE: Agentic Skill Programming through Iterative Robot Exploration
Authors: Runyu Lu, Yubo Wu, Ethan Kou, Letian Fu, Wenli Xiao, Ajay Mandlekar, Yinzhen Xu, Guanya Shi, Ken Goldberg, Ang Chen, Mosharaf Chowdhury, Yuke Zhu, Linxi "Jim" Fan, Guanzhi Wang
arXiv ID: 2607.00272
Problem: Traditional robot programming is difficult due to the need to orchestrate multimodal perception, manage contact dynamics, and handle diverse configurations and failures, with no existing system that autonomously writes, refines, and transfers reusable control programs across tasks and embodiments.
Key Methodology:
- A closed-loop robot execution engine that exposes fine-grained multimodal traces for autonomous failure diagnosis, repair synthesis, and validation
- A continually expanding skill library that distills validated fixes into reusable, transferable knowledge
- Evolutionary search that generates diverse task sequences and control programs to explore beyond single-trajectory refinement
Key Results:
- Surpasses prior methods by up to 77% on LIBERO-Pro manipulation under perturbation, 72% on Robosuite bimanual handover, and 32% on BEHAVIOR-1K long-horizon household tasks
- On LIBERO-Pro Long zero-shot generalization, ASPIRE achieves 31% success vs 4% for prior methods (despite their use of test-time reasoning and retries)
- Simulation-discovered skills transfer to real robots, substantially reducing real-world programming effort across different embodiments and robot APIs
Applied Context: Builders can use ASPIRE's approach to create robotic systems that self-improve over time, autonomously building a library of reusable skills that transfers across tasks and even across different robot hardware - dramatically reducing the manual programming burden for long-horizon manipulation.