IEEE RA-L 20252025-02-14
Shichao Fan, Quantao Yang, Yajie Liu, Kun Wu, Zhengping Che, Qingjie Liu, Min Wan
School of Mechanical Engineering and Automation, Beihang University, Beijing, China, School of Mechanical Engineering and Automation, BeiHang University, China, Beihang University, Division of Robotics, Perception and Learning (RPL), KTH Royal Institute of Technology, Stockholm, Sweden, Division of Robotics, Perception and Learning (RPL), KTH Royal Institute of Technology, Sweden, KTH Royal Institute of Technology, School of Computer Science and Engineering, Beihang University, Beijing, China, School of Computer Science and Engineering, BeiHang University, China, Beijing Innovation Center of Humanoid Robotics, Beijing, China, Beijing Innovation Center of Humanoid Robotics, China, Beijing Advanced Sciences and Innovation Center
视觉语言动作数据高效预训练高效推理高效训练操作
针对长程模仿学习中示教稀缺与误差累积导致的失败,DTP将语言和视觉输入先转化为扩散生成的任务相关2D轨迹,再作为额外条件指导VLA策略学习,相当于用可视化的中间运动意图缩小感知到动作的鸿沟。该模块可插入Transformer策略,并利用机器人视频预训练提升数据效率;在CALVIN上从零训练较SOTA平均成功率高25%,真实机器人实验也有明显提升。