SARA-RT: Scaling up Robotics Transformers with Self-Adaptive Robust Attention Figure 1
ICRA 20242023-12-04

SARA-RT: Scaling up Robotics Transformers with Self-Adaptive Robust Attention

Isabel Leal, Krzysztof Choromanski, Deepali Jain, Avinava Dubey, Jake Varley, Michael Ryoo, Yao Lu, Frederick Liu, Vikas Sindhwani, Quan Vuong, Tamas Sarlos, Ken Oslund, Karol Hausman, Kanishka Rao

Google,Mountain View,CA,94043, Google (United States)

视觉语言动作高效注意力高效推理高效架构机器人学习

机器人 Transformer/VLA 虽具备强语义泛化能力,但二次复杂度使 RT-1/RT-2 等难以在机器人端高频部署。SARA-RT 的核心是通过 up-training 将已预训练或微调的软最大注意力策略改造成自适应线性注意力,并训练投影以尽量保留尖锐注意力模式。实验覆盖 RT-2 类模型与大规模点云 PCT 控制器,在保持任务质量接近原模型的同时降低推理开销、提升速度。

Long-VLA: Unleashing Long-Horizon Capability of Vision Language Action Model for Robot Manipulation Figure 1
CoRL 20252025-08-27

Long-VLA: Unleashing Long-Horizon Capability of Vision Language Action Model for Robot Manipulation

Yiguo Fan, Pengxiang Ding, Shuanghao Bai, Xinyang Tong, Yuyang Zhu, Hongchao Lu, Fengqi Dai, Wei Zhao, Yang Liu, Siteng Huang, Zhaoxin Fan, Badong Chen, Donglin Wang

Westlake University, Zhejiang University, Xi’an Jiaotong University, Beijing Advanced Innovation Center for Future Blockchain and Privacy Computing, University, of Electronic Science and Technology of China

视觉语言动作高效注意力高效推理高效架构操作

针对现有 VLA 多聚焦短程操作、在长程多步骤任务中易受子任务依赖和技能衔接误差累积影响的问题,Long-VLA 保持端到端统一策略,不拆成多个局部模型,而是在输入层按移动/交互阶段进行相位感知掩码,使模型分别关注第三视角导航线索和自视角精细操作线索。作者还构建 L-CALVIN 评测,仿真与真实实验显示其较已有方法取得更高成功率和鲁棒性。

RetoVLA: Reusing Register Tokens for Spatial Reasoning in Vision-Language-Action Models Figure 1
2026 IEEE International Conference on Robotics and Automation (ICRA)2025-09-25

RetoVLA: Reusing Register Tokens for Spatial Reasoning in Vision-Language-Action Models

Jiyeon Koo, Taewan Cho, Hyunjoon Kang, Eunseom Pyo, Tae Gyun Oh, Taeryang Kim, Andrew Jaeyong Choi

School of Computing, Gachon University

视觉语言动作高效注意力高效推理高效架构模型压缩

面向轻量级VLA在压缩后易丢失3D布局与空间关系、难以实时部署的问题,RetoVLA将ViT中通常被丢弃的Register Tokens视为全局空间摘要,通过带门控的空间上下文注入路径送入动作规划模块,在不增加参数量的情况下补回场景理解。LIBERO与真实7自由度机械臂实验显示其优于SmolVLA基线,真实任务平均成功率由50.3%提升至67.4%,但高精度局部操作和反光物体仍有限制。

KV-Efficient VLA: A Method of Speed up Vision Language Model with RNN-Gated Chunked KV Cache Figure 1
arXiv preprint 20252025-09-20

KV-Efficient VLA: A Method of Speed up Vision Language Model with RNN-Gated Chunked KV Cache

Wanshun Xu, Long Zhuang, Lianlei Shan

University of Toronto, Tsinghua University

视觉语言动作高效注意力高效推理高效架构模型压缩令牌优化

本文针对 VLA 在长时序机器人控制中保留历史图像与动作上下文导致 KV cache 膨胀、注意力延迟和显存占用过高的问题,提出将 KV 按固定块压缩,并用 LSTM/RNN 门控按效用选择保留历史块的模型无关模块。在 LLaMA2-7B 与 Open X-Embodiment 微调设置下,报告平均节省 24.6% FLOPs、推理加速 1.34×、KV 内存降低 1.87×。

dVLA: Diffusion Vision-Language-Action Model with Multimodal Chain-of-Thought Figure 1
arXiv preprint 20252025-09-30

dVLA: Diffusion Vision-Language-Action Model with Multimodal Chain-of-Thought

Junjie Wen, Minjie Zhu, Jiaming Liu, Zhiyuan Liu, Yicun Yang, Linfeng Zhang, Shanghang Zhang, Yichen Zhu, Yi Xu

Midea Group Peking University, Shanghai Jiaotong University

视觉语言动作高效注意力高效推理高效架构扩散策略

针对现有 VLA 在图文动作联合训练中目标不一致、视觉生成与动作学习难统一且推理偏慢的问题,dVLA 将图像子目标、文本推理和动作序列离散化后置于统一扩散目标下重建,形成多模态 CoT,并用 prefix attention mask 与 KV cache 加速。其在 LIBERO 达到 96.4% 平均成功率,并在 Franka 实机含 bin-picking 任务中验证,推理最高约 2× 加速且性能损失较小。

Running VLAs at Real-time Speed Figure 1
arXiv preprint 20252025-10-30

Running VLAs at Real-time Speed

Yunchao Ma, Yizhuang Zhou, Yunhuan Yang, Tiancai Wang, Haoqiang Fan

视觉语言动作高效注意力高效推理高效架构实时控制

面向大参数 VLA 难以满足动态操作低时延需求的问题,本文指出 π0 级多视角 VLA 通过推理管线工程即可在单张消费级 RTX 4090 上实时运行:用 CUDA Graph 消除 CPU 调度开销,结合计算图折叠、QKV/RoPE 融合和内核级内存重排降低延迟,并提出全流式控制框架。实测双视角延迟达 27.3 ms,可处理 30Hz 视觉流、最高生成 480Hz 轨迹,在落笔抓取实验中达到 100% 成功率。

RoboMamba: Efficient Vision-Language-Action Model for Robotic Reasoning and Manipulation Figure 1
Advances in Neural Information Processing Systems 372024-06-06

RoboMamba: Efficient Vision-Language-Action Model for Robotic Reasoning and Manipulation

Jiaming Liu, Mengzhen Liu, Zhenyu Wang, Pengju An, Xiaoqi Li, Kaichen Zhou, Senqiao Yang, Renrui Zhang, Yandong Guo, Shanghang Zhang

State Key Laboratory of Multimedia Information Processing, School of Computer Science, Peking University; AI, Robotics, Beijing Academy of Artificial Intelligence (BAAI)

视觉语言动作Transformer替代架构高效推理高效架构操作轻量组件其他预训练策略高效训练

RoboMamba针对现有机器人VLA在复杂任务推理不足、微调与推理成本高的问题,将视觉编码器接入线性复杂度的Mamba语言模型,并通过视觉-语言对齐、机器人指令协同训练获得常识与机器人推理能力;在此基础上仅微调约0.1%参数的简单策略头来预测SE(3)位姿。实验显示其3.2B模型在通用/机器人推理基准上有竞争力,SAPIEN中达到SOTA,单A100数十分钟即可训练7MB策略头,推理速度约为既有VLA的3倍。

FlowRAM: Grounding Flow Matching Policy with Region-Aware Mamba Framework for Robotic Manipulation Figure 1
CVPR 20252025-06-19

FlowRAM: Grounding Flow Matching Policy with Region-Aware Mamba Framework for Robotic Manipulation

Sen Wang, Le Wang, Sanping Zhou, Jingyi Tian, Jiayi Li, Haowen Sun, Wei Tang

National Key Laboratory of Human-Machine Hybrid Augmented Intelligence, National Engineering Research Center for Visual Information and Applications, Institute of Artificial Intelligence and Robotics, Xi’an Jiaotong University, University of Illinois at Chicago

视觉语言动作Transformer替代架构高效架构操作扩散策略高效动作解码高效推理

FlowRAM针对扩散式机器人策略在高精度操作中推理慢、缺少任务相关局部几何感知的问题,将条件流匹配用于6DoF关键帧动作生成,并用动态半径调度从全局场景逐步收缩到关键区域,结合Mamba以线性复杂度融合多模态特征。在RLBench及高精度任务上达到SOTA,高精度平均成功率较既有方法提升12.0%,且真实任务中少于4步即可生成可行动作。

TinyVLA: Towards Fast, Data-Efficient Vision-Language-Action Models for Robotic Manipulation Figure 1
IEEE RA-L 20252024-09-19

TinyVLA: Towards Fast, Data-Efficient Vision-Language-Action Models for Robotic Manipulation

Junjie Wen, Yichen Zhu, Jinming Li, Minjie Zhu, Kun Wu, Zhiyuan Xu, Ning Liu, Ran Cheng, Chaomin Shen, Yaxin Peng, Feifei Feng, Jian Tang

Junjie Wen, Minjie Zhu, and Chaomin Shen are with East China Normal University, Shanghai 200042, China, Kun Wu is with Syracuse University, New York 13244, USA, Jinming Li and Yaxin Peng are with Shanghai University, Shanghai 201900, China

视觉语言动作高效动作解码高效推理操作机器人学习轻量组件高效架构其他预训练策略高效训练

TinyVLA针对现有VLA依赖大模型与自回归动作生成导致推理慢、且需大规模机器人预训练的问题,采用小型高速多模态骨干初始化策略,并在微调阶段接入扩散策略解码器直接生成连续动作。实验显示其无需OpenX级机器人预训练,在仿真和真实单/双臂任务中达到与OpenVLA相当或更好的成功率,TinyVLA-H真实任务高25.7%,且参数少5.5倍,泛化到新指令、物体、位置和环境变化表现稳定。

Accelerating vision-language-action model integrated with action chunking via parallel decoding Figure 1
IROS 20252025-03-04

Accelerating vision-language-action model integrated with action chunking via parallel decoding

Wenxuan Song, Jiayi Chen, Pengxiang Ding, Han Zhao, Wei Zhao, Zhide Zhong, Zongyuan Ge, Zhijun Li, Donglin Wang, Jun Ma, Lujia Wang, Haoang Li

Hong Kong University of Science and Technology, University of Hong Kong, Westlake University,AI Division, School of Engineering,Hangzhou,China, Westlake University, Monash University,Department of Data Science & AI,Melbourne,Australia, Monash University, Harbin Institute of Technology,School of Computer Science and Technology,Harbin,China, Harbin Institute of Technology

视觉语言动作高效动作解码高效推理

该文针对 VLA 结合 action chunking 后动作维度随块长线性增长、 autoregressive 解码拖慢控制频率的问题,提出 PD-VLA:将顺序动作解码重写为非线性方程组,并用并行不动点迭代同时预测多个 token。方法无需训练或改模型,可叠加其他加速手段;仿真中在保持相近成功率下,7 自由度机械臂执行频率达基础 VLA 的 2.52 倍,真实任务也验证了可用性。

Fine-Tuning Vision-Language-Action Models: Optimizing Speed and Success Figure 1
Robotics: Science and Systems 20252025-02-27

Fine-Tuning Vision-Language-Action Models: Optimizing Speed and Success

Moo Jin Kim, Chelsea Finn, Percy Liang

视觉语言动作高效动作解码高效推理高效训练监督微调

针对 VLA 迁移到新机器人时微调策略不清、且自回归动作生成难以满足高频控制的问题,论文系统比较解码方式、动作表示和训练目标,提出 OFT:并行解码结合动作分块、连续动作表示与 L1 回归。该配方在 LIBERO 将 OpenVLA 平均成功率从 76.5% 提升到 97.1%,动作生成吞吐提升 26 倍;在真实双臂 ALOHA 上也优于 π0、RDT-1B 及从头训练的模仿学习策略。

HybridVLA: Collaborative Diffusion and Autoregression in a Unified Vision-Language-Action Model Figure 1
arXiv preprint 20252025-03-13

HybridVLA: Collaborative Diffusion and Autoregression in a Unified Vision-Language-Action Model

Jiaming Liu, Hao Chen, Pengju An, Zhuoyang Liu, Renrui Zhang, Chenyang Gu, Xiaoqi Li, Ziyu Guo, Sixiang Chen, Mengzhen Liu, Chengkai Hou, Mengdi Zhao, KC alex Zhou, Pheng-Ann Heng, Shanghang Zhang

State Key Laboratory of Multimedia Information Processing, School of Computer Science, Peking University; Beijing Academy of Artificial Intelligence (BAAI); CUHK

视觉语言动作高效动作解码高效推理扩散策略令牌优化模型压缩

HybridVLA针对自回归VLA离散化动作损失精度、扩散VLA又未充分利用LLM令牌级推理的问题,将扩散去噪与下一令牌预测统一到同一LLM骨干,并用自回归置信度自适应融合两类动作;在大规模机器人数据预训练后,仿真和真实任务平均成功率较SOTA分别提升14%和19%,但部分增益可能来自数据与算力规模。

FreqPolicy: Efficient Flow-based Visuomotor Policy via Frequency Consistency Figure 1
NeurIPS 20252025-06-10

FreqPolicy: Efficient Flow-based Visuomotor Policy via Frequency Consistency

Yifei Su, Ning Liu, Dong Chen, Zhen Zhao, Kun Wu, Meng Li, Zhiyuan Xu, Zhengping Che, Jian Tang

Beijing Innovation Center of Humanoid Robotics, Institute of Automation of Chinese Academy of Sciences

视觉语言动作高效动作解码高效推理

生成式视觉运动策略虽能建模多峰动作,但多步采样难以满足实时控制;直接借用图像生成加速又忽略动作轨迹的时间连续性。FreqPolicy在flow-based策略中引入频域一致性,并用自适应频率损失强调随任务阶段变化的关键频率成分,从而支持一步动作生成。其在53个仿真任务优于现有一步生成器,接入VLA后在Libero 40任务实现约5倍加速且性能不降,真实机器人达到93.5Hz推理。

CEED-VLA: Consistency Vision-Language-Action Model with Early-Exit Decoding Figure 1
arXiv preprint 20252025-06-16

CEED-VLA: Consistency Vision-Language-Action Model with Early-Exit Decoding

Wenxuan Song, Jiayi Chen, Pengxiang Ding, Yuxin Huang, Han Zhao, Donglin Wang, Haoang Li

HKUST(GZ), Westlake University, Zhejiang University

视觉语言动作高效动作解码高效推理令牌优化

CEED-VLA针对VLA在高频、灵巧操作中受自回归动作解码拖慢的问题,指出普通Jacobi并行解码因训练分布不匹配和低效收敛迭代而加速有限。方法用一致性蒸馏让学生模型在一次迭代中预测多个正确动作令牌,并以混合标签监督抑制蒸馏误差,再通过早退解码放宽收敛条件。仿真与真实机器人实验显示其在不同基线实现约2–4.1×、总体超过4×推理加速,同时保持较高任务成功率。

MinD: Learning A Dual-System World Model for Real-Time Planning and Implicit Risk Analysis Figure 1
arXiv preprint 20252025-06-23

MinD: Learning A Dual-System World Model for Real-Time Planning and Implicit Risk Analysis

Xiaowei Chi, Kuangzhi Ge, Jiaming Liu, Siyuan Zhou, Peidong Jia, Zichen He, Yuzhen Liu, Tingguang Li, Lei Han, Sirui Han, Shanghang Zhang, Yike Guo

视觉语言动作高效动作解码高效推理世界模型实时控制层级系统高效架构

MinD针对视频生成世界模型虽能预测未来但扩散推理过慢、难以接入实时机器人控制的问题,提出低频视频“想象”与高频扩散策略并行的双系统架构;其关键洞察是控制并不需要完整去噪视频,单步低分辨率潜变量已可提供未来状态信号,并用DiffMatcher对齐视频与动作扩散中间表示。实验在RL-Bench达63%成功率、真实Franka达60%且11.3 FPS,并可提前识别约74%潜在失败。

VQ-VLA: Improving Vision-Language-Action Models via Scaling Vector-Quantized Action Tokenizers Figure 1
ICCV 20252025-07-01

VQ-VLA: Improving Vision-Language-Action Models via Scaling Vector-Quantized Action Tokenizers

Yating Wang, Haoyi Zhu, Mingyu Liu, Jiange Yang, Hao-Shu Fang, Tong He

Shanghai AI Lab Tongji USTC ZJU NJU SJTU

视觉语言动作高效动作解码高效推理模型压缩动作表示

VQ-VLA针对VLA连续动作输出冗长、推理慢且长程误差易累积的问题,将动作序列用卷积残差VQ-VAE压缩为离散token,并指出动作轨迹的仿真到真实域差较小,可用大规模合成轨迹低成本扩展tokenizer。实验在LIBERO和Franka真实平台显示,数据规模扩大带来近线性收益,推理更快、动作更平滑,长程真实任务成功率最高提升约30%。

Spec-VLA: Speculative Decoding for Vision-Language-Action Models with Relaxed Acceptance Figure 1
Proceedings of the 2025 Conference on Empirical Methods in Natural Language Processing2025-07-30

Spec-VLA: Speculative Decoding for Vision-Language-Action Models with Relaxed Acceptance

Songsheng Wang, Rucheng Yu, Zhihang Yuan, Chao Yu, Feng Gao, Yu Wang, Derek F. Wong

CT Lab, Department of Computer and Information Science, University of Macau, Tsinghua University, Zhongguancun Academy

视觉语言动作高效动作解码高效推理令牌优化

Spec-VLA针对VLA模型因大规模VLM骨干和自回归动作令牌生成带来的推理延迟,将LLM中的推测解码引入机器人动作预测;其关键洞察是普通严格验收在贪心VLA解码中收益有限,因此利用动作令牌表示的相对距离放宽验收。实验显示验收长度提升约25%–44%,相对OpenVLA最高达1.42×加速,且成功率基本不受影响。

Leveraging OS-Level Primitives for Robotic Action Management Figure 1
arXiv preprint 20252025-08-14

Leveraging OS-Level Primitives for Robotic Action Management

Wenxin Zheng, Boyang Li, Bin Xu, Erhu Feng, Jinyu Gu, Haibo Chen

Shanghai Jiao Tong University, Shanghai, Southern University of Science and Technology

视觉语言动作高效动作解码高效推理机器人学习令牌优化模型压缩

针对VLA机器人因训练数据不足而泛化差、动作片段不可中断且跨片段重复计算导致效率低的问题,论文提出从系统层而非重训模型入手的AMS,将动作片类比OS时间片,引入动作异常、动作上下文与动作重放来中断错误传播、复用推理状态并处理重复任务。仿真和真实机器人上,AMS使长程任务成功率提升7至24倍,端到端时间减少29%至74%。

NinA: Normalizing Flows in Action. Training VLA Models with Normalizing Flows Figure 1
arXiv preprint 20252025-08-23

NinA: Normalizing Flows in Action. Training VLA Models with Normalizing Flows

Denis Tarasov, Alexander Nikulin, Ilya Zisman, Albina Klepach, Nikita Lyubaykin, Andrei Polubarov, Alexander Derevyagin, Vladislav Kurenkov

AIRI, ETH Zürich &Alexander Nikulin, AIRI, Innopolis University &Andrei Polubarov, AIRI, Innopolis University

视觉语言动作高效动作解码高效推理高效训练扩散策略

针对VLA中扩散动作解码器需多步去噪、难以满足高频控制的问题,NinA将FLOWER的动作专家替换为条件归一化流,通过可逆变换实现一次采样,并保留精确似然训练;在LIBERO微调中,其成功率接近扩散版FLOWER,同时推理达到数量级加速且参数更少,但真实机器人验证仍未充分说明。

Discrete Diffusion VLA: Bringing Discrete Diffusion to Action Decoding in Vision-Language-Action Policies Figure 1
arXiv preprint 20252025-08-27

Discrete Diffusion VLA: Bringing Discrete Diffusion to Action Decoding in Vision-Language-Action Policies

Zhixuan Liang, Yizhuo Li, Tianshuo Yang, Chengyue Wu, Sitong Mao, Tian Nian, Liuao Pei, Shunbo Zhou, Xiaokang Yang, Jiangmiao Pang, Yao Mu, Ping Luo

The University of Hong Kong Shanghai AI Laboratory, Shanghai Jiao Tong University Huawei Cloud Computing Technologies Co., Ltd, Shanghai Jiao Tong University Huawei Cloud Computing Technologies Co

视觉语言动作高效动作解码高效推理扩散策略

针对现有 VLA 要么自回归逐 token 解码、要么外挂 MLP/连续扩散头导致信息路径割裂和推理低效的问题,本文将动作分箱为离散 chunk,并在同一 Transformer 内用离散扩散进行并行渐进细化;自适应“先易后难”解码和二次 re-masking 允许重访低置信动作 token。实验在 LIBERO 达到 96.3% 平均成功率,在 SimplerEnv-Fractal/Bridge 分别达 64.1%/54.2% overall,且函数调用少于 AR 基线。

CLIP-RT: Learning Language-Conditioned Robotic Policies from Natural Language Supervision Figure 1
Robotics: Science and Systems 20252024-11-01

CLIP-RT: Learning Language-Conditioned Robotic Policies from Natural Language Supervision

Gi-Cheon Kang, Junghyun Kim, Kyuhwan Shim, Jun Ki Lee, Byoung-Tak Zhang

Seoul National University, Tommoro Robotics

视觉语言动作轻量组件高效架构机器人学习人在回路数据采集数据采集数据增强

针对真实机器人示教依赖专家和遥操作硬件、难以规模化的问题,CLIP-RT把自然语言监督作为数据采集与策略学习接口:非专家用语言引导机器人生成示教,并通过随机轨迹增强扩展数据;模型改造预训练 CLIP,用对比模仿学习预测语言化运动基元。实机新技能上较 OpenVLA 平均成功率高 24%且参数少 7 倍,LIBERO 达约 93.1%成功率与 163Hz 推理吞吐。

Scalable, Training-Free Visual Language Robotics: a modular multi-model framework for consumer-grade GPUs Figure 1
SII 20252025-02-03

Scalable, Training-Free Visual Language Robotics: a modular multi-model framework for consumer-grade GPUs

Marie Samson, Bastien Muraccioli, Fumio Kanehiro

National Institute of Advanced Industrial Science and Technology (AIST),CNRS-AIST JRL (Joint Robotics Laboratory),Japan, National Institute of Advanced Industrial Science and Technology

视觉语言动作轻量组件高效架构高效训练机器人学习

针对现有 VLA 模型算力需求高、常需重训且难迁移到新机器人/任务的问题,论文提出免训练的模块化 SVLR,将 Mini-InternVL、CLIPSeg、Phi-3 与句向量匹配组合为高层控制器,用预定义任务及参数生成动作序列。系统可在移动版 RTX 2070 上运行,并在 UR10 抓取放置任务中表现可行;但评测任务较窄,与主流 VLA 的系统比较和泛化能力文中未充分说明。

NORA: A Small Open-Sourced Generalist Vision Language Action Model for Embodied Tasks Figure 1
arXiv preprint 20252025-04-28

NORA: A Small Open-Sourced Generalist Vision Language Action Model for Embodied Tasks

Chia-Yu Hung, Qi Sun, Pengfei Hong, Amir Zadeh, Chuan Li, U-Xuan Tan, Navonil Majumder, Soujanya Poria

Singapore University of Technology and Design

视觉语言动作轻量组件高效架构机器人学习

NORA针对现有VLA模型参数量大、视觉 grounding 不足而难以实时部署的问题,采用Qwen-2.5-VL-3B作为3B骨干,并用FAST+将连续动作高效离散化,在97万真实机器人示范上训练。实验覆盖真实任务与LIBERO,显示其在更低计算开销下优于若干更大VLA基线;但具体增益中来自骨干、数据规模或tokenizer的占比仍需进一步拆分说明。

SmolVLA: A vision-language-action model for affordable and efficient robotics Figure 1
arXiv preprint 20252025-06-02

SmolVLA: A vision-language-action model for affordable and efficient robotics

Mustafa Shukor, Dana Aubakirova, Francesco Capuano, Pepijn Kooijmans, Steven Palma, Adil Zouitine, Michel Aractingi, Caroline Pascal, Martino Russi, Andres Marafioti, Simon Alibert, Matthieu Cord, Thomas Wolf, Remi Cadene

视觉语言动作轻量组件高效推理高效架构机器人学习层剪枝模型压缩令牌优化互联网/跨域数据数据采集

SmolVLA针对现有VLA参数规模大、训练和部署成本高且依赖昂贵机器人数据的问题,提出面向社区数据和低成本硬件的轻量视觉语言动作模型。其核心是在小型预训练VLM上减少视觉token、跳过部分层,并用轻量交叉注意力动作专家与异步推理栈提升控制响应。文中显示其用少于3万公开视频/机器人回合训练,可在单GPU训练、消费级GPU或CPU部署,在仿真和真实任务上达到接近或超过约10倍规模VLA的表现。

EdgeVLA: Efficient Vision-Language-Action Models Figure 1
arXiv preprint 20252025-07-18

EdgeVLA: Efficient Vision-Language-Action Models

Wesley Maa, Matthew Freed, Jingxiang Mo, Winston Hsiao, Aaron Xie, Viraj Tipnis, Benjamin Bolte

Jingxiang Mo is with McGill University

视觉语言动作轻量组件高效推理高效架构实时控制

EdgeVLA瞄准OpenVLA等大规模VLA难以在移动机器人边缘设备上实时部署的问题,提出用约1B参数的Qwen2-0.5B结合SigLIP/DINOv2视觉编码器,并取消末端位姿自回归预测、改为一次性联合输出动作。早期在BridgeData V2和OpenX上的训练曲线接近OpenVLA,同时推理或训练迭代约快7倍、显存需求更低;但真实机器人多平台效果仍需后续验证。

No Figure
arXiv preprint 20252025

MiniVLA: A Better VLA with a Smaller Footprint

作者信息待提取

视觉语言动作轻量组件高效架构

针对 OpenVLA 7B 参数导致训练和推理开销高、单图输入与逐步动作离散化受限的问题,MiniVLA 将语言骨干换为 Qwen2.5-0.5B,使总规模约 1B,并引入 VQ 动作块表示、多图像历史/腕部视角支持。结果显示其在 LIBERO-90 上以约 7 倍更小模型接近 OpenVLA 基线,推理快 2.5 倍;结合动作块和多图输入后成功率由 62% 提升到约 82%。

GeRM: A Generalist Robotic Model with Mixture-of-experts for Quadruped Robot Figure 1
IROS 20242024-03-20

GeRM: A Generalist Robotic Model with Mixture-of-experts for Quadruped Robot

Wenxuan Song, Han Zhao, Pengxiang Ding, Can Cui, Shangke Lyu, Yaning Fan, Donglin Wang

MiLAB, Westlake University, China

视觉语言动作专家混合高效架构机器人学习数据高效预训练高效推理高效训练仿真数据采集数据采集仿真到现实

针对四足机器人多任务学习中人工示教昂贵、模仿学习受专家质量限制且大模型强化学习计算受限的问题,GeRM 将离线 RL 与 Transformer式 VLA 结合,并在 FFN 中引入稀疏 MoE(8 个专家、每 token 激活 2 个)以扩大容量而控制推理成本;同时自动采集 QUARD-Auto,包含成功与失败轨迹。实验在 99 个任务上优于对比方法,并显示更高训练/推理效率及一定涌现技能潜力。

FedVLA: Federated Vision-Language-Action Learning with Dual Gating Mixture-of-Experts for Robotic Manipulation Figure 1
ICCV 20252025-08-04

FedVLA: Federated Vision-Language-Action Learning with Dual Gating Mixture-of-Experts for Robotic Manipulation

Cui Miao, Tao Chang, Meihan Wu, Hongbin Xu, Chun Li, Ming Li, Xiaodong Wang

National University of Defense Technology, Bytedance Seed, University of Siedlce, Shenzhen MSU-BIT University, Shenzhen Technology University, Guangdong Laboratory of Artificial Intelligence and Digital Economy (SZ)

视觉语言动作专家混合高效架构操作机器人学习

FedVLA针对VLA机器人在家庭等用户场景微调时需上传隐私数据、且各客户端任务差异大的问题,将联邦学习引入视觉-语言-动作训练。其关键在于用指令引导场景解析提取任务相关物体特征,结合双门控MoE让token与自感知专家双向决定激活,并按专家相似性做服务器聚合。仿真和真实机器人实验显示,其成功率接近集中式训练,同时DGMoE较普通MoE降低计算开销。

Learning to See and Act: Task-Aware View Planning for Robotic Manipulation Figure 1
arXiv preprint 20252025-08-07

Learning to See and Act: Task-Aware View Planning for Robotic Manipulation

Yongjie Bai, Zhouxia Wang, Yang Liu, Kaijun Luo, Yifan Wen, Mingtong Dai, Weixing Chen, Ziliang Chen, Lingbo Liu, Guanbin Li, Liang Lin

Sun Yat-sen University, Pengcheng Laboratory, Nanyang Technological University, Shenzhen Institutes of Advanced Technology, Chinese Academy of Sciences, X-Era AI Lab

视觉语言动作专家混合高效架构操作世界模型其他预训练策略高效训练

针对多任务机器人操作中固定视角易遮挡、共享视觉编码器易产生任务干扰的问题,论文提出 TVVE:先从重建场景中按任务选择虚拟相机视角并重渲染观测,再用 TaskMoE 根据指令与场景动态路由到专家特征。作者还构建含视觉扰动和相机位姿变化的 RLBench-OG;在 RLBench、RLBench-OG 及真实机器人实验中,TVVE 相比强基线取得更高成功率,并在遮挡、视觉干扰和未见指令下表现更稳健。

HiRT: Enhancing Robotic Control with Hierarchical Robot Transformers Figure 1
arXiv preprint2024-09-12

HiRT: Enhancing Robotic Control with Hierarchical Robot Transformers

Jianke Zhang, Yanjiang Guo, Xiaoyu Chen, Yen-Jen Wang, Yucheng Hu, Chengming Shi, Jianyu Chen

Institute for Interdisciplinary Information Sciences, Tsinghua University, University of California, Berkeley, Shanghai Qizhi Institute

视觉语言动作层级系统高效架构操作机器人学习

HiRT针对大规模VLA依赖数十亿参数VLM导致推理慢、控制频率低,难以处理动态交互的问题,提出层级Robot Transformer:低频运行InstructBLIP提取相对稳定的语义/任务特征,高频轻量视觉策略在其条件引导下独立输出动作,并支持异步频率权衡。实验显示静态任务中控制频率约翻倍且成功率相当,真实动态操作任务成功率由48%提升到75%。

Towards Synergistic, Generalized, and Efficient Dual-System for Robotic Manipulation Figure 1
arXiv preprint 20242024-10-10

Towards Synergistic, Generalized, and Efficient Dual-System for Robotic Manipulation

Qingwen Bu, Hongyang Li, Li Chen, Jisong Cai, Jia Zeng, Heming Cui, Maoqing Yao, Yu Qiao

Shanghai Jiao Tong Univeristy, The University of Hong Kong, Shanghai AI Lab

视觉语言动作层级系统高效推理高效架构操作

面向VLA通用策略泛化强但推理慢、训练/适配成本高,而专用策略高效却泛化弱的问题,RoboDual将OpenVLA式通用模型作为高层条件源,以轻量扩散Transformer专用策略进行多步动作展开和多模态动作细化,实现训练数据与执行频率的解耦。实验显示,仅增加约20M可训练参数即可相对OpenVLA在真实场景提升26.7%、CALVIN提升12%,且5%示教数据下仍保持较强性能,真实部署控制频率提高3.8倍。

A Dual Process VLA: Efficient Robotic Manipulation Leveraging VLM Figure 1
arXiv preprint 20242024-10-21

A Dual Process VLA: Efficient Robotic Manipulation Leveraging VLM

ByungOk Han, Jaehong Kim, Jinhyeok Jang

视觉语言动作层级系统高效推理高效架构操作

这篇工作针对现有 VLA 依赖大 VLM/VLA 导致推理慢、动作不连续的问题,提出受双过程理论启发的层级 DP-VLA:低频运行的大型 System 2 负责视觉语言理解与意图/规划,小型 System 1 高频处理传感输入与运动控制,从而解耦推理与执行并便于替换更强 VLM。RoboCasa 实验显示其推理更快且任务成功率高于对比 VLA。

HAMSTER: Hierarchical Action Models For Open-World Robot Manipulation Figure 1
arXiv preprint2025-02-08

HAMSTER: Hierarchical Action Models For Open-World Robot Manipulation

Yi Li, Yuquan Deng, Jesse Zhang, Joel Jang, Marius Memmel, Raymond Yu, Caelan Reed Garrett, Fabio Ramos, Dieter Fox, Anqi Li, Abhishek Gupta, Ankit Goyal

NVIDIA, University of Washington, University of Southern California

视觉语言动作层级系统高效架构操作数据高效预训练高效推理高效训练

HAMSTER针对机器人数据昂贵且单体VLA难以利用离域数据的问题,将任务理解与精细控制解耦:高层VLM从图像和语言预测粗2D末端路径,可由无动作视频、草图或仿真廉价监督;低层3D策略据此执行精确操作。该层级表示被证明能跨形态、动力学、外观和语义差异迁移,真实机器人上相较OpenVLA在七类泛化轴平均成功率提升20个百分点,约50%相对增益。

Fast-in-Slow: A Dual-System Foundation Model Unifying Fast Manipulation within Slow Reasoning Figure 1
arXiv preprint 20252025-06-02

Fast-in-Slow: A Dual-System Foundation Model Unifying Fast Manipulation within Slow Reasoning

Hao Chen, Jiaming Liu, Chenyang Gu, Zhuoyang Liu, Renrui Zhang, Xiaoqi Li, Xiao He, Yandong Guo, Chi-Wing Fu, Shanghang Zhang, Pheng-Ann Heng

The Chinese University of Hong Kong, State Key Laboratory of Multimedia Information Processing, School of Computer Science, Peking University, Robotics, Beijing Academy of Artificial Intelligence (BAAI)

视觉语言动作层级系统高效推理高效架构操作

针对VLA模型虽具备视觉语言推理能力但执行频率低、现有双系统又将快慢模块割裂的问题,FiS将System 1执行模块嵌入预训练VLM的System 2中并部分共享参数,配合异步频率、异构输入和双感知协同训练,使快速控制能利用慢速推理表征。实验中其平均成功率较SOTA在仿真提升8%、真实任务提升11%,action chunk为8时达到117.7 Hz。

DeeR-VLA: Dynamic Inference of Multimodal Large Language Models for Efficient Robot Execution Figure 1
Advances in Neural Information Processing Systems 372024-11-04

DeeR-VLA: Dynamic Inference of Multimodal Large Language Models for Efficient Robot Execution

Yang Yue, Yulin Wang, Bingyi Kang, Yizeng Han, Shenzhi Wang, Shiji Song, Jiashi Feng, Gao Huang

Department of Automation, BNRist, Tsinghua University, ByteDance

视觉语言动作层剪枝高效推理模型压缩剪枝

面向机器人端部署多模态大模型时算力、显存与时延受限的问题,DeeR-VLA利用“多数控制状态较容易、无需完整模型”的洞察,在VLA模型中加入多出口并按动作一致性和资源预算动态早退,同时设计时序训练以稳定动作预测。在CALVIN上,LLM计算量降低5.2–6.5倍、显存降低2–6倍,性能基本不降。

MoLe-VLA: Dynamic Layer-skipping Vision Language Action Model via Mixture-of-Layers for Efficient Robot Manipulation Figure 1
Proceedings of the AAAI Conference on Artificial Intelligence 20262025-03-26

MoLe-VLA: Dynamic Layer-skipping Vision Language Action Model via Mixture-of-Layers for Efficient Robot Manipulation

Rongyu Zhang, Menghang Dong, Yuan Zhang, Liang Heng, Xiaowei Chi, Gaole Dai, Li Du, Yuan Du, Shanghang Zhang

Nanjing University; The Hong Kong Polytechnic University, State Key Laboratory of Multimedia Information Processing, School of Computer Science, Peking University; Beijing Academy of Artificial Intelligence, The Hong Kong University of Science and Technology

视觉语言动作层剪枝高效推理高效架构模型压缩

面向VLA模型在机器人实时控制中计算、存储开销过高的问题,MoLe-VLA将LLM各层视为“专家”,用具备视觉空间与语言/状态时序感知的STAR路由器动态选择需激活的层,避免早退丢失末层语义;同时以CogKD从全层教师中蒸馏认知特征,缓解跳层后的能力损失。在RLBench和真实环境中,相比基线平均成功率提升约8%,LLM计算开销最高降低5.6倍。

EfficientVLA: Training-Free Acceleration and Compression for Vision-Language-Action Models Figure 1
arXiv preprint 20252025-06-11

EfficientVLA: Training-Free Acceleration and Compression for Vision-Language-Action Models

Yantai Yang, Yuhao Wang, Zichen Wen, Luo Zhongwei, Chang Zou, Zhipeng Zhang, Chuan Wen, Linfeng Zhang

School of Artificial Intelligence, Shanghai Jiao Tong University, Harbin Institute of Technology, Xi’an Jiaotong University, University of Electronic Science and Technology of China

视觉语言动作层剪枝高效推理模型压缩高效训练令牌优化

面向扩散式 VLA 在机器人端推理中同时受 LLM 内存、视觉 token 计算和动作头迭代开销限制的问题,EfficientVLA 提出无需训练的整体压缩框架:按层间表示相似性剪除语言模块冗余层,按任务相关性与特征多样性筛选视觉 token,并缓存扩散动作头跨步中间特征。在 CogACT/SIMPLER 上实现 1.93× 加速、FLOPs 降至 28.9%,成功率仅下降 0.6%。

RLRC: Reinforcement Learning-based Recovery for Compressed Vision-Language-Action Models Figure 1
arXiv preprint 20252025-06-21

RLRC: Reinforcement Learning-based Recovery for Compressed Vision-Language-Action Models

Yuxuan Chen, Xiao Li

School of Mechanical Engineering, Shanghai Jiao Tong University

视觉语言动作层剪枝模型压缩强化学习剪枝量化

这篇论文针对 VLA 参数量大、延迟高、难以上机器人端侧部署的问题,先系统比较量化、剪枝、蒸馏等压缩手段,再提出 RLRC:对 VLA 中 LLM 部分做结构化层剪枝,用 SFT 与强化学习恢复控制性能,最后再做后训练量化。实验显示其可将显存占用最高降至 1/8、吞吐提升 2.3 倍,并在任务成功率上保持甚至超过原模型。

On-Device Diffusion Transformer Policy for Efficient Robot Manipulation Figure 1
ICCV 20252025-08-01

On-Device Diffusion Transformer Policy for Efficient Robot Manipulation

Yiming Wu, Huan Wang, Zhenghao Chen, Jianxin Pang, Dong Xu

School of Computing and Data Science, The University of Hong Kong, School of Engineering, Westlake University, School of Information and Physical Sciences, University of Newcastle, UBTech Robotics Corp

视觉语言动作层剪枝高效推理模型压缩操作

面向扩散策略在移动机器人上因多步去噪和大模型内存占用而难以实时部署的问题,论文提出 LightDP:先分析 DP-T、MDT-V 等结构并定位去噪网络为主要延迟瓶颈,再将可学习层剪枝与重训练统一优化,并结合一致性蒸馏减少采样步数。实验在 PushT、Robomimic、CALVIN、LIBERO 及真实机械臂上显示,其可在手机/Jetson 等设备实现实时或显著更快推理,同时保持接近原扩散策略的成功率。

FLOWER: Democratizing Generalist Robot Policies with Efficient Vision-Language-Action Flow Policies Figure 1
arXiv preprint 20252025-09-05

FLOWER: Democratizing Generalist Robot Policies with Efficient Vision-Language-Action Flow Policies

Moritz Reuss, Hongyi Zhou, Marcel Rühle, Ömer Erdinç, Fabian Otto, Rudolf Lioutikov

Intuitive Robots Lab, Karlsruhe Institute of Technology, Germany, Microsoft Research

视觉语言动作层剪枝高效推理模型压缩剪枝

面向通用机器人 VLA 部署中参数、显存和预训练成本过高的问题,FLOWER 的核心洞察是无需保留完整 VLM:通过中间层模态融合剪掉 30–50% VLM/LLM 层,把预算转给 flow action head,并用动作空间 Global-AdaLN 进一步减小头部参数。最终 950M 模型仅用约 200 H100 GPU 小时预训练,在 10 个仿真与真实基准、190 个任务上接近更大 VLA,并在 CALVIN ABC 达到 4.53。

OpenVLA: An Open-Source Vision-Language-Action Model Figure 1
arXiv preprint 20242024-06-13

OpenVLA: An Open-Source Vision-Language-Action Model

Moo Jin Kim, Karl Pertsch, Siddharth Karamcheti, Ted Xiao, Ashwin Balakrishna, Suraj Nair, Rafael Rafailov, Ethan Foster, Grace Lam, Pannag Sanketi, Quan Vuong, Thomas Kollar, Benjamin Burchfiel, Russ Tedrake, Dorsa Sadigh, Sergey Levine, Percy Liang, Chelsea Finn

视觉语言动作量化模型压缩监督微调高效训练

针对现有VLA多为闭源且缺少面向新机器人高效适配方法的问题,OpenVLA将Llama 2与融合DINOv2、SigLIP特征的视觉编码器结合,在97万真实机器人示范上监督微调成7B开源视觉语言动作模型。其在29个跨本体任务上以更少参数超过RT-2-X 16.5%成功率,微调后优于Diffusion Policy 20.4%,并显示LoRA与量化可在消费级GPU上适配和部署且性能基本不降。

Quantization-Aware Imitation-Learning for Resource-Efficient Robotic Control Figure 1
arXiv preprint 20242024-12-02

Quantization-Aware Imitation-Learning for Resource-Efficient Robotic Control

Seongmin Park, Hyungmin Kim, Wonseok Jeon, Juyoung Yang, Byeongwook Jeon, Yoonseon Oh, Jungwook Choi

Hanyang University

视觉语言动作量化高效推理模型压缩操作

面向机器人操作与自动驾驶中日益庞大的模仿学习/VLA策略,论文关注其在边缘设备上推理慢、能耗高且量化易扰动动作序列的问题。作者提出QAIL,在微调中显式注入量化,并用QBC让量化策略对齐FP32动作分布以抑制误差累积。OpenVLA在LIBERO上4-bit权重量化基本保持成功率,并在边缘GPU获最高2.5×加速和2.5×节能;CILRS自动驾驶4-bit权重/激活量化最高达3.7×加速、3.1×节能。

FAST: Efficient Action Tokenization for Vision-Language-Action Models Figure 1
Robotics: Science and Systems 20252025-01-16

FAST: Efficient Action Tokenization for Vision-Language-Action Models

Karl Pertsch, Kyle Stachowicz, Brian Ichter, Danny Driess, Suraj Nair, Quan Vuong, Oier Mees, Chelsea Finn, Sergey Levine

UC Berkeley, Stanford

视觉语言动作量化高效推理模型压缩动作表示令牌优化高效动作表示

该文针对自回归 VLA 在高频灵巧控制中因逐维逐时刻离散化导致动作 token 强相关、学习退化的问题,提出先用 DCT 将动作片段压缩到频域再量化/BPE 的 FAST,并训练 1M 轨迹的通用 FAST+。实验显示其能在原方法失效的任务上训练 VLA,结合 π0 可扩展到 1 万小时数据,性能接近扩散式 VLA,训练时间最高降至约 1/5,但推理速度仍较慢。

Saliency-Aware Quantized Imitation Learning for Efficient Robotic Control Figure 1
ICCV 20252025-05-21

Saliency-Aware Quantized Imitation Learning for Efficient Robotic Control

Seongmin Park, Hyungmin Kim, Sangwoo Kim, Wonseok Jeon, Juyoung Yang, Byeongwook Jeon, Yoonseon Oh, Jungwook Choi

Hanyang University

视觉语言动作量化高效推理模型压缩操作

面向VLA等模仿学习策略在机器人端侧部署时的高算力、能耗与低速问题,论文指出量化失败往往集中在少数任务关键状态而非全程误差累积,并提出SQIL:用策略敏感性/显著性识别关键状态,在量化感知训练中加权蒸馏这些动作。实验覆盖LIBERO、真实UR5、自动驾驶和物理控制;4-bit OpenVLA基本恢复全精度成功率,在边缘GPU上约2.5倍加速并节能。

BitVLA: 1-bit Vision-Language-Action Models for Robotics Manipulation Figure 1
arXiv preprint 20252025-06-09

BitVLA: 1-bit Vision-Language-Action Models for Robotics Manipulation

Hongyu Wang, Chuyan Xiong, Ruiping Wang, Xilin Chen

Key Laboratory of AI Safety, Institute of Computing Technology, Chinese Academy of Sciences, University of Chinese Academy of Sciences

视觉语言动作量化模型压缩操作机器人学习

面向边缘机器人部署,BitVLA针对现有VLA模型体量大、延迟高的问题,将1-bit LLM BitNet扩展为全参数三值化的视觉-语言-动作策略,并用Quantize-then-Distill把视觉编码器压到1.58-bit以缓解量化造成的表征错配。在LIBERO和真实操作任务中,其性能接近全精度OpenVLA-OFT,同时模型内存降至约1/11、端到端延迟降低4.4倍。

SQAP-VLA: A Synergistic Quantization-Aware Pruning Framework for High-Performance Vision-Language-Action Models Figure 1
arXiv preprint 20252025-09-11

SQAP-VLA: A Synergistic Quantization-Aware Pruning Framework for High-Performance Vision-Language-Action Models

Hengyu Fang, Yijiang Liu, Yuan Du, Li Du, Huanrui Yang

School of Electronic Science and Engineering, Nanjing University, University of Arizona

视觉语言动作量化模型压缩剪枝令牌优化

面向VLA模型在机器人端部署时计算与显存开销过高的问题,SQAP-VLA指出低比特量化会扭曲注意力分布、使常规token剪枝失效,因此将两者协同设计:用量化不敏感的top-k注意力保留、机械臂先验保护与空间采样稳定剪枝,并以Hadamard变换改进张量级量化。实验显示该无训练框架在标准VLA上实现约1.93倍加速,平均成功率最高还提升4.5%。

VLA-Cache: Towards Efficient Vision-Language-Action Model via Adaptive Token Caching in Robotic Manipulation Figure 1
arXiv preprint 20252025-02-04

VLA-Cache: Towards Efficient Vision-Language-Action Model via Adaptive Token Caching in Robotic Manipulation

Siyu Xu, Yunke Wang, Chenghao Xia, Dihao Zhu, Tao Huang, Chang Xu

School of Computer Science, University of Sydney, Australia, John Hopcropt Center for Computer Science, Shanghai Jiao Tong University, China

视觉语言动作令牌优化高效推理模型压缩操作

VLA-Cache针对VLA模型在闭环机器人控制中反复处理相邻帧静态视觉区域、导致语言解码开销过高的问题,提出免训练的跨帧视觉token KV缓存:先按帧间变化找可复用token,再用解码器注意力过滤夹爪/目标等任务相关区域,并按层注意力熵自适应调整复用比例。在LIBERO、SIMPLER及真实Kinova Jaco2上,方法最高约1.7× CUDA延迟加速、控制频率提升15%,成功率损失很小。

Think Twice, Act Once: Token-Aware Compression and Action Reuse for Efficient Inference in Vision-Language-Action Models Figure 1
arXiv preprint 20252025-05-27

Think Twice, Act Once: Token-Aware Compression and Action Reuse for Efficient Inference in Vision-Language-Action Models

Xudong Tan, Yaoxin Yang, Peng Ye, Jialin Zheng, Bizhe Bai, Xinyi Wang, Jia Hao, Tao Chen

Fudan University Shanghai AI Laboratory, The Chinese University of Hong Kong Zhangjiang Laboratory

视觉语言动作令牌优化高效推理模型压缩

这篇论文针对 VLA 模型在机器人实时控制中因视觉 token 计算和自回归动作解码带来的高延迟问题,指出连续动作存在可复用的时间冗余,视觉 token 也有大量低贡献冗余。作者提出无需训练、可插拔的 FlashVLA,通过 token 感知的动作复用跳过稳定阶段解码,并用信息贡献选择保留关键视觉 token。在 LIBERO 上其 FLOPs 降低 55.7%、延迟降低 36.0%,任务成功率仅下降 0.7%。

Fast ECoT: Efficient Embodied Chain-of-Thought via Thoughts Reuse Figure 1
arXiv preprint 20252025-06-09

Fast ECoT: Efficient Embodied Chain-of-Thought via Thoughts Reuse

Zhekai Duan, Yuan Zhang, Shikai Geng, Gaowen Liu, Joschka Boedecker, Chris Xiaoxuan Lu

Department of Computer Science, University College London, UK, Department of Computer Science, University of Freiburg, Germany, Cisco Research, USA

视觉语言动作令牌优化高效推理模型压缩机器人学习

ECoT 让 VLA 机器人在行动前生成可解释推理,但逐 token 自回归导致控制延迟过高。Fast ECoT 的核心洞察是相邻时刻的高层“想法”具有结构重复和时间局部性,因此在推理时缓存复用推理片段,并并行生成模块化步骤,再用异步调度解耦推理与动作解码。LIBERO 与真实机器人实验显示其最高降低 7.5× 延迟,同时保持或提升成功率与推理一致性。

CronusVLA: Transferring Latent Motion Across Time for Multi-Frame Prediction in Manipulation Figure 1
arXiv preprint 20252025-06-24

CronusVLA: Transferring Latent Motion Across Time for Multi-Frame Prediction in Manipulation

Hao Li, Shuai Yang, Yilun Chen, Xinyi Chen, Xiaoda Yang, Yang Tian, Hanqing Wang, Tai Wang, Dahua Lin, Feng Zhao, Jiangmiao Pang

视觉语言动作令牌优化模型压缩操作机器人学习监督微调高效训练

CronusVLA针对现有VLA受单帧输入限制、直接堆叠多帧又带来高计算与延迟的问题,提出先单帧动作token预训练、再用可学习特征与feature chunking进行多帧后训练的迁移框架,并通过跨帧解码器、特征调制与正则化聚合历史运动线索。实验显示其在SimplerEnv达70.9%成功率、LIBERO较OpenVLA提升26.8%,且在新建SimplerEnv-OR扰动基准上鲁棒性最高。

VOTE: Vision-Language-Action Optimization with Trajectory Ensemble Voting Figure 1
arXiv preprint 20252025-07-07

VOTE: Vision-Language-Action Optimization with Trajectory Ensemble Voting

Juyi Lin, Amir Taherin, Arash Akbari, Arman Akbari, Lei Lu, Guangyu Chen, Taskin Padir, Xiaomeng Yang, Weiwei Chen, Yiqian Li, Xue Lin, David Kaeli, Pu Zhao, Yanzhi Wang

Northeastern University

视觉语言动作令牌优化模型压缩

VOTE针对VLA动作生成需输出大量令牌、推理慢且历史动作预测利用不足的问题,将整段动作压缩为单个<ACT>特殊令牌并用瓶颈动作头并行预测,减少解码与训练开销;推理时再用历史与当前轨迹的加权投票集成稳定动作选择。实验显示其在LIBERO较OpenVLA平均成功率提升20%以上,在SimplerEnv WidowX较CogACT高7%,并在Jetson Orin上实现39×吞吐加速、46Hz部署。

CogVLA: Cognition-Aligned Vision-Language-Action Model via Instruction-Driven Routing & Sparsification Figure 1
arXiv preprint 20252025-08-28

CogVLA: Cognition-Aligned Vision-Language-Action Model via Instruction-Driven Routing & Sparsification

Wei Li, Renshan Zhang, Rui Shao, Jie He, Liqiang Nie

School of Computer Science and Technology, Harbin Institute of Technology, Shenzhen

视觉语言动作令牌优化模型压缩

CogVLA针对VLA模型后训练与推理开销高、现有稀疏化忽视视觉-语言-动作语义耦合的问题,提出由指令驱动的三阶段路由:在视觉编码器聚合任务相关token,在LLM中剪除无关视觉token,并用V-L-A耦合注意力保持动作生成一致性。实验显示其在LIBERO和真实机器人任务成功率达97.4%和70.0%,相较OpenVLA训练成本降2.5倍、推理延迟降2.8倍。

SpecPrune-VLA: Accelerating Vision-Language-Action Models via Action-Aware Self-Speculative Pruning Figure 1
arXiv preprint 20252025-09-06

SpecPrune-VLA: Accelerating Vision-Language-Action Models via Action-Aware Self-Speculative Pruning

Hanzhen Wang, Jiaming Xu, Yushun Xiang, Jiayi Pan, Yongkang Zhou, Yong-Lu Li, Guohao Dai

视觉语言动作令牌优化高效推理模型压缩剪枝

针对VLA推理中LLM前向占主要延迟、现有视觉token剪枝只看当前步局部信息而易损成功率的问题,SpecPrune-VLA利用连续动作帧的时空一致性,将上一轮深层注意力等全局历史与当前局部重要性结合,并按动作粗细自适应调节剪枝强度。该训练-free两级剪枝在LIBERO最高1.57×、真实机器人最高1.70×加速,成功率仅有轻微下降。

The Better You Learn, The Smarter You Prune: Towards Efficient Vision-language-action Models via Differentiable Token Pruning Figure 1
arXiv preprint 20252025-09-16

The Better You Learn, The Smarter You Prune: Towards Efficient Vision-language-action Models via Differentiable Token Pruning

Titong Jiang, Xuefeng Jiang, Yuan Ma, Xin Wen, Bailin Li, Kun Zhan, Peng Jia, Yahui Liu, Sheng Sun, Xianpeng Lang

School of Vehicle and Mobility, Tsinghua University

视觉语言动作令牌优化高效推理模型压缩剪枝

面向VLA在边缘机器人上因大量视觉token导致注意力计算和时延过高的问题,LightVLA提出任务性能驱动的可微视觉token剪枝:由视觉与指令交互生成动态查询,并用Gumbel-Softmax选择关键token,无需额外参数或手工保留比例。在LIBERO上相较OpenVLA-OFT降低59.1% FLOPs、38.2%延迟,同时成功率提升2.6%。

Action-aware Dynamic Pruning for Efficient Vision-Language-Action Manipulation Figure 1
arXiv preprint 20252025-09-26

Action-aware Dynamic Pruning for Efficient Vision-Language-Action Manipulation

Xiaohuan Pei, Yuxing Chen, Siyu Xu, Yunke Wang, Yuheng Shi, Chang Xu

School of Computer Science, The University of Sydney

视觉语言动作令牌优化高效推理模型压缩操作

该文针对VLA操作中密集视觉token导致长时序推理开销高、且不同操作阶段冗余程度不同的问题,提出ADP:先用文本相关性筛选视觉token,再依据近期末端执行器轨迹的运动强度动态决定剪枝比例,粗动作多剪、精细抓取少剪或不剪。LIBERO与真实机器人实验显示其可降低FLOPs和延迟,在OpenVLA-OFT上约1.35×加速,并在OpenVLA等基线上保持或提升成功率。

Focusing on What Matters: Object-Agent-centric Tokenization for Vision Language Action models Figure 1
arXiv preprint2025-09-28

Focusing on What Matters: Object-Agent-centric Tokenization for Vision Language Action models

Rokas Bendikas, Daniel Dijkman, Markus Peschl, Sanjay Haresh, Pietro Mazzaglia

Centre for Artificial Intelligence, UCL, Qualcomm AI Research

视觉语言动作令牌优化模型压缩

这篇论文针对 VLA 训练中视觉 patch token 过多导致的计算瓶颈,提出 Oat-VLA:用无监督物体中心掩码把场景物体聚合成少量 token,并额外选取夹爪附近的 agent-centric token 保留精细交互信息。在 224×224 输入下视觉 token 从 OpenVLA 的 256 个降到 16 个,LIBERO 上收敛至少快 2 倍,真实抓取放置任务成功率也高于 OpenVLA。

Latent Action Pretraining from Videos Figure 1
ICLR 20252024-10-15

Latent Action Pretraining from Videos

Seonghyeon Ye, Joel Jang, Byeongguk Jeon, Sejune Joo, Jianwei Yang, Baolin Peng, Ajay Mandlekar, Reuben Tan, Yu-Wei Chao, Bill Yuchen Lin, Lars Liden, Kimin Lee, Jianfeng Gao, Luke Zettlemoyer, Dieter Fox, Minjoon Seo

KAIST, University of Washington, Microsoft Research, NVIDIA, Allen Institute for AI

视觉语言动作数据高效预训练高效推理高效训练模仿学习高效动作表示

针对VLA预训练依赖人工遥操作动作标签、难以利用海量公开视频的问题,本文提出Latent Action Pretraining:先用VQ-VAE从相邻视频帧中无监督离散化“潜在动作”,再让VLA按语言和观测预测这些动作,最后用少量机器人数据映射到真实控制。实验显示其在无动作标签视频学习、跨环境/跨本体以及真实语言条件操作上优于既有方法,并比OpenVLA高约6.22%,预训练效率提升30倍以上;仅用人类操作视频也有正迁移。

Diffusion Trajectory-guided Policy for Long-horizon Robot Manipulation Figure 1
IEEE RA-L 20252025-02-14

Diffusion Trajectory-guided Policy for Long-horizon Robot Manipulation

Shichao Fan, Quantao Yang, Yajie Liu, Kun Wu, Zhengping Che, Qingjie Liu, Min Wan

School of Mechanical Engineering and Automation, Beihang University, Beijing, China, School of Mechanical Engineering and Automation, BeiHang University, China, Beihang University, Division of Robotics, Perception and Learning (RPL), KTH Royal Institute of Technology, Stockholm, Sweden, Division of Robotics, Perception and Learning (RPL), KTH Royal Institute of Technology, Sweden, KTH Royal Institute of Technology, School of Computer Science and Engineering, Beihang University, Beijing, China, School of Computer Science and Engineering, BeiHang University, China, Beijing Innovation Center of Humanoid Robotics, Beijing, China, Beijing Innovation Center of Humanoid Robotics, China, Beijing Advanced Sciences and Innovation Center

视觉语言动作数据高效预训练高效推理高效训练操作

针对长程模仿学习中示教稀缺与误差累积导致的失败,DTP将语言和视觉输入先转化为扩散生成的任务相关2D轨迹,再作为额外条件指导VLA策略学习,相当于用可视化的中间运动意图缩小感知到动作的鸿沟。该模块可插入Transformer策略,并利用机器人视频预训练提升数据效率;在CALVIN上从零训练较SOTA平均成功率高25%,真实机器人实验也有明显提升。

Humanoid-VLA: Towards Universal Humanoid Control with Visual Integration Figure 1
arXiv preprint 20252025-02-20

Humanoid-VLA: Towards Universal Humanoid Control with Visual Integration

Pengxiang Ding, Jianfei Ma, Xinyang Tong, Binghong Zou, Xinxin Luo, Yiguo Fan, Ting Wang, Hongchao Lu, Panzhong Mo, Jinxin Liu, Yuefan Wang, Huaicheng Zhou, Wenshuo Feng, Jiacheng Liu, Siteng Huang, Donglin Wang

视觉语言动作数据高效预训练高效推理高效训练互联网/跨域数据数据采集

针对现有人形控制多依赖被动模仿、缺少第一视角感知且视动数据昂贵的问题,Humanoid-VLA先用第三人称运动-语言数据做预对齐,再以参数高效的视觉条件模块注入自我中心场景,并通过运动遮挡/重建等自监督任务生成伪问答扩充训练。实验显示其在物体交互和环境探索中具备更强上下文感知与自主执行能力,但具体增益中数据扩展与结构设计的贡献仍需进一步拆分。

GraspVLA: a Grasping Foundation Model Pre-trained on Billion-scale Synthetic Action Data Figure 1
arXiv preprint 20252025-05-06

GraspVLA: a Grasping Foundation Model Pre-trained on Billion-scale Synthetic Action Data

Shengliang Deng, Mi Yan, Songlin Wei, Haixin Ma, Yuxin Yang, Jiayi Chen, Zhiqi Zhang, Taoyu Yang, Xuheng Zhang, Wenhao Zhang, Heming Cui, Zhizheng Zhang, He Wang

视觉语言动作数据高效预训练高效推理高效训练操作仿真数据采集数据采集仿真到现实

针对真实机器人动作数据采集昂贵、限制 VLA 扩展的问题,GraspVLA 探索完全用大规模合成抓取动作预训练:构建含十亿帧、强随机化和写实渲染的 SynGrasp-1B,并用 PAG 将视觉定位、抓取位姿与流匹配动作生成串成统一 CoT,使合成几何与互联网语义联合训练。实验显示其可直接 sim-to-real,在真实与 LIBERO 抓取中具备较强零样本泛化,并能少样本适配偏好,但部分增益可能主要来自 scaling / data。

Learning to Act Anywhere with Task-centric Latent Actions Figure 1
Robotics: Science and Systems 20252025-05-09

Learning to Act Anywhere with Task-centric Latent Actions

Qingwen Bu, Yanting Yang, Jisong Cai, Shenyuan Gao, Guanghui Ren, Maoqing Yao, Ping Luo, Hongyang Li

视觉语言动作数据高效预训练高效推理高效训练模仿学习高效动作表示动作表示

针对现有 VLA 依赖带动作标注数据、难以跨本体和环境迁移的问题,UniVLA 从跨机器人与人类视频中学习任务中心的离散潜在动作,并借助语言条件与 DINO 特征削弱相机运动等无关动态,再用轻量解码器落到具体机器人。实验显示其在操作、导航和真机任务上优于 OpenVLA,且预训练算力少于 1/20、下游数据约为 1/10。

Unified Vision-Language-Action Model Figure 1
arXiv preprint 20252025-06-24

Unified Vision-Language-Action Model

Yuqi Wang, Xinghang Li, Wenxuan Wang, Junbo Zhang, Yingyan Li, Yuntao Chen, Xinlong Wang, Zhaoxiang Zhang

视觉语言动作数据高效预训练高效推理高效训练

现有 VLA 多依赖 VLM 的静态语义理解,难以利用视频中的时序与因果结构。UniVLA 将视觉、语言、动作统一离散化到共享 token 空间,并以自回归序列和世界模型后训练学习环境动态,从而提升长程任务策略学习的数据与训练效率;在 CALVIN、LIBERO、SimplerEnv-Bridge 上达到 SOTA,LIBERO 平均成功率 95.5%,并展示到 ALOHA 与自动驾驶场景的迁移。

EgoVLA: Learning Vision-Language-Action Models from Egocentric Human Videos Figure 1
arXiv preprint 20252025-07-16

EgoVLA: Learning Vision-Language-Action Models from Egocentric Human Videos

Ruihan Yang, Qinxi Yu, Yecheng Wu, Rui Yan, Borui Li, An-Chieh Cheng, Xueyan Zou, Yunhao Fang, Xuxin Cheng, Ri-Zhao Qiu, Hongxu Yin, Sifei Liu, Song Han, Yao Lu, Xiaolong Wang

UC San Diego, MIT, NVIDIA

视觉语言动作数据高效预训练高效推理高效训练模仿学习高效动作表示动作表示互联网/跨域数据数据采集

EgoVLA针对机器人模仿学习受真实机器人采集规模限制的问题,主张先从丰富的第一视角人类操作视频学习VLA,再用逆运动学与手部重定向把人体腕手动作映射到机器人动作,并用少量机器人示范微调。论文还构建Ego Humanoid Manipulation Benchmark;在12个双臂仿真任务上,EgoVLA相较专用和通用基线在短/长程任务及视觉、空间泛化上均有明显提升。

AnyPos: Automated Task-Agnostic Actions for Bimanual Manipulation Figure 1
arXiv preprint 20252025-07-17

AnyPos: Automated Task-Agnostic Actions for Bimanual Manipulation

Hengkai Tan, Yao Feng, Xinyi Mao, Shuhe Huang, Guodong Liu, Zhongkai Hao, Hang Su, Jun Zhu

Tsinghua-Bosch Joint ML Center, Tsinghua University, Institute for AI, BNRist Center, THBI Lab

视觉语言动作数据高效预训练高效推理高效训练操作自探索数据采集数据采集

针对机器人操作数据昂贵且常与任务和本体绑定、难以跨任务/平台迁移的问题,AnyPos 将“想做什么”与“身体能做什么”解耦,通过自动化无任务探索采集安全多样的图像-动作对,并用逆动力学、手臂/末端解耦和方向感知解码学习可复用本体先验。其动作预测测试准确率较标准基线提升 51%,在微波炉、叠衣、浇水等真实双臂任务上成功率提升约 30–40%。

Being-H0: Vision-Language-Action Pretraining from Large-Scale Human Videos Figure 1
arXiv preprint 20252025-07-21

Being-H0: Vision-Language-Action Pretraining from Large-Scale Human Videos

Hao Luo, Yicheng Feng, Wanpeng Zhang, Sipeng Zheng, Ye Wang, Haoqi Yuan, Jiazheng Liu, Chaoyi Xu, Qin Jin, Zongqing Lu

Peking University, Renmin University of China

视觉语言动作数据高效预训练高效推理高效训练模仿学习互联网/跨域数据数据采集

Being-H0针对灵巧操作中遥操作数据规模小、仿真到现实差距大的瓶颈,主张把互联网人手视频作为“基础操作者”来预训练VLA。其核心是物理指令微调:结合人类视频预训练、3D物理空间对齐、机器人后训练,并用分部运动token化保留毫米级手部轨迹。实验显示模型在手部运动生成、指令跟随和真实机器人灵巧操作上随数据与模型规模提升而改进,但部分收益可能主要来自大规模UniHand数据。

RynnVLA-001: Using Human Demonstrations to Improve Robot Manipulation Figure 1
arXiv preprint 20252025-09-18

RynnVLA-001: Using Human Demonstrations to Improve Robot Manipulation

Yuming Jiang, Siteng Huang, Shengke Xue, Yaxi Zhao, Jun Cen, Sicong Leng, Kehan Li, Jiayan Guo, Kexiang Wang, Mingxiu Chen, Fan Wang, Deli Zhao, Xin Li

1]DAMO Academy, Alibaba Group, 2]Hupan Lab

视觉语言动作数据高效预训练高效推理高效训练操作高效动作表示模仿学习互联网/跨域数据数据采集

RynnVLA-001针对机器人操作数据昂贵稀缺的问题,尝试把大规模人类第一视角操作视频中的动力学先验迁移到VLA控制。其核心是两阶段预训练:先用1200万自我中心视频做语言条件未来帧生成,再联合预测人体关键点轨迹以缩小视觉预测与动作控制的差距,并用ActionVAE压缩动作块以提升表示和推理效率。在相同下游数据微调后,模型成功率超过GR00T N1.5和Pi0,但验证主要限于单一机械臂与相近环境。

LAWM: Latent Action Pretraining Through World Modeling Figure 1
arXiv preprint 20252025-09-22

LAWM: Latent Action Pretraining Through World Modeling

Bahey Tharwat, Yara Nasser, Ali Abouzeid, Ian Reid

Mohamed bin Zayed University of Artificial Intelligence (MBZUAI), Abu Dhabi, UAE, Author is with Alexandria University, Alexandria, Egypt

视觉语言动作数据高效预训练高效推理高效训练模仿学习高效动作表示

LAWM针对VLA依赖昂贵遥操作动作标注、现有潜动作方法模型过大的问题,将模仿学习模型与世界模型结合,用无标注人类/机器人视频的下一帧预测来自监督学习动作块表征,微调时仅保留策略模型。实验显示其在LIBERO和真实场景中优于使用真实动作监督预训练及同类大模型方法,并以BAKU、DreamerV3等小模型实现更高训练与部署效率。

cVLA: Towards Efficient Camera-Space VLAs Figure 1
arXiv preprint 20252025-07-02

cVLA: Towards Efficient Camera-Space VLAs

Max Argus, Jelena Bratulic, Houman Masnavi, Maxim Velikanov, Nick Heppert, Abhinav Valada, Thomas Brox

University of Freiburg, Germany

视觉语言动作高效动作表示高效推理动作表示仿真数据采集数据采集仿真到现实

cVLA针对现有VLA训练昂贵、真实多模态数据难采和评测依赖真机的问题,将PaliGemma类VLM微调用于在相机/图像坐标中直接预测末端关键路点,而非低层控制,从而降低训练成本并弱化机器人本体依赖。论文还考察深度输入、裁剪、多候选解码及beam-search-NMS和演示条件生成;模型主要用仿真数据训练,在DROID、ManiSkill和真实机械臂上展示可执行轨迹与一定sim-to-real能力,但任务范围限于桌面准静态操作。

VLA-Adapter: An Effective Paradigm for Tiny-Scale Vision-Language-Action Model Figure 1
Proceedings of the AAAI Conference on Artificial Intelligence 20262025-09-11

VLA-Adapter: An Effective Paradigm for Tiny-Scale Vision-Language-Action Model

Yihao Wang, Pengxiang Ding, Lingxiao Li, Can Cui, Zirui Ge, Xinyang Tong, Wenxuan Song, Han Zhao, Wei Zhao, Pengxu Hou, Siteng Huang, Yifan Tang, Wenhui Wang, Ru Zhang, Jianyi Liu, Donglin Wang

Beijing University of Posts and Telecommunications, Westlake University, Zhejiang University, State Key Laboratory of Networking and Switching Technology, The Hong Kong University of Science and Technology (Guangzhou)

视觉语言动作高效动作表示高效推理动作表示

针对现有 VLA 依赖大规模 VLM、机器人数据预训练且推理慢的问题,VLA-Adapter 系统分析不同视觉语言条件到动作空间的桥接效果,并提出带 Bridge Attention 的轻量 Policy 自动注入有效条件。实验显示其仅用 0.5B 骨干、无需机器人预训练,在仿真与真实任务达到接近或超过强基线的成功率,并可在单张消费级 GPU 上约 8 小时训练,推理速度也更快。

ReSET: Prepare Before You Act: Learning From Humans to Rearrange Initial States Figure 1
arXiv preprint 20252025-09-22

ReSET: Prepare Before You Act: Learning From Humans to Rearrange Initial States

Yinlong Dai, Andre Keyser, Dylan P Losey

Collaborative Robotics Lab ( Collab ), Dept. of Mechanical Engineering, Virginia Tech, Blacksburg, VA, Collaborative Robotics Lab ( Collab ), Dept

视觉语言动作高效动作表示高效推理动作表示

针对模仿学习在物体遮挡、位置异常等分布外初始状态下易失败且靠扩充示范代价高的问题,ReSET提出“先整理环境再执行任务”的两阶段思路:从无动作人类视频学习何时/如何重排场景,并用任务无关机器人玩耍数据映射到动作原语,将状态拉回锚点分布。理论上给出降低泛化差距的分析,真实桌面任务中相较扩散策略、VLA等在相同总数据量下更稳健。

Atomic Skill Library: An Atomic Skill Library Construction Method for Data-Efficient Embodied Manipulation Figure 1
arXiv preprint 20252025-01-25

Atomic Skill Library: An Atomic Skill Library Construction Method for Data-Efficient Embodied Manipulation

Dongjiang Li, Bo Peng, Chang Li, Ning Qiao, Qi Zheng, Lei Sun, Yusen Qin, Bangguo Li, Yifeng Luan, Bo Wu, Yibing Zhan, Mingang Sun, Tong Xu, Lusong Li, Hui Shen, Xiaodong He

JD Explore Academy, China, Jingdong Technology Information Technology Co., Ltd, University of Science and Technology of China, Shenzhen University, Haier Group, Tsinghua University, D-robotics, RealMan Intelligent Technology (Jiangsu/Beijing) Co., Ltd

视觉语言动作监督微调高效推理高效训练操作

针对端到端具身操作在新任务/新环境中数据需求随任务复杂度膨胀、技能难以复用的问题,本文将学习对象从整任务转向可组合原子技能:用 VLP 做任务分解与规划,经语义抽象生成技能定义,再围绕技能采集少量数据并微调 VLA,形成可动态扩展的技能库。真实机器人实验显示,该方法在保持较高执行成功率的同时显著降低数据成本,但具体增益中规划、技能抽象与微调数据量各自贡献仍需进一步拆分说明。

MoManipVLA: Transferring Vision-language-action Models for General Mobile Manipulation Figure 1
CVPR 20252025-03-17

MoManipVLA: Transferring Vision-language-action Models for General Mobile Manipulation

Zhenyu Wu, Yuheng Zhou, Xiuwei Xu, Ziwei Wang, Haibin Yan

Beijing University of Posts and Telecommunications Nanyang Technological University, Tsinghua University

视觉语言动作监督微调高效训练操作

针对移动操作缺少大规模训练、固定基座 VLA 难以直接协同底盘与机械臂的问题,MoManipVLA 将预训练 VLA 生成的末端路点转化为移动操作轨迹,并通过包含可达性、平滑性和避障的双层优化同时规划底盘与手臂。其在 OVMM 和真实场景中较 SOTA 成功率提升 4.2%,真实部署仅需 50 条专家轨迹,但能力仍受底层 VLA 与受限运动空间约束。

OpenHelix: A Short Survey, Empirical Analysis, and Open-Source Dual-System VLA Model for Robotic Manipulation Figure 1
arXiv preprint 20252025-05-06

OpenHelix: A Short Survey, Empirical Analysis, and Open-Source Dual-System VLA Model for Robotic Manipulation

Can Cui, Pengxiang Ding, Wenxuan Song, Shuanghao Bai, Xinyang Tong, Zirui Ge, Runze Suo, Wanqi Zhou, Yang Liu, Bofang Jia, Han Zhao, Siteng Huang, Donglin Wang

Westlake University, Zhejiang University, Xi’an Jiaotong University, HKUST(GZ)

视觉语言动作监督微调高效训练操作

OpenHelix针对双系统VLA缺少开放实现、难以分析高效部署的问题,梳理现有架构并从MLLM选择、轻量System 1设计、慢系统到快系统的latent传递等核心环节做经验分析;其主要洞察是用低频大模型保留多模态推理、高频小策略负责实时控制。论文还发布低成本开源模型供复现与扩展,但具体量化增益文中未充分说明。

ControlVLA: Few-shot Object-centric Adaptation for Pre-trained Vision-Language-Action Models Figure 1
arXiv preprint 20252025-06-19

ControlVLA: Few-shot Object-centric Adaptation for Pre-trained Vision-Language-Action Models

Puhao Li, Yingying Wu, Ziheng Xi, Wanlin Li, Yuzhe Huang, Zhiyuan Zhang, Yinghan Chen, Jianan Wang, Song-Chun Zhu, Tengyu Liu, Siyuan Huang

Tsinghua University State Key Lab of General Artificial Intelligence, BIGAI, Peking University Astribot Inc

视觉语言动作监督微调高效训练

ControlVLA针对预训练VLA在少样本真实操作任务中微调仍需大量数据的问题,将物体中心表征作为条件注入策略模型。其核心是借鉴ControlNet,在交叉注意力中加入零初始化KV投影,使模型逐步利用物体属性而尽量保留原有动作先验。真实机器人8类任务中,仅用10–20条示范达到76.7%成功率,显著高于基线20.8%,并展示了长程任务及未见物体、背景下的鲁棒性。

InstructVLA: Vision-Language-Action Instruction Tuning: From Understanding to Manipulation Figure 1
arXiv preprint 20252025-07-23

InstructVLA: Vision-Language-Action Instruction Tuning: From Understanding to Manipulation

Shuai Yang, Hao Li, Bin Wang, Yilun Chen, Yang Tian, Tai Wang, Hanqing Wang, Feng Zhao, Yiyi Liao, Jiangmiao Pang

University of Science and Technology of China, Zhejiang University, Shanghai Artificial Intelligence Laboratory

视觉语言动作监督微调高效训练操作机器人学习数据增强数据采集

InstructVLA针对现有VLA在动作微调中易遗忘VLM多模态推理、且缺少具身监督的问题,将机器人动作生成重定义为指令跟随的一部分;通过650K样本VLA-IT数据、两阶段动作预训练与MoE适配联合训练文本推理和潜在动作。在SimplerEnv较SpatialVLA提升33%,在80任务SimplerEnv-Instruct上较微调OpenVLA提升96%,并优于GPT-4o辅助动作专家29%。

RICL: Adding In-Context Adaptability to Pre-Trained Vision-Language-Action Models Figure 1
arXiv preprint2025-08-04

RICL: Adding In-Context Adaptability to Pre-Trained Vision-Language-Action Models

Kaustubh Sridhar, Souradeep Dutta, Dinesh Jayaraman, Insup Lee

University of Pennsylvania, University of British Columbia

视觉语言动作监督微调高效训练

RICL针对预训练VLA虽具泛化能力却缺少像LLM那样免更新快速适应接口的问题,提出用少量机器人示范对现成π0-FAST进行“上下文学习再训练”,并在测试时通过检索相关状态-动作片段放入上下文实现RAG+ICL适应。实验显示,每个新操作任务仅需约20条示范、无需参数更新即可在未见物体、新动作和新场景上显著提升表现;若允许继续用同批示范微调,性能还能进一步提高。

Align-Then-stEer: Adapting the Vision-Language Action Models through Unified Latent Guidance Figure 1
arXiv preprint 20252025-09-02

Align-Then-stEer: Adapting the Vision-Language Action Models through Unified Latent Guidance

Yang Zhang, Chenwei Wang, Ouyang Lu, Yuan Zhao, Yunfei Ge, Zhenglong Sun, Xiu Li, Chi Zhang, Chenjia Bai, Xuelong Li

1]Institute of Artificial Intelligence, China Telecom, 2]Tsinghua University, 3]The Chinese University of Hong Kong, Shenzhen, 4]Northwestern Polytechnical University

视觉语言动作监督微调高效训练

针对VLA在新机器人形态或新任务上微调时动作空间/分布错配、数据与算力需求高的问题,ATE先用带反向KL约束的VAE把目标动作嵌入预训练动作潜空间的模态,再以潜空间指导扩散/流式动作生成,无需改VLA结构。在仿真多任务成功率较直接微调最高提升9.8%,真实跨形态场景提升32%。

ConRFT: A Reinforced Fine-tuning Method for VLA Models via Consistency Policy Figure 1
Robotics: Science and Systems 20252025-02-08

ConRFT: A Reinforced Fine-tuning Method for VLA Models via Consistency Policy

Yuhui Chen, Shuai Tian, Shugao Liu, Yingting Zhou, Haoran Li, Dongbin Zhao

SKL-MAIS, Institute of Automation, Chinese Academy of Sciences, Beijing, China, School of Artificial Intelligence, University of Chinese Academy of Sciences, Beijing, China

视觉语言动作强化学习方法高效训练强化学习

针对VLA模型在真实接触丰富任务中仅靠监督微调易受少量、次优且不一致示范限制的问题,ConRFT将离线到在线强化微调统一到一致性策略目标下:离线结合行为克隆与Q-learning初始化策略和值函数,在线引入人类干预以安全高效探索。八个真实操作任务中,45–90分钟在线微调后平均成功率达96.3%,较监督方法提升144%,episode长度缩短1.9倍。

RPD: Refined Policy Distillation: From VLA Generalists to RL Experts Figure 1
IROS 20252025-03-06

RPD: Refined Policy Distillation: From VLA Generalists to RL Experts

Tobias Jülg, Wolfram Burgard, Florian Walter

the Department of Computer Science & Artificial Intelligence, University of Technology Nuremberg, Germany

视觉语言动作强化学习方法高效训练强化学习

这篇论文针对VLA虽具泛化性但成功率不足、换设置需再微调的问题,提出RPD:在PPO式在线强化学习中加入来自教师VLA动作的行为克隆约束,用其任务先验引导学生探索并蒸馏成小型专家策略。ManiSkill3上基于微调Octo/OpenVLA的实验显示,RPD在稠密和稀疏奖励下通常比纯RL收敛更快,并可超过教师VLA,对相机视角变化和部分任务变体也更稳健,但真实机器人部署仍受在线RL样本量限制。

RIPT-VLA: Interactive Post-Training for Vision-Language-Action Models Figure 1
arXiv preprint 20252025-05-22

RIPT-VLA: Interactive Post-Training for Vision-Language-Action Models

Shuhan Tan, Kairan Dou, Yue Zhao, Philipp Krähenbühl

UT Austin 1 , Nankai University

视觉语言动作强化学习方法高效训练强化学习

针对VLA依赖离线示范和监督微调、在低数据与新场景中难以适应的问题,RIPT-VLA把交互式强化学习作为第三阶段后训练,仅用稀疏成功/失败奖励,通过动态rollout采样与留一优势估计稳定更新策略,无需critic或奖励塑形。实验中它提升轻量QueST约21.2%,将OpenVLA-OFT推至97.5%成功率,并在单示范设置下15轮内由约4%提升到97%。

VLA-RL: Towards Masterful and General Robotic Manipulation with Scalable Reinforcement Learning Figure 1
arXiv preprint 20252025-05-24

VLA-RL: Towards Masterful and General Robotic Manipulation with Scalable Reinforcement Learning

Guanxing Lu, Wenkai Guo, Chubin Zhang, Yuheng Zhou, Haonan Jiang, Zifeng Gao, Yansong Tang, Ziwei Wang

Tsinghua Shenzhen International Graduate School, Tsinghua University, School of Electrical and Electronic Engineering, Nanyang Technological University

视觉语言动作强化学习方法高效训练操作强化学习

VLA-RL 针对现有 VLA 依赖离线示范、在分布外状态易失败的问题,将机器人操作轨迹表述为多模态多轮对话,并用在线强化学习微调自回归 VLA;同时以视觉语言模型构建过程奖励来缓解稀疏奖励,配合课程选择、向量化环境、批量解码和 critic warmup 提升训练效率。在 LIBERO 40 个任务上,OpenVLA-7B 较最强微调基线提升 4.5%,接近 π0-FAST,并显示测试时优化越多性能越好,增益可能部分来自 scaling。

CO-RFT: Efficient Fine-Tuning of Vision-Language-Action Models through Chunked Offline Reinforcement Learning Figure 1
arXiv preprint 20252025-08-04

CO-RFT: Efficient Fine-Tuning of Vision-Language-Action Models through Chunked Offline Reinforcement Learning

Dongchi Huang, Zhirui Fang, Tianle Zhang, Yihang Li, Lin Zhao, Chunhe Xia

视觉语言动作强化学习方法高效推理高效训练强化学习

针对VLA模型用监督微调依赖示范质量、OOD泛化弱,而在线/测试时RL又成本高或推理慢的问题,CO-RFT将离线RL与动作分块结合:先全参数IL适配工作空间和本体,再用扩展TD的Chunked RL按动作块学习Q值并优化策略。真实机器人实验中,在仅30–60条示范下相较SFT成功率提升57%,周期时间降低22.3%,未见位置成功率达44.3%。

Balancing Signal and Variance: Adaptive Offline RL Post-Training for VLA Flow Models Figure 1
Proceedings of the AAAI Conference on Artificial Intelligence 20262025-09-04

Balancing Signal and Variance: Adaptive Offline RL Post-Training for VLA Flow Models

Hongyin Zhang, Shiyuan Zhang, Junxi Jin, Qixin Zeng, Yifan Qiao, Hongchao Lu, Donglin Wang

视觉语言动作强化学习方法高效训练强化学习

本文针对 VLA flow 模型在下游复杂操作中仅靠模仿学习难以利用数据质量差异、动作精度不足的问题,提出 ARFM 离线强化学习后训练。其关键是在 flow loss 中按批次自适应调节 RL advantage 权重,通过偏差—方差目标和二分更新,在保留高优势样本信号的同时抑制梯度方差。仿真与真实机器人实验显示,该方法提升了泛化、抗扰动、少样本与持续学习表现。

SimpleVLA-RL: Scaling VLA Training via Reinforcement Learning Figure 1
arXiv preprint 20252025-09-11

SimpleVLA-RL: Scaling VLA Training via Reinforcement Learning

Haozhan Li, Yuxin Zuo, Jiale Yu, Yuhao Zhang, Zhaohui Yang, Kaiyan Zhang, Xuekai Zhu, Yuchen Zhang, Tianxing Chen, Ganqu Cui, Dehui Wang, Dingxiang Luo, Yuchen Fan, Youbang Sun, Jia Zeng, Jiangmiao Pang, Shanghang Zhang, Yu Wang, Yao Mu, Bowen Zhou, Ning Ding

Shanghai Jiao Tong University, Peking University, The University of Hong Kong, Shanghai AI Lab

视觉语言动作强化学习方法高效训练强化学习自探索数据采集数据采集

针对VLA依赖昂贵人工轨迹进行SFT且分布外泛化不足的问题,SimpleVLA-RL将veRL改造成面向机器人交互的在线强化学习框架,加入VLA轨迹采样、多环境并行渲染、结果奖励、探索增强和优化损失计算。基于OpenVLA-OFT在LIBERO达SoTA,在RoboTwin 1.0/2.0超过π0;单任务仅1条示范时LIBERO-Long由17.1%升至91.7%,并展示仿真到真实迁移及RL中出现“pushcut”新策略现象。

World-Env: Leveraging World Model as a Virtual Environment for VLA Post-Training Figure 1
arXiv preprint 20252025-09-29

World-Env: Leveraging World Model as a Virtual Environment for VLA Post-Training

Junjin Xiao, Yandan Yang, Xinyuan Chang, Ronghan Chen, Feng Xiong, Mu Xu, Wei-Shi Zheng, Qing Zhang

School of Computer Science and Engineering, Sun Yat-sen University, China, AMap, Alibaba Group Key Laboratory of Machine Intelligence and Advanced Computing, Ministry of Education, China

视觉语言动作强化学习方法高效训练强化学习世界模型自探索数据采集数据采集

针对VLA依赖大量示范、真实RL交互难以重置且有安全风险的问题,World-Env把视频世界模型用作虚拟环境进行后训练:通过几何感知特征注入提升预测的物理一致性,并用VLM即时反思器给出奖励与终止判断,减少成功后的冗余动作。实验显示在每任务仅5条专家示范下仍能提升复杂操作成功率与训练效率。

GCENT: Genie Centurion — Accelerating Scalable Real-World Robot Training with Human Rewind-and-Refine Guidance Figure 1
arXiv preprint 20252025-05-24

GCENT: Genie Centurion — Accelerating Scalable Real-World Robot Training with Human Rewind-and-Refine Guidance

Wenhao Wang, Jianheng Song, Chiming Liu, Jiayao Ma, Siyuan Feng, Jingyuan Wang, Yuxin Jiang, Kylin Chen, Sikang Zhan, Yi Wang, Tong Meng, Modi Shi, Xindong He, Guanghui Ren, Yang Yang, Maoqing Yao

视觉语言动作人在回路数据采集高效推理高效训练数据采集

针对真实机器人 VLA 策略依赖全程遥操作示教、成本高且难扩展的问题,GCENT 将部署与采集闭环结合:机器人先用不完美策略执行,失败或将失败时由 Task Sentinel 触发人工介入,并通过 rewind-and-refine 回到关键状态补充纠错数据。实验显示其在长程和精细操作中平均同帧数性能提升约 40%,达到相近效果仅需约 44.5% 数据,并支持单人监督双机器人仍保持约 1.9 的采集效率。

RoboTwin 2.0: A Scalable Data Generator and Benchmark with Strong Domain Randomization for Robust Bimanual Robotic Manipulation Figure 1
arXiv preprint 20252025-06-22

RoboTwin 2.0: A Scalable Data Generator and Benchmark with Strong Domain Randomization for Robust Bimanual Robotic Manipulation

Tianxing Chen, Zanxin Chen, Baijun Chen, Zijian Cai, Yibin Liu, Zixuan Li, Qiwei Liang, Xianliang Lin, Yiheng Ge, Zhenyu Gu, Weiliang Deng, Yubin Guo, Tian Nian, Xuanbing Xie, Qiangyu Chen, Kailun Su, Tianling Xu, Guodong Liu, Mengkang Hu, Huan-ang Gao, Kaixuan Wang, Zhixuan Liang, Yusen Qin, Xiaokang Yang, Ping Luo, Yao Mu

MoE key Lab of Artificial Intelligence, AI Institute, SJTU, Shanghai AI Lab, D-Robotics, SZU, THU, TeleAI

视觉语言动作仿真数据采集数据采集操作仿真到现实

面向双臂操作中真实示教昂贵、现有仿真数据缺少复杂性与跨本体泛化的问题,RoboTwin 2.0 用含731物体的资产库、MLLM+仿真闭环修正的专家代码生成,以及杂物、光照、背景、桌高和语言五轴域随机化来规模化采集数据。其覆盖50个任务和5种双臂平台,代码生成成功率提升10.9%;合成数据加10条真实示教使VLA较10-demo基线相对提升367%,纯合成零样本也提升228%,但增益可能主要来自数据规模与随机化共同作用。

ReBot: Scaling Robot Learning with Real-to-Sim-to-Real Robotic Video Synthesis Figure 1
IROS 20252025-03-15

ReBot: Scaling Robot Learning with Real-to-Sim-to-Real Robotic Video Synthesis

Yu Fang, Yue Yang, Xinghao Zhu, Kaiyuan Zheng, Gedas Bertasius, Daniel Szafir, Mingyu Ding

Xinghao Zhu is with Robotics and AI Institute, Broadway, Cambridge, MA 02142, USA

视觉语言动作仿真数据采集数据采集仿真到现实机器人学习

ReBot针对真实机器人数据昂贵、纯仿真又存在 sim-to-real gap 的瓶颈,提出 real-to-sim-to-real 数据扩展流程:在仿真中重放真实轨迹以替换/多样化操作物体,再与修复后的真实背景合成时序一致的视频。该方法可自动为目标域扩充 VLA 训练数据,在 SimplerEnv 和真实 Franka 上分别提升 Octo/OpenVLA 成功率,最高带来 21.8% 仿真域内增益和 20% 真实任务增益。

Real2Render2Real: Scaling Robot Data Without Dynamics Simulation or Robot Hardware Figure 1
arXiv preprint 20252025-05-14

Real2Render2Real: Scaling Robot Data Without Dynamics Simulation or Robot Hardware

Justin Yu, Letian Fu, Huang Huang, Karim El-Refai, Rares Andrei Ambrus, Richard Cheng, Muhammad Zubair Irshad, Ken Goldberg

University of California, Berkeley, Toyota Research Institute

视觉语言动作仿真数据采集数据采集仿真到现实

针对机器人学习数据依赖昂贵遥操作和真实硬件的瓶颈,R2R2R用手机物体扫描与单段人类演示视频,重建3DGS资产并跟踪6DoF运动,在关闭动力学/碰撞建模的渲染环境中合成大量视觉-动作数据。物理实验显示,单次演示生成的数据训练出的策略可接近150条遥操作示范的效果,增益可能主要来自数据规模与视觉多样性。

RealMirror: A Comprehensive, Open-Source Vision-Language-Action Platform for Embodied AI Figure 1
arXiv preprint 20252025-09-18

RealMirror: A Comprehensive, Open-Source Vision-Language-Action Platform for Embodied AI

Cong Tai, Zhaoyu Zheng, Haixu Long, Hansheng Wu, Haodong Xiang, Zhengbin Long, Jun Xiong, Rong Shi, Shizhuang Zhang, Gang Qiu, He Wang, Ruifeng Li, Jun Huang, Bin Chang, Shuai Feng, Tao Shen

Jun Xiong is with The Chinese University of Hong Kong, Shenzhen, China

视觉语言动作仿真数据采集数据采集仿真到现实

RealMirror针对人形机器人VLA研究中真实交互数据昂贵、缺少统一评测以及仿真到现实落差大的问题,构建开源的一体化仿真数据采集、训练、推理与评测平台,并通过生成式资产和3DGS提升环境与机器人模型真实感。其基准包含5类场景、1000余条轨迹和多种VLA模型评测;文中还展示仅用仿真数据训练即可零样本迁移到真实机器人执行任务。

EgoScaler: Developing Vision-Language-Action Model from Egocentric Videos Figure 1
arXiv preprint 20252025-09-26

EgoScaler: Developing Vision-Language-Action Model from Egocentric Videos

Tomoya Yoshida, Shuhei Kurita, Taichi Nishimura, Shinsuke Mori

Kyoto University, Kyoto 606-8501, Japan, National Institute of Informatics, Tokyo 101-8430, Japan, Institute of Science Tokyo, Sony Interactive Entertainment, Tokyo 108-0075, JapanContact

视觉语言动作互联网/跨域数据数据采集

针对 VLA 预训练依赖昂贵机器人遥操作数据、而以往自我中心视频方法又常需手姿态等辅助标注的问题,EgoScaler 从原始第一视角视频中抽取并清洗 6DoF 物体操作轨迹,用于构建大规模预训练数据。在 π0 上的仿真与真机实验显示,该数据较从零训练成功率提升超 20%,效果接近真实机器人数据,并与 BridgeData V2 合并后进一步增益,说明收益可能主要来自可扩展数据与显式轨迹监督。

MimicDreamer: Aligning Human and Robot Demonstrations for Scalable VLA Training Figure 1
arXiv preprint 20252025-09-26

MimicDreamer: Aligning Human and Robot Demonstrations for Scalable VLA Training

Haoyun Li, Ivan Zhang, Runqi Ouyang, Xiaofeng Wang, Zheng Zhu, Zhiqin Yang, Zhentao Zhang, Boyuan Wang, Chaojun Ni, Wenkang Qin, Xinze Chen, Yun Ye, Guan Huang, Zhenbo Song, Xingang Wang

Tsinghua University

视觉语言动作互联网/跨域数据高效训练数据采集模仿学习

MimicDreamer针对VLA训练中真实机器人数据昂贵、规模受限的问题,将低成本人类第一视角示范转成机器人可用监督。其核心是同时做三类对齐:用视频扩散模型生成机器人臂视觉外观,用单应变换与修复稳定视角,并将手部轨迹经约束IK映射为低抖动关节指令。实验显示,仅用合成示范也能少样本上真机,叠加更多人类数据后六项任务平均成功率较纯机器人数据基线提升14.7%,增益可能主要来自scaling/data与对齐质量共同作用。

EMMA: Generalizing Real-World Robot Manipulation via Generative Visual Transfer Figure 1
arXiv preprint 20252025-09-26

EMMA: Generalizing Real-World Robot Manipulation via Generative Visual Transfer

Zhehao Dong, Xiaofeng Wang, Zheng Zhu, Yirui Wang, Yang Wang, Yukun Zhou, Boyuan Wang, Chaojun Ni, Runqi Ouyang, Wenkang Qin, Xinze Chen, Yun Ye, Guan Huang, Zhen Lu, Yue Yang

Peking University, Tsinghua University

视觉语言动作互联网/跨域数据数据采集操作数据增强

EMMA针对真实机器人操作数据采集昂贵、视觉多样性不足导致VLA泛化差的问题,用DreamTransfer以文本引导生成多视角一致、几何可信的操作视频,并通过AdaMix按策略表现重加权困难样本。实验含仿真与真实1800余次试验,生成质量在多视角和深度一致性上分别相对提升42%和24%;零样本视觉外观下,相比仅用真实数据成功率相对提升超92%,AdaMix再带来17%增益。

Beyond Human Demonstrations: Diffusion-Based Reinforcement Learning to Generate Data for VLA Training Figure 1
arXiv preprint 20252025-09-24

Beyond Human Demonstrations: Diffusion-Based Reinforcement Learning to Generate Data for VLA Training

Rushuai Yang, Hangxing Wei, Ran Zhang, Zhiyuan Feng, Xiaoyu Chen, Tong Li, Chuheng Zhang, Li Zhao, Jiang Bian, Xiu Su, Yi Chen

Hong Kong University of Science and Technology, Hong Kong, China, Microsoft Research Asia, Beijing, China, Wuhan University, Wuhan, China, University of Chinese Academy of Sciences, Beijing, China, Tsinghua University, Beijing, China, Big Data Institute, Central South University, Changsha, China

视觉语言动作自探索数据采集高效训练数据采集强化学习

针对 VLA 训练高度依赖昂贵人工示范、难以规模化的问题,论文将扩散策略引入强化学习数据生成,用去噪过程的隐式正则化获得更平滑、低方差的长程操作轨迹,并以 BC 预热加在线 RL 微调构建数据流水线。在 LIBERO 130 个任务上,仅用该数据训练的 VLA 成功率达 81.9%,比人工数据高 5.3 个百分点、比高斯 RL 数据高 12.6 个百分点。

VLA-RFT: Vision-Language-Action Reinforcement Fine-Tuning with Verified Rewards in World Simulators Figure 1
arXiv preprint 20252025-10-01

VLA-RFT: Vision-Language-Action Reinforcement Fine-Tuning with Verified Rewards in World Simulators

Hengtao Li, Pengxiang Ding, Runze Suo, Yihao Wang, Zirui Ge, Dongyuan Zang, Kexian Yu, Mingyang Sun, Hongyin Zhang, Donglin Wang, Weihua Su

Westlake University, Zhejiang University, Fudan University, Zhengzhou University, Hebei University of Technology

视觉语言动作自探索数据采集高效训练数据采集强化学习

针对 VLA 主要依赖模仿学习、在分布偏移下易误差累积,而真实/传统仿真强化学习成本高或有 sim-to-real 问题,VLA-RFT 用真实交互数据训练可控世界模型,在其中按动作 rollout 预测视觉轨迹,并与成功参考轨迹比较生成密集验证奖励,用 GRPO 做强化微调。实验显示少于 400 步微调即可超过强监督微调基线,并在组合泛化和扰动场景中更稳健。

LLaRA: Supercharging Robot Learning Data for Vision-Language Policy Figure 1
ICLR 20252024-06-28

LLaRA: Supercharging Robot Learning Data for Vision-Language Policy

Xiang Li, Cristina Mata, Jongwoo Park, Kumara Kahatapitiya, Yoo Sung Jang, Jinghuan Shang, Kanchana Ranasinghe, Ryan Burgert, Mu Cai, Yong Jae Lee, Michael S. Ryoo

Stony Brook University of Wisconsin-Madison

视觉语言动作数据增强数据采集机器人学习

LLaRA针对少量机器人示范下难以直接把预训练VLM迁移为低层控制策略的问题,将行为克隆数据自动改写为视觉-文本对话式指令数据,并把动作对齐到图像像素坐标;同时用六个无需额外动作标注的自监督辅助任务增强数据。实验显示,在多种仿真和真实操作任务上,有限数据微调即可获得有意义动作决策并达到SOTA,同时保留一定语言模型泛化能力。

RoboChemist: Long-Horizon and Safety-Compliant Robotic Chemical Experimentation Figure 1
arXiv preprint2025-09-10

RoboChemist: Long-Horizon and Safety-Compliant Robotic Chemical Experimentation

Zongzheng Zhang, Chenghao Yue, Haobo Xu, Minwen Liao, Xianglin Qi, Huan-ang Gao, Ziwei Wang, Hao Zhao

Institute for AI Industry Research (AIR), Tsinghua Univeristy, Nanyang Technological University

视觉语言动作数据增强数据采集机器人学习

面向化学实验中长时序、透明器皿/可变形物质感知困难以及安全规范约束强的问题,RoboChemist 将 VLM 与 VLA 组成双闭环:VLM 负责任务分解、生成带框/关键点的视觉提示并监控完成度与合规性,VLA 按图像目标执行底层动作。在倒液、搅拌及酸碱中和、焰色反应等任务上,相比强 VLA 基线平均成功率提升 23.57%,合规率提升 0.298,并展示了对新试剂、容器和流程的泛化。

ERMV: Editing 4D Robotic Multi-view Images to Enhance Embodied Agents Figure 1
arXiv preprint 20252025-07-23

ERMV: Editing 4D Robotic Multi-view Images to Enhance Embodied Agents

Chang Nie, Guangming Wang, Zhe Lie, Hesheng Wang

University, Cambridge CB2 TN, UK

视觉语言动作数据增强数据采集机器人学习

针对机器人模仿学习中4D多视角序列采集昂贵、现有增强多停留在单帧/单视角的问题,ERMV用单帧编辑和机器人状态驱动整段序列编辑,引入运动感知极线注意力、稀疏时空模块及MLLM反馈干预来兼顾一致性、成本和关键物体语义。实验显示其增强数据可提升仿真与真实环境中VLA策略的鲁棒性和泛化,并可将仿真图像风格真实化以缓解sim-to-real差距。