About Me

I am a Research Engineer at Alibaba DAMO Academy. I lead a small (but great) team to build multimodal foundation models that perceive, reason and act in physical world.

Research Interests

• Multimodal LLMs for Robot Perception & Manipulation
• Generalist Vision-Language-Action Models
• Action Policy Beyond 2D Vision

Education

• 2016-2021, Ph.D., The Chinese University of Hong Kong, Department of Systems Engineering and Engineering Management. Advisor: Prof. Wai Lam
• 2012-2016, B.Eng., Sun Yat-Sen University, School of Computer Science and Engineering

Experiences

• Mar 2021-, Research Engineer, Alibaba DAMO Academy.
• Jul 2020-Feb 2021, Research Intern, Huawei Noah’s Ark Lab. Mentor: Dr. Yi Liao.
• Jan 2018-Apr 2018, Research Intern, Tencent AI Lab. Mentor: Dr. Lidong Bing and Dr. Piji Li.
• Jul 2015-Jun 2016, Research intern, Microsoft Research Asia. Mentor: Dr. Chin-Yew Lin and Dr. Jing Liu
• Sep 2014-July 2015, Research assistant, SentiNet Group, Sun Yat-Sen University. Supervisor: Prof. Rao Yanghui

Recent Works [Full list]

• RynnValue: Scaling Robotic Value Foundation Models with Temporal Distance
  Dongchi Huang^* , Hongyin Zhang^* , Bohan Hou^* , Siteng Huang^† , Zhian Su, Hang Guo, Tong Lu, Zhaofeng Xu, Jiahao Tang, Jianfei Yang, Donglin Wang, Peixi Peng^† , Mingxiu Chen, Deli Zhao, Xin Li.
  [code][checkpoints & demos][modelscope]
• RynnBrain 1.1: Towards More Capable and Generalizable Embodied Foundation Model
  Kehan Li^*† , Bohan Hou^* , Minghao Zhu^* , Tianyi Zhang^* , Zesen Cheng^* , Zhikai Wang^* , Sicong Leng^* , Xin Li^*† , Xiao Lin^* , Biying Yao^* , Minghua Zeng^* , Jiangpin Liu^* , Ronghao Dang^* , Jiayan Guo^* , Siteng Huang, Haoyu Zhao, Heng Ping, Yaxi Zhao, Tong Zhao, Kexiang Wang, Tong Lu, Shengke Xue, Jiahao Tang, Yulei Wang, Zejing Wang, Jianwei Gao, Shijian Lu, Chengju Liu, Jianfei Yang, Mingxiu Chen, Deli Zhao.
  [code][checkpoints & demos][modelscope]
• RynnWorld-Teleop: An Action-Conditioned World Model for Digital Teleoperation
  Haoyu Zhao^* , Xingyue Zhao^* , Hangyu Li, Biao Gong, Kehan Li, Siteng Huang^† , Xin Li, Deli Zhao^† , Zhongyu Li^† .
  [code][checkpoint][modelscope]
• RynnWorld-4D: 4D Embodied World Models for Robotic Manipulation
  Haoyu Zhao^* , Xingyue Zhao^* , Siteng Huang^† , Xin Li, Deli Zhao^† , Zhongyu Li^† .
  [code][checkpoint][modelscope]
• RynnBrain: Open Embodied Foundation Models
  Ronghao Dang^* , Jiayan Guo^* , Bohan Hou^* , Sicong Leng^* , Kehan Li^* , Xin Li^* , Jiangping Liu^* , Yunxuan Mao^* , Zhikai Wang^* , Yuqian Yuan^* , Minghao Zhu^* , Xiao Lin, Yang Bai, Qian Jiang, Yaxi Zhao, Minghua Zeng, Junlong Gao, Yuming Jiang, Jun Cen, Siteng Huang, Liuyi Wang, Wenqiao Zhang, Chengju Liu, Jianfei Yang, Shijian Lu, Deli Zhao.
  [code (RynnScale)][code (eval & infer)][checkpoints & demos]
• RynnVLA-002: A Unified Vision-Language-Action and World Model
  Jun Cen^* , Siteng Huang^* , Yuqian Yuan^* , Kehan Li^* , Hangjie Yuan, Chaohui Yu, Yuming Jiang, Jiayan Guo, Xin Li, Hao Luo, Fan Wang, Deli Zhao, Hao Chen.
  [code][checkpoint]
• RynnVLA-001: Using Human Demonstrations to Improve Robot Manipulation
  Yuming Jiang, Siteng Huang, Shengke Xue, Yaxi Zhao, Jun Cen, Sicong Leng, Kehan Li, Jiayan Guo, Kexiang Wang, Mingxiu Chen, Fan Wang, Deli Zhao, Xin Li.
  [code][blog][checkpoint]
  To appear in ICRA 2026 (Full paper)
• VideoLLaMA 3: Frontier Multimodal Foundation Models for Image and Video Understanding
  Boqiang Zhang^* , Kehan Li^* , Zesen Cheng^* , Zhiqiang Hu^* , Yuqian Yuan^* , Guanzheng Chen^* , Sicong Leng^* , Yuming Jiang^* , Hang Zhang^* , Xin Li^* , Peng Jin, Wenqi Zhang, Fan Wang, Lidong Bing, Deli Zhao.
  arXiv:2501.13106
  [code][checkpoints & demos]
• VideoRefer Suite: Advancing Spatial-Temporal Object Understanding with Video LLM
  Yuqian Yuan, Hang Zhang, Wentong Li, Zesen Cheng, Boqiang Zhang, Long Li, Xin Li, Deli Zhao, Wenqiao Zhang, Yueting Zhuang, Jianke Zhu, Lidong Bing.
  In CVPR 2025 (Full paper, poster)
  [code (new)][videorefer-700k][videorefer-bench][videorefer-videollama3 (new)]

Featured Projects

• Foundation Models that Perceive, Reason and Act in Physical World (2025.05-Present)
  • Embodied foundation models that unify egocentric understanding, spatio-temporal grounding, chain-of-point reasoning, and task planning (RynnBrain)
  • Scaling embodied foundation models to 100B (RynnBrain 1.1)
  • Training robotic value foundation models over 7K+ hours data without any preference or progress annotations (RynnValue)
• Embodied World Models for Robot Manipulation (2025.05-Present)
  • Embodied world models that jointly predict RGB, depth, and optical flow (RynnWorld-4D)
  • Decoupling data collection from physical constraints by replacing the real robot with a generative world model (RynnWorld-Teleop)
• Enhanced Robot Manipulation Policies (2024.11-Present)
  • Boosting manipulation performance via massive video generative pretraining (RynnVLA-001)
  • Jointly modeling actions and the visual changes of environment (RynnVLA-002)
  • Modeling Highly Heterogeneous Embodiments via Unified Action Space (RynnBrain-VLA)
• Multimodal Foundation Models for Video Understanding (2023.04-2025.05)
  • The first instruction-tuned LLM for joint audio-visual understanding (VideoLLaMA)
  • Advancing audio-visual understanding capabilities of Video-LLMs via massive multi-task fine-tuning (VideoLLaMA 2)
  • Fine-grained video understanding with arbitrary visual prompt (VideoRefer, PixelRefer)
  • Building highly capable multimodal LLMs for both image and video (VideoLLaMA 3)
• Recipes for Improving General VLMs/LLMs (2023.06-2025.05)
  • Long-context modeling in LLMs (CLEX, LongPO)
  • Multimodal textbook for vision-language pretraining (MM-Textbook)
  • Mitigating object hallucinations in VLMs via visual contrastive decoding (VCD)
  • Multilingual LLMs tailored for Southeast Asian languages (SeaLLMs)

Honors & Awards

• AAAI Student Scholarship, 2020
• Outstanding Graduates Awards, Sun Yat-Sen University.
• Excellent Undergraduate Thesis award, Sun Yat-Sen University.

Professional Activities

• Reviewer (or PC Member):
  • ACL 2020-2023, EMNLP 2018-2023, NAACL 2021
  • AAAI 2019-2020
  • WSDM 2023, CIKM 2021
  • ACM Transactions on Knowledge Discovery from Data (TKDD)
  • ACM Transactions on Asian and Low-Resource Language Information Processing (TALLIP)
  • IEEE Transaction on Multimedia (TMM)
  • Neurocomputing
  • Computational Intelligence

Some Useful Notes & Links

• Introduction to Neural Word Embeddings. [pdf]
• Tutorial for Statistical Machine Translation.
• Tutorial for Neural Machine Translation.
• How to write a great research paper.

Hobbies

• Playing basketball
• Swimming
• Hiking