Me
Jun CEN
DAMO Academy, Alibaba
jcenaa [at] connect.ust.hk

About

I am an algorithm expert in DAMO Academy, Alibaba, where I focus on multi-modality model and embodied AI. I got my Ph.D. at the Robotics Institute, The Hong Kong University of Science and Technology, where I work on computer vision and autonomous driving under the supervision of Prof. Qifeng Chen and Prof. Michael Yu Wang. I'm always looking for related colaboration. If you are interested to chat with me, feel free to drop me an email.

Interests

Education

The Hong Kong University of Science and Technology
The Hong Kong University of Science and Technology
  • M.Sc in Mechanical Engineering, Sep. 2019 - Oct. 2020
  • Mentor: Prof. Lilong Cai
Zhejiang University
  • B.Eng in Mechatronics Engineering, Sep. 2015 - Oct. 2019
  • Mentor: Prof. Jinghua Xu

Experiences

Aug, 2024 - Now
Algorithm Expert @ DAMO Academy, Alibaba
Join as  AliStar, focus on Embodied AI and VLA models.
Aug, 2023 - May, 2024
Research Intern @ MSRA
Supervisor: Dr. Chenfei Wu, Dr. Nan Duan
Aug, 2023 - July, 2024
Visiting Student @ SUSTech
Supervisor: Prof. Jianguo Zhang
Mar, 2023 - Sep, 2023
Visiting Student @ MMLab, NTU
Supervisor: Prof. Ziwei Liu
Mar, 2022 - Mar, 2023
Research Intern @ Tongyi Lab, Alibaba
Supervisor: Dr. Shiwei Zhang
Mar, 2020 - July, 2020
Algorithm Intern @ HIKVISION
Supervisor: Dr. Yushi Zhu
Nov, 2019 - Feb, 2020
Research Assistant @ CUHK(SZ)
Supervisor: Dr. Liguang Zhou, Prof. Tin Lun Lam

Publications ( / )

WorldVLA: Towards Autoregressive Action World Model
Jun Cen, Chaohui Yu, Hangjie Yuan, Yuming Jiang, Siteng Huang, Jiayan Guo, Xin Li, Yibing Song, Hao Luo, Fan Wang, Deli Zhao, Hao Chen
Arxiv 2025
[Paper] [Code (Star 400+)]
RynnVLA-001: Using Human Demonstrations to Improve Robot Manipulation
Yuming Jiang, Siteng Huang, Shengke Xue, Yaxi Zhao, Jun Cen, Sicong Leng, Kehan Li, Jiayan Guo, Kexiang Wang, Mingxiu Chen, Fan Wang, Deli Zhao, Xin Li
Arxiv 2025
[Paper] [Code]
Self-evolved Imitation Learning in Simulated World
Yifan Ye, Jun Cen, Jing Chen, Zhihe Lu
Arxiv 2025
[Paper] [Code]
Discovering Hierarchical Manipulation Concepts from Unlabeled Multi-Modal Data
Ruizhe Liu, Pei Zhou, Qian Luo, Li Sun, Jun Cen, Yibing Song, Yanchao Yang
NeurIPS 2025
[Paper] [Code]
Generative Artificial Intelligence in Robotic Manipulation: A Survey
Kun Zhang*, Peng Yun*, Jun Cen, Junhao Cai, Didi Zhu, Hangjie Yuan, Chao Zhao, Tao Feng, Michael Yu Wang, Qifeng Chen, Jia Pan, Bo Yang†, Hua Chen†
Arxiv 2025
[Paper] [Code]
Lumos-1: On Autoregressive Video Generation from a Unified Model Perspective
Hangjie Yuan, Weihua Chen, Jun Cen, Hu Yu, Jingyun Liang, Shuning Chang, Zhihui Lin, Tao Feng, Pengwei Liu, Jiazheng Xing, Hao Luo, Jiasheng Tang, Fan Wang, Yi Yang
Arxiv 2025
[Paper] [Code]
Calib3D: Calibrating Model Preferences for Reliable 3D Scene Understanding
Lingdong Kong*, Xiang Xu*, Jun Cen, Wenwei Zhang, Liang Pan, Kai Chen, Ziwei Liu
WACV 2025 oral
[Paper] [Code]
Continual Learning for Segment Anything Model Adaptation
Jinglong Yang, Yichen Wu, Jun Cen, Wenjian Huang, Hong Wang, Jianguo Zhang
Arxiv 2024
[Paper] [Code]
SimCMF: A Simple Cross-modal Fine-tuning Strategy from Vision Foundation Models to Any Imaging Modality
Chenyang Lei*, Liyi Chen*, Jun Cen, Xiao Chen, Zhen Lei, Felix Heide, Qifeng Chen, Zhaoxiang Zhang
Arxiv 2024
[Paper] [Code]
iControl3D: An Interactive System for Controllable 3D Scene Generation
Xingyi Li, Yizheng Wu, Jun Cen, Juewen Peng, Kewei Wang, Ke Xian, Zhe Wang, Zhiguo Cao, Guosheng Lin
ACMMM 2024
[Paper] [Code]
Using Left and Right Brains Together: Towards Vision and Language Planning
Jun Cen*, Chenfei Wu*, Xiao Liu, Shengming Yin, Yixuan Pei, Jinglong Yang, Qifeng Chen, Nan Duan, Jianguo Zhang
ICML 2024
[Paper] [Code]
Self-Supervised Class-Agnostic Motion Prediction with Spatial and Temporal Consistency Regularizations
Kewei Wang, Yizheng Wu, Jun Cen, Zhiyu Pan, Xingyi Li, Zhe Wang, Zhiguo Cao, Guosheng Lin
CVPR 2024
[Paper] [Code]
Consistent Prompting for Rehearsal-Free Continual Learning
Zhanxin Gao, Jun Cen, Xiaobin Chang
CVPR 2024
[Paper] [Code]
CMDFusion: Bidirectional Fusion Network with Cross-modality Knowledge Distillation for LIDAR Semantic Segmentation
Jun Cen, Shiwei Zhang, Yixuan Pei, Kun Li, Hang Zheng, Maochun Luo, Yingya Zhang, Qifeng Chen
RA-L 2023
[Paper] [Code]
SAD: Segment Any RGBD
Jun Cen, Yizheng Wu, Kewei Wang, Xingyi Li, Jingkang Yang, Yixuan Pei, Lingdong Kong, Ziwei Liu, Qifeng Chen
NeurIPS 2023 workshop
[Paper] [Code (Star 800+)] [Twitter]
Segment Any Point Cloud Sequences by Distilling Vision Foundation Models
Youquan Liu*, Lingdong Kong*, Jun Cen, Runnan Chen, Wenwei Zhang, Liang Pan, Kai Chen, Ziwei Liu
NeurIPS 2023 Spotlight
[Paper] [Code]
4D Panoptic Scene Graph Generation
Jingkang Yang, Jun Cen, Wenxuan Peng, Shuai Liu, Fangzhou Hong, Xiangtai Li, Kaiyang Zhou, Qifeng Chen, Ziwei Liu
NeurIPS 2023 Spotlight
[Paper] [Code]
CLIP-guided Prototype Modulating for Few-shot Action Recognition
Xiang Wang, Shiwei Zhang, Jun Cen, Changxin Gao, Yingya Zhang, Deli Zhao, Nong Sang
IJCV 2023
[Paper] [Code]
Enlarge Instance-specific and Class-specific Information for Open-set Action Recognition
Jun Cen, Shiwei Zhang, Xiang Wang, Yixuan Pei, Zhiwu Qing, Yingya Zhang, Qifeng Chen
CVPR 2023
[Paper] [Code]
The Devil is in the Wrongly-classified Samples: Towards Unified Open-set Recognition
Jun Cen*, Di Luan*, Shiwei Zhang, Yixuan Pei, Yingya Zhang, Deli Zhao, Shaojie Shen, Qifeng Chen
ICLR 2023
[Paper] [Code]
Learning a Condensed Frame for Memory-Efficient Video Class-Incremental Learning
Yixuan Pei*, Zhiwu Qing*, Jun Cen, Xiang Wang, Shiwei Zhang, Yaxiong Wang, Mingqian Tang, Nong Sang, Xueming Qian
NeurIPS 2022
[Paper]
Open-world Semantic Segmentation for LIDAR Point Clouds
Jun Cen, Peng Yun, Shiwei Zhang, Junhao Cai, Di Luan, Michael Yu Wang, Ming Liu, Mingqian Tang
ECCV 2022
[Paper] [Code]
Real-Time Collision-Free Grasp Pose Detection With Geometry-Aware Refinement Using High-Resolution Volume
Junhao Cai, Jun Cen, Haokun Wang, Michael Yu Wang
RA-L with ICRA 2022
[Paper] [Project Page and Code]
Open-set 3D Object Detection
Jun Cen, Peng Yun, Junhao Cai, Michael Yu Wang, Ming Liu
3DV 2021
[Paper]
Conflicts between Likelihood and Knowledge Distillation in Task Incremental Learning for 3D Object Detection
Peng Yun, Jun Cen, Ming Liu
3DV 2021
[Paper]
Deep Metric Learning for Open World Semantic Segmentation
Jun Cen, Peng Yun, Junhao Cai, Michael Yu Wang, Ming Liu
ICCV 2021
[Paper] [Code] [Video]
BORM: Bayesian Object Relation Model for Indoor Scene Recognition
Liguang Zhou, Jun Cen, Xingchao Wang, Zhenglong Sun, Tin Lun Lam, Yangsheng Xu
IROS 2021
[Paper] [Code] [Video]
Precision forward design for 3D printing using kinematic sensitivity via Jacobian matrix considering uncertainty
Jinghua Xu, Xueqing Feng, Jun Cen, Shuyou Zhang
IJAMT 2020
[Paper]

GitHub

WorldVLA: Towards Autoregressive Action World Model.
Python
Star   – Fork   –
Generative Artificial Intelligence in Robotic Manipulation: A Survey.
Markdown
Star  – Fork  –
SegmentAnyRGBD.
Python
Star  – Fork  –
[NeurIPS 2023] 4D Panoptic Scene Graph Generation.
Python
Star  – Fork  –
[NeurIPS 2023] Segment Any Point Cloud Sequences by Distilling Vision Foundation Models.
Python
Star  – Fork  –
[RA-L 2023] CMDFusion: Bidirectional Fusion Network with Cross-modality Knowledge Distillation for LIDAR Semantic Segmentation.
Python
Star  – Fork  –
[CVPR 2023] Enlarge Instance-specific and Class-specific Information for Open-set Action Recognition.
Python
Star  – Fork  –
[ICLR 2023] The Devil is in the Wrongly-classified Samples: Towards Unified Open-set Recognition.
Python
Star  – Fork  –
[ECCV 2022] Open-world Semantic Segmentation for LIDAR Point Clouds.
Python
Star  – Fork  –
[ICCV 2021] Deep Metric Learning for Open World Semantic Segmentation.
Python
Star  – Fork  –

Selected Honors & Awards

Academic Services

Conference Reviewer:
Conference on Computer Vision and Pattern Recognition (CVPR)
International Conference on Computer Vision (ICCV)
European Conference on Computer Vision (ECCV)
Conference on Neural Information Processing Systems (NeurIPS)
International Conference on Learning Representations (ICLR)
International Conference on Machine Learning (ICML)
International Conference on Robotics and Automation (ICRA)
International Conference on Intelligent Robots and Systems (IROS)
Journal Reviewer:
International Journal of Computer Vision (IJCV)
Transactions on Pattern Analysis and Machine Intelligence (TPAMI)
Transactions on Neural Networks and Learning Systems (TNNLS)
Transactions on Circuits and Systems for Video Technology (TCSVT)
Transactions on Multimedia (TMM)
Robotics and Automation Letters (RA-L)
Top