Jun CEN

HKUST
Clear Water Bay, Kawloon
Hong Kong

About Me

I am a Ph.D. student at the Robotics Institute, The Hong Kong University of Science and Technology, where I work on computer vision and autonomous driving under the supervision of Prof. Qifeng Chen and Prof. Michael Yu Wang.

My research lies in computer vision, multi-modality learning, robotics, etc. My previous work focus on multi-modality LLM, 2D semantic segmentation, 3D object detection, 3D LIDAR semantic segmentation, video recognition and image recognition. I'm always looking for related colaboration. If you are interested to chat with me, feel free to drop me an email.

I will be on the job market in the summer of 2024. Here is my CV. Please feel free to drop me an email if you have any interests.

Education

MSc Student | The Hong Kong University of Science and Technology
Time: Sep 2019 - July 2020. Mentor: Prof. Lilong Cai

Undergraduate Student | Zhejiang University
Time: Sep 2015 - Sep 2019. Mentor: Prof. Jinghua Xu

Experience

Research Intern | Natural Language Computing Group, Microsoft Research Asia
Time: Aug 2023 - Present. Mentor: Dr. Chenfei Wu, Dr. Nan Duan

Visiting Student | CVIP Group, Southern University of Science and Technology
Time: Aug 2023 - Present. Mentor: Prof. Jianguo Zhang

Visiting Student | MMLab, Nanyang Technological University
Time: Mar 2023 - Present. Mentor: Prof. Ziwei Liu

Reseach Intern | City Brain & Autonomous Driving Group, Damo Academy, Alibaba
Time: Mar 2022 - Mar 2023. Mentor: Dr. Shiwei Zhang

Algorithm Intern | Autonomous Driving Group, Hikvision Reseach Institute
Time: Mar 2020 - July 2020. Mentor: Dr. Yushi Zhu

Research Assistant | Robotics and Artificial Intelligence Laboratory, The Chinese University of Hong Kong (Shenzhen)
Time: Nov 2019 - Feb 2020. Mentor: Prof. Tin Lun Lam

Publications [Google Scholar]

Using Left and Right Brains Together: Towards Vision and Language Planning
Jun Cen*, Chenfei Wu*, Xiao Liu, Shengming Yin, Yixuan Pei, Jinglong Yang, Qifeng Chen, Nan Duan, Jianguo Zhang
[Arxiv 2024]
[Paper] [Code]

CMDFusion: Bidirectional Fusion Network with Cross-modality Knowledge Distillation for LIDAR Semantic Segmentation
Jun Cen, Shiwei Zhang, Yixuan Pei, Kun Li, Hang Zheng, Maochun Luo, Yingya Zhang, Qifeng Chen
[RA-L 2023] Robotics and Automation Letters, 2023
[Paper] [Code]

Segment Any Point Cloud Sequences by Distilling Vision Foundation Models
Youquan Liu*, Lingdong Kong*, Jun Cen, Runnan Chen, Wenwei Zhang, Liang Pan, Kai Chen, Ziwei Liu
[NeurIPS 2023 Spotlight] Conference on Neural Information Processing, 2023
[Paper] [Code]

4D Panoptic Scene Graph Generation
Jingkang Yang, Jun Cen, Wenxuan Peng, Shuai Liu, Fangzhou Hong, Xiangtai Li, Kaiyang Zhou, Qifeng Chen, Ziwei Liu
[NeurIPS 2023 Spotlight] Conference on Neural Information Processing, 2023
[Paper] [Code]

CLIP-guided Prototype Modulating for Few-shot Action Recognition
Xiang Wang, Shiwei Zhang, Jun Cen, Changxin Gao, Yingya Zhang, Deli Zhao, Nong Sang
[IJCV 2023] International Journal of Computer Vision, 2023
[Paper] [Code]

Enlarge Instance-specific and Class-specific Information for Open-set Action Recognition
Jun Cen, Shiwei Zhang, Xiang Wang, Yixuan Pei, Zhiwu Qing, Yingya Zhang, Qifeng Chen
[CVPR 2023] Computer Vision and Pattern Recognition Conference, 2023
[Paper] [Code]

The Devil is in the Wrongly-classified Samples: Towards Unified Open-set Recognition
Jun Cen*, Di Luan*, Shiwei Zhang, Yixuan Pei, Yingya Zhang, Deli Zhao, Shaojie Shen, Qifeng Chen
[ICLR 2023] International Conference on Learning Representations, 2023
[Paper] [Code]

Learning a Condensed Frame for Memory-Efficient Video Class-Incremental Learning
Yixuan Pei*, Zhiwu Qing*, Jun Cen, Xiang Wang, Shiwei Zhang, Yaxiong Wang, Mingqian Tang, Nong Sang, Xueming Qian
[NeurIPS 2022] Conference on Neural Information Processing, 2022
[Paper]


Open-world Semantic Segmentation for LIDAR Point Clouds
Jun Cen, Peng Yun, Shiwei Zhang, Junhao Cai, Di Luan, Michael Yu Wang, Ming Liu, Mingqian Tang
[ECCV 2022] European Conference on Computer Vision, 2022
[Paper] [Code]

Real-Time Collision-Free Grasp Pose Detection With Geometry-Aware Refinement Using High-Resolution Volume
Junhao Cai, Jun Cen, Haokun Wang, Michael Yu Wang
[RA-L with ICRA 2022] Robotics and Automation Letters, 2022
[Paper] [Project Page and Code]

Open-set 3D Object Detection
Jun Cen, Peng Yun, Junhao Cai, Michael Yu Wang, Ming Liu
[3DV 2021] International Conference on 3D Vision, 2021
[Paper]

Conflicts between Likelihood and Knowledge Distillation in Task Incremental Learning for 3D Object Detection
Peng Yun, Jun Cen, Ming Liu
[3DV 2021] International Conference on 3D Vision, 2021
[Paper]

Deep Metric Learning for Open World Semantic Segmentation
Jun Cen, Peng Yun, Junhao Cai, Michael Yu Wang, Ming Liu
[ICCV 2021] International Conference on Computer Vision, 2021
[Paper] [Code] [Video]

BORM: Bayesian Object Relation Model for Indoor Scene Recognition
Liguang Zhou, Jun Cen, Xingchao Wang, Zhenglong Sun, Tin Lun Lam, Yangsheng Xu
[IROS 2021] International Conference on Intelligent Robots and Systems, 2021
[Paper] [Code] [Video]



Precision forward design for 3D printing using kinematic sensitivity via Jacobian matrix considering uncertainty
Jinghua Xu, Xueqing Feng, Jun Cen, Shuyou Zhang
The International Journal of Advanced Manufacturing Technology, 2022
[Paper]

Projects

SAD: Segment Any RGBD
Jun Cen, Yizheng Wu, Xingyi Li, Jingkang Yang, Yixuan Pei, Lingdong Kong
Got more than 600 stars on Github, and more than 1K Likes and 140K Views on Twitter!
NeurIPS 2023 workshop on Robustness of Few-shot and Zero-shot Learning in Foundation Models, 2023
[Code] [Technical Report] [Huggingface] [Twitter]

Services

  • Conference and Journal Reviewer:
     IEEE Transactions on Pattern Analysis and Machine Intelligence (TPAMI)
     IEEE Transactions on Neural Networks and Learning Systems (TNNLS)
     Conference on Neural Information Processing Systems (NeurIPS)
     IEEE/CVF Computer Vision and Pattern Recognition Conference (CVPR)
     IEEE/CVF International Conference on Computer Vision (ICCV)
     IEEE/RAS International Conference on Intelligent Robots and Systems (IROS)
     IEEE/RAS International Conference on Robotics and Automation (ICRA)
     IEEE/CAA Journal of Automatica Sinica (JAS) (中国自动化学报)

Awards

  • Outstanding Research Intern, Alibaba Group, 2023. (15/1500, 4 candidates for computer vision per year)
  • Overseas Research Award, HKUST, 2023.
  • Postgraduate Studentship, 2020-2024, HKUST.
  • School of Engineering Excellent Student Scholarship, 2019-2020, HKUST. (Top 5%)
  • School of Engineering Entrance Scholarship, 2019-2020, HKUST. (Top 2%)