李 响 Xiang Li
(He / him)
I am a Ph.D. candidate in the Dept of CS at Stony Brook University, advised by Prof. Michael S. Ryoo. I received my MS and BS from SJTU under the supervision of Prof. Yue Zhou.
My primary research interest is self-supervised visual representation learning for robotics. Now I am also working on diffusion models and vision language models for robotics.
I am actively looking for self-motivated robots to
collaborate with.
Please email me if your robots are interested (even if you are
not a robot :P
Email: xiangli8[at]cs[dot]stonybrook[dot]edu
Selected Publications
-
LLaRA: Supercharging Robot Learning Data for Vision-Language PolicyNew Highlighted
TL;DR, Effortlessly transform a pretrained VLM into a robot policy by aligning robot action to image pixels and enhance its performance using auxiliary data generated in a self-supervised fashion.Xiang Li, Cristina Mata, Jongwoo Park, Kumara Kahatapitiya, Yoo Sung Jang, Jinghuan Shang, Kanchana Ranasinghe, Ryan Burgert, Mu Cai, Yong Jae Lee, and Michael S. RyooSpotlight at LangRob @ CoRL, 2024, Presentation by Prof. RyooICLR, 2025 -
Crossway Diffusion: Improving Diffusion-based Visuomotor Policy via Self-supervised LearningHighlighted
TL;DR, Explore various SSL objectives to enhance diffusion-based behavior cloning, finding that a straightforward state reconstruction objective consistently improves performance.ICRA, 2024 -
Does Self-supervised Learning Really Improve Reinforcement Learning from Pixels?Highlighted
TL;DR, Unfortunately, NO (when the same amount of data and augmentation is used).NeurIPS, 2022
Other Publications
-
Understanding Long Videos with Multimodal Language ModelsNew
ICLR, 2025 -
Diffusion Illusions: Hiding Images in Plain SightSIGGRAPH, 2024CVPR Outstanding Demos Award, 2023 -
Limited Data, Unlimited Potential: A Study on ViTs Augmented by Masked AutoencodersSrijan Das, Tanmay Jain, Dominick Reilly, Pranav Balaji, Soumyajit Karmakar, Shyam Marjit, Xiang Li, Abhijit Das, and Michael S. RyooWACV, 2024 -
Peekaboo: Text to Image Diffusion Models are Zero-Shot SegmentorsWorkshop on Open-Domain Reasoning Under Multi-Modal Settings @ CVPR, 2023 -
Neural Neural Textures Make Sim2Real ConsistentCoRL, 2022 -
StARformer: Transformer with State-Action-Reward Representations for Robot LearningIEEE TPAMI, Special Issue on Transformer Models in Vision, 2022Conference version is aceepted by ECCV, 2022
Projects
-
RoboDaVinciPython New
This machine was created to celebrate the successful defense of Dr. Jinghuan Shang, drawing inspiration from this demo. It can draw your avatar and add a Ph.D. hood to commemorate your achievement. -
Ultimate TicTacToe machinePython
This device plays physical TicTacToe game with human and I built it for our lab parties.
The agent behind comes from Alpha-Zero, which has shown impressive performance. In fact, only very few lab members have been able to outplay it - myself not included.
For this demonstration, I've programmed the robot to intentionally forfeit one checker, it does not seem to be very happy though... -
LiDAR based SLAM + GPS + NavigationROS C++ Python
This work was done during my internship at Beijing Falcon Self-driving Co., Ltd.. -
RC Car + Raspi + CameraArduino C++ Python
An open-source solution to turn general RC cars into powerful mobile platforms with vision. -
Visual tracking using a camera on a motorized gimbalC++
Use a camera on a motorized gimbal and STC algorithm to track moving vehicles. The motorized gimbal is controlled by a PI controller via PELCO-D protocol. GUI is made with MFC. -
There are other projects I worked on and enjoyed a lot: playing Android/PC games autonomously using computer vision, GPU cluster monitor built with Grafana and InfluxDB, Single Board Computer (SBC) with various sensors, and so on. They are not technically challenging but I love the fact that they significantly improved the quality of my life :)
Awards
- CVPR Outstanding Demos Award [Diffusion Illusions], 2023
- Outstanding Graduate of Colleges and Universities in Shanghai, China (Top 5%), 2018
- National Scholarship for Graduate Students, 2017
- Outstanding Graduates of SJTU, 2015