李 响 Xiang Li
(He / him)
I am a Ph.D. candidate in the Dept of CS at Stony Brook University, advised by Prof. Michael S. Ryoo. I received my MS and BS from SJTU under the supervision of Prof. Yue Zhou.
My primary research interest is self-supervised visual representation learning for robotics. Now I am also working on diffusion models and vision language models.
I am actively looking for self-motivated robots to
collaborate with.
Please email me if your robots are interested (even if you are
not a robot :P
Email: xiangli8[at]cs[dot]stonybrook[dot]edu
Selected Publications
-
LLaRA: Supercharging Robot Learning Data for Vision-Language PolicyNew Highlighted
TL;DR, Effortlessly transform a pretrained VLM into a robot policy and enhance its performance using auxiliary data generated from the same behavior cloning dataset.Xiang Li, Cristina Mata, Jongwoo Park, Kumara Kahatapitiya, Yoo Sung Jang, Jinghuan Shang, Kanchana Ranasinghe, Ryan Burgert, Mu Cai, Yong Jae Lee, and Michael S. Ryoopreprint, 2024 -
Crossway Diffusion: Improving Diffusion-based Visuomotor Policy via Self-supervised LearningHighlighted
TL;DR, Explore various SSL objectives to enhance diffusion-based behavior cloning, finding that a straightforward state reconstruction objective consistently improves performance.ICRA, 2024 -
Does Self-supervised Learning Really Improve Reinforcement Learning from Pixels?Highlighted
TL;DR, Unfortunately, NO (when the same amount of data and augmentation is used).NeurIPS, 2022
Other Publications
-
Understanding Long Videos in One Multimodal Language Model PassNew
preprint, 2024 -
Diffusion Illusions: Hiding Images in Plain SightSIGGRAPH, 2024CVPR Outstanding Demos Award, 2023 -
Limited Data, Unlimited Potential: A Study on ViTs Augmented by Masked AutoencodersSrijan Das, Tanmay Jain, Dominick Reilly, Pranav Balaji, Soumyajit Karmakar, Shyam Marjit, Xiang Li, Abhijit Das, and Michael S. RyooWACV, 2024 -
Peekaboo: Text to Image Diffusion Models are Zero-Shot SegmentorsWorkshop on Open-Domain Reasoning Under Multi-Modal Settings @ CVPR, 2023 -
Neural Neural Textures Make Sim2Real ConsistentCoRL, 2022 -
StARformer: Transformer with State-Action-Reward Representations for Robot LearningIEEE TPAMI, Special Issue on Transformer Models in Vision, 2022Conference version is aceepted by ECCV, 2022
Projects
-
RoboDaVinciPython New
This machine was created to celebrate the successful defense of Dr. Jinghuan Shang, drawing inspiration from this demo. It can draw your avatar and add a Ph.D. hood to commemorate your achievement. -
Ultimate TicTacToe machinePython
This device plays physical TicTacToe game with human and I built it for our lab parties.
The agent behind comes from Alpha-Zero, which has shown impressive performance. In fact, only very few lab members have been able to outplay it - myself not included.
For this demonstration, I've programmed the robot to intentionally forfeit one checker, it does not seem to be very happy though... -
LiDAR based SLAM + GPS + NavigationROS C++ Python
This work was done during my internship at Beijing Falcon Self-driving Co., Ltd.. -
RC Car + Raspi + CameraArduino C++ Python
An open-source solution to turn general RC cars into powerful mobile platforms with vision. -
Visual tracking using a camera on a motorized gimbalC++
Use a camera on a motorized gimbal and STC algorithm to track moving vehicles. The motorized gimbal is controlled by a PI controller via PELCO-D protocol. GUI is made with MFC. -
There are other projects I worked on and enjoyed a lot: playing Android/PC games autonomously using computer vision, GPU cluster monitor built with Grafana and InfluxDB, Single Board Computer (SBC) with various sensors, and so on. They are not technically challenging but I love the fact that they significantly improved the quality of my life :)
Awards
- CVPR Outstanding Demos Award [Diffusion Illusions], 2023
- Outstanding Graduate of Colleges and Universities in Shanghai, China (Top 5%), 2018
- National Scholarship for Graduate Students, 2017
- Outstanding Graduates of SJTU, 2015