Xiaoxiang (Tony) Dong

Xiaoxiang (Tony) Dong

Robotics Institute  ·  Carnegie Mellon University

I am an undergraduate student at Carnegie Mellon University, majoring in Computer Science with a focus on Robotics. I work at the intersection of robot learning and manipulation. My research focuses on learning from demonstrations, perception for manipulation, and dexterous robotic control. I am advised by Prof. Jeffrey Ichnowski in the Momentum Lab and Prof. Weiming Zhi from the University of Sydney.

scroll

News

Experience

Momentum Lab

09/2025 – Present

Undergraduate Researcher  ·  Summer Undergraduate Research Fellowship (SURF)

Advised by Prof. Jeffrey Ichnowski

CMU Robotics Institute

  • Zero-Shot Functional Manipulation (1st Author): Formulated SemAnCorr, a semantic-anchored correspondence framework enabling zero-shot manipulation skill transfer across topologically varied object categories (IEEE RA-L Under Review).
  • Dexterous Policy Learning (2nd Author): Co-developed a framework for learning dexterous manipulation policies directly from unconstrained RGB human videos via 3D hand-object trajectory reconstruction (IEEE RA-L Under Review).
  • Implementation: Built visual feature matching and pose estimation pipelines using PyTorch, training policies across multi-DOF robotic hands and simulated environments.

DROP Lab

03/2025 – 08/2025

Research Intern

Advised by Prof. Matthew Johnson-Roberson & Prof. Weiming Zhi

CMU Robotics Institute  ·  Vanderbilt University College of Connected Computing

  • Video-to-Motion Generation (1st Author): Developed a joint flow trajectory optimization framework to generate kinematically and dynamically feasible robot motions directly from RGB video demonstrations (ICRA 2026).
  • Cross-Modal Conditioning (2nd Author): Co-designed a cross-modal instruction architecture leveraging Vision-Language Models (VLMs) to condition generative motion models for zero-shot robot execution (ICRA 2026).
  • Generative Modeling: Implemented flow-matching based generative pipelines and evaluated object-centric motion synthesis across complex tabletop manipulation tasks.

Kantor's Lab

05/2024 – 05/2025

Undergraduate Researcher

Advised by Prof. George Kantor

CMU Robotics Institute

  • Robotic System: Designed and built an autonomous mobile robot to locate and collect scattered badminton shuttles in real-time.
  • Perception & Planning Pipeline: Fine-tuned a custom Mask R-CNN model for shuttle detection, integrated a particle filter for state localization, and implemented a greedy Travelling Salesman Problem (TSP) algorithm for energy-efficient path planning.

Publications

Paper thumbnail

SemAnCorr: Semantic Anchored Correspondence for Zero-Shot Manipulation Skill Transfer

Xiaoxiang Dong, William Baron, Hongyi Chen, Uksang Yoo, Jeffrey Ichnowski, Weiming Zhi

Under Review IEEE Robotics and Automation Letters (RA-L)

Paper Code Project Page
Paper thumbnail

Dexterous Manipulation Policies from RGB Human Videos via 3D Hand-Object Trajectory Reconstruction

Hongyi Chen, Xiaoxiang Dong, Tiancheng Wu, Liquan Wang, Yash Jangir, Yaru Niu, Yufei Ye, Homanga Bharadhwaj, Zackory Erickson, Jeffrey Ichnowski

Under Review IEEE Robotics and Automation Letters (RA-L)

Paper Code Project Page
Paper thumbnail

Joint Flow Trajectory Optimization For Feasible Robot Motion Generation from Video Demonstrations

Xiaoxiang Dong, Matthew Johnson-Roberson, Weiming Zhi

ICRA 2026 IEEE International Conference on Robotics and Automation (ICRA)

Paper Project Page
Paper thumbnail

Cross-Modal Instructions for Robot Motion Generation

William Baron, Xiaoxiang Dong, Matthew Johnson-Roberson, Weiming Zhi

ICRA 2026 IEEE International Conference on Robotics and Automation (ICRA)

Paper

Projects

Project demo

Reinforcement Learning for Block Pickup with Visual State Estimation

Investigating tradeoffs between PPO, SAC, and MBRL for block pick-and-place. Explored teacher-student distillation with changing input modality.

Reinforcement Learning Manipulation Teacher-Student Distillation 16-831 Introduction to Robot Learning
Report Code Project Page
Project demo

Mini-Beam Diffusion: Searching for Better Paths Through Noise

Explored an inference-time search strategy for diffusion models that maintains multiple candidate denoising trajectories, pruning less promising samples as generation progresses.

Diffusion Models Image Generation Inference-Time Search 10-403 Generative AI
Report Code Project Page
Project demo

Optimizing Collection System: A Greedy Travelling Salesman Approach for Path-Following Agent

Built an autonomous robot that detects and collects scattered badminton shuttlecocks using a custom Mask R-CNN model, a greedy TSP path planner, and a particle filter localization module.

Path Planning Semantic Segmentation Robotic System SLAM ROS2
Poster Code Project Page

Leadership

15-210: Parallel and Sequential Data Structures and Algorithms

08/2026 – Present 08/2025 – 05/2026

Head Teaching Assistant (prev. Teaching Assistant)

Carnegie Mellon University

  • Course staff for Parallel and Sequential Data Structures and Algorithms; designed lecture and homework materials, taught recitations, held office hours, graded assignments.

CMU Badminton Club

08/2025 – 05/2026 08/2023 – 05/2025

President & Team Captain (prev. Student Athlete)

Carnegie Mellon University

  • Organized collegiate team tournaments and hosted scrimmages with universities across the Ohio–Pennsylvania region.

Academic Service

Paper Reviewer

  • 3× 2026 IEEE International Conference on Robotics and Automation (ICRA)
  • 1× 2026 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS)

Conference Volunteering

  • Student Volunteer, 2026 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS)

Awards & Honors

Summer Undergraduate Research Fellowship (SURF)

Carnegie Mellon University

2026

School of Computer Science Dean's List

CMU School of Computer Science · Named all semesters attended

2023, 2024, 2025, 2026

Governor General's Academic Medal

Office of the Governor General of Canada · Best academic performance in high school

2023

Canadian Mathematics Olympiads

Canadian Mathematics Society · Finished top 50 across country

2023

United States Computing Olympiads - Gold Division

United States Computing Olympiad Association

2022