Biography

Dr. Yixin Zhu received a Ph.D. degree (‘18) from UCLA advised by Prof. Song-Chun Zhu. His research builds interactive AI by integrating high-level common sense (functionality, affordance, physics, causality, intent) with raw sensory inputs (pixels and haptic signals) to enable richer representation and abstract reasoning on objects, scenes, shapes, numbers, and agents. He is a co-organizer of Vision Meets Cognition (FPIC) workshops, 3D Scene Understanding for Vision, Graphics, and Robotics workshops, and Virtual Reality Meets Physical Reality workshops.

During his Ph.D. and postdoc studies, his work was supported by DARPA MSEE, DARPA SIMPLEX, DARPA XAI, ONR MURI on Scene Understanding, and ONR Cognitive Systems for Human-Machine Teaming.

His group is looking for highly motivated undergrads, Ph.D. students, and postdocs with exceptional programming skills and solid math backgrounds to work on 3D computer vision, abstract reasoning, physics-based simulation, and cognitive robot.

[CV][Dissertation]

Interests
  • computer vision
  • artificial intelligence
  • human-robot interaction
Education

  • Ph.D. in statistics, 2018

    UCLA

  • M.S. in computer science, 2013

    UCLA

  • B.Eng. in software engineering, 2012

    Xi'an Jiaotong University

Topics

Affordance Analogy AR Causality Communication Container Functionality Glove HRI Intent IQ Manipulation MPM Optimization Physics Raven Scene Parsing Simulation VR XAI

Pre-Print

[arXiv22] PartAfford: Part-level Affordance Discovery from 3D Objects
[arXiv21] Emergent Graphical Conventions in a Visual Communication Game
[arXiv21] Learning Algebraic Representation for Systematic Generalization in Abstract Reasoning
[arXiv21] HALMA: Humanlike Abstraction Learning Meets Affordance in Rapid Problem Solving
[arXiv21] A HINT from Arithmetic: On Systematic Generalization of Perception, Syntax, and Semantics

Publications

[RA-L/IROS22] Object Gathering with a Tethered Robot Duo
Yao Su, Yuhong Jiang, Yixin Zhu, Hangxin Liu
[AAIL21] Patching interpretable And-Or-Graph knowledge representation using augmented reality
Hangxin Liu, Yixin Zhu, Song-Chun Zhu
[NeurIPS21] Unsupervised Foreground Extraction via Deep Region Competition
Peiyu Yu, Sirui Xie, Xiaojian Ma, Yixin Zhu, Ying Nian Wu, Song-Chun Zhu
[RA-L22] Synthesizing Diverse and Physically Stable Grasps with Arbitrary Hand Structures using Differentiable Force Closure Estimator
Tengyu Liu, Zeyu Liu, Ziyuan Jiao, Yixin Zhu, Song-Chun Zhu
[ICCV21] YouRefIt: Embodied Reference Understanding with Language and Gesture
Yixin Chen, Qing Li, Deqian Kong, Yik Lun Kei, Song-Chun Zhu, Tao Gao, Yixin Zhu, Siyuan Huang
[ICCV21] Spatio-temporal Self-Supervised Representation Learning for 3D Point Clouds
Siyuan Huang, Yichen Xie, Song-Chun Zhu, Yixin Zhu
[IROS21] Consolidating Kinematic Models to Promote Coordinated Mobile Manipulations
Ziyuan Jiao, Zeyu Zhang, Xin Jiang, David Han, Song-Chun Zhu, Yixin Zhu, Hangxin Liu
[IROS21] Efficient Task Planning for Mobile Manipulation: a Virtual Kinematic Chain Perspective
Ziyuan Jiao, Zeyu Zhang, Weiqi Wang, David Han, Song-Chun Zhu, Yixin Zhu, Hangxin Liu
[IROS21] Communicative Learning with Natural Gestures for Embodied Navigation Agents with Human-in-the-Scene
Qi Wu, Cheng-Ju Wu, Yixin Zhu, Jungseock Joo
[CogSci21] Individual vs. Joint Perception: a Pragmatic Model of Pointing as Communicative Smithian Helping
Kaiwen Jiang, Stephanie Stacy, Chuyu Wei, Adelpha Chan, Federico Rossano, Yixin Zhu, Tao Gao
[ACL-Findings21] GRICE: A Grammar-based Dataset for Recovering Implicature and Conversational rEasoning
Zilong Zheng, Shuwen Qiu, Lifeng Fan, Yixin Zhu, Song-Chun Zhu
[CVPR21] Learning Triadic Belief Dynamics in Nonverbal Communication from Videos
Lifeng Fan, Shuwen Qiu, Zilong Zheng, Tao Gao, Song-Chun Zhu, Yixin Zhu
[CVPR21] ACRE: Abstract Causal Reasoning Beyond Covariation
Chi Zhang, Baoxiong Jia, Mark Edmonds, Song-Chun Zhu, Yixin Zhu
[CVPR21] Abstract Spatial-Temporal Reasoning via Probabilistic Abduction and Execution
Chi Zhang, Baoxiong Jia, Song-Chun Zhu, Yixin Zhu
[ICRA21] Reconstructing Interactive 3D Scene by Panoptic Mapping and CAD Model Alignments
Muzhi Han, Zeyu Zhang, Ziyuan Jiao, Xu Xie, Yixin Zhu, Song-Chun Zhu, Hangxin Liu
[ICRA21] Congestion-aware Multi-agent Trajectory Prediction for Collision Avoidance
Xu Xie, Chi Zhang, Yixin Zhu, Ying Nian Wu, Song-Chun Zhu
[IJNME21] Lagrangian‐Eulerian Multi‐Density Topology Optimization with the Material Point Method
Yue Li, Xuan Li, Minchen Li, Yixin Zhu, Bo Zhu, Chenfanfu Jiang
[ECCV20] LEMMA: A Multi-view Dataset for Learning Multi-agent Multi-task Activities
Baoxiong Jia, Yixin Chen, Siyuan Huang, Yixin Zhu, Song-Chun Zhu
[IROS20] Human-Robot Interaction in a Shared Augmented Reality Workspace
Shuwen Qiu, Hangxin Liu, Zeyu Zhang, Yixin Zhu, Song-Chun Zhu
[IROS20] Graph-based Hierarchical Knowledge Representation for Robot Task Transfer from Virtual to Physical World
Zhenliang Zhang, Yixin Zhu, Song-Chun Zhu
[Engineering20] Dark, Beyond Deep: A Paradigm Shift to Cognitive AI with Humanlike Common Sense
Yixin Zhu, Tao Gao, Lifeng Fan, Siyuan Huang, Mark Edmonds, Hangxin Liu, Feng Gao, Chi Zhang, Siyuan Qi, Ying Nian Wu, Josh Tenenbaum, Song-Chun Zhu
[SIGGRAPH20] A Massively Parallel and Scalable Multi-GPU Material Point Method
Xinlei Wang, Yuxing Qiu, Stuart Slattery, Yu Fang, Minchen Li, Song-Chun Zhu, Yixin Zhu, Min Tang, Dinesh Manocha, Chenfanfu Jiang
[SIGGRAPH20] IQ-MPM: An Interface Quadrature Material Point Method for Non-sticky Strongly Two-Way Coupled Nonlinear Solids and Fluids
Yu Fang, Ziyin Qu, Minchen Li, Xinxin Zhang, Yixin Zhu, Mridul Aanjaneya, Chenfanfu Jiang
[ICRA20] Joint Inference of States, Robot Knowledge, and Human (False-)Beliefs
Tao Yuan, Hangxin Liu, Lifeng Fan, Zilong Zheng, Tao Gao, Yixin Zhu, Song-Chun Zhu
[ICRA20] Congestion-aware Evacuation Routing using Augmented Reality Devices
Zeyu Zhang, Hangxin Liu, Ziyuan Jiao, Yixin Zhu, Song-Chun Zhu
[ScienceRobotics19] A tale of two explanations: Enhancing human trust by explaining robot behavior
Mark Edmonds, Feng Gao, Hangxin Liu, Xu Xie, Siyuan Qi, Brandon Rothrock, Yixin Zhu, Ying Nian Wu, Hongjing Lu, Song-Chun Zhu
[AAAI20] Theory-based Causal Transfer: Integrating Instance-level Induction and Abstract-level Structure Learning
Mark Edmonds, Xiaojian Ma, Siyuan Qi, Yixin Zhu, Hongjing Lu, Song-Chun Zhu
[AAAI20] Machine Number Sense: A Dataset of Visual Arithmetic Problems for Abstract and Relational Reasoning
Wenhe Zhang, Chi Zhang, Yixin Zhu, Song-Chun Zhu
[NeurIPS19] Learning Perceptual Inference by Contrasting
Chi Zhang, Baoxiong Jia, Feng Gao, Yixin Zhu, Hongjing Lu, Song-Chun Zhu
[NeurIPS19] PerspectiveNet: 3D Object Detection from a Single RGB Image via Perspective Points
Siyuan Huang, Yixin Chen, Tao Yuan, Siyuan Qi, Yixin Zhu, Song-Chun Zhu
See all publications

Students

*
All Cognition and Reasoning Computer Vision
Mar 31, 2022
Shiqian Li
Shiqian Li
Jul 21, 2021
Manjie Xu
Manjie Xu
Jul 21, 2021
Nan Jiang
Nan Jiang

Alumni

  • Wenhe Zhang, Master, Computer Science, UCLA, 2020 Fall
  • Xiaojian Ma, Master, Computer Science, UCLA, 2019 Fall
  • Xiaolin Fang, Ph.D., CSAIL, MIT, 2019 Fall
  • Shu Wang, Ph.D., Statistics, UCLA, 2018 Fall
  • Wenwen Si, Master, Computer Vision, CMU, 2018 Fall
  • Hangxin Liu, Ph.D., Computer Science, UCLA, 2018 Spring
  • Jenny Lin, Ph.D., Computer Science, CMU, 2017 Fall
  • Mark Edmonds, Ph.D., Computer Science, UCLA, 2017 Fall
  • Tian Ye, Master, Robotics, CMU, 2017 Fall
  • Feng Gao, Master, Statistics, UCLA, 2017 Fall
  • Xu Xie, Master, Statistics, UCLA, 2017 Fall
  • Xingwen Guo, Master, Computer Science, Yale, 2017 Fall
  • Chi Zhang, Master, Computer Science, UCLA, 2017 Fall
  • Jingyu Shao, Master, Statistics, UCLA, 2016 Winter
  • Yutong Zhang, Master in Computer Science, UCLA, 2015 Fall