Yixin Zhu

Yixin Zhu

Assistant Professor

PKU Institute for AI

Biography

Dr. Yixin Zhu received a Ph.D. degree (‘18) from UCLA advised by Prof. Song-Chun Zhu. His research builds interactive AI by integrating high-level common sense (functionality, affordance, physics, causality, intent) with raw sensory inputs (pixels and haptic signals) to enable richer representation and cognitive reasoning on objects, scenes, shapes, numbers, and agents. Dr. Zhu directs the PKU CoRe Lab, working on abstract reasoning, visually grounded reasoning, and interactive reasoning.

[CV] [Dissertation]

Getting Started 帮你自学大学四年AI的所有内容
通班相关文章 汇集官方和权威渠道对通班的报道
博雅一号（成都） 目前仅限AI院全职PI和北大通班学生使用

Topics

Activity Affordance Analogy AR Causality Communication Container EBM Functionality Generalization HOI Intent IQ Manipulation Motion Synthesis MPM Number Sense Optimization Personality Physics Programming Language Representation Learning Scene Parsing Scene Reconstruction Scene Synthesis TAMP Teaming Tool UAV VR XAI

Pre-Print

[arXiv22] CHAIRS: Towards Full-Body Articulated Human-Object Interaction
Nan Jiang, Tengyu Liu, Zhexuan Cao, Jieming Cui, Yixin Chen, He Wang, Yixin Zhu, Siyuan Huang
PDF Cite Code Web Request Dataset
[arXiv22] To think inside the box, or to think out of the box? Scientific discovery via the reciprocation of insights and concepts
Yu-Zhe Shi, Manjie Xu, Wenjuan Han, Yixin Zhu
PDF Cite
[arXiv22] On the Complexity of Bayesian Generalization
Yu-Zhe Shi, Manjie Xu, John E. Hopcroft, Kun He, Joshua B. Tenenbaum, Song-Chun Zhu, Ying Nian Wu, Wenjuan Han, Yixin Zhu
PDF Cite Code Web
[arXiv22] Neural-Symbolic Recursive Machine for Systematic Generalization
Qing Li, Yixin Zhu, Yitao Liang, Ying Nian Wu, Song-Chun Zhu, Siyuan Huang
PDF Cite Code
[arXiv22] EST: Evaluating Scientific Thinking in Artificial Agents
Manjie Xu, Guangyuan Jiang, Chi Zhang, Song-Chun Zhu, Yixin Zhu
PDF Cite
[arXiv22] MPI: Evaluating and Inducing Personality in Pre-trained Language Models
Guangyuan Jiang, Manjie Xu, Song-Chun Zhu, Wenjuan Han, Chi Zhang, Yixin Zhu
PDF Cite
[arXiv22] PartAfford: Part-level Affordance Discovery from 3D Objects
Chao Xu, Yixin Chen, He Wang, Song-Chun Zhu, Yixin Zhu, Siyuan Huang
PDF Cite
[arXiv21] HALMA: Humanlike Abstraction Learning Meets Affordance in Rapid Problem Solving
Sirui Xie, Xiaojian Ma, Peiyu Yu, Yixin Zhu, Ying Nian Wu, Song-Chun Zhu
PDF Cite Web

Publications

[CVPR23] Diffusion-based Generation, Optimization, and Planning in 3D Scenes
Siyuan Huang, Zan Wang, Puhao Li, Baoxiong Jia, Tengyu Liu, Yixin Zhu, Wei Liang, Song-Chun Zhu
PDF Cite Code Video Supp Web Hugging Face Checkpoints
[ICLR23] Understanding Embodied Reference with Touch-Line Transformer
Yang Li, Xiaoxue Chen, Hao Zhao, Jiangtao Gong, Guyue Zhou, Federico Rossano, Yixin Zhu
PDF Cite Code Web
[ICLR23] A Minimalist Dataset for Systematic Generalization of Perception, Syntax, and Semantics
Qing Li, Siyuan Huang, Yining Hong, Yixin Zhu, Ying Nian Wu, Song-Chun Zhu
PDF Cite Code Dataset Video Web Weights & Bias
[ICRA23] GenDexGrasp: Generalizable Dexterous Grasping
Puhao Li, Tengyu Liu, Yuyang Li, Yiran Geng, Yixin Zhu, Yaodong Yang, Siyuan Huang
PDF Cite Code Dataset Video Web
[ICRA23] Rearrange Indoor Scenes for Human-Robot Co-Activity
Weiqi Wang, Zihang Zhao, Ziyuan Jiao, Yixin Zhu, Song-Chun Zhu, Hangxin Liu
PDF Cite Code Video Web
[Engineering23] A Reconfigurable Data Glove for Reconstructing Physical and Virtual Grasps
Hangxin Liu, Zeyu Zhang, Ziyuan Jiao, Zhenliang Zhang, Minchen Li, Chenfanfu Jiang, Yixin Zhu, Song-Chun Zhu
PDF Cite Video Web
[NeurIPS22] On the Learning Mechanisms in Physical Reasoning
Shiqian Li, Kewen Wu, Chi Zhang, Yixin Zhu
PDF Cite Code Poster Video Supp Web Presentation 北大新闻网 北大AI院官网 北大科研官微 北大新工科官微 北大AI院官微 VALSE论文速讲
[NeurIPS22] Emergent Graphical Conventions in a Visual Communication Game
Shuwen Qiu, Sirui Xie, Lifeng Fan, Tao Gao, Jungseock Joo, Song-Chun Zhu, Yixin Zhu
PDF Cite Code Poster Video Supp Web Presentation 北大新闻网 北大AI院官网 北大新工科官微 北大AI院官微
[NeurIPS22] HUMANISE: Language-conditioned Human Motion Generation in 3D Scenes
Zan Wang, Yixin Chen, Tengyu Liu, Yixin Zhu, Wei Liang, Siyuan Huang
PDF Cite Code Dataset Poster Video Supp Web Presentation 北大新闻网 北大AI院官网 北大新工科官微 北大AI院官微
[IJCV22] Scene Reconstruction with Functional Objects for Robot Autonomy
Muzhi Han, Zeyu Zhang, Ziyuan Jiao, Xu Xie, Yixin Zhu, Song-Chun Zhu, Hangxin Liu
PDF Cite Code Web 北大新闻网 北大AI院官网 北大AI院官微
[ScienceRobotics22] In situ bidirectional human-robot value alignment
Luyao Yuan, Xiaofeng Gao, Zilong Zheng, Mark Edmonds, Ying Nian Wu, Federico Rossano, Hongjing Lu, Yixin Zhu, Song-Chun Zhu
PDF Cite Background & Summary in Layperson Language Code & Data Movie S1 & S2 MDAR Reproducibility Checklist Supp PDF Invited Talk at DeepMind 新华社 CGTN 科技日报/新华网 中国科学报/科学网 北大新闻网 北大AI院官网 北大AI院官微 TechXplore
[ECCV22] Learning Algebraic Representation for Systematic Generalization in Abstract Reasoning
Chi Zhang, Sirui Xie, Baoxiong Jia, Ying Nian Wu, Song-Chun Zhu, Yixin Zhu
PDF Cite Code Dataset Supp Blog Web 北大新闻网 北大AI院官网 北大AI院官微
[IROS22] Sequential Manipulation Planning on Scene Graph
Ziyuan Jiao, Yida Niu, Zeyu Zhang, Song-Chun Zhu, Yixin Zhu, Hangxin Liu
PDF Cite Code Video Web 北大新闻网 北大AI院官网 北大AI院官微
[IROS22] Downwash-aware Control Allocation for Over-actuated UAV Platforms
Yao Su, Chi Chu, Meng Wang, Jiarui Li, Liu Yang, Yixin Zhu, Hangxin Liu
PDF Cite Video Web 北大新闻网 北大AI院官网 北大AI院官微
[RA-L/IROS22] Understanding Physical Effects for Effective Tool-use
Zeyu Zhang, Ziyuan Jiao, Weiqi Wang, Yixin Zhu, Song-Chun Zhu, Hangxin Liu
PDF Cite Code Video 北大新闻网 北大AI院官网 北大AI院官微
[ICML22] Latent Diffusion Energy-Based Model for Interpretable Text Modeling
Peiyu Yu, Sirui Xie, Xiaojian Ma, Baoxiong Jia, Bo Pang, Ruiqi Gao, Yixin Zhu, Ying Nian Wu, Song-Chun Zhu
PDF Cite Code Supp Presentation
[JCB2022] Sharing Rewards Undermines Coordinated Hunting
Minglu Zhao, Ning Tang, Annya Dahmani, Yixin Zhu, Federico Rossano, Tao Gao
PDF Cite
[CogSci22] What Is the Point? A Theory of Mind Model of Relevance
Kaiwen Jiang, Annya Dahmani, Stephanie Stacy, Boxuan Jiang, Federico Rossano, Yixin Zhu, Tao Gao
PDF Cite
[RA-L/ICRA22] Object Gathering with a Tethered Robot Duo
Yao Su, Yuhong Jiang, Yixin Zhu, Hangxin Liu
PDF Cite Code Video Web
[AAIL21] Patching interpretable And-Or-Graph knowledge representation using augmented reality
Hangxin Liu, Yixin Zhu, Song-Chun Zhu
PDF Cite
See all publications

Students

*
All Core:Causality CoRe:Generalization CoRe:Physics CV:Activity CV:HOI CV:Scene Parsing RO:Human-Robot Teaming
Chuyue Tang

Chuyue Tang

CoRe, TongClass@PKU ‘20, Harvard Visiting Student

Kewen Wu

Kewen Wu

CoRe, Undergrad@THU, ‘19

Shiji Xin

Shiji Xin

CoRe, TuringClass@PKU, ‘19

Yida Niu

Yida Niu

Human-Robot Teaming, Ph.D.@PKU, ‘23

Jieming Cui

Jieming Cui

CV, Ph.D.@PKU, ‘23

Shiqian Li

Shiqian Li

CoRe, Ph.D.@PKU, ‘22

Manjie Xu

Manjie Xu

CoRe, M.Sc.@BIT, ‘22

Nan Jiang

Nan Jiang

CV, Ph.D.@PKU, ‘22

Guangyuan Jiang

Guangyuan Jiang

CoRe, TongClass@PKU, ‘20

Yuzhe Shi

Yuzhe Shi

CoRe, Undergrad@HUST, ‘18

Alumni

  • Wenhe Zhang, Master, Computer Science, UCLA, 2020 Fall
  • Xiaojian Ma, Master, Computer Science, UCLA, 2019 Fall
  • Xiaolin Fang, Ph.D., CSAIL, MIT, 2019 Fall
  • Shu Wang, Ph.D., Statistics, UCLA, 2018 Fall
  • Wenwen Si, Master, Computer Vision, CMU, 2018 Fall
  • Hangxin Liu, Ph.D., Computer Science, UCLA, 2018 Spring
  • Jenny Lin, Ph.D., Computer Science, CMU, 2017 Fall
  • Mark Edmonds, Ph.D., Computer Science, UCLA, 2017 Fall
  • Tian Ye, Master, Robotics, CMU, 2017 Fall
  • Feng Gao, Master, Statistics, UCLA, 2017 Fall
  • Xu Xie, Master, Statistics, UCLA, 2017 Fall
  • Xingwen Guo, Master, Computer Science, Yale, 2017 Fall
  • Chi Zhang, Master, Computer Science, UCLA, 2017 Fall
  • Jingyu Shao, Master, Statistics, UCLA, 2016 Winter
  • Yutong Zhang, Master in Computer Science, UCLA, 2015 Fall