Postdoc

VCLA lab, UCLA

Biography

I am a postdoctoral scholar in Prof. Song-Chun Zhu’s VCLA lab at UCLA.

My research builds interactive AI by integrating high-level common sense ( functionality, affordance, physics, causality ) with raw sensory inputs (pixels and haptic signals) to enable richer representation and abstract reasoning on objects, scenes, shapes, numbers, and agents.

My work is currently supported by ONR MURI on Scene Understanding, DARPA XAI, and ONR Cognitive Systems for Human-Machine Teaming. I am a co-organizer of Vision Meets Cognition (FPIC) workshops, 3D Scene Understanding for Vision, Graphics, and Robotics workshops, and Virtual Reality Meets Physical Reality workshops.

We are looking for highly motivated students with exceptional programming skills and solid math background to work on 3D computer vision, abstract reasoning, physics-based simulation, and robotics. If you are a UCLA student and interested in working with me, please read some papers from the reading list before sending me an email. For a partial list of ongoing projects in the lab, see VCLA Project Bulletin.

Interests

Computer Vision
Artificial Intelligence
Human–Robot Interaction

Education

PhD in Statistics, 2018

UCLA
MS in Computer Science, 2013

UCLA
BEng in Software Engineering, 2012

Xi'an Jiaotong University

Publication

See all publications

[ECCV20] LEMMA: A Multi-view Dataset for Learning Multi-agent Multi-task Activities

Understanding and interpreting human actions is a long-standing challenge and a critical indicator of perception in artificial …

Baoxiong Jia, Yixin Chen, Siyuan Huang, Yixin Zhu, Song-Chun Zhu

PDF Code Dataset Supp Presentation Web

[ECCV20] LEMMA: A Multi-view Dataset for Learning Multi-agent Multi-task Activities

[IROS20] Human-Robot Interaction in a Shared Augmented Reality Workspace

We design and develop a new shared Augmented Reality (AR) workspace for Human-Robot Interaction (HRI), which establishes a …

Shuwen Qiu, Hangxin Liu, Zeyu Zhang, Yixin Zhu, Song-Chun Zhu

PDF Code Video Presentation Web

[IROS20] Human-Robot Interaction in a Shared Augmented Reality Workspace

[IROS20] Graph-based Hierarchical Knowledge Representation for Robot Task Transfer from Virtual to Physical World

We study the hierarchical knowledge transfer problem using a cloth-folding task, wherein the agent is first given a set of human …

Zhenliang Zhang, Yixin Zhu, Song-Chun Zhu

PDF Code Video Presentation Web

[IROS20] Graph-based Hierarchical Knowledge Representation for Robot Task Transfer from Virtual to Physical World

[Engineering20] Dark, Beyond Deep: A Paradigm Shift to Cognitive AI with Humanlike Common Sense

Recent progress in deep learning is essentially based on a “big data for small tasks” paradigm, under which massive amounts …

Yixin Zhu, Tao Gao, Lifeng Fan, Siyuan Huang, Mark Edmonds, Hangxin Liu, Feng Gao, Chi Zhang, Siyuan Qi, Ying Nian Wu, Josh Tenenbaum, Song-Chun Zhu

PDF

[Engineering20] Dark, Beyond Deep: A Paradigm Shift to Cognitive AI with Humanlike Common Sense

[SIGGRAPH20] A Massively Parallel and Scalable Multi-GPU Material Point Method

Harnessing the power of modern multi-GPU architectures, we present a massively parallel simulation system based on the Material Point …

Xinlei Wang, Yuxing Qiu, Stuart Slattery, Yu Fang, Minchen Li, Song-Chun Zhu, Yixin Zhu, Min Tang, Dinesh Manocha, Chenfanfu Jiang

PDF Code Video Supp Presentation Web

[SIGGRAPH20] A Massively Parallel and Scalable Multi-GPU Material Point Method

[SIGGRAPH20] IQ-MPM: An Interface Quadrature Material Point Method for Non-sticky Strongly Two-Way Coupled Nonlinear Solids and Fluids

We propose a novel scheme for simulating two-way coupled interactions between nonlinear elastic solids and incompressible fluids. The …

Yu Fang, Ziyin Qu, Minchen Li, Xinxin Zhang, Yixin Zhu, Mridul Aanjaneya, Chenfanfu Jiang

PDF Code Video Presentation

[SIGGRAPH20] IQ-MPM: An Interface Quadrature Material Point Method for Non-sticky Strongly Two-Way Coupled Nonlinear Solids and Fluids

[arXiv20] LETO: Hybrid Lagrangian-Eulerian Method for Topology Optimization

We propose LETO, a new hybrid Lagrangian-Eulerian method for topology optimization. At the heart of LETO lies in a hybrid particle-grid …

Yue Li, Xuan Li, Minchen Li, Yixin Zhu, Bo Zhu, Chenfanfu Jiang

PDF Code Video

[arXiv20] LETO: Hybrid Lagrangian-Eulerian Method for Topology Optimization

[ICRA20] Joint Inference of States, Robot Knowledge, and Human (False-)Beliefs

Aiming to understand how human (false-)belief—a core socio-cognitive ability—would affect human interactions with robots, …

Tao Yuan, Hangxin Liu, Lifeng Fan, Zilong Zheng, Tao Gao, Yixin Zhu, Song-Chun Zhu

PDF Video Presentation

[ICRA20] Joint Inference of States, Robot Knowledge, and Human (False-)Beliefs

[ICRA20] Congestion-aware Evacuation Routing using Augmented Reality Devices

We present a congestion-aware routing solution for indoor evacuation, which produces real-time individual-customized evacuation routes …

Zeyu Zhang, Hangxin Liu, Ziyuan Jiao, Yixin Zhu, Song-Chun Zhu

PDF Code Video Presentation

[ICRA20] Congestion-aware Evacuation Routing using Augmented Reality Devices

[ScienceRobotics19] A tale of two explanations: Enhancing human trust by explaining robot behavior

The ability to provide comprehensive explanations of chosen actions is a hallmark of intelligence. Lack of this ability impedes the …

Mark Edmonds, Feng Gao, Hangxin Liu, Xu Xie, Siyuan Qi, Brandon Rothrock, Yixin Zhu, Ying Nian Wu, Hongjing Lu, Song-Chun Zhu

PDF Code Video Summary Video Supp Smithsonian IEEE Spectrum UCLA Samueli School of Engineering Web

[ScienceRobotics19] A tale of two explanations: Enhancing human trust by explaining robot behavior

[AAAI20] Theory-based Causal Transfer: Integrating Instance-level Induction and Abstract-level Structure Learning

Learning transferable knowledge across similar but different settings is a fundamental component of generalized intelligence. In this …

Mark Edmonds, Xiaojian Ma, Siyuan Qi, Yixin Zhu, Hongjing Lu, Song-Chun Zhu

PDF Code Poster Supp Web

[AAAI20] Theory-based Causal Transfer: Integrating Instance-level Induction and Abstract-level Structure Learning

[AAAI20] Machine Number Sense: A Dataset of Visual Arithmetic Problems for Abstract and Relational Reasoning

As a comprehensive indicator of mathematical thinking and intelligence, the number sense (Dehaene 2011) bridges the induction of …

Wenhe Zhang, Chi Zhang, Yixin Zhu, Song-Chun Zhu

PDF Code Dataset Poster Web

[AAAI20] Machine Number Sense: A Dataset of Visual Arithmetic Problems for Abstract and Relational Reasoning

[NeurIPS19] Learning Perceptual Inference by Contrasting

‘Thinking in pictures,’ [1] i.e., spatial-temporal reasoning, effortless and instantaneous for humans, is believed to be a …

Chi Zhang, Baoxiong Jia, Feng Gao, Yixin Zhu, Hongjing Lu, Song-Chun Zhu

PDF Code Poster Slides Blog Web

[NeurIPS19] Learning Perceptual Inference by Contrasting

[NeurIPS19] PerspectiveNet: 3D Object Detection from a Single RGB Image via Perspective Points

Detecting 3D objects from a single RGB image is intrinsically ambiguous, thus requiring appropriate prior knowledge and intermediate …

Siyuan Huang, Yixin Chen, Tao Yuan, Siyuan Qi, Yixin Zhu, Song-Chun Zhu

PDF Code Poster Supp Web

[NeurIPS19] PerspectiveNet: 3D Object Detection from a Single RGB Image via Perspective Points

[ICCV19] Holistic++ Scene Understanding: Single-view 3D Holistic Scene Parsing and Human Pose Estimation with Human-Object Interaction and Physical Commonsense

We propose a new 3D holistic++ scene understanding problem, which jointly tackles two tasks from a single-view image: (i) holistic …

Yixin Chen, Siyuan Huang, Tao Yuan, Yixin Zhu, Siyuan Qi, Song-Chun Zhu

PDF Code Poster Video Supp Web

[ICCV19] Holistic++ Scene Understanding: Single-view 3D Holistic Scene Parsing and Human Pose Estimation with Human-Object Interaction and Physical Commonsense

See all publications

Postdoc

VCLA lab, UCLA

Biography

Interests

Education

Publication

[ECCV20] LEMMA: A Multi-view Dataset for Learning Multi-agent Multi-task Activities

[IROS20] Human-Robot Interaction in a Shared Augmented Reality Workspace

[IROS20] Graph-based Hierarchical Knowledge Representation for Robot Task Transfer from Virtual to Physical World

[Engineering20] Dark, Beyond Deep: A Paradigm Shift to Cognitive AI with Humanlike Common Sense

[SIGGRAPH20] A Massively Parallel and Scalable Multi-GPU Material Point Method

[SIGGRAPH20] IQ-MPM: An Interface Quadrature Material Point Method for Non-sticky Strongly Two-Way Coupled Nonlinear Solids and Fluids

[arXiv20] LETO: Hybrid Lagrangian-Eulerian Method for Topology Optimization

[ICRA20] Joint Inference of States, Robot Knowledge, and Human (False-)Beliefs

[ICRA20] Congestion-aware Evacuation Routing using Augmented Reality Devices

[ScienceRobotics19] A tale of two explanations: Enhancing human trust by explaining robot behavior

[AAAI20] Theory-based Causal Transfer: Integrating Instance-level Induction and Abstract-level Structure Learning

[AAAI20] Machine Number Sense: A Dataset of Visual Arithmetic Problems for Abstract and Relational Reasoning

[NeurIPS19] Learning Perceptual Inference by Contrasting

[NeurIPS19] PerspectiveNet: 3D Object Detection from a Single RGB Image via Perspective Points

[ICCV19] Holistic++ Scene Understanding: Single-view 3D Holistic Scene Parsing and Human Pose Estimation with Human-Object Interaction and Physical Commonsense

Downloads

Demos

Students Mentored

Postdoc

Biography

Interests

Education

Tags

Publication

Downloads

Demos

Students Mentored