[PKU 04804023] Cognitive Models for Visual Commonsense

  • Course Info
    • Schedule: 3 hours per week, for 16 lectures
    • Lecture: Thursday 15:10-18:00 @ 三教507
    • Credits: 3
  • Instructor Info
    • Professor: Yixin Zhu (yixin.zhu@pku.edu.cn)
    • Office Hours: Friday 9:30-11:30 by appointment (email, not WeChat)
    • Office: 资源西楼2208A(南门外)
  • TA: TBA
  • Reference: Song-Chun Zhu and Yixin Zhu, Cognitive Models for Visual Commonsense, Springer [early draft]


  • understand the significance of commonsense in modern AI
  • master basic knowledge of cognitive science for modern AI
  • derive statistical computational models of commonsense to solve challenging AI problems
  • code and build complex system to model certain aspect(s) of commonsense


The course provides an in-depth introduction to the knowledge related to visual commonsense reasoning with a focus on both theories and computational methods. In particular, it introduces the “dark matter” of AI by studying the physical and social commonsense, invisible in terms of pixels but nevertheless drive the creation, maintenance, and development of visual scenes.


  • basic knowledge of statistics, e.g., Bayes rule
  • comfortable with programming in Linux using modern deep learning libraries
  • excellent English skills for reading literature, presentation, and writing homework and reports
  • familiar with Latex for writing homework and reports


  • strict rule: no late submission will be accepted after the due date
  • attending: 1% $\times$ 15 weeks = 15%
  • paper presentation (in English): 17%
  • homework (in English): 3% $\times$ 11 lectures = 33%
  • project presentation (in English): 25%
  • project report (in English): 10%


This course is made possible by a group of collaborators and students

my advisors,

my long-term collaborators,

the most awesome peers in the world (in alphabetical order),

and the smartest students/postdocs in the world (in alphabetical order)

  • Dr. Bo Dai (PKU, BIGAI)
  • Baoxiong Jia (UCLA, BIGAI)
  • Shuwen Qiu (UCLA)
  • Liangru Xiang (THU, BIGAI)
  • Chao Xu (UCLA)
  • Luyao Yuan (UCLA)
  • Chi Zhang (UCLA, BIGAI)
  • Zeyu Zhang (UCLA, BIGAI)
  • Dr. Fangwei Zhong (PKU, BIGAI)