Overview
This first week introduces the course structure and grading, the cognitive-science framing behind the course, and the standards for presentations and academic integrity that apply for the rest of the semester. Students will also form groups, review candidate project topics, and submit their topic preferences.
Instructor
Yixin Zhu, PKU
Topics Covered
- Course logistics: schedule, office hours, and how to reach the instructor and TA
- Why cognitive science, not just benchmark scores, matters for evaluating AI: do current models perceive, reason, and understand others, or just score well?
- The four-layer evaluation framework used throughout the course: SOTA benchmark reproduction, open-source dataset testing, non-open-source paradigm replication, and original test suite design
- Grading breakdown and the three-round group presentation schedule
- Oral/written presentation rubric: Soundness, Novelty, Depth, Presentation, and Technical Details
- Academic integrity and the ten common forms of plagiarism (Clone, Ctrl+C, Find-Replace, Remix, Recycle, Hybrid, Mashup, 404 Error, Aggregator, Re-Tweet), and how to avoid them
In-class Activities
- Review the full course schedule, grading breakdown, and presentation rubric.
- Walk through the four-layer evaluation framework with concrete examples (e.g., a causal-reasoning subset of BIG-Bench, I-PhyRe for intuitive physics).
- Introduce candidate project topics and the topic-preference process.
- Discuss the academic integrity policy and review examples of the ten common plagiarism patterns.
Assignment
There is no formal graded assignment for Week 1. Students should:
- Find teammates and form a project group; choose a team lead who will later report on individual contributions.
- Rank their top 3 preferred project topics in the course questionnaire (topics with too many preferences are adjusted; remaining students may be randomly assigned).
- Watch for the topic lottery link and results, announced via WeChat before the next lecture.
- Read the plagiarism page and the course policy on academic integrity.
- Prepare questions for the Week 2 Q&A session (see Notes below).
Notes
- Week 2 is a Q&A session: students ask questions about the topic and readings they selected in Week 1, and the instructor answers, so come prepared having actually read the assigned papers.
- Start early on writing, figures, and anything involving software or video production — these always take longer than expected once the project is underway.
- Slides for this lecture are distributed separately to enrolled students; no video recording is provided for this course.