Module 6: AI Hardware and Acceleration

Overview

This week delves into the world of AI Hardware and Acceleration, exploring how specialized hardware and acceleration techniques can significantly enhance the performance of AI systems. We will cover various aspects of hardware optimization and acceleration methods used in modern AI applications.

Instructors

Xiyuan Tang and Yaoyu Tao, PKU

Topics Covered

• Introduction to AI hardware acceleration
• Data sensing and processing for edge intelligent
• Software-hardware co-design for AI chip architectures
• Emerging AI computation acceleration technologies

Assignments

Full Assignment: Compare matrix multiplication performance using NumPy and PyTorch with varying matrix sizes and dimensions. Analyze the impact of GPU acceleration on training time and accuracy for an MNIST model.

Short Assignment: Compare matrix multiplication performance using NumPy and PyTorch with varying matrix sizes and dimensions.

Additional Resources

• Efficient Processing of Deep Neural Networks by Vivienne Sze, Yu-Hsin Chen, Tien-Ju Yang, Joel Emer
• AI and Memory Wall by Amir Gholami
• Hardware Acceleration for Efficient Deep Learning by Intel
• Edge AI: The Future of Artificial Intelligence by NVIDIA
• Slurm Workload Manager - Official documentation for Slurm
• Slurm Quick Start Tutorial - A beginner’s guide to using Slurm
• Using GPUs with Slurm - Guide on how to request and use GPUs in a Slurm environment
• Slurm for GPU Scheduling - Detailed guide on submitting GPU jobs with Slurm

Notes

• This module integrates concepts from previous weeks, particularly in machine learning and robotics, with a focus on hardware optimization.
• For the full assignment, you will need access to a GPU. You are encouraged to utilize the computing resources provided for this course (see Computing Resources). Note that the course does not provide GPU compute or compensation for GPU hours to non-Tong Class students; you will need to arrange GPU access yourself, whether through your own resources, rented GPU hours from a cloud provider, or a free tier such as Google Colab or Kaggle Notebooks. If your cluster uses Slurm for job scheduling, familiarize yourself with basic Slurm commands such as srun, sbatch, and squeue.
• For the short assignment, try to connect the hardware advancement you discuss with potential applications or improvements in AI systems.
• As always, document your code thoroughly and use version control (Git) for your project.
• Submit your assignments via Gradescope.