18-453 Introduction to XR Systems
Fall 2024

XR Class Concept

Course Description

Digitization of reality is at the cusp of widespread adoption; it also embodies a unique convergence of techniques that span the gamut of electrical and computer engineering. This course introduces students to augmented/virtual/mixed reality systems, or what is broadly referred to as extended reality (XR). The course covers hardware and software topics central to XR systems, namely, content capture, streaming, and display. The course also aims to make students comfortable with some of the basic tools required to digitize, optimize, render, control, and visualize content.

In addition to lectures spanning the eco-system of topics central to XR systems, the course will feature labs where students will implement techniques to digitize 3D content using RGBD cameras, importing and manipulating digital content in modern XR development workflows, building controllers with 6 DOF, and finally building a 3D display. By the end of the course, all of these components will be connected to complete the cycle between reality to digital reality and back.

The course will have one project that will require implementing and demonstrating a feature on an XR display. The project will have multiple deliverables, including a mid-semester presentation that will involve a deep dive into related technical articles.

Course Goal

At the culmination of the course, the student should be able to understand of gamut of techniques underlying XR systems, including different flavors of XR systems that are in vogue. The course will provide hands-on experience in building different components of an XR system, and the ability to critically analyze the tradeoffs in each.

Office Hours

Office hours timings will be announced soon.

Pre-requisites

18-290 Signals and Systems OR 18-213 Computer Systems

Class Schedule

Date Topics Labs / Homework
Tu, Aug 26 Introduction to XR
Th, Aug 28 XR Landscape
Fr, Aug 29 Recitation 1 Lab 1 released
Tu, Sept 3 Human Perception I
Th, Sept 5 Human Perception II
Fr, Sept 7 Recitation 2 Lab 1 Checkpoint 1 Due
Tu, Sept 10 Recitation 3 Lab 2 released
Th, Sept 12 XR Tools
Fr, Sept 13 Software Tools and Real-time Rendering Lab 1 Final Due
Tu, Sept 17 Display I: Stereoscopic
Th, Sept 19 Display II: Integral
Fr, Sept 20 Recitation 4
Tu, Sept 24 Display III: Varifocal / Multifocal
Th, Sept 26 Display IV: Holographic and Exotic
Fr, Sept 27 Recitation 5 Lab 2 Due, Lab 3 released
Tu, Oct 1 Localization I
Th, Oct 3 Localization II
Fr, Oct 4 Recitation 6 Lab 3 Due / Lab 4 released
Tu, Oct 8 Gordon Wetzstein guest lecture
Th, Oct 12 Deep dive into holograms
Th, Oct 13 Project pitch
Tu, Oct 15 Fall Break No Class
Th, Oct 17 Fall Break No Class
Fr, Oct 18 Fall Break No Class
Tu, Oct 22 Reality Capture I: Basics of 3D scanning
Th, Oct 24 Reality Capture II: Appearance / BRDFs / Light Stages
Fr, Oct 25 Recitation 8
Tu, Oct 29 Reality Capture III: Digitizing the Body - Human Capture
Th, Oct 31 Reality Capture IV: Mapping and Scene Reconstruction
Fr, Nov 1 Recitation 9 Lab 4 due
Tu, Nov 5 Democracy Day No Class
Th, Nov 7 David Lindlbauer guest lecture
Fr, Nov 8 Recitation 10
Tu, Nov 12 Chris Harrison group guest lecture
Th, Nov 14 Frontiers of VR I
Fr, Nov 15 Recitation 11
Tu, Nov 19 Frontiers of VR II
Th, Nov 21 Frontiers of VR III
Fr, Nov 22 Recitation 12
Tu, Nov 26 Guest Lecture
Th, Nov 28 Thanksgiving No Class
Fr, Nov 29 Thanksgiving No Class
Tu, Dec 3 Final Project Presentation and demo Final Project Due
Th, Dec 5 Final Project Presentation and demo
Fr, Dec 6 Final Project Presentation and demo

Labs

The labs comprises of:

Experience Sessions

Experience sessions lets you to get familiar with various state-of-the-art technologies and demos and will be sporadically scheduled during the semester.

Acknowledgments

This course borrows material and inspiration from a number of sources: