18-453 Introduction to XR Systems
Fall 2023

XR Class Concept

Course Description

Digitization of reality is at the cusp of widespread adoption; it also embodies a unique convergence of techniques that span the gamut of electrical and computer engineering. This course introduces students to augmented/virtual/mixed reality systems, or what is broadly referred to as extended reality (XR). The course covers hardware and software topics central to XR systems, namely, content capture, streaming, and display. The course also aims to make students comfortable with some of the basic tools required to digitize, optimize, render, control, and visualize content.

In addition to lectures spanning the eco-system of topics central to XR systems, the course will feature labs where students will implement techniques to digitize 3D content using RGBD cameras, importing and manipulating digital content in modern XR development workflows, building controllers with 6 DOF, and finally building a 3D display. By the end of the course, all of these components will be connected to complete the cycle between reality to digital reality and back.

The course will have one project that will require implementing and demonstrating a feature on an XR display. The project will have multiple deliverables, including a mid-semester presentation that will involve a deep dive into related technical articles.

Course Goal

At the culmination of the course, the student should be able to understand of gamut of techniques underlying XR systems, including different flavors of XR systems that are in vogue. The course will provide hands-on experience in building different components of an XR system, and the ability to critically analyze the tradeoffs in each.

Selected Final Project

XR Class Project

Recitation and Office Hours

Recitation: Fridays 11:00 am - 12:20 pm ET, WEH 5409

Office hours will be held during the semester at the following times:


18-290 Signals and Systems OR 18-213 Computer Systems

Class Schedule

Date Topics Homework
Tu, Aug 29 Introduction to XR
Th, Aug 31 XR Landscape
Tu, Sept 5 Human Perception I
Th, Sept 7 Human Perception II Lab 1 Checkpoint 1 Due
Tu, Sept 12 XR Tools
Th, Sept 14 Software Tools and Real-time Rendering Lab 1 Final Due
Tu, Sept 19 Stereoscopic Displays
Th, Sept 21 Integral Displays
Tu, Sept 26 Focal Planes - Varifocal / Multifocal Systems Lab 2 Due
Th, Sept 28 Optical Tools and Holographic Displays
Tu, Oct 3 Exotic Displays
Th, Oct 5 Localization and Tracking I Lab 3 Due
Tu, Oct 10 Localization and Tracking II
Th, Oct 12 Basics of 3D Scanning
Tu, Oct 17 Fall Break No Class
Th, Oct 19 Fall Break No Class
Tu, Oct 24 Appearance / BRDFs / Light Stages
Th, Oct 26 Scene Reconstruction I Lab 4 Due
Tu, Oct 31 Scene Reconstruction II Final Project Out
Th, Nov 2 Guest Lecture (Larry Shea, CMU School of Drama)
Tu, Nov 7 Democracy Day No Class
Th, Nov 9 Digitizing the Body - Human Capture
Tu, Nov 14 Guest Lecture (David Lindlbauer, CMU HCII)
Th, Nov 16 Collaborative Virtual Environments
Tu, Nov 21 Guest Lecture (Mallesh Dasari, NEU)
Th, Nov 23 Thanksgiving No Class
Tu, Nov 28 Guest Lecture (Doug Lanman, Meta)
Th, Nov 30 Frontiers of XR
Tu, Dec 5 Final Project Presentation Final Project Due
Th, Dec 7 Final Project Presentation


The final grade is consisted of:


The labs comprises of:

Experience Sessions

Experience sessions lets you to get familiar with various state-of-the-art technologies and demos and will be sporadically scheduled during the semester.