Demo

View demo video on YouTube

The Motivation

We are all tired of wearing headphones sometimes. But playing music out loud can bring much inconvenience to other people in the room. So we would like to build an indoor device to enable a zero-weight, contact-free and personalizable audio experience which is only audible to one person in a certain range of 3D space.

The Project

Acoustic Phantom is a directional speaker system with object recognition and real-time tracking. With this speaker, the sound will follow the designated user and only this user however he moves. Users can register themselves with our system and play audios wirelessly.

The Architecture

Other Details

Design Requirements

Persistently track and beam directional sound to a specific user in the room.

User can use the smartphone app to interact with the system via WiFi/Bluetooth (e.g. initiate tracking and play music).

Memorize the user’s settings and preferences when he/she re-enters the room.

Perform its functionality when components are placed in different locations of the room or in another room.

Technical Specification

Tracking camera

Raspberry Pi compatible wide angle HD webcam that transmits video data to the RPi via CSI bus.

Directional speaker

A Soundlazer directional speaker that uses ultrasonic waves to transmit audio to some user in a beam of sound.

Pan-tilt platform

The pan-tilt platform holds the camera, the speaker and the RPi, and is driven by two servos which are controlled by the PWM signals sent from the RPi.

Central processing hub

A Raspberry Pi is responsible for running CV algorithms, communicating with iOS app, controlling platform servos, and outputting audio to the speaker.

Computer vision algorithm

We use OpenGL in C++ to perform multiple color tracking, and OpenCV to track larger features.

Smartphone app

An iOS app is used to turn on/off the system, complete the registration steps and performing AirPlay by communicating with RPi (sending/receiving commands and data).

Competitive Analysis

Shadow Sound System

A research group at Korea Advanced Institute of Science and Technology has developed a similar system called the “Shadow Sound System” which achieved user tracking via a webcam mounted on a raised pan-tilt platform. Captured video stream is sent to a laptop for computer vision processing, and the resulting movement commands are sent to an Arduino for servoing the speaker.

Ultrasonic Audio Technologies

This company is also developing products including musical interface controllers based on computer vision, directional speakers and mobile applications. However, the company’s current products only include directional speakers and sets of directional speakers and sensors.