We are all tired of wearing headphones sometimes. But playing music out loud can bring much inconvenience to other people in the room. So we would like to build an indoor device to enable a zero-weight, contact-free and personalizable audio experience which is only audible to one person in a certain range of 3D space.
Acoustic Phantom is a directional speaker system with object recognition and real-time tracking. With this speaker, the sound will follow the designated user and only this user however he moves. Users can register themselves with our system and play audios wirelessly.
Persistently track and beam directional sound to a specific user in the room.
User can use the smartphone app to interact with the system via WiFi/Bluetooth (e.g. initiate tracking and play music).
Memorize the user’s settings and preferences when he/she re-enters the room.
Perform its functionality when components are placed in different locations of the room or in another room.
Raspberry Pi compatible wide angle HD webcam that transmits video data to the RPi via CSI bus.
A Soundlazer directional speaker that uses ultrasonic waves to transmit audio to some user in a beam of sound.
The pan-tilt platform holds the camera, the speaker and the RPi, and is driven by two servos which are controlled by the PWM signals sent from the RPi.
A Raspberry Pi is responsible for running CV algorithms, communicating with iOS app, controlling platform servos, and outputting audio to the speaker.
We use OpenGL in C++ to perform multiple color tracking, and OpenCV to track larger features.
An iOS app is used to turn on/off the system, complete the registration steps and performing AirPlay by communicating with RPi (sending/receiving commands and data).
A research group at Korea Advanced Institute of Science and Technology has developed a similar system called the “Shadow Sound System” which achieved user tracking via a webcam mounted on a raised pan-tilt platform. Captured video stream is sent to a laptop for computer vision processing, and the resulting movement commands are sent to an Arduino for servoing the speaker.
This company is also developing products including musical interface controllers based on computer vision, directional speakers and mobile applications. However, the company’s current products only include directional speakers and sets of directional speakers and sensors.