Creating Spatialized Audio Playback System for Mixed-Reality Application

Intel Corporation Mathematics, 2018-19

Liaison(s): Charles Johnson II, Peter Sankhagowit
Advisor(s): Alfonso Castro
Students(s): Jacob Cordeiro, Greg Derecho, Jeffrey Rutledge, Brenner Ryan, Thitaree Tanprasert (PM), Nina Thiebaut

Intel TrueView is a system for displaying realistic replays of sports scenes from all points of view in the form of mixed reality (MR). An important factor in creating a believable MR scene is to create a suitable audio replay. Our project aims to develop a system to turn dry sound into realistic, spatialized audio so each object in an MR scene sounds like it’s in the same environment as the user, and at the position displayed in the scene. We explore two approaches to this problem, which are recognition of room properties via the room impulse response (RIR) and via photos of the room or parts of the room. We are performing user testing by implementing and integrating each approach into a simple MR scene.