The Swiss lakeside town of Montreux was once again host for its famous Jazz festival. Over the past 50 years, the festival has seen some of the greatest acts in music history, including Nina Simone, Ray Charles, Miles Davis, Prince and David Bowie, among others.
For this year’s festival, audioborn developed a project allowing the festival to be captured through 360° / 3D video and sound recording technology for the first time. This content has been rendered and can be used in virtual reality applications, permitting users to experience the selected acts in real-life quality.
Commissioned by the Metamedia Center of the University École polytechnique fédérale de Lausanne (EPFL) together with the Montreux Jazz Festival, project lead and official technology partner audioborn has partnered with PRG Lab for the video expertise. audioborn was responsible for capturing and reproducing all concerts in 3D audio with its software real-time processor called Auratorium. The software system was designed for 3D audio and virtual reality applications. It uses a ray-tracing approach to calculate physically correct and 100% natural sound.
PRG Lab, a division of PRG XL Video, based in Germany, develops customized interactive and leading software solutions for the event and exhibition market. PRG Lab supported the project with a 360° / 3D Nokia OZO camera system and capture a 360° spherical video of chosen acts at the festival. The cooperation between audioborn and PRG Lab was one of the first commercial uses of the OZO system in a production of this size. In addition to the standard multitrack recordings, audioborn captured the sound in the form of a 3D recording, using the Sennheiser VR microphone prototype, to reproduce highly realistic 3D audio for the recordings.
“The methods we used to capture sound – which is the most important part of Montreux’s Jazz Festival – will be pioneering for 3D audio sound recording as well as reproduction. Using our 3D software audio processor Auratorium, we will be able to achieve the highest possible degree of immersion for the jazz archive.” says Dr. Dirk Schröder, CEO of audioborn. “We had to determine how each instrument and sound would react depending on the position of the camera in the room. When sound and video are combined for VR playback, the virtual user should feel as if he or she is actually at the festival in person. Our goal is a total immersive playback experience in sound and vision. We are proud to work closely together with PRG Lab as our professional partner for 3D video capturing.”
“This project is outstanding in its dimensions. We recorded a huge amount of data during the 16 festival days with the OZO system. This was the most demanding challenge we had to face during the project.” commented Michael Ochs of PRG Lab. “This was very exciting, state-of-the-art technology being used at the Montreux Jazz Festival. It was the first project of its kind and we could not draw from previous experiences. ”
One of the challenges of this project was the large amount of data collected through 3D recording. A concert of three hours created approximately 2 TB of compressed data. Uncompressed it multiplies later in postproduction by factor 12.
PRG Lab is continuously expanding its 3D capabilities and will be using OZO production computers as well as the audioborn Auratorium software to create immersive experiences. The audio mixes are being rendered for multichannel 3D audio systems or binaural headphone reproduction.