audioborn and PRG Lab create VR experience for the archives of the Montreux Jazz Festival . . .
Switzerland - Over the past 50 years, the Montreux Jazz Festival has seen some of the greatest acts in music history, including Nina Simone, Ray Charles, Miles Davis, Prince and David Bowie - and many others. This year’s 50th anniversary festival was host to a brand new technical experiment developed by Cologne-based audioborn, a specialist in room acoustics simulation and 3D audio reproduction. audioborn has developed a project which is allowing the Montreux Jazz Festival to be captured through 360° / 3D video and sound recording technology for the first time. This content can later be rendered and used in virtual reality applications, permitting users to experience the selected acts “in real-life quality”. Commissioned by the Metamedia Center of the University École Polytechnique Fédérale de Lausanne (EPFL) together with the Montreux Jazz Festival, project lead and official technology partner audioborn is partnering with PRG Lab for the video expertise. audioborn is responsible for capturing and reproducing all concerts in 3D audio, using its software real-time processor called Auratorium. The software system, designed for 3D audio and virtual reality applications, uses a ray-tracing approach to calculate physically correct and 100% natural sound, the company says.
PRG Lab, a division of PRG XL Video, also based in Germany, develops customised interactive and leading software solutions for the event and exhibition market. PRG Lab is supporting the project with a 360° / 3D Nokia OZO camera system to capture a 360° spherical video of chosen acts at the festival. The cooperation between audioborn and PRG Lab is one of the first commercial uses of the OZO system in a production of this size. In addition to the standard multitrack recordings, audioborn will capture the sound in the form of a 3D recording, using the Sennheiser VR microphone prototype, and will be able to reproduce highly realistic 3D audio for the recordings. “The methods we use to capture sound - which is the most important part of Montreux’s Jazz Festival - will be pioneering for 3D audio sound recording as well as reproduction. Using our 3D software audio processor Auratorium, we will be able to achieve the highest possible degree of immersion for the jazz archive,” says Dr. Dirk Schröder, CEO of audioborn. “We will have to determine how each instrument and sound will react depending on th position of the camera in the room. When sound and video are combined for VR playback, the virtual user should feel as if he or she is actually at the festival in person. Our goal is a total immersive playback experience in sound and vision. We are proud to work closely together with PRG Lab as our professional partner for 3D video capturing.”
“This project is outstanding in its dimensions. We recorded a huge amount of data during the 16 days of the festival with the OZO system. This is the most demanding challenge we have to face during the project,” commented Michael Ochs of PRG Lab. “This is very exciting, state-of-the-art technology being used at the Montreux Jazz Festival. It is the first project of its kind and we can’t draw from previous experiences.” As Ochs comments, dealing with the very large amount of data collected through 3D recording is a major challenge. For example, recording a three-hour-concert creates approximately 2TB of compressed data. Uncompressed it multiplies later in post-production by factor 12. PRG Lab is continuously expanding its 3D capabilities and will be using OZO production computers as well as the audioborn Auratorium software to create immersive experiences. The audio mixes can be rendered for multichannel 3D audio systems or binaural headphone reproduction.
Setting up the system is relatively straightforward, as the Nokia OZO is quite a low profile unit: including its mount, it measures 264 x 170 x 238mm and weighs just 4.2kg. The team simply has to find a safe and secure position from which to record the show, where it doesn’t disturb sight lines. The suitability of the systemalso depends to some extent on the type of act being recorded. Classical music is relatively easy, as long as no sightlines are blocked and the artists arehappy with the presence of the VR recording hardware. But with acts where there is a lot of movement around the stage or a lot of bass, or both, the recording can be disturbed by vibrations. But audioborn has many other applications in mind, including corporate events, nature filming, classical music productions in concert halls or churches, sport events, theatre shows and more. From the recordings made this summer, three titles will be produced for the Montreux Jazz Festival’s YouTube channel. “There will be definitely be a 2D 360˚ version with spatial audio, and we may also upload a 3D 360˚ video with spatial audio,” say audioborn.
(We thank LSi Online for the permission to share this article. Copyright September, 2016 by LSi Online - PLASA Media Ltd.. Please visit LSi Online for further information. You can read the whole LSi issue when registering for the LSi Digital Issue).