On November 17, 2024, as part of the HEAT project, ATRAE, UTBV, UNICA, and ORO collaborated to organize the first version of the XR-Blues Pilot at the Rockstadt Club in Brașov, Romania. This pre-pilot event laid the foundation for an innovative fusion of extended reality (XR) and live music experiences.
The pre-pilot revolved around a blues concert organized by ATRAE, featuring renowned Brazilian blues guitarist Fred Sunwalk and his band. This event provided an ideal setting to test and refine the technological components necessary for the XR-Blues Pilot.
During the soundcheck, the HEAT team recorded the three band members using six RGB-D LiDAR cameras, with two cameras dedicated to each musician. Alongside the visual capture, a multitrack audio recording was taken directly from the stage audio mixer. During the concert itself, additional recordings were conducted to build a comprehensive dataset for further pilot development.
ORO undertook an evaluation of the network capacity at Rockstadt Club to assess its suitability for future XR applications. This included measuring key 5G network parameters such as maximum throughput, latency, ping, and streaming video quality, which are essential for delivering real-time immersive experiences.
On the day following the concert, the HEAT team utilized drone technology to map the interior of the Rockstadt Club. This process enabled the generation of a detailed 3D environment that will be used for the XR-Blues pre-pilot.
Following the event, efforts were focused on processing the collected data. This included:
- Developing a VR representation of the Rockstadt Club.
- Creating avatars for the stage performers.
- Mapping the recorded audio to specific stage coordinates to enhance spatial sound realism.
The XR Blues Pre-Pilot was a crucial step toward merging live music performances with immersive XR technology. The insights gained from this event will contribute to the refinement of the full XR-Blues Pilot, bringing audiences closer to a new era of musical engagement through cutting-edge virtual experiences.
