Coldplay mixed reality promo at ARRI Stage London combines best of two worlds

British rock band Coldplay delivered a spectacular show in early 2022 as part of the delayed Dubai Expo2020. The mixed reality promotional film was shot at ARRI Stage London two months earlier, featuring a track from the band's Music of the Spheres album. To date, the clip has had more than 1.3 million views across the Expo's social channels.

Production company NorthHouse wanted to create a vast cosmic environment that simulated its upcoming projection-mapped performance with the band within a shimmering Al Wasl dome. The brief was to make them appear as if they were in this environment, which hadnt yet been fully finished and fully created, says DP James Medcraft. The client initially wanted a green screen shoot, which I felt wasnt going to provide quality of lighting on the band. You'd spend a lot of time closing down your set with black flags to hide green screen spill.

The project was well suited to Medcraft, whose varied background takes in design, 3D animation, photography and cinematography; his credits include commercials for global brands such as Toyota and Nissan, and music promos for Becky Hill and Ellie Goulding.

Copyright Will Case, Creative Technology

The promo brief encompassed a science-fiction concept, describing planets and stars moving through a stylized Al Wasl dome.

The production chose ARRI Stage London for the shoot. Opened in 2021 and equipped with an LED volume created in partnership with Creative Technology, the stage is amongst the largest permanent facilities of its kind in Europe. Constructed from ROE Visual LED screens, the wraparound volume comprises a 30 m x 5 m main curved wall, an 18 x 4.2 m rear curved wall, plus a dynamic motorized ceiling. The main wall is made of four 4K screens driven by an 8K processor, while 3D backgrounds are rendered in real time by Unreal Engine.

Most green screen studios wouldn't have allowed for the kind of camera moves required by the production, Medcraft continues. ARRI Stage London's size was not only large enough for what the production needed, but also meant we could embed the band into the environment, using the lighting to shoot all in one go. We were able to take a hybrid approach, blending the benefits of virtual production with green screen.

Medcraft believes this is an untapped use of this technology. Basically, we projected a green window wherever the camera was facing but used the low-resolution content as an ambient lighting plate, giving the best of both worlds.

Copyright Alice Backham

A hybrid approach blended the benefits of virtual production with green screen.

Vicon camera tracking, delivered by Audiomotion, and FIZ (focus/iris/zoom) data relayed by ARRI UMC-4s allowed the system to drive a frustum for each camera that was rendered green-the area of the LED screens seen on camera at any given moment. The remaining screen area rendered the virtual environment of the Al Wasl dome in real-time, providing highly authentic interactive light on the foreground talent.

The emissive green provided by the LED walls was easy to key out in postproduction, and because it filled only the minimum area needed to cover the frame, there was no green spill on the talent. Instead, the band were lit realistically by the rough virtual backgrounds displayed on the rest of the volume. It meant we could shoot earlier, light perfectly for the postproduction Unreal scene, and then drop the final virtual scene in at the last minute, so it helped every stage of the process, says Medcraft.

The production shot on two ARRI AMIRA cameras outfitted with Cooke S5 lenses. The A' camera was mounted on a Technocrane, with an extra camera for close-ups, picking up detail. It was important to create a great sense of scale with sweeping, wide camera shots, explains Tom Bairstow, creative director at NorthHouse. But we also wanted to get in close to the band to really feel their performance.

As well as driving the green frustums, tracking data from the Vicon crowns and UMC-4s was fed to separate servers providing simulcam -live images which were chromakeyed in real-time with the virtual backgrounds, showing the crew an approximation of the final shots. Camera position, FIZ data and timecode were also recorded in Unreal Engine from both cameras, making for a more efficient postproduction.

I think selling reality is all about subtle details that you dont concentrate on in the frame, says Medcraft. Its all about subtle reflections in the eyes and reflective objects like the cymbals, and also for the band-it gives a better experience of being within an environment and therefore a better performance.

Watch Coldplay's mixed reality promo clip, shot at ARRI Stage London

The stage also boasts a ring of 50 ARRI Orbiter LED fixtures, which can be controlled by Unreal Engine to complement the light from the video walls. We used some of those to bring out some animated highlights in the skin, Medcraft notes. Its really visible in Chris [Martin]'s eyes.

NorthHouse utilized Creative Technology's VFX suite to make adjustments to the virtual backgrounds whilst connected to the volume. Medcraft and the crew spent a day and a half rehearsing in the volume before Coldplay arrived. The band performed on a turntable and this, combined with the ease of flipping the virtual backgrounds on the screens, meant that reverse shots were achievable within the tight timeframe.

The DP enjoyed his experience at ARRI Stage London. Youve got all the kit available and expertise in all the different departments; you've got the best people running the screens, the Unreal team, the camera department support in Rental, you've got all of the lighting experience here. It does feel like a completely welcoming, supported environment.

Opening photo: Alice Backham

Related Products ARRI So

LINK:	https://www.arri.com/news-en/mixed-reality-coldplay-promo-arri-stage-l...
	See more stories from arri

More from ARRI