Camera Automation For Live Orchestra Performances

The challenges of shooting orchestra

Orchestral performances are large and complicated to produce and often rely on public funding to cover the cost. Bringing these performances to a wider audience is key in their viability. However, to live stream the concert in a visually interesting way requires multiple cameras and operators, as well as the director and production team. This, in turn, makes the performances even more costly and is therefore difficult to justify.

The use of robotic pan-tilt-zoom (PTZ) cameras has helped to reduce these production costs, as one operator can control multiple cameras. These cameras are able to save up to 40 presets, which allows a decent range of framing options. They don’t, however, have the same level of maneuverability as a tracking camera on a jib or a crane, so more cameras are often required.

Even with the PTZ cameras, a number of experienced operators are needed to manually recall the preset shots at the right time in the score. Due to the length of the pieces, this can be an intense and stressful task. The shot list has to be carefully planned in advance, and as rehearsal time in the venue is limited, this is often based on previous recordings or the score itself.

A new way to cue

There is already software available that is designed to simplify the multi-camera live performance, by pre-planning the entire shot list for the operators. CuePilot provides the software used by the Eurovision Song Contest to cover its momentous live performances. It allows a complete shot list to be generated from rehearsal performances. These are then fed to the camera operators for them to follow and make the switches between cameras and crops. The software can also be used to control the vision switcher, to move between different feeds automatically.

The pre-production work can encompass video roughly shot on any device, audio clips, lyrics, and scores. The vision cues for each camera and crop can be added to the timeline to create a full run-down. The camera operators receive a digital cue card in the provided app, which not only gives the camera and preset information, but also a countdown for the length of time each shot lasts for.

The British Broadcasting Company (BBC) worked with CuePilot to develop a system that would work for its live broadcasts of orchestral performances. For its purpose, support was added for PTZ cameras to the CuePilot system, to allow these presets to be controlled by the software.

The arrangement

An initial test used a total of seven feeds into the vision switcher. Four PTZ cameras fed into a single control panel and two static cameras covering a wide view of the whole orchestra and a shot of the conductor. It also used one camera on a tripod at the back of the hall with a long lens, with a dedicated operator. Both the PTZ operator and the dedicated camera operator accessed the software app on tablets. This set up was upgraded to six PTZ cameras, losing one of the static cameras, for a total of eight cameras for subsequent tests.

To prepare the shot list in advance, the director marked up the score as normal. The difficulty came when using a commercial recording of the piece to mark down the shot. Though the system is designed to work from a master tempo with beat markers, the tempo variations in the classical scores made this difficult implement. Instead, they experimented with matching the number of bars and simply using four-second gaps.  

The system now allows you to drop in shots on the fly, when playing back a recording. This allows the director more artistic freedom to cut by sight – or by ear.

Keeping in time

With the timings on the cue cards, the camera operators were too focused on keeping to the timings rather than the shot being used. This was solved by having the operator press a button to advance to the next cue. This allowed them to jump earlier or later if required while not upsetting the overall timeline.

If a particular shot or camera became unusable, there needed to be a way to manually override the upcoming cue. This was also added in, allowing the operator to select another available camera from the corresponding keypad number.

They also discovered that having the director responsible for the CuePilot software and advancing the shots, meant that they became too focused on the score. Even when there was a dedicated crew member calling out the shot numbers. A better option was to have the crew member following the score make the advances in shots and let the director have a second keypad, simply for artistic changes and unplanned changes.

Scaling up

While the BBC’s use of the CuePilot software was still very manual compared to what it is capable of, it did allow the potential for the operation to be scaled up to include a more complex camera set-up without the need for extra staff 

Its production arrangement still allowed for artist direction by an experienced director in addition to the vision switcher operator, when in fact the basic operation could be followed by a single less experienced crew member. It is in the pre-production where the experience is required, to plan the shots in advance.

With further development, the production during the live recording could be completely automated with just a single crew member available in case of failed cameras or changes to the performance. If the pre-production process could be adapted to read from the score, rather than just the recording, it could be possible to automate much of the initial work as well.

This technology has shown ways to enhance the visual output of live orchestral performances without increasing the budget. It also offers the potential to deliver coverage of events in a more basic way that would otherwise not be covered. Something that, in turn, means more people will have access to performing art, and that’s a good thing.