Broadcast Facilities: Scalable IP-Based Infrastructures
Hot Button Discussion
by Michael Goldman
New methods for broadcast facilities to distribute video and audio data and metadata around their studios have been evolving for some time. Indeed, a revolution is currently underway in this arena thanks to the use of IP-based networks for professional AV media. This development has radically changed just about everything, and as discussed in a 2018 issue of Newswatch, it’s been radically enabled by the move away from SDI and toward incorporating the SMPTE ST 2110 suite of standards for Professional Media Over Managed IP networks.
ST 2110, after all, has allowed entities to send video, audio, and metadata as separate essences over networks using Ethernet or fiber cables linked to switches. This approach has greatly increased flexibility and efficiency across the industry as bi-directional signals have generally allowed broadcasters to reduce cabling, among other improvements. The new standard also relies on the IEEE 1588 Precision Time Protocol (PTP), permitting particularly fine resolution in terms of making sure all signals can efficiently be synched up at the receiving end.
These are all major steps forward. However, Rob Porter, Project Manager for the Advanced Technology Team of Sony’s European Professional Engineering Group and someone who has been deeply involved in working on these issues, emphasizes that the move toward an IP-based foundation for broadcast facilities involves “lots more than simply replacing one kind of cable with another. The idea is to route signals seamlessly from one piece of equipment to another, and now, we are able to use standard IT equipment for this—things like network switches or commercial off-the-shelf [COTS] switches. Those are the kind of switches they now use in major data centers. It’s becoming much cheaper to do it with general purpose IT equipment, rather than specialized broadcast equipment.”
Nevertheless, the goal of a single, interoperable IP-based methodology for professional media work has other hurdles to clear, as there are a number of additional issues facing the industry, including the issue of proving how reliable large-scale broadcast facility networks can be over the long term. Fortunately, according to Porter, major strides have been made recently in this regard, particularly in terms of work done by the Advanced Media Workflow Association (AMWA) to develop Networked Media Open Specifications (NMOS).
“ST 2110 is a very good thing, because it is getting all of us working together and interoperating,” Porter says. “But there are still some proprietary protocols out there. So the other thing we have tried to improve for interoperability is the control layer. How do you discover a new device when you plug it into the network, and make other devices able to subscribe to its video stream, and get it onto your monitor or into your switcher?
“This is where the AMWA specifications come in. As part of them, we have developed and published two interface specifications—IS-04 for discovery and registration of NMOS nodes, and IS-05 for connection management between nodes. The benefit of these protocols is that they are completely open, so everyone can access them, everyone can write software to make sure that their devices use these protocols. So ultimately, as soon as you plug something into your network, other things on the network should be able to find them through the Registry and be able to connect to them.”
Porter emphasizes that the AMWA strategy is essentially “a software development approach. We can have things go up on the GitHub [open-source development] repository system to represent the software interfaces we are using. We are using RESTful APIs for IS-04 and IS-05 to facilitate communication between devices, for instance. It’s basically the same kind of commands that you talk to Web servers with—HTTP POST, PATCH, GET, DELETE, and so on, making it easy to develop and for everyone to use.”
One industry challenge on an IP-landscape, naturally, is security. But Porter emphasizes that the IT concept of repeatability has largely insured that large, robust broadcast facilities can typically make sure data is never lost altogether
“Even within the studio, we are talking about redundancy,” he says. “Every signal has two different paths that it goes down. If you lose IP packets on one path, you get them over the other path. This is the part of ST 2022 known as ST 2022-7—the method of having two separate paths, and then recombining any lost packets on the receiving end.
“And then, in terms of making sure data is secure and no one can intercept it, certainly on the API side, since these are Web-based API’s we are using—we can largely use standard Web-based technology to make things secure. And we have Best Common Practice documents defining how to do that. Here, we largely rely on standard IT methods—we aren’t trying to reinvent the wheel.”
A more pressing challenge has involved making sure APIs are scalable across large facilities and networks. To address this issue, according to Porter, Sony recently led an AMWA NMOS Scalability Study, authored by Porter and his Sony Europe colleague Gareth Sylvester-Bradley, to test literally thousands of media devices across a massive virtual network. According to the study, these tests included “confirming that operations such as registration of thousands of Nodes can occur within an acceptable timeframe; testing recovery of a Registry after a failure; testing behavior with multiple clustered Registries; testing different methods of Registry discovery; testing connection management at scale; and confirming behavior for architectures with redundant network interfaces.”
According to Porter, “the idea was to provide some reassurance to the industry that you could have thousands of devices attached to a network, and they could all talk to each other without overloading the network or overloading each other. We basically proved that running things on a virtualized network—you could have thousands of Nodes registering within a couple of minutes so that your broadcast center can get up and online pretty quickly. We also proved other things, like resilience to failure—if a Registry goes down, another one can take over. It was a virtualized network because we couldn’t hook up thousands and thousands of end points to a real switch system, and we were only really testing the software APIs at that stage anyway.”
The AMWA study was presented at the SMPTE Technical Conference last fall, and this month’s SMPTE Motion Imaging Journal examines the study in more depth. However, Porter points out that the AMWA NMOS initiative has now begun additional work on new IS specifications that are currently in earlier stages of development. Among them is IS-06, an API that permits control of a network by broadcast controllers; IS-07, which Porter says is called “an event and tally specification,” meaning a mechanism that defines permitted event types along a network, permitted transport types, and an API definition that allows a network to adjust to changes in event states; and finally, IS-08, a protocol for remapping audio channels.
"These are all slightly more embryonic at this stage, but we recognize that we can use the same infrastructure to do things we used to do in different ways,” Porter explains. “In particular, it’s been recognized that having a way of actually managing flows through a network can be very valuable, and having an open way of doing that, authorizing devices on a network and managing bandwidth—that can be extremely beneficial. So that’s what we are trying to do through the IS-06 specification, for instance.”
Porter emphasizes that IP-based workflows have benefits beyond the infrastructure within the studio or OB truck, and are also a key enabler for remote production. “One of the advantages of going to an IT infrastructure is that your equipment can be anywhere,” he says. “You could have a remote production with your studio in one place and all the cameras somewhere halfway across the country. Using a low latency IT infrastructure, the operations can still be carried out as if the equipment were in the same place.”
He says that Sony, for instance, has embraced the interoperability that the open specifications of ST 2110 and AMWA NMOS bring to the industry. He adds that ST 2110 is already well supported in their lineup of live production products, and NMOS IS-04 and IS-05 are fully supported in the company’s system camera chains and broadcast controller, and he fully expects the rest of the industry to pursue a similar course.
Indeed, Porter explains that as well as placing this support into their own products, Sony has contributed to the widespread adoption of the NMOS specifications generally by offering their implementation as open source software on GitHub. This includes implementations of a fully functional Registry and the Node software required for end devices.
“We developed nmos-cpp as an implementation of the NMOS specifications that are completely free to use by anyone,” Porter says. “A major concern for broadcasters is that vendors are not adopting the specifications as quickly as they might be, so the aim of releasing our software as open-source is to allow others to quickly adopt NMOS specifications and allow true multi-vendor interoperability.”
To ensure that vendors’ equipment interoperates successfully, the industry regularly holds interoperability events or “plugfests” where vendors can test their devices ability to successfully receive video, audio, and other data from other vendors’ device's, as well as integrate with the NMOS APIs. To develop more industry confidence going into these events, AMWA now offers a set of online test suites to which Sony and others have contributed.
“Because the NMOS specifications are software-based APIs, much of the functionality can be tested with Web-based test suites,” he says. “These are freely available on GitHub, and allow developers to download the test software and run a fairly exhaustive series of tests against their implementations. This gives implementers confidence when joining interoperability events, and should be another key factor in accelerating the adoption of these open specifications throughout the industry.”
Microsoft Invests Big in OpenAI Initiative
A recent piece in the New York Times reports that a major funding step has been taken to potentially facilitate the pursuit of taking artificial intelligence (AI) technology to an entirely new level—theoretically, the development of an Artificial General Intelligence (AGI) machine capable of replicating many functions of the human brain. The report says that the OpenAI lab initiative founded by Silicon Valley tech whiz Sam Altman and Elon Musk has received a billion dollar investment from Microsoft to help it pursue such goals. OpenAI is now run by Altman, who is transforming it into a for-profit company capable of luring such financing, in order to “create broadly beneficial AGI,” Altman states in the article. The project is of great interest to the technology community, with similar projects like Google’s DeepMind pursuing a similar agenda. The article quotes Altman and others as admitting the goal of creating a machine that thinks like humans do could take decades, or even centuries, but suggests the fundamental concept behind their agenda is reasonable in the AI world. That concept revolves around the notion of using AI reinforcement learning techniques to eventually teach a machine to make human-like decisions by building enough super-computing power to process reams of data that describes “everything humans deal with on a daily basis” and then analyzing that data efficiently to result in human-like behavioral decisions.
Restoring Moon Footage
With all the recent coverage of the 50th anniversary of the Apollo 11 moon landing, a recent article in The Hollywood Reporter notes that Hollywood played a key role in assisting NASA in restoring imagery from the historic mission for the world to enjoy. The article reports that NASA failed to properly archive all the images that were recorded on the day of the moon landing, and that in 2006, the agency launched an extensive search for masters of much of that imagery—a search that largely failed, as the agency admitted in 2009. Therefore, NASA began an Apollo 11 restoration initiative that relied on locating the best available live broadcast footage of the moon landing and moonwalk from several different sources, and then turning to Burbank-based Lowry Digital, a well-known restoration company for major Hollywood films, to bring that material to life as clear HD imagery. The “Lowry Process” of mathematically tracking consecutive frames of the images to clean it up allowed the company to transform some of the worst quality footage it has ever worked with into pristine HD masters.
The Computing Moonshot
Speaking of the moon landing anniversary, an interesting recent column on the Display Daily site reminds us of what a historical computing achievement the event was considering the fact that the two portable 1960s-era computers that guided the Apollo 11 capsule to the moon and back again possessed last than a half-percent of the computing power of a typical smartphone today: a 1 MHz clock, four 16-bit registers, 4K of RAM, and 32K ROM. Among the legacies of this work, the column says, was the fact that it helped revolutionize the miniaturization of sophisticated electronics leading to today’s digital revolution. Among other things, the NASA rocket computing systems led to the development of the Texas Instruments integrated circuit based on research by computer engineering pioneers Jack Kilby and Robert Noyce, the co-founder of Fairchild Semiconductor and later, the Intel Corporation.