Detail in 4K imaging

Pierre Routhier, Vice-President of 3D strategy for Technicolor spoke about "4K: Model for motion control to ensure true 4K detail at capture." One critical component for maintaining image detail in 4K imagery is motion blur, dictated by optical flow (motion as seen by the sensor) and shutter speed (to ensure smoothness of motion). Routhier pointed out that resolution is merely a container for detail. "Let's stop talking just about resolution,"he said. "The detail cannot exceed resolution but detail can be inferior to the working resolution"

Image detail is actually quite complex, Routhier noted. He showed a graph of factors affecting image detail. "A good cinematographer understands all these issues and knows how to make the best choices to get the best image," he said. "A lot of these factors being contradictory, he or she has to make the best compromises to get that best possible image." He showed two side-by-side images where both are identical; the only difference was frame rate. The image on the left looks like it has higher resolution but its resolution is actually less; it has a high temporal rate. "What we've observed is that temporal resolution trumps spatial resolution," he said.

He addressed the issue of the depth of field - as sensors get bigger and bigger and you can't light your set like a Christmas tree, the iris won't be fully closed and you won't have the depth of field you want," he said. Noise is the third characteristic. "The idea is that the sensor has little buckets to capture photon," he said. "What I'm seeing in the room is the reflection of light on different features that bounce back to me. The camera works in the same way and if not enough photons reach the sensor, it won't know what's there."

The lessons learned thus far is that getting good images in 4K is more than trading in a 2K camera for a 4K one. "2160p30 is inferior to 720p60 and does not show more detail than 1080p30," he said. "2160p60 is superior to 720p60 but only marginally better than 1080p60. And 1080p120 is superior to 2160p60. The Holy Grail for sports is 120; that's when you start to get interesting results."

Motion is critical to detail for 4K cinema. "At 24 and 48 fps, conventional shutter speeds are insufficient to prevent motion blur in a vast majority of shots," he said. "On the movie set, the cinematographer can calculate depth of field; now he needs to calculate motion as well. In order to obtain detail, it is imperative that we manage and control motion."

In his paper, he said, he created two simple models to analyze types of motion. He showed an image with a full sensor and 50mm lens at 25 feet, scaled to 4K. He then showed the effective level of detail noting that it decreases dramatically with speed. "Motion cannot be higher than one-tenth of normal walking speed. If not, you get in the territory of SD or the lowest resolution at which you would not show blur." The formula for rotational motion is approximately the same and dependent on the focal length used; you need very short focals to have wide images at high-resolution, he says. A 90-Degree Pan at 50mm, if I want a high level of detail, that takes almost two minutes. "That's the kind of time you need," he said. "It's very slow. Basically you're not moving the camera. Any time you pan, you lose detail." 

With the research team, they built a detail analyzer that makes it easy to understand for "right-brain" creative people. They start with an uncompressed source, and the resolution at which they want to display it. They down-scale to a benchmark resolution (for example, 4K to 2K and then scale it up without adding edge enhancement. "Just blowing it up to put it in the 4K container," he said. "Then we chop it up and present it in blocks, analyzing it in ways that people can see and understand." He showed videos that made it very clear how a cinematographer or director would see the loss of detail in an image. "The minute you have motion, you lose detail," he said.

"I work on a lot of movies and unfortunately people are disappointed," he said. "They think they've got the best camera, but there isn't always compatibility between what you need to do to get best detail and what you need to do to create a movie. And motion destroys detail." Recommendations to maximize detail is to manage motion as carefully as possible. "Shoot with wider lenses and from farther away and limit camera and subject motion to a minimum," he said. To maximize cost effectiveness in post, analyze the detail before determining post workflow for a shot. "And that's especially true for 2D/3D conversion and VFX," he adds. "Think about the level of detail before you begin processing." In answer to a question, Routhier reported that they are doing tests on variable frame rate.