Creating the Cinema Look in HDR with a New Approach to Frame Rates

In their paper, “Achieving Cinematic Motion with High-Dynamic Range,” published in the August’s SMPTE Motion Imaging Journal https://www.smpte.org/motion-imaging-journal, a team from Dolby’s Applied Vision Science Group investigates how viewers compare the smoothness of scenes with varying frame rates, color gamut, luminance levels and motion types. Building on existing research, they present their own psychophysical experiment, examining how these variables can help content creators craft their desired look.

Smooth, realistic motion with lack of judder is one of the distinguishing factors between cinematic content and live sports and gaming. Yet it is clear that higher brightness and contrast in a large screen format can negatively alter viewers’ perception of this motion. In response, some content creators may adopt slower camera motion rates or limit HDR’s expanded potential, which can compromise the creative vision. Or, they may adopt a higher frame rate for the entire production, which can expand production costs. The paper’s authors designed an experiment to answer the question, “What if creators selected the most appropriate frame rate for each scene based on its type of motion?”

Past experiments tended to focus on viewers’ personal preferences for images at various frame rates. However, the Dolby team utilized real content produced with conventional cinematographic compositing, lighting, cameras, lenses and color correction, and designed their experiment to ask viewers to compare the motion smoothness not against their personal preferences, but against a cinematic reference at 24 fps frame rate and 48 cd/m2 peak luminance. The 14 test sequences included a wide variety of motion types and speeds for both camera and object, and varied greatly in contrast, color and brightness. To test multiple variables, these test sequences used frame rates of 30 fps and 40 fps, and a peak luminance of 300 cd/m2.

The 16 test subjects of differing ages, roughly half of them using corrective lenses, participated from their homes due to COVID-19 restrictions. They were all experienced in video quality analysis, and focused their trained eyes on LG monitors using a custom Dolby Vision Cinema Mode that bypassed nearly all tone-mapping and post-processing in the television to approximate the consistency of a reference monitor. After two viewings of each reference video and then two of its corresponding test sequence, the participants each compared the smoothness of the test video on a 7-point scale from “Much More Smooth” to “Much Less Smooth” than the reference video.

Their resulting analysis shows that perception of motion smoothness in HDR is affected both by frame rate and peak luminance, and depends greatly on the video’s motion characteristics. While the motion smoothness was actually independent of the motion of the objects on screen, there was a strong correlation with the motion of the camera itself. In brighter scenes with fast camera pans, increasing frame rates dramatically boosted the smoothness rating. Some darker scenes with no camera motion did not require a frame rate change with increased peak luminance, and in fact scored as a worse match with increasing frame rate. The increased frame rate on scenes that don’t require it created a motion that was too smooth and felt fake to the test viewers.

Ultimately, the Dolby team conclude that content creators using the expanded palette of color and light offered by HDR can achieve their desired cinematic look with scene-dependent frame rates. They note, “For an HDR production, a variable frame rate that depends on the characteristics of the scene will be needed to maintain the appearance of motion that content creators and viewers have grown accustomed to. For dark, low motion scenes we anticipate little difference between 24, 30, and 40 fps. As the peak luminance and camera motion increase, there is a higher sensitivity to small frame rate adjustments and the perceptual smoothness difference between 24/30/40 is much greater.”

Read the complete article for all the details about the Dolby team’s work:https://www.smpte.org/motion-imaging-journal