HDR Video Part 4: Shooting for HDR

To kick off our new weekly blog here on mysterybox.us, we’ve decided to publish five posts back-to-back on the subject of HDR video.  This is Part 4: Shooting for HDR.

The first HDR project I graded was a set space shuttle launch shots, filmed on the RED ONE camera by NASA.  The footage wasn’t filmed with HDR in mind.  In fact, HDR wasn’t anything close to ‘a thing’: the shuttle last flew in 2011, and Dolby didn’t present their proposition of “Perceptual Signal Coding for More Efficient Usage of Bit Codes” (what we now call PQ), until 2012.  And yet despite the age of the footage and the lack of consideration for HDR when it was filmed, I still had no problem grading the footage into HDR space, and getting a pretty awesome image out of it.

HDR Grade from NASA Archive Footage, shot on Red One, circa 2011

I bring this up simply as a point of perspective; while I’m going to offer some suggestions here to help make your footage better in HDR, it’s important to realize that, in general, all footage is better in HDR, regardless of its age or how it was filmed.  That being said, there are things you can do while filming to best prepare for an HDR finish, which is what we’re going to discuss here.


The Kit

Choosing a digital camera today is like how choosing film stock used to be - each one responds differently than others, and can create slightly different looks.  This doesn’t change when you’re shooting HDR.  Beyond a specific point, your camera choice doesn’t matter; camera choice is a creative (or budgetary) decision.

But there are a few features that are important, if not essential, when planning an HDR shoot.  Think of these as the minimum level of kit needed.  I’m going to outline those first.  Then, there are niceties you can add on that will make your life a little easier, and I’ll outline those next.

First and foremost for HDR recording: never, ever, EVER shoot with a standard Rec 709 / BT.1886 / Gamma 2.4 contrast curve.

It’s possible to grade footage that uses one, but the results are pretty poor.  There’s too much clipping of the darks and whites, and the loss of detail kills you.  Linear is okay if it’s a high enough bit depth; a LOG format is better, but native RAW is really the best.  LOG and RAW will preserve more of the detail through the darks while retaining better roll-off into the whites, that will make HDR grading easier / possible.

When you’re shooting with HDR mastery in mind, use the highest bit depth (and bit rate) available.  If you’re using a camera that stores its footage in a compressed 8 bit format, you’re going to do yourself a world of disservice when it comes to grading in HDR.  The same reason that all HDR formats require 10 bits minimum applies to the camera - 8 bits causes stepping.  If you’ve shot in a LOG format, it’s possible to get away with using 8 bit sources, but you can’t push the footage as far as it’s able to go, and are very likely to see stepping in the whites.

If an 8 bit camera is your only option, and you still want / need HDR, consider using an external ProRes, DNxHR, or other high bitrate, intraframe, 10b+ per channel recorder.  You’ll save yourself a world of hurt in post.

Of course, the ideal format to shoot in is a camera RAW format, like Cinema DNG, RED RAW, ARRI RAW, Sony RAW, Phantom RAW, etc.  You’ll love yourself in HDR post for using RAW, even if you typically prefer the turnaround speed offered by a ProRes or DNxHR workflow.  Here’s why:

  1. Most ProRes and DNxHR workflows normalize the RAW footage into a LOG format, which is fine, but they collapse the bit depth range used to 10 bits. With RAWs you typically have access to the full 12, 14 or 16 bits offered by the sensor! Your grading application typically uses even higher bit depth internals for color processing, so even if you’re only grading in 10 bit at the display, having the extra 2, 4, or 6 bits per channel, per pixel, is a major advantage in grading latitude, something I can’t emphasize enough as being important to HDR.

  2. ProRes and DNxHR workflows typically normalize the camera primaries into Rec. 709 color space. Most professional cameras use primaries that are wider than Rec. 709, but the video signals they output are typically conformed to Rec. 709 for ‘no brains’ compatibility, that is, it’ll just work. While this isn’t a problem per-se for HDR, it does restrict the volumes of colors available for grading, and will require a LUT or manual shifting of the primaries in grading to match HDR’s BT.2020 or DCI-P3 space.

    Typically, RAW formats record their data using the camera native RGB values, and your RAW interpreter in your color grading program can then renormalize them to whatever target space you’re looking for. Since BT.2020 is the widest of all display spaces, you’ll be able to better reproduce what your camera is already capturing.

  3. RAW formats often provide highlight and lowlight recovery not available in fixed video formats, even when using LOG or linear recording. The RAW formats here give you access to as much information as the sensor actually recorded, which is invaluable in post. Because of the extended dynamic range of the HDR environment, you’ll want as much of the highlights as you possibly can get, and may even at times push further into the noise floor, because the noise is less perceptible in the deep HDR darks.

If you’re shooting at the professional level, and using professional cinematography equipment, what you already have is probably okay for shooting in HDR.  Cameras like the RED Epic or Weapon (or pretty well all of RED’s cameras because of their RAW format), Sony’s F55 or F65, Arri’s Alexa, or any camera in the same class are perfect.  As are most film sources (S35mm or greater, for resolution), when captured with a high bit depth scanner.  Using any of these, you’ll be well suited for HDR, assuming you follow the format’s best shooting practices, which I’ll discuss in the technique section below.

If you’re using a prosumer or entry level professional camera, taking a few preparatory steps to set up how you’ll actually be capturing the image can mean the difference between getting footage that can be used in HDR mastering, vs. footage that can’t.

So in summary, when choosing your kit for HDR, consider:

  1. 16 bpc > 14 bpc > 12 bpc > 10 bpc > 8 bpc: 10 bpc should be your minimum spec.

  2. RAW > LOG > Linear > Gamma 2.4: avoid baking in your gamma at all costs!

  3. Camera Native / BT.2020 > DCI-P3 > Rec. 709 color primaries

  4. RAW > Compressed RAW > Intraframe compression (ProRes, DNxHR, AVC/HEVC Intra) > Interframe compressed (AVC/H.264, HEVC).

As a quick side note, some cameras offer a SMPTE ST.2084 signal or other HDR signal out of the camera for use with an external recorder.  These are a useful replacement to recording externally in either a LOG or gamma format - they can lead to faster turnaround times, but will require an HDR grade (or a dedicated step out of HDR) vs. being ready to be graded in HDR with the option of grading normally.


The Technique

First things first, some general, good advice: take some time learning how your camera and lenses respond to various lighting situations.  How is its roll-off into the highs?  What’s its noise level in the darks?  How does the color response change in different exposure levels?  While this is generally good practice, it’s the kind of forethought you really need in planning an HDR shoot.

When shooting for HDR mastery, you may find that you’ll need to modify your typical shooting technique.  There are three things that are important above all others: protecting your highlights, protecting your darks, and planning the expansion of the dynamic range of your scene.


Protecting Your Highlights

Most of us are rightfully excited about the creative possibilities that come with increased brightnesses at the display, and the expanded range of highlight detail that comes with it.  The catch is that there are some things that used to work well that, frankly, now look like shit.

Clipping.  May you rot in hell.

The large area to the right of the sun has no detail retention in the RAW.

All sensors clip at some level of exposure.  Film does too.  It’s unavoidable.  The goal for HDR shooting is to expose your whites to eliminating clipping the RAW data when possible, and minimizing it when it’s not.

Unlike traditional mastering workflows, where images clipped to white are simple to correct (set the clipped area to true white), clipping in HDR becomes problematic very quickly.  In HDR there is no longer such a thing as “true white”.  Instead, in the grading process (which we’ll discuss in Part 5), we make a creative decision about how bright white should be, and how to roll into it.  That roll into whatever white you pick is essential to tricking the eye to believe whatever you’ve picked to be white is, in fact, white.

The same shot can be graded with different white points in HDR, depending on the goals of the cinematographer & colorist. Both of these grades work with the snow reading as white; the lighter image feels brighter, while the darker image feels more oppressive and foreboding

The human visual system perceives any object within, or region of a scene, as colored a shade of white (that is, not as a shade of grey, but varying intensities of whites) so long as three conditions are met:

  1. The brightness level is above a specific threshold relative to the rest of the scene, which is usually around 100 nits

  2. The chromatic characteristics are relatively balanced (that is, low saturation)

  3. The area is not completely uniform in brightness level and juxtaposed with a scene (or part of the same frame) with a brighter or more natural roll-into the whites

When talking about clipping, it’s that third condition that ends up being a problem.  Clipped footage typically has large swatches of ‘white’ with an abrupt transition into the patch (once the rest of the footage is graded to a normalized brightness level).

Gentle rolls into clipped white areas appear more natural than abrupt transitions

Deciding what brightness level to place this ‘white’ at becomes problematic for a couple of reasons.

First, you have to limit the brightness of the white patch with respect to the rest of the scene - if it’s too much brighter than everything else (say, everything is under 100 nits and you put the patch at 1000 nits), without roll into the whites you have an obnoxiously bright patch that dominates and overwhelms the rest of the scene.

Second, because these patches typically have a large area, that is, make up a significant portion of the pixels used on the screen, they end up skewing the distribution of brightnesses when calculating the MaxFALL, meaning that everything else in the scene has to be significantly darker than you might like, or you have to bring down the brightness of the white to bring up the brightness of everything else.

The overall brightness around the sun limits the overall peak image brightness due to MaxFALL. For contrast, I've included both the direct SDR down grade (roll into white between 200 and 500 nits), and the same with the white point restored to full

Third, with the first two effects limiting the overall brightness of the uniform patch, it’s likely to appear grey when cut together with footage that has proper roll into the whites, since that footage is likely to have parts that are much brighter than whatever white you’re able to use for this clipped value.  The overall effect: grossness that pulls you out of the ‘magic’ that HDR creates.

In this sequence, the peak available white point of the middle shot is lower than the two shots that surround it, due to MaxFALL. In the final grade, the first and third shots were graded with lower peak whites to match

Stop down or use ND: protect your highlights and avoid clipping like the plague.

Some parts of your image, like the sun or bright lights for instance, may clip and that’s okay, so long as they don’t dominate your scene.  You can typically roll into these whites much more subtly than larger clipped areas.

Not all clipping is unnatural, even in HDR

White, puffy clouds also tend to want to clip on most cameras, but don’t let them, if possible.  Because of the frequency that most people see clouds, and see the details in them, you need to preserve as much of that as you can or risk your viewers looking into them and being jarred at the bright uniform shapes that come with clipping, vs. the gentle gradients that come with the more rounded textures.

In HDR the contrast in clouds is much more significant than in SDR, and the clipping in the clouds hurts the realism of the scene

Coupled with this, is the idea that you can’t assume that you’ll get to hide things on the other side of bright windows.  If you camera retains any detail through a window or doorway, it’ll probably be visible.  If you’re hiding crew or equipment through a blown out window, you’ll need to be doubly sure that the window will, in fact, be blown out in HDR. (The same, by the way, is true for your darks - don’t assume they’ll be crushed out.  More on that in a second).

If your monitor out from your camera allows for separate colorimetry than your recorded image signal, you may want to switch it to a LOG curve out so that you can see on your field display or eyepiece where the scene is clipping, if at all, and what details are visible in the brights.


Protecting Your Darks

While the brights tend to get the love when talking about HDR, personally, I love what HDR does for the darks.

Just like with the whites in the image, we have to get rid of the concept of ‘true black’ when discussing HDR.  Instead, we have a range of blacks, just like we have a range of whites.  Two of the three conditions we discussed above describing how the brain perceives whites is true for blacks as well: they need to be below a certain value threshold, and they can’t be large uniform areas juxtaposed with darker regions.  The only difference is that below the brightness threshold the brain typically stops perceiving chromatic value anyway: saturation doesn’t matter (unless you’re trying to supersaturate your darks?)

Just like with our whites, eventually sensors clip to black.  In most video signals, this will be a hard clip, but in many RAW formats (especially those that offer ISO adjustments in post production), the blacks are typically recoverable into the noise floor of the camera.

If you’re planning on using a PQ HDR mastery workflow, you’ll need to assume that most of these darks are in fact visible, beyond what you’d normally consider available.  Which means you may need to be concerned about overall exposure level for the detail in the darks - you can’t necessarily hide equipment there, and need to make sure your production design moves deep into the visible darks.

Details often lost in darks in SDR are often visible in HDR

Even worse, or better, depending on whether you’ve planned for it or not, even after the image is properly graded, areas that appear black with the full HDR grade can ‘open up’ to the eye just by obscuring with your hand the brightest regions of the image, just like how blocking the light from a spotlight pointed at you allows you to see behind the light source.

Simulated images showing details visible in the darks when you block the lights in HDR. This does not happen in SDR.

The good news is that noise is far less perceivable in the darkest depths of HDR than when the footage is normalized into SDR, largely because of the lack of saturation and our vision’s greater degree of tolerance to luminance noise than to chromatic noise.  So while it’s important to keep important details above the noise floor, it’s not as essential as protecting against clipping.

To the darks the same rules apply: open up or increase your ISO to avoid clipping in the darks.


Planning the Scene in HDR

You may have noticed the two pieces of contradictory advice from the last two considerations: stop down to protect your highlights, and open up to protect your darks: a paradox.  Something has to give: how do you plan for that?

Don’t worry: planning your scene for HDR is actually even more complicated than that.

When you shoot for HDR, you can’t assume that every consumer display will be HDR.  So you need to consider that how the darks and lights will play in both HDR and SDR.  With the whites, it’s relatively simple to adjust your roll into or clip so that it plays well in SDR, but crushing the blacks isn’t always the best option.  Creatively, you may want to highlight action or detail in the darks in a way that will be lost with a simple crush.

Crushing the darks in SDR maintains the mood of the HDR image, but at the expense of detail retention

A solution, of course, would be to bring up part of the darks during post, which increases the visible noise in SDR and may require a clipping or flattening of the whites to maintain the contrast and detail across the scene.

Noise is more perceptible in SDR darks than in HDR darks

Alternatively, you can adjust your lighting to bring up the darks and compress the range, then re-expand the range while color grading in HDR space.  So long as you’re shooting in a RAW format and capturing 12+ bits / channel, you won’t see stepping with this technique, since your mid gradients on a log curve are allocated sufficient bits that expansion is possible.

Another thing to consider when planning the scene is the MaxFALL limitation of HDR mastering.  The overall dynamic range of the scene needs to be planned in a way that the super bright / HDR elements are restricted to a small portion of the overall frame, so as to not push up the frame average light level.  Shooting interiors with a few bright windows or patches of direct sunlight tend to be fine, larger bay windows with cloudy or limited outdoor light also work so long as the eternal ambient isn’t too high (dusk / dawn, not noonday sun).

Both of these shots were done in the same space, about a year apart. The time of day plays an important role to how much the windows affect the MaxFALL of the scene, with the blown out windows limiting overall brightness.

Particularly problematic are blue skies.  Why?  Because blue skies often take up a much bigger part of a frame than you expect, and contribute more to the MaxFALL since our eyes perceive blue values of similar absolute brightnesses as darker than those of other colors.  What we see as mid range blues can suddenly push up MaxFall and limit your overall scene brightness while still looking ‘normal’ or ‘average’ to the eye.  Exposing for blue skies often means keeping the blueness in the traditional light level range, which can leave the rest of your brights muddied (especially when shooting into the sun).

The amount of the image take up by the blue sky limited the overall MaxFALL of this image. The result: in HDR, the sky never felt 'bright' like the trees or the tower.

Essentially, when designing your scene for HDR you need to plan the bulk of each frame to land below the traditional film standard light levels, so as to not push up your MaxFALL / average light level.  Of particular concern here is planning your edits for HDR - small patches of direct sun in a darker scene are fine, until you move in for the close up and that small patch behind the actor’s face dominates part of the scene.

In this wide and close pair, the wide shot is only limited by the available peak brightness of the display, while the close up is limited by the MaxFALL

While as an individual shot it’s fine to limit the MaxCLL / peak light level of a close up’s bright patch, when you’re cutting between two shots you’ll need to adjust the wider shot’s MaxCLL to match the MaxCLL permitted by the close up’s MaxFALL.

Or, in plain english, you’ll be limited on the maximum brightness in the wide because the maximum brightness of the close-up will be more limited, if the brighter areas take up more of the frame.  If you’re looking to push the 1000 nits limit of current HDR displays for creative reasons, your scene blocking needs to take in account that average brightness for the close up: plan on minimizing bright areas around the talent or inserts to keep the bright patches bright across a sequence.

Otherwise the shifting brightness levels can be much more visible and leave a ‘greying’ feeling in the more restricted close ups (which the eye would normally perceive as white, except in contrast with something whiter).

Because of the expanded darkness range of HDR, you can design much more ‘dark and moody’ lighting setups than you normally would for standard film or video exhibition.  Whole detailed-filled scenes can play out in levels under 30 nits!  However, be aware that this is a bad idea if you’re intending your work to end up on consumer displays.  In a darkened reference environment, our eyes will adjust to the lower light levels and we’ll see deeper into the darks.  But in a consumer’s home, where the ambient light level around the display may be higher than cinema or grading spaces, the viewer’s adjustment levels may be limited.

You can, however, still allow scenes to play much darker than typically available in television exhibition, keeping your maximum brightness below 80 nits.  This can be used with great effect when cutting between darker average to lighter average scenes: it’s in this contrast that HDR really pops.

In HDR darker footage can be cut in with brighter footage without the details in the blacks feeling milky, or the drop in brightness being jarring.


RED HDRx

If you don’t shoot using the RED ecosystem, ignore this section (seriously). But if you do, all this talk of shooting in HDR may make you tempted to shoot using RED’s HDRx.  I’m not saying this is a bad idea, but I am saying this is difficult to execute.

The real problem here is getting the HDR grade right using the HDRx footage.  We shot with it once, and our takeaway from that is has been: only shoot it if you absolutely, certainly, without a doubt, need it.  Which, in this case, means an increase in the recorded dynamic range of the scene (or scene elements).

RED HDRx Blend in HDR and SDR

The reason not to use it comes down to grading.  Blending the two separately exposed elements is fine in REDCINE, but you’re going to run into difficulties with HDR Grading in REDCINE, simply because of the limited grading toolset.  When you grade in DaVinci, you run into severe performance issues using the API blending tool in the RAW decoding.  DaVinci’s split input tool is better, but you still run into problems compressing the larger dynamic range and maintaining the overall look of HDR video.

In the end, the most efficient (inefficient) workflow was actually grade the shot twice in HDR - once for the standard exposure to grade the darks, and a second time with the blended exposure to grade the lights.  Then, both shots need to be passed through a compositing program like After Effects to selectively decide which set of contrast you want for which parts of the image - far more like traditional HDR photography than HDR video.

Dark and Light Plates in HDR and SDR with Final Image Blend

You can get great results this way, but it’s way, way more involved.

 

A Grain of Salt

Cameras, settings, best practices, planning.  Here’s the caveat: take all this advice with a grain of salt, not as a set of hard and fast rules.

Going back to the story that I opened with, even footage never planned to be shown in HDR can give excellent results.  Comparing the SDR version of the shuttle launch footage to the HDR grade, the HDR looks better.  The darks are darker while preserving all the details, and the range is higher.  This is, of course, an ideal case since high quality RAWs were available; the same is true for film sources when negatives are available.

We’ve done a lot of HDR regrading of our back catalog of footage, and I haven’t found a single shot that looks worse in HDR than SDR (even when ignoring the benefits of BT.2020 and 10 bit displays).

 

But even when you’re limited to just 8 bit log or standard gamma footage, you can often find more detail within an scene when grading in HDR then is perceptible in SDR.  While you’ll want to be far more cautious with how far you push the footage, but you’ll still be able to get good results.

Detail recovery is often possible when grading from sufficiently high quality SDR graded sources



Generally, if you are already following best practices for digital cinematography, and if you spend a little bit of time reviewing HDR grades of your existing footage with a colorist, you’ll quickly get a feel for how the HDR space works and what you can do with it, and that’s when you can unleash your own creative potential.


But once it’s shot and edited, what happens next?  Grading, mastering, and delivering in HDR is our next topic, so stay tuned for Part 5.

Written by Samuel Bilodeau, Head of Technology and Post Production

HDR Video Part 3: HDR Video Terms Explained

To kick off our new weekly blog here on mysterybox.us, we’ve decided to publish five posts back-to-back on the subject of HDR video.  This is Part 3: HDR Video Terms Explained.

In HDR Video Part 1 we explored what HDR video is, and what makes it different from traditional video.  In Part 2, we looked at the hardware you need to view HDR video in a professional environment.  Since every new technology comes with a new set of vocabulary, here in Part 3, we’re going to look at all of the new terms that you’ll need to know when working with HDR video.  These fall into three main categories: key terms, standards, and metadata.


Key Terms

HDR / HDR Video - High Dynamic Range Video - Any video signal or recording using one of the new transfer functions (PQ or HLG) to capture, transmit, or display a dynamic range greater than the traditional CRT gamma or BT.1886 Gamma 2.4 transfer functions at 100-120 nits reference.

The term can also be used as a compatibility indicator, to describe any camera capable of capturing and recording a signal this way, or a display that either exhibits the extended dynamic range natively or is capable of automatically detecting an HDR video signal and renormalizing the footage for its more limited or traditional range.


SDR / SDR Video - Standard Dynamic Range Video - Any video signal or recording using the traditional transfer functions to capture, transmit, or display a dynamic range limited to the traditional CRT gamma or BT.2886 Gamma 2.4 transfer functions at 100-120 nits reference. SDR video is fully compatible with all pre-existing video technologies.


nit - A unit of brightness density, or luminance. It’s the colloquial term for the SI units of candelas per square meter (1 nit = 1 cd/m2). It directly converts with the United States customary unit of foot-lamberts (1 fl = 1 cd/foot2), with 1 fl = 3.426 nits = 3.426 cd/m2.

Note that the peak nits / foot-lamberts value of a projector is often lower than that of a display, even in HDR video: because a projected image covers more area and the image is viewed in a darker environment than consumer’s homes, the same psychological and physiological responses exist at lower light levels.

For instance, a typical digital cinema screen will have a maximum brightness of 14fl or 48 cd/m2 vs. the display average of 80-120nits for reference and 300 for LCDs and Plasmas in the home. HDR cinema actual light output ranges in theaters are adjusted accordingly, since 1000 cd/m2 on a theater’s 30 foot screen is perceived to be far brighter than on a 65” flat screen.


EOTF - Electro-Optical Transfer Function - A mathematical equation or set of instructions that translate voltages or digital values into brightness values. It is the opposite of the Optical-Electro Transfer Function, or OETF, that defines how to translate brightness levels into voltages or digital values.

Traditionally, the OETF and EOTF were incidental to the behavior of the cathode ray tube, which could be approximated by a 0-1 exponential curve with a power value (gamma) of 2.4. Now they are defined values like ‘Linear”, “Gamma 2.4” or any of the various LOG formats. OETFs are used at the acquisition end of the video pipeline (by the camera) to convert brightness values into voltages/digital values, and EOTFs are used by displays to translate voltages/digital values into brightness values for each pixel.


PQ - Perceptual Quantization - Name of the EOTF curve developed by Dolby and standardized in SMPTE ST.2084, designed to allocate bits as efficiently as possible with respect to how the human vision perceives changes in light levels.

Perceptual Quantization (PQ) Electro-Optical Transfer Function (EOTF) with Gamma 2.4 Reference

Dolby’s tests established the Barten Threshold (also called the Barten Limit or the Barten Ramp), the point at what the difference in light levels between two values does that difference become visible.

PQ is designed that when operating at 12 bits per channel, the stepping between single digital values is always below the Barten threshold, for the whole range from 0.0001 to 10,000 nits, without being so far below that threshold that the resolution between bits is wasted. At 10 bits per channel, the PQ function is just slightly above the Barten threshold, where in some (idealized) circumstances stepping may be visible, but in most cases should be unnoticeable.

Barten Thesholds for 10 bit and 12 bit Rec. 1886 and PQ curves. Source

For comparison, current log formats waste bits on the low end (making them suitable for acquisition to preserve details in the darks, but not transmission and exhibition), while the current standard gamma functions waste bits on the high end, while creating stepping in the darks.

HDR systems using PQ curves are not directly backwards compatible with standard dynamic range video.


HLG - Hybrid Log Gamma - A competing EOTF curve to PQ / SMPTE ST.2084 designed by the BBC and NHK to preserve a small amount of backwards compatibility.

Hybrid Log Gamma (HLG) Electro-Optical Transfer Function (EOTF) with Gamma 2.4 Reference

HLG vs. SDR gamma curve with and without knees.  Source

HLG vs. SDR gamma curve with and without knees. Source

On this curve, the first 50% of the curve follows the output light levels of standard Gamma 2.4, while the top 50% steeply diverges along a log curve, covering the brightness range from about 100 to 5000 nits. As with PQ, 10 bits per channel is the minimum permitted.

HLG does not expand the range of the darks like PQ curve, and as an unfortunate side effect of the backwards compatibility coupled with the max-fall necessitated by the technology of HDR displays, whites can appear grey, when viewed in standard gamma 2.4, especially when compared to footage natively graded in gamma 2.4.


Standards

SMPTE ST. 2084 - First official standardization of HDR video transfer function by a standardization body, and is at the moment (October 2016), the most widely implemented. SMPTE ST.2084 officially defines the PQ EOTF curve for translating a set of 10 bit, or 12 bit per channel digital values into a brightness range of 0.0001 to 10,000 nits. SMPTE ST.2084 provides the basis for HDR 10 Media Profile and Dolby Vision implementation standards.

This is the transfer function to select in HEVC encoding to signal a PQ HDR curve.


ARIB STD-B67 - Standardized implementation of Hybrid Log Gamma by the Association of Radio Industries and Businesses. Defines the use of the HLG curve, with 10 or 12 bits per channel color and the same color primaries as BT.2020 color space.

This is the transfer function to select in HEVC encoding to signal an HLG HDR curve.


ITU-T BT.2100 - ITU-T Recommendation BT.2100 - ITU-T’s standardization of HDR for television broadcast. Ratified in 2016, this document is the HDR equivalent of ITU-T Recommendation BT.2020 (Rec.2020 / BT.2020). When compared with BT.2020, BT.2100 includes the FHD (1920x1080) frame size in addition to the UHD and FUHD, and defines two acceptable transfer functions (PQ and HLG) for HDR broadcast, instead of the single transfer function (BT.1886 equivalent) found in BT.2020.

BT.2100 uses the same color primaries and the same RGB to YCbCr signal format transform as BT.2020, and includes similar permissions of 10 or 12 bits per channel as BT.2020, although BT.2100 also permits full range code values in 10 or 12 bits where BT.2020 is limited only to traditional legal.

BT.2100 also includes considerations for a chroma subsampling methodology based on the LMS color space (human visual system tristimulus values), called ICTCP, and a transform for ‘gamma weighting’ (in the sense of the PQ and HLG equivalent of gamma weighting) the LMS response as L’M’S’.


HDR 10 Media Profile - The Consumer Technologies Association (CTA)’s official HDR video standard for use in HDR Televisions. HDR 10 requires the use of the SMPTE ST.2084 EOTF, BT.2020 color space, 10 bits per channel, 4.2.0 chroma subsampling, and the inclusion of SMPTE ST.2086 and associated MaxCLL and MaxFALL metadata values.

HDR 10 Media Profile defines the signal televisions can decode for the inclusion of “HDR compatibility” term in the marketing of televisions.

Note that “HDR compatibility” does not necessarily define the ability to display in the higher dynamic range, simply to the compatibility to decode and renormalize footage in the HDR 10 specification for whatever the dynamic range and color space of the display happen to be.


Dolby Vision - Dolby’s proprietary implementation of the PQ curve, for theatrical setups and home devices. Dolby Vision supports both the BT.2020 and the DCI-P3 color space, at 10 and 12 bits per channel, for home and theater, respectively.

The distinguishing feature of Dolby Vision is the inclusion of shot-by-shot transform metadata that adapts the PQ graded footage into a limited range gamma 2.4 or gamma 2.6 output for SDR displays and projectors. The colorist grades the film in the target HDR space, and then runs a second adaptation pass to adapt the HDR grade into SDR, and the transform is saved into the rendered HDR output files as metadata. This allows for a level of backwards compatibility with HDR transmitted footage, while still being able to make the most of the SDR and the HDR ranges.

Because Dolby Vision is a proprietary format, it requires a license issued by Dolby and the use of qualified hardware, which at the moment (October 2016) is only the Dolby PRM-4220, the Sony BVM-X300, or the Canon DP-V2420 displays


Metadata

MaxCLL Metadata - Maximum Content Light Level - An integer metadata value defining the maximum light level, in nits, of any single pixel within an encoded HDR video stream or file. MaxCLL should be measured during or after mastering. However if you keep your color grade within the MaxCLL of your display’s HDR range, and add a hard clip for the light levels beyond your display’s maximum value, you can use your display’s maximum CLL as your metadata MaxCLL value.


MaxFALL Metadata - Maximum Frame Average Light Level - An integer metadata value defining the maximum average light level, in nits, for any single frame within an encoded HDR video stream or file. MaxFALL is calculated by averaging the decoded brightness values of all pixels within each frame (that is, converting the digital value of each frame into its corresponding nits value, and averaging all of the nits values within each frame).

MaxFALL is an important value to consider in mastering and color grading, and is usually lower than the MaxCLL value. The two values combined define how bright any individual pixel within a frame can be, and how bright the frame as a whole can be.

Displays are limited differently on both of those values, though typically only the peak (single pixel) brightness of a display is reported. As pixels get brighter and approach their peak output, they draw more power and heat up. With current technology levels, no display can push all of its pixels into the maximum HDR brightness level at the same time - the power draw would be extremely high, and the heat generated would severely damage the display.

As a result, displays will abruptly notch down the overall image brightness when the frame average brightness exceeds the rated MaxFALL, to keep the image under the safe average brightness level, regardless of what the peak brightness of the display or encoded image stream may be.

For example, while the BVM-X300 has a peak value of 1000 nits for any given pixel (MaxCLL = 1000), on average, the frame brightness cannot exceed about 180 nits (MaxFALL = 180). The MaxCLL and MaxFALL metadata included in the HDR 10 media profile allows consumer displays to adjust the entire stream’s brightness to match their own display limits.


SMPTE ST.2086 Metadata - Metadata Information about the display used to grade the HDR content. SMPTE ST.2086 includes information on six values: the three RGB primaries used, the white point used, and the display maximum and minimum light levels.

The RGB primaries and the white point values are recorded as ½ of their (X,Y) values from the CIE XYZ 1931 chromaticity standard, and expressed as the integer portion of the the first five significant digits, without a decimal place. Or, in other words:

f(XPrimary) = 100,000 × XPrimary ÷ 2

f(YPrimary) = 100,000 × YPrimary ÷ 2.

For example, the (X,Y) value of DCI-P3’s ‘red’ primary is (0.68, 0.32) in CIE XYZ; in SMPTE ST.2086 terms it’s recorded as

R(34000,16000)

because

for R(0.68,0.32):

f(XR) = 100,000 × 0.68 ÷ 2 = 34,000

f(YR) = 100,000 × 0.32 ÷ 2 = 16,000

Maximum and minimum luminance values are recorded as nits × 10,000, so that they too end up as positive integers. For instance, a display like the Sony BVM-X300 with a range from 0.0001 to 1000 nits would record its luminance as

L(10000000,1)

The full ST.2086 Metadata is ordered Green, Blue, Red, White Point, Luminance with the values as

G(XG,YG)B(XB,YB)R(XR,YR)WP(XWP,YWP)L(max,min)

all strung together, and without spaces. For instance, the ST.2086 for a DCI-P3 display with a maximum luminance of 1000 nits, a minimum of 0.0001 nit would be, and using white point D65 would be:

G(13250,34500)B(7500,3000)R(34000,16000)WP(15635,16450)L(10000000,1)

while a display like the Sony BVM-X300, using BT.2020 primaries, with a white point of D65 and the same max and min brightness would be:

G(8500,39850)B(6550,2300)R(35400,14600)WP(15635,16450)L(10000000,1)

In an ideal situation, it would be best to use a colorimeter and measure the display’s native R-G-B and white point values; however, in all practicality the RGB and white point values the display conforms to that was used in mastering, are sufficient in communicating information about the mastery to the end unit display.


That should be a good overview of the new terms that HDR video has (so far) introduced into the extended video technologies vocabulary, and are a good starting point for diving deeper into learning about and using HDR video on your own, at the professional level.

In Part 4 of our series we’re going to take the theory of HDR video and start talking about the practice, and look specifically about how to shoot with HDR in mind.

Written by Samuel Bilodeau, Head of Technology and Post Production