HDR Video Part 3: HDR Video Terms Explained

To kick off our new weekly blog here on mysterybox.us, we’ve decided to publish five posts back-to-back on the subject of HDR video.  This is Part 3: HDR Video Terms Explained.

In HDR Video Part 1 we explored what HDR video is, and what makes it different from traditional video.  In Part 2, we looked at the hardware you need to view HDR video in a professional environment.  Since every new technology comes with a new set of vocabulary, here in Part 3, we’re going to look at all of the new terms that you’ll need to know when working with HDR video.  These fall into three main categories: key terms, standards, and metadata.


Key Terms

HDR / HDR Video - High Dynamic Range Video - Any video signal or recording using one of the new transfer functions (PQ or HLG) to capture, transmit, or display a dynamic range greater than the traditional CRT gamma or BT.1886 Gamma 2.4 transfer functions at 100-120 nits reference.

The term can also be used as a compatibility indicator, to describe any camera capable of capturing and recording a signal this way, or a display that either exhibits the extended dynamic range natively or is capable of automatically detecting an HDR video signal and renormalizing the footage for its more limited or traditional range.


SDR / SDR Video - Standard Dynamic Range Video - Any video signal or recording using the traditional transfer functions to capture, transmit, or display a dynamic range limited to the traditional CRT gamma or BT.2886 Gamma 2.4 transfer functions at 100-120 nits reference. SDR video is fully compatible with all pre-existing video technologies.


nit - A unit of brightness density, or luminance. It’s the colloquial term for the SI units of candelas per square meter (1 nit = 1 cd/m2). It directly converts with the United States customary unit of foot-lamberts (1 fl = 1 cd/foot2), with 1 fl = 3.426 nits = 3.426 cd/m2.

Note that the peak nits / foot-lamberts value of a projector is often lower than that of a display, even in HDR video: because a projected image covers more area and the image is viewed in a darker environment than consumer’s homes, the same psychological and physiological responses exist at lower light levels.

For instance, a typical digital cinema screen will have a maximum brightness of 14fl or 48 cd/m2 vs. the display average of 80-120nits for reference and 300 for LCDs and Plasmas in the home. HDR cinema actual light output ranges in theaters are adjusted accordingly, since 1000 cd/m2 on a theater’s 30 foot screen is perceived to be far brighter than on a 65” flat screen.


EOTF - Electro-Optical Transfer Function - A mathematical equation or set of instructions that translate voltages or digital values into brightness values. It is the opposite of the Optical-Electro Transfer Function, or OETF, that defines how to translate brightness levels into voltages or digital values.

Traditionally, the OETF and EOTF were incidental to the behavior of the cathode ray tube, which could be approximated by a 0-1 exponential curve with a power value (gamma) of 2.4. Now they are defined values like ‘Linear”, “Gamma 2.4” or any of the various LOG formats. OETFs are used at the acquisition end of the video pipeline (by the camera) to convert brightness values into voltages/digital values, and EOTFs are used by displays to translate voltages/digital values into brightness values for each pixel.


PQ - Perceptual Quantization - Name of the EOTF curve developed by Dolby and standardized in SMPTE ST.2084, designed to allocate bits as efficiently as possible with respect to how the human vision perceives changes in light levels.

Perceptual Quantization (PQ) Electro-Optical Transfer Function (EOTF) with Gamma 2.4 Reference

Dolby’s tests established the Barten Threshold (also called the Barten Limit or the Barten Ramp), the point at what the difference in light levels between two values does that difference become visible.

PQ is designed that when operating at 12 bits per channel, the stepping between single digital values is always below the Barten threshold, for the whole range from 0.0001 to 10,000 nits, without being so far below that threshold that the resolution between bits is wasted. At 10 bits per channel, the PQ function is just slightly above the Barten threshold, where in some (idealized) circumstances stepping may be visible, but in most cases should be unnoticeable.

Barten Thesholds for 10 bit and 12 bit Rec. 1886 and PQ curves. Source

For comparison, current log formats waste bits on the low end (making them suitable for acquisition to preserve details in the darks, but not transmission and exhibition), while the current standard gamma functions waste bits on the high end, while creating stepping in the darks.

HDR systems using PQ curves are not directly backwards compatible with standard dynamic range video.


HLG - Hybrid Log Gamma - A competing EOTF curve to PQ / SMPTE ST.2084 designed by the BBC and NHK to preserve a small amount of backwards compatibility.

Hybrid Log Gamma (HLG) Electro-Optical Transfer Function (EOTF) with Gamma 2.4 Reference

HLG vs. SDR gamma curve with and without knees.  Source

HLG vs. SDR gamma curve with and without knees. Source

On this curve, the first 50% of the curve follows the output light levels of standard Gamma 2.4, while the top 50% steeply diverges along a log curve, covering the brightness range from about 100 to 5000 nits. As with PQ, 10 bits per channel is the minimum permitted.

HLG does not expand the range of the darks like PQ curve, and as an unfortunate side effect of the backwards compatibility coupled with the max-fall necessitated by the technology of HDR displays, whites can appear grey, when viewed in standard gamma 2.4, especially when compared to footage natively graded in gamma 2.4.


Standards

SMPTE ST. 2084 - First official standardization of HDR video transfer function by a standardization body, and is at the moment (October 2016), the most widely implemented. SMPTE ST.2084 officially defines the PQ EOTF curve for translating a set of 10 bit, or 12 bit per channel digital values into a brightness range of 0.0001 to 10,000 nits. SMPTE ST.2084 provides the basis for HDR 10 Media Profile and Dolby Vision implementation standards.

This is the transfer function to select in HEVC encoding to signal a PQ HDR curve.


ARIB STD-B67 - Standardized implementation of Hybrid Log Gamma by the Association of Radio Industries and Businesses. Defines the use of the HLG curve, with 10 or 12 bits per channel color and the same color primaries as BT.2020 color space.

This is the transfer function to select in HEVC encoding to signal an HLG HDR curve.


ITU-T BT.2100 - ITU-T Recommendation BT.2100 - ITU-T’s standardization of HDR for television broadcast. Ratified in 2016, this document is the HDR equivalent of ITU-T Recommendation BT.2020 (Rec.2020 / BT.2020). When compared with BT.2020, BT.2100 includes the FHD (1920x1080) frame size in addition to the UHD and FUHD, and defines two acceptable transfer functions (PQ and HLG) for HDR broadcast, instead of the single transfer function (BT.1886 equivalent) found in BT.2020.

BT.2100 uses the same color primaries and the same RGB to YCbCr signal format transform as BT.2020, and includes similar permissions of 10 or 12 bits per channel as BT.2020, although BT.2100 also permits full range code values in 10 or 12 bits where BT.2020 is limited only to traditional legal.

BT.2100 also includes considerations for a chroma subsampling methodology based on the LMS color space (human visual system tristimulus values), called ICTCP, and a transform for ‘gamma weighting’ (in the sense of the PQ and HLG equivalent of gamma weighting) the LMS response as L’M’S’.


HDR 10 Media Profile - The Consumer Technologies Association (CTA)’s official HDR video standard for use in HDR Televisions. HDR 10 requires the use of the SMPTE ST.2084 EOTF, BT.2020 color space, 10 bits per channel, 4.2.0 chroma subsampling, and the inclusion of SMPTE ST.2086 and associated MaxCLL and MaxFALL metadata values.

HDR 10 Media Profile defines the signal televisions can decode for the inclusion of “HDR compatibility” term in the marketing of televisions.

Note that “HDR compatibility” does not necessarily define the ability to display in the higher dynamic range, simply to the compatibility to decode and renormalize footage in the HDR 10 specification for whatever the dynamic range and color space of the display happen to be.


Dolby Vision - Dolby’s proprietary implementation of the PQ curve, for theatrical setups and home devices. Dolby Vision supports both the BT.2020 and the DCI-P3 color space, at 10 and 12 bits per channel, for home and theater, respectively.

The distinguishing feature of Dolby Vision is the inclusion of shot-by-shot transform metadata that adapts the PQ graded footage into a limited range gamma 2.4 or gamma 2.6 output for SDR displays and projectors. The colorist grades the film in the target HDR space, and then runs a second adaptation pass to adapt the HDR grade into SDR, and the transform is saved into the rendered HDR output files as metadata. This allows for a level of backwards compatibility with HDR transmitted footage, while still being able to make the most of the SDR and the HDR ranges.

Because Dolby Vision is a proprietary format, it requires a license issued by Dolby and the use of qualified hardware, which at the moment (October 2016) is only the Dolby PRM-4220, the Sony BVM-X300, or the Canon DP-V2420 displays


Metadata

MaxCLL Metadata - Maximum Content Light Level - An integer metadata value defining the maximum light level, in nits, of any single pixel within an encoded HDR video stream or file. MaxCLL should be measured during or after mastering. However if you keep your color grade within the MaxCLL of your display’s HDR range, and add a hard clip for the light levels beyond your display’s maximum value, you can use your display’s maximum CLL as your metadata MaxCLL value.


MaxFALL Metadata - Maximum Frame Average Light Level - An integer metadata value defining the maximum average light level, in nits, for any single frame within an encoded HDR video stream or file. MaxFALL is calculated by averaging the decoded brightness values of all pixels within each frame (that is, converting the digital value of each frame into its corresponding nits value, and averaging all of the nits values within each frame).

MaxFALL is an important value to consider in mastering and color grading, and is usually lower than the MaxCLL value. The two values combined define how bright any individual pixel within a frame can be, and how bright the frame as a whole can be.

Displays are limited differently on both of those values, though typically only the peak (single pixel) brightness of a display is reported. As pixels get brighter and approach their peak output, they draw more power and heat up. With current technology levels, no display can push all of its pixels into the maximum HDR brightness level at the same time - the power draw would be extremely high, and the heat generated would severely damage the display.

As a result, displays will abruptly notch down the overall image brightness when the frame average brightness exceeds the rated MaxFALL, to keep the image under the safe average brightness level, regardless of what the peak brightness of the display or encoded image stream may be.

For example, while the BVM-X300 has a peak value of 1000 nits for any given pixel (MaxCLL = 1000), on average, the frame brightness cannot exceed about 180 nits (MaxFALL = 180). The MaxCLL and MaxFALL metadata included in the HDR 10 media profile allows consumer displays to adjust the entire stream’s brightness to match their own display limits.


SMPTE ST.2086 Metadata - Metadata Information about the display used to grade the HDR content. SMPTE ST.2086 includes information on six values: the three RGB primaries used, the white point used, and the display maximum and minimum light levels.

The RGB primaries and the white point values are recorded as ½ of their (X,Y) values from the CIE XYZ 1931 chromaticity standard, and expressed as the integer portion of the the first five significant digits, without a decimal place. Or, in other words:

f(XPrimary) = 100,000 × XPrimary ÷ 2

f(YPrimary) = 100,000 × YPrimary ÷ 2.

For example, the (X,Y) value of DCI-P3’s ‘red’ primary is (0.68, 0.32) in CIE XYZ; in SMPTE ST.2086 terms it’s recorded as

R(34000,16000)

because

for R(0.68,0.32):

f(XR) = 100,000 × 0.68 ÷ 2 = 34,000

f(YR) = 100,000 × 0.32 ÷ 2 = 16,000

Maximum and minimum luminance values are recorded as nits × 10,000, so that they too end up as positive integers. For instance, a display like the Sony BVM-X300 with a range from 0.0001 to 1000 nits would record its luminance as

L(10000000,1)

The full ST.2086 Metadata is ordered Green, Blue, Red, White Point, Luminance with the values as

G(XG,YG)B(XB,YB)R(XR,YR)WP(XWP,YWP)L(max,min)

all strung together, and without spaces. For instance, the ST.2086 for a DCI-P3 display with a maximum luminance of 1000 nits, a minimum of 0.0001 nit would be, and using white point D65 would be:

G(13250,34500)B(7500,3000)R(34000,16000)WP(15635,16450)L(10000000,1)

while a display like the Sony BVM-X300, using BT.2020 primaries, with a white point of D65 and the same max and min brightness would be:

G(8500,39850)B(6550,2300)R(35400,14600)WP(15635,16450)L(10000000,1)

In an ideal situation, it would be best to use a colorimeter and measure the display’s native R-G-B and white point values; however, in all practicality the RGB and white point values the display conforms to that was used in mastering, are sufficient in communicating information about the mastery to the end unit display.


That should be a good overview of the new terms that HDR video has (so far) introduced into the extended video technologies vocabulary, and are a good starting point for diving deeper into learning about and using HDR video on your own, at the professional level.

In Part 4 of our series we’re going to take the theory of HDR video and start talking about the practice, and look specifically about how to shoot with HDR in mind.

Written by Samuel Bilodeau, Head of Technology and Post Production


HDR Video Part 2: HDR Video Reference Hardware

UPDATE 18 December 2017: We've posted a new blog about using Production HDR monitors for grading in HDR.  This puts HDR grading displays in the sub $4,000 USD range.  Read our post about how to do that and what you'll need here.

To kick off our new weekly blog here on mysterybox.us, we’ve decided to publish five posts back-to-back on the subject of HDR video.  This is Part 2: HDR Video Reference Hardware.

In HDR Video Part 1 we explored what HDR video is, and what makes it different from traditional video.  Here in Part 2, we’re going to look at what hardware is needed for proper HDR grading (as of October 2016), and how to partially emulate HDR to get a feel for grading in the standard before investing in a full HDR setup.


New Standard, New Hardware

Alright, first, the bad news.  Professional grade reference displays for HDR are expensive.  And there are only two that are commercially available for purchase*: The Sony BVM-X300 and the Dolby PRM-4220.  Both cover 100% of DCI-P3 space, but the BVM-X300 operates in and covers most of BT.2020, has a 4K resolution, a peak brightness of 1000 nits, and uses OLED panels for more detail through the darks.  The PRM-4220 is an excellent display, but is only 2K in resolution and 600 nits max, though it operates with a 12 bit panel for better DCI reference.

At the time of this writing, these are the only two commercially available HDR reference monitors.

At the time of this writing, these are the only two commercially available HDR reference monitors.

At this time, I can’t find any DCI projectors advertising HDR capabilities, though I think that a bright enough laser projector with the right LUT could emulate one, in a small environment - essentially using the LUT trick I’m going to describe below while using a projector that’s 10x brighter than it should be for the reference environment.  That doesn’t mean they don’t exist, it just means you’ll need to talk to the manufacture directly.  I haven’t tested this, though, so don’t quote me on it.

There's at least one reference display that claims to be HDR compatible, but really isn’t - the Canon DP-V2410.  Frankly, the display is actually gorgeous and comparable to the Sony for color rendering and detail level, but it’s max brightness is only 300 nits and it’s HDR mode downscales the SMPTE ST.2084 0.0001 - 1000 nit range into the 0.01-300 nit range.  This leaves the overall image rather lackluster and less impactful, though you could use it to grade in a pinch, since its response curve is right.  But I wouldn’t, primarily because of MaxFALL, which I’ll cover extensively in Parts 4 and 5.

At Mystery Box we decided to go with the Sony BVM-X300 for our HDR grading.  I can’t praise the look of this display enough, though I do have my gripes (I mean, who doesn’t?), but I’ll save that review for another time.

Sony BVM-X300 (Right) in Mystery Box's grading environment (lights on, for detail clarity)


HDR Video on Consumer Displays

DaVinci Resolve Studio 12.5+ Settings for enabling HDR metadata over HDMI

The most affordable option for grading in HDR is to use an HDR television.  The Vizio Reference series have nice color with a 300 nit peak (in HDR mode), while the LG 2016 OLED HDR series displays have phenomenal color, with max brightness levels approaching 1000 nits.

The catch is, of course, that there is still more variation in the color of the display than in a reference display, so unless you know for certain that you’re going to be exhibiting on that specific display, be cautious when using them to grade.  They also lack SDI inputs, but that’s solvable.

DaVinci Resolve Studio version 12.5+ has an option to inject flags for HDR and BT.2020 into the HDMI output of your DeckLink or UltraStudio hardware.  To grade in HDR using a consumer HDR television with HDMI input, simply hook up the display over HDMI, toggle the option in your DaVinci settings and the display will automatically switch into HDR mode:

If you’re not using DaVinci Resolve Studio 12.5+, or if for whatever reason you have to route SDI out, you can inject the right metadata into the HDMI stream once you’ve converted from SDI to HDMI.  What you’ll need is an Integral by HD Fury.  This box, which is annoyingly only configurable under Windows, will add the right metadata into the HDMI connection between host and device, allowing you to flag any HDMI stream as BT.2020 and HDR.

Marketing shot of Integral by HD Fury, a box that will allow you to manually alter HDMI metadata

BE CAREFUL if you’re using the Integral though.  It can be tempting to use the HDMI output of your computer to just patch your desktop into HDR.  This is a bad idea.  Any interface lines will also be translated into HDR, which will limit the displays overall brightness (because you can’t switch your desktop into HDR mode), and any static elements risk burn-in.  Most HDR displays use OLED panels and OLEDs are susceptible to burn-in!

If you are already using SDI outputs for your I/O, and want to switch to the BVM-X300 or the PRM-4220, you shouldn’t NEED to upgrade your I/O card or box to drive HDR - 10b 4:2:2 works for grading HDR.  You might want to upgrade though if you want higher output resolutions (4K vs 2K/1080), higher frame rates at the higher resolutions (50/60p) or RGB/4:4:4 instead of 4:2:2 Chroma Subsampling.

Everything else should work with your existing color correction hardware.


Emulating HDR Video

Okay, so if you’re not ready to spring for the new reference hardware, but want to emulate HDR just to get a feel for how it works, here’s a trick you can do using a standard broadcast display, or a display like the HP Dreamcolor Z27x (which I used when doing my first tests) to partially emulate HDR.

Use a reference display with native BT.2020 color support, if you can.  If you’re using Rec 709, but still want to get a feel for grading in BT.2020, there’s a fix for that using LUTs, but it’s not elegant.  You can get a feel for the HDR curve in a Rec 709 color space, but you won’t get a feel for how the primaries behave slightly differently, or how saturation works in BT.2020.**

In addition, if possible, try to use a reference display with a 10 bit panel.  There’s no cheat for this one, you either have it or you don’t.  8 bits will give you an idea what you’re looking at, but won’t be as clear as possible.  In many cases it won’t make a difference, you’ll just lose your ability to see specific fine details.

Now, calibrate the display and your environment, to emulate HDR.  Turn your maximum brightness to full (on the Dreamcolor Z27x, it peaks at 250 nits; your display may be different).  Turn off all ambient lighting (as pitch black as possible).  Then, turn down the brightness of the host interface display to the lowest setting that it’s still useable.  Do the same for any video scopes or other external monitoring hardware that may also be hooked up to the output signal.

HP Dreamcolor Z27x HDR Approximation Settings

This should make your reference display the brightest thing in the room, by a factor of 2 to 4x.  This is important.  While the display will still lack ‘umph’, at very least it’ll dominate your perception of brightness.  That’s key to creating the illusion of the HDR effect in this case; without it your screen will just look muted and dull.

HDR Approximation Environment Calibration: Lights off, scopes dimmed, interface display as low as possible while retaining visibility (6%, in this case)

At this point, what we’ve done by adjusting the ambient and display brightness is emulated the greater brightness range of HDR without using a display that pushes into the HDR range.  Next what we need to do is adjust the display’s response so that it matches the HDR curve we want to emulate.  Essentially, we need to eliminate the display’s native gamma curve for either PQ or HLG curve.

DaVinci Resolve Studio's LUTs for scaling HDR into Gamma 2.4 / Gamma 2.6

This is actually pretty easy to do in DaVinci Resolve Studio - DaVinci has a set of 3D LUTs you can attach to your output that will automatically do it for you.  You’ll find them written as “HDR <value in nits> nits to Gamma <target display gamma>” (ex. HDR 1000 nits to Gamma 2.4) for the SMPTE 2084 / PQ curve, and “HLG to Gamma <target display gamma>” (ex. HLG to Gamma 2.2) for the Hybrid Log Gamma curve.

What these LUTs do, essentially, is add a 1/gamma (ex 1/2.4) contrast curve to the output signal, combined with the selected contrast curve, i.e., the one you want to see.  The gamma reciprocal adjustment combines with the display’s native or selected gamma to linearlize the overall signal, as the two curves cancel each other left.  The only contrast curve you’re left with, then, is the HDR contrast curve you’ve added to the signal, the HDR curve being translated into your display’s native or adapted luminance range.**

Using one of these LUTs on your monitor or output will allow your display to respond as if it were operating natively with the HDR curve, though you'll notice that your display is only showing the first 100 nits of HDR curve.  We'll fix that next.

The final step is to calibrate your display’s brightness and contrast.  I add a timeline node and scale the gain and gamma adjustments to bring the full HDR range back into the display's native signal range.  As for adjusting the contrast, though, there’s not much I can say about how to do that, other than to use a reference image or two graded in the target space to calibrate the display until it ‘looks right’.  Here are a couple that I graded in SMPTE 2084 that you can use for calibration:

Mystery Box ST.2084 Calibration Images, normalized for Rec.709. Follow this link to download the DPX and individual reference PNGs.

All of this LUT business and brightness scaling, by the way, is exactly what the Canon DP-V2410 does, it just does it internally with a mode switch instead of needing manual setup.  Don’t get me wrong - in every other respect, the DP-V2410 is an amazing display, but in HDR mode it’s equivalent to this setup for HDR emulation, rather than true HDR performance.*


Emulated HDR vs. True HDR

So how does an emulated HDR display compare to a true HDR reference display?  Well, poorly is an understatement.  It's not terrible, but emulated HDR lacks the power of the true HDR, the ability to grade with lights on and see how your footage holds up through the large punch of the whites.  With an 8 bit panel you’re going to see stepping while grading in an emulated HDR mode, because most of the region you’ll be working in ends up compressed to 50 or so code values.

This compression in the darks means you won’t get a feel for just how deep SMPTE 2084 can go while still preserving details - you can grade whole shots with full detail in the darks and a few hundred levels of contrast, that land between codes 4 and 14 (full range) on a standard 8 bit display (especially an LED or CFL backed LCD).

You’ll also be tempted in this mode to grade ‘hot’, that is, push more into the brights of the image, since you don’t have any actual limits for frame average light levels, like all true HDR displays do.  That’s not necessarily a problem, but you’ll run into trouble if you try to use the footage elsewhere.  You also miss the great psychological response the actual dark and light levels of a true HDR range give you.

So why emulate then?  Well, right now, HDR reference hardware is expensive.  And if you want to practice grading and mastering in HDR, without having to invest in the hardware, emulation is a fantastic place to start.  You’ll get to see how the mids and highs roll into the whites in SMPTE 2084, and develop tricks to make your grading more natural when you make the switch to a proper HDR display.  You may even be able to grade using emulated HDR so long as you have a proper HDR television to internally QC before sending out to a client - assuming your mastering of the HDR file is right, you can check it on a television and make sure it at very least looks good there, contrast and curve wise, before sending it out to a client.

Of course, mastering HDR video is problem in and of itself, but I’m saving it for last, in Part 5 of our series.  First, though, we’re going to look at the new terminology introduced with HDR video, because even if you’ve been working with video for decades, most of this is likely to be new.

Written by Samuel Bilodeau, Head of Technology and Post Production


Endnotes

* The day I went to post this I found Canon had updated their website to include the Canon DP-V2420, which they claim supports full HDR in both the ST.2084 and the HLG specifications, and be Dolby Vision qualified;  I haven't had time to look into these claims.

The Dolby PRM-4220 requires a workaround to get it to operate in an HDR mode.  It can be loaded with a custom gamma curve that can match the HDR EOTF, or you can add a custom LUT that scales the 0.01 - 600 nits of SMPTE ST2084 into gamma 2.4 while operating the display in 600 nits mode.

The Dolby Pulsar and the Dolby PRM-32FHD are both HDR capable displays, operating at 4000 and 2000 nits respectively, but I elected not to mention them because they are not, to the best of my knowledge, generally available for purchase.

** If you’re using the LUT on your output to emulate the HDR curve, but only have a Rec. 709 display and want to get a feel for BT.2020, you may consider using a BT.2020 to Rec. 709 LUT and stacking it with the gamma compensating LUT.  In DaVinci you can do this by adding one LUT to the output, and a second LUT for the monitor, or by attaching one of the LUTs to a global node for a timeline.  As a last resort, you can attach as many LUTs as you want to individual grades. You should be able to do something similar in pretty much all other color grading or mastering softwares like Scratch or Nuke.