Multi-Space Color Correction: The New Paradigm

Our last technical blog post talked about color management, including considerations for maintaining color accuracy through the post process by keeping displays & projectors calibrated and understanding how each application manages the colors you see.  We alluded to the fact that at face value it can seem pretty complicated to manage color between all of the different kinds of camera technologies and technical standards, especially when dealing with multiple delivery specifications.

With this post we want to go into more detail about the issue of color grading for multiple standards, and how the new paradigm for color correction simplifies the process and keeps your grades more futureproof.  We’re going to examine a few things today.  First, how do computers process colors; second, where the problems we’re trying to fix come from; third, what the solutions are; and lastly, how to implement the solutions and why.


Color Engines

In previous posts (here, here, and here) we’ve mentioned the importance of using a color space agnostic color engine when doing specific color correction tasks - something like DaVinci Resolve Studio.  As a quick review, a color space agnostic engine is like a glorified R-G-B or R-G-B-Y calculator: It takes a specific set of decoded R-G-B or R-G-B-Y data, applies a specific set of mathematical operations based on the user input, and outputs the new R-G-B or R-G-B-Y data.

Agnostic color engines don’t care about what the data is supposed to mean, instead simply producing the results of an operation.  It’s up to the user to know if the results are right, or if the operation has put values out of spec or create unwanted distortions.  This is a double edged tool, since it places far more importance on user understanding to get things right, while being powerful enough to apply its corrections to any combination of custom situations.

As an example of how an agnostic engine works, let’s look at three of the simplest color correction operations: lift, gamma, and gain, operating strictly on the brightness (Y) component of the image.

Lift operates essentially as a global addition value: add or subtract a specific amount to each pixel’s value.  Because of the way that traditional EOTFs work and our human perception of brightness changes, lift tends to have the greatest effect on the darks: quickly raising or lowering the blacks while having a much more reduced effect on the mids and lights.

Gain operates essentially as a global multiplication value: multiply or divide the value of each pixel by a specific amount.  Since the operation essentially affects all tones within the image evenly, all parts of the image see a similar increase or decrease in brightness, though once again because of the EOTF considerations it has the greatest effect in the brights.

Gamma operates as an exponential value adjustment, affecting the linearity of values between the brights and the darks.  Lowering the gamma value has the effect of pulling more of the middle values towards the darks, while raising the gamma value has the effect of pushing the middle values brighter.  Once again, it still affects the brights and the darks, but at a much lower rate.

Notice that these operations don’t take into account what the data is supposed to mean.  And with new HDR EOTFs, especially with the Perceptual Quantization EOTF, you may find extreme changes across the image with very small values, which is why I recommend adding a roll-off curve as the last adjustment to your HDR grading workflow.

The combination of lift, gamma, and gain allow the colorist to adjust the overall brightness and contrast of the image, with fairly fine granularity and image control.

Compare these functions of an agnostic engine to their equivalents in a color space dependant engine.  In a color space dependent engine you’re more likely to find only two adjustments for controlling brightness and contrast: brightness and contrast.

The same color transformation operation has different effects on the image. Here, I've applied the same hue adjustment curve is applied to eight different color spaces and the effects on the vec

The brightness and contrast controls tend to be far more color space dependent, since they’re designed to affect the image in a way that more evenly affects the brightness or contrast along the expected EOTFs.  For the end user, this works a far simpler and often faster approach for minor corrections, at the expense of power, precision, and adaptability.  Which hasn’t been too bad of a trade off, so long as all digital video data operated in the same color space.

But adding support for new color spaces and EOTFs to a brightness and contrast operation requires rewriting the rules for how each of the new color spaces behave as digital RGB values.  That takes time to get right, and is oftimes not done at all.  Meaning that color space dependant engines tend to adapt more slowly to the emerging standards, and there’s no clear path for how to implement the upgrades.

Every color engine, whether we’re talking about a computer application or a chip found in a camera or display, makes assumptions about how to interpret the operations it's instructed to do.  Where the engines lie on the scale from fully color managed to completely color agnostic defines how the operations work, and what effect the ‘same’ color transformation has on the image.

The overall point here is that the same color transformations applied to different color spaces have different effects on the end image.  A hue rotation will accomplish something completely different in Rec. 709 than it will in Rec. 2020; standard gain affects HDR curves in ways that are somewhat unpredictable when compared with SDR curves.  Color engines can either try to compensate for this, or simply assume the user knows what he or she is doing.  And the more assumptions any single operation within an engine makes about the data, the more pronounced the differences if it’s applied to another color space.  These seemingly small differences can create massive problems in today’s color correction and color management workflows.


Understanding the Problem

With that background in mind, let’s explore where these problems come from.

Here’s something that may come as a shock if you haven’t dived into color management before: every camera ‘sees’ the world differently.  We’re not just talking about the effects of color temperature of light or the effects of the lenses (though those are important to keep in mind), but we’re talking about the camera sensors themselves.  All things being equal, different makes and models of camera will ‘see’ the same scene with different RGB values at the sensor.  In inexpensive cameras you may even see variation between individual cameras of the same make and model.

This isn’t a problem, it’s just how cameras work.  Variations in manufacturing process, decisions about which microfilters, microlenses, and OLPF to use, and the design of the sensor circuitry all play a part in changing the raw values the sensor sees.  But to keep things consistent, these unique camera RGB color values are almost always conformed to an output color standard using the camera’s image processor (or by the computer’s RAW interpreter) before you see them.

In the past, all video cameras conformed to analog video color spaces: NTSC/SMPTE-C, PAL, etc., and their early digital successors conformed to the digital equivalent standards: first Rec. 601 and then Rec. 709.

When it comes to conforming camera primaries to standard primaries, manufacturers had two choices: apply the look effects before or after the conforming step.  If you apply color transformations before the conforming step, you often have more information available for the change.  But by conforming first to the common color space, color correction operations would behave the same way between different camera makes and models.  

Most camera manufacturers took a hybrid approach, applying some transformations like gain and white balance before the conforming step, and then applying look effects after the conforming step.  And everything was golden, until the advent of digital cinema cameras.

Digital cinema class cameras started edging out film as the medium of choice for high quality television and feature film production a decade ago, and now vastly outnumber the quantity of film-first productions.  And here’s where we run into trouble.  Because digital cinema uses a different color space than digital video: DCI-P3.  Oh, and recently the video broadcast standards shifted to a much wider color space, Rec. 2020, to shake off the limiting shackles of the cathode ray tube.

Color space selection suddenly became an important part of the camera selection and workflow process, one that few people talked about.  Right from the get go the highest end cameras started offering multiple spaces that you could shoot and conform to, one of which was usually camera RGB.  But changing the conforming space means that any corrections or effects added to the image after conforming behaved differently than they did before, and many user generated looks would be color space dependent.

To fix this, many (but not all) digital cinema camera manufactures moved the ‘look’ elements of their color processing to before the conforming step.  This way, regardless of which color space you, as the operator, chooses any looks you apply will have the same effect on the final image.

Which is fine in camera, and fine through post production.  Unless your color correcting platform doesn’t understand what the primary color values mean, or if it can’t directly transform the values into your working space.  Then you need to create and add additional conforming elements as correction layers, which can increase the computational complexity and reduce the overall image quality.

Oh, and if you start working with multiple cameras with different look settings available, you can get into trouble almost instantly, since there isn’t usually a simple way of conforming all of them to your working space if it’s not Rec. 709.

Oh, and you may have to deliver to all of the different color spaces: Rec. 2020 for 4K television broadcast, DCI-P3 for your digital cinema delivery, Rec. 709 for HD Blu-ray and traditional broadcast, and Rec. 601 for DVDs.  And for sanitiy’s sake, let’s add HDR.

And don’t break the bank.


The Solution

What if there was a way to make sure that a) all of your looks would move simply between cameras, regardless of make and manufacturer, and b) you could color grade once and deliver in all formats simply, without needing to manage multiple grades?

There is a way to do it: create a new RGB color space that encompasses all possible color values, and do all of your color corrections there.

Here’s the block diagram:

Camera RGB -> Very Wide Gamut Working Space (Log or Linear) -> Color Correction / Looks -> Tone Map to Standard Space

By mapping all of the camera sensor values into the same log or linear space, with very wide RGB color primaries1, you can make sure that you have access to all of the image data captured by every camera and that all operations will have the same effect on all images.

But what do I mean by a “very wide gamut RGB space”?  There are two types of gamuts I’m talking about here, both of which have advantages and disadvantages.

The first kind is a color space with virtual RGB primaries:  the RGB color primaries land outside of the CIE 1931 color colorimetry diagram.  Remember that CIE 1931 maps the combinations of various wavelengths of light and the perceivable colors they produce onto an X-Y coordinate plane, and that any color space requires at least three primary color vertices on this chart.  But since the chart is bigger than all mapped colors, you can put these values outside of the actual set of real colors.

By putting the values outside of the visible color ranges you’re defining ‘red’, ‘green’, and ‘blue’ values that simply don’t and can’t exist. But they end up being quite useful, because when you map your primaries this way you can define an RGB color space that includes up to all possible color values.  Yes, you could simply use CIE XYZ values to map all colors, but all of the math needed for color manipulations would need to be redefined and rebuilt from the ground up (and it always requires at least 16 bits precision).  But an RGB space with virtual primaries allows you to use standard RGB math, while maintaining as many colors as possible.

Comparison of eight very wide gamut color spaces, five with virtual primaries, three with real (or mostly real) primaries.

Examples of color spaces using virtual RGB primaries include the color space defined by the Academy of Motion Picture Arts & Sciences, ACES AP0, and many manufacturer specific spaces like Sony S-Gamut3 / S-Gamut3.cine, ARRI Log C Wide Gamut, Canon Cinema Gamut, and the new RedWideGamutRGB found in RED’s IPP2.

The catch with these virtual primaries is that many operations that you as a colorist may be accustomed to doing won’t behave exactly the same way.  The reason being is that the RGB balance as it relates to hues and saturations doesn’t quite apply the same way.  Without getting mired in the details, the effects of these operations are related to the relative shape of the triangle produced by the RGB color primaries, and the color space triangles using all virtual primaries tend to be more dissimilar with the traditional RGB color spaces than the RGB spaces themselves.

So instead some wide gamut formats use all real, or mostly all real primaries to somewhat match the shape (i.e. color correction feel) of working with the smaller color gamuts.  A couple of examples here are Rec. 2020 (called Wide Gamut on 4K televisions), Adobe Wide Gamut, and ACES AP1.  While not covering all possible color values, these spaces cover very large portions of the visible color gamut, making them very useful for color correction working spaces.

Whichever very wide color space you choose to work in is up to you and your needs.  If your company or workflow requires ACES, use ACES.   If you’re only using one type of camera, such as a RED Weapon or an ARRI Alexa, you may find it beneficial to work in as specific manufacturer’s RGB space.

For most of the work we do here at Mystery Box that’s destined for anything other than web, I typically conform everything to Rec. 2020 and do my coloring and mastering in that space.  There are a couple of reasons for this:

  1. As a defined color spaces it uses real, pure wavelength primaries. Meaning that so long as only 3 color primaries are used for image reproduction, it’s about as wide as we’ll ever go.

  2. It encompasses 100% of Rec. 709 / sRGB and 99.98% of DCI-P3 (only losing a tiny amount of the reds)

  3. It encompasses 99.9% of Pointer’s gamut, a gamut that maps all real-world reflectable colors (not perceivable colors, just those found in the real world) onto the CIE XYZ gamut - essentially every color producible through the subtractive primaries.

  4. While it behaves differently than DCI-P3 and Rec. 709, they all behave fairly similarly so the learning curve is low.

  5. It requires fewer tone mapping corrections for the final output.

Whether these reasons are convincing enough for you or not is up to you.  Personally, I don’t find the 0.02% of DCI-P3 it doesn’t cover to actually matter, nor do the set of greens and blue-greens it doesn’t store (and no three color system can produce). These differences are so small that only in the absolute best side-by-sides in a lab could you hope to see a difference.

Whatever you do choose to use as a working space, it’s worth investing the time to pick one and stick with it.  Since the grading transformations do behave differently in the different color spaces, it’s easiest to pick one and refine your technique there to get the best possible results.


Conversions Implementation

Looking at the generalized workflow block diagram you’ll want to consider how to implement the different conversion steps for your own productions, in order to maintain the highest quality image pipeline with the lowest time and resource costs.  So let’s go into the two main places that you need to make new choices in the pipeline, and how to plan for them.

Conforming Camera RGB to Very Wide Gamut Working Space

Moving from Camera RGB to a very wide gamut space is a slightly different process for each camera system, and can depend on whether you’re capturing RAW data or a compressed video image.

When you’re using RAW formats, you’ll manage this step in the color correction or DIT software, which is the prefered workflow when image quality is paramount.  If you’re recording to non-raw intermediate format, like ProRes, DNxHR, or H.264 or any other flavor of MPEG video, you’ll need to select camera settings that best match your target wide gamut space.

Most RAW formats ignore camera looks applied by the operator and store the color decisions as metadata, but most video formats don’t.  Once again, camera settings vary, so it’s important to look at your specific system and run tests to find out where looks are applied in your camera’s image pipeline, and whether you can add a separate look on the video outputs for on-set monitoring while capturing a flattened LOG or linear image.

If your camera can’t separate the looks applied to the video files and the video output, and you want to capture a flat image but need to see it normalized on set, loading LUTs into on-set monitors is the ideal choice for image monitoring.  The process of creating and applying monitoring LUTs varies with your workflow, but we often find ourselves using a two-step process that uses Lattice to generate color space conversion LUTs, which we bring into DaVinci Resolve Studio to add creative looks and generate the final monitoring LUT.

Some cameras or DIT applications export their look settings as CDLs, LUTs, or other metadata for you to use later in the grading process, which you can then apply in post as the starting point for the grading process.  Again, workflows vary.

Generally you’ll want to move directly from camera RGB into the working space to preserve as much sensor information as possible.  That implies that you need to decide what your working space will be before capture (ACES AP0 or Rec. 2020 are recommended for future broadest compatibility), though it’s sometimes not an option.  While RAWs maintain camera primaries and allow you to jump directly into a wide working space later, if you’re forced to conform to standardized RGB for video intermediates you'll need to make that decision as early as possible.  In that case, put them into the widest color space available by the camera, whether that’s Rec. 2020, DCI-P3, or the manufacturer’s proprietary Wide Gamut space.

If RAWs aren’t an option, using a 12 bit log format video is your next best choice.  10 bit is fine too, but you won’t get as clean of corrections later, and may see some banding in fine gradients.  Anything less than 10 bits per channel create severe problems when color grading and really should only be used as a last resort.  When recording using an 8 bit format, you should only use a standard SDR EOTF (never LOG) - LOG with only 8 bits of precision can create MASSIVE amounts of banding.

To summarize: To maintain the highest image quality with the smallest resource pain, use RAW formats when possible, convert to the working or widest color space if you have to record as video files, and use LUTs on display outputs to avoid baking camera looks into the video data.

Tone Mapping the Working Space to Output

Moving from a wide working space to a final deliverable space is generally relatively simple process: simply convert each color value from the working space to the color value equivalent of the target space, and discard any data that lands outside of the target range.  In most Rec. 2020 -> DCI-P3, or Rec. 2020 -> Rec. 709 conversions, this is completely fine.  You may find minor clipping in a few of the most saturated colors, but overall you shouldn’t see too many places where the color is so bad you can’t live with it.

Where you do run into problems is when you’ve graded using an HDR transfer function and are moving into SDR.  A straight translation here results in very, very large amounts of clipping.  I haven’t mentioned EOTFs much yet, simply because most color engines where you’ll be doing wide gamut work use linear internals, since that tends to offer the most dynamic range and manipulation potential.

However, displays rarely offer a linear EOTF and so you’ll have to be monitoring in some transfer function or another.  Display monitoring is another reason I typically grade in BT.2020 (and usually in HDR), since displays need to be set to a specific color space and EOTF.  Which means that if you’re using a very wide working space, you must apply a tone map to your monitoring output, regardless of whether you’re grading in HDR or SDR (especially when you’re working in linear light).

The first series we published here on the blog about HDR video included a section on “Grading Mastering, and Delivering HDR”, where we presented a few bezier curves you can apply as the last element in your node structure for HDR grading in PQ or HLG.  These bezier curves are essentially luminance tone maps, converting the linear light values into the specific range of digital values you use for HDR.

A full tone map typically includes considerations for converting color information as well.  Just like the bezier curves control the roll off of the lights into your target range, tone maps roll off color values between color spaces, to minimize the amount of hard clipping.  Here its important to exercise caution and experiment with your specific needs before selecting a tone map, since this step can create hue or saturation shifts you don’t expect.

Tone mapping is the golden goose of simplifying multi-space color corrections.  It’s what brings everything together by making it possible to very, very quickly move from your working space into your delivery space.

If you grade in ACES AP0 or AP1, the tone maps are already prepared for your conversions.  Simply apply the tone map for the target system and voila, the conversion’s ready for rendering, preserving all (or rather most - they aren’t quite perfect) of the authorial intent of the grade.  We did this on our Yellowstone video to generate the HDR master.2

Grading in other wide color spaces often requires custom tone maps, or on-the-fly maps generated by a program such as DaVinci Resolve Studio.  RED Digital Cinema, for instance, has produced LUT based tone maps for converting their RedWideGamutRGB Log3G10 footage into various HDR and SDR color spaces.  The entire Dolby Vision format is essentially a shot by shot set of tone maps for various screen brightnesses.

Or, you may find yourself doing what we’ve done - spend time to create your own tone mapped LUTs for converting HDR and SDR of various formats, and refining these maps for each individual piece of HDR content so that you end up with the optimal SDR look for that work.


Why Bother?

Wide color space corrections and tone mapping for various output systems is the way that color correction will be treated and handled in the future.  With the arrival of BT. 2020 and HDR transforms, in just the last few years the number of delivery color encoding formats has increased three fold at the very least.  The only way to ensure your content will be compatible in the future is to adopt the new paradigm and multi-space coloring workflow.

DaVinci Resolve Studio’s latest update (release of version 14) saw a significant overhaul of the color management engine in the last few beta versions to optimize the core functionality for this kind of color management workflow.  If you’re using DaVinci Color Management or ACES color management in the latest version, DaVinci will automatically select the optimal RAW interpretation of your footage and conform it to your working space, removing the ambiguity of how to interpret your footage and maintaining the maximum image quality.

Another manufacturer who’s natively implemented a similar color pipeline is RED, with their new IPP2 color workflow.  They’ve moved all of their in-camera looks and apply them to the image data after the sensor RGB is converted into RedWideGamutRGB, tone mapping all outputs to your monitoring space.  With that they now allow you to select whether color adjustments in camera are burned into the ProRes files, or simply attached as a LUT or CDL.  This way, regardless of what your monitoring or eventual mastering space is, the color changes you make in camera will have the exact same effect across the board.

This is the workflow of the future.  Like with HDR, which we can assume will be the EOTF of the future, the efficiencies and simplicities of this particular workflow are so great that the sooner you get on board with it the better your position will be in a few years time.  Grading in ACES AP0 offers a level of future proofing that not even BT.2020 provides.  While BT.2020 still exceeds what current technologies can really do, ACES AP0 ensures that regardless of where color science heads in the future (4+ color primaries?), your footage will already be common format that’s simple to convert to the new standard, preserving all color data.

While there is a learning curve to this workflow, at a technical level it’s simpler to learn and apply than even understanding how HDR video works.  Yes, it takes some getting used to, but it’s worth learning.  Because in the end, you’ll find better quality than you can otherwise hope for.

Written by Samuel Bilodeau, Head of Technology and Post Production
 

Footnotes:

1 Yes, I’m making up the term “Very Wide Gamut” or “Very Wide Gamut RGB” simply because “Wide Gamut” and “Wide Gamut RGB” can refer to many different specific spaces, depending on the circumstances. Here I’m referring to any of these typical wide gamut spaces, or any space that covers a very large portion of the perceivable gamut.

2 A caveate note about ACES tone mapping: We used ACES AP0 with an ACEScc EOTF for our Yellowstone video. The tone mapping into HDR was fantastic and allowed me to skip my own range limiting map, and the ability to select different input transforms for each shot was fantastic. However, ACES failed when trying to generate an SDR version of the film: instead of tone mapping the higher dynamic range into the smaller SDR range, it clipped at the limits of SDR. This limitation makes me hesitant about recommending ACES for mixed-dynamic range work. It works wonderful for one or the other, but don't expect it to tone map directly between the two.

How to Upload HDR Video to YouTube (with a LUT)

Today YouTube announced via their blog official HDR streaming support.  I alluded to the fact that this was coming in my article about grading in HDR because we've been working with them the past month to get our latest HDR video onto the platform. It's officially live now, so we can go into detail.


How to Upload HDR Video to YouTube

Similar to VR support, there are no flags on the platform itself that will allow the user to manually flag the video as HDR after it's been uploaded, so the uploaded file must include the proper HDR metadata.  But YouTube doesn't support uploading in HEVC, so there are two possible pathways to getting the right metadata into your file: DaVinci Resolve Studio 12.5.2 or higher, or the YouTube HDR Metadata Tool.  They are generally outlined in the YouTube support page, but not very clearly, so I think more detail is useful.

I did include a lengthy description on how to manage HDR metadata in DaVinci Resolve Studio 12.5.2+, with a lot more detail than they include on their support page, so if you want to use the Resolve method, head over there and check that out.  I've covered it once, so I don't see the need to cover the how-to's again.

I should note that Resolve doesn't include the necessary metadata for full HDR10 compatibility, lacking fields for MaxFALL, MaxCLL, and the Mastering Display values of SMPTE ST.2086.  It does mark the BT.2020 primaries and the transfer characteristics as either ST.2084 (PQ) or ARIB STD-B67 (HLG), which will let YouTube recognize the file as HDR Video.  YouTube will then fill in the missing metadata for you when it prepares the streaming version for HDR televisions, by assuming you're using the Sony BVM-X300.  So this works, and is relatively easy.  BUT, you don't get to include your own SDR cross conversion LUT; for that you'll need to use YouTube's HDR Metadata Tool.

 

***UPDATE: April 20, 2017*** We've discovered in our testing that if you pass uncompressed 24 bit audio into your QuickTime container out of some versions of Adobe Media Encoder / Adobe Premiere into the mkvmerge tool described below the audio will be distorted.  We recommend using 16 bit uncompressed audio or AAC instead until the solution is found.

 

YouTube's HDR Metadata Tool

Okay, let's talk about option two: YouTube's HDR Metadata Tool.  

Alright, not to criticize or anything here, but the VR metadata tool comes in a nice GUI, but the link to the HDR tool sends you straight to GitHub.  Awesome.  Don't panic, just follow the link, download the whole package, and un-Zip the file.

So the bad news: whether you're working on Windows or on a Mac, you're going to need to use the command line to run the utility.  Fire up Command Prompt (Windows) or Terminal (MacOS) to get yourself a shell.

So the really bad news: If you're using a Mac, the binary you need to run is actually inside the app package mkvmerge.app.  If you're on Windows, drag the 32 or 64 bit version of mkvmerge.exe into Command Prompt to get thing started; if you're on MacOS, right click on mkvmerge.app, select "Show Package Contents", and drag the binary file ./Contents/MacOS/mkvmerge into Terminal to get started:

Right click on mkvmerge.app and select "Show Package Contents"

Drag the mkvmerge binary into Terminal

The README.md file includes some important instructions and the default syntax to run the tool, with the assumption that you're using the Sony BVM-X300 and mastering in SMPTE ST.2084.  I've copied the relevant syntax here (I'm using a Mac; delete anything in bold before copying the command over, and replace the file paths in the **s with your content:)

./hdr_metadata-master/macos/mkvmerge.app/Contents/MacOS/mkvmerge \
-o *yourfilename.mkv* \
--colour-matrix 0:9 \
--colour-range 0:1 \
--colour-transfer-characteristics 0:16 \
--colour-primaries 0:9 \
--max-content-light 0:1000 \
--max-frame-light 0:300 \
--max-luminance 0:1000 \
--min-luminance 0:0.01 \
--chromaticity-coordinates 0:0.68,0.32,0.265,0.690,0.15,0.06 \
--white-colour-coordinates 0:0.3127,0.3290 \

If using a LUT, add the lines
--attachment-mime-type application/x-cube \
--attach-file *file-path-to-your-cube-LUT* \

In all cases end with
*yourfilename.mov*

Beyond the initial call to the binary or executable, the syntax is identical on MacOS and Windows.

The program's full syntax can be found here, but it's a little overwhelming.  If you want to look it up, just focus on section 2.8, which include the arguments we're using here.   The first four arguments set the color matrix (BT.2020 non-constant), color range (Broadcast), transfer function (ST.2084), and color space (BT.2020) by referencing specific index values, which you can find on the linked page.  If you want to use HLG instead of PQ, switch the value of --colour-transfer-characteristics to 0:18, which will flag for ARIB STD-B67.

(Note to the less code savvy: the backslashes at the end of each line allow you to break the syntax across multiple lines in the command prompt or terminal window.  You'll need them at the end of every line you copy and paste in, except for the last one)

The rest of the list of video properties should be fairly self explanatory, and match the metadata required by HDR10, which I go over in more detail here.

Now, if you want to include your own SDR cross conversion LUT, you'll need to include the arguments --attachment-mime-type application/x-cube, which tells the program you want to attach a file that's not processed (specifically, a cube LUT), and --attach-file filepath, which is the actual file you're attaching.

If you don't attach your own LUT, YouTube will handle the SDR cross conversion with their own internal LUT.  It's not bad, but personally I don't like the hard clipping above 300 nits and the loss of detail in the reds, but that's largely a personal preference.  See the comparison screenshots below to see how theirs works.

Once you've pasted in all of the arguments and set your input file path, hit enter to let it run and it'll make a new MKV.  It doesn't recompress any video data, just copies it over, so if you gave it ProRes, it'll still be the same ProRes stream but with the included HDR metadata and LUT that YouTube needs to recognize the file.

Overall, it's It's a pretty fast tool, and extremely useful beyond just YouTube applications.  You can see what it's done in this set of screenshots below.  The first is the source ProRes clip, the second is the same after passing it through mkvmerge to add the metadata only, and the third went through mkvmerge to get the metadata and my own LUT:

ProRes 422 UHD Upload Without Metadata Injection

ProRes 422 UHD Upload in MKV File. Derived from the ProRes File above and passed through the mkvmerge tool to add HDR Metadata, but no LUT.

ProRes 422 UHD Upload in MKV file. Derived from the ProRes file above and passed through the mkvmerge tool to add HDR Metadata and including our SDR cross conversion LUT. Notice the increased detail in the brights of the snake skin, and the regained detail in the red flower.


All of us at Mystery Box are extremely excited to see HDR support finally widely available on YouTube.  We've been working in the medium for over a year, and haven't been able to distribute any of our HDR content in a way that consumers would actually be able to use.  But now, there's a general content distribution platform available with full HDR support, and we're excited to see what all creators can do with these new tools!

Written by Samuel Bilodeau, Head of Technology and Post Production

HDR Video Part 5: Grading, Mastering, and Delivering HDR

To kick off our new weekly blog here on mysterybox.us, we’ve decided to publish five posts back-to-back on the subject of HDR video.  This is Part 5: Grading, Mastering, and Delivering HDR.

In our series on HDR so far, we’ve covered the basic question of “What is HDR?”, what hardware you need to see it, the new terms that apply to it, and how to prepare and shoot with HDR in mind.  Arguably, we’ve saved the most complicated subject for last: grading, mastering, and delivering.

First, we’re going to look at setting up an HDR grading project, and the actual mechanics of grading in the two HDR spaces.  Next, we’re going to look at how to prepare cross conversion grades to convert from one HDR space to the other, or from HDR to SDR spaces.  Then, we’re going to look at suitable compression options for master & intermediate files, before discussing how to prepare files suitable for end-user delivery.

Now, if you don’t handle your own coloring and mastering, you may be tempted simply to ignore this part of our series.  I’d recommend you don’t - not just because I’ve taken the time to write it, but I sincerely believe that if you work at any step along an image pipeline, from acquisition to exhibition, your work will benefit from learning how the image is treated in other steps along the way.  Just my two cents.

Let’s dive in.

NOTE: Much of this information will be dated, probably within the next six months to a year or so. As programs incorporate more native HDR features, some of the workarounds and manual processes described here will likely be obsolete.


Pick Your Program

Before diving into the nitty gritty of technique, we need to talk applications.  Built-in color grading tools or plugins for Premiere, Avid, or FCP-X are a no-no.  Until all of the major grading applications have full and native HDR support, you’re going to want to pick a program that offers full color flexibility and precision in making adjustments.

I’m going to run you through my workflow using DaVinci Resolve Studio, which I’ve been using to grade in HDR since October 2015, long before Resolve contained any native HDR tools.  My reasoning here is threefold: one, it’s the application I actually use for grading on a regular basis; two, the tricks I developed to grade HDR in DaVinci can be applied to most other color grading applications; and three, it offers some technical benefits that we find important to HDR grading, including:

  • 32 bit internal color processing

  • Node based corrections offering both sequential and parallel operations

  • Color space agnostic processing engine

  • Extensive LUT support, including support for multiple LUTs per shot

  • Ability to quickly apply timeline & group corrections

  • Extensive, easily accessible correction toolset with customizable levels of finesse

  • Timeline editing tools for quick edits or sequence changes

  • Proper metadata inclusion in QuickTime intermediate files

Now, I’m not going to say that DaVinci Resolve is perfect.  I have a laundry list of beefs that range from minor annoyances to major complaints (but the same is basically true for every program that I’ve used…), but for HDR grading its benefits outweigh its drawbacks.

My philosophy tends to be that if you can pretty easily make a program you’re familiar with do something, use that program.  So while we’re going to look at how to grade in DaVinci Resolve Studio, you should be able to use any professional color grading application to achieve similar results, by translating the technique of the grade into that application’s feature set.*

If you are using DaVinci Resolve Studio, I recommend upgrading to version 12.5.2 or higher, for reasons I’ll clarify in a bit.

DaVinci Resolve Studio version 12.5.2 has features that make it very useful for HDR grading and encoding.


Grading in HDR

So now that we’re clear on what we need in a color grading program, let’s get to the grading technique itself.  For starters, I’m going to focus on grading with the PQ EOTF rather than HLG, simply because there’s a lot of overlap between the two.  The initial subsections will focus on PQ grading, but I’ll conclude the section with a bit about how to adapt the advice (and your PQ grade!) to grading in HLG.

Set up the Project

I assume, at this point, that you’re familiar with how to import and set up a DaVinci Resolve Studio project for normal grading using your own hardware, adding footage, and importing the timeline with your sequence.  Most of that hasn’t changed, so go ahead and set up the project as usual, and then take a look at the settings that need to be different for HDR.

First, under your Master Project Settings you’re going to want to turn on DaVinci’s integrated color management by changing the Color Science value to “Davinci YRGB Color Managed”.  Enabling DaVinci’s color management allows you to set the working color space, which as of Resolve Studio 12.5.2 and higher will embed the correct color space, transfer function, and transform matrix metadata to QuickTime files using ProRes, DNxHR, H.264, or Uncompressed codecs.  As more and more applications become aware of how to move between color spaces, especially BT.2020 and the HDR curves, this is invaluable.

Enabling DaVinci YRGB Color Management as a Precursor for HDR Grading

Side note: I’m actually not recommending using their color management for input color space transformations; in fact, for my HDR grades, I actually set the input to “bypass” and the timeline and output color space values to the same values, because I don’t like how these transformations affect how basic grading operations act.  Color management is however a useful starting point for HDR and SDR cross conversions, which we’ll discuss in a bit.

Once color management is turned on, you’ll want to set it up for the HDR grade.  Move to the Color Management pane of the project settings and enable setting “Use Separate Color Space and Gamma”.  This will give you fine-tuneable controls over the input, timeline, and output values.  If you want to keep these flat, i.e. preventing any actual color management by DaVinci, set the Input Color Space to “Bypass” and the Timeline and Output Color Space to “Rec.2020” - “ST.2084”.  This will enable the proper metadata in your renders without affecting any grading operations.

For the purposes of what I’m demonstrating here, if you are using DaVinci’s color management for color transformations, use these settings:

  • Input Color Space - <”Bypass,” Camera Color Space or Rec 709> - <”Bypass,” Camera Gamma or Rec 709>

  • Timeline Color Space - “Rec.2020” - “ST.2084”

  • Output Color Space - “Rec.2020” - “ST.2084”

DaVinci Resolve Studio for embedding HDR metadata in master files, without affecting overall color management.

NOTE: At the time of this writing DaVinci's ACES doesn’t support HLG at all, or PQ within the BT.2020 color space; in the future, this may be a better option to use, if you’re comfortable grading in ACES.

After setting your color management settings, you’ll want to enable your HDR scopes by flagging “Enable HDR Scopes for ST.2084” in the Color settings tab of the project settings.  This changes the scale on DaVinci’s integrated scopes from 10 bit digital values to a logarithmic brightness scale showing the output brightness of each pixel in nits.

How to Enable HDR Scopes for ST.2084 in DaVinci Resolve Studio 12.5+

DaVinci Resolve Studio scopes in standard digital scale, and in ST.2084 nits scale.

If you’re connected to your HDMI reference display, under Master Project Settings flag “Enable HDR Metadata over HDMI”, and under Color Management flag “HDR Mastering is for X nits” to trigger the HDR mode on your HDMI display.

How to enable HDR Metadata over HDMI to trigger HDR on consumer displays.

If you’re connected to a reference monitor over SDI, set the display’s color space to BT.2020 and its gamma curve to ST.2084 (and its Transform Matrix to BT.2020 or BT.709, depending on whether you’re using subsampling and what your output matrix is).

Settings for enabling SMPTE ST.2084 HDR on the Sony BVM-X300

That’s it for settings.  It’s really that simple.


Adjusting the Brightness Range

Now that we’ve got the project set up properly, we’re going to add the custom color management compensation that will allow the program’s mathematical engine to process changes in brightness and contrast in a way more conducive to grading in ST.2084.

The divergence of the PQ EOTF from a linear scale is pretty hefty, especially in the high values.  Internally, the mathematical engine operates on the linear digital values, with a slight weighting towards optimization for Gamma 2.4.  What we want to do is make the program respond more uniformly to the brightness levels (output values) of HDR, rather than to the digital values behind them (input values).

We’re going to do this by setting up a bezier curve that compresses the lights and expands the darks:

Bezier curve for expanding the darks and compressing the whites of ST.2084, for grading with natural movement between exposure values in HDR

For best effect, we need to add the curve to a node after the rest of the corrections, either as a serial node after other correctors on individual clips, on the timeline as a whole (timeline corrections are processed in serial, after clip corrections), or exported as a LUT and attached to the overall output.

Where to attach the HDR bezier curve for best HDR grading experience - serial to each clip, or serial to all clips by attaching it to the timeline.

So what effect does this have on alterations?  Look at the side by side effect of the same gain adjustment on the histogram with and without the custom curve in serial:

Animated GIF of brightness adjustments with and without the HDR Bezier Curve

Without the curves, the upper range of brightnesses race through the HDR brights.  This is, as you can imagine, very unnatural and difficult to control.  On the other hand, the curve forces the bright ranges to move more slowly, still increasing, but at a pace that’s more comparable to a linear adjustment of brightnesses, rather than a linear adjustment of digital values: exactly what we want.

NOTE: DaVinci Resolve Studio includes a feature called “HDR Mode”, accessible through the context menu on individual nodes, that in theory is supposed to accomplish a similar thing.  I’ve found it has really strange effects on Lift - Gamma - Gain that I can’t figure out how is supposed to help HDR grading: Gain races faster through the brights, Gamma is inverted and seems to compress the space, and so does Lift, but at different rates.  If you’ve figured out how to make these controls useful, let me know…

If you've figured out how to use HDR Mode in DaVinci Resolve Studio for HDR grading, let me know!

Once that curve’s in place, grading in HDR becomes pretty normal, in some ways even easier than grading for SDR.  But there are a few differences that need to be noted, and a couple more tricks that will get your images looking the best.  And the first one of these we’ll look at is the HDR frenemy, MaxFALL.


Grading with MaxFALL

If you read the last part in this HDR series about shooting for HDR, you’ll remember that MaxFALL was an important consideration when planning the full scene for HDR.  In color grading you’re likely going to discover why MaxFALL is such an important consideration: it can become frustratingly limiting to what you think you want to do.

Just a quick recap: MaxFALL is the maximum frame average light level permitted by the display.  We calculate each frame average light level by measuring the light level, in nits, of each pixel and taking the average across each individual frame.  The MaxFALL value is the maximum encoded within an HDR stream, or permitted by a display.  The MaxFALL permitted by your reference or target display is what we really need to think about with respect to color grading.

Without getting into the technical reasons behind the MaxFALL, you can imagine it as collapsing all of the color and brightness within a frame into a single, full frame, uniform grey screen, and the MaxFALL is how bright that grey (white) screen can be before the display would be damaged.  Every display has a MaxFALL value, and will hard-limit the overall brightness by dimming the overall image when you send it a signal that exceeds the MaxFALL.

Average Pixel Brightness with Accompanying Source Image

On the BVM-X300, you’ll notice the over range indicator turns on when you exceed the MaxFALL, so that when you look at the display, you can see when the display is limiting the brightness.  On consumer television displays, there is no such indicator, so if the dimming happens when you’re looking away from the screen, you’re likely to not notice the decreased range.  Use the professional reference when it’s available!

BVM-X300 Over Range Indicator showing MaxFALL Exceeded

Just like with CRT displays, the MaxFALL tends to be lower on reference displays than on consumer displays with the same peak brightness, since the size of the consumer displays often reduces the damage produced through the heat generated from the higher current, and the tolerable color deviation in consumer displays allows for lower color fidelities with higher MaxFALLs than a reference display.

So what do we do in grading that can be limited by the MaxFALL attribute?  Here are some scenarios that I’ve run into limitations with MaxFALL:

  1. Bright, sunny, outdoors scenes

  2. Large patches of blue skies

  3. Large patches of white clouds

  4. Large patches of clipped whites

  5. Large gradients into the brightest whites

When I first started grading in HDR, running into MaxFALL was infuriating.  You’re working at a nice clip, when suddenly, no matter how much I raise the brightness of the scene, it just never got brighter!  I didn’t understand initially, since I was looking at the scopes and I was well below the peak brightness available on my display, yet every time I added gain, the display bumped up, then suddenly dimmed down.

When MaxFALL is exceeded, the Over Range indicator lights up and the display brightness is notched down to prevent damage.

Now that I know what I was fighting against, it’s less infuriating, but still annoying.  In generally, I know that I need to keep the majority of the scene around 100-120 nits, and pull only a small amount into the superwhites of HDR.  When my light levels are shifting across frames, as in this grade with the fire breather, I’ll actually allow a few frames to exceed the display’s MaxFALL temporarily, so long as it’s very, very brief, so as not to damage the display when it temporarily averages brighter.

Grading with brief exceeding of target MaxFALL.

When I’m grading content that’s generally bright, with long sets of even brighter, such as this outdoor footage from India, it can be a good idea to keyframe an upper-brightness adjustment to drop the MaxFALL, basically dropping the peak white values as the clipped or white patch takes up more and more of the scene.  This can be visible, though, as a greying of the whites, so be careful.

Tilt-up Shot of Taj Mahal where brightness keyframes were required to limit MaxFALL. In an ideal world, no keyframes would have been necessary and the final frame would have been much brighter (as shot) than the first.

In other cases, it may be necessary to drop the overall frame brightness, to allow for additional peak brightness in a part of the frame, such as what happened with this shot of Notre Dame Cathedral, where I dropped the brightness of the sky, tree, and cathedral to less than what I wanted to allow the clouds to peak higher into the HDR white range.

Average brightness was limited so that more of the cloud details would push higher into the superwhites without exceeding MaxFALL

In some cases, you really have no choice but to darken the entire image and reduce the value of peak white, such as this shot of the backflip in front of the direct sun - the gradient created nearby the sun steps if I pull the center up to the peak white of the sun, while the MaxFALL is exceeded if I pull up the overall brightness of the image.

MaxFALL limited the white point to only 200 nits because of the quantity of the bright portion of the image and the softness of the gradient around the sun.

The last consideration with MaxFALL comes with editing across scenes, and is more important when maintaining consistency across a set of shots that should look like they’re in the same location.  You may have to decrease the peak white within the series of shots so that on no edit does the white suddenly appear grey, or rather, ‘less white’ than the shot before it.

Three shots with their possible peak brightnesses (due to MaxFALL limitations of the BVM-X300) vs the values I graded them at.

What do I mean by ‘less white’?  I mentioned it in Part 4: Shooting for HDR, but to briefly reiterate and reinforce:


In HDR grading, there’s no such thing as absolute white and black.


HDR Whites & Blacks

From a grading paradigm point of view, this may be the biggest technical shift: in HDR, there is no absolute white or absolute black.

Okay, well, that’s not entirely true, since there is a ‘lowest permitted digital code’ which is essentially the blackest value possible, and a ‘highest permitted digital code’ the can be called the peak brightness - essentially the whitest value possible within the system (encoded video + display).  However, in HDR, there is a range of whites available through the brights, and a range of blacks available through the darks.

Black and white have always been a construct in video systems, limited by the darkest and brightest values displays could produce.  There were the hard-coded limits of the digital and voltage values available.  In traditional SDR color grading, crushing to blacks was simply: push the darks below the lowest legal dark value, and you have black.  Same thing with whites - set the brightness to the highest legal value and that was the white that was available: anything less than that tends to look grey, especially in contrast with ‘true white’ or ‘legal white’.

But in the real world, there is a continuum that exists between blacks and whites.  With the exception of a black hole, there is nothing that is truly ‘black’, and no matter how bright an object is, there’s always something brighter, or whiter than it.

Of course, that’s not how we see the world - we see blacks and whites all around us.  Because of the way that the human visual system works, we perceive as blacks any part of a scene (that is, what is in front of our eyes) that is either very low in relative illumination and reflects all wavelengths of light relatively uniformly, or that is very low in relative illumination such that few of our cones are activated in our eyes and we therefore can’t perceive the ratio of wavelengths reflected with any degree of certainty.  Or, in other words, everything that is dark with little saturation, or so dark that we can’t see the saturation, we perceive as black.

The same thing is true with whites, but in reverse.  Everything illuminated or emitting brightness beyond a specific value, with little wavelength variance (or along the normal distribution of wavelengths) we see as white, or if things are so bright that we can’t differentiate between the colors reflected or emitted, we see it as white.

Why do I bring this up?  Because unlike in SDR video where there is a coded black and coded white, in HDR video, there are ranges of blacks and whites (and colors of blacks and whites), and as a colorist you have the opportunity to decide what level of whiteness and blackness you want to add to the image.

Typically, any area that’s clipped should be pushed as close as possible to the scene-relative white level where the camera.  Or, in other words, as high as possible in a scene with a very large range of exposure values, or significantly lower when the scene was overexposed and therefore clipped at a much lower relative ratio.

Clipping in an image with wide range of values and tones vs clipping in image with limited range of values and tones

Since this is different for every scene and every camera, it’s hard to recommend what that level should be.  I usually aim for the maximum value of the display or the highest level permitted by MaxFALL if my gradient into the white or size of the clipped region won’t permit it to be brighter.

So long as the light level is consistent across edits, the whites will look the same and be seen as white.  If, within a scene, you have to drop the peak brightness level of one shot because of MaxFALL or other considerations, it’s probably going to look best if you drop the brightness level of the whites across every other shot within that same scene.  In DaVinci, you can do this quickly by grouping your shots and applying a limiting corrector (in the Group Post-Clip, to maintain the fidelity of any shot-based individual corrections).

Sometimes you may actually want a greyer white, or a colored white that reads more blue or yellow, depending on the scene.  In fact, when nothing within the image is clipping and you don’t have other MaxFALL considerations, it’s very liberating to decide the absolute white level within an image.  Shots without any ‘white’ elements can still have colored brights at levels well above traditional white, which helps separate the relative levels within a scene in a way that could not be possible with traditional SDR video.

The only catch, and this is a catch, is that when you do an SDR cross conversion, some of that creativity can translate into gross looking off-whites, but if you plan specifically for it in your cross conversion to SDR, you should be able to pull it off in HDR without any issues.

Blacks have a similar tonal range available to them.  You have about 100 levels of black available below traditional SDR’s clipping point, and that in turn creates some fantastic creative opportunities.  Whole scenes can play out with the majority of values below 10 nits.  Some parts of the darks can be so dark that they appear uniform black, until you block out the brighter areas of the screen and suddenly find that you can see even deeper into the blacks.  Noise, especially chromatic noise, disappears more in these deep darks, making the image appear cleaner than it would in SDR.  All of these offer incredible creative opportunities when planning for production, and I discussed them in more detail in Part 4: Shooting for HDR.

So how do you play with these whites and blacks?

The two tools I use on a regular basis to adjust my HDR whites and blacks are the High and Low LOG adjustments within DaVinci.  These tools allow me to apply localized gamma corrections to specific parts of the image, that is, those above a specific value for the highs adjustment, and those below a specific value for the lows adjustment.

DaVinci Resolve Studio's LOG Adjustment Panel

In SDR video, I typically use LOG adjustments on the whites to extend contrast, or to adjust the color of the near-whites.  In HDR, I first adjust the “High Range” value to ‘bite’ the part of the image that I want, and then pull it towards the specific brightness value I’m looking for.  This often (but not always) involves pulling up a specific part of the whites (say, the highlights on the clouds) to a higher brightness value in the HDR range, for a localized contrast enhancement, though I do use it to adjust the peak brightness too.

Effect of LOG Adjustments on an HDR Image with Waveform. Notice the extended details in the clouds.

In SDR video, I’d typically use the low adjustment to pull down my blacks to ‘true black’, or to fix a color shift in the blacks I’d introduced with another correction (or the camera captured). In HDR, I use the same adjustment to bite a portion of the lows and extend them through the range of blacks, increasing the local contrast in the darks to make the details that are already there more visible.

The availability of the LOG toolset is one of the major reasons I have a preference for color grading in DaVinci, and what it lets you do quickly with HDR grading really helps speed up the process.  When it’s not available its functionality is difficult to emulate, with finesse, using tools such as curves or lift-gamma-gain.  Typically, I’ve found it generally requires a secondary corrector limited to a specific color range and then using a gamma adjustment, which is a very inelegant workaround, but one that works.


Futureproofing

Once the grade is nearly finalized, there’s a couple of things that you may consider doing to clean up the grade and make it ‘futureproof’, or, to make sure that things you do now don’t come back to haunt the grade later.

If you’ve been grading by eye, any value above the maximum brightness of your reference display will be invisible, clipped at the maximum display value.  If you’re only ever using the footage internally, and on that display only, don’t worry about making it future proof.  If, however, you’re intending on sharing that content with anyone else, or upgrading your display later, you’ll want to actually add the clip to your grade.

The reasoning here I think is pretty easy to see: if you don’t clip it your video signal, your master will contain information that you can’t actually see.  In the future, or on a different display with greater latitude, it may be visible.

There are a couple of ways of doing this.

One that’s available in DaVinci is to generate a soft-clip LUT in the Color Management of the project settings, setting the top clip value to the 10 bit digital value of your display’s maximum nits value (767, for instance for 1000 nits max brightness display using PQ space).  Once you generate the LUT, attach it to the output and you’ve got yourself a fix.

Generating a Soft Clipping LUT for ST.2084 at 1000 nits in DaVinci Resolve

Alternatively, you can adjust your roll off curve that we’re using for making uniform brightness adjustments so that it comes as close to limiting the maximum displayable value as you can get, by extending the bezier curve into a near flat line that lands at your target maximum

Bezier curve for HDR grading with flatter whites to minimize peak range

But sometimes you may want to leave those values there, so that when the next generation of brighter displays comes around, you may find a little more detail in the lights.  What’s really important here is that you make white white, and not accidentally off-white.

If you’re working with RAW footage that allows you to adjust the white balance later, you may find that where white ‘clipped’ on the sensor isn’t uniform in all three channels.  This can happen too with a grading correction that adjusts the color balance of the whites - you can end up with separate clips in the red, green, and blue channels that may be clipped an invisible on your display, but will show up in the future.

Waveform of clipped whites with separated RGB Channels. This is common with RAW grading with clipped whites at the sensor and the ability to control decoded color temperature.

The simple fix here is to add a serial node adjustment that selects, as a gradient, all values above a specific point, and desaturate the hell out of.  Be careful to limit your range to low saturation values only (so long as they encompass what you’re trying to hit) so that you don’t accidentally desaturate other more intentionally colorful parts of the image that just happen to be bright.

How to fix RGB separated clipped whites: add a serial node with a Hue/Saturation/Luminance restriction to just the whites and reduce their saturation to zero.

Working with Hybrid Log Gamma

Up to this point the grading techniques I’ve been discussing have been centered on grading in PQ space.  Grading in Hybrid Log Gamma is slightly different in a couple of important ways.

As a quick refresher, Hybrid Log Gamma is an HDR EOTF that intends to be partially backwards compatible with traditional gamma 2.4 video.  This is a benefit and a drawback when it comes to HDR grading.

If you have multiple reference displays available, this is an important time to break them out.  Ideally, one display should be set up in HLG with a system gamma of 1.2 (or whatever your target system gamma is), and the second should be set up in regular gamma 2.4.  That way, whatever grading you do you can see the effect immediately on both target systems.  Otherwise you’ll need to flip back and forth between two HDR and SDR modes on a single reference display in your search for ‘the happy medium’.

Grading HLG with two reference displays - one in HDR, one in SDR, to ensure the best possible contrast in both.

Most of the project and grading setup is identical to grading with the PQ EOTF, with the exception of the bezier curve in serial that adjusts the brightness response.  In HLG we don’t want to expand the darks, since the HLG darks are identical to the gamma 2.4 darks, so we want that part of the curve to be more linear, before easing into our compression of the highs.

Bezier curve for HDR grading in Hybrid Log Gamma. This curve replaces the ST.2084 Bezier curve added earlier.

Once that’s in place, the rest of the grading process is similar to grading in PQ.  In fact, you can replace the ST.2084 bezier curve with this curve and your grade should be nearly ready to go in HLG.  The major exception to this is that you still need to regularly be evaluating how the image looks in SDR, on a shot by shot basis.

The biggest complaint I have with grading in HLG is the relative contrast between the HDR and the SDR images.  Because HLG runs up to 5000 nits with its top digital values, if you’re grading in 1000 nits you end up with a white level in the SDR version below the usual peak white.  This often means that the whites in the SDR version look muddied and lower contrast than the same content graded for SDR natively.  This is especially true when the MaxFALL dictates a darker image is necessary and a lower white point is necessary, landing values solidly in the middle ranges of brightness.

Hybrid Log Gamma occasionally has much dimmer and muddied whites, when compared to SDR natively graded footage, due to MaxFALL limitations.

And as if muddied whites weren’t enough, it’s difficult in HLG to find a contrast curve that works for both the HDR and the SDR image: because of how our brains perceive contrast, when the contrast looks right and natural in HDR, it looks flat in SDR because of the more limited dynamic range, while when it looks right in SDR it looks overly contrasty in HDR.

Personally, I find grading in HLG to compounds the minor problems of HDR with the problems of SDR, which I find extremely irritating.  Rather than being happy with the grade, I’m often left with a sense of “It’s okay, I guess”.

But on the other hand, when it’s done, you won’t necessarily have to regrade for other target gamma systems, which is what you have to do when working in PQ.



Cross Converting HDR to HDR & HDR to SDR

Let’s be honest.  A PQ encoded image displayed in standard gamma 2.4 rendering looks disgusting.  The trouble is, we only really want to do the bulk of the grading once, so how can we cheat and make sure we don’t have to regrade every project two or more times?

LUTs, LUTs, and more LUTs!  Also, Dolby Vision.

Dolby Vision is an optional (paid to Dolby) add-in for DaVinci Resolve Studio that allows you to encode the metadata for the SDR cross conversion into your output files.  Essentially, the PQ HDR image is transmitted with metadata that describes how to transform the HDR into a properly graded SDR image.  It’s a nifty process that seeks to solve the dilemma of backwards compatibility.

But I’ve never used it, because we’ve had no need and I don’t have a license.  DaVinci Resolve’s documentation on how to use it with DaVinci is extensive though, and it requires a similar process to doing a standard SDR cross conversion, so take that as you will.  I’ve also heard rumors that some major industry players are looking for / looking to create a royalty-free dynamic metadata alternative that everyone can use as a global standard for transmitting this information - but that’s just a rumor.

For everyone not using Dolby Vision, you’re going to have to render the SDR versions separately from the HDR versions as separate video files.  Here at Mystery Box, we prefer to render the entire HDR sequence as set of clip-separated 12bit intermediate files to make the SDR grade from them, versus adding additional corrector elements to the HDR grade.  This tends to render faster, because you only render from the RAWs once, and make any other post-processing adjustments once instead of on every version.

NOTE: I’m going to cover the reason why later, but it’s important that you use a 12 bit intermediate if you want a 10 bit master, since the cross conversion from PQ to any other gamma system cuts the detail levels preserved by about 2-4 times, or an effective loss of 1-2 bits of information per channel.

When I’m cross converting from PQ in the BT.2020 space to gamma 2.4 in the BT.2020 space, after reimporting and reassembling the HDR sequence (and adding any logos or text as necessary), I’ll duplicate the HDR sequence and add a custom LUT to the timeline.

The fastest way to build this LUT is to use the built-in DaVinci Color Management (set the sequence gamma to ST.2084 and the output gamma to Gamma 2.4) or the HDR 1000 nits to Gamma 2.4 LUT, and then add a gain and gamma adjustment to bring the brightness range and contrast back to where you want it to be.  It’s a pretty good place to start building your own LUT on, and while these tools weren’t available when I started building my first cross conversion LUT, the process they use is nearly identical to what I did.

Using DaVinci Resolve Studio to handle HDR to SDR cross conversion

Using DaVinci Resolve Studio to handle HDR to SDR cross conversion

Once you’ve attached that correction to the timeline, it’s a pretty fast process to run through each shot and simply do minor brightness, contrast, white point, and black point adjustments - Using DaVinci’s built-in LUT / Color Management I can do a full SDR cross conversion for 5 minutes of footage in less than half an hour using this LUT method.  Using my own custom LUT this processes can take less than five minutes.

HDR to SDR Cross Conversions using a Custom LUT vs. using DaVinci Resolve Studio's integrated conversion + brightness adjustment, Image 01

HDR to SDR Cross Conversions using a Custom LUT vs. using DaVinci Resolve Studio's integrated conversion + brightness adjustment, Image 02

HDR to SDR Cross Conversions using a Custom LUT vs. using DaVinci Resolve Studio's integrated conversion + brightness adjustment, Image 03

HDR to SDR Cross Conversions using a Custom LUT vs. using DaVinci Resolve Studio's integrated conversion + brightness adjustment, Image 03

HDR to SDR Cross Conversions using a Custom LUT vs. using DaVinci Resolve Studio's integrated conversion + brightness adjustment, Image 04

Notice the detail loss in the pinks, reds, and oranges because of over saturation in the simple downconversion process (images 01 and 04), the milkiness and hue shifting in the darks (images 02) and the fluorescence of the pinks and skin tones (images 03) with a straight downconversion.  This happens largely in the BT.2020 to BT.709 color space conversion, when colors land outside of the BT.709 gamut.  Building a custom LUT can be a great solution to retain the detail.

After prepping the BT.2020 version, making a BT.709 version, for web or demonstration purposes is incredibly easy.  All that you have to do is duplicate the BT.2020 sequence (this is why I like adding LUTs to timelines, instead of to the output globally) and add an additional LUT to the timeline that does a color space cross conversion from BT.2020 to BT.709.  (Alternatively, change the color management settings).  Since the BT.2020 and BT.709 contrast is the same, all I need to do then is run through the sequence looking for regions where reds, blues, or greens end up out of gamut, and bring those back in.  That’s usually less than 5 minutes for a 5 minute project.

Stacked LUTs on a Timeline to combine transformations.

Cross converting from HLG to PQ is fairly simple, since PQ encompasses a larger range of brightnesses than the HLG range and it can fairly easily be directly moved over with a simple LUT or color management tool; you may want to adjust your low-end blacks to take advantage of the deeper PQ space, but it’s otherwise straightforward.

Cross-grading from PQ to HLG is a different animal altogether.  It’s still faster to work from the intermediate than the RAWs themselves, but it’s more than just a simple LUT or color management solution.  Because of the special considerations for HLG, that its contrast has to look good in both HLG and gamma 2.4, you have a lot more work to do finessing the contrast then when you convert ST.2084 into gamma 2.4.  You’ll also run into issues with balancing the MaxFALL in HLG, which in some cases you’ll just have to ignore.

DaVinci’s built-in color management is actually quite good starting point for cross converting from HLG to PQ or PQ to HLG.  It’s important, though, to be aware of how color management injects metadata into QuickTime files, which I’ll address in a second, so that you don’t accidentally flag the incorrect color space or gamma in your master files.

Using DaVinci Color Management to apply an HLG to ST.2084 cross conversion.

Understanding how LUTs work to handle SDR cross conversions is really important, because until there’s a universal metadata method for including SDR grades with HDR content, which in and of itself would essentially be a version of a shot-by-shot LUT, display manufacturers and content delivery system creators rely on LUTs (or their mathematical equivalent) to convert your HDR content into something that can be shown on SDR displays!


Metadata & DaVinci’s Color Management

If you’re using color management to handle parts of your color space and gamma curve transformations, you’re going to need to adjust the Output Color Space each time you change sequences, to match the targeted space of that timeline (in addition to switching the settings on your reference display).  This is actually the biggest reason I prefer using LUTs over color management - it just becomes a hassle to continually have to reset the color management when I’m grading.

Even if you’re not using the color management to handle color space conversions, you’re going to need to make some changes to the color management settings when rendering out QuickTime masters, so that the correct metadata is included into the master files.

Proper Metadata Inclusion for BT.2020 / ST.2084 QuickTime File, encoded in ProRes 4444 out of DaVinci Resolve Studio.

The settings you use depend when you go to render will depend on whether you’re using color management for the transformation or not.  If you are using color management for the transform, change just the Output Color Space to match the target color space and gamma of the timeline to be rendered.  If you aren’t using color management to handle the color conversion, switch both the Timeline Color Space and the Output Color Space to match your target color space and gamma immediately before rendering the matching timeline.  Again, and unfortunately, you will need to make this adjustment every time you go to render a new sequence.  Sorry, no batch processing.

DaVinci Resolve Studio Color Management Settings for transforming color and adding metadata, and adding metadata only.

Grading in HDR isn’t as hard as it originally seems, once you figure out the tricks that allow the grading system to respond to your input as you would expect and predict.  And despite how different HDR is from traditional video, SDR and HDR cross conversions aren’t as hard as they seems, especially when you’re using prepared LUTs specifically designed for that process.


Mastering in HDR

When it comes to picking an appropriate master or intermediate codec for HDR video files, the simplest solution would always be to pick an uncompressed format with an appropriate per-channel bit depth.  Other than the massive file size considerations (especially when dealing with 4K+ video), there are a few cautions here.  

First, for most of the codecs available today that use chroma subsampling, the transfer matrix that converts from RGB to YCbCr is the BT.709 transfer matrix, and not the newer BT.2020 transfer matrix, which should be used with the BT.2020 color space.  This isn’t a problem per-se, and actually benefits out of date decoders that don’t honor the BT.2020 transfer matrix, even with the proper metadata.  It’s also possible to use the use the BT.2020 transfer matrix and improperly flag the matrix used when working with a transcoding application that requires manual flagging instead of metadata flagging.  At its very worst, it can create a very small amount of color shifting on decode.

As slightly more concerning consideration, however, is the availability of high quality 12+ bit codecs for use in intermediate files.  Obviously any codec using 8 bits / channel only are out of the question for HDR masters or intermediates, since 10 bits are required by all HDR standards.  10 bit encoding is completely fine for mastering space, and codecs like ProRes 422, DNxHR HQX/444, 10 bit DPX, or any of the many proprietary ‘uncompressed’ 10 bit formats you’ll find with most NLEs and color correction softwares should all work effectively.

However, if you’re considering which codecs to use as intermediates for HDR work, especially if you’re planning on an SDR down-grade from these intermediates, 12 bits per channel as a minimum is important.  I don’t want to get sidetracked into the math behind it, but just a straight cross conversion from PQ HDR into SDR loses about ½ bit of precision in data scaling, and another ¼ - ½ bit precision in redistributing the values to the gamma 2.4 curve, leaving a little more 1 bit of precision available for readjusting the contrast curve (these are not uniform values).  So, to end up with an error-free 10 bit master (say, for UHD broadcast) you need to encode 12 bits of precision into your HDR intermediate.

ProRes 4444 / 4444 (XQ), DNxHR 444, 12 bit DPX, Cineform RGB 12 bit, 16 bit TIFFs, or OpenEXR (Half Precision) are all suitable intermediate codecs,** though it’s important to double check all of your downstream applications to make sure that whichever you pick will work later.  Similarly, any of these codecs should be suitable for mastering, with the possibility of creating a cross converted grade from the master later.

I just want to note before anyone actually asks: intermediate and master files encapsulating HDR video are still reeditable after rendering - they can be assembled, cut, combined, etc just like regular video files.  You don’t need to be using an HDR display to do that either - they just look a little flatter on a regular display (except if you’re using HLG).  So long as you don’t pass them through a process that drops the precision of the encoded video, you should be fine to work with them in other applications as usual, though you may want to return to DaVinci to add the necessary metadata to whatever your final sequence ends up being.


Metadata

After you’ve made the master, it’s easy to assume you’re done.  But HDR specifications call for display referenced metadata during encoding of the final deliverable stream, so it’s actually important to record this metadata at the time of creation, if you aren’t handling the final encode yourself.  Unfortunately, currently none of the video file formats have a standardized place to record this metadata.

Your options are fairly limited; the simplest solution is to include a simple text file with a list of attribute:value pairs.

Text file containing necessary key : value pairs for an HDR master file that doesn't provide embedded metadata.

What metadata should you include?  It’s a good idea to include everything that you’d need to include in the standard VUI for HDR transmission:

  • Color Primaries

  • Transfer Matrix

  • Transfer Characteristics (for chroma subsampled video)

  • MaxCLL

  • MaxFALL

  • Master Display

When you’re creating distribution files, each of these values need to be properly set to flag a stream as HDR Video to the decoding display.  It’s possible to guess many of these (color space, transfer matrix, etc) if you’ve been provided with a master file without metadata, but it’s much easier to record and provide this metadata at the time of creation so that no matter how long down the line you come back to the master, none of the information is lost.


Distributing HDR

If you’ve made it this far through the HDR creation process, there should really only be one major question remaining: how do we encode HDR video in a way that consumers can see it?

First, the bad news.  There’s no standardization for HDR in digital cinema yet.  So if your intention is a theatrical HDR delivery, you’re probably need to work with Dolby.  At the moment, they’re the only ones with the actual installations that can display HDR, and they have specialists who will handle that step for you.  For most people, what we want to know is how to get an HDR capable television to display the video file properly.

This is where things get more tricky.

I don’t say that because it’s a necessarily complicated process, but only because there’s no ‘drop in’ solutions that are generally available to do it (other than YouTube, very soon).

There are only three codecs that can, by specification, actually be used for distributing HDR video, HEVC, VP9 and AV1 (AV1 is the successor to VP9), and within these only specific operational modes support HDR.  And of these three, the only real option at the moment is HEVC, simply because HDR televisions support hardware based 10 bit HEVC decoding - it’s the same hardware decoder needed for the video stream of UHD broadcasts.

HEVC encoding support is still rather limited, and finding an application with an encoder that supports all of the optional features needed to encode HDR is still difficult.  Adobe Media Encoder, for instance, supports 10 bit HEVC rendering, but doesn’t allow for the embedding of VUI metadata, which means that the file won’t trigger the right mode in the end-viewer’s televisions.

Unfortunately, there’s only one encoder freely available that gives you access to all of the options you need for HDR video encoder: x265 through FFmpeg.

If you’re not comfortable using FFmpeg through a command line, I seriously recommend downloading Hybrid (http://www.selur.de), which is one of the best, if not the best, FFmpeg frontend I’ve found.

Here are the settings that I typically use for encoding HEVC using FFmpeg for a file graded in SMPTE ST.2084 HDR using BT.2020 primaries on our BVM-X300, at a UHD resolution with a frame rate of 59.94fps:

Profile: Main 10
Tier: Main
Bit Depth: 10-bit
Encoding Mode: Average Bitrate (1-Pass)
Target Bitrate: 18,000 - 50,000 kbps
GOP: Closed
Primaries: BT.2020
Matrix: BT.2020nc
Transfer Characteristics: SMPTE ST.2084
MaxCLL: 1000 nits
MaxFALL: 180 nits
Master Display: G(8500,39850)B(6550,2300)R(35400,14600)WP(15635,16450)L(10000000,1)
Repeat Headers: True
Signaling: HRD, AUD, SEI Info

I’ve only listed the settings that I are different from the default x265 settings, so let me run through what they do, and why I use these values.

First, x265 needs to output a 10-bit stream in order to be compliant with UHD broadcast, SMPTE ST.2084, ARIB STD-B67 or HDR10 standards.  To trigger that mode, that I set the Profile to Main 10 and the Bit Depth to 10-bit.  Unless you’re setting a really high bit rate, or using 8K video, you shouldn’t need a higher Tier than Main.

Next, I target 18 - 50 mbps as an average bitrate, with a 1 pass encoding scheme.  If you can tolerate a little flexibility in the final bitrate, I prefer using this mode, simply because it balances render time with quality, without padding the final result.  If you need broadcast compliant UHD, you’ll need to drop the target bitrate from 18 to 15 mbps, to leave enough headroom on the 20 mbps available bandwidth for audio programs, closed captions, etc.

x265 Main Compression Settings for HDR Delivery (Using Hybrid FFMPEG front-end)

However, I’ve found that 15mbps does introduce some artifacts, in most cases, when using high frame rates such as 50 or 60p.  18 seems to be about the most that many television decoders can handle seamlessly, though individual manufacturers vary and it does depend significantly on the content you’re transmitting.  Between 30 and 50 mbps you end up with a near-lossless encode, so if you happen to know the final display system can handle it, pushing the bitrate up can give you better results.  Above 50 mbps, there are no perceptual benefits to raising the bitrate.

A closed GOP is useful for random seeks and to minimize the amount of memory used by the decoder.  By default, x265 uses a GOP of at most 250 frames, so reference frames can end up being stored for quite some time when using an open GOP; it’s better just to keep it closed.

x265 Frames Compression Settings for HDR Delivery (Using Hybrid FFMPEG front-end)

Next we add the necessary HDR metadata into the Video Usability Information (VUI).  This is the metadata required by HDR10, and records information about your mastering settings, including color space, which HDR EOTF you’re using, the MaxCLL of the encoded video, the MaxFALL of the encoded video (if you’ve kept your MaxFALL below your display’s peak, you can estimate this value using the display’s MaxFALL), and the SMPTE ST.2086 metadata that records the primaries, white point, and brightness range of the display itself.

x265 Video Usability Information Compression Settings for HDR Delivery (Using Hybrid FFMPEG front-end)

This metadata is embedded into the headers of the video stream itself, so even f you change containers the information will still be there.  To make sure that the metadata is stored at regular intervals, and to enable smoother random access to the video stream, the last step is to turn on the option for repeating the headers and to include HRD, AUD, and SEI Info.

x265 Stream Settings for HDR Delivery (Using Hybrid FFMPEG front-end)

The HEVC stream can be wrapped in either a .mp4 or a .ts container; both are valid MPEG containers and should work properly on HDR televisions.  Be aware that it can take a while to get your settings right on the encode; if you’re using Hybrid you may need to tweak some of the settings to get 10-bit HEVC to encode without crashing (I flag on “Prefer FFmpeg” and “Use gpu for decoding” to get it to run stable) - don’t leave testing to the last minute!


Grading, mastering, and delivering HDR are the last pieces you need to understand to create excellent quality HDR video.  We hope that the information in this guide to HDR video will help you to be confident in working in this new and exciting video format.

HDR Video is the future of video.  It’s time to get comfortable with it, because it’s not going anywhere.  The sooner you get on board with it and start working with the medium, the more prepared you’ll be for the forthcoming time when HDR video becomes the defacto video standard.

Written by Samuel Bilodeau, Head of Technology and Post Production


Endnotes


*The rationale behind the technical requirements will become clear over the course of the article.  I would recommend that you look at the documentation for the application you use to make sure it meets the same minimum technical requirements as DaVinci Resolve when grading in HDR.  Most major color grading programs meet most or all of these technical criteria, and it’s always better to grade in the program you know than in the program you don’t.


However, if you are looking to pick a program right off the bat, I’d recommend DaVinci Resolve Studio, primarily since you can learn on regular Resolve level to learn the application and toolset before even having to spend a dime.


** You should always test that these codecs actually perform as expected with HDR in your workflow, even if you’ve used them for other applications in the past.  I’ve run into an issue where certain applications decode the codecs in different ways that have little effect in SDR, but create larger shifts and stepping in HDR.

HDR Video Part 2: HDR Video Reference Hardware

UPDATE 18 December 2017: We've posted a new blog about using Production HDR monitors for grading in HDR.  This puts HDR grading displays in the sub $4,000 USD range.  Read our post about how to do that and what you'll need here.

To kick off our new weekly blog here on mysterybox.us, we’ve decided to publish five posts back-to-back on the subject of HDR video.  This is Part 2: HDR Video Reference Hardware.

In HDR Video Part 1 we explored what HDR video is, and what makes it different from traditional video.  Here in Part 2, we’re going to look at what hardware is needed for proper HDR grading (as of October 2016), and how to partially emulate HDR to get a feel for grading in the standard before investing in a full HDR setup.


New Standard, New Hardware

Alright, first, the bad news.  Professional grade reference displays for HDR are expensive.  And there are only two that are commercially available for purchase*: The Sony BVM-X300 and the Dolby PRM-4220.  Both cover 100% of DCI-P3 space, but the BVM-X300 operates in and covers most of BT.2020, has a 4K resolution, a peak brightness of 1000 nits, and uses OLED panels for more detail through the darks.  The PRM-4220 is an excellent display, but is only 2K in resolution and 600 nits max, though it operates with a 12 bit panel for better DCI reference.

At the time of this writing, these are the only two commercially available HDR reference monitors.

At the time of this writing, these are the only two commercially available HDR reference monitors.

At this time, I can’t find any DCI projectors advertising HDR capabilities, though I think that a bright enough laser projector with the right LUT could emulate one, in a small environment - essentially using the LUT trick I’m going to describe below while using a projector that’s 10x brighter than it should be for the reference environment.  That doesn’t mean they don’t exist, it just means you’ll need to talk to the manufacture directly.  I haven’t tested this, though, so don’t quote me on it.

There's at least one reference display that claims to be HDR compatible, but really isn’t - the Canon DP-V2410.  Frankly, the display is actually gorgeous and comparable to the Sony for color rendering and detail level, but it’s max brightness is only 300 nits and it’s HDR mode downscales the SMPTE ST.2084 0.0001 - 1000 nit range into the 0.01-300 nit range.  This leaves the overall image rather lackluster and less impactful, though you could use it to grade in a pinch, since its response curve is right.  But I wouldn’t, primarily because of MaxFALL, which I’ll cover extensively in Parts 4 and 5.

At Mystery Box we decided to go with the Sony BVM-X300 for our HDR grading.  I can’t praise the look of this display enough, though I do have my gripes (I mean, who doesn’t?), but I’ll save that review for another time.

Sony BVM-X300 (Right) in Mystery Box's grading environment (lights on, for detail clarity)


HDR Video on Consumer Displays

DaVinci Resolve Studio 12.5+ Settings for enabling HDR metadata over HDMI

The most affordable option for grading in HDR is to use an HDR television.  The Vizio Reference series have nice color with a 300 nit peak (in HDR mode), while the LG 2016 OLED HDR series displays have phenomenal color, with max brightness levels approaching 1000 nits.

The catch is, of course, that there is still more variation in the color of the display than in a reference display, so unless you know for certain that you’re going to be exhibiting on that specific display, be cautious when using them to grade.  They also lack SDI inputs, but that’s solvable.

DaVinci Resolve Studio version 12.5+ has an option to inject flags for HDR and BT.2020 into the HDMI output of your DeckLink or UltraStudio hardware.  To grade in HDR using a consumer HDR television with HDMI input, simply hook up the display over HDMI, toggle the option in your DaVinci settings and the display will automatically switch into HDR mode:

If you’re not using DaVinci Resolve Studio 12.5+, or if for whatever reason you have to route SDI out, you can inject the right metadata into the HDMI stream once you’ve converted from SDI to HDMI.  What you’ll need is an Integral by HD Fury.  This box, which is annoyingly only configurable under Windows, will add the right metadata into the HDMI connection between host and device, allowing you to flag any HDMI stream as BT.2020 and HDR.

Marketing shot of Integral by HD Fury, a box that will allow you to manually alter HDMI metadata

BE CAREFUL if you’re using the Integral though.  It can be tempting to use the HDMI output of your computer to just patch your desktop into HDR.  This is a bad idea.  Any interface lines will also be translated into HDR, which will limit the displays overall brightness (because you can’t switch your desktop into HDR mode), and any static elements risk burn-in.  Most HDR displays use OLED panels and OLEDs are susceptible to burn-in!

If you are already using SDI outputs for your I/O, and want to switch to the BVM-X300 or the PRM-4220, you shouldn’t NEED to upgrade your I/O card or box to drive HDR - 10b 4:2:2 works for grading HDR.  You might want to upgrade though if you want higher output resolutions (4K vs 2K/1080), higher frame rates at the higher resolutions (50/60p) or RGB/4:4:4 instead of 4:2:2 Chroma Subsampling.

Everything else should work with your existing color correction hardware.


Emulating HDR Video

Okay, so if you’re not ready to spring for the new reference hardware, but want to emulate HDR just to get a feel for how it works, here’s a trick you can do using a standard broadcast display, or a display like the HP Dreamcolor Z27x (which I used when doing my first tests) to partially emulate HDR.

Use a reference display with native BT.2020 color support, if you can.  If you’re using Rec 709, but still want to get a feel for grading in BT.2020, there’s a fix for that using LUTs, but it’s not elegant.  You can get a feel for the HDR curve in a Rec 709 color space, but you won’t get a feel for how the primaries behave slightly differently, or how saturation works in BT.2020.**

In addition, if possible, try to use a reference display with a 10 bit panel.  There’s no cheat for this one, you either have it or you don’t.  8 bits will give you an idea what you’re looking at, but won’t be as clear as possible.  In many cases it won’t make a difference, you’ll just lose your ability to see specific fine details.

Now, calibrate the display and your environment, to emulate HDR.  Turn your maximum brightness to full (on the Dreamcolor Z27x, it peaks at 250 nits; your display may be different).  Turn off all ambient lighting (as pitch black as possible).  Then, turn down the brightness of the host interface display to the lowest setting that it’s still useable.  Do the same for any video scopes or other external monitoring hardware that may also be hooked up to the output signal.

HP Dreamcolor Z27x HDR Approximation Settings

This should make your reference display the brightest thing in the room, by a factor of 2 to 4x.  This is important.  While the display will still lack ‘umph’, at very least it’ll dominate your perception of brightness.  That’s key to creating the illusion of the HDR effect in this case; without it your screen will just look muted and dull.

HDR Approximation Environment Calibration: Lights off, scopes dimmed, interface display as low as possible while retaining visibility (6%, in this case)

At this point, what we’ve done by adjusting the ambient and display brightness is emulated the greater brightness range of HDR without using a display that pushes into the HDR range.  Next what we need to do is adjust the display’s response so that it matches the HDR curve we want to emulate.  Essentially, we need to eliminate the display’s native gamma curve for either PQ or HLG curve.

DaVinci Resolve Studio's LUTs for scaling HDR into Gamma 2.4 / Gamma 2.6

This is actually pretty easy to do in DaVinci Resolve Studio - DaVinci has a set of 3D LUTs you can attach to your output that will automatically do it for you.  You’ll find them written as “HDR <value in nits> nits to Gamma <target display gamma>” (ex. HDR 1000 nits to Gamma 2.4) for the SMPTE 2084 / PQ curve, and “HLG to Gamma <target display gamma>” (ex. HLG to Gamma 2.2) for the Hybrid Log Gamma curve.

What these LUTs do, essentially, is add a 1/gamma (ex 1/2.4) contrast curve to the output signal, combined with the selected contrast curve, i.e., the one you want to see.  The gamma reciprocal adjustment combines with the display’s native or selected gamma to linearlize the overall signal, as the two curves cancel each other left.  The only contrast curve you’re left with, then, is the HDR contrast curve you’ve added to the signal, the HDR curve being translated into your display’s native or adapted luminance range.**

Using one of these LUTs on your monitor or output will allow your display to respond as if it were operating natively with the HDR curve, though you'll notice that your display is only showing the first 100 nits of HDR curve.  We'll fix that next.

The final step is to calibrate your display’s brightness and contrast.  I add a timeline node and scale the gain and gamma adjustments to bring the full HDR range back into the display's native signal range.  As for adjusting the contrast, though, there’s not much I can say about how to do that, other than to use a reference image or two graded in the target space to calibrate the display until it ‘looks right’.  Here are a couple that I graded in SMPTE 2084 that you can use for calibration:

Mystery Box ST.2084 Calibration Images, normalized for Rec.709. Follow this link to download the DPX and individual reference PNGs.

All of this LUT business and brightness scaling, by the way, is exactly what the Canon DP-V2410 does, it just does it internally with a mode switch instead of needing manual setup.  Don’t get me wrong - in every other respect, the DP-V2410 is an amazing display, but in HDR mode it’s equivalent to this setup for HDR emulation, rather than true HDR performance.*


Emulated HDR vs. True HDR

So how does an emulated HDR display compare to a true HDR reference display?  Well, poorly is an understatement.  It's not terrible, but emulated HDR lacks the power of the true HDR, the ability to grade with lights on and see how your footage holds up through the large punch of the whites.  With an 8 bit panel you’re going to see stepping while grading in an emulated HDR mode, because most of the region you’ll be working in ends up compressed to 50 or so code values.

This compression in the darks means you won’t get a feel for just how deep SMPTE 2084 can go while still preserving details - you can grade whole shots with full detail in the darks and a few hundred levels of contrast, that land between codes 4 and 14 (full range) on a standard 8 bit display (especially an LED or CFL backed LCD).

You’ll also be tempted in this mode to grade ‘hot’, that is, push more into the brights of the image, since you don’t have any actual limits for frame average light levels, like all true HDR displays do.  That’s not necessarily a problem, but you’ll run into trouble if you try to use the footage elsewhere.  You also miss the great psychological response the actual dark and light levels of a true HDR range give you.

So why emulate then?  Well, right now, HDR reference hardware is expensive.  And if you want to practice grading and mastering in HDR, without having to invest in the hardware, emulation is a fantastic place to start.  You’ll get to see how the mids and highs roll into the whites in SMPTE 2084, and develop tricks to make your grading more natural when you make the switch to a proper HDR display.  You may even be able to grade using emulated HDR so long as you have a proper HDR television to internally QC before sending out to a client - assuming your mastering of the HDR file is right, you can check it on a television and make sure it at very least looks good there, contrast and curve wise, before sending it out to a client.

Of course, mastering HDR video is problem in and of itself, but I’m saving it for last, in Part 5 of our series.  First, though, we’re going to look at the new terminology introduced with HDR video, because even if you’ve been working with video for decades, most of this is likely to be new.

Written by Samuel Bilodeau, Head of Technology and Post Production


Endnotes

* The day I went to post this I found Canon had updated their website to include the Canon DP-V2420, which they claim supports full HDR in both the ST.2084 and the HLG specifications, and be Dolby Vision qualified;  I haven't had time to look into these claims.

The Dolby PRM-4220 requires a workaround to get it to operate in an HDR mode.  It can be loaded with a custom gamma curve that can match the HDR EOTF, or you can add a custom LUT that scales the 0.01 - 600 nits of SMPTE ST2084 into gamma 2.4 while operating the display in 600 nits mode.

The Dolby Pulsar and the Dolby PRM-32FHD are both HDR capable displays, operating at 4000 and 2000 nits respectively, but I elected not to mention them because they are not, to the best of my knowledge, generally available for purchase.

** If you’re using the LUT on your output to emulate the HDR curve, but only have a Rec. 709 display and want to get a feel for BT.2020, you may consider using a BT.2020 to Rec. 709 LUT and stacking it with the gamma compensating LUT.  In DaVinci you can do this by adding one LUT to the output, and a second LUT for the monitor, or by attaching one of the LUTs to a global node for a timeline.  As a last resort, you can attach as many LUTs as you want to individual grades. You should be able to do something similar in pretty much all other color grading or mastering softwares like Scratch or Nuke.