Misalignment Between VFX and Color
Color pipeline misalignment between color and vfx departments is a growing issue in post production today. Often the crafts are distinct: different artists, separate departments, each working with their own workflows and approaches.
When it comes time to exchange files between the two, there are often misalignments in color, metadata or formatting. These misalignments can be very costly for a project. Often these issues are an unseen problem until close to the end of a project. Therefore, it is critical to align teams on the color pipeline as soon as possible.
If you’d like to go directly to the resource ($10) for managing color pipelines between color and vfx, click on this link below:
GO DIRECTLY TO THE COLOR PIPELINE PDF HERE
If you’d like to buy me a coffee, click on this link below:
While the two disciplines are distinct historically, the lines between color and vfx have become more and more blurry today. There are a few key overlapping reasons for this.
- The rise of color management
- The availability of color grading tools without expensive hardware
- All digital workflows without physical media like film or tape
- Remote workflows and new display standards.
Each team has their own way of working within the context of these modern changes. Often workflows reflect the methods of the past or how things have traditionally been done. If one team is working in a more traditional approach and another in a completely different modern context, there can be a lot of workflow issues that arise between vfx and color departments.
These differences in workflow can lead to total customization per project.
When each new project is a new workflow, it’s easy for teams can make assumptions. Often they don’t have adequate context of the whole process. Teams and artists make the best decisions they can with the information they have. During post, time is critical so teams usually need to make color critical decisions quickly. On top of that, delivery dates can be difficult to pin down. So the work begins with high hopes, but unclear guidelines.
Two Trains, One Destination
To use a train metaphor, let’s assume vfx and color are two trains leaving the station when work begins. In general, the vfx train will leave first. Vfx usually has a longer journey than color. Eventually the color train will leave from a different station. Eventually they need to meet at the same station. There they can exchange passengers and bring them to their final destination. To make this happen, both trains need a timetable and a map.
This metaphor makes it clear that there needs to be some way to communicate between these two trains of vfx and color departments. So that there is a clear layout of the workflow and process. Without this clear layout, projects can easily become more complicated and difficult to plot out. A map is key to a smooth process without dead ends or backtracking.
Modern Problems and Old Standards
Why have things become so complicated and difficult today as opposed to the past? Complexity used to be handled at the hardware level with engineers. Essentially, the hardware dictated the standards and processes. This is why things used to cost so much and require specific processes. The industry required those limitations.
Today, our digital workflows and powerful computers have largely removed those limitations. However, there hasn’t been a new standard to replace the old. This has led to a wild west of workflow.
This wild west has led to companies managing their own workflows internally. Usually these workflows are dictated by the work that needs to get done. Companies do their best to streamline their own processes with the tools available. Or writing up workflow documentation. While these approaches are good intentioned, they tend to fail at addressing the real roots of the issue.
Most of these approaches are top down and reactive to problems. Then fix after fix is applied to shore up the process. Often this myopic focus on fixing problems becomes like whack a mole. There is always another mole.
Ultimately, the focus on fixing problems entrenches them further. Like a house built on a sandy foundation, problems will inevitably come due to the unsustainable structure. All the fixes, documentation and patches to fix the issues becomes that Achilles Heel that threatens a project’s growth.
“To address these issues, I’ve created a resource that provides structure to support teams. This resource addressed color space and gamma alignment issues head on. That way anyone can reference how files are processed from beginning to end. These choices are usually difficult to track during a project. By having a reference point, teams can avoid getting bogged down by all the thorny workflow issues.“
Teams of any size and scale can use this information. By leveraging information, smaller teams without big facility resources have access to high end workflows without the overhead. These new workflows can seem daunting and overwhelming for those of us just getting started in color and vfx. But there is also so much potential for growth and adaptation.
If you’d like to skip immediately to that resource, click the link below:
GO DIRECTLY TO THE COLOR PIPELINE PDF HERE
Color Pipeline, Color Management and Workflow
A little background is required to understand how and why we need a resource to track color space and gamma transformations during the course of a project. To start, it’s important to understand a few basic concepts. The first is the color pipeline. I will define the color pipeline as all of the color space and gamma transformations from capture through delivery.
Color pipelines include raw file interpretations, camera encodings, LUTs, transforms, working color spaces and delivery formats. During the post process, color evolves from set to a display.
Another important concept within the color pipeline concept is color management. Color management seems simple in post. A color managed project means that the application interprets or transforms colors from one color space to another. The application is managing the color conversions for the user under the hood.
An unmanaged color project means that the application does not interpret or transform colors from one color space to another. The application does not change or affect the color space of the files with unmanaged workflows.
In general, color managed workflows have a working color space and a display color space. Operators transforms files from their source color space to a working color space. Then they transform them into a display color space for viewing.
In contrast, unmanaged color workflows don’t have a particular working color space. Artists manipulate files in whatever form they already exist. There are no transforms between different color spaces. With unmanaged workflows, what you see is what you get.
Scene Referred vs. Display Referred Color Pipelines
Another key concept to understand within color management is scene referred vs. display referred. In simplest terms, scene referred means images formatted and encoded to encapsulate scene information. This scene information can come from real footage or computer generated imagery. Display referred means images formatted and encoded for ONE particular display standard like rec709.
Scene Referred Images
Scene referred images allow us to manipulate light closer to how it occurs in the real world. These images are usually log or linear with larger color gamuts to preserve more range of light and color information. Often these images are a higher bit depth and in a larger container like EXR, DPX or ProRes444. These higher quality formats and encodings contain all that high quality scene information efficiently.
Scene referred images will not look correct on a given rec709 or computer display. They will either look flat if they are log or very contrasty and clipped if they are linear. They need to be converted properly for display viewing.
Example of a Log to Linear node in DaVinci Resolve’s Fusion page
Display Referred Images
Scene referred images are not dependent on a particular display format. In contrast, display referred images look correct on ONE particular display format like rec709 for example. Display referred images have a display based gamma curve. This display based gamma curve is either applied in camera or more commonly today applied in post.
Because display based images target a particular display format, they don’t require the higher bit depths or formatting that scene referred images do. They aren’t preserving scene information, only display based information so often 10-bit encoding in a codec like ProResHQ or DHxHD is sufficient.
Scene Referred Images and Color Management
Scene referred images are essential to color managed workflows. While display referred images look correct on ONE particular display type, scene referred images in conjunction with color management, can look correct on a variety of displays.
In a color managed workflow, color transforms map scene referred images from their input color space into a working color space. The working space is also scene referred. Then color transforms convert the image to a particular target display color space.
Scene referred workflows maintain a connection to how light works in the real world which has many benefits for post production. Artists create work on images within a wider, scene referred space and monitor the images in the target display space.
When artists or teams need to pass files around, they exchange scene referred images. These images don’t contain any baked display color space conversions. Once the team finishes working on scene referred images, they convert them for viewing or deliverables.
Critical Differences Between Display and Scene Referred Workflows
In scene referred workflows, artists use tools that work well with scene referred imagery. For colorists this will usually be log. For vfx artists, often this is linear. During the process, images are monitored via a viewer or output transform that targets a particular display like rec709.
In display referred workflows, artist are typically working in a display format like rec709 without any viewer or output transforms. Their tools work well in that particular display space.
Once artists finish their work in a scene referred space like log or linear, often they are exported in log or linear with that display based viewer transform. Once artists finish their work in a display space like rec709, the work is exported directly in that display space without any need to remove a viewer or output transform.
So for scene referred workflows, it is much more critical to align teams on the color pipeline as there are considerations around working color spaces and display transforms. Display based workflows in contrast are simpler in that way, but also more limited in terms real world, accurate scene based image information.
Real World Color Management
While color management as a concept is simple, it is not so simple in real world workflows. For example, you might be working within an unmanaged color project. And let’s say you want to transform a particular group of shots from one color space to another.
Traditionally a camera LUT would do this for example. But you want to use a color space transform. So you throw on a color space transform to go from the camera color space to a display space. Technically, you are color managing one shot. So even within an unmanaged project, color management plays a role.
Because of the blurry nature of color managed and unmanaged, workflows within applications and between applications can become complicated. A colorist might work within one color management system. And a vfx team might work in a completely different color managed system.
Depending on subjective choices within those systems, it can become a complicated workflow knot. The more complicated the knot, the more difficult it is to untangle towards the end of the process.
Color Management Systems
Color management systems like ACES exist to address these issues. ACES is a color management system that uses the same color transforms between different applications. This helps with alignment between teams using different software. It is a unified system. So in theory, there isn’t as much room for error when exchanging files between applications.
ACEScct with an ACEScg output transform in DaVinci Resolve
ACES can be the entire color pipeline of a given show or film. Or ACES can be only one part of a particular show or department. For example, vfx might use ACEScg for their compositing and the rest of the show is something else. While this type of system can unify workflows, often artists prefer to work within their own color management systems.
Usually an artist’s own color management systems are software specific like DaVinci Wide Gamut and Truelight for Baselight. One piece of software contains that color pipeline and all the math and transforms exist only within that software. Often software companies design the tools in their software to work well with their own color management systems.
So even within a color managed show or feature, there can be a variety of software specific, color managed systems. Each piece of the process can be a self contained system as opposed to one unifying color system like ACES.
Raw Camera Files in Color Pipelines
Within color managed workflows, raw camera files can be tricky. When I say raw files, I mean camera files in a raw format. Raw camera files are not static video. They are data interpreted by software into video. So depending on the software, they translate differently.
Camera files recorded as ProRes444 for example are not raw. They are video. They cannot be changed after shooting like camera raw files. Raw files become video after debayering. Artists can choose a variety of encoding types and color spaces depending on the camera and SDK.
This is an important distinction between the different types of files that cameras record. Often post production people call any source camera file raw. This can be confusing in post. It’s critical to distinguish between recorded raw files and encoded video.
If a camera file isn’t actually raw, a better way to describe flat camera files is “log encoded video.” It may seem like a minor misperception, but this technicality has a ripple effect for workflows. Those misperceptions can lead to real decisions about transcoding, interpretation and formatting.
Operators can adjust raw file settings even after the shoot. Therefore, vfx teams and colorists with access to these files can change those settings independently of each other. On top of that, different software interprets raw files differently. So even with the same SDK settings, images can come out looking different. Misalignment can easily occur with raw files because they are so adaptable.
An Example of a Raw Files in the Color Pipeline
For example, the IPP2 workflow interprets RED files as Log3G10 and Red Wide Gamut. There are also legacy interpretations like RedLogFilm and RedDragon and many others as well as Linear or rec709 based interpretations.
Let’s say a colorist begins with one interpretation of that red file like Log3G10 and Red Wide Gamut. And a vfx artist interprets that red file as linear gamma and some version of sRGB. There will definitely be misalignment when it comes time to integrate the vfx with the color or vice versa.
First, an operator debayers and renders a raw file to a video format. Once this is done, the new video file does not contain color space and gamma metadata anymore. Unless it is manually tagged or added. When artists exchange these files, there is no explicit encoding information.
Without that critical piece of information, artists receiving the files can interpret the files incorrectly. In a color managed system, this will lead to shifts in color and gamma. Therefore, it is critical to track raw file encoding so that all parties are on the same page.
Aligning the Color Pipeline for Color and VFX
This brings us to the crux of our issue: alignment between color and vfx. The most important part of any vfx and color workflow is to establish the starting point input color space and gamma and the ending point color space and gamma output of each team that is part of the finishing process. In other words, producers and artists need to align on the color pipeline before starting the work.
Real World Complications for Color Pipelines
While color pipelines are simple in theory, in the real world they are complicated. Often separate companies handle the vfx and color. There isn’t always a conversation around the color pipeline before the works begins.
Producers are busy, supervisors are busy, artists are busy and often these decisions are automatic. The colorist receives a turnover and starts working right away. The vfx team received prep weeks ago and they are already compositing and sending renders back to editorial.
Then at some point, the colorist needs to color a vfx render and it all falls apart. The color grade applied to the new shot looks completely different for some reason. This leads to confusion for the whole project. There are many calls back and forth, angry artists, and confused producers. It will take more work and time to fix a preventable issue.
Workflows Between Commercial, Episodic and Feature Worlds
In addition, workflows can be wildly different within the film world or commercial world. Every project is different. Sometimes a colorist might only work with camera raw files. Then render those files out in rec709 and finish their part of the process. Other times the colorist might work on EXR ACES files processed by the data prep team. Sometimes vfx companies manage all the camera files. Then create their own plates in whatever formatting they prefer. Then they send them back to the colorist.
Sometimes one company manages all the camera pulls. They distributes files to each part of the team. The files are in a specific color space or overarching color management system like ACES. Even if there is alignment on color space and gamma, images can shift around in gamma or exposure. During the composting process or there might be a levels changes in a render.
Soft Skills, Intentionality and Team Management
It is possible to avoid these issues before any work in done with one important key ingredient: intentionality. Regardless of the type of work, teams need to meet and talk about the color pipeline.
Most of the time, a vfx or color team knows how their own preferred color pipeline. Even with those preferences, sometimes it makes more sense to align to the vfx team or the range of cameras.
And sometimes, preferences can be flexible to ensure a better fit and alignment based on the needs of the project. Once teams have met and discussed the best approach, the next best thing to do is testing the pipeline.
It is one thing to discuss the approach and another to bring it into the real world. Even a discussion about who will be managing the camera files or plate pulls helps immensely. After teams work out all the details, there should be no confusion on any side about file formatting. Or who is doing what with camera files. Or which color spaces to use and when to exchange files during the process.
Every project is different. Each colorist is different. Each vfx person and company is different. Therefore, it is up to that specific team to establish a color workflow that makes sense for the project. That makes sense for the timeline. And that makes sense for the technical resources available.
The team needs to discuss the workflow in detail. For example, pulling camera files, showLUTs, CDLS, raw formatting, file exchange and mattes between color and vfx. Then it becomes easy to make decisions about file formatting and color management.
The Importance of Testing for Teams
Because of this variety, it’s really important to test out the color pipeline early. Often during the heat of battle on jobs, producers and artists begin work as soon as possible jumping right in. Time is always of the essence in post. But jumping in before working out a plan between artists and producers is counter productive in the long term.
Teams that have worked together consistently have their own established color pipelines. Often there is a shorthand that doesn’t require further discussion for every project. Technical directors establish their own specific pipeline for shows or commercial work.
The beauty of scene referred workflows is that there are so many ways to do it. So once teams are comfortable establishing their own workflow, there is room to try new things. And get more integrated between artists or departments.
A Useful PDF Tool for Mapping Out the Color Pipeline
It can feel overwhelming to map out color pipelines for a given project. If a project involves CG, a variety of cameras, a large list of deliverables, lots of artists, etc. things can get out of hand very quickly. Key department heads need to come together for projects like this to make key decisions. While it can feel overwhelming, the best approach is to lay out a guide.
I’ve created a one sheet that makes mapping out the color pipeline much easier and straightforward. This one sheet can help teams looking to easily manage complicated exchanges and deliveries between teams.
This pdf has many color space and gamma drop-downs for cameras, vfx teams, colorist and delivery targets. By aligning before production, each team can add their information and discuss exchange formats.
Color space and gamma is one of the most opaque metadata problems facing high end post production. This PDF solves that particular issue.
As an example, each category in the PDF contains multiple dropdown options for color space and gamma. Below is a sampling of the options.
- Camera Color Spaces
- Arri
- RED
- Sony
- and more
- VFX Shot Pulls Color Spaces
- ACES AP0
- ACES AP1
- LogC3
- LogC4
- VFX Working Color Spaces
- ACEScg
- Linear
- Color Working Color Spaces
- ACEScc
- ACEScct
- Color Deliver Target Color Spaces
- Web
- rec709, gamma 2.2
- Broadcast
- rec709, gamma 2.4
- Theatrical
- P3-DCI, gamma 2.6
- HDR
- P3-D65, ST.2084 1000 nit
- Web
By using the tool with the drop downs, each team can easily see how the other teams are working. They can see what the input and outputs need to be. And how to efficiently share files between teams.
Often this information gets lost in emails, spreadsheets or buried in manuals. By making a pdf, it is easy to reference during the storms of projects. It is a valuable resource for teams on sets around the world and in edit and post bays. Often post supervisors or producers don’t get to see into the color pipeline process.
This resource changes that by providing a window into the actual color pipeline process. If there is misalignment or particular issues with file formatting, referencing this document can ensure a smooth, easy workflow.
The Color Pipeline for Color and VFX PDF
Thanks for checking out the PDF! Please feel free to reach out if you’d like to add more color space or gamma options to the pdf. Currently, it’s easy enough to add your own to the pdf. Please reach out if you have feedback on the one sheet.
To Wrap It All Up
Color pipelines can really make or break projects. Teams don’t often openly discuss them holistically. Artists or teams stay in their own corner and manage their own color workflows.
While there are reasons for this, it’s critical for vfx and color to align on color management. Without this overarching guidance or a roadmap, misalignments need work to correct. This work can add countless hours to projects and lead to artist burnout and frustrated producers.
It is important to bring these issues to the light of day with transparency, testing and communication. Then teams can be confident that their workflows are solid and on the right footing.
Thank You and A New Tip Jar!
Thank you for taking the time to read through this article. I’ve decided to add a tip jar to my current and future posts.
These tips will cover the costs of keeping the site running. And also make it possible for me to continue to write articles and resources. I’d appreciate any amount that you feel the content is worth to you. Thanks so much for reading. I’ll keep the articles coming.
Click on the picture below if you’d like to contribute!
^Click the Picture Above^
I’d like to specially thank Jason Bowdach, a colorist and finishing artist at FOX, and Mark Todd Osborne, a Senior Colorist who runs his own company MTO Color Inc. for their feedback and contributions to this article. Both are amazing colorists and high level professionals in the post community.
In addition to coloring, Jason also builds his own color grading tools here at https://pixeltoolspost.com/. Mark runs his own color grading boutique grading mostly feature films at his company MTO Color Inc. here: https://mtocolor.com/. Check them both out and give them a shout if you run across them out there in the wilds of the internet.
Leave a Reply