Digital compositing

Digital compositing is the process of digitally assembling multiple images to make a final image, typically for print, motion pictures or screen display. It is the digital analogue of optical film compositing .

Mathematics

The basic operation used in digital compositing is known as ‘ alpha blending ‘, where an opacity value, ‘α’, is used to control the proportions of two input pixel values ​​that end up a single output pixel.

As a simple example, suppose two images of the same size are available and they are to be composited. The input images are referred to the foreground image and the background image. Each image consists of the same number of pixels . Composing is performed by mathematically combining information from the corresponding pixels in the image and the result of a third image, which is called the composite image.

Consider three pixels;

  • a foreground pixel, f
  • A pixel background, b
  • a pixel composited

and

  • α, the opacity value of the pixel foreground. (α = 1 for opaque foreground, α = 0 for a completely transparent foreground). A monochrome raster image Where the pixel values are to be construed as alpha values is Known as a matte .

Then, considering all three color channels, and assuming that the color channels are expressed in a color = 1 color space (which is to say, the measured values ​​are proportional to light intensity), we have:

r = α f r + (1 – α) b r
g = α f g + (1 – α) b g
b = α f b + (1 – α) b b

Note that if the operations are performed in a color space where is not equal to 1 then the operation will lead to non-linear effects which can be seen as aliasing artifacts (or ‘ jaggies ‘) along sharp edges in the matte. More, nonlinear compositing can have such effects as “halos” around composited objects, because the influence of the alpha channel is non-linear. It is possible for a compositing artist to compensate for the effects of compositing in non-linear space.

Performing alpha blending is an expensive operation performed on an entire image or 3D scene. If this operation is done in real time there is an easy trick to boost performance.

out = α f in + (1 – α) b in
out = α f in + b in – α b in
out = b in + α (f in – b in )

By simply rewriting the mathematical expression one can save 50% of the multiplications required.

Algebraic properties

When we are in the process of compositing together, it is worthwhile to consider the algebraic properties of compositing operators used. Specifically, the associativity and commutativity determines when repeated calculation can or can not be avoided.

Consider the case when we have layers to the final image: F = A * (B * (C * D)) where A, B, C, D are partially transparent image layers and “*” denotes a compositing operator (with the left layer on the right side). If only layer C changes, we should find a way to avoid re-blending all of the layers when computing F. Without any special considerations, four full-image blends would need to occur. For compositing operators that are commutative , such as additive blending , it is safe to re-order the blending operations. In this case, we might compute T = A * (B * D) only once and simply blend T * C to produce F, a single operation. Unfortunately, most operators are not commutative. However, many are associative, suggesting it is safe to re-group operations to F = (A * B) * (C * D), ie without changing their order. In this case we can compute S: = A * B ounce and save this result. To form F with an associative operator, we need only two additional compositing operations to integrate the new layer S, by computing F: = S * (C * D). Note that this expression indicates compositing with all of the layers below it in one step and then blending all of the layers with the previous result to produce the final image in the second step.

If there is a change in the composition of a composite film, the composativity of a compositing operator can still be exploited to speed up computation through parallelism even when there is no gain from pre-computation. Again, consider the image F = A * (B * (C * D)). Each compositing operation in this expression depends on the next, leading to serial computation. However, associativity can allow us to rewrite F = (A * B) * (C * D) where there are two operations that can not be executed in parallel. In general, we can build a tree of pair-wise compositing operations with a logarithmic in the number of layers.

Software

The most historically significant nonlinear compositing system was Cineon , which operated in a logarithmic color space, which is more closely related to film emulsions (the Cineon system, made by Kodak, is no longer in production). Due to the limitations of processing speed and memory, compositing artists do not usually have the luxury Over time, the limitations are becoming much more significant, and nowadays more precisely in a linear color space, even in cases where the source is a logarithmic color space.

Compositing also includes scaling, retouching and color correction of images.

Node-based and layer-based compositing

There are two radically different digital compositing workflows: node-based compositing and layer-based compositing.

Node-based compositing represents an entire composite as a tree graph, linking media objects and effects in a procedural map, intuitively laying out the progression from source input to final output, and is in fact the way all compositing applications internally handle composites. This kind of compositing interface Allows great flexibility, Including the Ability to modify the parameters of an Earlier image processing step “in context” (while viewing the final composite ). Node-based compositing packages often handle keyframing and time effects poorly, as their workflow does not stem directly from a timeline, as do layer-based compositing packages. Software which incorporates Apple Shake , Blender , Blackmagic Fusion, and The Foundry’s Nuke .

Layer-based compositing represents each media object in a composite as a separate layer within a timeline, each with its own time bounds, effects, and keyframes. All the layers are stacked, one above the next, in any desired order; Abstract: Composed of a composite layer, the composite layer has been rendered in the final composite. Layer-based compositing is very well suited for rapid 2D and limited 3DSuch effects in motion graphics, but becomes more complex composites entailing a large number of layers. A partial solution to this is some programs’ ability to view the composite-order of elements (such as images, effects, or other attributes) with a visual diagram called a flowchart to nest compositions, or “comps,” directly into other compositions, Compositional complexity of the first composition by compositing layers in the future composition, then combining the result with images from the proceeding composition, and so on. An example of this exists in the Adobe After Effects program .

See also

  • Broadcast designer
  • Chroma key
  • Digital asset
  • Digital cinema
  • Digital on-screen graphic (BUG)
  • Gamma correction
  • Graphics coordinator
  • Motion graphic
  • Motion graphic design

Further reading

  • Mansi Sharma; Santanu Chaudhury; Brejesh Lall (2014). Content-aware seamless stereoscopic 3D compositing . Proceedings of the 2014 Indian Conference on Computer Vision Graphics and Image Processing, ACM New York, NY, USA. doi : 10.1145 / 2683483.2683555 .
  • T. Porter and T. Duff, “Compositing Digital Images”, Proceedings of SIGGRAPH ’84, 18 (1984).
  • The Art and Science of Digital Compositing ( ISBN 0-12-133960-2 )