cr = α fr + (1 − α) br cg = α fg + (1 − α) bg cb = α fb + (1 − α) bb

Taken from Wiki http://en.wikipedia.org/wiki/Digital_compositing

The basic operation used is known as 'alpha blending', where an opacity value, 'α' is used to control the proportions of two input pixel values that end up a single output pixel.
Consider three pixels;
a foreground pixel, f
a background pixel, b
a composited pixel, c
and
α, the opacity value of the foreground pixel. (α=1 for opaque foreground, α=0 for a completely transparent foreground). A monochrome raster image where the pixel values are to be interpreted as alpha values is known as a matte.
Then, considering all three colour channels, and assuming that the colour channels are expressed in a γ=1 colour space (that is to say, the measured values are proportional to light intensity), we have:

cr = α fr + (1 − α) br
cg = α fg + (1 − α) bg
cb = α fb + (1 − α) bb

Note that if the operations are performed in a colour space where γ is not equal to 1 then the operation will lead to non-linear effects which can potentially be seen as aliasing artifacts (or 'jaggies') along sharp edges in the matte. More generally, nonlinear compositing can have effects such as "halos" around composited objects, because the influence of the alpha channel is non-linear. It is possible for a compositing artist to compensate for the effects of compositing in non-linear space.