I have verified that the conversion here follows ITU-R BT.601 except for
the offsets 16/256 and 128/256 which should be 16/255 and 128/255
respectively.
I used to following octave script to verify this:
rf = 0.299;
gf = 0.587;
bf = 0.114;
crdiv = 1.402;
cbdiv = 1.772;
M = [ rf, gf, bf ;
-rf / cbdiv, -gf / cbdiv, (1 - bf) / cbdiv;
(1 - rf) / crdiv, -gf / crdiv, -bf / crdiv ];
YCbCr = [ 'Y'; 'Cb'; 'Cr' ];
RGB = [ 'R'; 'G'; 'B' ];
eq = [ ' '; '='; ' ' ];
l = [ ' [ '; ' [ '; ' [ ' ];
r = [ ' ] '; ' ] '; ' ] ' ];
mat = [
sprintf('%9f %9f %9f', M(1,:));
sprintf('%9f %9f %9f', M(2,:));
sprintf('%9f %9f %9f', M(3,:));
];
[ l YCbCr r eq l mat r l RGB r ]
R = inv(M);
mat = [
sprintf('%9f %9f %9f', R(1,:));
sprintf('%9f %9f %9f', R(2,:));
sprintf('%9f %9f %9f', R(3,:));
];
[ l RGB r eq l mat r l YCbCr r ]
[ R(:,1), R(:,2:3) .* (255/224) ]
The final matrix printed is what the shader uses down to +/- one digit,
so at least 7 correct decimals.
Signed-off-by: Pekka Paalanen <pekka.paalanen@collabora.com>
9a59303a4f