From 663415b914d48ea996a9b770f57b14f9ec8136db Mon Sep 17 00:00:00 2001 From: wm4 Date: Mon, 7 Dec 2015 23:41:29 +0100 Subject: vo_opengl: fix issues with some obscure pixel formats The computation of the tex_mul variable was broken in multiple ways. This variable is used e.g. by debanding for moving expansion of 10 bit fixed-point input to normalized range to another stage of processing. One obvious bug was that the rgb555 pixel format was broken. This format has component_bits=5, but obviously it's already sampled in normalized range, and does not need expansion. The tex_mul-free code path avoids this by not using the colormatrix. (The code was originally designed to work around dealing with the generally complicated pixel formats by only using the colormatrix in the YUV case.) Another possible bug was with 10 bit input. It expanded the input by bringing the [0,2^10) range to [0,1], and then treating the expanded input as 16 bit input. I didn't bother to check what this actually computed, but it's somewhat likely it was wrong anyway. Now it uses mp_get_csp_mul(), and disables expansion when computing the YUV matrix. --- video/csputils.h | 2 ++ 1 file changed, 2 insertions(+) (limited to 'video/csputils.h') diff --git a/video/csputils.h b/video/csputils.h index df5da4a1c6..0a9b11af9d 100644 --- a/video/csputils.h +++ b/video/csputils.h @@ -251,6 +251,8 @@ struct mp_cmat { void mp_get_cms_matrix(struct mp_csp_primaries src, struct mp_csp_primaries dest, enum mp_render_intent intent, float cms_matrix[3][3]); +double mp_get_csp_mul(enum mp_csp csp, int input_bits, int texture_bits); + void mp_get_xyz2rgb_coeffs(struct mp_csp_params *params, struct mp_csp_primaries prim, enum mp_render_intent intent, struct mp_cmat *xyz2rgb); void mp_get_yuv2rgb_coeffs(struct mp_csp_params *params, struct mp_cmat *yuv2rgb); -- cgit v1.2.3