vo_opengl: handle grayscale input better, add YA16 support

Simply clamp off the U/V components in the colormatrix, instead of doing
something special in the shader.

Also, since YA8/YA16 gave a plane_bits value of 16/32, and a colormatrix
calculation overflowed with 32, add a component_bits field to the image
format descriptor, which for YA8/YA16 returns 8/16 (the wrong value had
no bad consequences otherwise).

1 files changed, 2 insertions, 2 deletions

diff --git a/video/csputils.c b/video/csputils.c
index ea83739e8e..922f3ee80f 100644
--- a/video/csputils.c
+++ b/video/csputils.c
@@ -564,8 +564,8 @@ void mp_get_yuv2rgb_coeffs(struct mp_csp_params *params, struct mp_cmat *m)
 
     // Hue is equivalent to rotating input [U, V] subvector around the origin.
     // Saturation scales [U, V].
-    float huecos = params->saturation * cos(params->hue);
-    float huesin = params->saturation * sin(params->hue);
+    float huecos = params->gray ? 0 : params->saturation * cos(params->hue);
+    float huesin = params->gray ? 0 : params->saturation * sin(params->hue);
     for (int i = 0; i < 3; i++) {
         float u = m->m[i][1], v = m->m[i][2];
         m->m[i][1] = huecos * u - huesin * v;