sub: find GBRP format automatically when rendering to RGB

This removes the need to define IMGFMT_GBRAP, which fixes compilation
with the current Libav release.

This also makes it automatically pick up a GBRP format with the same bit
width. (Unfortunately, it seems libswscale does not support conversion
to AV_PIX_FMT_GBRAP16, so our code falls back to 8 bit, removing
precision for video covered by subtitles in cases this code is used.)

Also, when the source video is e.g. 10 bit YUV, upsample to 16 bit.
Whether this is good or bad, it fixes behavior with alpha. Although I'm
not sure if the alpha range is really correct ([0,2^16-1] vs.
[0,255*256]). Keep in mind that libswscale doesn't even agree with the
way we do it.

1 files changed, 3 insertions, 3 deletions

diff --git a/video/img_format.c b/video/img_format.c
index 42b4df45ab..3535807993 100644
--- a/video/img_format.c
+++ b/video/img_format.c
@@ -275,12 +275,12 @@ struct mp_imgfmt_desc mp_imgfmt_get_desc(int mpfmt)
     return desc;
 }
 
-// Find a format that is MP_IMGFLAG_YUV_P with the following configuration.
-int mp_imgfmt_find_yuv_planar(int xs, int ys, int planes, int component_bits)
+// Find a format that has the given flags set with the following configuration.
+int mp_imgfmt_find(int xs, int ys, int planes, int component_bits, int flags)
 {
     for (int n = IMGFMT_START + 1; n < IMGFMT_END; n++) {
         struct mp_imgfmt_desc desc = mp_imgfmt_get_desc(n);
-        if (desc.id && (desc.flags & MP_IMGFLAG_YUV_P)) {
+        if (desc.id && ((desc.flags & flags) == flags)) {
             if (desc.num_planes == planes && desc.chroma_xs == xs &&
                 desc.chroma_ys == ys && desc.plane_bits == component_bits &&
                 (desc.flags & MP_IMGFLAG_NE))