From f30c2c99d16527c045caee90c9f7b2a7a7c85c00 Mon Sep 17 00:00:00 2001 From: wm4 Date: Sun, 1 Dec 2013 20:45:44 +0100 Subject: mp_image: deal with FFmpeg PSEUDOPAL braindeath We got a crash in libavutil when encoding with Y8 (GRAY8). The reason was that libavutil was copying an Y8 image allocated by us, and expected a palette. This is because GRAY8 is a PSEUDOPAL format. It's not clear what PSEUDOPAL means, and it makes literally no sense at all. However, it does expect a palette allocated for some formats that are not paletted, and libavutil crashed when trying to access the non-existent palette. --- video/img_format.c | 5 +++++ 1 file changed, 5 insertions(+) (limited to 'video/img_format.c') diff --git a/video/img_format.c b/video/img_format.c index b3d0c8bd91..9e396832bf 100644 --- a/video/img_format.c +++ b/video/img_format.c @@ -233,6 +233,11 @@ struct mp_imgfmt_desc mp_imgfmt_get_desc(int mpfmt) desc.bytes[p] = desc.bpp[p] / 8; } + // PSEUDOPAL is a complete braindeath nightmare, however it seems various + // parts of FFmpeg expect that it has a palette allocated. + if (pd->flags & (PIX_FMT_PAL | PIX_FMT_PSEUDOPAL)) + desc.flags |= MP_IMGFLAG_PAL; + if ((desc.flags & MP_IMGFLAG_YUV) && (desc.flags & MP_IMGFLAG_BYTE_ALIGNED)) { bool same_depth = true; -- cgit v1.2.3