From b5b0350371e25a1a54ab087e078ac24cd5c104e7 Mon Sep 17 00:00:00 2001 From: Philip Langdale Date: Sat, 22 Jun 2019 10:38:44 -0700 Subject: vo_gpu: hwdec_vaapi: Count planes rather than layers in Vulkan interop We saw a segfault when trying to use the intel-media-driver (iHD) rather than the normal intel va driver. This happened because the iHD driver reports P010 (and maybe other formats) with multiple layers to represent the interleaved UV plane. The normal va driver reports one UV layer to match the plane. This threw off my logic which assumed that the number of layers could not exceed the number of planes. There's a way one could fix this in a fully generalised form, but I'm just going to do what the EGL path does and assume that: * Layer 'n' is on Plane 'n' for n < total number of planes * These layers always start at offset 0 on the plane You can imagine ways that these assumptions are violated, but at least the failure will look the same for both EGL and Vulkan paths. --- video/out/hwdec/hwdec_vaapi.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) (limited to 'video') diff --git a/video/out/hwdec/hwdec_vaapi.c b/video/out/hwdec/hwdec_vaapi.c index 8ca2ab134e..5bb5d81614 100644 --- a/video/out/hwdec/hwdec_vaapi.c +++ b/video/out/hwdec/hwdec_vaapi.c @@ -472,7 +472,7 @@ static int mapper_map(struct ra_hwdec_mapper *mapper) if (!ra_get_imgfmt_desc(mapper->ra, mapper->dst_params.imgfmt, &desc)) goto esh_failed; - for (int n = 0; n < p->desc.num_layers; n++) { + for (int n = 0; n < p->num_planes; n++) { if (p->desc.layers[n].num_planes > 1) { // Should never happen because we request separate layers MP_ERR(mapper, "Multi-plane VA surfaces are not supported\n"); -- cgit v1.2.3