autoconvert: destroy sub filter immediately if reconfiguration is needed

I'm currently not convinced that the way the output_chain is handled as
part of reconfiguration is correct. If there is an event requiring
reconfiguration, such as toggling the use of hwdec, we currently do not
ensure that the filter chain is fully drained first. This creates a
situation where the filter chain is invalidated while the autoconvert's
sub filter (that does the real work) still has a frame to process and
pass on.

As the autoconvert code calls mp_subfilter_drain_destroy(), it returns
early to allow for draining before destroying the subfilter, but that
means the subfilter is still present in its original configuration, and
no actually draining is done before the existing filters reinitialise
themselves.

This leads to a situation where, if a hardware scaling filter is being
used by autoconvert, that filter is still present and responds when
told to reinitialise. But it cannot successfully reinitialise if the
triggering event is disabling hw decoding, as the input frame to the
filter will now be a software frame, so reinit fails, leading to total
failure of the filter chain, which is a fatal error, and we exit.

I think this was never noticed before, because I think it's not
possible for the hwtransfer filter to be active in a situation where
you can dynamically change the state such that the input or output
formats of the output chain are invalidated.

eg: If the autoconverter is activated because of `--vf=format=vaapi`,
it is actually not possible to toggle hwdec off, as the explicit user
filter ensure the hwdec is always present and active.

So, my solution here is to destroy the sub filter, regardless of
whether it needs draining or not. We simply have no opportunity to
drain and reconfigure in the correct order, and we must consider the
remaining frame in the filter as a casualty of the toggling process.

I'm sure there is a more substantial rework of the output_chain
reconfiguration process that could ensure draining before
reconfiguration begins, but my ambitions do not currently extend that
far.

1 files changed, 2 insertions, 2 deletions

diff --git a/filters/f_autoconvert.c b/filters/f_autoconvert.c
index 739a45b496..7f5c054d6f 100644
--- a/filters/f_autoconvert.c
+++ b/filters/f_autoconvert.c
@@ -358,8 +358,8 @@ static void handle_video_frame(struct mp_filter *f)
     }
 
     if (!mp_subfilter_drain_destroy(&p->sub)) {
-        p->in_imgfmt = p->in_subfmt = 0;
-        return;
+        MP_VERBOSE(f, "Sub-filter requires draining but we must destroy it now.\n");
+        mp_subfilter_destroy(&p->sub);
     }
 
     p->in_imgfmt = img->params.imgfmt;