From f37f4de8496556afaa024e39e2efb433eb1680d4 Mon Sep 17 00:00:00 2001 From: wm4 Date: Wed, 6 Nov 2019 21:36:02 +0100 Subject: stream: turn into a ring buffer, make size configurable In some corner cases (see #6802), it can be beneficial to use a larger stream buffer size. Use this as argument to rewrite everything for no reason. Turn stream.c itself into a ring buffer, with configurable size. The latter would have been easily achievable with minimal changes, and the ring buffer is the hard part. There is no reason to have a ring buffer at all, except possibly if ffmpeg don't fix their awful mp4 demuxer, and some subtle issues with demux_mkv.c wanting to seek back by small offsets (the latter was handled with small stream_peek() calls, which are unneeded now). In addition, this turns small forward seeks into reads (where data is simply skipped). Before this commit, only stream_skip() did this (which also mean that stream_skip() simply calls stream_seek() now). Replace all stream_peek() calls with something else (usually stream_read_peek()). The function was a problem, because it returned a pointer to the internal buffer, which is now a ring buffer with wrapping. The new function just copies the data into a buffer, and in some cases requires callers to dynamically allocate memory. (The most common case, demux_lavf.c, required a separate buffer allocation anyway due to FFmpeg "idiosyncrasies".) This is the bulk of the demuxer_* changes. I'm not happy with this. There still isn't a good reason why there should be a ring buffer, that is complex, and most of the time just wastes half of the available memory. Maybe another rewrite soon. It also contains bugs; you're an alpha tester now. --- DOCS/man/options.rst | 26 ++++++++++++++++++++++++++ 1 file changed, 26 insertions(+) (limited to 'DOCS/man') diff --git a/DOCS/man/options.rst b/DOCS/man/options.rst index 8dcbe1bcd8..3ae3264969 100644 --- a/DOCS/man/options.rst +++ b/DOCS/man/options.rst @@ -4196,6 +4196,32 @@ Cache Currently, this is used for ``--cache-on-disk`` only. +``--stream-buffer-size=`` + Size of the low level stream byte buffer (default: 4KB). This is used as + buffer between demuxer and low level I/O (e.g. sockets). Generally, this + can be very small, and the main purpose is similar to the internal buffer + FILE in the C standard library will have. + + Half of the buffer is always used for guaranteed seek back, which is + important for unseekable input. + + There are known cases where this can help performance to set a large buffer: + + 1. mp4 files. libavformat may trigger many small seeks in both + directions, depending on how the file was muxed. + + 2. Certain network filesystems, which do not have a cache, and where + small reads can be inefficient. + + In other cases, setting this to a large value can reduce performance. + + Usually, read accesses are at half the buffer size, but it may happen that + accesses are done alternating with smaller and larger sizes (this is due to + the internal ring buffer wrap-around). + + See ``--list-options`` for defaults and value range. ```` options + accept suffixes such as ``KiB`` and ``MiB``. + Network ------- -- cgit v1.2.3