stream: turn into a ring buffer, make size configurable

In some corner cases (see #6802), it can be beneficial to use a larger
stream buffer size. Use this as argument to rewrite everything for no
reason.

Turn stream.c itself into a ring buffer, with configurable size. The
latter would have been easily achievable with minimal changes, and the
ring buffer is the hard part. There is no reason to have a ring buffer
at all, except possibly if ffmpeg don't fix their awful mp4 demuxer, and
some subtle issues with demux_mkv.c wanting to seek back by small
offsets (the latter was handled with small stream_peek() calls, which
are unneeded now).

In addition, this turns small forward seeks into reads (where data is
simply skipped). Before this commit, only stream_skip() did this (which
also mean that stream_skip() simply calls stream_seek() now).

Replace all stream_peek() calls with something else (usually
stream_read_peek()). The function was a problem, because it returned a
pointer to the internal buffer, which is now a ring buffer with
wrapping. The new function just copies the data into a buffer, and in
some cases requires callers to dynamically allocate memory. (The most
common case, demux_lavf.c, required a separate buffer allocation anyway
due to FFmpeg "idiosyncrasies".) This is the bulk of the demuxer_*
changes.

I'm not happy with this. There still isn't a good reason why there
should be a ring buffer, that is complex, and most of the time just
wastes half of the available memory. Maybe another rewrite soon.

It also contains bugs; you're an alpha tester now.
This commit is contained in:
wm4
2019-11-06 21:36:02 +01:00
parent abb089431d
commit f37f4de849
13 changed files with 314 additions and 175 deletions

View File

@@ -4196,6 +4196,32 @@ Cache
Currently, this is used for ``--cache-on-disk`` only.
``--stream-buffer-size=<bytesize>``
Size of the low level stream byte buffer (default: 4KB). This is used as
buffer between demuxer and low level I/O (e.g. sockets). Generally, this
can be very small, and the main purpose is similar to the internal buffer
FILE in the C standard library will have.
Half of the buffer is always used for guaranteed seek back, which is
important for unseekable input.
There are known cases where this can help performance to set a large buffer:
1. mp4 files. libavformat may trigger many small seeks in both
directions, depending on how the file was muxed.
2. Certain network filesystems, which do not have a cache, and where
small reads can be inefficient.
In other cases, setting this to a large value can reduce performance.
Usually, read accesses are at half the buffer size, but it may happen that
accesses are done alternating with smaller and larger sizes (this is due to
the internal ring buffer wrap-around).
See ``--list-options`` for defaults and value range. ``<bytesize>`` options
accept suffixes such as ``KiB`` and ``MiB``.
Network
-------