It was stupid. The only thing that still effectively used it was
sd_lavc_conv - all other "filters" were the subtitle decoder/renderers
for text (sd_ass) and bitmap (sd_lavc) subtitles.
While having a subtitle filter chain was interesting (and actually
worked in almost the same way as the audio/video ones), I didn't
manage to use it in a meaningful way, and I couldn't e.g. factor
secondary features like fixing subtitle timing into filters.
Refactor the shit and drop unneeded things as it goes.
This affects non-ASS text subtitles (those which go through libavcodec's
subtitle converter), which are muxed with video/audio. (Typically srt
subs in mkv.)
The problem is that seeking in the file can send a subtitle packet to
the decoder multiple times. These packets are interlaved with video,
and thus can't be all read when opening the file. Rather, subtitle
packets can essentially be randomly skipped or repeated (by seeking).
Until recently, this was solved by scanning the libass event list for
duplicates. Then our builtin srt-to-ass converter was removed, and
the problem was handled by fully clearing the subtitle list on each
seek.
This resulted in sub-seek not working properly for this type of file.
Since the subtitle list was cleared on seek, it was not possible to
do e.g. sub-seeks to subtitles before the current playback position.
Fix this by not clearing the list, and intead explicitly rejecting
duplicate packets. We use the packet file position was unique ID for
subtitles; this is confirmed working for most file formats (although
it is slightly risky - new demuxers may not necessarily set the file
position to something unique, or at all).
The list of seen packets is sorted, and the lookup uses binary search.
This is to avoid quadratic complexity when subtitles are added in
bulks, such as when opening a text subtitle file.
In some places, the code has to be adjusted to pass through the packet
file position correctly.
With the FFmpeg subtitle decoder used for _all_ non-ASS text subtitle
format, this code is simply unused now.
Ironically, the FFmpeg subtitle decoder does not handle things correctly
in a bunch of cases. Should it turn out they actually matter, they will
have to hack back.
The extend_event one is a candidate, although even though there were
allegedly files which need it, I couldn't get samples from the user who
originally reported such files. As such, extend_event was only confirmed
to handle trailing events with no (endless) duration like with MicroDVD
and LRC, but FFmpeg "fudges" these anyway, so no special handling is
needed.
This code also had logic to handle seeking with muxed srt subtitles,
which made the sub-seek command work. But this has been broken before
this commit already. Currently, seeking with muxed srt subs will clear
all subtitles, as the broken FFmpeg ASS format output by the libavcodec
subtitle converters does not check for duplicates. Since the subtitles
are all cleared, ass_step_sub() can not work properly and sub-seek can
not seek to already seen subtitles.
This has no reason to be there. Put the functionality into another
function instead. While we're at it, also adjust for possible accuracy
issues with high bit depth YUV (matters for rendering subtitles into
screenshots only).
Fixes a reported sample, that has a sign interrupted by a few frames
(for which --sub-fix-timing would remove the wanted gap).
The list of tags in has_overrides() is taken from libass. It has a
similar function (which even checks whether the tag are within the { }
delimiters). Unfortunately, this function is not public, so we just have
a simpler one which does roughly the same. It doesn't matter that this
function sometimes returns false positives.
I feel like it's better there. Note that there is no reduced
functionality, as bitmaps subs (i.e. not handled by sd_ass.c) were never
fully read on init, and thus never went through sub_read_all_packets().
On the other hand, this might lead to confusion, as --sub-fps etc. will
now also affect muxed subtitles (which makes not much sense).
Instead of messing with the subtitle packet timestamps, do it on output.
We work on the libass event list. If there is an unwanted gap or
overlap, we render the timestamp at another position where there is no
gap or overlap.
This is somewhat more robust, and even works with demuxed subs (to some
degree - depends whether the subs are prefected soon enough).
It's active even for native ASS subs. I wonder if this is a problem with
extended type setting. If it is, the heuristic that tries to avoid
interrupting such cases has to be improved.
While it probably would be ideal to do this after the subtitle decoder,
certain aspects are at least currently handled better in this place.
If a second subtitle is shown, it should be forced to display on top
of the screen. This was working only if --no-ass was passed, because
otherwise the subtitle was rendered normally (i.e. usually on the
bottom).
This was used with --no-sub-ass (aka --no-ass). This option (which is
not yet removed) strips all styling from the subtitles, and renders them
as plaintext only. For some reason, it originally seemed convenient to
reuse all the OSD text rendering code (osd_libass.c). While this was
indeed simple, it had a bad influence on the rest of the code. For
example, it had to decide whether to go through the OSD code path, or
the proper subtitle renderer in sd_ass.c.
Kill the OSD subtitle renderer. Reimplement --no-sub-ass and also
"secondary" subtitles in sd_ass.c. fill_plaintext() contains some rather
minor code duplication with osd_libass.c for setting up a dummy
ASS_Event and escaping the stripped text. Since sd_ass.c already has to
handle "normal" text subtitles, and has code for stripping ASS tags,
this remains all relatively simple.
Remove all the unnecessary crap from the rest of the code.
The FFmpeg can officially not distinguish between unknown subtitle
durations, and subtitle durations being 0. (It documents the value 0
meaning unknown duration.)
In practice, at least the LRC demuxer signals unknown subtitle durations
with a negative value.
Assume negative durations mean unknown duration. Show subtitles with
unknown duration forever. Unless there's a subtitle event following it;
then reset the duration so that it ends on the new subtitle event.
Fixes#2244.
Each subtitle track gets its own decoder instance (sd_ass). But they use
a shared ASS_Renderer. This is done mainly because of fontconfig.
Initializing fontconfig is very slow when using it with memory fonts, so
there's a practical need to cache this memory font state, which is done
by not creating separate ASS_Renderers. This is very dirty and very
evil, but we probably can't get rid of it any time soon.
The shared ASS_Renderer was not properly synchronized. While the program
logic guarantees that only one sd_ass instance is visible at a time,
there are other interactions that require synchronization. In
particular, I suspect concurrent execution of mp_ass_configure_fonts()
and sd_ass.get_bitmaps cause issues in a newer libass development
branch.
So here's a shitty hack that hopefully fixes things, hopefully only
until libass becomes less dependent on fontconfig.
Remove the colorspace-related top-level options, add them to vf_format.
They are rather obscure and not needed often, so it's better to get them
out of the way. In particular, this gets rid of the semi-complicated
logic in command.c (most of which was needed for OSD display and the
direct feedback from the VO). It removes the duplicated color-related
name mappings.
This removes the ability to write the colormatrix and related
properties. Since filters can be changed at runtime, there's no loss of
functionality, except that you can't cycle automatically through the
color constants anymore (but who needs to do this).
This also changes the type of the mp_csp_names and related variables, so
they can directly be used with OPT_CHOICE. This probably ended up a bit
awkward, for the sake of not adding a new option type which would have
used the previous format.
Update options like --sub-text-margin-y at runtime. These are somewhat
of a problem, because ass_set_selective_style_override() (intentionally)
does not override them. This should be fixed in libass (by providing
additional override modes), but for now this will do.
Also change the signature of mp_ass_configure(), so we can get access to
the track.
Additionally, drop the redundant setting of the style Alignment (it's
overwritten by mp_ass_set_style()).
See #1622 (again).
Now --ass-use-margins doesn't apply to normal subtitles anymore. This is
probably the inverse from the mpv behavior users expected so far, and
thus a breaking change, so rename the option, that the user at least has
a chance to lookup the option and decide whether the new behavior is
wanted or not.
The basic idea here is:
- plain text subtitles should have a certain useful defalt behavior,
like actually using margins
- ASS subtitles should never be broken by default
- ASS subtitles should look and behave like plaintext subtitles if
the --ass-style-override=force option is used
This also subtly changes --sub-scale-with-window and adds the --ass-
scale-with-window option. Since this one isn't so important, don't
bother with compatibility.
Instead of parsing the ASS file in demux_libass.c and trying to pass the
ASS_Track to the subtitle renderer, just read all file data in
demux_libass.c, and let the subtitle renderer pass the file contents to
ass_process_codec_private(). (This happens to parse full files too.)
Makes the code simpler, though it also relies harder on the (messy)
probe logic in demux_libass.c.
--ass-style-override=force now attempts to override the 'Default' style.
May or may not work. In some situations it will work, but also mess up
seemingly unrelated things like signs typeset with ASS.
If, for some reason, the subtitle renderer attempts to render a
subtitle before SD_CTRL_SET_VIDEO_PARAMS was called, it passed a
value calculated from invalid values. This can happen with --vf=sub
and --start. The crash happens if 1. there was a subtitle packet that
falls into the timestamp of the rendered video frame, 2. the playloop
hasn't informed the subtitle decoder about the video resolution yet
(normally unneeded, because that is used for weird corner cases only,
so this code is a bit fuzzy), and 3. something actually requests a
frame to be drawn from the subtitle renderer, like with vf_sub.
The actual crash was due to passing NaN as pixel aspect to libass,
which then created glyphs with ridiculous sizes, involving a few
integer overflows and unchecked mallocs.
The sd_lavc.c and sd_spu.c cases probably don't crash, but I'm not
sure, and it's better fix them anyway.
Not bothering with sd_spu.c, this crap is for compatibility and will
be removed soon.
Note that this would have been no problem, had the code checked whether
SD_CTRL_SET_VIDEO_PARAMS was actually called. This commit adds such a
check (although it basically checks after using the parameters).
Regression since 49caa0a7 and 633fde4a.
Since m_option.h and options.h are extremely often included, a lot of
files have to be changed.
Moving path.c/h to options/ is a bit questionable, but since this is
mainly about access to config files (which are also handled in
options/), it's probably ok.
The previous code used the output video's pixel aspect for stretching
purposes, breaking rendering with e.g. -vf scale in the chain. Now
subtitles are stretched using the input video's pixel aspect only,
matching the intentions of the original subtitle author.
First, don't try to seek if the result is 0 (i.e. nothing found, or
subtitle event happens to be exactly on spot).
Second, since we never can make sure that we actually seek to the exact
subtitle PTS (seeking "snaps" to video PTS), offset the seek by 10ms.
Since most subtitle events are longer than 10ms, this should work fine.
This code was made inactive some months ago. At this time it wasn't
entirely clear whether this code was still needed, but now I'm pretty
sure it isn't. Even if it is, it didn't work anymore.
This means the direct libass usage can be removed from command.c, and no
weird hacks for retrieving the ASS_Track are needed.
Also fix a bug when using this feature with ordered chapters.
Actually check the newly added text for whitespace, and not the
uninitialized buffer after it. Also, if an even is only whitespace,
don't add it at all.
sd_ass contains some code that treats subtitle events with duration 0
specially, and adjust their duration so that they will disappear with
the next event.
This is most likely not needed anymore. Some subtitle formats allow
omitting the duration so that the event is visible until the next one,
but both subreader.c as well as libavformat subtitle demuxers already
handle this.
Subtitles embedded in mp4 files (movtext) used to trigger this code. But
these files appear to export subtitle duration correctly (at least
libavcodec's movtext decoder is using this assumption). Since commit
6dbedd2 changed demux_lavf to actually copy the packet duration field,
the code removed with this commit isn't needed anymore for correct
display of movtext subtitles. (The change in sd_movtext is for dropping
empty subtitle events, which would now be "displayed" - libavcodec does
the same.)
On the other hand, this code incorrectly displayed hidden events in .srt
subtitles. See for example the first event in SubRip_capability_tester.srt
(part of FFmpeg's FATE). These intentionally have a duration of 0, and
should not be displayed. (As of with this commit, they are still
displayed in external .srt subs because of subreader.c hacks.)
However, we can't be 100% sure that this code is really unneeded, so
just comment the code. Hopefully it can be removed if there are no
regressions after some weeks or months.
Currently, we are filtering libavformat style ASS packets by checking
whether they are prefixed "Dialogue: ". Unfortunately, comment packets
are demuxed too. These start with "Comment: ", so they are not caught.
Change the filtering, and use the codec ID instead. libavformat uses
"ssa" as codec ID for ASS subtitles, while mpv uses "ass". Also, at
least FFmpeg will change the ASS packet format to the same format mpv
and Matroska use, and identify these with "ass" as codec ID, so this is
works out nicely.
The default style is added by mp_ass_default_track(), but not by
ass_new_track(). Considering this, the previous condition at this point
didn't make much sense anymore: the actual (converted) subtitle format
doesn't matter much for what styling should be applied. What matters is
if the subtitle was originally ASS, or if it was converted to it.
Change the code such that the default style is added if there aren't
any, even after reading sub extradata. (The extradata contains the ASS
header, including the style section.) This might change behavior with
scripts that don't define any styles. The change is either with this
commit or with an earlier commit in this branch, depending on the
situation - there are multiple places where default styles are added
in libass API functions, and it's all a big mess.
Other than with very old or broken files (where different behavior
doesn't matter much), the current code should be pretty safe, though.
