Only timestamps that enter or leave the demuxer API should be adjusted
by ts_offset (which is usually the start time). queue_seek() is also
used by backward demux seeks, which uses an internal timestamp.
Raw audio formats can be accessed sample-wise, and logically audio
packets demuxed from it would contain only 1 sample. This is
inefficient, so raw audio demuxers typically "bundle" multiple samples
in one packet.
The problem for the demuxer cache and backward playback is that they
need properly aligned packets to make seeking "deterministic". The
requirement is that if you read some packets, and then seek back, you
eventually see the same packets again. demux_raw basically allowed to
seek into the middle of a previously returned packet, which makes it
impossible to make the transition seamless. (Unless you'd be aware of
the packet data format and cut them to make it seamless, which is too
complex for such a use case.)
Solve this by always aligning seeks to packet boundaries. This reduces
the seek accuracy to the arbitrarily chosen packet size. But you can use
hr-seek to fix this. The gain from not making raw audio an awful special
case pays in exchange for this "stupid" suggestion to use hr-seek.
It appears this also fixes that it could and did seek into the middle of
the frame (not sure if this code was ever tested - it goes back to
removing the code duplication between the former demux_rawaudio.c and
demux_rawvideo.c).
If you really cared, you could introduce a seek flag that controls
whether the seek is aligned or not. Then code which requires
"deterministic" demuxing could set it. But this isn't really useful for
us, and we'd always set the flag anyway, unless maybe the caching were
forced disabled.
libavformat's wav demuxer exhibits the same issue. We can't fix it (it
would require the unpleasant experience of contributing to FFmpeg), so
document this in otions.rst. In theory, this also affects seek range
joining, but the only bad effect should be that cached data is
discarded.
This is for uncompressed data, so every frame is a "keyframe". This is
part of making this demuxer work with the demuxer layer caching and
backward playback.
See manpage additions. This is a huge hack. You can bet there are shit
tons of bugs. It's literally forcing square pegs into round holes.
Hopefully, the manpage wall of text makes it clear enough that the whole
shit can easily crash and burn. (Although it shouldn't literally crash.
That would be a bug. It possibly _could_ start a fire by entering some
sort of endless loop, not a literal one, just something where it tries
to do work without making progress.)
(Some obvious bugs I simply ignored for this initial version, but
there's a number of potential bugs I can't even imagine. Normal playback
should remain completely unaffected, though.)
How this works is also described in the manpage. Basically, we demux in
reverse, then we decode in reverse, then we render in reverse.
The decoding part is the simplest: just reorder the decoder output. This
weirdly integrates with the timeline/ordered chapter code, which also
has special requirements on feeding the packets to the decoder in a
non-straightforward way (it doesn't conflict, although a bugmessmass
breaks correct slicing of segments, so EDL/ordered chapter playback is
broken in backward direction).
Backward demuxing is pretty involved. In theory, it could be much
easier: simply iterating the usual demuxer output backward. But this
just doesn't fit into our code, so there's a cthulhu nightmare of shit.
To be specific, each stream (audio, video) is reversed separately. At
least this means we can do backward playback within cached content (for
example, you could play backwards in a live stream; on that note, it
disables prefetching, which would lead to losing new live video, but
this could be avoided).
The fuckmess also meant that I didn't bother trying to support
subtitles. Subtitles are a problem because they're "sparse" streams.
They need to be "passively" demuxed: you don't try to read a subtitle
packet, you demux audio and video, and then look whether there was a
subtitle packet. This means to get subtitles for a time range, you need
to know that you demuxed video and audio over this range, which becomes
pretty messy when you demux audio and video backwards separately.
Backward display is the most weird (and potentially buggy) part. To
avoid that we need to touch a LOT of timing code, we negate all
timestamps. The basic idea is that due to the navigation, all
comparisons and subtractions of timestamps keep working, and you don't
need to touch every single of them to "reverse" them.
E.g.:
bool before = pts_a < pts_b;
would need to be:
bool before = forward
? pts_a < pts_b
: pts_a > pts_b;
or:
bool before = pts_a * dir < pts_b * dir;
or if you, as it's implemented now, just do this after decoding:
pts_a *= dir;
pts_b *= dir;
and then in the normal timing/renderer code:
bool before = pts_a < pts_b;
Consequently, we don't need many changes in the latter code. But some
assumptions inhererently true for forward playback may have been broken
anyway. What is mainly needed is fixing places where values are passed
between positive and negative "domains". For example, seeking and
timestamp user display always uses positive timestamps. The main mess is
that it's not obvious which domain a given variable should or does use.
Well, in my tests with a single file, it suddenly started to work when I
did this. I'm honestly surprised that it did, and that I didn't have to
change a single line in the timing code past decoder (just something
minor to make external/cached text subtitles display). I committed it
immediately while avoiding thinking about it. But there really likely
are subtle problems of all sorts.
As far as I'm aware, gstreamer also supports backward playback. When I
looked at this years ago, I couldn't find a way to actually try this,
and I didn't revisit it now. Back then I also read talk slides from the
person who implemented it, and I'm not sure if and which ideas I might
have taken from it. It's possible that the timestamp reversal is
inspired by it, but I didn't check. (I think it claimed that it could
avoid large changes by changing a sign?)
VapourSynth has some sort of reverse function, which provides a backward
view on a video. The function itself is trivial to implement, as
VapourSynth aims to provide random access to video by frame numbers (so
you just request decreasing frame numbers). From what I remember, it
wasn't exactly fluid, but it worked. It's implemented by creating an
index, and seeking to the target on demand, and a bunch of caching. mpv
could use it, but it would either require using VapourSynth as demuxer
and decoder for everything, or replacing the current file every time
something is supposed to be played backwards.
FFmpeg's libavfilter has reversal filters for audio and video. These
require buffering the entire media data of the file, and don't really
fit into mpv's architecture. It could be used by playing a libavfilter
graph that also demuxes, but that's like VapourSynth but worse.
The demuxer layer can start a thread to decouple the rest of the player
from blocking I/O (such as network accesses). But this particular
function does not support running with the thread enabled. The mutex use
within it is only since thread_work() may temporarily unlock the mutex,
and unlocking an unlocked mutex is not allowed. Most of the rest of the
code still does proper locking, even if it's pointless and effectively
single-threaded.
To make this look slightly cleaner, extend the mutex around the rest of
the code (like threaded code would have to do). This is mostly a
cosmetic change.
The demuxer cache benefits slightly from knowing where the current file
or stream begins. For example, seeking "left most" when the start is
cached would not trigger a low level seek (which would be followed by
messy range joining when it notices that the newly demuxed packets
overlap with an existing range).
Unfortunately, since multimedia is so crazy (or actually FFmpeg in its
quite imperfect attempt to be able to demux anything), it's hard to tell
where a file starts. There is no feedback whether a specific seek went
to the start of the file. Packets are not tagged with a flag indicating
they were demuxed from the start position. There is no index available
that could be used to cross-check this (even if the file contains a full
and "perfect" index, like mp4). You could go by the timestamps, but who
says streams start at 0? Streams can start somewhere at an extremely
high timestamps (transport streams like to do that), or they could start
at negative times (e.g. files with audio pre-padding will do that), and
maybe some file formats simply allow negative timestamps and could start
at any negative time. Even if the affected file formats don't allow it
in theory, they may in practice. In addition, FFmpeg exports a
start_time field, which may or may not be useful. (mpv's internal mkv
demuxer also exports such a field, but doesn't bother to set it for
efficiency and robustness reasons.)
Anyway, this is all a huge load of crap, so I decided that if the user
performs a seek command to time 0 or earlier, we consider the first
packet demuxed from each stream to be at the start of the file. In
addition, just trust the start_time field. This is the "shitty" part of
this commit.
One common case of negative timestamps is audio pre-padding. Demuxers
normally behave sanely, and will treat 0 as the start of the file, and
the first packets demuxed will have negative timestamps (since they
contain data to discard), which doesn't break our assumptions in this
commit. (Although, unfortunately, do break some other demuxer cache
assumptions, and the first cached range will be shown as starting at a
negative time.)
Implementation-wise, this is quite simple. Just split the existing
initial_state flag into two, since we want to deal with two separate
aspects. In addition, this avoids the refresh seek on track switching
when it happens right after a seek, instead of only after opening the
demuxer.
This is a minor benign hack that reorders the MPV_FORMAT_NODE output.
The order of members is not supposed to matter, but it's how the OSD
renders them as raw output. Normally this isn't used, but
demuxer-cache-state is a "prominent" case. Moving the seek ranges to the
end avoids that the more important other fields are not cut off by going
out of the screen on the bottom.
Also output the seek ranges in reverse. The order doesn't matter either
(as declared by input.rst). Currently, the demuxer orders them by least
recent use. Reversing it makes the most recently used range (the current
range) show up on top.
In other words, this commit does basically nothing but fudge stuff in a
cosmetic way to make debugging easier for me, and you've wasted your
time reading this commit message and the diff. Good.
This will properly notify observed properties if the player hasn't
started actual playback yet, such as with --demuxer-cache-wait.
This also happens to cause the main loop more often, which triggers
MPV_EVENT_IDLE, and fixes the OSC display. (See previous commit
message.)
The OSC's (osc.lua) event handling is fundamentally broken. It waits for
MPV_EVENT_TICK to update the UI, and MPV_EVENT_TICK has become entirely
meaningless, except as a hack for the OSC. There are many situations
where the OSC doesn't properly update because the TICK event it expects
isn't sent.
Fix one of them: it doesn't update the cache state if the VO window is
forced and --demuxer-cache-wait is used. Make it so that the tick event
is sent even if playback initialization is taking time.
This is still slightly broken, because it works only if the mainloop is
actually run, which depends on random circumstances (such as moving the
mouse over the VO window). The next commit will add another such
circumstance which will make it appear to work, although it's still
conceptually broken. If we "fixed" it and strictly woke up the player
if the idle timer ran out, we'd send tick events all the time, even
if nothing is going on, which we don't want. Fucking shitshow.
There were 3 packet reading functions: the "old" demux_read_packet()
that blocked (leftover from MPlayer times, but was still used until
recently by some obscure code), the "new" demux_read_packet_async(), and
the special demux_read_any_packet(), that is used by pseudo-demuxers
like demux_edl.
The first two could be used both in threaded and un-threaded mode. This
made 5 cases in total. Some bits of logic was spread across all of them.
Unify the logic. A recent commit made demux_read_packet() private, and
the code for it in threaded mode disappears. The difference between
threaded and un-threaded is minimized.
It's possible that this commit causes random regression. Enjoy.
There are 3 packet reading functions in the demux API, which all
function completely differently. One of them, demux_read_packet(), has
only 1 caller, which is in dec_sub.c. Change this caller to use
demux_read_packet_async() instead. Since it really wants to do a
blocking call, setup some proper waiting. This uses mp_dispatch_queue,
because even though it's overkill, it needs the least code.
In practice, waiting actually never happens. This code is only called on
code paths where everything is already read into memory (libavformat's
subtitle demuxers simply behave this way). It's still a bit of a
"coincidence", so implement it properly anyway.
If suubtitle decoder init fails, we still need to unset the demuxer
wakeup callback. Add a sub_destroy() call to the failure path. This also
happens to fix a missed pthread_mutex_destroy() call (in practice this
was a nop, or a memory leak on BSDs).
This is just redundant and slightly annoying, at least for normal
command line usage. If there are multiple entries, still print it
(because you want to know where you are). Also still print it if the
player was redirected (because you want to know where you got redirected
to).
I'm not sure about this, but it looks like a bug. If a stream didn't
have packets, but the joined range does, the stream should obviously
read the packets added by the joined range. Until now, due to
reader_head being NULL, reading was only resumed if a _new_ packet was
added by actual demuxing (in add_packet_locked()), which means the
stream would suddenly skip ahead, past the original end of the joined
range.
Change it so that it will pick up the new range.
Also, clear the skip_to_keyframe flag. Nothing useful can come from this
flag being set; in the first place, the first packet of a range (that
isn't the current range) should start with a keyframe. Some code
probably enforced it (although it's fuzzy).
Completely untested.
When doing a seek to the end of the cache, ds->skip_to_keyframe can be
set to true. Then some packets passed to add_packet_locked() may have to
be skipped. In some aspects, the skipped packet was still treated as if
it was going to be returned to the reader.
It almost doesn't matter though: it only caused a redundant wakeup_ds()
call, and could pass the packet to the stream recorder. Fix it anyway.
UB-sanitizer complains that we shift bits into the sign (when a is
used). Change it to unsigned, which in theory is more correct and
silences the warning.
Doesn't matter in practice, both the "bug" and the fix have 0 impact.
If a DASH-hack EDL has an init fragment is set, it opens the init
fragment as such to get the track layout (including codec etc.) and
avoids opening actual fragments until actual playback. It does not get
added to the source array, so it leaks on exit, which triggers an
obscure (but very justified) assertion in thread_tools.c:106. Fix the
leak by adding the additional demuxer instance to the sources arrays,
which gets it freed.
This is a regression from when I rewrote some of the timeline handling.
I decided that in order to make memory management slightly simpler,
freeing a timeline should only free elements in the sources array. That
is OK; I just didn't re-test with pseudo-DASH that has init fragments,
and just hit a video that uses that by accidents. These videos are
rather scarce (apparently) so it happened only now.
The real solution would probably be adding demuxer reference counting.
This EDL memory management is just too messy, and throwing refcounting
at such problems is an effective and popular fix. Then you'd get
debugging nightmares with incorrect refcounts too, though.
ytdl_hook.lua essentially uses these headers to implement parts of DASH.
Hopefully the FFmpeg DASH demuxer gets usable at some point, and/or mpv
gets a proper DASH demuxer. In any case, these EDL hacks could get
removed as soon as they get unnecessary and too annoying.
If you have a EDL stream with separate sources for audio and video
stream (like ytdl_hook now creates), you can get the problem that the
video stream seeks to a different position than audio due to different
key frame granularity.
In particular, if you seek backward, the video might undershoot the seek
target by a lot. Then video will resume from an earlier position than
audio, and the player plays silence. This is annoying.
Fix this by explicitly implementing a heuristic to detect separate
audio/video streams, determining where a video seek ends up, and then
seeking the audio stream to the video destination. This also makes sure
to not seek audio with SEEK_FORWARD, so it will always seek before the
video position. Non-precise seeks still skip audio to the video target,
so this helps with ensuring that audio is present at the final seek
target.
The implementation is very annoying, because the only way to determine
the seek target is to actually read a packet. Thus a 1-packet queue
needs to be added. In theory, we could get the seek target from the
index of the video file (especially if it's mp4), but libavformat does
not have public API that exports this index, so we're stuck with this
roundabout generic method.
Note that this is only for non-precise seeks. If precise seeks are done,
the problem is handled by the frontend by skipping unwanted video
frames. But non-precise seeking should still work. (Personally I prefer
non-precise seek mode by default because they're still significantly
faster.)
It also needs to be said that this is the 4th implementation of this
seek adjustment thing in mpv. The 1st implementation is in the frontend
(look for MPContext.seek_slave). This works only if the external audio
stream is known as such on the frontend level. The 2nd implementation is
in the demuxer level packet cache (top of execute_cache_seek()). This is
similar to code that any demuxer needs to handle non-precise seeks
sufficiently nicely. The 3rd is in demux_mkv.c. Since mkv is an
interleaved format, this implementation mostly consists on trying to
pick index entries for video packets if a video stream is selected.
Maybe these "redundant" implementations could be avoided by exposing
separate streams through the demuxer API (and making them individually
seekable) or something like this, but this is messy and not without
problems for multiple reasons. So for now this commit is the best way to
fix the observed behavior.
Instead of just using "edl/" for the file format, report mkv_oc if it's
generated from ordered chapters, "cue/" if from .cue, "multi/" if it's
from EDL but only for adding separate streams, "dash/" if it's from EDL
but only using the DASH hack, and "edl/" for everything else.
The EDL variants are mostly special-cased to the variants the ytdl
wrapper usually generates.
This has no effect other than what the command.c file-format property
returns.
Remove the singly linked list hack, replace it with a slightly more
proper data structure. This probably gets rid of a few minor bugs along
the way, caused by the awkward nonsensical sharing/duplication of some
fields.
Another change (because I'm touching everything related to timeline
anyway) is that I'm removing the special semantics for parts[num_parts].
This is now strictly out of bounds, and instead of using the start time
of the next/beyond-last part, there is an end time field now.
Unfortunately, this also requires touching the code for cue and mkv
ordered chapters. From some superficial testing, they still seem to
mostly work.
One observable change is that the "no_chapters" header is per-stream
now, which is arguably more correct, and getting the old behavior would
require adding code to handle it as special-case, so just adjust
ytdl_hook.lua to the new behavior.
I noticed that some ytdl streams have a start time other than 0. There's
currently no mechanism inside of the EDL stuff that determines this
start time correctly, so it can happen that if the start time is high,
demux_timeline.c tries to clip off the entire video and audio, resulting
in failure of playback.
As a counter measure, use the no_clip header, which entirely disables
clipping against time ranges in demux_timeline.c. (It's basically a
hack.)
Used by the next commit. It mostly exposes part of mp4_dash
functionality. It actually makes little sense other than for ytdl
special-use. See next commit.
E.g. "mpv null:// --demuxer=rawvideo" will "hang" by waiting for video
EOF forever. It's not signalled correctly because of the last-frame
corner case, which attempts to wait until the current frame is finally
displayed (which is signalled by whether a new frame can be queued, see
commit 1a339fa09d for some details). If no frame was ever queued, the VO
is not configured, and vo_is_ready_for_frame() never returns true.
Fix this by using vo_has_frame(), which seems to be exactly the correct
thing we need.
Init fragments are not a necessity for DASH, but this code assumed so.
Maybe the check was to prevent worse. But using normal EDL here leads to
very shitty behavior where it tries to open hundreds or thousands of
fragments, each with its own demuxer and HTTP connection. (This behavior
is fine for normal uses of EDLs, but completely unacceptable when
emulating fragmented streaming protocols. I'm not sure why the normal
EDL code is needed here, but I think someone claimed some obscure sites
just need it.)
This happens in the same situation as the one described in the previous
commit.
Otherwise we'd just use the base URL as media URL, which would fail with
a 404 error.
Not sure if there's a deeper reason why the audio path was explicitly
different from the video one. But this actually works now for a video
that returned fragmented DASH audio with the default format selection.
(This affects streams on that well known site of a big evil Silicon
Valley company. Typically happens after live stream gets converted to a
normal video, though after some time passes, this fragmented version is
deleted, and replaced by a non-fragmented one. I've observed this
several times and this seems to be the "normal" behavior.)
Normal EDL needs to clip packets coming from the underlying demuxer to
the segment range (including complicated stuff due to frame reordering).
This is unwanted In pseudo-DASH mode. A broken or subtly incorrect
manifest would lead to "bad stuff" happening. The intention of the
pseudo-DASH mode is to literally concatenate fragments.
This fixes that there were weird delay ("buffering") when seeking into
the last part of a seekable range. The exact case which triggers it if
SEEK_FORWARD is used, and the seek pts is after the second-last
keyframe, but before the end of the range. In that case,
find_seek_target() returned NULL, and the cache layer waited until the
_next_ keyframe the underlying demuxer returned until resuming playback.
find_seek_target() returned NULL, because the last keyframe had
kf_seek_pts unset. This field contains the lowest PTS in the packet
range from the keyframe until the next keyframe (or EOF). For normal
seeks, this is needed because keyframes don't necessarily have the
minimum PTS in the packet range, so it needs to be computed by waiting
for all packets until the next keyframe (or EOF).
Strictly speaking, this behavior was correct, but it meant that the
caller would set ds->skip_to_keyframe, which waits for the next newly
demuxed keyframe. No packets were returned to the decoder until this
happened, usually resulting in the frontend entering "buffering" mode.
What it really needs to do is returning the last keyframe in the cache.
In this situation, the seek target points in the middle of the last
completely cached packet range (as delimited by keyframes), and
SEEK_FORWARD is supposed to skip to the next keyframe. This is in line
with the basic assumptions the packet cache makes (e.g. the keyframe
flag means it's possible to start decoding, and the frames decoded from
it and following packets will strictly have PTS values above the
previous keyframe range). This means in this situation the kf_seek_pts
value doesn't matter either.
So fix this situation by explicitly detecting it and then returning the
last cached keyframe.
Should the search loop look at all packets, instead of only keyframe
ones? This would mean it can know that it's within the last keyframe
range (without looking at queue->seek_end). Maybe this would be a bit
more natural for the SEEK_FORWARD case, but due to PTS reordering it
doesn't sound like a useful thing to do.
Should skip_to_keyframe be checked by the code that sets kf_seek_pts to
a known value? This wouldn't help too much; the frontend would still go
into "buffering" mode for no reason until the packet range is completed,
although it would resume from the correct range.
Should a NULL return always unconditionally use keyframe_latest? This
makes sense because the seek PTS is usually already in the cached range,
so this is the only case that should happen. But there are scary special
cases, like sparse subtitle streams, or other uses of find_seek_target()
which could be out of range now or in future. Basically, don't "risk"
it.
One other potential problem with this is that the "adjust seek target"
code will be disabled in this case. It checks kf_seek_pts, and if it's
unset, the adjustment is not done. Maybe this could be changed to use
the queue's seek_end time, but I'm not sure if this is fully kosher. On
the other hand, I think the main use for this adjustment is with
backwards seeks, so this shouldn't matter.
A previous commit dealing with audio/video stream merging mentioned how
seeking forward entered "buffering" mode for unknown reasons; this
commit fixes this issue.
demux_timeline doesn't do any transport accesses itself. The slave
demuxers do this (these will actually access the stream layer and
perform e.g. network accesses). As a consequence, demux_timeline always
reported 0 bytes read, and network speed display didn't work.
Fix this by awkwardly reporting the amount of read bytes upwards. This
is not very nice, and requires explicit calls whenever the slave "might"
have read data.
Due to the way the reporting is done, it only works if the slaves do not
run demuxer threads, which makes things even less nice. (Fortunately
they don't anyway, because it would be a waste of resources.) Some
identifiers contain the word "hack" as a warning.
Some of the stupidity comes from the fact that demux.c itself resets the
stats randomly in order to calculate the bytes_per_second value, which
is useless for a slave, but of course is still done, because demux.c
itself is not aware of whether it's on the slave or top-level layer.
Unfortunately, this must do.
In theory, the demuxer thread/cache layer should be separated from
demuxer implementations. This would get rid of all the awkwardness and
nonsense. For example, the only threading involved would be the caching
layer, completely separate from demuxers themselves. It'd be the only
thing calculates speed rates for the player frontend, too (instead of
doing it for each demuxer, even if unused).
It was an ugly hack, and the next commit will make it even uglier.
Slightly reduce the ugliness to prevent death of too many brain cells,
though it's still an ugly hack.
The cleanup is really minor, but I guess the following commit would be
much worse otherwise. In particular, this commit checks accesses
(instead of having a public field with evil access rules), which should
avoid misunderstandings and incorrect use. Strictly speaking, the added
field is redundant, but the next commit complicates it a bit.
I think this is better. On the other hand, this is a behavior change.
The EDL "spec" says that unknown fields are igored. But strictly
speaking, unknown headers are not "fields", but unknown entities.
EDL "headers" were always an afterthought, and kind of hacked on top of
the existing code. Improve it slightly, and make it follow the
conventions of the normal parsing. Basically use the same code structure
for them, just that they use different field names.
This merges separate audio and video tracks into one virtual stream,
which helps the mpv caching layer. See previous EDL commit for details.
It's apparently active for most of evil Silicon Valley giant's streaming
videos.
Initial tests seem to work fine, except it happens pretty often that
playback goes into buffering immediately even when seeking within a
cached range, because there is not enough forward cache data yet to
fully restart playback. (Or something like this.)
The audio stream title used to be derived from track.format_note; this
commit stops doing so. It seemed pointless anyway. If really necessary,
it could be restored by adding new EDL headers.
Note that we explicitly don't do this with subtitle tracks. Subtitle
tracks still have a chance with on-demand loading or loading in the
background while video is already playing; merging them with EDL would
prevent this. Currently, subtitles are still added in a "blocking"
manner, but in theory this could be loosened. For example, the Lua API
already provides a way to run processes asynchronously, which could be
used to add subtitles during playback. EDL will probably be never
flexible enough to provide this. Also, subtitles are downloaded at
once, rather than streamed like audio and video.
Still missing: disabling EDL's pointless chapter generation, and
propagating download speed statistics through the EDL wrapper.
The ytdl wrapper can resolve web links to playlists. This playlist is
passed as big memory:// blob, and will contain further quite normal web
links. When playback of one of these playlist entries starts, ytdl is
called again and will resolve the web link to a media URL again.
This didn't work if playlist entries resolved to EDL URLs. Playback was
rejected with a "potentially unsafe URL from playlist" error. This was
completely weird and unexpected: using the playlist entry directly on
the command line worked fine, and there isn't a reason why it should be
different for a playlist entry (both are resolved by the ytdl wrapper
anyway). Also, if the only EDL URL was added via audio-add or sub-add,
the URL was accessed successfully.
The reason this happened is because the playlist entries were marked as
STREAM_SAFE_ONLY, and edl:// is not marked as "safe". Playlist entries
passed via command line directly are not marked, so resolving them to
EDL worked.
Fix this by making the ytdl hook set load-unsafe-playlists while the
playlist is parsed. (After the playlist is parsed, and before the first
playlist entry is played, file-local options are reset again.) Further,
extend the load-unsafe-playlists option so that the playlist entries are
not marked while the playlist is loaded.
Since playlist entries are already verified, this should change nothing
about the actual security situation.
There are now 2 locations which check load_unsafe_playlists. The old one
is a bit redundant now. In theory, the playlist loading code might not
be the only code which sets these flags, so keeping the old code is
somewhat justified (and in any case it doesn't hurt to keep it).
In general, the security concept sucks (and always did). I can for
example not answer the question whether you can "break" this mechanism
with various combinations of archives, EDL files, playlists files,
compromised sites, and so on. You probably can, and I'm fully aware that
it's probably possible, so don't blame me.
This commit adds an extension to mpv EDL, which basically allows you to
do the same as --audio-file, --external-file, etc. in a single EDL file.
This is a relatively quick & dirty implementation. The dirty part lies
in the fact that several shortcuts are taken. For example, struct
timeline now forms a singly linked list, which is really weird, but also
means the other timeline using demuxers (cue, mkv) don't need to be
touched. Also, memory management becomes even worse (weird object
ownership rules that are just fragile WTFs). There are some other
dubious small changes, mostly related to the weird representation of
separate streams.
demux_timeline.c contains the actual implementation of the separate
stream handling. For the most part, most things that used to be on the
top level are now in struct virtual_source, of which one for each
separate stream exists. This is basically like running multiple
demux_edl.c in parallel. Some changes could strictly speaking be split
into a separate commit, such as the stream_map type change.
Mostly untested. Seems to work for the intended purpose. Potential for
regressions for other timeline uses (like ordered chapters) is probably
low. One thing which could definitely break and which I didn't test is
the pseudo-DASH fragmented EDL code, of which ytdl can trigger various
forms in obscure situations. (Uh why don't we have a test suite.)
Background:
The intention is to use this for the ytdl wrapper. A certain streaming
site from a particularly brain damaged and plain evil Silicon Valley
company usually provides streams as separate audio and video streams.
The ytdl wrapper simply does use audio-add (i.e. adding it as external
track, like with --audio-file), which works mostly fine. Unfortunately,
mpv manages caching completely separately for external files. This has
the following potential problems:
1. Seek ranges are rendered incorrectly. They always use the "main"
stream, in this case the video stream. E.g. clicking into a cached range
on the OSC could trigger a low level seek if the audio stream is
actually not cached at the target position.
2. The stream cache bloats unnecessarily. Each stream may allocate the
full configured maximum cache size, which is not what the user intends
to do. Cached ranges are not pruned the same way, which creates disjoint
cache ranges, which only use memory and won't help with fast seeking or
playback.
3. mpv will try to aggressively read from both streams. This is done
from different threads, with no regard which stream is more important.
So it might happen that one stream starves the other one, especially if
they have different bitrates.
4. Every stream will use a separate thread, which is an unnecessary
waste of system resources.
In theory, the following solutions are available (this commit works
towards D):
A. Centrally manage reading and caching of all streams. A single thread
would do all I/O, and decide from which stream it should read next. As
long as the total TCP/socket buffering is not too high, this should be
effective to avoid starvation issues. This can also manage the cached
ranges better. It would also get rid of the quite useless additional
demuxer threads. This solution is conceptually simple, but requires
refactoring the entire demuxer middle layer.
B. Attempt to coordinate the demuxer threads. This would maintain a
shared cache and readahead state to solve the mentioned problems
explicitly. While this sounds simple and like an incremental change,
it's probably hard to implement, creates more messy special cases,
solution A. seems just a better and simpler variant of this. (On the
other hand, A. requires refactoring more code.)
C. Render an intersection of the seek ranges across all streams. This
fixes only problem 1.
D. Merge all streams in a dedicated wrapper demuxer. The general demuxer
layer remains unchanged, and reading from separate streams is handled as
special case. This effectively achieves the same as A. In particular,
caching is simply handled by the usual demuxer cache layer, which sees
the wrapper demuxer as a single stream of interleaved packets. One
implementation variant of this is to reuse the EDL infrastructure, which
this commit does.
All in all, solution A would be preferable, because it's cleaner and
works for all external streams in general.
Some previous commit tried to prepare for implementing solution A. This
could still happen. But it could take years until this is finally
seriously started and finished. In any case, this commit doesn't block
or complicate such attempts, which is also why it's the way to go.
It's worth mentioning that original mplayer handles external files by
creating a wrapper demuxer. This is like a less ideal mixture of A. and
D. (The similarity with A. is that extending the mplayer approach to be
fully dynamic and without certain disadvantages caused by the wrapper
would end up with A. anyway. The similarity with D. is that due to the
wrapper, no higher level code needs to be changed.)
