SmoothMotion is a way to time and blend frames made popular by MadVR. It's
intended behaviour is to remove stuttering caused by mismatches between the
display refresh rate and the video fps, while preserving the video's original
artistic qualities (no soap opera effect). It's supposed to make 24fps video
playback on 60hz monitors as close as possible to a 24hz monitor.
Instead of drawing a frame once once it's pts has passed the vsync time, we
redraw at the display refresh rate, and if we detect the vsync is between two
frames we interpolated them (depending on their position relative to the vsync).
We actually interpolate as few frames as possible to avoid a blur effect as
much as possible. For example, if we were to play back a 1fps video on a 60hz
monitor, we would blend at most on 1 vsync for each frame (while the other 59
vsyncs would be rendered as is).
Frame interpolation is always done before scaling and in linear light when
possible (an ICC profile is used, or :srgb is used).
And remove all uses of the VFCAP_CSP_SUPPORTED* constants. This is
supposed to reduce conversions if many filters are used (with many
incompatible pixel formats), and also for preferring the VO's natively
supported pixel formats (as opposed to conversion).
This is worthless by now. Not only do the main VOs not use software
conversion, but also the way vf_lavfi and libavfilter work mostly break
the way the old MPlayer mechanism worked. Other important filters like
vf_vapoursynth do not support "proper" format negotation either.
Part of this was already removed with the vf_scale cleanup from today.
While I'm touching every single VO, also fix the query_format argument
(it's not a FourCC anymore).
We still need to send the VO a duration in these cases. Disabling
framedrop has logically absolutely nothing to do with these cases; it
was overlooked in commit 918b06c4.
So we always send the frame duration (or a guess for it), and check
whether framedropping is actually enabled in the VO code. (It would
be cleaner to send framedrop as a flag, but I don't care about that
right now.)
The last video frame is another case that has a separate code path,
although it's pretty similar to the one in commit 73e5aa87. Fix this
in a different way, which also takes care of the last frame case,
although without context the code becomes slightly more tricky.
As further cleanup, move the decision about framedropping itself to
the same place, so the check in vo.c becomes much simpler. The check
for the vo->driver->encode flag, which is remvoed completely, was
redundant too.
Fixes#1480.
Don't use vo_control() for sending VOCTRL_RESET when starting a seek.
This means vo_seek_reset() won't wait until the VO actually processed
VOCTRL_RESET. It happens asynchronously instead.
The impact of this change should be minimal, unless the VO is somehow
too busy (like blocking on vsync).
There are currently 568 pixel formats (actually fewer, but the namespace
is this big), and for each format elaborate synchronization was done to
call it synchronously on the VO. This is completely unnecessary, and we
can do with just a single call.
Involve detection of software renderers in the probing properly. Other
VOs could handle probing also more gracefully, and e.g. produce less
noise if an API is unavailable. (Although other than the OpenGL VOs,
only vo_wayland will.)
Now the "sw" suboption for vo_opengl[_old] is strictly speaking not
needed anymore. Doing "--vo=opengl" disables the probing logic, and will
always force it, if possible.
Includes some simplifications as well.
Commit d38bc531 is incorrect: the 50ms queue-ahead value and the flip
queue offset have different functions. The latter is about calling
flip_page in advance, so the change attempted to show video frames 50ms
in advance on all VOs.
The change was for vo_opengl_cb, but that can be handled differently.
This adds API to libmpv that lets host applications use the mpv opengl
renderer. This is a more flexible (and possibly more portable) option to
foreign window embedding (via --wid).
This assumes that methods like context sharing and multithreaded OpenGL
rendering are infeasible, and that a way is needed to integrate it with
an application that uses a single thread to render everything.
Add an example that does this with QtQuick/qml. The example is
relatively lazy, but still shows how relatively simple the integration
is. The FBO indirection could probably be avoided, but would require
more work (and would probably lead to worse QtQuick integration, because
it would have to ignore transformations like rotation).
Because this makes mpv directly use the host application's OpenGL
context, there is no platform specific code involved in mpv, except
for hw decoding interop.
main.qml is derived from some Qt example.
The following things are still missing:
- a way to do better video timing
- expose GL renderer options, allow changing them at runtime
- support for color equalizer controls
- support for screenshots
Add a generic mechanism to the VO to relay "extra" events from VO to
player. Use it to notify the core of window resizes, which in turn will
be used to mark all affected properties ("window-scale" in this case) as
changed.
(I refrained from hacking this as internal command into input_ctx, or to
poll the state change, etc. - but in the end, maybe it would be best to
actually pass the client API context directly to the places where events
can happen.)
Especially with other components (libavcodec, OSX stuff), the thread
list can get quite populated. Setting the thread name helps when
debugging.
Since this is not portable, we check the OS variants in waf configure.
old-configure just gets a special-case for glibc, since doing a full
check here would probably be a waste of effort.
When the VO was moved it its own thread, responsibility for redrawing
was given to the VO thread itself. So if there was a condition that
indicated that redrawing was required, like expose events or certain
VOCTRLs, the VO thread was redrawing itself.
This worked fine, but there are some corner cases where this works
rather badly. E.g. if I fullscreen the player and hit panscan controls
with mpv's default autorepeat rate, playback stops. This happens because
the VO redraws itself after every panscan change command. Running each
(repeated) command takes so long due to redrawing and (involuntary)
waiting on vsync, that it never leaves the input processing loop while
the key is held down. I suspect that in my case, redrawing in fullscreen
mode just gets slow enough that it takes 2 vsyncs instead of 1 on
average, and the processing time gets larger than the autorepeat delay.
Fix this by taking redraw control from the VO, and instead let the
playloop issue a "real" redraw command to the VO if needed. This
basically reverts redraw handling to what it was before moving the VO to
a thread.
CC: @mpv-player/stable
When pausing after a frame was just dropped, we're logically at the
dropped frame, and thus should redraw the dropped frame. This was
implemented, but didn't work after unpausing for the second time,
because of a minor logic bug.
vo_vdpau uses its own framedrop code, mostly for historic reasons. It
has some tricky heuristics, of which I'm not sure how they work, or if
they have any effect at all, but in any case, I want to keep this code
for now. One day it might get fully ported to the vo.c framedrop code,
or just removed.
But improve its interaction with the user-visible framedrop controls.
Make --framedrop actually enable and disable the vo_vdpau framedrop
code, and increment the number of dropped frames correctly.
The code path for other VOs should be equivalent. The vo_vdpau behavior
should, except for the improvements mentioned above, be mostly
equivalent as well. One minor change is that frames "shown" during
preemption are always count as dropped.
Remove the statement from the manpage that vo_vdpau is the default; this
hasn't been the case for a while.
There's no reason to let the core wait until the frame is done
displaying. In practice, the core normally didn't need this additional
wakeup, and the VO was quick enough to fetch the new frame, before the
core even attempted to queue a new frame. But it wasn't entirely clean,
and the correct wakeup handling might matter in some cases.
This was kept in the codebase because it is slightly faster than --vo=opengl
on really old Intel cards (from the GMA era). Time to kill it, and let it rest.
Fixes#1061
bstr.c doesn't really deserve its own directory, and compat had just
a few files, most of which may as well be in osdep. There isn't really
any justification for these extra directories, so get rid of them.
The compat/libav.h was empty - just delete it. We changed our approach
to API compatibility, and will likely not need it anymore.
If duration<0, it means the duration is unknown. Disable framedropping,
because end_time makes no sense in this case.
Also, strictly never drop the first frame.
This fixes weird behavior with the cover-art case (for the 100th time).
This could be used by VO implementations to report a recent vsync time
to the generic VO code, which in turn will use it and the display FPS
to estimate at which point in time the next vsync will happen.
For debugging (drawing fun plots with TOOLS/stats-conv.py).
Also move last_flip under the correct comment: it's not protected by the
lock, and can be accessed by the VO thread only.
Nvidia's vdpau implementation is pretty good, but other factors make it
much less attractive for use as default VO. For example, Mesa often has
low quality drivers (mess up things with the presentation queue and the
vdpau API time source). Intel ruins things completely, and we're likely
to run on emulation via OpenGL. Compositing has unknown effects (to me
anyway), but appears to reduce the vdpau advantages.
One important reason to prefer vo_vdpau was that it could do proper
framedropping. Framedropping got fixed for the other VOs, so this reason
is going away.
This works only on X11, and only if the refresh rate changes due to the
window being moved to another screen (detected by us). It doesn't
include system screen reconfiguration yet.
This calls VOCTRL_GET_DISPLAY_FPS on every frame, which makes me uneasy.
It means extra thread communication with the win32 and Cocoa backends.
On the other hand, a frame doesn't happen _that_ often, and the
communication should still be pretty cheap and fast, so it's probably
ok.
Also needs some extra fuzz for vo_vdpau.c, because that does everything
differently.
Since the display FPS is currently detected on X11 only (and even there
it's known to be wrong on certain setups), it seems like a good idea to
make this user-configurable.
This mostly uses the same idea as with vo_vdpau.c, but much simplified.
On X11, it tries to get the display framerate with XF86VM, and limits
the frequency of new video frames against it. Note that this is an old
extension, and is confirmed not to work correctly with multi-monitor
setups. But we're using it because it was already around (it is also
used by vo_vdpau).
This attempts to predict the next vsync event by using the time of the
last frame and the display FPS. Even if that goes completely wrong,
the results are still relatively good.
On other systems, or if the X11 code doesn't return a display FPS, a
framerate of 1000 is assumed. This is infinite for all practical
purposes, and means that only frames which are definitely too late are
dropped. This probably has worse results, but is still useful.
"--framedrop=yes" is basically replaced with "--framedrop=decoder". The
old framedropping mode is kept around, and should perhaps be improved.
Dropping on the decoder level is still useful if decoding itself is too
slow.
The previous commit broke these things, and fixing them is separate in
this commit in order to reduce the volume of changes.
Move the image queue from the VO to the playback core. The image queue
is a remnant of the old way how vdpau was implemented, and increasingly
became more and more an artifact. In the end, it did only one thing:
computing the duration of the current frame. This was done by taking the
PTS difference between the current and the future frame. We keep this,
but by moving it out of the VO, we don't have to special-case format
changes anymore. This simplifies the code a lot.
Since we need the queue to compute the duration only, a queue size
larger than 2 makes no sense, and we can hardcode that.
Also change how the last frame is handled. The last frame is a bit of a
problem, because video timing works by showing one frame after another,
which makes it a special case. Make the VO provide a function to notify
us when the frame is done, instead. The frame duration is used for that.
This is not perfect. For example, changing playback speed during the
last frame doesn't update the end time. Pausing will not stop the clock
that times the last frame. But I don't think this matters for such a
corner case.
The VO is run inside its own thread. It also does most of video timing.
The playloop hands the image data and a realtime timestamp to the VO,
and the VO does the rest.
In particular, this allows the playloop to do other things, instead of
blocking for video redraw. But if anything accesses the VO during video
timing, it will block.
This also fixes vo_sdl.c event handling; but that is only a side-effect,
since reimplementing the broken way would require more effort.
Also drop --softsleep. In theory, this option helps if the kernel's
sleeping mechanism is too inaccurate for video timing. In practice, I
haven't ever encountered a situation where it helps, and it just burns
CPU cycles. On the other hand it's probably actively harmful, because
it prevents the libavcodec decoder threads from doing real work.
Side note:
Originally, I intended that multiple frames can be queued to the VO. But
this is not done, due to problems with OSD and other certain features.
OSD in particular is simply designed in a way that it can be neither
timed nor copied, so you do have to render it into the video frame
before you can draw the next frame. (Subtitles have no such restriction.
sd_lavc was even updated to fix this.) It seems the right solution to
queuing multiple VO frames is rendering on VO-backed framebuffers, like
vo_vdpau.c does. This requires VO driver support, and is out of scope
of this commit.
As consequence, the VO has a queue size of 1. The existing video queue
is just needed to compute frame duration, and will be moved out in the
next commit.
With software decoding, images were uploaded to vdpau surfaces as they
were queued to the VO. This makes it slightly more complicated
(especially later on), and has no advantages - so stop doing it.
The only reason why this was done explicitly was due to attempts to keep
the code equivalent (instead of risking performance regressions). The
original code did this naturally for certain reasons, but now that we
can measure that it has no advantages and just requires extra code, we
can just drop it.
Follow up on commit 760548da. Mouse handling is a bit confusing, because
there are at least 3 coordinate systems associated with it, and it
should be cleaned up. But that is hard, so just apply a hack which gets
the currently-annoying issue (VO backends needing access to the VO) out
of the way.
VO backends which are or will run in their own thread have a problem
with vo_mouse_movement() calling vo_control(). Restrict this to VOs
which actually need this.
Until now, failure to allocate image data resulted in a crash (i.e.
abort() was called). This was intentional, because it's pretty silly to
degrade playback, and in almost all situations, the OOM will probably
kill you anyway. (And then there's the standard Linux overcommit
behavior, which also will kill you at some point.)
But I changed my opinion, so here we go. This change does not affect
_all_ memory allocations, just image data. Now in most failure cases,
the output will just be skipped. For video filters, this coincidentally
means that failure is treated as EOF (because the playback core assumes
EOF if nothing comes out of the video filter chain). In other
situations, output might be in some way degraded, like skipping frames,
not scaling OSD, and such.
Functions whose return values changed semantics:
mp_image_alloc
mp_image_new_copy
mp_image_new_ref
mp_image_make_writeable
mp_image_setrefp
mp_image_to_av_frame_and_unref
mp_image_from_av_frame
mp_image_new_external_ref
mp_image_new_custom_ref
mp_image_pool_make_writeable
mp_image_pool_get
mp_image_pool_new_copy
mp_vdpau_mixed_frame_create
vf_alloc_out_image
vf_make_out_image_writeable
glGetWindowScreenshot
Let the VOs draw the OSD on their own, instead of making OSD drawing a
separate VO driver call. Further, let it be the VOs responsibility to
request subtitles with the correct PTS. We also basically allow the VO
to request OSD/subtitles at any time.
OSX changes untested.
While I'm not very fond of "const", it's important for declarations
(it decides whether a symbol is emitted in a read-only or read/write
section). Fix all these cases, so we have writeable global data only
when we really need.
Playing a video and then an audio file with cover art kept displaying
the last frame of the video. This was because the hasframe flag was set,
perhaps due to redrawing the last video frame before the cover art image
is decoded.
