Notes:
- converts the (image) write() api to filenames, because using avio
with FILE* is a pain.
- adds more debug logs for screenshots.
build: rename av1 dependency to avif_muxer
wscript: unify lavf dependency with meson
Today, the only way to make mpv consider multiple hwdecs and pick the
first one that works is to use one of the `auto` modes. But the list
that is considered in those cases is hard-coded. If the user wants to
provide their own list, they are out of luck.
And I think that there is now a significant reason to support this -
the new Vulkan hwdec is definitely not ready to be in the auto list,
but if you want to use it by default, it will not work with many codecs
that are normally hardware decodable (only h.264, hevc and av1 if you
are very lucky). Everything else will fall back to software decoding.
Instead, what you really want to say is: use Vulkan for whatever it
supports, and fall back to my old hwdec for everything else.
One side-effect of this implementation is that you can freely mix
hwdec names and special values like `auto` and `no`. The behaviour will
be correct, so I didn't try and prohibit any combinations. However,
some combinations will be silly - eg: sticking any further values after
`no` will result in them being ignored. On the other hand, a
combination like `vulkan,auto` could be very useful as that will use
Vulkan if possible, and if not, run the normal auto routine.
Fixes#11797
this changes mp_image_new_ref() to handle allocation failure itself
instead of doing it at its many call-sites (some of which never checked
for failure at all).
also remove MP_HANDLE_OOM() from the call sites since this is not
necessary anymore.
not all the call-sites have been touched, since some of the caller might
be relying on `mp_image_new_ref(NULL)` returning NULL.
Fixes: https://github.com/mpv-player/mpv/issues/11840
when vo_drm_init() fails inside of preinit(), uninit() will be called as
part of cleanup with vo->drm being NULL and thus `drm->fd` would lead to
null dereference.
and since vo_drm_uninit() closes drm->fd, destroy_framebuffer() ends up
using a closed fd.
according to the drm-gem manpage [0]:
> If you close the DRM file-descriptor, all open dumb-buffers are
> automatically destroyed.
so remove the destroy_framebuffer() loop entirely, which fixes both the
issues.
[0]: https://www.systutorials.com/docs/linux/man/7-drm-gem/
I originally left `drmprime_overlay` as higher priority because
`drmprime` was new, and because I didn't have any hardware where both
worked (only one or the other) so I couldn't compare relative
performance, and if only one worked, the priority didn't matter.
But with time and more usage, we've reached a point where we can say we
would recommend using `drmprime` in situations where both work, and
we've also been able to identify hardware where both do indeed work and
it seems that `drmprime` is more reliable.
So, let's flip them.
It was done once before but later reverted for testing reasons. This
time it's permanent though since I can test this VO on ARM and with an
up to date system.
When using a display-* video-sync mode, it is possible for buffers with
a matching id to already have an image associated with them (i.e. the
compositor hasn't released it yet). Previously, it was thought that we
could just unref, return null, and make a new buffer but this eventually
leads to a fatal error that originates from libwayland itself which
stops playback. Admittedly, the reason for the error is a bit nebulous
but likely it seems to be some kind of mismatch between dmabuf params
and the associated image with the buffer.
However, we can simplify this process greatly. Instead when the
previously mentioned edge case happens, the old image can simply be
freed and we give the buffer the new image. This saves creating a new
buffer and also avoids that nasty libwayland error. A nice win-win all
around. Fixes#11773.
vo_dmabuf_wayland has a pool of wl_buffers that it cycles through when
drawing frame. There needs to be at least some minimum number otherwise
a flickering artifact occurs where old frames are mistakenly repeated.
When using display-resample and other similar modes, it seems more
buffers are required (more drawing happens so it makes sense) and the
current minimum of 8 isn't good enough. Let's just bump this to 15. It's
also a random ad hoc number, but as far as I know there's not really a
way to predict how many buffers a random video may need. From testing,
it works fine and overall 15 is still a tiny amount of objects to create
considering the lifetime of a video, so we'll just go with this.
Some platforms (wayland) apparently have a lot of trouble with drag and
drop. The default behavior is still the same which is basically obeying
what we get from the window manager/compositor, but the --drag-and-drop
option allows forcibly overriding the drag and drop behavior. i.e. you
can force it to always replace the playlist or append at the end. This
only implements this in X11 and Wayland but in theory windows and macos
could find this option useful (both hardcode the shift key for
appending). Patches welcome.
In data_offer_actions, it's possible to get the
WL_DATA_DEVICE_MANAGER_DND_ACTION_NONE action which would set
wl->dnd_action to DND_APPEND (did nothing in practice) but also log a
message which is confusing and misleading. Instead, just ignore and
don't do anything when we get this case.
This is not technically necessary, because we never touch the fd again
after passing to cuda, but having it there could lead to future code
accidentally using it.
All hwdecs should respect the probing flag and demote their lgoging to
verbose level, so that initialisation failures during probing do not
spam the user. I forgot to do this for the Vulkan hwdec.
The dmabuf-wayland vo has a stub ra implementation that doesn't
have a swapchain. That means that it's currently not safe to call
ra_vk_ctx_get on that ra_ctx, but it must be safe to call on all ra
implementations as this is how we discover if it is a vulkan ra.
This hasn't been an issue before because no Vulkan code paths would be
triggered when using dmabuf-wayland, but with the new vulkan hwdec, it
becomes possible to trigger when hwdecs are probed.
The libplacebo sync abstraction is deprecated and we should be using
the more explicit Vulkan semaphore helpers instead. This means that
more of the book keeping moves to our side, but it's not too bad.
There are two main things going on here:
1. After a lot of back and forth, I decided to write the new code with
timeline semaphores to streamline things, and that also means all
the variables are separate - which makes the #ifdefs easier to read.
Which is important because:
2. While pl_sync owned the exported fd/handle, pl_vulkan_sem does not,
so we are responsible for managing them. That means reversing the
previous logic - we now can pass an original fd to CUDA and never
think about it again, while we have to clean up a Win32 Handle
because CUDA will not take ownership.
AV1 support in Vulkan is extremely bleeding edge - to the point that
the extension is not present in official Khronos releases, but it has
a reserved identifier and we can look it up with a string literal for
now.
This will be skipped and ignored if the driver doesn't support it, so
it's safe if/when the name changes later (it'll just never be activated
in that case).
I originally wrote this trying to avoid doing an explicit version check
on the headers, but it just makes things more confusing, and the
requirements harder to understand.
So, Vulkan interop now takes a dependency on the header release where
they finalised the video decode headers. VK_EXT_descriptor_buffer was
added in 1.3.235, so that's covered as well.
Along the way I fixed a bug in the waf build where it was depending
on libplacebo-next instead of libplacebo-decode.
ffmpeg was previously allocating images for frames as the code size,
rather than the presentation one (1088 vs 1080 in the most common
example). Using the coded size when wrapping images for libplacebo
resulted in incorrect scaling from 1088 -> 1080, but even using the
presentation size wasn't perfect, as discussed in the original
commit.
However, ffmpeg has now been updated to use the presentation size for
the frame images, after discussions that concluded this must be done
because there is no way for a frame consumer to fix the dimensions
without copying the frame.
With that ffmpeg change, we can just use the normal layout information
like all the other hwdecs.
Although we can support vulkan multiplane images, cuda lacks any such
support, and so cannot natively import such images for interop. It's
possible that we can do separate exports for each plane in the image
and have it work, but for now, we can selectively disable multiplane
when we know that we'll be consuming cuda frames.
As a reminder, even though cuda is the frame source, interop is one way
so the vulkan images have to be imported to cuda before we copy the
frame contents over.
This logic here is slightly more complex than I'd like but you can't
just set the flag blindly, as it will cause hwframes ctx creation to
fail if the format is packed or if it's planar rgb. Oh well.
Vulkan hwdec interop with the ffmpeg 6.1 vulkan code will require
additional features beyond those activated by libplacebo by default.
Enabling these features requires both requesting the features'
extensions and then explicitly turning on the features. libplacebo
handles detecting unsupported features and dropping them, to avoid
failing to create the vulkan device.
We then leave it to ffmpeg to decide if any missing features are
required for functionality, and error out if necessary.
As ffmpeg requires at least one bleeding edge extension (descriptor
buffers), all of this logic is gated on the presence of sufficiently
new Vulkan headers.
Vulkan Video Decoding has finally become a reality, as it's now
showing up in shipping drivers, and the ffmpeg support has been
merged.
With that in mind, this change introduces HW interop support for
ffmpeg Vulkan frames. The implementation is functionally complete - it
can display frames produced by hardware decoding, and it can work with
ffmpeg vulkan filters. There are still various caveats due to gaps and
bugs in drivers, so YMMV, as always.
Primary testing has been done on Intel, AMD, and nvidia hardware on
Linux with basic Windows testing on nvidia.
Notable caveats:
* Due to driver bugs, video decoding on nvidia does not work right now,
unless you use the Vulkan Beta driver. It can be worked around, but
requires ffmpeg changes that are not considered acceptable to merge.
* Even if those work-arounds are applied, Vulkan filters will not work
on video that was decoded by Vulkan, due to additional bugs in the
nvidia drivers. The filters do work correctly on content decoded some
other way, and then uploaded to Vulkan (eg: Decode with nvdec, upload
with --vf=format=vulkan)
* Vulkan filters can only be used with drivers that support
VK_EXT_descriptor_buffer which doesn't include Intel ANV as yet.
There is an MR outstanding for this.
* When dealing with 1080p content, there may be some visual distortion
in the bottom lines of frames due to chroma scaling incorporating the
extra hidden lines at the bottom of the frame (1080p content is
actually stored as 1088 lines), depending on the hardware/driver
combination and the scaling algorithm. This cannot be easily
addressed as the mechanical fix for it violates the Vulkan spec, and
probably requires a spec change to resolve properly.
All of these caveats will be fixed in either drivers or ffmpeg, and so
will not require mpv changes (unless something unexpected happens)
If you want to run on nvidia with the non-beta drivers, you can this
ffmpeg tree with the work-around patches:
* https://github.com/philipl/FFmpeg/tree/vulkan-nvidia-workarounds
We will need the full ra_ctx to be able to look up all the state
required to initialise an ffmpeg vulkan hwcontext, so pass let's
pass the ra_ctx instead of just the ra.
This is motivated by a need to access it from vo_gpu_next's opengl
wrapping code, and justified by it being an inherent property of the GL
context itself,
It was unsafe to return pointer to memory that was freed on another
thread, just copy the string to caller owned sturcture.
Fixes crashes when displaying passes stats with gpu-next.
vo_dmabuf_wayland worked by allocating entries to a pool and then having
a lot of complex logic dealing with releasing buffers, pending entries,
etc. along with some other not so nice things. Instead, we can rewrite
this logic so that the wl_buffers created by the imported dmabuf is
instead stored in a linked list, wl_list. We can simply append our
buffers to the list when needed and destroy everything at the end. On
every frame, we can check the ids of our surfaces and reuse existing
buffers, so in practice there will only ever be a handful at a time.
Some other small changes were made in an attempt to organize the
vaapi/drmprime code a little better as well.
An important change is to always enforce at least a minimum number of
buffers. Certain formats would not make enough unique buffers, and this
results in flickering/artifacts occuring. The old way to attempt to deal
with this was to clear out all the existing buffers and remake them, but
this gets complicated and also didn't always work. An easy solution to
this is just create more buffers which appears to solve this problem.
The actual number needed is not really based on anything solid, but 8
is a reasonable number to create for the lifetime of a file and it seems
to do the trick.
Additionally, seeking/loading new files can result in flicker artificts
due to buffers being reused when they shouldn't. When that happens, we
flip a bool so all the buffers get destroyed in draw_frame to avoid any
visual glitches.
This just replaces the API calls to get rid of deprecation warnings, it
doesn't yet expand the enum, nor replace them by the proper options.
The translation from tone map modes to hybrid mix parameters is taken
from the libplacebo source code.
Previously, if vo_drm_init failed at the start of drm_egl_init it
caused a use-after-free in drm_egl_uninit when it tried to access the
non-existant drm context. At that point vo_drm_uninit had already been
called resulting in two different null pointer dereference. In this
situation the DRM private context had also not been allocated.
Filtering globally D3D11_MESSAGE_ID_CREATETEXTURE2D_INVALIDDIMENSIONS is
suboptimal, because can also hide other invalid usages. In the same time
it is not enough, because not only this message is emitted, but also one
about E_INVALIDARG. Just flush queue before and clear messages after to
ignore this part of code.
As a side note, I don't believe this texture size lookup is in fact
useful, but since it is there and is relatively harmless, let's leave
it as is.
The current implementation is order dependent and assumes that getting
keyboard input happens before the toplevel is activated. This isn't
necessarily the case and indeed mutter activates the toplevel first.
Improve this by simply spinning off the check to a function and calling
it in the three places where it would be needed: the toplevel
configuration event, keyboard entering, and keyboard leaving. This
fixes#11694.
