Pull AOs work off of a callback that relies on mpv's internal timer. So
like with the related video changes, convert all of these to nanoseconds
instead. In many cases, the underlying audio API does actually provide
nanosecond resolution as well.
There's a lot of wild 1e6, 1000, etc. lying around in the code. A macro
is much easier to read and understand at a glance. Add some helpers for
this. We don't need to convert everything now but there's some simple
things that can be done so they are included in this commit.
Why a bigger search-interval is required:
scaletempo2 doesn't do a good job when the signal contains frequencies
less then 1/search_interval. With a search interval of 30ms that means
anything below 33.333Hz sounds bad.
Depending on the genre it's very for music to contain frequencies down
to 30Hz, and sometimes even a little bit below that. Therefore a higher
default value is needed to handle such cases.
Based on that an argument can be made for a value of 50, as that should
work down to 20Hz, or something even higher because movies sometimes
have some infrasonic content.
However the downside of big search intervals is increased CPU usage and
intelligibility at higher speeds, as it effectively leads to parts of
the audio being skipped.
A value of 40 can handle frequencies down to 25Hz, enough for all music
except very rare edge cases, while still providing decent
intelligibility.
Why a smaller window-size is required:
Large values reduce intelligibility at high speeds and therefore small
values are preferred.
However when values get too small it starts to sound weird
(similar to librubberband).
In my testing a value of 10 already works well, but adding a small
safety margin seems like a good idea, especially since it made no
noticeable difference to intelligibility, which is why 12 was chosen.
Linux and macOS already use nanosecond resolution for their sleep
functions. It was just being converted from microseconds before. Since
we have mp_time_ns now, go ahead and bump the precision here. The timer
for windows uses some timeBeginPeriod thing which I'm not sure what it
does really but whatever just convert the units to ms like they were
doing before. There's really no reason to keep the mp_sleep_us helper
around. A multiplication by 1000 is trivial and underlying OS clocks
have nanosecond precision.
This is the most supported in standard layout, if we request more it
tends to fallback to stereo instead. Also channels mask is 32-bit and it
can get truncated.
A bit different from the OPT_REPLACED/OPT_REMOVED ones in that the
options still possibly do something but they have a deprecation
message. Most of these are old and have no real usage. The only
potentially controversial ones are the removal of --oaffset and
--ovoffset which were deprecated years ago and seemingly have no real
replacement. There's a cryptic message about --audio-delay but who
knows. The less encoding mode code we have, the better so just chuck
it.
Avoid generating too much audio after EOF.
Note: This often has no effect, because less audio is produced than
required.
Usually this comes to effect with the userspeed filter at high speed
(4x) and going back to 1x speed to remove the filter.
After the final input packet, the filter padded with silence to allow
one more iteration. That was not enough to process the final frames.
Continue padding the end of `input_buffer` with silence until the final
frames have been processed.
Implementation: Instead of padding when adding final samples, pad before
running WSOLA iteration. Count number of added silent frames and
remaining input frames for time keeping.
This changes the emitted pts values from the start of the search block
to the center of the search block. Change initial `output_time`
accordingly. Initial `search_block_index` is irrelevant, because it's
overwritten before the first iteration.
Using the `output_time` removes the rounding of `search_block_index`,
which also fixes the <20 microsecond gaps in timestamps between output
packets.
Rationale:
The variance in audio position was in the range `0..search-interval`.
With this change, the range is
(- search-interval / 2)..(search-interval / 2)`
which ensures lower maximum offset.
Target block can be anywhere in the previous search-block, varying by
`search-interval` while the filter is active. This resulted in constant
audio offset when returning to 1x playback speed.
- Move the search block to the target block to sync up exactly.
- Drop old frames to minimize input_buffer usage.
The internal time update function involved multiple problems:
- Time was updated after WSOLA iteration. The means speed was updated
one iteration later than it could be.
- The update functions caused spikes of too many or too few samples
advanced, leading to audio glitches on speed changes.
- The inconsistent updates made it very difficult to produce gapless
audio packets.
- The `output_time` update function involved complicated feedback:
`search_block_index` influenced how many frames from `input_buffer`
are retained, which influenced how much `output_time` is changed,
which influenced `search_block_index`.
With these changes:
- Time is updated before WSOLA iterations. Speed changes are effective
instantly.
- There are no spikes in playback speed during speed changes.
- No significant gaps are introduced in output packets.
- The time update function becomes (function calls omitted for brevity)
output_time += ola_hop_size * playback_rate
Functions received a `playback_rate` parameter to check how many samples
are needed before iteration. Internal state is only updated when the
iteration is actually run, so the speed is allowed to change until
enough data is received.
The first WSOLA iteration overlapped audio with whatever was in the
`wsola_output` buffer. This was either silence (if not run before), or
old frames (if switching to 1x and back to a different speed).
Track the state of the output buffer and memcpy the whole window for the
first iteration instead.
`output_time` is used to set the center of the search block. Init of
both `search_block_index` and `output_time` with 0 caused inconsistent
search block movement for the first iterations.
Initialize with `search_block_center_offset` for consistency with initial
`search_block_index`.
Fixes#12028
There was an additional issue that audio was always delayed by half the
configured search-interval. This was caused by the `out` buffer length
not being included in the delay calculation.
Notes:
- Every WSOLA iteration advances the input buffer by _some amount_, and
produces data in the output buffer always of size `ola_hop_size`.
- `mp_scaletempo2_fill_buffer` is always called with `ola_hop_size`
- Thus, the rendered frames are always cleared immediately after
processing, and `num_complete_frames` is 0 in the delay calculation.
- The factors contributing to delay are:
- the pending samples in the input buffer according to the search
block position, and
- the pending rendered samples in the output buffer (always empty in
practice).
The frame_delay code looked like that of the rubberband filter, which
might not work for scaletempo2. Sometimes a different amount of input
audio was consumed by scaletempo2 than expected. It may have been caused
by speed changes being a more dynamic process in scaletempo2. This can
be seen by where `playback_rate` is used in `run_one_wsola_iteration`:
`playback_rate` is only referenced after the iteration, when updating
the time and removing old data from buffers.
In scaletempo2, the playback speed is applied by changing the amount the
search block is moved. That apparently averages out correctly at
constant playback speed, but when the speed changes, the error in this
assumption probably spikes. This error accumulated across all speed
changes because of the persistent `frame_delay` value.
With the removal of the persistent `frame_delay`, there should be no way
for the audio to drift off. By deriving the delay from filter buffer
positions, and the buffers are filled only as much as needed, the delay
always stays within buffer bounds.
The idea behind period_size is that it's the minimum amount of data
that your audio subsystem wants to fetch.
For PulseAudio, this is given by the minreq buffer attribute, which
is in bytes for all channels. Hence the divisions.
This change sets the device_buffer member of the ao struct for
the JACK ao to whatever value we read during init.
While JACK allows changing the audio buffer size on-the-fly
(you can do this for example through DBus), having it somehow
reconfigure the entire audio buffer logic of mpv that depends
on device_buffer in some way when that happens only leads to
audio dropout and weird code. device_buffer's role is mostly for
prebuffer from what I understand, which means that simply setting
it to its current value in the init function is fine.
In addition to the patch, appropriate additions to the mpv.conf file
will of course be needed for this to work. For example on my system:
audio-device=oss//dev/dsp4
audio-spdif=ac3,dts,dts-hd,eac3,truehd
This has been tested using recent FreeBSD-13.1-stable, using external
surround processors (both a Trinnov Altitude 16 and an LG OLED that
supports Dolby Atmos, and connected with HDMI to an NVidia RTX 2070).
Author and tester: David G Lawrence <dg1007@dglawrence.com>
There's an edge cause with gapless audio and pausing. Since, gapless
audio works by sending an EOF immediately, it's possible to pause on the
next file before audio actually finishes playing and thus the sound gets
cut off. The fix is to simply just always do an ao_drain if the ao is
about to set a pause on EOF and we still have audio playing.
Fixes#8898.
The difference this makes is that the OS API will notice
when we underrun (as opposed to being fed silence).
Other AOs mostly seem to not do this because they've committed
to filling a buffer of a certain size no matter what, but I have
not observed any ill effects for AudioTrack in my testing.
This looks like a pretty bad bug but only became a problem
with the last commit that allows rates like 22.5kHz to pass through
directly instead of being resampled.
wireplumber uses the media role when the node is first created.
To have the property available at this point reliably we need to set it
directly when creating the stream/node.
As the role property is interpreted by wireplumber it can only be
evaluated when creating the stream. The current, dynamic mechanism is
racy so revert it.
See: #11253Fixes: #12041
This reverts commit 535cd6f313.
The avpkt was created once on decoder init but destroyed each time the
filter was destroyed, this obviously can't work. Move the packet alloc
to the filter init function instead.
fixes: 4574dd5dc6
Doing a pw_thread_loop_wait() without checking conditions is invalid.
The thread loop could be signalled for other reasons and in this case
the wait needs to continue.
PipeWire added such additional signaling in
commit 33be898130f0 ("thread-loop: signal when started").
This meant that for_each_sink would return before the callbacks have
fired and session_has_sink() would incorrectly return "false", failing
the initialization of ao_pipewire.
Fixes#11995
