As os_gettime_ns() gets large the current scaling methods, mostly by casting
to uint64_t, may lead to numerical overflows. Sweep the code and use
util_mul_div64() where applicable.
Signed-off-by: Hans Petter Selasky <hps@selasky.org>
sse-intrin.h is a required header now, but the implicit path
breaks building addons once the headers are installed.
Fix this by making the path explicit.
Code submissions have continually suffered from formatting
inconsistencies that constantly have to be addressed. Using
clang-format simplifies this by making code formatting more consistent,
and allows automation of the code formatting so that maintainers can
focus more on the code itself instead of code formatting.
Fixes handling of the `obs_source_frame::full_range` member variable,
which is often set to false by default by many plugins even when using
RGB, which would cause RGB to be marked as "partial range". This change
is crucial for when partial range RBG support is implemented.
Adds `obs_source_frame2` structure that replaces the `full_range` member
variable with a `range` variable, which uses the `video_range_type` enum
to allow handling default range values. This member variable treats
VIDEO_RANGE_DEFAULT as full range if the format is RGB, and partial
range if the format is YUV.
Also adds `obs_source_output_video2` and `obs_source_preload_video2`
functions which use the `obs_source_frame2` structure instead of the
`obs_source_frame` structure.
When using the original `obs_source_frame`, `obs_source_output_video`,
and `obs_source_preload_video` functions, RGB will always be full range
by default for backward compatibility purposes.
A mono source is currently upmixed by swresampler in the following way:
- for stereo output, FL=FR=input/sqrt(2)
- for other speaker layouts of the outputs, FC=input, other channels
are zeroed.
In the case of stereo output, this leads to a 3dB level decrease which
users have issue with [1].
The obvious fix of adding a 3dB gain is reported to be adding distortions
on some setups [2].
Note that the "Downmix to Mono" does not fix this upmix problem, since
it just makes all output channels identical by summing all input channels
and normalizing (by dividing by the number of output channels). This last
normalization step results in a level reduction for a mono input.
[1] This fixes https://obsproject.com/mantis/view.php?id=960.
[2] See also: https://obsproject.com/forum/threads/please-allow-for-mono-recording-of-microphones-ill-explain-why.84834
Normally, the total and skipped frame count for the encoder is performed
in the video-io thread. However, because a new thread is being
introduced for texture-based encoding, the frontend has no way of being
able to query that. So, instead of making the frontend query that data
separately, just make the texture encoder thread increment the values of
video-io. That way, the frontend doesn't need to change any code, and
can continue using the same functions for determining the total/skipped
frame count.
This can cause the frame count to be doubled if both a texture-based
encoder and a raw data encoder is active at the same time, but it's an
acceptable alternative.
(This commit also modifies the obs-ffmpeg module)
The default channel layouts from aac spec are implemented in FFmpeg
native aac encoder as follows:
AV_CH_LAYOUT_MONO,
AV_CH_LAYOUT_STEREO,
AV_CH_LAYOUT_SURROUND,
AV_CH_LAYOUT_4POINT0,
AV_CH_LAYOUT_5POINT0_BACK,
AV_CH_LAYOUT_5POINT1_BACK,
AV_CH_LAYOUT_7POINT1,
The correspondence of speaker layouts to AV_CH_LAYOUT from FFmpeg is
changed to reflect the previous table.
Although FFmpeg native aac encoder can now encode all the layouts listed
in avutil channel_layout.h (on master), there might be issues with older
FFmpeg binaries.
Note that 2.1 speaker layout will be encoded as AV_CH_LAYOUT_SURROUND
(FL FR FC) because it is not listed as the default layout for three
channels.
This just means some optimizations for LFE channel will not be used by
the encoder which will treat it as an SCE (single channel element).
Closesjp9000/obs-studio#1182
When using more than two channels, the channel map of pulse-audio is incorrect.
Add an API for getting a speaker map based on OBS speaker layout. Then use the
speaker map when connecting to a pulse-audio device, for both source and
monitor output.
The memset in custom_audio_render() did not clear all audio buffers when
the number of output channels was less then 8. This caused wrong audio
output on mixes that did not get cleared.
Closesjp9000/obs-studio#1123
(also obs, deps/media-playback, libobs/audio-monitoring, decklink,
linux-alsa, linux-pulseaudio, mac-capture, obs-ffmpeg, win-dshow,
win-wasapi)
Default channel layout for 4 channels is 4.0 in FFmpeg.
Replacing quad with 4.0 will improve compatibility since FFmpeg has
better support of its default channel layouts.
(also modifies obs-ffmpeg, audio-monitoring, win-wasapi, decklink,
obs-outputs)
Removes speaker layouts which are not exposed in UI. The speaker
layouts selectable by users in the UI are the most common ones. It is
not necessary to keep other layouts. (This basically removes
5POINT1_SURROUND, 7POINT1_SURROUND, SURROUND =3.0).
The AVCodecParameters weren't introduced until avcodec version 57.48.101
(FFmpeg version 3.1), so this will make sure to still use the older
avcodec_copy_context if the detected FFmpeg version is earlier.
(This commit also modifies the following modules: UI,
deps/media-playback, coreaudio-encoder, decklink, linux-alsa,
linux-pulseaudio, mac-capture, obs-ffmpeg, obs-filters, obs-libfdk,
obs-outputs, win-dshow, and win-wasapi)
Adds surround sound audio support to the core, core plugins, and user
interface.
Compatible streaming services: Twitch, FB 360 live
Compatible protocols: rtmp / mpeg-ts tcp udp
Compatible file formats: mkv mp4 ts (others untested)
Compatible codecs: ffmpeg aac, fdk_aac, CoreAudio aac,
opus, vorbis, pcm (others untested).
Tested streaming servers: wowza, nginx
HLS, mpeg-dash : surround passthrough
Html5 players tested with live surround:
videojs, mediaelement, viblast (hls+dash), hls.js
Decklink: on win32, swap channels order for 5.1 7.1
(due to different channel mapping on wav, mpeg, ffmpeg)
Audio filters: surround working.
Monitoring: surround working (win macOs linux (pulse-audio)).
VST: stereo plugins keep in general only the first two channels.
surround plugins should work (e.g. mcfx does).
OS: win, macOs, linux (alsa, pulse-audio).
Misc: larger audio bitrates unlocked to accommodate more channels
NB: mf-aac only supports mono and stereo + 5.1 on win 10
(not implemented due to lack of usefulness)
Closesjp9000/obs-studio#968
(This commit also modifies the deps/media-playback, obs-ffmpeg, and
win-dshow modules)
More fixes due to ffmpeg renaming some constants and deprecating
AVFMT_RAWPICTURE and AV_PIX_FMT_VDA_VLD.
Latter replaced by AV_PIX_FMT_VIDEOTOOLBOX per ffmpeg dev advice.
Closesjp9000/obs-studio#1061
This function had a number of bugs and just wasn't working properly at
all. This function is currently not used in public builds because GPU
is used for color conversion instead (hence why it had probably not
really been tested), but a need came up where CPU conversion was useful
for the sake of testing something else in the back-end, and it needed to
be fixed before CPU conversion could be used.
Prevents lagged frames (frames that took too long to render) to be
counted as skipped frames (frames that are skipped due to encoding
taking too long to process)
When frames are skipped the skipped frame count would increment, but the
total frame count would not increment, causing the percentage
calculation to fail.
Additionally, the skipped frames log reporting has been moved to
media-io/video-io.c instead of each output.
(Note: Also modified the obs-ffmpeg plugin module)
Allows the ability for frame data to pass 8-bit grayscale images (Y800
color format).
Closesjp9000/obs-studio#515
(Note: This commit breaks libobs compilation. Skip if bisecting)
This variable is somewhat redundant. Volume is already known/accessible
to front-ends.
(Note: This commit breaks libobs compilation. Skip if bisecting)
Uses a callback and allows the caller to mix audio. Additionally,
allows the callback to return audio later, allowing it to buffer as much
as it needs.
The skipped frame count (dropped frames due to encoding being
overloaded) would erroneously include lagged frames (dropped frames due
to render stalls). This will make diagnosing actual issues a user might
be having a bit easier.
With certain devices (AVerMedia C985 and LGP), audio timestamps are
bad, and a 50ms threshold of audio data "smoothing" (making consecutive
audio packets seamless with one another) isn't enough to handle bad
consecutive timestamp values. After testing, 70ms sufficiently solves
the issue.
This improves logging for when audio data insertion is way out of bounds
or is getting cut off in the front due to a bad negative sync offset.
Instead of throwing out a log message for every time this happens with
each piece of data, it now states when the out of bounds or cutoff has
started and stopped only.
This fixes a case where an insertion of audio data would pass
valid_timestamp_range yet the insert position would cause a negative
integer position and thus an unsigned integer overflow.
