Due to all the threads in libobs it wouldn't be safe to make that
parameter reconfigurable after libobs is initialized without adding
even more synchronization. On the other hand, adding a function to set
the name store before calling obs_startup would solve the problem of
passing a name store into libobs, but it can lead to more complicated
semantics for obs_get_profiler_name_store (e.g., should it always return
the current name store even if libobs isn't initialized until someone
calls set_name_store(NULL)? should obs_shutdown call
set_name_store(NULL)? Passing it as obs_startup parameter avoids
these (and hopefully other) potential misunderstandings
(Non-compiling commit: windowless-context branch)
API Changed:
---------------------
Removed functions:
- obs_add_draw_callback
- obs_remove_draw_callback
- obs_resize
- obs_preview_set_enabled
- obs_preview_enabled
Removed member variables from struct obs_video_info:
- window_width
- window_height
- window
Summary:
---------------------
Changes the core libobs API to not be dependent upon a main window/view.
If you wish to draw to a window/view, use an obs_display object to
handle it.
This allows the use of libobs without requiring a window to be present
on the system. This is also prunes code that had to be needlessly
duplicated to handle the "main" window.
(Non-compiling commit: windowless-context branch)
Changes API from:
---------------------
EXPORT int gs_create(graphics_t **graphics, const char *module,
const struct gs_init_data *data);
To:
---------------------
EXPORT int gs_create(graphics_t **graphics, const char *module,
uint32_t adapter);
Summary:
---------------------
Changes the gs_create function to use an adapter parameter instead of
requiring a gs_init_data with window/color/etc information.
Intentionally breaks compilation when trying to compile the specific
merged commits within the windowless-context branch. This is meant to
be used in conjunction with a merge commit so that bisecting will never
see any non-compiling commits.
Microsoft basically deprecated GetVersion/GetVersionEx, so now you have
to query the file version of kernel32.dll in order to get the actual
windows version. Because of the steps involved in getting the windows
version are fairly complicated, this is an exported libobs helper
function that can be used to get the windows version if needed.
(Microsoft does have its own set of helper functions for this but the
API feels a bit.. awkward, and you can't actually get the real windows
version with them)
A minor optimization: in copy_rgbx_frame (used when libobs is set to
output RGBA frames instead of YUV frames), if the line sizes for the
source and destination match, just use a single memcpy call for all of
the data instead of multiple memcpy calls.
Added cast to unsigned and the assert because microsoft's compiler
doesn't support "%zu"
Actual warning:
libobs/graphics/effect-parser.c:1387:4: warning: format specifies type
'unsigned int' but the argument has type 'size_t' (aka 'unsigned long')
[-Wformat]
Certain plugins/modules that are loaded on windows may be dependent on
libraries that are located in the same directory as the module in
question. This makes is so that LoadLibrary will also search for
dependent libraries for that module in the module's directory.
When globbing for modules, the intent was to ignore the . and ..
directories that might also be included in the glob search. It was
mistakenly doing a string compare on the 'path' variable which contains
the full path, rather than the 'file' variable which just contains the
found file itself, so the string compare always failed.
In case the encoder has to use a different sample rate (due to the
sample rate being unsupported), we need an API function for the encoder
to get the sample rate that the encoder is actually running at.
When an audio filter is applied to a video source that also has
accompanying audio, it would cause the video from the source to stop
rendering.
The original code this was from was to prevent audio-only sources from
rendering video, but I neglected to make sure that this would not apply
to filters, and thus when an audio filter is on a source with video, the
code would kill the video.
The size parameter is the size of the elements, not the size of the
data. The size parameter should be 1, and the elements should be the
number of bytes.
The reason why I'm making this change is because the fread/fwrite would
fail when the parameters were swapped.
In the (unlikely) event of multiple concurrent calls to
input_method_changed it was possible that the log messages would appear
out of order with respect to which layout would actually be active
after the last log message
Due to the fact that async timestamps themselves can be susceptible to
minor jitter from certain types of inputs, increase the allowable jitter
compensation value to ensure that the rendered frame timing from async
video sources is always as close as possible to the compositor.
When the framerate of the source is the same as the framerate as the
compositor, this (combined with the fact that clamped video timing now
being used with async video frames) helps ensure that buffered async
video sources will sync up their rendering to the compositor as
accurately as possible despite jitter from the source's timestamps.
If there is no jitter in the source's timestamps then it'll always sync
up perfectly with the compositor, thanks to clamped video timing.
When playing back buffered async frames, this reduces the probability
that new frames will be missed/skipped due to jitter in the system
timestamps.
If a buffered async source is playing at the same framerate as the
compositor and there is no jitter in the async source's timestamps, then
the async source will play back perfectly in sync with the compositor
thanks to this change, ensuring that there's no skipped or missed frames
in video playback.
The "clamped" video time is the system time per video frame that is
closest to the current system time, but always divisible by the frame
interval. For example, if the last frame system timestamp was 1600 and
the new frame is 2500, but the frame interval is 800, then the
"clamped" video time is 2400.
This clamped value is useful to get the relative system time without any
jitter.
When buffering is enabled for an async video source, sometimes minor
drift in timestamps or unexpected delays to frames can cause frames to
slowly buffer more and more in memory, in some cases eventually causing
the system to run out of memory.
The circumstances in which this can happen seems to depend on both the
computer and the devices in use. So far, the only known circumstances
in which this happens are with heavily buffered devices, such as
hauppauge, where decoding can sometimes take too long and cause
continual frame playback delay, and thus continual buffering until
memory runs out. I've never been able to replicate it on any of my
machines however, even after hours of testing.
This patch is a precautionary measure that puts a hard limit on the
number of async frames that can be currently queued to prevent any case
where memory might continually build for whatever reason. If it goes
over the limit, it clears the cache to reset the buffering.
I had a user with this problem test this patch with success and positive
feedback, and the intervals between buffering resets were long to where
it wasn't even noticeable while streaming/recording.
Ideally when decoding frames (such as from those devices), frame
dropping should be used to ensure playback doesn't incur extra delay,
although this sort of hard limit on the frame cache should still be
implemented regardless just as a safety precaution. For DirectShow
encoded devices I should just switch to faruton's libff for decoding and
enable the frame dropping options. It would probably explain why no
one's ever reported it for the media source, and pretty much only from
DirectShow device usage.
Found via UBSan, actual errors (addresses not pruned for illustrative purposes):
"runtime error: store to misaligned address 0x7f9a9178e84c for type
'size_t' (aka 'unsigned long'), which requires 8 byte alignment"
"runtime error: load of misaligned address 0x7f9a9140f2cf for type
'size_t' (aka 'unsigned long'), which requires 8 byte alignment"
The screen index returned from XDefaultScreen is 0-based, and we were
decrementing it before the check to see if it had reached 0 rather than
after, so in the default_screen function it would always end up getting
either the wrong screen or no screen.
When xcb_query_pointer and xcb_query_pointer_reply was called with no
valid screen, it would fail with an error, thus making it so that the
mouse buttons could not be properly captured as hotkeys.
Implements exponential backoff for consecutive reconnects, which is
useful to prevent too many connections from trying to reconnect back to
a service at once over a short period of time in the case of potential
service downtime. Exponential backoff causes each subsequent reconnect
attempt to double its timeout duration.
This optionally allows the front-end to know what the current timeout
value in seconds is set to for the reconnection without having to call
an extra API function to find that out.
Implement the log_processor_info function on bsd and add ifdefs to only
build the implementation specific to the platform.
Also add an ifdef around the call to that function to make sure it will
only be called on platforms where it is actually implemented.
Split the function logging the processor information on nix into two
parts. The part logging the number of logical cores is portable and
works on all systems that support POSIX.1 while the other part is
specific to linux.
Add the relevant header file needed on FreeBSD and utilize yet another
ifdef to call pthread_set_name_np as the function name differs from
those on the other platforms.
Add definition on FreeBSD to enable getline in stdio, as it is not
(yet ?) available by default. According to the manpage getline was a
GNU extension but was standardized in POSIX.1-2008.
API Changed (in struct obs_encoder_info):
----------------------------------------
bool (*get_audio_info)(void *data, struct audio_convert_info *info);
bool (*get_video_info)(void *data, struct video_scale_info *info);
To:
----------------------------------------
void (*get_audio_info)(void *data, struct audio_convert_info *info);
void (*get_video_info)(void *data, struct video_scale_info *info);
The encoder video/audio information callbacks no longer need to manually
query the libobs video/audio information, that information is now passed
via the parameter, which the callbacks can modify.
The refactor that reduces boilerplate in the encoder video/audio
information callbacks also removes the need for their return values, so
change the return types to void.
I realized that the get_video_info and get_audio_info encoder callbacks
always have to manually query the libobs audio/video information.
This fixes that problem by passing the libobs video/audio information in
the structures passed to those callbacks so they don't have to query it
each time, reducing needless boilerplate code for encoders.
Allows the ability to hint at encoders what format should be used.
This is particularly useful if libobs is currently operating in planar
4:4:4, but you want to force an encoder used for streaming to convert to
NV12 to prevent streaming issues.
Fixes a crash that could happen if any of the mutexes are used in the
create callback, or before the obs_source_init function is called.
I'm not sure how this function order slipped because it seems fairly
obvious that these mutexes should be created before the create callback.
Had this crash happen to me when creating a WASAPI output source, the
create callback of the WASAPI source creates a thread which outputs
audio, and that thread managed to call obs_source_output_audio before
the obs_source_init function was called, which in turn caused it to try
to use a null mutex.
When caching a new frame, keep a reference to the frame while copying to
ensure that the frame is not potentially destroyed for whatever reason
while that data is being copied.
The obs_source::async_reset_texture variable can cause a data race
between threads to occur because it could be set to true in one thread
then changed back to false in another thread. This could cause the
async texture to not update its size when it's supposed to, which can
cause a crash or corruption when copying data from a frame of a
differing size.
The solution to this is to:
- Delete the async_reset_texture variable, and make the
set_async_texture_size function change the texture size if the
async_width, async_height, or async_format variables differ from the
frame's width/height/format. Those variables are then only ever set
in the libobs graphics thread.
- Make the cache_video function use separate variables from other
functions to detect a change in size (due to the fact that the texture
size should only be resized in the libobs graphics thread). These
variables are async_cache_width, async_cache_height, and
async_cache_format, which are only be set in the thread that calls
obs_source_output_video.
How to replicate the data race:
- On OSX, use window capture on a textedit window, then continually
resize the textedit window.
The bilinear lowres scale effect was using 'output' for a variable,
which is apparently a reserved keyword in GLSL on macs. This slipped
by me due to the fact that this didn't occur with OpenGL on my windows
machine.
The minimum and maximum color range values were not being set by the
video_format_get_parameters function when full range was in use; I
assume it's because the expected min/max values of full range is
{0.0, 0.0, 0.0} and {1.0, 1.0, 1.0}, so I'm just making it so that it
sets those values if using full range.
Way to replicate the issue (windows):
1.) Create a win-dshow device capture source
2.) Select a device and set it to a YUV color format to enable YUV color
conversion
3.) Select "Full Range" in the color range property.
4.) Restart OBS, the device will then start up, but will display green
due to the fact that the min/max range values were never set, and are
left at their default value of 0.
Due to a bad 'if' expression, when a filter that is not last in the
chain is disabled or being bypassed, it ends up still calling the
filter's video processing function unintentionally.
This fix makes sure that it only calls the appropriate render functions
if the next filter target is the source, otherwise it will just call
obs_source_video_render to process the next filter in the chain.
How to replicate the bug:
1. Create two crop filters on the same source
2. Give each crop filter a different distinct value
3. Disable both crop filters
4. The image would still be cropped
The normal scaling methods cannot sample enough pixels to create an
accurate output image when the output size is under half the base size,
so use the bilinear low resolution scaling effect in that case instead
to ensure a more accurate low resolution image.
If you don't need to see what's displayed, then this is particularly
useful for two reasons:
1. It reduces the number of draw/present calls
2. It can prevent issues with certain hardware setups where rendering on
a monitor hooked up to a separate card can experience slowdowns
This reverts commit 99c674e41f.
These commits were originally added to allow multiple user interfaces to
use the same plugins, but I soon realized that multiple user interfaces
can use multiple libobs versions, so each user interface should have its
own set of plugins to manage. Some user interfaces may not wish to use
certain plugins anyway, so this fixes that issue as well.
This reverts commit 92d800cc18, reversing
changes made to 35a4acede0.
These commits were originally added to allow multiple user interfaces to
use the same plugins, but I soon realized that multiple user interfaces
can use multiple libobs versions, so each user interface should have its
own set of plugins to manage. Some user interfaces may not wish to use
certain plugins anyway, so this fixes that issue as well.
Instead of manually setting the blend state to the desired values, use
gs_reset_blend_state to ensure we have the default blend state (which
for color is a typical srcalpha/invsrcalpha alpha blending operation,
then the alpha channels are added together).
This fixes an issue where filtered scenes would look strange due to the
fact that alpha was not being blended properly.
This allows the ability to separate the blend states of color and alpha.
The default blend state has also changed so that alpha is always added
together to ensure that the destination image always gets an alpha value
that is actually usable after the operation (for render targets).
Old default state:
color source: GS_BLEND_SRCALPHA, color dest: GS_BLEND_INVSRCALPHA
alpha source: GS_BLEND_SRCALPHA, alpha dest: GS_BLEND_INVSRCALPHA
New default state:
color source: GS_BLEND_SRCALPHA, color dest: GS_BLEND_INVSRCALPHA
alpha source: GS_BLEND_ONE, alpha dest: GS_BLEND_ONE
This fixes an issue where cache frames would not free at all after
having been allocated with no upper limit on the cached frame size. If
cached frames go unused for a specific period of time, they are
deallocated and removed from the cache.
This is preferable to having an upper cache limit due to the potential
for async delay filtering.
Under certain circumstances the cache could be prone to growing too
large unintentionally. Setting a hard maximum limit should prevent
memory from growing if we suddenly get a lot of frames.
Async frames are only swapping when rendering, or when not visible.
This is a flawed design due to the fact that there are certain
circumstances where the source is neither visible nor currently
rendering.
This is what caused a memory leak when scene items were marked as
invisible, because if a source has an async child source and decides not
to render that source for whatever reason, the child source would not
process the async frames at all, and the cache would just grow.
To fix this, simply moving the async frame cycle to tick fixes the issue
due to the fact that tick is always called regardless of circumstance.
Filtering the video before it's output to the texture means that it
happens after all the processing on the timestamps and such of the video
data. This way, the video filter does not have to worry about what's
currently buffered, and it won't affect timing.
When OBS is shutting down, if for some reason the filter is destroyed
before the parent source is destroyed, it would try to remove itself
from the source, but it would decrement the reference and try to destroy
itself again while already in the process of destroying itself.
So, the solution was simply to make sure that if it's removing itself
from the source that it doesn't decrement its own reference.
The "last" filter that's rendered is technically at filter index 0, so
enumeration needs to be from the last index in the list to the first
index in the list.
When rendering a filter to a texture, the target is empty and unused, so
there's no reason for blending to be on when rendering the filter to a
render target.
obs_source_process_filter tried to do everything in a single function,
but the problem is that effect parameters would not properly be
accounted for due to the way it internally draws, therefore it was
necessary to split the functions in to two, you first call
obs_source_process_filter_begin, then you set your effect parameters,
then you finally call obs_source_process_filter_end. This ensures that
when the filter is drawn, that the effect parameters are set.
When the filter chain finally reaches the source and the last filter in
the chain is set to not render directly (meaning it has to render to
texture), it would not render the source with any effect due to the fact
that it expects a filter to be present.
Adds a source callback function that is used when a filter is removed
from a source. This ensures that any data that might be associated with
that source can be cleaned up if necessary.
There are a few more conditions which need to be checked to ensure
whether we can actually bypass or not; in this particular case we are
using the YUV texture shaders, plus the image can also be flipped, so we
can't really use the bypass optimization in those situations.
These functions are primarily for use with filters, filters need to be
able to get the width/height of a target source without it necessarily
getting the post-filtered dimensions.
Previously I had it set so that it would set the async_width and
async_height variables to 0,0 if a null frame occurs, but the problem
with this is that it was inadvertently cause the frame cache to clear
when it starts receiving textures again because it detects it as a size
change.
So instead of changing async_width and async_height to 0 when a null
frame occurs, change it so that if the source is set to an inactive
state, make obs_source_get_width/_get_height return 0 for their
dimensions instead.
When a frame is processed by a filter, it comes directly from the
source's video frame cache. However, if a filter is using or processing
those frames for whatever reason, there would be no guarantee that the
frames would persist during processing, and frames could eventually be
deallocated unexpected, for example when the resolution or format
changes.
So the solution is to implement simple reference counting for the frames
so that the frames will exist until they have been released by any
source or filter that's using them.
Fix a bug where when a source first starts up its async cache, it
unintentionally resets its cache, which means that the first few frames
would be lost.
This optionally specifies that the int/float value should be displayed
with a slider rather than and up/down control. UIs are not necessarily
required to implement it, it's meant to be more of a hint.
If negative was specified it would not parse the negative sign and thus
would not interpret that number as negative, and would cause
shader/effect parsing to simply fail on the file.
This is particularly important for the filter pipeline in order to
ensure that when the last filter is reached that the original blend
state is properly reset.
When render targets are used, they output to the render target inverted
due to the way that opengl works. This fixes that issue by inverting
the projection matrix so that it renders the image upside down and
inverting the front face from counterclockwise to clockwise.
The wrong function was being used to recurse through the filter chain in
obs_source_process_filter, obs_source_get_[width/height] would get the
post-effect dimensions rather than the pre-effect dimensions.
The frame is definitely meant to be modifiable if needed, so it
shouldn't be const. I originally meant for these frames to be
duplicated, but with the source video cache that's both unnecessary and
would reduce performance.
If the audio data had the same format/samplerate as the obs audio
subsystem, it would fail to simply destroy the resampler and set it to
NULL, and then any audio data going through would use the resampler that
was being used before that, causing audio to become garbage.
This bug only started appearing when I recently changed the libobs
internal audio subsystem format to non-interleaved floating point, which
is a common format, and thus caused this bug to actually occur more
often.
I when a source has both async audio/video capability, it would ignore
audio until the video has started. There's really no need to do this,
when the video starts it'll just fix up the timing automatically.
This should fix the case where things like the media source would not be
able to play audio-only files.
Core API functions changed:
-----------------------------
EXPORT bool obs_reset_audio(struct audio_output_info *aoi);
EXPORT bool obs_get_audio_info(struct audio_output_info *aoi);
To:
-----------------------------
EXPORT bool obs_reset_audio(const struct obs_audio_info *oai);
EXPORT bool obs_get_audio_info(struct obs_audio_info *oai);
Core structure added:
-----------------------------
struct obs_audio_info {
uint32_t samples_per_sec;
enum speaker_layout speakers;
uint64_t buffer_ms;
};
Non-interleaved (planar) floating point output is standard with audio
filtering, so to prevent audio filters from having to worry about
different audio format implementations and for the sake consistency
between user interfaces, make it so that audio is always set to
non-interleaved floating point output.
API changed from:
------------------------
EXPORT void obs_service_apply_encoder_settings(obs_service_t *service,
obs_encoder_t *video_encoder,
obs_encoder_t *audio_encoder);
void obs_service_info::apply_encoder_settings(void *data
obs_encoder_t *video_encoder,
obs_encoder_t *audio_encoder);
To:
------------------------
EXPORT void obs_service_apply_encoder_settings(obs_service_t *service,
obs_data_t *video_encoder_settings,
obs_data_t *audio_encoder_settings);
void obs_service_info::apply_encoder_settings(void *data
obs_data_t *video_encoder_settings,
obs_data_t *audio_encoder_settings);
These changes make it so that instead of an encoder potentially being
updated more than once with different settings, that these functions
will be called for the specific settings being used, and the settings
will be updated according to what's required by the service.
This fixes that design flaw and ensures that there's no case where
obs_encoder_update is called where the settings might not have
service-specific settings applied.
This code was originally meant to skip some checks as an optimization,
but it did not account for async video sources and would call
obs_source_default_render (which is only for synchronous video sources),
this fixes the issue by just calling obs_source_video_render.
