So, scene editing was interesting (and by interesting I mean
excruciating). I almost implemented 'manipulator' visuals (ala 3dsmax
for example), and used 3 modes for controlling position/rotation/size,
but in a 2D editing, it felt clunky, so I defaulted back to simply
click-and-drag for movement, and then took a similar though slightly
different looking approach for handling scaling and reszing.
I also added a number of menu item helpers related to positioning,
scaling, rotating, flipping, and resetting the transform back to
default.
There is also a new 'transform' dialog (accessible via menu) which will
allow you to manually edit every single transform variable of a scene
item directly if desired.
If a scene item does not have bounds active, pulling on the sides of a
source will cause it to resize it via base scale rather than by the
bounding box system (if the source resizes that scale will apply). If
bounds are active, it will modify the bounding box only instead.
How a source scales when a bounding box is active depends on the type of
bounds being used. You can set it to scale to the inner bounds, the
outer bounds, scale to bounds width only, scale to bounds height only,
and a setting to stretch to bounds (which forces a source to always draw
at the bounding box size rather than be affected by its internal size).
You can also set it to be used as a 'maximum' size, so that the source
doesn't necessarily get scaled unless it extends beyond the bounds.
Like in OBS1, objects will snap to the edges unless the control key is
pressed. However, this will now happen even if the object is rotated or
oriented in any strange way. Snapping will also occur when stretching
or changing the bounding box size.
This covers the basics of devices. Mostly functional but not at 100%
yet. Uses 'libdshowcapture' library to capture directshow video/audio.
Both libdshowcapture and the plugin still need some work. Should at
least capture basic webcams and capture cards for the time being.
Added github gist API uploading to the help menu to help make problems a
bit easier to debug in the future. It's somewhat vital that this
functionality be implemented before any release in order to analyze any
given problem a user may be experiencing.
This doesn't add FLV file output to the user interface yet, but we'll
get around to that eventually. This just adds an FLV output type.
Also, removed ftello/fseeko because off_t is a really annoying data
type, and I'd rather have a firm int64_t for large sizes, so I named it
to os_fseeki64 and os_ftelli64 instead, and changed the file size
function to return an int64_t.
Add a 'source selection' dialog to replace the 'enter a name' dialog.
This new dialog allows you to make new instances of pre-existing sources
so that you can add a pre-existing source to a different scene, or in to
the same scene more than once.
Also started implementing locale.
Comtemplating switching to JSON-based locale later, so we can add things
like descriptions/disambiguation, and so we can use jansson's built-in
hash table when doing the string lookup.
- Add volume control
These volume controls are basically nothing more than sliders. They
look terrible and hopefully will be as temporary as they are
terrible.
- Allow saving of specific non-user sources via obs_load_source and
obs_save_source functions.
- Save data of desktop/mic audio sources (sync data, volume data, etc),
and load the data on startup.
- Make it so that a scene is created by default if first time using the
application. On certain operating systems where supported, a default
capture will be created. Desktop capture on mac, particularly. Not
sure what to do about windows because monitor capture on windows 7 is
completely terrible and is bad to start users off with.
Starting with VS2012, microsoft made a terribly stupid change to the way
working directory is handled. It basically will not work properly if
you need to run your executable from a specific location, it would only
run properly from debug if you also had your target directory set to an
environment variable, which was incredibly stupid. The visual studio
development team is run by chimpanzees with tiny brains, headed by
a broken AI core designed by gilbert gottfried.
I made a workaround for it so that it's much easier to set up debugging
in VS2013, so that all you need to do is enter in the the working
directory for it to work properly. Still somewhat of a pain but the Qt
addin kept overwriting the god forsaken environment path, and made
setting it up much more difficult than it was supposed to be.
- Updated the services API so that it links up with an output and
the output gets data from that service rather than via settings.
This allows the service context to have control over how an output is
used, and makes it so that the URL/key/etc isn't necessarily some
static setting.
Also, if the service is attached to an output, it will stick around
until the output is destroyed.
- The settings interface has been updated so that it can allow the
usage of service plugins. What this means is that now you can create
a service plugin that can control aspects of the stream, and it
allows each service to create their own user interface if they create
a service plugin module.
- Testing out saving of current service information. Saves/loads from
JSON in to obs_data_t, seems to be working quite nicely, and the
service object information is saved/preserved on exit, and loaded
again on startup.
- I agonized over the settings user interface for days, and eventually
I just decided that the only way that users weren't going to be
fumbling over options was to split up the settings in to simple/basic
output, pre-configured, and then advanced for advanced use (such as
multiple outputs or services, which I'll implement later).
This was particularly painful to really design right, I wanted more
features and wanted to include everything in one interface but
ultimately just realized from experience that users are just not
technically knowledgable about it and will end up fumbling with the
settings rather than getting things done.
Basically, what this means is that casual users only have to enter in
about 3 things to configure their stream: Stream key, audio bitrate,
and video bitrate. I am really happy with this interface for those
types of users, but it definitely won't be sufficient for advanced
usage or for custom outputs, so that stuff will have to be separated.
- Improved the JSON usage for the 'common streaming services' context,
I realized that JSON arrays are there to ensure sorting, while
forgetting that general items are optimized for hashing. So
basically I'm just using arrays now to sort items in it.
This plugin is just a generic service plugin for basic RTMP streaming
service stuff.
This just has a 'common' service that has a list of common/simple
streaming services that don't have their own custom service modules, and
then a 'custom' service that allows you to enter in the stream URL and
key manually, without a service/server list.
Also, copy the jansson VS projects file (don't modify the old one) so
that it's located in the vs/2013 directory, so that other libraries can
properly link with it without having to enter in extra information just
to include jansson
Add API for streaming services. The services API simplifies the
creation of custom service features and user interface.
Custom streaming services later on will be able to do things such as:
- Be able to use service-specific APIs via modules, allowing a more
direct means of communicating with the service and requesting or
setting service-specific information
- Get URL/stream key via other means of authentication such as OAuth,
or be able to build custom URLs for services that require that sort
of thing.
- Query information (such as viewer count, chat, follower
notifications, and other information)
- Set channel information (such as current game, current channel title,
activating commercials)
Also, I reduce some repeated code that was used for all libobs objects.
This includes the name of the object, the private data, settings, as
well as the signal and procedure handlers.
I also switched to using linked lists for the global object lists,
rather than using an array of pointers (you could say it was..
pointless.) ..Anyway, the linked list info is also stored in the shared
context data structure.
- Implement the RTMP output module. This time around, we just use a
simple FLV muxer, then just write to the stream with RTMP_Write.
Easy and effective.
- Fix the FLV muxer, the muxer now outputs proper FLV packets.
- Output API:
* When using encoders, automatically interleave encoded packets
before sending it to the output.
* Pair encoders and have them automatically wait for the other to
start to ensure sync.
* Change 'obs_output_signal_start_fail' to 'obs_output_signal_stop'
because it was a bit confusing, and doing this makes a lot more
sense for outputs that need to stop suddenly (disconnections/etc).
- Encoder API:
* Remove some unnecessary encoder functions from the actual API and
make them internal. Most of the encoder functions are handled
automatically by outputs anyway, so there's no real need to expose
them and end up inadvertently confusing plugin writers.
* Have audio encoders wait for the video encoder to get a frame, then
start at the exact data point that the first video frame starts to
ensure the most accrate sync of video/audio possible.
* Add a required 'frame_size' callback for audio encoders that
returns the expected number of frames desired to encode with. This
way, the libobs encoder API can handle the circular buffering
internally automatically for the encoder modules, so encoder
writers don't have to do it themselves.
- Fix a few bugs in the serializer interface. It was passing the wrong
variable for the data in a few cases.
- If a source has video, make obs_source_update defer the actual update
callback until the tick function is called to prevent threading
issues.
Now that we have the priorties window in we can finally be able to
select windows for capture source such as window capture. Only took
about an hour or two to write.
Also, fixed some depednency issues on winmm.lib with obs-outputs
- Add start/stop code to obs-output module
- Use a circular buffer for the buffered encoder packets instead of a
dynamic array
- Add pthreads.lib as a dependency to obs-output module on windows in
visual studio project files
- Fix an windows export bug for avc parsing functions on windows.
Also, rename those functions to be more consistent with each other.
- Make outputs use a single function for encoded data rather than
multiple functions
- Add the ability to make 'text' properties be passworded
- obs-outputs module: Add preliminary code to send out data, and add
an FLV muxer. This time we don't really need to build the packets
ourselves, we can just use the FLV muxer and send it directly to
RTMP_Write and it should automatically parse the entire stream for us
without us having to do much manual code at all. We'll see how it
goes.
- libobs: Add AVC NAL packet parsing code
- libobs/media-io: Add quick helper functions for audio/video to get
the width/height/fps/samplerate/etc rather than having to query the
info structures each time.
- libobs (obs-output.c): Change 'connect' signal to 'start' and 'stop'
signals. 'start' now specifies an error code rather than whether it
simply failed, that way the client can actually know *why* a failure
occurred. Added those error codes to obs-defs.h.
- libobs: Add a few functions to duplicate/free encoder packets
- Add a properties window for sources so that you can now actually edit
the settings for sources. Also, display the source by itself in the
window (Note: not working on mac, and possibly not working on linux).
When changing the settings for a source, it will call
obs_source_update on that source when you have modified any values
automatically.
- Add a properties 'widget', eventually I want to turn this in to a
regular nice properties view like you'd see in the designer, but
right now it just uses a form layout in a QScrollArea with regular
controls to display the properties. It's clunky but works for the
time being.
- Make it so that swap chains and the main graphics subsystem will
automatically use at least one backbuffer if none was specified
- Fix bug where displays weren't added to the main display array
- Make it so that you can get the properties of a source via the actual
pointer of a source/encoder/output in addition to being able to look
up properties via identifier.
- When registering source types, check for required functions (wasn't
doing it before). getheight/getwidth should not be optional if it's
a video source as well.
- Add an RAII OBSObj wrapper to obs.hpp for non-reference-counted
libobs pointers
- Add an RAII OBSSignal wrapper to obs.hpp for libobs signals to
automatically disconnect them on destruction
- Move the "scale and center" calculation in window-basic-main.cpp to
its own function and in its own source file
- Add an 'update' callback to WASAPI audio sources
- Implement OBS encoder interface. It was previously incomplete, but
now is reaching some level of completion, though probably should
still be considered preliminary.
I had originally implemented it so that encoders only have a 'reset'
function to reset their parameters, but I felt that having both a
'start' and 'stop' function would be useful.
Encoders are now assigned to a specific video/audio media output each
rather than implicitely assigned to the main obs video/audio
contexts. This allows separate encoder contexts that aren't
necessarily assigned to the main video/audio context (which is useful
for things such as recording specific sources). Will probably have
to do this for regular obs outputs as well.
When creating an encoder, you must now explicitely state whether that
encoder is an audio or video encoder.
Audio and video can optionally be automatically converted depending
on what the encoder specifies.
When something 'attaches' to an encoder, the first attachment starts
the encoder, and the encoder automatically attaches to the media
output context associated with it. Subsequent attachments won't have
the same effect, they will just start receiving the same encoder data
when the next keyframe plays (along with SEI if any). When detaching
from the encoder, the last detachment will fully stop the encoder and
detach the encoder from the media output context associated with the
encoder.
SEI must actually be exported separately; because new encoder
attachments may not always be at the beginning of the stream, the
first keyframe they get must have that SEI data in it. If the
encoder has SEI data, it needs only add one small function to simply
query that SEI data, and then that data will be handled automatically
by libobs for all subsequent encoder attachments.
- Implement x264 encoder plugin, move x264 files to separate plugin to
separate necessary dependencies.
- Change video/audio frame output structures to not use const
qualifiers to prevent issues with non-const function usage elsewhere.
This was an issue when writing the x264 encoder, as the x264 encoder
expects non-const frame data.
Change stagesurf_map to return a non-const data type to prevent this
as well.
- Change full range parameter of video scaler to be an enum rather than
boolean
- Implement windows monitor capture (code is so much cleaner than in
OBS1). Will implement duplication capture later
- Add GDI texture support to d3d11 graphics library
- Fix precision issue with sleep timing, you have to call
timeBeginPeriod otherwise windows sleep will be totally erratic.
- Add WASAPI audio capture for windows, input and output
- Check for null pointer in os_dlopen
- Add exception-safe 'WinHandle' and 'CoTaskMemPtr' helper classes that
will automatically call CloseHandle on handles and call CoTaskMemFree
on certain types of memory returned from windows functions
- Changed the wide <-> MBS/UTF8 conversion functions so that you use
buffers (like these functions are *supposed* to behave), and changed
the ones that allocate to a different naming scheme to be safe
- Signals and dynamic callbacks now require declarations to be made
before being used. What this does is allows us to get information
about the functions dynamically which can be relayed to the user and
plugins for future extended usage (this should have big implications
later for scripting in particular, hopefully).
- Reduced the number of types calldata uses from "everything I could
think of" to simply integer, float, bool, pointer/object, string.
Integer data is now stored as long long. Floats are now stored as
doubles (check em).
- Use a more consistent naming scheme for lexer error/warning macros.
- Fixed a rather nasty bug where switching to an existing scene would
cause it to increment sourceSceneRefs, which would mean that it would
never end up never properly removing the source when the user clicks
removed (stayed in limbo, obs_source_remove never got called)
I can't believe I wasn't doing this. This is why file output was
getting corrupted. Audio and video send in data from separate threads.
I should be embarassed for not having considered that.
Key lesson: Increase threading paranoia levels. Apparently my
threading paranoid levels are lackluster.
Added a "master" volume for the entire audio subsystem.
Also, added a "presentation" volume for both the master volume and for
each invidiaul source. The presentation volume is used to control
things like transitioning volumes, preventing sources from outputting
any audio when they're inactive, as well as some other uses in the
future.
Add a scaler interface (defaults to swscale), and if a separate output
wants to use a different scale or format than the default output format,
allow a scaler instance to be created automatically for that output,
which will then receive the new scaled output.
- Changed glMapBuffer to glMapBufferRange to allow invalidation. Using
just glMapBuffer alone was causing some unacceptable stalls.
- Changed dynamic buffers from GL_DYNAMIC_WRITE to GL_STREAM_WRITE
because I had misunderstood the OpenGL specification
- Added _OPENGL and _D3D11 builtin preprocessor macros to effects to
allow special processing if needed
- Added fmod support to shaders (NOTE: D3D and GL do not function
identically with negative numbers when using this. Positive numbers
however function identically)
- Created a planar conversion shader that converts from packed YUV to
planar 420 right on the GPU without any CPU processing. Reduces
required GPU download size to approximately 37.5% of its normal rate
as well. GPU usage down by 10 entire percentage points despite the
extra required pass.
There were a *lot* of warnings, managed to remove most of them.
Also, put warning flags before C_FLAGS and CXX_FLAGS, rather than after,
as -Wall -Wextra was overwriting flags that came before it.
Originally, the rendering system was designed to only display sources
and such, but I realized there would be a flaw; if you wanted to render
the main viewport in a custom way, or maybe even the entire application
as a graphics-based front end, you wouldn't have been able to do that.
Displays have now been separated in to viewports and displays. A
viewport is used to store and draw sources, a display is used to handle
draw callbacks. You can even use displays without using viewports to
draw custom render displays containing graphics calls if you wish, but
usually they would be used in combination with source viewports at
least.
This requires a tiny bit more work to create simple source displays, but
in the end its worth it for the added flexibility and options it brings.
The API used to be designed in such a way to where it would expect
exports for each individual source/output/encoder/etc. You would export
functions for each and it would automatically load those functions based
on a specific naming scheme from the module.
The idea behind this was that I wanted to limit the usage of structures
in the API so only functions could be used. It was an interesting idea
in theory, but this idea turned out to be flawed in a number of ways:
1.) Requiring exports to create sources/outputs/encoders/etc meant that
you could not create them by any other means, which meant that
things like faruton's .net plugin would become difficult.
2.) Export function declarations could not be checked, therefore if you
created a function with the wrong parameters and parameter types,
the compiler wouldn't know how to check for that.
3.) Required overly complex load functions in libobs just to handle it.
It makes much more sense to just have a load function that you call
manually. Complexity is the bane of all good programs.
4.) It required that you have functions of specific names, which looked
and felt somewhat unsightly.
So, to fix these issues, I replaced it with a more commonly used API
scheme, seen commonly in places like kernels and typical C libraries
with abstraction. You simply create a structure that contains the
callback definitions, and you pass it to a function to register that
definition (such as obs_register_source), which you call in the
obs_module_load of the module.
It will also automatically check the structure size and ensure that it
only loads the required values if the structure happened to add new
values in an API change.
The "main" source file for each module must include obs-module.h, and
must use OBS_DECLARE_MODULE() within that source file.
Also, started writing some doxygen documentation in to the main library
headers. Will add more detailed documentation as I go.
Implement a properties definition interface to allow modules to export
general properties associated with objects of libobs.
The properties definition interface allows the option for automatic
settings UI generation (which will make simple plugins easier to develop
without the need for user interface), as well as allow real-time
property editing of values of things like sources/outputs/etc without
having to open property dialogs. More property types can be added in
the future as needed as well.
Add the ability to be able to call and use toolkit-specific or
program-specific user interface in modules.
User interface code can be either bundled with the module, or 'split'
out in to separate libraries (recommended).
There are three reasons why splitting is recommended:
1.) It allows plugins to be able to create custom user interface for
each toolkit if desired.
2.) Often, UI will be programmed in one language (the language of the
toolkit), and core logic may be programmed in another. This
allows plugins to keep the languages separated if necessary.
3.) It prevents direct linkage of UI toolkits libraries with core
module logic.
Splitting is not required, though is recommended if you want your plugin
to be more flexible with other user interface toolkits or programs.
Will implement a generic properties lookup next, which will be used for
automatic UI handling so that plugin UI isn't necessarily required.
Add a fairly easy to use settings interface that can be passed to
plugins, and replaced the old character string system that was being
used before. The new data interface allows for an easier method of
getting/altering settings for plugins, and is built to be serializable
to/from JSON.
Also, removed another wxFormBuilder file that was no longer in use.
- Move over the last of the original settings dialog code to QT. It was
actually a bit easier to write in the QT version. wxWidgets was
definitely not ideal for that because the pages would fully
create/destroy every time.
- [Win32] Fix os_dlopen so that it only appends .dll if not present
- [MacOS] Fix name dialog text edit widget issue (it would be better if
we could just use the list widget for editing labels, will have to
look in to that in the future)
- Tweak the settings UI a bit more and make 30 FPS default
- Add a macro to convert a QString to a UTF-8 const char * string
- Rename build/plugins to build/obs-plugins
- Remove the last of the wxWidgets code
Fixed a few files that went over 80 columns, mostly just a nitpack on my
part.
libobs/obs-nix.c had a rather bad case of leading whitespace.
Also, fixed the x86 obs-studio project files so that it would properly
output to the right directory. It couldn't find libobs.lib because
obs-studio's project settings had it outputting to a different place
than the rest of the projects.
--------------------------------------------------
Notes and details
--------------------------------------------------
Why was this done? Because wxWidgets was just lacking in many areas. I
know wxWidgets is designed to be used with native controls, and that's
great, but wxWidgets just is not a feature-complete toolkit for
multiplatform applications. It lacks in dialog editors, its code is
archaic and outdated, and I just feel frustrated every time I try to do
things with it.
Qt on the other hand.. I had to actually try Qt to realize how much
better it was as a toolkit. They've got everything from dialog editors,
to an IDE, a debugger, build tools, just everything, and it's all
top-notch and highly maintained. The focus of the toolkit is
application development, and they spend their time trying to help
people do exactly that: make programs. Great support, great tools,
and because of that, great toolkit. I just didn't want to alienate any
developers by being stubborn about native widgets.
There *are* some things that are rather lackluster about it and design
choices I disagree with though. For example, I realize that to have an
easy to use toolkit you have to have some level of code generation.
However, in my personal and humble opinion, moc just feels like a
terrible way to approach the problem. Even now I feel like there are a
variety of ways you could handle code generation and automatic
management of things like that. I don't like the idea of circumventing
the language itself like that. It feels like one giant massive hack.
--------------------------------------------------
Things that aren't working properly:
--------------------------------------------------
- Settings dialog is not implemented. The dialog is complete but the
code to handle the dialog hasn't been constructed yet.
- There is a problem with using Qt widgets as a device target on
windows, with at least OpenGL: if I have the preview widget
automatically resize itself, it seems to cause some sort of video
card failure that I don't understand.
- Because of the above, resizing the preview widget has been disabled
until I can figure out what's going on, so it's currently only a
32x32 area.
- Direct3D doesn't seem to render correctly either, seems that the
viewport is messed up or something. I'm sort of confused about
what's going on with it.
- The new main window seems to be triggering more race conditions than
the wxWidgets main window dialog did. I'm not entirely sure what's
going on here, but this may just be existing race conditions within
libobs itself that I just never spotted before (even though I tend to
be very thorough with race conditions any time I use variables
cross-thread)
- Added some code for FFmpeg output that I'm still playing around with.
Right now I'm just trying to get it to output to file and try to
understand the FFmpeg/libav APIs. Hopefully in the future this plugin
can be used for any sort of output to FFmpeg.
- Fixed a cast warning in audio-io.c with size_t -> uint32_t
- Renamed the 'video_info' and 'audio_info' structures to
'video_conver_info' and 'audio_convert_info' to better represent their
actual purpose, and to avoid confusion with 'audio_output_info' and
'video_output_info' structures.
- Removed a few macros from obs-def.h that were at one point going to be
used but no longer going to be used (at least for now)
- First, I redid the output interface for libobs. I feel like it's
going in a pretty good direction in terms of design.
Right now, the design is so that outputs and encoders are separate.
One or more outputs can connect to a specific encoder to receive its
data, or the output can connect directly to raw data from libobs
output itself, if the output doesn't want to use a designated encoder.
Data is received via callbacks set when you connect to the encoder or
raw output. Multiple outputs can receive the data from a single
encoder context if need be (such as for streaming to multiple channels
at once, and/or recording with the same data).
When an encoder is first connected to, it will connect to raw output,
and start encoding. Additional connections will receive that same
data being encoded as well after that. When the last encoder has
disconnected, it will stop encoding. If for some reason the encoder
needs to stop, it will use the callback with NULL to signal that
encoding has stopped. Some of these things may be subject to change
in the future, though it feels pretty good with this design so far.
Will have to see how well it works out in practice versus theory.
- Second, Started adding preliminary RTMP/x264 output plugin code.
To speed things up, I might just make a direct raw->FFmpeg output to
create a quick output plugin that we can start using for testing all
the subsystems.
