0
0
mirror of https://github.com/OpenVPN/openvpn3.git synced 2024-09-20 12:12:15 +02:00
Commit Graph

301 Commits

Author SHA1 Message Date
James Yonan
ee5fe53607
HTTP server: implement throttling
Signed-off-by: James Yonan <james@openvpn.net>
2020-01-31 10:24:31 +02:00
James Yonan
4f91530f2f
HTTP: added Alt Routing support to HTTP client/server
Alt Routing allows services on meshed PG nodes to communicate
with services on other nodes via the secure mesh, and without
requiring SSL.

Signed-off-by: James Yonan <james@openvpn.net>
2020-01-31 10:24:30 +02:00
James Yonan
daa98d1e45
WS::Server changes:
* Cancel general timeouts for websockets

* CoarseTimer must always be reset when its associated
  AsioTimer is cancelled.

Signed-off-by: James Yonan <james@openvpn.net>
2020-01-31 10:24:30 +02:00
James Yonan
fc19e73bcd
WS::Server: use C++11 member initializers
Signed-off-by: James Yonan <james@openvpn.net>
2020-01-31 10:24:30 +02:00
James Yonan
fddf4c534b
HTTP layer: added Websocket support
Signed-off-by: James Yonan <james@openvpn.net>
2020-01-31 10:24:30 +02:00
James Yonan
179d00e15b
Added i/o abstraction layer.
Created a lightweight abstraction layer so that another i/o
reactor can be dropped in place of asio.

This commit includes:

* Added ASIO=1 to many "go" scripts that require asio
* Renamed "asio::" to "openvpn_io::".

Signed-off-by: James Yonan <james@openvpn.net>
2020-01-31 10:24:29 +02:00
James Yonan
979c831466
Moved asio files from openvpn/common to openvpn/asio
Signed-off-by: James Yonan <james@openvpn.net>
2020-01-31 10:24:29 +02:00
James Yonan
50fba999f6
Moved openvpn::to_string() to separate source file.
Signed-off-by: James Yonan <james@openvpn.net>
2020-01-31 10:24:29 +02:00
James Yonan
1bea8dace5
copyright: updated to 2017
Signed-off-by: James Yonan <james@openvpn.net>
2020-01-31 10:24:28 +02:00
James Yonan
7dedd61bfa
license : this branch (common) is proprietary, so remove all GPL headers.
Signed-off-by: James Yonan <james@openvpn.net>
2020-01-31 10:24:28 +02:00
James Yonan
2ac9e18a34
copyright : updated to 2016 2020-01-31 10:24:28 +02:00
James Yonan
f09a0c4402
HTTP server : struct Factory now inherits from OPENVPN_HTTP_SERV_RC instead of being hardcoded to RC<thread_unsafe_refcount>. 2020-01-31 10:24:27 +02:00
James Yonan
9649eb83f2
In HTTP client and server, enable TCP_NODELAY on socket. 2020-01-31 10:24:27 +02:00
James Yonan
cbc9a2ec6a
In WS::Server::Config, remove the constructor and use C++11 member initializers. 2020-01-31 10:24:27 +02:00
James Yonan
d03dd13f1e
In WS::Server::Listener::Client, added new virtual method http_pipeline_peek() to allow HTTP servers to observe and possibly modify/delete incoming out-of-context raw socket data before the data is written to the out-of-context pipeline (where it would be later processed by SSL and/or HTTP layers). 2020-01-31 10:24:26 +02:00
James Yonan
323db959d9
Added unix_fd() method to WS::Server::Listener::Client to allow derived classes to obtain the raw unix domain socket. 2020-01-31 10:24:25 +02:00
James Yonan
7185e45ff0
Minor change to httpserv.hpp -- Bracket unix domain socket code with #ifdef ASIO_HAS_LOCAL_SOCKETS instead of #if !defined(OPENVPN_PLATFORM_WIN) 2020-01-31 10:24:25 +02:00
James Yonan
0b2865cac4
In httpserv.hpp, added WS::Server::Listener::Client::get_parent() method. 2020-01-31 10:24:25 +02:00
James Yonan
ab6c29cfbc
Factored out HTTP Server acceptor classes into openvpn/acceptor. 2020-01-31 10:24:25 +02:00
James Yonan
a6ddc344fa
In HTTP server (httpserv.hpp), allow users to specify the Windows SDDL string used to set the permissions on created named pipes.
Also, refactor some Windows-specific code into secattr.hpp
(SECURITY_ATTRIBUTES stuff) and npinfo.hpp (getting info
about named pipe peer).
2020-01-31 10:24:24 +02:00
James Yonan
257ba661a0
For Windows named pipe implementation in httpserv.hpp, removed PIPE_WAIT flag from CreateNamedPipeA call, as it seems incompatible with FILE_FLAG_OVERLAPPED. 2020-01-31 10:24:24 +02:00
James Yonan
7f10bd7fae
Added Windows named pipe support to HTTP client/server library. 2020-01-31 10:24:24 +02:00
James Yonan
7723bcba40
Ported httpserv.hpp to Windows, mostly by removing Unix-only constructs for the Windows build such as unix domain socket support. 2020-01-31 10:24:23 +02:00
James Yonan
5815a9a483
Minor change to httpserv.hpp -- include openvpn/common/core.hpp instead of openvpn/linux/core.hpp for portability between Linux and Mac OS X. 2020-01-31 10:24:23 +02:00
James Yonan
288bdf879f
Fixed preprocessor issue with httpserv.hpp:
* Moved ifndef/define OPENVPN_WS_HTTPSERV_H to the
  top of the file.
2020-01-31 10:24:23 +02:00
James Yonan
96910692d6
Asio async_wait lambdas should always check error status. 2020-01-31 10:24:22 +02:00
James Yonan
a108ba302b
WS::Server::Listener can now handle Listen::List lists that have mixed SSL/non-SSL items. 2020-01-31 10:24:22 +02:00
James Yonan
e9f5907138
Added WS::Server::Listener::Client::is_local() method to test if socket is a unix domain socket. 2020-01-31 10:24:22 +02:00
James Yonan
fb2a6e3d93
In HTTP client/server, added msg_overhead_bytes member to Config class to allow for accounting of message overhead.
This is to prevent attacks where a large number of very
small messages (such as 1 byte each) are sent to the
server to force it to consume more memory than the
max_content_bytes limit would normally allow.
2020-01-31 10:24:22 +02:00
James Yonan
cc83e8c352
Added ContentInfo::content_encoding on the server side (httpserv.hpp). 2020-01-31 10:24:21 +02:00
James Yonan
241623621b
Added AGPL license header to HTTP, Java, and common code. 2020-01-31 10:24:21 +02:00
James Yonan
8830d26123
HTTP changes.
Both Client/Server side:

1. Support asynchronous sending of content via
   set_async_out() and http_content_out_finish()
   methods and http_content_out_needed() callback.

2. Added ContentInfo::extra_headers for caller-defined
   extra HTTP headers.

3. Made ContentInfo::CHUNKED into a constexpr
   type to match ContentInfo::length member var.

4. Set FD_CLOEXEC on socket.

5. Added remote_ip_port() method to allow remote IP
   address and port of socket to be obtained.

Client side:

1. In Host, added hint string to override transport host
   when a specific IP address should be used for host
   instead of resolving host via DNS.

2. Added Host::host_port_str() method.

3. Make general_timeout work like a true timeout, where
   traffic resets the timer (this is how server-side
   already works).

4. Added new method remote_endpoint_str() to match
   the same method on server-side.

5. Added new method host_hint() to return the current Host
   object, but set the hint/port fields to the live
   remote IP address/port of the connection.

6. Added new callback http_mutate_resolver_results() to
   allow user to modify the order of endpoint list returned
   by resolver.

Server side:

1. Make content_len_t into a 64-bit signed int since one
   of its possible values is -1 for CHUNKED.

2. Added ContentInfo::no_cache member var to trigger headers
   telling clients to not cache the content.

3. Added Factory::stop() virtual method for users to
   optionally override.

4. Made get_client_id() method public.

5. Fixed issue where code that allocates a client_id
   wasn't actually calling new_client_id().
2020-01-31 10:24:21 +02:00
James Yonan
f085fa9352
HTTP client/server -- now that socket is a smart pointer, make sure to check it is defined before calling close() method. 2020-01-31 10:24:20 +02:00
James Yonan
dfba2a6f28
In HTTP client/server stop() methods, do socket->close() instead of socket.reset(), as the latter may free the underlying socket while asio still holds dangling pointers to it. 2020-01-31 10:24:20 +02:00
James Yonan
e7222bc361
HTTP client/server changes:
* HTTP client and server now support unix domain sockets
  via AsioPolySock abstraction.

* HTTP server now supports Basic auth credentials.

* HTTP server now supports peercred authentication
  over unix domain sockets.

* HTTP server now supports file creation permission
  bits on unix domain socket.

* Added udstest tool to test HTTP client over unix domain
  sockets.
2020-01-31 10:24:20 +02:00
James Yonan
b357c56ee3
Don't use deprecated asio features. 2020-01-31 10:24:19 +02:00
James Yonan
941274e31a
Obsoleted asiodispatch in favor of C++11 lambdas. 2020-01-31 10:24:19 +02:00
James Yonan
b3071d8e56
HTTP server can now listen on unix domain sockets (in addition to TCP sockets). Server can also listen on multiple sockets concurrently. 2020-01-31 10:24:19 +02:00
James Yonan
a56bf236bd
Boost dependency elimination -- final removal of Boost dependency. Asio is now included as a standalone, header-only dependency. 2020-01-31 10:24:18 +02:00
James Yonan
944b4d5d22
pgproxy changes:
* Moved ASIO resolver object out of connection-scoped
  PortShare and into thread-scoped ThreadSpecific.

* Fixed issue where delayed release of proxy socket and/or
  resolver was causing termination delays.
2020-01-31 10:24:18 +02:00
James Yonan
0cd244a53b
Make remote_endpoint_str() method const in httpserv.hpp. 2020-01-31 10:24:18 +02:00
James Yonan
4c21518c84
HTTP server:
* Fixed issue where HTTP server did not detect
  mid-session client disconnect.

* Implement parent_handoff (for proxies).

* Improved handling of residual content, allowing
  HTTP 1.1 pipelining to be supported.
2020-01-31 10:24:18 +02:00
James Yonan
15fd7a0965
OPENVPN_HTTP_SERV_RC may be defined to override WS::Server::Listener::Client RC root class. 2020-01-31 10:24:17 +02:00
James Yonan
3c46024674
Refactored methods for parsing/validating hosts/ports into openvpn/common/hostport.hpp 2020-01-31 10:24:17 +02:00
James Yonan
870ac130f3
Renamed boost::intrusive_ptr<T> usage to RCPtr<T>. 2020-01-31 10:24:17 +02:00
James Yonan
db9a8d557a
Moved array_size() function to its own header file. 2020-01-31 10:24:16 +02:00
James Yonan
72b5cae69a
Refactored ScopedPtr usage to std::unique_ptr. Removed ScopedPtr (moved to test/unused). 2020-01-31 10:24:16 +02:00
James Yonan
ea6592f94c
Extended WS::Client to allow the use of any OpenVPN client transport driver for the HTTP/S transport layer. This enables use of HTTP CONNECT and OBFS proxy drivers. To enable, set the WS::Client::Config::transcli member to any TransportClientFactory instance. 2020-01-31 10:24:16 +02:00
James Yonan
9eaa36d031
HTTP/S server changes:
In WS::Server::Listener::Client::Initializer, save the
socket in a ScopedPtr, for safety against possible leaks.

This requires that users of Initializer declare it as
non-const rather than const.
2020-01-31 10:24:15 +02:00
James Yonan
2bb6e33b8a
Added pgproxy, an HTTP proxy supporting both CONNECT and OBFS. 2020-01-31 10:24:15 +02:00
James Yonan
32440efd6a
Refactored web services classes (openvpn/ws) to support HTTP/S servers. 2020-01-31 10:24:15 +02:00
James Yonan
993d0c4a4d
Initial commit of test HTTP server. 2020-01-31 10:24:15 +02:00
James Yonan
f250c4c921 WS::ClientSet: silently ignore when Asio is missing the results.randomize() method
If HTTPCLI_RANDOMIZE_RESULTS_REQUIRED is defined, cause a compile-time
error if Asio is not compiled with results.randomize() method.

If HTTPCLI_RANDOMIZE_RESULTS_REQUIRED is NOT defined, opportunistically
compile results.randomize() usage only if available in Asio.

Signed-off-by: James Yonan <james@openvpn.net>
2020-01-29 09:27:46 -07:00
Arne Schwabe
9c50badeb4
Fix integer comparison problems introduced by the merge of released
Signed-off-by: Arne Schwabe <arne@openvpn.net>
2020-01-16 15:34:32 +01:00
Lev Stipakov
a4d9989d81
Update license on files moved from common
Signed-off-by: Lev Stipakov <lev@openvpn.net>
2020-01-09 12:24:20 +02:00
James Yonan
50196ed5fa
JsonClient/AppServer: full HTTP compression support
* gzip or lz4 supported for standard HTTP.

* lz4 support for websocket.

Signed-off-by: James Yonan <james@openvpn.net>
2020-01-09 12:24:20 +02:00
James Yonan
7297015895
websocket: make Sender::frame() const
Sender::frame() was already const-ready, it just didn't
have the const attribute specified.

Signed-off-by: James Yonan <james@openvpn.net>
2020-01-09 12:24:20 +02:00
James Yonan
6055aaf34c
HTTP layer: added Websocket support
Signed-off-by: James Yonan <james@openvpn.net>
2020-01-09 12:24:19 +02:00
James Yonan
14379360c1
WS::Creds::load_from_header(): use the simpler string::split() rather than Split::by_char()
Signed-off-by: James Yonan <james@openvpn.net>
2020-01-09 12:24:19 +02:00
James Yonan
c8b4fe322e
WS::Creds: use atomic_thread_fence(std::memory_order_acq_rel) instead of OPENVPN_COMPILER_FENCE
Signed-off-by: James Yonan <james@openvpn.net>
2020-01-09 12:24:19 +02:00
James Yonan
95f1654921
WS::Creds: use crypto::str_neq() when comparing credentials against expected values
Signed-off-by: James Yonan <james@openvpn.net>
2020-01-09 12:24:19 +02:00
James Yonan
0ac1999cd0
copyright: updated to 2017
Signed-off-by: James Yonan <james@openvpn.net>
2020-01-09 12:24:18 +02:00
James Yonan
d3751a65c2
license : this branch (common) is proprietary, so remove all GPL headers.
Signed-off-by: James Yonan <james@openvpn.net>
2020-01-09 12:24:18 +02:00
James Yonan
4dc149ffd9
copyright : updated to 2016 2020-01-09 12:24:18 +02:00
James Yonan
4ebd66bfbb
httpcreds : redefined WS::Creds::load_from_options() to pass through the UserPass flags value. 2020-01-09 12:24:18 +02:00
James Yonan
c86fd2a5de
Added write_private() methods from core (no Windows implementation yet).
In httpcreds.hpp, use write_private() for creating creds
files rather than UMask.
2020-01-09 12:24:17 +02:00
James Yonan
3b20b9bf6a
Fix to httpcreds.hpp for Windows: #ifdef out umask functionality. 2020-01-09 12:24:17 +02:00
James Yonan
94419fd21c
Update httpcreds.hpp with SplitLines change where S_OK was renamed to S_OKAY due to Windows symbol conflict. 2020-01-09 12:24:17 +02:00
James Yonan
8ca935bcd4
In httpcreds.hpp, added WS::Creds::load_from_options() method. 2020-01-09 12:24:17 +02:00
James Yonan
5965e046ed
Added AGPL license header to HTTP, Java, and common code. 2020-01-09 12:24:16 +02:00
James Yonan
75acd76f40
HTTP client/server changes:
* HTTP client and server now support unix domain sockets
  via AsioPolySock abstraction.

* HTTP server now supports Basic auth credentials.

* HTTP server now supports peercred authentication
  over unix domain sockets.

* HTTP server now supports file creation permission
  bits on unix domain socket.

* Added udstest tool to test HTTP client over unix domain
  sockets.
2020-01-09 12:24:16 +02:00
Arne Schwabe
99dbc6848b
Fix adding int to a string
Compiler warning:

openvpn/ws/httpcommon.hpp:157:135: warning: adding 'int' to a string does not append to the string [-Wstring-plus-int]
2020-01-09 12:24:16 +02:00
James Yonan
f1546d6b8b
WS::HTTPBase: added more detail to http_content_out_finish() exception on unexpected state
Signed-off-by: James Yonan <james@openvpn.net>
2020-01-09 12:24:16 +02:00
James Yonan
7b6d4cf6cb
AppServer, WS::HTTPBase: added support for ssl_did_full_handshake() method added to SSL layer
Signed-off-by: James Yonan <james@openvpn.net>
2020-01-09 12:24:15 +02:00
James Yonan
73f8525f9e
Implemented TLS session ticket support in the HTTP layers.
Added support for PGDB server and clients, and the chat test.

Signed-off-by: James Yonan <james@openvpn.net>
2020-01-09 12:24:15 +02:00
James Yonan
9e7470c37d
AppServer, WS::HTTPBase: change update_max_content_bytes() to reduce_max_content_bytes()
Signed-off-by: James Yonan <james@openvpn.net>
2020-01-09 12:24:15 +02:00
James Yonan
77f7e06786
Change to commit "httpcommon: fixed race that can occur on high-speed persistent connections"
The original commit has some unintended side effects
that break server-side code.

This commit tries a different approach: do an early
return from http_in() when buffer size is zero.

Signed-off-by: James Yonan <james@openvpn.net>
2020-01-09 12:24:15 +02:00
James Yonan
8e6e8e759d
httpcommon: fixed race that can occur on high-speed persistent connections
I observed a case where http_in() (running as a client) called
parent().base_http_done_handler() twice for the same transaction!
Normally the 'ready' var blocks this sort of behavior, but with
a high-speed persistent session, the 'ready' var can transition
so quickly as to create a window for a double-done race.

The fix is to use a more robust filter against unsolicited input
after base_http_done_handler() is called by setting rr_status to
REQUEST_REPLY::Parser::undefined.  This value is never matched
in httpcommon, so it effectively turns http_in() into a no-op when
set.

There is also the question of whether unsolicited input should
be considered a fatal error on a persistent session.  It probably
should, but this fix focuses on a corner case where http_in()
is called with a zero-length buffer, presumably from the SSL/TLS
layer.

Signed-off-by: James Yonan <james@openvpn.net>
2020-01-09 12:24:14 +02:00
James Yonan
fa37101597
httpcommon: minor comment addition
Signed-off-by: James Yonan <james@openvpn.net>
2020-01-09 12:24:14 +02:00
James Yonan
111cabd518
AppServer: minor debug logging change of E_GENERAL_TIMEOUT
At debug_level 1, only show timeouts if they occur
within a request/reply transaction, not if they
are triggered outside of a transaction by the
expiration of keepalive.

Signed-off-by: James Yonan <james@openvpn.net>
2020-01-09 12:24:14 +02:00
James Yonan
f7e8719852
websocket: don't debit websocket stream traffic to max content size limit
Signed-off-by: James Yonan <james@openvpn.net>
2020-01-09 12:24:14 +02:00
James Yonan
b8886e0482
httpcommon: make is_deferred() public
Signed-off-by: James Yonan <james@openvpn.net>
2020-01-09 12:24:13 +02:00
James Yonan
4e8dffd02b
HTTP layer: added Websocket support
Signed-off-by: James Yonan <james@openvpn.net>
2020-01-09 12:24:13 +02:00
James Yonan
2136c62a35
copyright: updated to 2017
Signed-off-by: James Yonan <james@openvpn.net>
2020-01-09 12:24:13 +02:00
James Yonan
0d62ae1fb0
license : this branch (common) is proprietary, so remove all GPL headers.
Signed-off-by: James Yonan <james@openvpn.net>
2020-01-09 12:24:13 +02:00
James Yonan
d48c197d01
HTTP library : extended debug flag OPENVPN_DEBUG_HTTP to log incoming as well as outgoing network data. 2020-01-09 12:24:12 +02:00
James Yonan
54b91242d3
copyright : updated to 2016 2020-01-09 12:24:12 +02:00
James Yonan
8c341d7d11
Fixed a bug in update_max_content_bytes() in the HTTP common code, where the code was modifying max_content_bytes in the global configuration instead of the specific HTTP instance.
Because pgapi and ccds use this method as an anti-DoS measure,
it means that POSTS of > 4096 bytes that fail authentication
would permanently lower the future max POST size to 4096 bytes
until server restart.
2020-01-09 12:24:12 +02:00
James Yonan
017571afad
In httpcommon.hpp, added a static_assert() to ensure that CONTENT_LENGTH_TYPE type is signed. 2020-01-09 12:24:12 +02:00
James Yonan
a095cd1237
Fixed a minor issue in httpcommon.hpp -- SSL read was not respecting reserved tailroom. 2020-01-09 12:24:11 +02:00
James Yonan
36a2a401e7
Added new HTTP common method update_max_content_bytes() to allow the configuration limit max_content_bytes to be modified mid-session. This can be useful to dial-down the limit on unauthenticated sessions. 2020-01-09 12:24:11 +02:00
James Yonan
f0aea806e3
In HTTP client/server, added msg_overhead_bytes member to Config class to allow for accounting of message overhead.
This is to prevent attacks where a large number of very
small messages (such as 1 byte each) are sent to the
server to force it to consume more memory than the
max_content_bytes limit would normally allow.
2020-01-09 12:24:11 +02:00
James Yonan
1eb31f13e6
In httpcommon.hpp, added HTTPBase::http_config() method. 2020-01-09 12:24:11 +02:00
James Yonan
0e31ffa928
Added AGPL license header to HTTP, Java, and common code. 2020-01-09 12:24:10 +02:00
James Yonan
d9173907e2
HTTP changes.
Both Client/Server side:

1. Support asynchronous sending of content via
   set_async_out() and http_content_out_finish()
   methods and http_content_out_needed() callback.

2. Added ContentInfo::extra_headers for caller-defined
   extra HTTP headers.

3. Made ContentInfo::CHUNKED into a constexpr
   type to match ContentInfo::length member var.

4. Set FD_CLOEXEC on socket.

5. Added remote_ip_port() method to allow remote IP
   address and port of socket to be obtained.

Client side:

1. In Host, added hint string to override transport host
   when a specific IP address should be used for host
   instead of resolving host via DNS.

2. Added Host::host_port_str() method.

3. Make general_timeout work like a true timeout, where
   traffic resets the timer (this is how server-side
   already works).

4. Added new method remote_endpoint_str() to match
   the same method on server-side.

5. Added new method host_hint() to return the current Host
   object, but set the hint/port fields to the live
   remote IP address/port of the connection.

6. Added new callback http_mutate_resolver_results() to
   allow user to modify the order of endpoint list returned
   by resolver.

Server side:

1. Make content_len_t into a 64-bit signed int since one
   of its possible values is -1 for CHUNKED.

2. Added ContentInfo::no_cache member var to trigger headers
   telling clients to not cache the content.

3. Added Factory::stop() virtual method for users to
   optionally override.

4. Made get_client_id() method public.

5. Fixed issue where code that allocates a client_id
   wasn't actually calling new_client_id().
2020-01-09 12:24:10 +02:00
James Yonan
18c3789943
HTTP server:
* Fixed issue where HTTP server did not detect
  mid-session client disconnect.

* Implement parent_handoff (for proxies).

* Improved handling of residual content, allowing
  HTTP 1.1 pipelining to be supported.
2020-01-09 12:24:10 +02:00
James Yonan
7925880b7f
OPENVPN_HTTP_SERV_RC may be defined to override WS::Server::Listener::Client RC root class. 2020-01-09 12:24:10 +02:00
James Yonan
24070412f3
Renamed types.hpp to size.hpp since it now only defines size_t and ssize_t. 2020-01-09 12:24:09 +02:00
James Yonan
51603baee6
Refactored ScopedPtr usage to std::unique_ptr. Removed ScopedPtr (moved to test/unused). 2020-01-09 12:24:09 +02:00
James Yonan
2343e0aff0
Added pgproxy, an HTTP proxy supporting both CONNECT and OBFS. 2020-01-09 12:24:09 +02:00