We were formally targeting ES2015 and relying on TypeScript desugaring
the use of newer features like async/await, which is unnecessary.
Desktop targets ES2020; we can too. One thing TypeScript was *not*
desugaring was BigInt literals, which we can now use freely.
Fix issues that made this harder to use in client libraries:
- mutable arguments made Net.cdsiLookup harder to call
- a missing error code field made discriminating a RateLimitedError un-idiomatic
Use the async CDSI lookup function introduced in libsignal-net and expose it via
the bridging layer to node. Add a typescript library that provides a more
convenient interface for callers and exposes a close-to-drop-in-compatible API
for the desktop client.
This adds integration bits for the new webpsan, a WebP image sanitizer -- which
currently simply checks the validity of a WebP file input, so that passing a
malformed file to an unsafe parser can be avoided. The integration pretty much
just leverages the integration work that was already done for mp4san.
Allows a client to request a credential for a backup-id without
revealing the backup-id to the issuing server. Later, the client may use
this to make requests for the backup-id without identifying themselves
to the server.
bridge_fn already supported async functions for Node, by running them
on the Node microtask queue using the work in the signal_neon_futures
crate. This PR fits that into the AsyncRuntime trait added for
bridge_io, allowing async bridge_fn and async bridge_io to share code
and the same basic structure when compiling for Node.
For the most part this should happen transparently without any
explicit adoption, like the previous change, but for Java code the
NoSessionException is now properly declared on SessionCipher.encrypt.
(This was always technically possible, but clients were expected to
have previously checked for session validity before using
SessionCipher; now that there's an expiration involved, that's not
strictly possible.)
And consolidate the implementations of these two separate checks; now
they both check for a valid session by looking for a sender chain
instead of just *some* current session, in addition to the new check
for an expired unacknowledged session. At the Rust level, this is now
one check named has_usable_sender_chain; at the app levels, the old
names of hasSenderChain (Java) and hasCurrentState (Swift, TypeScript)
have been preserved.
Tests to come in the next commit.
Like ProtocolAddresses in 88a2d5c, these APIs will eventually only
support ACIs, so introducing strong types now helps move in that
direction. However, the existing APIs that produce strings have not
been removed yet.
These work the same as the equivalent factory methods on ServiceId,
but throw if the resulting parsed ServiceId doesn't match the specific
type you were trying to parse.
Only the iOS client ever used this extra parameter, and it's one
that's easily stored alongside the reference to a store. This is
massively simpler than having it threaded down to the Rust
libsignal_protocol and back up through the bridging layer.
Signal Desktop only supports Ubuntu 20.04 and newer, so we no longer
need to build against Ubuntu 16.04 to ensure compatibility. And
bullseye-slim is a smaller base image than the Ubuntu images, so if we
don't specifically need an Ubuntu package this should be an easy
improvement.
In a future release ProtocolAddresses will *only* support ServiceIds,
so these APIs are designed to be the nullable version of the signature
they'll eventually have. Since ProtocolAddresses are created by the
client app in nearly all cases, they should be able to ignore the null
case if they only use ServiceIds in their input.
The JNI tests have also been conditionalized in case we want to take
this out for Android as well. (Node still unconditionally depends on
it being present.) I've given it a separate feature flag from just
ffi/jni/node so that we can preserve the tests Jessa wrote for each
platform.
This MP4 format "sanitizer" currently only transforms (when necessary) outgoing media on iOS, Android, or Desktop to
make it suitable for streaming playback by the recepient. In the future, it will validate and be able to either repair
or reject outbound AND inbound media, to prevent malformed media from being fed to third party or OS media players.
An generic io module was added to the libsignal rust bridge containing the InputStream trait, modeled loosely after
Java's InputStream, which calls back into the client language to perform reads or skips. This infrastructure could
potentially also be for any other future large data inputs to libsignal functions.
This is very similar to the AuthCredential used by the group server,
but using CallLinkParams to encrypt the user ID rather than
GroupParams (and using GenericServerParams to issue the credential
rather than the group server's ServerParams).
This will allow a user to request to create a call link from the chat
server without revealing anything about the room, and then later
actually create it by giving the room ID to the calling server without
identifying themself.
This involves a new, stripped-down GenericServer{Secret,Public}Params,
which currently only contains a generic "zkcredential" key. Apart from
the calling server not needing to handle all the credentials that the
group storage server supports, the structure of zkcredential means it
is safe to use the same key for multiple kinds of credentials.
Similarly, CallLink{Secret,Public}Params plays the same role as
Group{Secret,Public}Params for encrypting user IDs when talking to the
calling server.
Following from that, the APIs for CreateCallLinkCredentials are
located on the individual types (RequestContext, Request, Response,
Credential, Presentation) rather than all being on the Server*Params
types; adding a new credential type won't change the API of the
Server*Params types at all.
The main Server*Params may make use of zkcredential in the future as
well, but for now it's only for new Signal servers that want to use
zero-knowledge credentials.
At this point, the only special behavior of bridge_fn_buffer is to
support multiple return values for the C bridge (a pointer/length
pair), and that doesn't pull its weight. Remove it in favor of a plain
bridge_fn.
This did reveal that Username_Hash was using bridge_fn_buffer and now
produces a fixed-size array, imported into Swift as a tuple, so this
commit also factors out a new helper invokeFnReturningFixedLengthArray.
Follow-up to ec4faf26 where we tried to guess if the build directory
path was short enough for non-long-path-aware build tools. Give up on
that and just always use $RUNNER_TEMP if that's shorter.
-i (interactive) and -t (allocate a tty) allow the shell running
inside Docker to handle Ctrl-C (^C) and other shell commands, so you
can stop a command in the interactive process you ran it. However,
they only work if the containing shell (the one where you ran `docker
run`) is also interactive with a tty hooked up, so we test for that
first in both scripts that invoke `docker run`, using `test -t`.
--init passes signals from *outside* Docker down to its subprocesses,
so that cancellation from *another* context works for our Docker
images. This includes the Cancel button in GitHub Actions.
This was previously in the Java layer because it only really affects
the server, but it's more consistent to have all verification in the
Rust layer. We do lose the separate exception type for it, though.
The only supported way to target an older glibc is to build against
that glibc; consequently, we need to build on an Ubuntu 16 system (or
similar) to target Ubuntu 16. This requires downloading second-party
versions of Clang and CMake, which are too old in the default Ubuntu
repository, as well as building our own Python.
Do all this in a new Dockerfile based on Ubuntu 16.04. This isn't as
rigorous as the Java "reproducible build" Dockerfile, since we're not
pinning the base image or the repositories we're fetching from, but
it's still an image with the environment and tools we need.
Implements (a subset of) Intel's DCAP attestation,
making heavy use of 'boring' for X509 and ECDSA.
Cds2Client is now ready for use!
Co-authored-by: Jordan Rose <jrose@signal.org>
Co-authored-by: Ravi Khadiwala <ravi@signal.org>
This annoying function is implemented separately for each bridge
because it produces two results, and the optimal way of doing that for
each bridge differs.
Otherwise, we can run into paths that exceed the classic Windows path
limit due to the nesting of build systems (GitHub Actions > node-gyp >
Cargo > CMake > Visual Studio). Unfortunately, at least some of Visual
Studio's tools are not long-path-aware.
And tweak the test file to remind that the top-level zkgroup/index.ts
exists, though since we still don't reference most types by name in
the tests this wouldn't have actually caught the oversight.
This is a variant of AuthCredential that carries two UUIDs, intended
to be a user's ACI and PNI. Why? Because when you've been invited to a
group, you may have been invited by your ACI or by your PNI, or by
both, and it's easier for clients to treat all those states the same
by having a credential that covers both identities. The downside is
that it's larger (both the data, obviously, but also the zkgroup proof
of validity, unsurprisingly).
AnyAuthCredentialPresentation gains a 'get_pni_ciphertext' method,
which will return `None` for the existing presentations and
`Some(encrypted_pni)` for the new credential. Having a separate
credential type but a common presentation type makes it easier for the
server to handle all possible credentials uniformly.
Without this, two ByteArray types without any additional operations
are structurally equivalent, and so TypeScript permits passing one as
the other. (Thanks, Fedor!)
Like ProfileKeyCredential, but with an expiration timestamp embedded
in it. This has its own credential type and response type, but uses
the same request type as a "classic" ProfileKeyCredential, and
generates presentations usable with AnyProfileKeyCredential-
Presentation, so that existing server code accepting presentations
will automatically do the right thing.
Adoption for servers:
- Update secret params
- When presentations are saved in group state, use
ProfileKeyCredentialPresentation.getStructurallyValidV1PresentationBytes()
to maintain backwards compatibility with existing clients.
- Add an endpoint to issue ExpiringProfileKeyCredentials
- (future) Remove the endpoint that issues regular ProfileKeyCredentials
Adoption for clients, after the server has updated:
- Update public params
- Start fetching and using ExpiringProfileKeyCredentials instead of
regular ProfileKeyCredentials (the old endpoint will eventually
go away)
- Node: To bring types into harmony, a receipt's expiration time has
been changed to a `number` instead of a `bigint`
Upcoming work in `attest` requires additional X509 support, and swapping these libraries
is a negligible impact on binary size. This uses a fork of `cloudflare/boring`, as
we have some additions that haven’t yet been contributed upstream.
...which drops our dependency on Electron altogether. We originally
tested with electron-mocha to more closely resemble the Desktop app,
but libsignal-client doesn't actually use anything Electron-specific,
and because it uses N-API we don't have to sync up versions exactly
(and indeed we haven't been updating the Electron in this repo as
often as the Desktop app has taken new Electrons).
Two benefits of this: you can now run the tests on headless systems
(see the change to the CI script), and `yarn install` has less to
download.
- Update .nvmrc to match Desktop
- Update electron, electron-mocha, and typescript to match Desktop
- Update indirect dependency 'minimist' per `yarn audit`
In signal-neon-futures test package:
- Update mocha
- `yarn upgrade` to clear audit advisories
Optimize presentation of credentials (AuthCredentialPresentationV2, ProfileKeyCredentialPresentationV2, PniCredentialPresentationV2). Server will accept V1 or V2 presentations. Clients will produce V2.
Various improvements to FFI to support this, and some minor optimizations (in particular "lazy statics" to avoid redundant loading of SystemParams).
In Java these are subclasses of IllegalStateException, a
RuntimeException, so that every session operation isn't annotated as
throwing InvalidSessionException. Swift and TypeScript don't have
typed errors, so they're just additional specific cases that can be
caught.
The PublicKey type includes a type byte prefix to support different
kinds of public keys in the future, but that doesn't really apply to
HsmEnclave keys. This matches the Java implementation and the
recently-updated Swift implementation.
Native.d.ts only applies in TypeScript mode, not plain JavaScript.
This also confuses esbuild. Rather than try to fight that, return to
having Native not be exported at all, using a translation layer.
Also, make sure we use "true" TypeScript enums rather than `const
enum`, for the same reasons.
- Java: on IdentityKeyPair and IdentityKey, respectively
- Swift: on IdentityKeyPair and IdentityKey, respectively
- Node: on IdentityKeyPair and PublicKey; Node doesn't have a separate
IdentityKey API
For convenience, exposes IdentityKeyPair.generate() in Java and Node
as well. (This API already existed in Swift.)
Previously all APIs were available through the top-level index.ts, but
now the zkgroup APIs are in their own module. To access that in the
old packaging required writing
import * from '@signalapp/signal-client/node/dist/zkgroup'
This commit moves the package root into the node/ directory to
eliminate the 'node/' component, then adds a top-level, precompiled
zkgroup.js/.d.ts so that clients can use
import * from '@signalapp/signal-client/zkgroup'
Previously this was defined in the app layers, because zkgroup's
original codegen didn't support custom exception types. However, we
can now move it to a common implementation in Rust.
These APIs are designed to match the generated "simpleapi" entry
points in the original zkgroup repository, to make it easier to adapt
the existing Java, Swift, and TypeScript code to libsignal-client.
The cbindgen-generated signal_ffi.h now includes constants, so that
the fixed-size arrays used to serialize zkgroup types can use named
constants in Rust. This meant filtering out some constants that were
getting picked up but that should not be included.
Note that this commit makes references to Java exception types that
will be added in a later commit.
This will be used by zkgroup. Note that in order to print the type
correctly in C, a type `Serialized<FooBar>` will be translated to
`[u8; FOO_BAR_LEN]`, where 'FOO_BAR_LEN' has to be a constant that's
in scope.
u64 can't be represented as a primitive in Java or TypeScript (and for
the latter, Neon doesn't support bigint yet). However, for timestamps
represented as milliseconds, the integer-safe range of float64 still
covers more than 285,000 years, so it's reasonably safe to use
TypeScript's 'number' or Java's 'long' to represent these
ostensibly-64-bit values. Indicate this with a new Timestamp wrapper
type in the bridge layer.
In theory we could push this new Timestamp type down to the
libsignal-protocol crate. However, the protocol itself doesn't impose
any restrictions on the timestamp fields, so I figured it was best to
put it at the bridge layer, to indicate that it's about how Signal
specifically uses these fields.
This commit paves the way for being stricter about *other* u64 values
that might want to use the full 64-bit space.
Unlike bridge_get or bridge_get_bytearray, bridge_deserialize doesn't
do any complicated transformation of the return value to accept
optional or non-optional, failable and non-failable results alike. At
the same time, its syntax has been subtly different from the other
bridge_fn macros, dating from when we were first setting up this
library. Since the extra parameters to rename or disable a particular
bridge's entry point were rarely used, this commit removes them and
replaces those use sites with spelled-out bridge_fns. This in turn
allows removing the custom per-bridge implementations of
bridge_deserialize in favor of a bridge_fn-based implementation like
bridge_get already has.
Previously, we had HKDF-for-session-version-3, which matches RFC 5869,
and HKDF-for-session-version-2, which produced slightly different
results. However, nothing in the current versions of Signal uses
anything but the RFC-compliant version. Therefore, this commit removes
support for version 2 and deprecates the entry points that take a
version:
- Java: The HKDFv3 class is deprecated in favor of static methods on
the HKDF class.
- Swift: The hkdf function that takes a 'version' parameter is
deprecated in favor of a new overload that does not.
- TypeScript: The HKDF class is deprecated in favor of a top-level
hkdf function.
- Rust: The libsignal-protocol implementation of HKDF has been removed
entirely in favor of the hkdf crate.
There are no significant benchmark deltas from this change, and a
minimal code size increase that's the cost for removing our own
implementation of HKDF. The deprecations can be removed as a later
breaking change.
On Windows, node-gyp tries to be helpful by changing forward slashes
to backslashes and making paths relative to the source root rather
than the build directory, and indeed we rely on that. However, it has
to guess what's a path and what isn't. Previously, we worked around
that by manually stripping the prepended "..\", but the node-gyp
maintainers suggested a better workaround of using joined arguments
instead (`--foo=bar` instead of `--foo bar`).
This dedicated error is thrown when a recipient has a registration ID
that's out of the range used by Signal [0, 0x3FFF]. These IDs cannot
be encoded in the sealed sender v2 format and are not supported, even
though they don't cause any problems for 1:1 messages.
This allows a device to know whether it's the one that sent a bad
message, and take action accordingly.
We could have a slightly more typesafe API here by using
ProtocolAddress and extracting the device ID, but that doesn't match
up with getting the device ID out of a sealed sender certificate.
- Default: sender will not resend; an error should be shown
immediately
- Resendable: sender will try to resend; delay any error UI if
possible
- Implicit: don't show any error UI at all; this is something sent
implicitly like a typing message or a receipt
This checks if there is an active sender state using the given ratchet
key, for use with decryption error messages. In this case, the app may
choose to archive the current session, or take even stronger actions
such as fetching new prekeys for the recipient.
The app-visible change is that sealedSenderMultiRecipientEncrypt now
takes a SessionStore as well. Sessions will be looked up in bulk using
a new SessionStore API, 'loadExistingSessions' or
'getExistingSessions`. The registration ID is then loaded from each
session and included in the resulting SSv2 payload.
The implementation is a bit of a divergence from some other APIs in
libsignal-client in that the "look up in bulk" step is performed in
the Java, Swift, or TypeScript layer, with the resulting sessions
passed down to Rust. Why? Because otherwise we'd pass a list of
addresses into Rust, which would have to turn them back into a Java,
Swift, or TypeScript array to call the SessionStore method. This would
be (1) a bunch of extra work to implement, and (2) a waste of CPU when
we already /have/ a list of addresses in the correct format: the
argument to sealedSenderMultiRecipientEncrypt.
This is an example of "the boundaries between the Rust and
Java/Swift/TypeScript parts of the library don't have to be perfect;
they're internal to the overall product". In this case, we've taken
that a little further than usual: usually we try to make the
libsignal-protocol API as convenient as possible as well, but here it
had to be a bit lower-level to satisfy the needs of the app language
wrappers. (Specifically, callers need to fetch the list of
SessionRecords themselves.)
P.S. Why doesn't v1 of sealed sender include registration IDs? Because
for SSv1, libsignal-client isn't producing the entire request body to
upload to the server; it's only producing the message content that
will be decrypted by the recipient. With SSv2, the serialized message
the recipient downloads has both shared and per-recipient data in it,
which the server must assemble from the uploaded request. Because of
this, SSv2's encrypt API might as well produce the entire request.
Registration IDs are used to detect if a device ID has been reused,
since the new device will (with high probability) use a different
randomly-generated registration ID from the old one. The server should
be able to validate this for SSv2 like it does for SSv1, though the
handling of this for SSv1 is in the various apps.
We still encode the "current" version in the message version byte, but
the part that the receiver will check is now based on the session's
original message version rather than the "current" version in the
sender. (Note that these are the /same/ version right now, so this
change won't have any effect on the current wire format.)
This matches the behavior of SignalMessage and PreKeySignalMessage.
The parameters for these constructors cannot be chosen correctly in
isolation; messages always need to be created as part of a session.
The Node APIs have been renamed with leading underscores rather than
removed because we *did* have tests that explicitly constructed
messages just to test that they faithfully preserve information. In
this case the parameters are plausible values but not used for
anything.
This was MessageCounterError in libtextsecure; it's an "error" in that
it interrupts processing of an individual message, but that message
is then be discarded in practice.
Within a new 'Errors' module, we have a base class that indicates this
is a strongly-typed error, and an enum to identify what kind of error
it is and thus what extra properties it might have. TypeScript's type
narrowing support make this possible to do safely as long as all
instances of the base class do in fact have the extra properties that
match their code (and do not have an invalid code).
To expose this to Rust (via Neon):
- After loading the Rust library into Node, set an 'Errors' property
with the relevant error types on the module object.
- Whenever a bridge_fn produces an error, pass it to a new
SignalNodeError::throw API along with the 'this' object, which is
assumed to have to be the object with the 'Errors' property.
This is a little less tidy than how we do Java exceptions, but it
comes from not having access to the error classes by some kind of
absolute name. Alternate approaches considered include:
- Use an initialized-once global. Downside: would not work if you ever
had more than one Node engine live in a process, or quit and restarted
it.
- Store the errors on the global object under some long, complicated
key (like "org.signal.libsignal-client.Errors"). Downside: pollutes
the global object.
- Generate the base class using Neon instead of writing it in
TypeScript. Downsides: inconvenient, difficult to maintain, harder to
use /from/ TypeScript.
We're optimizing for size overhead in this encoding, so forego the
flexibility of protobufs in favor of a flat encoding (though one that
still uses protobuf's varints). Additionally, this encoding includes
the recipients inline in the message so the client can dump it all to
server in one go.
As a side effect, this means an SSv2 message encoded for sending no
longer has the same format as one encoded for receiving when there's
only one recipient. Consequently, all the tests need to be modified to
"fan out" a multi-recipient message to several single-recipient
messages. For simplicity, the wrapper language tests only support this
operation for SSv2 messages sent to exactly one recipient.
- Add a new "multi-recipient encrypt" entry point
- Add an "encrypt v1 sealed sender from UnidentifiedSenderMessage-
Content" entry point
- Add a public constructor for UnidentifiedSenderMessageContent
- Change group_encrypt to return a CiphertextMessage instead of bytes,
so it can be used with the above
- Java: add SenderKeyStore to SignalProtocolStore requirements
It's a payload message, something that would go inside a SignalMessage
or PreKeySignalMessage. Drop it from all the enums, and while we're
here let's sync up the CiphertextMessageType::SenderKey case with the
sealed sender content type and the envelope content type.
This slipped in the original translation to Rust, but it doesn't
matter in practice because both nibbles of the version byte have had
the same value for a long time.
With distribution IDs embedded in SenderKeyMessage and
SenderKeyDistributionMessage, the abstraction of SenderKeyName (a
sender address + distribution ID tuple) is no longer pulling its
weight. Remove it from the implementation and the public API.
The distribution ID is used to identify which key a particular sender
is using to encrypt their SenderKeyMessage, so it has to be known as
part of decryption. The previous design had the distribution ID stored
alongside each message (perhaps on the "envelope" structure that's
received from the server), but that's harder to keep track of, and it
would only be present for certain message kinds anyway.
Clarifies the use of "ID" in SenderKey-related APIs. I've left
deprecated entry points for Java but not for Swift and TypeScript
(which are not in use yet).
- SenderKeyMessage::key_id -> chain_id (avoids double "key" in name)
- SenderKeyDistributionMessage::id -> chain_id (to match SKM)
- SenderKeyName::group_id -> distribution_id (it's not the global group ID)
Specifically, use 'NativeImpl' to refer to the run-time bindings
(instead of 'SC'), and 'Native' to refer to the TypeScript module for
those bindings (instead of 'SignalClient'). This makes
compile-time diagnostics clearer, since the overall Node package is
named 'signal-client'.
Previously we defined one entry point for FFI (Swift) that took an
extra "context" parameter (to pass through iOS's database
transactions), and one for JNI+Node that did not (no context needed
currently). But this is all in our glue layer, which doesn't need to
be a perfect reflection of the outside interface. Remove that
duplication by accepting a Context parameter for both JNI and Node
that, for now, must be null.
This will be necessary for async, which needs to keep the boxed Rust
values alive. (Node N-API only allows keeping JS objects alive; boxed
values don't count on their own.)
