tlsv1 alert protocol version (Synapse can't make TLS 1.3 requests, FreeBSD)

[pacija]

Description

Hi,

some admins who (like me) run their own homeservers told me their servers won't get room updates directly from my server, it takes another server to accept them, and relay them to their homeservers.

Steps to reproduce

• send a message in room
• get feedback from some users that my messages lag, that they don't get updates directly from my homeserver, it needs for some other server to accept them, then relay them to their homeserver
• check log and observe abovementioned INFO message

Homeserver

mimar.rs

Synapse Version

1.104.0

Installation Method

Other (please mention below)

Database

PostgreSQL

Workers

Single process

Platform

FreeBSD jail (14.0 x64) running on FreeBSD host (14.0 x64).

Configuration

I added federation_client_minimum_tls_version: "1.2" but the problem remains.

Relevant log output

(real domain changed to example.org)
2024-04-04 10:33:42,479 - synapse.http.matrixfederationclient - 799 - INFO - federation_transaction_transmission_loop-4667- {PUT-O-4699} [example.org] Request failed: PUT matrix-federation://example.org/_matrix/federation/v1/send/1712218846484: RequestTransmissionFailed:[Error([('SSL routines', '', 'tlsv1 alert protocol version')])]

Anything else that would be useful to know?

Same situation happens with current FreeBSD port/package (1.98.0). I created my custom port and upgraded to 1.104.0 (will send patches to FreeBSD after a few days of running it if no problems appear), but the problem remains.

[jrick]

My server (zettaport.com) is one of the ones @pacija's is unable to federate with (in the mimar.rs -> zettaport.com direction; federation updates from my server to his work properly). I'm using OpenBSD relayd as my reverse proxy which only accepts TLS 1.2+ connections. My relayd logs look like:

Apr  4 08:57:08 domain relayd[76588]: relay matrix, session 2148 (9 active), 0, 193.53.106.128 -> :8008, TLS handshake error: handshake failed: error:1402610B:SSL routines:ACCEPT_SR_CLNT_HELLO:wrong version number: Invalid argument

so it appears that mimar.rs is continuing to use an older, insecure TLS version for some reason.

[reivilibre]

Do you have the ability to run something like openssl s_client -connect zettaport.com:8448 in the same environment (jail?) as what Synapse is using?

That should pretty closely simulate what Synapse does to establish a connection. The output might be useful.

[pacija]

Hi,

in the meantime I upgraded to py39-matrix-synapse-1.105.1, current version in ports, the problem remains. I have attached the output you asked for.

zettaport.txt

[reivilibre]

Thanks for that.

I notice that your attached log file uses TLS 1.3. I get a similar result locally, notably you can see the server certificate etc.

However if I tell OpenSSL to use TLS 1.2, I get an alert and it doesn't show a server certificate

$ openssl s_client -tls1_2 -connect zettaport.com:8448
CONNECTED(00000003)
40B7E2C9907F0000:error:0A00042E:SSL routines:ssl3_read_bytes:tlsv1 alert protocol version:ssl/record/rec_layer_s3.c:1590:SSL alert number 70
---
no peer certificate available
---
No client certificate CA names sent
---
SSL handshake has read 7 bytes and written 210 bytes
Verification: OK
---
New, (NONE), Cipher is (NONE)
Secure Renegotiation IS NOT supported
Compression: NONE
Expansion: NONE
No ALPN negotiated
SSL-Session:
    Protocol  : TLSv1.2
    Cipher    : 0000
    Session-ID: 
    Session-ID-ctx: 
    Master-Key: 
    PSK identity: None
    PSK identity hint: None
    SRP username: None
    Start Time: 1717151048
    Timeout   : 7200 (sec)
    Verify return code: 0 (ok)
    Extended master secret: no
---

So it seems zettaport.com doesn't accept TLS 1.2. And your server seemingly can't speak TLS 1.3 for some reason.

Is there any chance you can ask your distro packager to see if they know why it can't speak TLS 1.3?

We use PyOpenSSL through Twisted. Twisted sets these options:
https://github.com/twisted/twisted/blob/c465c460c68bec3cf481c2815d1039da6796eb03/src/twisted/internet/ssl.py#L151-L155
but they seem correct to me.

You've already confirmed that the openssl CLI tool can use TLS v1.3, so the only thing that comes to mind is that Synapse is somehow using a different build or configuration of OpenSSL...?

It sounds like other servers are managing, so it doesn't seem like a direct Synapse problem. I also can't find any clues online but I don't know much about FreeBSD!

[reivilibre]

and @jrick from your message it sounds like you intended to accept TLS 1.2+, but right now you only accept TLS 1.3. This is maybe a separate problem for you, I don't know how widespread TLS 1.3 acceptance is by now but I also personally wouldn't expect it to be universal..

[jrick]

Thanks for that heads up. I took a second look at my config and it does intentionally disable 1.2.

[ofcaah]

https://cgit.freebsd.org/ports/commit/?id=e3f34f8a65e6aae0f177b5098b704ca1d0a62c4a
Amount of shortsightedness here is just astonishing.

Anyway, removing post-patch section from py-twisted port's Makefile, and doing a make reinstall restores ability to establish TLS 1.3 connections, at least on my FreeBSD 14.1