OIDC login creates @=22username=22:somedomain.tld instead of @username:somedomain.tld

[jessebot]

Description

Using Zitadel (it's like a modern version of keycloak) as an OIDC provider for matrix and element works, but creates a weird username with =22 in it, like @=22username=22:somedomain.tld. I had followed the docs, and previously, maybe 6 months ago when I setup my last server, it wasn't a problem. Not sure if there's anything I can specify to strip out those weird characters.

Steps to reproduce

1. setup a zitadel instance (which uses the same oidc config block as keycloak)

2. Setup matrix and element via docker using the following in your homeserver.yaml as per docs:

oidc_providers:
  - idp_id: zitadel
    idp_name: zitadel
    scopes:
      - openid
      - profile
    user_mapping_provider:
      config:
        display_name_template: '"{{ user.name }}"'
        localpart_template: '"{{ user.preferred_username }}"'
    issuer: https://zitadel.domain.tld
    client_id: 123456789012345@core
    client_secret: jfdaf1hjdsakhflkadshfjkldahsjkfldhsajklfh4sajklfhjdksalhfjkds

3. try to login to element via zitadel

4. it lets you login but, the username comes back as @=22myusername=22:domain.tld and there's no way to reset it.

5. I also tried to use python to strip the weird characters, but it didn't work. Example of what I tried:

    user_mapping_provider:
      config:
        display_name_template: '"{{ user.name }}"'
        localpart_template: '"{{ user.preferred_username.replace("=22","") }}"'

Homeserver

personally hosted homeserver using matrixdotorg/synapse:v1.102.0

Synapse Version

{"server_version":"1.102.0"}

Installation Method

Docker (matrixdotorg/synapse)

Database

one fresh v14.x postgresql server and it has not been restored from a backup

Workers

Single process

Platform

I'm using a helm chart to deploy this stack which includes synapse (matrixdotorg/synapse:v1.102.0) and element (vectorim/element-web:v1.11.61). Zitadel is running in the same cluster and it does technically work, but the username is always really weird. It's more of an inconvenience than it is a breaking change.

Configuration

oidc_providers:
  - idp_id: zitadel
    idp_name: zitadel
    scopes:
      - openid
      - profile
    user_mapping_provider:
      config:
        display_name_template: '"{{ user.name }}"'
        localpart_template: '"{{ user.preferred_username }}"'
    issuer: https://zitadel.domain.tld
    client_id: 123456789012345@core
    client_secret: jfdaf1hjdsakhflkadshfjkldahsjkfldhsajklfh4sajklfhjdksalhfjkds

Relevant log output

Defaulted container "synapse" out of: synapse, add-secret-values-to-config (init), generate-signing-key (init), pip-install-extra-packages (init)
Starting synapse with args -m synapse.app.homeserver --config-path /data/homeserver.yaml
Synapse requires that a list of trusted key servers are specified in order to
provide signing keys for other servers in the federation.

This homeserver does not have a trusted key server configured in
homeserver.yaml and will fall back to the default of 'matrix.org'.

Trusted key servers should be long-lived and stable which makes matrix.org a
good choice for many admins, but some admins may wish to choose another. To
suppress this warning, the admin should set 'trusted_key_servers' in
homeserver.yaml to their desired key server and 'suppress_key_server_warning'
to 'true'.

In a future release the software-defined default will be removed entirely and
the trusted key server will be defined exclusively by the value of
'trusted_key_servers'.
--------------------------------------------------------------------------------
2024-03-19 10:34:20,200 - root - 355 - WARNING - main - ***** STARTING SERVER *****
2024-03-19 10:34:20,201 - root - 356 - WARNING - main - Server /usr/local/lib/python3.11/site-packages/synapse/app/homeserver.py version 1.102.0
2024-03-19 10:34:20,201 - root - 361 - WARNING - main - Copyright (c) 2023 New Vector, Inc
2024-03-19 10:34:20,201 - root - 362 - WARNING - main - Licensed under the AGPL 3.0 license. Website: https://github.com/element-hq/synapse
2024-03-19 10:37:28,214 - synapse.api.auth.internal - 258 - WARNING - GET-40 - Invalid access token in auth: <class 'pymacaroons.exceptions.MacaroonDeserializationException'> cannot determine data format of binary-encoded macaroon.
2024-03-19 10:43:21,411 - synapse.handlers.sso - 839 - WARNING - GET-117 - failed to save the user avatar
2024-03-19 10:57:44,940 - synapse.storage.txn - 786 - WARNING - PUT-328 - [TXN OPERROR] {add_user_account_data-3ea} could not serialize access due to concurrent update
 0/5

Anything else that would be useful to know?

I cross posted this to the zitadel github repo: zitadel/zitadel#7596 Thanks for any guidance you can provide. Happy to provide additional logs from element if you need them. Above logs are from synapse container in matrix pod.

[reivilibre]

=22 is the encoded form of " (it's like URL-encoding but with =s. If you look at an ASCII table for 22 (hex) you see ").

In this case I think your issue is that your template has quotation marks in it:

replace

        display_name_template: '"{{ user.name }}"'
        localpart_template: '"{{ user.preferred_username }}"'

with

        display_name_template: '{{ user.name }}'
        localpart_template: '{{ user.preferred_username }}'

You will possibly need to recreate your user, as I think the username is generated on first login and will remain associated afterwards — I think the identity provider usually supplies a stable internal ID for identifying users without using their name.

[jessebot]

thank you for your really fast response! 🙏 I will try that in just a bit and report back!

[jessebot]

=22 is the encoded form of " (it's like URL-encoding but with =s. If you look at an ASCII table for 22 (hex) you see ").

In this case I think your issue is that your template has quotation marks in it:

replace

        display_name_template: '"{{ user.name }}"'
        localpart_template: '"{{ user.preferred_username }}"'

with

        display_name_template: '{{ user.name }}'
        localpart_template: '{{ user.preferred_username }}'

You will possibly need to recreate your user, as I think the username is generated on first login and will remain associated afterwards — I think the identity provider usually supplies a stable internal ID for identifying users without using their name.

That was it! Sorry, this got confusing because I'm deploying synapse through Argo CD ApplicationSet -> Argo CD App -> Helm -> Docker container in pod on k8s, and so the escaping of things can get a little weird and produce some weird results. For anyone using the above helm chart via Argo CD or anything else that needs you to escape this templating, this is what you'd put into your values to escape a layer of gotemplating:

# previously I had '{{`"{{ user.preferred_username }}"`}}' which broke things
localpart_template: "{{`{{ user.preferred_username }}`}}"
display_name_template: "{{`{{ user.name }}`}}"

example Argo CD ApplicationSet to deploy matrix

---
apiVersion: argoproj.io/v1alpha1
kind: ApplicationSet
metadata:
  name: matrix-web-app-set
  namespace: argocd
spec:
  goTemplate: true
  # generator allows us to source specific values from an external k8s secret
  generators:
    - plugin:
        configMapRef:
          name: secret-var-plugin-generator
        input:
          parameters:
            secret_vars:
              - matrix_hostname
  template:
    metadata:
      name: matrix-web-app
      annotations:
        argocd.argoproj.io/sync-wave: "3"
    spec:
      project: matrix
      destination:
        server: https://kubernetes.default.svc
        namespace: matrix
      syncPolicy:
        syncOptions:
          - ApplyOutOfSyncOnly=true
        automated:
          prune: true
          selfHeal: true
      source:
        repoURL: 'https://small-hack.github.io/matrix-chart'
        targetRevision: 7.0.2
        chart: matrix
        helm:
          valuesObject:
            # Runtime configuration for Synapse and settings related to the Matrix protocol
            matrix:
              oidc:
                enabled: true
                existingSecret: "synapse-oidc"
                secretKeys:
                  issuer: "issuer"
                  client_id: "client_id"
                  client_secret: "client_secret"

                providers:
                  - idp_id: zitadel
                    idp_name: zitadel
                    scopes:
                      - "openid"
                      - "profile"
                    user_mapping_provider:
                      config:
                        localpart_template: "{{`{{ user.preferred_username }}`}}"
                        display_name_template: "{{`{{ user.name }}`}}"

Thanks, @reivilibre ! 👋