Supplied key (org.apache.sshd.common.config.keys.OpenSshCertificateImpl) is not a RSAPublicKey instance

[liuziyu1226]

Version

2.8.0

Bug description

I overwrote the authenticate function of the PublickeyAuthenticator.java and registered it with the ssh server.

sshServer.setPublickeyAuthenticator(publicKeyAuth);

But when the client provides the key and certificate, MINA SSHD displays an error:

2024-12-25 17:29:30.882 sshd-SshServer[53600a30](port=2222)-nio2-thread-5 WARN  org.apache.sshd.server.session.ServerUserAuthService.warn - handleUserAuthRequestMessage(ServerSessionImpl[null@/127.0.0.1:56562]) Failed (InvalidKeyException) to authenticate using factory method=publickey: Supplied key (org.apache.sshd.common.config.keys.OpenSshCertificateImpl) is not a RSAPublicKey instance

After debugging, I found that the problem is that when MINA SSHD initializes the signature verifier of the public key, the public key passed in is of type OpenSshCertificateImpl, while Signature expects an RSAPublicKey object. Why does this problem occur? How can I solve it?

Actual behavior

I rewrote the initVerifier function in SignatureRSA.java and now I can login correctly.
Old initVerifier function:

public void initVerifier(SessionContext session, PublicKey key) throws Exception {
      super.initVerifier(session, key);
      RSAKey rsaKey = ValidateUtils.checkInstanceOf(key, RSAKey.class, "Not an RSA key");
      verifierSignatureSize = getVerifierSignatureSize(rsaKey);
}

I overwrote the initVerifier function:

public void initVerifier(SessionContext session, PublicKey key) throws Exception {
      if (key instanceof OpenSshCertificate){
          super.initVerifier(session, ((OpenSshCertificate) key).getCertPubKey());
          RSAKey rsaKey = ValidateUtils.checkInstanceOf(((OpenSshCertificate) key).getCertPubKey(), RSAKey.class, "Not an RSA key");
          verifierSignatureSize = getVerifierSignatureSize(rsaKey);
      }else {
          super.initVerifier(session, key);
          RSAKey rsaKey = ValidateUtils.checkInstanceOf(key, RSAKey.class, "Not an RSA key");
          verifierSignatureSize = getVerifierSignatureSize(rsaKey);
      }
  }

I am still curious as to why this happens？ And whether there are any hidden dangers in my modification.
`

Relevant log output

No response

Other information

No response

[tomaswolf]

The server side of user authentication with OpenSshCertificates was not fully implemented, and apparently is lacking tests.

See issue SSHD-1161.

The real cause of the problem you encounter is in

mina-sshd/sshd-core/src/main/java/org/apache/sshd/server/auth/pubkey/UserAuthPublicKey.java

Line 95 in 7b19f6b

if (key instanceof OpenSshCertificate) {

and

mina-sshd/sshd-core/src/main/java/org/apache/sshd/server/auth/pubkey/UserAuthPublicKey.java

Line 132 in 7b19f6b

verifier.initVerifier(session, key);

If the key is an OpenSshCertificate, initVerifier should be called with key.getCertPubKey(), not with key.

So, to answer your questions: it happens because there is a bug. There are no dangers to your modification, but it'll work only for RSA keys. The real fix has to be in UserAuthPublicKey (server side) so that it works for all key types. Similar to how it's done in

mina-sshd/sshd-core/src/main/java/org/apache/sshd/client/kex/DHGClient.java

Lines 184 to 186 in 7b19f6b

	if (serverKey instanceof OpenSshCertificate) {
	OpenSshCertificate openSshKey = (OpenSshCertificate) serverKey;
	serverPublicHostKey = openSshKey.getCertPubKey();

and

mina-sshd/sshd-core/src/main/java/org/apache/sshd/client/kex/DHGClient.java

Line 222 in 7b19f6b

verif.initVerifier(session, serverPublicHostKey);

Plus there needs to be a test for this.

Note that there are other bits of authenticating with a certificate missing on the server side:

mina-sshd/sshd-core/src/main/java/org/apache/sshd/server/auth/pubkey/UserAuthPublicKey.java

Lines 108 to 110 in 7b19f6b

	// TODO: cert.getCaKey() must be either in authorized_keys, marked as a CA key
	// and not revoked, or in TrustedUserCAKeys and then also match
	// AuthorizedPricipalsFile, if present.

[liuziyu1226]

Yes, the fundamental problem is that when initializing Signature, instead of getting cert publickey from OpenSshCertificate, OpenSshCertificate is directly passed in as the public key.
I tested the above method and found two problems:

1. It can only adapt to certs with RSA encryption algorithm
2. Even if the initVerifier() function of all encryption algorithms is rewritten, the signature will fail when the encryption method of cert is different from that of cert publickey.
   Can I please raise a PR to fix the issue in UserAuthPublicKey?

[tomaswolf]

Sure, if you can provide a PR with the fix and tests with different certificates/key types, that'd be great.

[liuziyu1226]

OK, I will write my PR in the next few weeks after work, and I hope my work can make a small contribution to this open source project.