[crypto,x509] fix tls-server-end-point signature algorithm selection

This reverts commit 00baf58a71. That
change appears to have been incorrect. It's described as simplying
retrieving the "default signature digest", but it actually changed the
function's behavior entirely. The function wasn't retrieving defaults
previously.

A certificate contains, among other things, a public key and a
signature. The public key is the public key of the subject. However, the
signature was generated by the issuer. That is, if I get a certificate
from a CA, the public key will be my public key and the signature will
be my CA's signature over the certificate contents.

Now, the original code returned the digest used in the certificate's
signature. That is, it tells you which signature algorithm did my *CA*
use to sign my certificate.

The new code extracts the certificate's public key (my public key, not
the CA's). This doesn't necessarily tell you the signature algorithm, so
it then asks OpenSSL what the "default" signature algorithm would it use
with the key. This notion of "default" is ad-hoc and has changed over
time with OpenSSL releases. It doesn't correspond to any particular
protocol semantics. It's not necessarily the signature algorithm of the
certificate.

Now, looking at where this function is used, it's called by
freerdp_certificate_get_signature_alg, which is called by
tls_get_channel_binding to compute the tls-server-end-point channel
binding. That code cites RFC 5929, which discusses picking the hash
algorithm based on the certificate's signatureAlgorithm:

https://www.rfc-editor.org/rfc/rfc5929#section-4.1

That is, the old version of the code was correct and the
"simplification" broke it. Revert this and restore the original version.

I suspect this went unnoticed because, almost all the time, both the old
and new code picked SHA-256 and it was fine. But if the certificate was,
say, signed with SHA-384, the new code would compute the wrong channel
binding.
This commit is contained in:
David Benjamin 2024-02-05 14:52:52 -05:00 committed by akallabeth
parent 2fffcd64b9
commit bee7f94e93
1 changed files with 4 additions and 6 deletions

View File

@ -720,15 +720,13 @@ WINPR_MD_TYPE x509_utils_get_signature_alg(const X509* xcert)
{
WINPR_ASSERT(xcert);
EVP_PKEY* evp = X509_get0_pubkey(xcert);
WINPR_ASSERT(evp);
const int nid = X509_get_signature_nid(xcert);
int nid = 0;
const int res = EVP_PKEY_get_default_digest_nid(evp, &nid);
if (res <= 0)
int hash_nid = 0;
if (OBJ_find_sigid_algs(nid, &hash_nid, NULL) != 1)
return WINPR_MD_NONE;
switch (nid)
switch (hash_nid)
{
case NID_md2:
return WINPR_MD_MD2;