This partially reverts commit 67f3fff2. The legacy provider is now loaded
from the `winpr_InitializeSSL` function, so it is not needed to load it from
the `winpr_RC4_New` function again.
* winpr/crypto: Exit cleanly when EVP_EncryptInit_ex fails
The `EVP_EncryptInit_ex` function may fail in certain configurations.
Consequently, FreeRDP segfaults in `EVP_CIPHER_CTX_set_key_length`.
Let's handle the `EVP_EncryptInit_ex` failures and exit cleanly in
such case.
* winpr/crypto: Load legacy provider to fix rc4 with OpenSSL 3.0
Currently, the `EVP_EncryptInit_ex` function fails for rc4 with OpenSSL 3.0.
This is becuase rc4 is provided by the legacy provider which is not loaded
by default. Let's explicitly load the legacy provider to make FreeRDP work
with OpenSSL 3.0.
Relates: https://github.com/openssl/openssl/issues/14392
Fixes: https://github.com/FreeRDP/FreeRDP/issues/6604
This option will ensure that NLA is disabled(since NTLM uses weak crypto algorithms), FIPS
encryption is enabled, and ensure fips mode is enabled for openssl.
Selectively override specific uses of MD5/RC4 with new API calls specifically tailored to override FIPS.
Add comments on why overriding the use of these algorithms under FIPS is acceptable for the locations where overrides happen.
Remove check of server proprietary certificate which was already being ignore to avoid use of MD5.
Initialize winpr openssl earlier to ensure fips mode is set before starting using any crypto algorithms.
SSL functions like OpenSSL_add_all_digests should be invoked at very beginning as they are not MT safe.
If not we might meet double free exception as following:
#0 0x00007f23ddd71c37 in raise () from /lib/x86_64-linux-gnu/libc.so.6
#1 0x00007f23ddd75028 in abort () from /lib/x86_64-linux-gnu/libc.so.6
#2 0x00007f23dddae2a4 in ?? () from /lib/x86_64-linux-gnu/libc.so.6
#3 0x00007f23dddba55e in ?? () from /lib/x86_64-linux-gnu/libc.so.6
#4 0x00007f23dc6ecfcd in CRYPTO_free () from /lib/x86_64-linux-gnu/libcrypto.so.1.0.0
#5 0x00007f23dc6ef8d1 in OBJ_NAME_add () from /lib/x86_64-linux-gnu/libcrypto.so.1.0.0
#6 0x00007f23dc77dcd8 in EVP_add_digest () from /lib/x86_64-linux-gnu/libcrypto.so.1.0.0
#7 0x00007f23dc782321 in OpenSSL_add_all_digests () from /lib/x86_64-linux-gnu/libcrypto.so.1.0.0
#8 0x00007f23c781da28 in winpr_openssl_get_evp_md (md=4) at /home/zihao/workspace/zihao_FreeRDP/winpr/libwinpr/crypto/hash.c:52
#9 0x00007f23c781dccb in winpr_Digest_Init (ctx=0x7f22d064d470, md=<optimized out>) at /home/zihao/workspace/zihao_FreeRDP/winpr/libwinpr/crypto/hash.c:344
#10 0x00007f23d486139b in security_salted_mac_signature (rdp=0x7f23859f5a20, data=0x7f238542d4fb "\004\204\022\004", length=4743, encryption=<optimized out>, output=0x7
at /home/zihao/workspace/zihao_FreeRDP/libfreerdp/core/security.c:378
#11 0x00007f23d488d73f in fastpath_send_update_pdu (fastpath=<optimized out>, updateCode=4 '\004', s=0x7f23859f5f40, skipCompression=true)
at /home/zihao/workspace/zihao_FreeRDP/libfreerdp/core/fastpath.c:1076
#12 0x00007f23d4891c4f in update_send_surface_frame_bits (context=0x7f23859f5540, cmd=0x7f22b2ffcc80, first=true, last=true, frameId=6)
at /home/zihao/workspace/zihao_FreeRDP/libfreerdp/core/update.c:1041
Related reports: https://rt.openssl.org/Ticket/Display.html?id=2216&user=guest&pass=guest
- fixed invalid, missing or additional arguments
- removed all type casts from arguments
- added missing (void*) typecasts for %p arguments
- use inttypes defines where appropriate
- winpr_HMAC_New() now just returnes the opaque WINPR_HMAC_CTX* pointer
which has to be passed to winpr_HMAC_Init() for (re)initialization
and since winpr_HMAC_Final() no more frees the context you always have to
use the new function winpr_HMAC_Free() once winpr_HMAC_New() has succeded
- winpr_Digest_New() now just returns the opaque WINPR_DIGEST_CTX* pointer
which has to be passed to winpr_Digest_Init() for (re)initialization
and since winpr_Digest_Final() no more frees the context you always have to
use the new function winpr_Digest_Free() once winpr_Digest_New() has succeded
