The `winpr_enable_fips` function is called twice from the `winpr_InitializeSSL`
function after commit 49936a7b. I believe this is a bug, which might lead to
some undesired behavior. Let's call that only once.
With OpenSSL 3.O, FreeRDP log contains errors like:
```
4036740A4C7F0000:error:0308010C:digital envelope routines:
inner_evp_generic_fetch:unsupported:crypto/evp/evp_fetch.c:346:
Global default library context, Algorithm (MD4 : 85), Properties ()
```
This leads to connection failures in some cases. This is because algorithms
like MD4 are now part of the legacy provider, which is not loaded by
default. Let's explicitly load that provider. With this change, also the
other provides has to be explicitely loaded.
FreeRDP fails to build with OpenSSL 3.0 because of usage of the `FIPS_mode`
and `FIPS_mode_set` functions, which were removed there. Just a note that
the FIPS mode is not supported by OpenSSL 1.1.* although the mentioned
functions are still there (see https://wiki.openssl.org/index.php/FIPS_modules).
Let's make FreeRDP build with OpenSSL 3.0 and fix the FIPS mode support.
See: https://bugzilla.redhat.com/show_bug.cgi?id=1952937
The assistance file requires primitives from the ssl wrapper.
Enable these before parsing the file.
Additionally split the FIPS mode enablement from the one time
initializer to avoid ignoring that flag.
FreeRDP aborts if OpenSSL operates in FIPS mode and +fipsmode is not
manually specified. Let's prevent the abortion and enable the necessary
options in that case automatically.
This option will ensure that NLA is disabled(since NTLM uses weak crypto algorithms), FIPS
encryption is enabled, and ensure fips mode is enabled for openssl.
Selectively override specific uses of MD5/RC4 with new API calls specifically tailored to override FIPS.
Add comments on why overriding the use of these algorithms under FIPS is acceptable for the locations where overrides happen.
Remove check of server proprietary certificate which was already being ignore to avoid use of MD5.
Initialize winpr openssl earlier to ensure fips mode is set before starting using any crypto algorithms.
SSL functions like OpenSSL_add_all_digests should be invoked at very beginning as they are not MT safe.
If not we might meet double free exception as following:
#0 0x00007f23ddd71c37 in raise () from /lib/x86_64-linux-gnu/libc.so.6
#1 0x00007f23ddd75028 in abort () from /lib/x86_64-linux-gnu/libc.so.6
#2 0x00007f23dddae2a4 in ?? () from /lib/x86_64-linux-gnu/libc.so.6
#3 0x00007f23dddba55e in ?? () from /lib/x86_64-linux-gnu/libc.so.6
#4 0x00007f23dc6ecfcd in CRYPTO_free () from /lib/x86_64-linux-gnu/libcrypto.so.1.0.0
#5 0x00007f23dc6ef8d1 in OBJ_NAME_add () from /lib/x86_64-linux-gnu/libcrypto.so.1.0.0
#6 0x00007f23dc77dcd8 in EVP_add_digest () from /lib/x86_64-linux-gnu/libcrypto.so.1.0.0
#7 0x00007f23dc782321 in OpenSSL_add_all_digests () from /lib/x86_64-linux-gnu/libcrypto.so.1.0.0
#8 0x00007f23c781da28 in winpr_openssl_get_evp_md (md=4) at /home/zihao/workspace/zihao_FreeRDP/winpr/libwinpr/crypto/hash.c:52
#9 0x00007f23c781dccb in winpr_Digest_Init (ctx=0x7f22d064d470, md=<optimized out>) at /home/zihao/workspace/zihao_FreeRDP/winpr/libwinpr/crypto/hash.c:344
#10 0x00007f23d486139b in security_salted_mac_signature (rdp=0x7f23859f5a20, data=0x7f238542d4fb "\004\204\022\004", length=4743, encryption=<optimized out>, output=0x7
at /home/zihao/workspace/zihao_FreeRDP/libfreerdp/core/security.c:378
#11 0x00007f23d488d73f in fastpath_send_update_pdu (fastpath=<optimized out>, updateCode=4 '\004', s=0x7f23859f5f40, skipCompression=true)
at /home/zihao/workspace/zihao_FreeRDP/libfreerdp/core/fastpath.c:1076
#12 0x00007f23d4891c4f in update_send_surface_frame_bits (context=0x7f23859f5540, cmd=0x7f22b2ffcc80, first=true, last=true, frameId=6)
at /home/zihao/workspace/zihao_FreeRDP/libfreerdp/core/update.c:1041
Related reports: https://rt.openssl.org/Ticket/Display.html?id=2216&user=guest&pass=guest
freerdp/winpr had the following issues:
* The non reentrant SSL_library_init() was called concurrently (crash)
* Missing code/api to set the eventually required OpenSSL static and dynamic locking callbacks
* Missing code/api to free the application-global or thread-local OpenSSL data and tables
This commit creates two new winpr functions:
BOOL winpr_InitializeSSL(DWORD flags):
Use the flag WINPR_SSL_INIT_ALREADY_INITIALIZED if you want to tell winpr that
your application has already initialized OpenSSL.
If required use the flag WINPR_SSL_INIT_ENABLE_LOCKING to tell winpr that it
should set the OpenSSL static and dynamic locking callbacks.
Otherwise just call it with the flag WINPR_SSL_INIT_DEFAULT.
The recommended way is that your application calls this function once before
any threads are created. However, in order to support lazy OpenSSL library
initialization winpr_InitializeSSL() can also safely be called multiple times
and concurrently because it uses the new InitOnceExecuteOnce() function to
guarantee that the initialization is only performed successfully once during
the life time of the calling process.
BOOL winpr_CleanupSSL(DWORD flags):
If you create a thread that uses SSL you should call this function before the
thread returns using the flag WINPR_SSL_CLEANUP_THREAD in order to clean up
the thread-local OpenSSL data and tables.
Call the function with the flag WINPR_SSL_CLEANUP_GLOBAL before terminating
your application.
Note: This commit only replaced the current occurences of the
SSL_load_error_strings(); SSL_library_init(); pairs in the freerdp source
with winpr_InitializeSSL(). None of the server or client applications has been
changed according to the recommended usage described above (TBDL).
