FreeRDP aborts if OpenSSL operates in FIPS mode and +fipsmode is not
manually specified. Let's prevent the abortion and enable the necessary
options in that case automatically.
This option will ensure that NLA is disabled(since NTLM uses weak crypto algorithms), FIPS
encryption is enabled, and ensure fips mode is enabled for openssl.
Selectively override specific uses of MD5/RC4 with new API calls specifically tailored to override FIPS.
Add comments on why overriding the use of these algorithms under FIPS is acceptable for the locations where overrides happen.
Remove check of server proprietary certificate which was already being ignore to avoid use of MD5.
Initialize winpr openssl earlier to ensure fips mode is set before starting using any crypto algorithms.
WLog_PrintMessagePrefixVA is called with format being a stack variable.
Always copy the data to message->PrefixString otherwise the information
will be lost whenever the stack is destroyed.
SSL functions like OpenSSL_add_all_digests should be invoked at very beginning as they are not MT safe.
If not we might meet double free exception as following:
#0 0x00007f23ddd71c37 in raise () from /lib/x86_64-linux-gnu/libc.so.6
#1 0x00007f23ddd75028 in abort () from /lib/x86_64-linux-gnu/libc.so.6
#2 0x00007f23dddae2a4 in ?? () from /lib/x86_64-linux-gnu/libc.so.6
#3 0x00007f23dddba55e in ?? () from /lib/x86_64-linux-gnu/libc.so.6
#4 0x00007f23dc6ecfcd in CRYPTO_free () from /lib/x86_64-linux-gnu/libcrypto.so.1.0.0
#5 0x00007f23dc6ef8d1 in OBJ_NAME_add () from /lib/x86_64-linux-gnu/libcrypto.so.1.0.0
#6 0x00007f23dc77dcd8 in EVP_add_digest () from /lib/x86_64-linux-gnu/libcrypto.so.1.0.0
#7 0x00007f23dc782321 in OpenSSL_add_all_digests () from /lib/x86_64-linux-gnu/libcrypto.so.1.0.0
#8 0x00007f23c781da28 in winpr_openssl_get_evp_md (md=4) at /home/zihao/workspace/zihao_FreeRDP/winpr/libwinpr/crypto/hash.c:52
#9 0x00007f23c781dccb in winpr_Digest_Init (ctx=0x7f22d064d470, md=<optimized out>) at /home/zihao/workspace/zihao_FreeRDP/winpr/libwinpr/crypto/hash.c:344
#10 0x00007f23d486139b in security_salted_mac_signature (rdp=0x7f23859f5a20, data=0x7f238542d4fb "\004\204\022\004", length=4743, encryption=<optimized out>, output=0x7
at /home/zihao/workspace/zihao_FreeRDP/libfreerdp/core/security.c:378
#11 0x00007f23d488d73f in fastpath_send_update_pdu (fastpath=<optimized out>, updateCode=4 '\004', s=0x7f23859f5f40, skipCompression=true)
at /home/zihao/workspace/zihao_FreeRDP/libfreerdp/core/fastpath.c:1076
#12 0x00007f23d4891c4f in update_send_surface_frame_bits (context=0x7f23859f5540, cmd=0x7f22b2ffcc80, first=true, last=true, frameId=6)
at /home/zihao/workspace/zihao_FreeRDP/libfreerdp/core/update.c:1041
Related reports: https://rt.openssl.org/Ticket/Display.html?id=2216&user=guest&pass=guest
- fixed invalid, missing or additional arguments
- removed all type casts from arguments
- added missing (void*) typecasts for %p arguments
- use inttypes defines where appropriate
