Because cURL can do connection caching behind the scenes, we
need to have a cache for the SSL certificate chains which we
send onward on first header back from cURL.
This uses the new hashmap implementation to mean that we cache
chains on a hostname:port basis.
Signed-off-by: Daniel Silverstone <dsilvers@digital-scurf.org>
Instead of extracting information from the X509 certificate chain in
the fetcher the entire chain is propagated in Distinguished Encoding
Rules (DER) format.
This allows all the information contained in a certificate chain to be
retained which can subsequently be presented to the user
* Changed ETag storage to be time_t, rather than int.
* Changed `If-None-Match` value parsing to use proper
time_t parsing, rather than `atoi`.
We emit FETCH_NOTMODIFIED if the resource hasn't changed.
Any errors from the fetch which are not already handled are
reported with an internal query page instead of a modal
dialog.
This is much less invasive for the user and much more in
keeping with how this is handled by other browsers.
The handler is similar to the timeout handler but the
functionality is kept separate as it is intended timeout
handling be extended in future.
cURL will prevent channel reuse if NTLM auth is enabled because
NTLM authenticates a channel not a request. As such we were
unable to reuse curl handles since we handed off connection
reuse to curl instead of our own handle cache. This mitigates
the effect, though curl authors are looking at fixing it upstream
too.
Fixes: #2707
Signed-off-by: Daniel Silverstone <dsilvers@digital-scurf.org>
The curl fetcher can operate without openssl library being
available, additionaly curl itself may be compiled with a
different TLS library.
In either case this will simply cause the "unknown" error to be
reported for all TLS failiures and page information to lack any
certificate information.
Since OpenSSL 1.0.2 there has been hostname verification support
which cURL doesn't turn on for some reason. Turn it on so that
we get better hostname verification handling.
Signed-off-by: Daniel Silverstone <dsilvers@digital-scurf.org>
In doing this, also propagate why the certificates were bad
so that the page can display a reason. We will need FatMessages
for all these.
Signed-off-by: Daniel Silverstone <dsilvers@digital-scurf.org>
* Fetchers now provide the certificates before headers
* This is propagated all the way to the browser window
* When a query occurs, we retrieve it from there and fire
the query with those stored certificates.
* The serial number is a bignum, store it as hex.
Signed-off-by: Daniel Silverstone <dsilvers@digital-scurf.org>
Because we need to be slightly more careful now, lodge ourselves
`inside_curl` whenever we might be processing via the cURL messages
or similar.
Signed-off-by: Daniel Silverstone <dsilvers@digital-scurf.org>
In order to cope better with modern cURL which prevents making
cURL calls when inside a callback from within cURL, defer fetch
start when we are processing in `fetch_curl_data()`.
Signed-off-by: Daniel Silverstone <dsilvers@digital-scurf.org>
stop curl fetcher logging being special case to standard error and
use the fetch catagory at DEBUG level instead.
The special suppress_curl_debug option is currently still obeyed
On ubuntu 19.4, curl is built with HTTP2 support, and we
segfault.
==18174== Invalid read of size 1
==18174== at 0x4ACCE7D: ??? (in /usr/lib/x86_64-linux-gnu/libcurl.so.4.5.0)
==18174== by 0x4B054B1: ??? (in /usr/lib/x86_64-linux-gnu/libcurl.so.4.5.0)
==18174== by 0x4AD398A: ??? (in /usr/lib/x86_64-linux-gnu/libcurl.so.4.5.0)
==18174== by 0x4AD7A0B: ??? (in /usr/lib/x86_64-linux-gnu/libcurl.so.4.5.0)
==18174== by 0x4AE93EE: ??? (in /usr/lib/x86_64-linux-gnu/libcurl.so.4.5.0)
==18174== by 0x4AEA8A8: curl_multi_perform (in /usr/lib/x86_64-linux-gnu/libcurl.so.4.5.0)
==18174== by 0x1F2EF7: fetch_curl_poll (curl.c:1209)
==18174== by 0x1EEC5C: fetcher_poll (fetch.c:271)
==18174== by 0x2A1ED4: schedule_run (schedule.c:160)
==18174== by 0x15F941: framebuffer_run (gui.c:596)
==18174== by 0x15F941: main (gui.c:2206)
==18174== Address 0x9de95a8 is 3,224 bytes inside a block of size 6,304 free'd
==18174== at 0x483997B: free (in /usr/lib/x86_64-linux-gnu/valgrind/vgpreload_memcheck-amd64-linux.so)
==18174== by 0x4AD497B: ??? (in /usr/lib/x86_64-linux-gnu/libcurl.so.4.5.0)
==18174== by 0x4AE158C: curl_easy_cleanup (in /usr/lib/x86_64-linux-gnu/libcurl.so.4.5.0)
==18174== by 0x1F30DE: fetch_curl_cache_handle (curl.c:761)
==18174== by 0x1F30DE: fetch_curl_stop (curl.c:840)
==18174== by 0x1F30DE: fetch_curl_done (curl.c:1122)
==18174== by 0x1F30DE: fetch_curl_poll (curl.c:1223)
==18174== by 0x1EEC5C: fetcher_poll (fetch.c:271)
==18174== by 0x2A1ED4: schedule_run (schedule.c:160)
==18174== by 0x15F941: framebuffer_run (gui.c:596)
==18174== by 0x15F941: main (gui.c:2206)
==18174== Block was alloc'd at
==18174== at 0x483AB35: calloc (in /usr/lib/x86_64-linux-gnu/valgrind/vgpreload_memcheck-amd64-linux.so)
==18174== by 0x4AE165F: curl_easy_duphandle (in /usr/lib/x86_64-linux-gnu/libcurl.so.4.5.0)
==18174== by 0x1F15EB: fetch_curl_get_handle (curl.c:738)
==18174== by 0x1F15EB: fetch_curl_start (curl.c:750)
==18174== by 0x1EEB22: fetch_dispatch_job (fetch.c:156)
==18174== by 0x1EEB22: fetch_choose_and_dispatch (fetch.c:187)
==18174== by 0x1EEB22: fetch_dispatch_jobs (fetch.c:247)
==18174== by 0x1EF1BB: fetch_start (fetch.c:573)
==18174== by 0x26C779: llcache_object_refetch (llcache.c:916)
==18174== by 0x26D5E4: llcache_object_fetch (llcache.c:979)
==18174== by 0x26D5E4: llcache_object_retrieve_from_cache (llcache.c:1767)
==18174== by 0x26D5E4: llcache_object_retrieve (llcache.c:1865)
==18174== by 0x26E42C: llcache_fetch_redirect (llcache.c:2110)
==18174== by 0x26E42C: llcache_fetch_callback (llcache.c:2810)
==18174== by 0x1F1295: fetch_curl_process_headers (curl.c:922)
==18174== by 0x1F13A0: fetch_curl_data (curl.c:1324)
==18174== by 0x4ACD4C3: ??? (in /usr/lib/x86_64-linux-gnu/libcurl.so.4.5.0)
==18174== by 0x4AE00DA: ??? (in /usr/lib/x86_64-linux-gnu/libcurl.so.4.5.0)
