mirrors/netsurf
1
0
Fork 0
mirror of https://github.com/netsurf-browser/netsurf synced 2024-12-01 18:37:10 +03:00
Code Issues Packages Projects Releases Wiki Activity
13,599 Commits 6 Branches 12 Tags 33 MiB
43f8e77df7
Commit Graph

246 Commits

Author SHA1 Message Date
Daniel Silverstone
3d80e825e1 content/fetchers/curl.c: Defer fetch start if inside cURL 
In order to cope better with modern cURL which prevents making
cURL calls when inside a callback from within cURL, defer fetch
start when we are processing in `fetch_curl_data()`.

Signed-off-by: Daniel Silverstone <dsilvers@digital-scurf.org>
 2019-06-05 19:51:47 +01:00
Vincent Sanders
13a5b8f74a make curl fetcher log debug using NSLOG 
stop curl fetcher logging being special case to standard error and
 use the fetch catagory at DEBUG level instead.

The special suppress_curl_debug option is currently still obeyed
 2019-05-28 11:16:49 +01:00
Michael Drake
65b16f4c54 Curl fetcher: Force HTTP1.1 to fix HTTP2 crash. 
On ubuntu 19.4, curl is built with HTTP2 support, and we
segfault.

==18174== Invalid read of size 1
==18174==    at 0x4ACCE7D: ??? (in /usr/lib/x86_64-linux-gnu/libcurl.so.4.5.0)
==18174==    by 0x4B054B1: ??? (in /usr/lib/x86_64-linux-gnu/libcurl.so.4.5.0)
==18174==    by 0x4AD398A: ??? (in /usr/lib/x86_64-linux-gnu/libcurl.so.4.5.0)
==18174==    by 0x4AD7A0B: ??? (in /usr/lib/x86_64-linux-gnu/libcurl.so.4.5.0)
==18174==    by 0x4AE93EE: ??? (in /usr/lib/x86_64-linux-gnu/libcurl.so.4.5.0)
==18174==    by 0x4AEA8A8: curl_multi_perform (in /usr/lib/x86_64-linux-gnu/libcurl.so.4.5.0)
==18174==    by 0x1F2EF7: fetch_curl_poll (curl.c:1209)
==18174==    by 0x1EEC5C: fetcher_poll (fetch.c:271)
==18174==    by 0x2A1ED4: schedule_run (schedule.c:160)
==18174==    by 0x15F941: framebuffer_run (gui.c:596)
==18174==    by 0x15F941: main (gui.c:2206)
==18174==  Address 0x9de95a8 is 3,224 bytes inside a block of size 6,304 free'd
==18174==    at 0x483997B: free (in /usr/lib/x86_64-linux-gnu/valgrind/vgpreload_memcheck-amd64-linux.so)
==18174==    by 0x4AD497B: ??? (in /usr/lib/x86_64-linux-gnu/libcurl.so.4.5.0)
==18174==    by 0x4AE158C: curl_easy_cleanup (in /usr/lib/x86_64-linux-gnu/libcurl.so.4.5.0)
==18174==    by 0x1F30DE: fetch_curl_cache_handle (curl.c:761)
==18174==    by 0x1F30DE: fetch_curl_stop (curl.c:840)
==18174==    by 0x1F30DE: fetch_curl_done (curl.c:1122)
==18174==    by 0x1F30DE: fetch_curl_poll (curl.c:1223)
==18174==    by 0x1EEC5C: fetcher_poll (fetch.c:271)
==18174==    by 0x2A1ED4: schedule_run (schedule.c:160)
==18174==    by 0x15F941: framebuffer_run (gui.c:596)
==18174==    by 0x15F941: main (gui.c:2206)
==18174==  Block was alloc'd at
==18174==    at 0x483AB35: calloc (in /usr/lib/x86_64-linux-gnu/valgrind/vgpreload_memcheck-amd64-linux.so)
==18174==    by 0x4AE165F: curl_easy_duphandle (in /usr/lib/x86_64-linux-gnu/libcurl.so.4.5.0)
==18174==    by 0x1F15EB: fetch_curl_get_handle (curl.c:738)
==18174==    by 0x1F15EB: fetch_curl_start (curl.c:750)
==18174==    by 0x1EEB22: fetch_dispatch_job (fetch.c:156)
==18174==    by 0x1EEB22: fetch_choose_and_dispatch (fetch.c:187)
==18174==    by 0x1EEB22: fetch_dispatch_jobs (fetch.c:247)
==18174==    by 0x1EF1BB: fetch_start (fetch.c:573)
==18174==    by 0x26C779: llcache_object_refetch (llcache.c:916)
==18174==    by 0x26D5E4: llcache_object_fetch (llcache.c:979)
==18174==    by 0x26D5E4: llcache_object_retrieve_from_cache (llcache.c:1767)
==18174==    by 0x26D5E4: llcache_object_retrieve (llcache.c:1865)
==18174==    by 0x26E42C: llcache_fetch_redirect (llcache.c:2110)
==18174==    by 0x26E42C: llcache_fetch_callback (llcache.c:2810)
==18174==    by 0x1F1295: fetch_curl_process_headers (curl.c:922)
==18174==    by 0x1F13A0: fetch_curl_data (curl.c:1324)
==18174==    by 0x4ACD4C3: ??? (in /usr/lib/x86_64-linux-gnu/libcurl.so.4.5.0)
==18174==    by 0x4AE00DA: ??? (in /usr/lib/x86_64-linux-gnu/libcurl.so.4.5.0)
 2019-05-06 17:12:14 +01:00
Michael Drake
5e1f4c406d data url handler: drop logging to DEEPDEBUG. 2019-05-05 20:39:26 +01:00
Michael Drake
fb1d9862f9 data url handler: Keep nsurl to avoid copy. 2019-05-05 20:36:19 +01:00
Michael Drake
47e47244a9 data url handling: avoid needless allocation / copy. 2019-05-05 20:23:25 +01:00
Michael Drake
c76b5ef4d5 data URL handler: constify some string pointers. 2019-05-05 20:11:21 +01:00
Michael Drake
97f9e2d9f5 Curl fetcher: Jiggle logging levels. 2018-08-09 16:43:37 +01:00
Michael Drake
ca0514ea92 Curl fetcher: Move poll logging from INFO to DEEPDEBUG. 2018-08-09 14:58:49 +01:00
John-Mark Bell
a8bf9b05aa HTTPS: restrict ciphersuites 2018-04-22 12:54:23 +00:00
John-Mark Bell
7d4349035d Explicitly re-enable TLS session tickets. 
Libcurl disables this by default.
 2018-04-22 12:24:56 +00:00
Daniel Silverstone
70b6ca4376 Force SSL session caching on, Ubuntu seems to have it off by default 2018-04-22 13:04:21 +01:00
Daniel Silverstone
5efc6407d3 Revert "Disable SSL session ID caching." 
This reverts commit debe0e345b.
 2018-04-22 12:58:39 +01:00
Vincent Sanders
75018632a9 Use coccinelle to change logging macro calls in c files 
for F in $(git ls-files '*.c');do spatch --sp-file foo.cocci --in-place ${F};done

@@ expression E; @@
-LOG(E);
+NSLOG(netsurf, INFO, E);
@@ expression E, E1; @@
-LOG(E, E1);
+NSLOG(netsurf, INFO, E, E1);
@@ expression E, E1, E2; @@
-LOG(E, E1, E2);
+NSLOG(netsurf, INFO, E, E1, E2);
@@ expression E, E1, E2, E3; @@
-LOG(E, E1, E2, E3);
+NSLOG(netsurf, INFO, E, E1, E2, E3);
@@ expression E, E1, E2, E3, E4; @@
-LOG(E, E1, E2, E3, E4);
+NSLOG(netsurf, INFO, E, E1, E2, E3, E4);
@@ expression E, E1, E2, E3, E4, E5; @@
-LOG(E, E1, E2, E3, E4, E5);
+NSLOG(netsurf, INFO, E, E1, E2, E3, E4, E5);
@@ expression E, E1, E2, E3, E4, E5, E6; @@
-LOG(E, E1, E2, E3, E4, E5, E6);
+NSLOG(netsurf, INFO, E, E1, E2, E3, E4, E5, E6);
@@ expression E, E1, E2, E3, E4, E5, E6, E7; @@
-LOG(E, E1, E2, E3, E4, E5, E6, E7);
+NSLOG(netsurf, INFO, E, E1, E2, E3, E4, E5, E6, E7);
 2017-09-06 18:45:27 +01:00
Vincent Sanders
6075feb487 create netsurf inttypes header to have portable integer formatting macros 2017-01-21 14:20:55 +00:00
Vincent Sanders
3f88c12315 Fix OpenSSL X509 cert API detection 
LibreSSL defines a OPENSSL_VERSION_NUMBER of 2.1 but only provides the
old OpenSSL 1.0.x API.
 2016-11-21 18:29:23 +00:00
Vincent Sanders
e8a9e37445 fix openSSL 1.1.0 X509 certificate handling 2016-11-20 12:15:31 +00:00
Vincent Sanders
61a00c3832 fix unecessary base64 header include 2016-11-19 17:11:23 +00:00
Daniel Silverstone
1fb06929d9 Remove internal BASE64 in favour of nsutils' one 2016-11-19 14:40:10 +00:00
Vincent Sanders
fa1af79e7c document file fetcher being locale dependant 2016-09-10 23:05:24 +01:00
Michael Drake
fe00eac8cb Buildsystem: Make curl/openssl usage build-time options. 2016-08-13 13:31:07 +01:00
Michael Drake
fa2e3b7784 URL unescape: return the new length to the caller. 
The avoids situations were we threw away the length, only for
the caller to have to strlen the returned string.

Note, there seems to be a case of the amiga front end writing
beyond end of allocation.  Added a TODO for now.
 2016-07-24 14:03:16 +01:00
Michael Drake
7202ff2f64 Data URL handling: Use url_unescape rather than curl. 2016-07-24 12:23:42 +01:00
Michael Drake
f9870c41f5 Whitespace: Convert spaces to tab for indent. 2016-07-24 12:07:31 +01:00
Daniel Silverstone
a468b40990 Refactor the fdset acquisition into the fetchers to stop fetch.c including curl.h 2016-06-27 21:00:58 +01:00
Vincent Sanders
ab6c03f311 Fix handling of certificate chains 
When processing a x509 certificate chain from openssl it is necessary
to allow teh entire chain to be processed rather than halting
processing at the first certificate with an error. This allows errors
with a certificate authority to be examined.
 2016-06-27 13:39:07 +01:00
Vincent Sanders
8861923455 Allow include directories to be added by sub makefiles 2016-06-06 14:47:27 +01:00
Vincent Sanders
76a68d7dd0 Update content to split public and internal API 2016-06-06 09:01:38 +01:00
Vincent Sanders
527756cca0 move fetch header into public API 2016-05-30 11:29:39 +01:00
Vincent Sanders
fe7eb85614 move misc header into public API 2016-05-30 11:23:32 +01:00
Vincent Sanders
3224d7121a move image content handlers to accomodate core build changes 2016-05-23 23:32:16 +01:00
Vincent Sanders
6a36d4ec2b remove all core usage of warn_user API 2016-04-26 12:50:16 +01:00
Vincent Sanders
2bb68cb874 need strings.h for strncasecmp on freebsd 2016-04-21 23:04:00 +01:00
Vincent Sanders
7d9c9dba36 Split utils header into string functions and everything else 
split out the string handling API from the rest of the utils header
and fix up all the fallout.
 2016-04-21 22:36:21 +01:00
Vincent Sanders
798654f910 remove curl fetchers use of the wallclock API 
The wallclock() API uses gettimeofday which can be affected by the the
systems clock being changed etc. The curl fetcher usage of this API is
to generate a timing delta and does not cope with the gettimeofday
issues.

This changes the fetcher to use the nsutils library monotonic time
function which does not suffer from the issues with gettimeofday.
 2016-04-19 14:18:09 +01:00
Vincent Sanders
afea659fef move dir_sort_alpha function from generic utils to one specific place it is used 2016-04-19 12:01:05 +01:00
Vincent Sanders
1bc010665e Reduce unecessary use of directory entry headers 
The config header was causing many source files to unecessarily
include the dirent headers causing extra dependancies. This has been
fixed by providing a utility dirent header that provides a common API
across all platforms while removing the unecessary dirent header usage.
 2016-04-19 11:21:06 +01:00
Vincent Sanders
ade0cd8bb9 Reduce unecessary use of networking headers 
The utility configuration header dragged in a number of bsd sockets
and related API as a side effect of setting up the configuration. By
splitting the header and API setup into a separate header only the
small number of places that need the functionality explitly include
it.
 2016-04-17 13:42:28 +01:00
Vincent Sanders
d15ab96a51 Fix size_t printf formatting 
The printf formatting for size_t is set in c99 as %zu but in windows
it is %Iu this is solved by adding and inttypes style PRI macro for size_t

This also uses this macro everywhere size_t is formatted.
 2016-03-17 22:00:54 +00:00
Michael Drake
4055ecf796 Tiny simplification of curl debug logging. 2016-01-21 09:27:27 +00:00
Michael Drake
401cc36924 Add extra logging to curl poll fn when not suppressing curl debug. 2015-11-11 10:39:12 +00:00
Daniel Silverstone
4d1ef3bac4 Add support for retrying timed-out cURL fetches. 
This is an attempt to amelioriate the situation found in #2384 where
we see the cURL connect() failing to complete.  Based on the pcap
from the bug log, we believe that RISC OS is likely failing to signal
the completion of the connection to cURL.  As such, cURL times out.

This change permits retries of timed out connections in the hope that
a fresh socket FD might subsequently function correctly.  The defaults
chosen mean that the previous behaviour of 30 seconds before timeout
is reported will remain the same, but in that time we will make 3 separate
attempts to connect the socket.
 2015-11-10 21:51:54 +00:00
Vincent Sanders
e9b89f776d Allow the resource scheme to provide data directly. 
This allows front ends to provide resources from compiled in data
instead of requiring the resources to be available on disc and forcing
a redirect.
 2015-06-17 21:35:40 +01:00
Vincent Sanders
c105738fa3 Change LOG() macro to be varadic 
This changes the LOG macro to be varadic removing the need for all
callsites to have double bracketing and allows for future improvement
on how we use the logging macros.

The callsites were changed with coccinelle and the changes checked by
hand. Compile tested for several frontends but not all.

A formatting annotation has also been added which allows the compiler
to check the parameters and types passed to the logging.
 2015-05-28 16:08:46 +01:00
Vincent Sanders
a487f7e611 Move win32 window operations into their own module 
This splits up a great deal of the win32 window code out from other
gui code. It also remove large quantities of unused and junk
variables and functions.
 2015-03-12 23:48:53 +00:00
Vincent Sanders
669df172ec Fix my spelling of teh everywhere 2014-11-18 17:16:26 +00:00
Vincent Sanders
d4c7a0ff77 Improve data scheme fetcher allocation 
The data scheme fetcher was over allocating the space for decoded
base64 encoded urls and not using the base64 API that allocated the
correct size storage.
 2014-11-17 16:05:17 +00:00
Vincent Sanders
71d016aba5 Remove unecessary form debug 
Additionaly we make the curl fetcher report errors instead of
uninitialised buffer.
 2014-11-15 18:23:44 +00:00
Vincent Sanders
56bb1b03c5 Doxygen cleanups 2014-11-09 12:50:30 +00:00
Vincent Sanders
8f64d5d223 Improve Doxygen documentation 2014-11-08 12:35:11 +00:00