Commit Graph

218 Commits

Author SHA1 Message Date
Vincent Sanders
5a2f69388c hoist the Referer header generation logic up to the low level cache
This removes the need for the fetchers to have any interaction with
 the Referer header. It has not been completely removed from the
 fetch interface as fetch.c:fetch_set_cookie() still uses it for
 unverifiable cookie decision logic. (There is an anchient todo here)
2021-03-25 23:36:41 +00:00
Vincent Sanders
c74509cdf5 move about fetcher into its own directory 2020-09-20 10:54:48 +01:00
Vincent Sanders
0908925ca6 move the file fetcher sources into a single directory 2020-07-03 18:07:21 +01:00
Michael Drake
c5aaf45d8e Fetch: Squash use of uninitialised value in fetch_send_callback()
The `last_msg` member of `struct fetch` was not initialised on
creation of the structure.
2020-03-12 17:22:28 +00:00
Daniel Silverstone
4c256408c5
fetch: Bump last_msg rather than always assigning it
If we are aborting a cURL fetch then it's possible entry to
cURL to clean up could cause a PROGRESS message to happen which
we need to not reset last_msg for, otherwise we spuriously attempt
to send an ERROR message during abort which is not a safe time for
our message callback to be used.

Signed-off-by: Daniel Silverstone <dsilvers@digital-scurf.org>
2020-02-24 17:43:21 +00:00
Daniel Silverstone
97870122dd
fetch: When aborting, ensure we note this to not multi-fail
When aborting a fetch, it may be cleaned up, in which case we
would otherwise send a spurious failure message too early. Instead
we record that the abort has occurred to suppress such an error.

Signed-off-by: Daniel Silverstone <dsilvers@digital-scurf.org>
2020-02-24 10:24:23 +00:00
Daniel Silverstone
083ba385d7
fetch: Automatically handle fetches which fail to finish
Signed-off-by: Daniel Silverstone <dsilvers@digital-scurf.org>
2020-02-24 09:47:52 +00:00
Daniel Silverstone
8469f4cc8e Reimplement handling of BAD_AUTH inside browser_window
We now handle authentication requests via an `about:` page
which presents a nice form built into the browser window.
In order to do this, we add internal navigation as a concept
to the browser window and we strip the 401login support from all
frontends except monkey.

The 401login callback is now intended for password safe type support
rather than an immediately interactive prompt.

Signed-off-by: Daniel Silverstone <dsilvers@digital-scurf.org>
2019-08-06 11:26:47 +01:00
Vincent Sanders
71225624f6 update fetch debug logging to use catagory 2017-09-06 18:45:35 +01:00
Vincent Sanders
75018632a9 Use coccinelle to change logging macro calls in c files
for F in $(git ls-files '*.c');do spatch --sp-file foo.cocci --in-place ${F};done

@@ expression E; @@
-LOG(E);
+NSLOG(netsurf, INFO, E);
@@ expression E, E1; @@
-LOG(E, E1);
+NSLOG(netsurf, INFO, E, E1);
@@ expression E, E1, E2; @@
-LOG(E, E1, E2);
+NSLOG(netsurf, INFO, E, E1, E2);
@@ expression E, E1, E2, E3; @@
-LOG(E, E1, E2, E3);
+NSLOG(netsurf, INFO, E, E1, E2, E3);
@@ expression E, E1, E2, E3, E4; @@
-LOG(E, E1, E2, E3, E4);
+NSLOG(netsurf, INFO, E, E1, E2, E3, E4);
@@ expression E, E1, E2, E3, E4, E5; @@
-LOG(E, E1, E2, E3, E4, E5);
+NSLOG(netsurf, INFO, E, E1, E2, E3, E4, E5);
@@ expression E, E1, E2, E3, E4, E5, E6; @@
-LOG(E, E1, E2, E3, E4, E5, E6);
+NSLOG(netsurf, INFO, E, E1, E2, E3, E4, E5, E6);
@@ expression E, E1, E2, E3, E4, E5, E6, E7; @@
-LOG(E, E1, E2, E3, E4, E5, E6, E7);
+NSLOG(netsurf, INFO, E, E1, E2, E3, E4, E5, E6, E7);
2017-09-06 18:45:27 +01:00
Vincent Sanders
0f69965805 clean up some of the doxygen generation warnings 2017-02-19 11:32:17 +00:00
Vincent Sanders
1f52b2d514 remove unused fetch acessor 2017-01-31 22:56:45 +00:00
Vincent Sanders
70d700b26c rationalise use of utils/utils.h header 2017-01-19 21:15:08 +00:00
Michael Drake
fe00eac8cb Buildsystem: Make curl/openssl usage build-time options. 2016-08-13 13:31:07 +01:00
Daniel Silverstone
47ccd9855d Move fetcher_fdset to fetch.h (and rename to fetch_fdset). Maybe not ideal but better 2016-06-27 21:09:39 +01:00
Daniel Silverstone
a468b40990 Refactor the fdset acquisition into the fetchers to stop fetch.c including curl.h 2016-06-27 21:00:58 +01:00
Vincent Sanders
fe7eb85614 move misc header into public API 2016-05-30 11:23:32 +01:00
Vincent Sanders
7d9c9dba36 Split utils header into string functions and everything else
split out the string handling API from the rest of the utils header
and fix up all the fallout.
2016-04-21 22:36:21 +01:00
Vincent Sanders
33c7df0c40 complete the rename of the gui browser table
When the operations tables were created the browser table was renamed
to miscellaneous except the actual rename patch was never applied,
this fixes that situation.
2016-04-16 23:50:21 +01:00
Vincent Sanders
5d3c38a704 remove reliance on GLib from monkey frontend
This changes the monkey implemntation to use a simple scheduler list
and select instead of glib events.
2016-02-12 13:37:50 +00:00
Vincent Sanders
73e45ff024 Fix error reporting from fetch_start
Any fetch start error was being reported as "out of memory" which was
clearly insufficient. Foe example bad urls (reported was file:// with
a missing /) were causing a warn_user with out of memory. This change
now at least causes a "bad url" message.
2015-06-24 10:31:13 +01:00
Vincent Sanders
0d811963c1 fixup all the remaining logging macro callsites
The semantic patch tool appears to have missed some difficult to
reason about callsites which have now been manually cleaned up
2015-05-28 17:06:18 +01:00
Vincent Sanders
c105738fa3 Change LOG() macro to be varadic
This changes the LOG macro to be varadic removing the need for all
callsites to have double bracketing and allows for future improvement
on how we use the logging macros.

The callsites were changed with coccinelle and the changes checked by
hand. Compile tested for several frontends but not all.

A formatting annotation has also been added which allows the compiler
to check the parameters and types passed to the logging.
2015-05-28 16:08:46 +01:00
Vincent Sanders
d1f2d29e10 Doxygen cleanups and documentation additions 2014-11-10 17:00:15 +00:00
Vincent Sanders
53f0f432dd update fetcher registration API to return an error code.
By using an error code return we can gracefully handle fetcher
registration faliures instead of just immediately aborting.

The curl handler was also cleaned up and documentation improved
as a side effect.
2014-10-25 17:15:23 +01:00
Vincent Sanders
d06df231bc Update the core to use the split operations table headers
Second in the series to decouple the core API usage from the frontends.
2014-10-16 22:33:43 +01:00
Vincent Sanders
02bbaa3e88 split version info into its own header and remove unnecessary nesurf.h includes
The netsurf.h header should *only* contain the registration, core
initialisation and finalisation methods. Version information is best
placed in its own header.

Also remove any unneeded inclusion of this header limiting it to
solely the places the relevant API is required.
2014-10-13 15:08:16 +01:00
Vincent Sanders
b99ba0998b fixup haiku rsrc fetcher initialisation 2014-09-05 00:40:52 +01:00
Vincent Sanders
37755fb135 add incomplete scheme fetcher for javascript urls 2014-08-25 15:59:52 -07:00
Vincent Sanders
0694f345da fix leak on error path (coverity 1224429) 2014-07-08 15:57:48 +01:00
Vincent Sanders
a71865b328 turn off verbose fetch debugging 2014-07-02 17:07:29 +01:00
Vincent Sanders
29e29cccd7 fix dumb error in previous commit and get scheme name element correct 2014-07-02 15:01:24 +01:00
Vincent Sanders
516dcb96e1 fix unhelpful assert at quit 2014-07-02 14:51:11 +01:00
Vincent Sanders
8944edd649 convert all frontends to scheduled fetch operation 2014-06-26 19:05:59 +01:00
Vincent Sanders
1b7aa7ffe5 make fetchers scheduled 2014-06-26 19:05:59 +01:00
Vincent Sanders
4b2101ba6a clean up the fetcher factory and improve its API 2014-06-19 18:27:24 +01:00
Michael Drake
c4e2fff5a4 Use corestrings for fetch module's lwc strings. 2014-01-24 19:25:07 +00:00
Vincent Sanders
4987a3a8a7 remove forward refs from content/fetch.c and cleanup doc comments 2014-01-19 21:18:47 +00:00
Daniel Silverstone
33f94c265e Support cloning rawfile data 2014-01-04 21:49:03 +00:00
Daniel Silverstone
de42f8880e Add a bunch of logging to try and help track down the issue with rawfile handling, sorry guys 2014-01-04 21:20:17 +00:00
Daniel Silverstone
581d877576 In theory, store raw filenames and pass them through for file upload. Untested due to no file-upload in GTK frontend just yet 2014-01-04 19:34:18 +00:00
Michael Drake
21d63c3542 Pedantic wrap fix. 2013-10-23 15:15:08 +01:00
Vincent Sanders
ee6e6eec05 move options include 2013-05-28 14:16:10 +01:00
John-Mark Bell
3dff750ae2 Downgrade TLS version support if it turns out the server can't cope with TLSv1.1. 2013-01-04 23:13:23 +00:00
Michael Drake
2e869ade2a Cleanup. 2012-10-11 14:38:10 +01:00
Michael Drake
5d7ad9db46 Port to new urldb. 2012-10-08 20:32:57 +01:00
Vincent Sanders
78620910e8 check all return values of string comparisons 2012-08-09 17:30:02 +01:00
Vincent Sanders
1490b52a6b NetSurf options rework (a=vince r=daniels,jmb)
svn path=/trunk/netsurf/; revision=13548
2012-03-22 09:34:34 +00:00
John Mark Bell
07309888d6 return;
Fix bug #3442642: allow scheme-specific fetchers to have a say in whether an URL can be fetched.

svn path=/trunk/netsurf/; revision=13182
2011-11-27 14:14:36 +00:00
John Mark Bell
dcbafe6b87 Clean up fetch callback API. Inject some long-needed type safety.
svn path=/trunk/netsurf/; revision=13137
2011-11-08 21:51:42 +00:00