Commit Graph

171 Commits

Author SHA1 Message Date
Vincent Sanders
1490b52a6b NetSurf options rework (a=vince r=daniels,jmb)
svn path=/trunk/netsurf/; revision=13548
2012-03-22 09:34:34 +00:00
John Mark Bell
07309888d6 return;
Fix bug #3442642: allow scheme-specific fetchers to have a say in whether an URL can be fetched.

svn path=/trunk/netsurf/; revision=13182
2011-11-27 14:14:36 +00:00
John Mark Bell
dcbafe6b87 Clean up fetch callback API. Inject some long-needed type safety.
svn path=/trunk/netsurf/; revision=13137
2011-11-08 21:51:42 +00:00
Vincent Sanders
5d06dbf974 prevent odd curl non completion behaviour
svn path=/trunk/netsurf/; revision=13024
2011-10-09 17:22:57 +00:00
John Mark Bell
d5bb9d98e7 Build resource: map at start time
Remove stylesheet url globals -- the core will always use resource:{default,quirks,adblock}.css as appropriate

svn path=/trunk/netsurf/; revision=12949
2011-10-05 08:32:52 +00:00
Michael Drake
73dbd82b7d Optimise fetch item selection when fetching many items from same host.
svn path=/trunk/netsurf/; revision=12920
2011-10-02 12:10:02 +00:00
Michael Drake
6cfd37e60f Convert fetchers to nsurl.
svn path=/trunk/netsurf/; revision=12910
2011-09-29 15:31:54 +00:00
Michael Drake
e3211bf4fc fetch_can_fetch can take const nsurl.
svn path=/trunk/netsurf/; revision=12902
2011-09-27 14:42:45 +00:00
Michael Drake
e1f7a37f15 Port fetch layer to nsurl. Remove unused fetch_get_referer function.
svn path=/trunk/netsurf/; revision=12899
2011-09-27 11:07:32 +00:00
Michael Drake
c94271edf5 Fetchers register with an lwc_string, rather than a string.
svn path=/trunk/netsurf/; revision=12891
2011-09-26 21:07:19 +00:00
Michael Drake
393b1afd4f Disable fetch debug loops when not debugging.
svn path=/trunk/netsurf/; revision=12890
2011-09-26 15:22:38 +00:00
Michael Drake
8435195ea4 Remember to free host, now it's not kept for anything.
svn path=/trunk/netsurf/; revision=12886
2011-09-26 13:43:34 +00:00
Michael Drake
460648872f Remove unused host from fetch info struct.
svn path=/trunk/netsurf/; revision=12885
2011-09-26 13:38:06 +00:00
François Revel
955896ac46 Revert r12412, the doi: URI scheme hasn't been kept around it seems, even ACM doesn't use it. And kyllikki was jumping on his seat anyway... It's still in the svn history if needed. I have some stuff for view-source: and man: but I'm not use they would be appreciated either :-D
svn path=/trunk/netsurf/; revision=12414
2011-05-16 19:28:40 +00:00
François Revel
d25e5568bb Add a redirecting fetcher for the doi: URI scheme. Not much used but just in case...
svn path=/trunk/netsurf/; revision=12412
2011-05-15 20:01:25 +00:00
Vincent Sanders
0231675abf add resource handling
move gtk and framebuffer to use generic resource handling

svn path=/trunk/netsurf/; revision=11772
2011-02-23 22:27:55 +00:00
John Mark Bell
daf23b0860 Add missing include
svn path=/trunk/netsurf/; revision=11694
2011-02-15 23:26:34 +00:00
Vincent Sanders
fa98e3d76a add about: fetcher
rename fetchers to be more sensible 

svn path=/trunk/netsurf/; revision=11692
2011-02-15 23:18:10 +00:00
John Mark Bell
3fcb02afa1 Revert r10650/1. We no longer require noisy fetch/cache logging.
svn path=/trunk/netsurf/; revision=11223
2011-01-06 00:31:54 +00:00
Daniel Silverstone
b85b1f5ba5 Make fetch.c use new RING_COUNTBYLWCHOST
svn path=/trunk/netsurf/; revision=11108
2010-12-20 10:32:15 +00:00
John Mark Bell
2077918805 Merge branches/vince/netsurf-file-fetcher to trunk
r=jmb

svn path=/trunk/netsurf/; revision=10750
2010-09-09 21:45:59 +00:00
John Mark Bell
183974b437 Enable noisy logging
svn path=/trunk/netsurf/; revision=10650
2010-07-17 08:17:42 +00:00
John Mark Bell
b0f1b7b170 Improve const-correctness of fetch_start
svn path=/trunk/netsurf/; revision=10342
2010-04-10 17:13:53 +00:00
John Mark Bell
24c853281d Return cookies to the same level of working/brokenness as they had before new-cache.
svn path=/trunk/netsurf/; revision=10340
2010-04-10 12:24:25 +00:00
Daniel Silverstone
6c9e4bea8e Purge FETCH_TYPE which is unused throughout NetSurf. Clearing a todo in llcachec.
svn path=/trunk/netsurf/; revision=10300
2010-04-08 12:16:52 +00:00
Daniel Silverstone
2b1146bcb3 Remove parent from fetches, clearing a todo from llcache.c
svn path=/trunk/netsurf/; revision=10299
2010-04-08 11:47:05 +00:00
Daniel Silverstone
270ef59a98 Merge jmb/new-cache; r=dsilvers,rs=vince
svn path=/trunk/netsurf/; revision=10180
2010-03-28 12:56:39 +00:00
Vincent Sanders
6990c2cd04 remove uneccissary includes
svn path=/trunk/netsurf/; revision=9931
2010-01-29 11:32:53 +00:00
John Mark Bell
355799ce0b Merge branches/MarkieB/gtkmain to trunk.
svn path=/trunk/netsurf/; revision=9729
2009-12-17 23:55:02 +00:00
Chris Young
0cd96bdd5e Correct definition
svn path=/trunk/netsurf/; revision=8899
2009-07-29 22:00:07 +00:00
Chris Young
d2c29dbe95 Keep a copy of the next_fetcher pointer to avoid a situation where the fetch gets
freed during the poll routine and the next pointer can no longer be read.

svn path=/trunk/netsurf/; revision=8898
2009-07-29 18:51:35 +00:00
John Mark Bell
5feb7018c5 Replace parent_url with a pointer to the parent content.
svn path=/trunk/netsurf/; revision=8428
2009-07-10 00:26:37 +00:00
John Mark Bell
d58baf8ea4 Permit referer to propagate to https hosts from http
svn path=/trunk/netsurf/; revision=7875
2009-06-19 11:15:06 +00:00
John Mark Bell
059b6a0e15 Make logic surrounding extraction of URL + referer scheme more sensible.
Always require the fetch URL to have a scheme -- without it, we can't find a fetcher.

svn path=/trunk/netsurf/; revision=7597
2009-05-28 11:56:56 +00:00
John Mark Bell
243c1ba03f Limit scope of manually-defined NDEBUG. Purge a bunch of redundant #undef NDEBUG.
svn path=/trunk/netsurf/; revision=7593
2009-05-28 10:39:53 +00:00
John Mark Bell
847b776285 Purge NETSURF_USE_SSL and WITH_SSL
svn path=/trunk/netsurf/; revision=6599
2009-02-22 15:07:50 +00:00
John Mark Bell
79b4cee9d3 Make redirects behave as a NOP wrt the verifiability of fetches.
Apparently, the interweb depends on this.

svn path=/trunk/netsurf/; revision=6542
2009-02-16 20:24:54 +00:00
François Revel
f28f4cc760 First round of C89 fixes. Those should be obvious, hope I didn't introduce any side effect.
svn path=/trunk/netsurf/; revision=4788
2008-07-29 09:05:36 +00:00
James Bursa
21a3c99d3d Remove unused members of struct fetch (were moved to fetch_curl but not removed from here).
svn path=/trunk/netsurf/; revision=4232
2008-05-31 16:57:53 +00:00
John Mark Bell
a76164ab51 Wrap very verbose logging with #ifdef, so as to silence it.
svn path=/trunk/netsurf/; revision=4225
2008-05-29 19:32:31 +00:00
John Mark Bell
74a1095cd6 Remove unnecessary includes
svn path=/trunk/netsurf/; revision=4144
2008-05-11 10:09:44 +00:00
Rob Kendrick
8e79665248 Initial attempt at implementing data: URLs. Needs cleaning and verifying that I'm using the fetcher API correctly. Appears to work, though.
svn path=/trunk/netsurf/; revision=3882
2008-03-05 14:21:29 +00:00
John Mark Bell
7f70acaca6 Silence noisy logging
svn path=/trunk/netsurf/; revision=3830
2008-02-03 18:21:49 +00:00
John Mark Bell
cc3481a2ff Make core fetching code handle verifiability of transactions
Make core fetching code be responsible for inserting cookies into the urldb
Provide accessor to a fetch's parent url (this is defined as being the URL of the verifiable fetch which caused this one to occur)
Make fetchcache's 3xx handling use the parent url when spawning new fetches

svn path=/trunk/netsurf/; revision=3809
2008-01-30 19:56:41 +00:00
Vincent Sanders
2f864eeecc Update all source code file headers to reflect GPL version 2 only and contain appropriate licence text
svn path=/trunk/netsurf/; revision=3486
2007-08-08 16:16:03 +00:00
John Mark Bell
81f3fcbafd Fix crash when building error page -- fetch will be NULL, so fetch_get_referer will fail.
svn path=/trunk/netsurf/; revision=3476
2007-08-06 22:17:04 +00:00
John Mark Bell
17ab65e69a Fix uninitialised fetch structure entries.
This was causing the Referer header to be sent regardless (assuming the referer string was non-NULL).

svn path=/trunk/netsurf/; revision=3387
2007-07-08 11:35:53 +00:00
James Bursa
594eb74c4c Fix deadlock when fetching stylesheets when max_fetchers_per_host is reached by splitting fetch_can_be_freed() into fetch_remove_from_queues() and fetch_free().
svn path=/trunk/netsurf/; revision=3380
2007-07-04 18:44:13 +00:00
James Bursa
960c11de1a Rename register_curl_fetchers() to fetch_curl_register(). Add declarations for static functions and reorder functions.
svn path=/trunk/netsurf/; revision=3379
2007-07-04 18:05:16 +00:00
Daniel Silverstone
e671fe3707 Add debug about ring sizes. Ensure we dispatch queued jobs when a job is freed.
svn path=/trunk/netsurf/; revision=3333
2007-06-10 18:12:30 +00:00