Commit Graph

210 Commits

Author SHA1 Message Date
Michael Drake
8435195ea4 Remember to free host, now it's not kept for anything.
svn path=/trunk/netsurf/; revision=12886
2011-09-26 13:43:34 +00:00
Michael Drake
460648872f Remove unused host from fetch info struct.
svn path=/trunk/netsurf/; revision=12885
2011-09-26 13:38:06 +00:00
François Revel
955896ac46 Revert r12412, the doi: URI scheme hasn't been kept around it seems, even ACM doesn't use it. And kyllikki was jumping on his seat anyway... It's still in the svn history if needed. I have some stuff for view-source: and man: but I'm not use they would be appreciated either :-D
svn path=/trunk/netsurf/; revision=12414
2011-05-16 19:28:40 +00:00
François Revel
d25e5568bb Add a redirecting fetcher for the doi: URI scheme. Not much used but just in case...
svn path=/trunk/netsurf/; revision=12412
2011-05-15 20:01:25 +00:00
Vincent Sanders
0231675abf add resource handling
move gtk and framebuffer to use generic resource handling

svn path=/trunk/netsurf/; revision=11772
2011-02-23 22:27:55 +00:00
John Mark Bell
daf23b0860 Add missing include
svn path=/trunk/netsurf/; revision=11694
2011-02-15 23:26:34 +00:00
Vincent Sanders
fa98e3d76a add about: fetcher
rename fetchers to be more sensible 

svn path=/trunk/netsurf/; revision=11692
2011-02-15 23:18:10 +00:00
John Mark Bell
3fcb02afa1 Revert r10650/1. We no longer require noisy fetch/cache logging.
svn path=/trunk/netsurf/; revision=11223
2011-01-06 00:31:54 +00:00
Daniel Silverstone
b85b1f5ba5 Make fetch.c use new RING_COUNTBYLWCHOST
svn path=/trunk/netsurf/; revision=11108
2010-12-20 10:32:15 +00:00
John Mark Bell
2077918805 Merge branches/vince/netsurf-file-fetcher to trunk
r=jmb

svn path=/trunk/netsurf/; revision=10750
2010-09-09 21:45:59 +00:00
John Mark Bell
183974b437 Enable noisy logging
svn path=/trunk/netsurf/; revision=10650
2010-07-17 08:17:42 +00:00
John Mark Bell
b0f1b7b170 Improve const-correctness of fetch_start
svn path=/trunk/netsurf/; revision=10342
2010-04-10 17:13:53 +00:00
John Mark Bell
24c853281d Return cookies to the same level of working/brokenness as they had before new-cache.
svn path=/trunk/netsurf/; revision=10340
2010-04-10 12:24:25 +00:00
Daniel Silverstone
6c9e4bea8e Purge FETCH_TYPE which is unused throughout NetSurf. Clearing a todo in llcachec.
svn path=/trunk/netsurf/; revision=10300
2010-04-08 12:16:52 +00:00
Daniel Silverstone
2b1146bcb3 Remove parent from fetches, clearing a todo from llcache.c
svn path=/trunk/netsurf/; revision=10299
2010-04-08 11:47:05 +00:00
Daniel Silverstone
270ef59a98 Merge jmb/new-cache; r=dsilvers,rs=vince
svn path=/trunk/netsurf/; revision=10180
2010-03-28 12:56:39 +00:00
Vincent Sanders
6990c2cd04 remove uneccissary includes
svn path=/trunk/netsurf/; revision=9931
2010-01-29 11:32:53 +00:00
John Mark Bell
355799ce0b Merge branches/MarkieB/gtkmain to trunk.
svn path=/trunk/netsurf/; revision=9729
2009-12-17 23:55:02 +00:00
Chris Young
0cd96bdd5e Correct definition
svn path=/trunk/netsurf/; revision=8899
2009-07-29 22:00:07 +00:00
Chris Young
d2c29dbe95 Keep a copy of the next_fetcher pointer to avoid a situation where the fetch gets
freed during the poll routine and the next pointer can no longer be read.

svn path=/trunk/netsurf/; revision=8898
2009-07-29 18:51:35 +00:00
John Mark Bell
5feb7018c5 Replace parent_url with a pointer to the parent content.
svn path=/trunk/netsurf/; revision=8428
2009-07-10 00:26:37 +00:00
John Mark Bell
d58baf8ea4 Permit referer to propagate to https hosts from http
svn path=/trunk/netsurf/; revision=7875
2009-06-19 11:15:06 +00:00
John Mark Bell
059b6a0e15 Make logic surrounding extraction of URL + referer scheme more sensible.
Always require the fetch URL to have a scheme -- without it, we can't find a fetcher.

svn path=/trunk/netsurf/; revision=7597
2009-05-28 11:56:56 +00:00
John Mark Bell
243c1ba03f Limit scope of manually-defined NDEBUG. Purge a bunch of redundant #undef NDEBUG.
svn path=/trunk/netsurf/; revision=7593
2009-05-28 10:39:53 +00:00
John Mark Bell
847b776285 Purge NETSURF_USE_SSL and WITH_SSL
svn path=/trunk/netsurf/; revision=6599
2009-02-22 15:07:50 +00:00
John Mark Bell
79b4cee9d3 Make redirects behave as a NOP wrt the verifiability of fetches.
Apparently, the interweb depends on this.

svn path=/trunk/netsurf/; revision=6542
2009-02-16 20:24:54 +00:00
François Revel
f28f4cc760 First round of C89 fixes. Those should be obvious, hope I didn't introduce any side effect.
svn path=/trunk/netsurf/; revision=4788
2008-07-29 09:05:36 +00:00
James Bursa
21a3c99d3d Remove unused members of struct fetch (were moved to fetch_curl but not removed from here).
svn path=/trunk/netsurf/; revision=4232
2008-05-31 16:57:53 +00:00
John Mark Bell
a76164ab51 Wrap very verbose logging with #ifdef, so as to silence it.
svn path=/trunk/netsurf/; revision=4225
2008-05-29 19:32:31 +00:00
John Mark Bell
74a1095cd6 Remove unnecessary includes
svn path=/trunk/netsurf/; revision=4144
2008-05-11 10:09:44 +00:00
Rob Kendrick
8e79665248 Initial attempt at implementing data: URLs. Needs cleaning and verifying that I'm using the fetcher API correctly. Appears to work, though.
svn path=/trunk/netsurf/; revision=3882
2008-03-05 14:21:29 +00:00
John Mark Bell
7f70acaca6 Silence noisy logging
svn path=/trunk/netsurf/; revision=3830
2008-02-03 18:21:49 +00:00
John Mark Bell
cc3481a2ff Make core fetching code handle verifiability of transactions
Make core fetching code be responsible for inserting cookies into the urldb
Provide accessor to a fetch's parent url (this is defined as being the URL of the verifiable fetch which caused this one to occur)
Make fetchcache's 3xx handling use the parent url when spawning new fetches

svn path=/trunk/netsurf/; revision=3809
2008-01-30 19:56:41 +00:00
Vincent Sanders
2f864eeecc Update all source code file headers to reflect GPL version 2 only and contain appropriate licence text
svn path=/trunk/netsurf/; revision=3486
2007-08-08 16:16:03 +00:00
John Mark Bell
81f3fcbafd Fix crash when building error page -- fetch will be NULL, so fetch_get_referer will fail.
svn path=/trunk/netsurf/; revision=3476
2007-08-06 22:17:04 +00:00
John Mark Bell
17ab65e69a Fix uninitialised fetch structure entries.
This was causing the Referer header to be sent regardless (assuming the referer string was non-NULL).

svn path=/trunk/netsurf/; revision=3387
2007-07-08 11:35:53 +00:00
James Bursa
594eb74c4c Fix deadlock when fetching stylesheets when max_fetchers_per_host is reached by splitting fetch_can_be_freed() into fetch_remove_from_queues() and fetch_free().
svn path=/trunk/netsurf/; revision=3380
2007-07-04 18:44:13 +00:00
James Bursa
960c11de1a Rename register_curl_fetchers() to fetch_curl_register(). Add declarations for static functions and reorder functions.
svn path=/trunk/netsurf/; revision=3379
2007-07-04 18:05:16 +00:00
Daniel Silverstone
e671fe3707 Add debug about ring sizes. Ensure we dispatch queued jobs when a job is freed.
svn path=/trunk/netsurf/; revision=3333
2007-06-10 18:12:30 +00:00
John Mark Bell
87ca206e13 Warning fixes.
svn path=/trunk/netsurf/; revision=3332
2007-06-10 18:08:22 +00:00
Daniel Silverstone
e6d754cc96 Remove a time consuming LOG(()) and add some whitespace to help me think
svn path=/trunk/netsurf/; revision=3331
2007-06-10 17:55:07 +00:00
Daniel Silverstone
1dd7e97eb3 Merge scheme switcher branch in.
svn path=/trunk/netsurf/; revision=3330
2007-06-10 17:46:44 +00:00
Daniel Silverstone
6807b4208a Remove the netsurf/ from the include paths and rationalise use of <> vs "" in includes
NetSurf includes are now done with ""s and other system includes with <>s as C intended.
The scandeps tool has been updated to only look for ""ed includes, and to verify that the
files exist in the tree before adding them to the dependency lines. The depend rule has
therefore been augmented to make sure the autogenerated files are built before it is run.

This is untested under self-hosted RISC OS builds. All else tested and works.


svn path=/trunk/netsurf/; revision=3307
2007-05-30 22:39:54 +00:00
Daniel Silverstone
25e22eb1f5 Add a -v option to enable debug logging. Pass -v by default to the RO runimage.
svn path=/trunk/netsurf/; revision=3306
2007-05-30 22:27:58 +00:00
James Bursa
5d2a2c42b2 Log curl_version() in fetch_init().
svn path=/trunk/netsurf/; revision=3267
2007-05-06 05:55:13 +00:00
James Bursa
21407fe8ab Fix some warnings in recent gcc.
svn path=/trunk/netsurf/; revision=3261
2007-05-02 02:47:39 +00:00
James Bursa
0547422534 Move user-agent generation into fetch.c and simplify.
svn path=/trunk/netsurf/; revision=3260
2007-05-02 02:38:38 +00:00
John Mark Bell
91cfb11592 Fix crash on WWW-Authenticate header with no realm (1686714)
svn path=/trunk/netsurf/; revision=3216
2007-03-23 22:39:10 +00:00
John Mark Bell
6d0795b923 Handle redirects with partial bodies.
svn path=/trunk/netsurf/; revision=3209
2007-03-18 13:02:07 +00:00
John Mark Bell
cf17b38065 Ensure handle options are reset appropriately (fixes 1668354)
svn path=/trunk/netsurf/; revision=3189
2007-02-25 16:52:53 +00:00
Daniel Silverstone
657e384a84 Fix RING_FINDBYHOST to actually find things. jmb found the bug, I verified his patch and committed it
svn path=/trunk/netsurf/; revision=3186
2007-02-22 11:48:39 +00:00
John Mark Bell
25213dca24 Fix handling of cookies in unverifiable transactions caused by a redirect from a fetch into a browser window which was varifiable.
svn path=/trunk/netsurf/; revision=3165
2007-02-02 23:08:13 +00:00
John Mark Bell
8cf2e40daa Bring handling of submission of blank file inputs in line with other browsers.
Tidy up fetch_post_convert while I'm at it.

svn path=/trunk/netsurf/; revision=3159
2007-01-30 23:19:21 +00:00
Rob Kendrick
5426a708a9 Generates and use a User-Agent: string based on new netsurf_version_major/minor values, and results of uname().
svn path=/trunk/netsurf/; revision=3158
2007-01-30 19:51:54 +00:00
James Bursa
a0b6661eb6 Make GTK build compile on FreeBSD.
svn path=/trunk/netsurf/; revision=3154
2007-01-29 22:27:15 +00:00
John Mark Bell
df4dbaf4cf Handle cookies in unverifiable transactions
svn path=/trunk/netsurf/; revision=3151
2007-01-27 20:58:20 +00:00
John Mark Bell
f629a4f00f Revert hack around broken pipe (not caching fetch handles used for SSL
connections).
Ignore SIGPIPE completely as nothing other than OpenSSL is expected to 
generate it.

svn path=/trunk/netsurf/; revision=3145
2007-01-18 01:26:58 +00:00
James Bursa
e76140557d Add fetch_http_code() to retrieve the HTTP status code of a fetch.
svn path=/trunk/netsurf/; revision=3139
2007-01-13 00:19:02 +00:00
Daniel Silverstone
cf0718f649 Update the file comment for fetch.c as requested.
svn path=/trunk/netsurf/; revision=3133
2007-01-02 13:07:28 +00:00
John Mark Bell
98b451ffae Add option to set Accept-Charset header (Stefaan Claes)
svn path=/trunk/netsurf/; revision=3124
2006-12-28 00:42:14 +00:00
John Mark Bell
abcce3dfe7 Fix typo.
Default type -> text/plain

svn path=/trunk/netsurf/; revision=3123
2006-12-28 00:36:37 +00:00
Michael Drake
ba23e4b693 Update project URL.
svn path=/trunk/netsurf/; revision=3073
2006-11-27 15:35:18 +00:00
Chris Williams
f4e4ec00b7 Ignore body data from 401 replies to prevent disruption of auth login handling
svn path=/trunk/netsurf/; revision=2972
2006-10-01 13:17:52 +00:00
John Mark Bell
5bbd5f259e Don't cache curl handles that were used to fetch https URLs; with any
luck, this should fix broken pipe crashes.

svn path=/trunk/netsurf/; revision=2723
2006-07-09 01:25:06 +00:00
John Mark Bell
8c3e9b882b Fix crash when url_* fails
svn path=/trunk/netsurf/; revision=2678
2006-06-29 22:04:56 +00:00
John Mark Bell
83ef76adc7 Only accept cookies from cookie-enabled transactions
svn path=/trunk/netsurf/; revision=2668
2006-06-29 00:01:24 +00:00
John Mark Bell
4f249f9d0a Merge cookies changes into head - unvalidated transactions and a UI
still need implementing.

svn path=/trunk/netsurf/; revision=2632
2006-06-19 21:49:25 +00:00
James Bursa
312bece1d0 Make static function names consistent.
svn path=/trunk/netsurf/; revision=2537
2006-04-16 10:30:51 +00:00
John Mark Bell
ef301b8619 Fix URL file loading and add support for file:/// URLs to urldb
Convert file:/... to file:///... (the former isn't a valid URL)

svn path=/trunk/netsurf/; revision=2524
2006-04-12 08:09:27 +00:00
John Mark Bell
c09eb457df Unify information databases
svn path=/trunk/netsurf/; revision=2519
2006-04-09 23:21:13 +00:00
Daniel Silverstone
7ddb6b6f19 [project @ 2006-03-14 15:01:57 by dsilvers]
Urgh, unconditional variables made my brain hurt

svn path=/import/netsurf/; revision=2129
2006-03-14 15:01:57 +00:00
Daniel Silverstone
40260f177a [project @ 2006-03-14 14:21:00 by dsilvers]
Multi-parallel fetch. Needs documenting, works in basic GTK tests, makes things way faster on GTK port. Needs testing on RO and in general more testing is good.

svn path=/import/netsurf/; revision=2127
2006-03-14 14:21:01 +00:00
Michael Drake
b48ed604f5 [project @ 2006-03-04 10:48:40 by tlsa]
Tweak timeouts.

svn path=/import/netsurf/; revision=2103
2006-03-04 10:48:40 +00:00
John Mark Bell
d4d3e5ee1c [project @ 2006-02-23 15:06:53 by jmb]
Handle invalid SSL certificates better - UI still needs work.
Modify fetch callback data parameter type to remove compiler warnings.
Constify things.
Lose global ssl_verify_certificates option.
Fix issue when closing a dialog without input focus.

svn path=/import/netsurf/; revision=2092
2006-02-23 15:06:54 +00:00
John Mark Bell
5ce5fe084c [project @ 2006-02-19 18:26:23 by jmb]
Rewrite HTTP authentication.
Fix extraction of realm from WWW-Authenticate header.
Tidy up login dialog code.

svn path=/import/netsurf/; revision=2085
2006-02-19 18:26:23 +00:00
James Bursa
07d55db910 [project @ 2006-02-13 23:04:32 by bursa]
Disable sending of Expect: 100-continue for POSTs (fixes using the bug tracker). Remove unnecessary setting of Host header.

svn path=/import/netsurf/; revision=2080
2006-02-13 23:04:32 +00:00
John Mark Bell
e724672302 [project @ 2006-02-08 00:35:05 by jmb]
Handle case where no cache expiry headers are sent; use (now - last_modified) / 10. This should reduce the frequency of cache entry validation.

svn path=/import/netsurf/; revision=2064
2006-02-08 00:35:05 +00:00
John Mark Bell
6be0b8e60c [project @ 2006-02-07 00:44:52 by jmb]
Squash leaks and fake ETag header for local objects.

svn path=/import/netsurf/; revision=2060
2006-02-07 00:44:52 +00:00
John Mark Bell
0f228ada91 [project @ 2006-02-06 00:10:09 by jmb]
Implement HTTP caching algorithm; this should avoid stale cache entries being used.

svn path=/import/netsurf/; revision=2059
2006-02-06 00:10:09 +00:00
John Mark Bell
745deb7a9d [project @ 2006-01-08 01:51:33 by jmb]
Make data file locations user-configurable (no UI for this as yet)
Reduce intrusiveness of ncos modifications
Fix GTK build
Remove Cookies file details from Messages (this data never belonged in there anyway)
Make gui_init more robust against memory exhaustion.

svn path=/import/netsurf/; revision=2014
2006-01-08 01:51:33 +00:00
John Mark Bell
07d4f5c57b [project @ 2006-01-07 02:33:36 by jmb]
Fix 1347726.

svn path=/import/netsurf/; revision=2011
2006-01-07 02:33:36 +00:00
John Mark Bell
05bb6641e1 [project @ 2006-01-03 17:01:17 by jmb]
Fix 1371902.
fetch_filetype now receives the unix path with the leading / (which it always should have done, tbh).

svn path=/import/netsurf/; revision=1976
2006-01-03 17:01:17 +00:00
James Bursa
ad13eb98af [project @ 2005-12-19 21:54:51 by bursa]
Fix busy polling in gtk build.

svn path=/import/netsurf/; revision=1900
2005-12-19 21:54:51 +00:00
James Bursa
666cdaf8dd [project @ 2005-12-11 21:54:30 by bursa]
Disable some log messages.

svn path=/import/netsurf/; revision=1896
2005-12-11 21:54:30 +00:00
Adrian Lees
dbcfa470c4 [project @ 2005-07-16 16:23:08 by adrianl]
Bounds checking in Content-Length handling

svn path=/import/netsurf/; revision=1799
2005-07-16 16:23:08 +00:00
James Bursa
c4d3ac1d8f [project @ 2005-04-09 10:00:00 by bursa]
Serialize fetches with no host (should fix multiple local files not loading in certain cases).

svn path=/import/netsurf/; revision=1611
2005-04-09 10:00:00 +00:00
Richard Wilson
422df90089 [project @ 2005-01-23 16:09:05 by rjw]
Further work on theme installing

svn path=/import/netsurf/; revision=1463
2005-01-23 16:09:05 +00:00
James Bursa
1fb2e5f2c3 [project @ 2005-01-22 22:42:48 by bursa]
Fix proxy authentication bug (libcurl does not copy string options)

svn path=/import/netsurf/; revision=1461
2005-01-22 22:42:48 +00:00
John Mark Bell
efce778a9e [project @ 2004-11-11 23:55:23 by jmb]
Fix for redirects causing crashes

svn path=/import/netsurf/; revision=1360
2004-11-11 23:55:23 +00:00
John Mark Bell
49ce807e3c [project @ 2004-10-01 21:31:55 by jmb]
A somewhat better implementation of referrers which no longer sends the referer if the URL schemes don't match.

Things to do:
1) Preservation of referer across redirects (see comment in browser.c:284)
2) GUI templates/code for configuration of referer sending (simple on/off toggle only)
3) Make referer sending when fetching objects/stylesheets for a page pay attention to option_send_referer?
4) Handle the case where the referer is in the form of http://moo:foo@mysite.com/ (ie the login details embedded in the referer - not good).

svn path=/import/netsurf/; revision=1297
2004-10-01 21:31:55 +00:00
James Bursa
fb5082605c [project @ 2004-09-03 22:33:05 by bursa]
Fix bugs relating to aborting fetches.

svn path=/import/netsurf/; revision=1264
2004-09-03 22:33:05 +00:00
John Tytgat
7264ae50e5 [project @ 2004-08-14 15:07:19 by joty]
- Rename len() to css_len2px().
- Less compiler warnings concerning float/int implicit casts.
- More stddef.h type usuage.

svn path=/import/netsurf/; revision=1232
2004-08-14 15:07:21 +00:00
John Tytgat
402c4ca66c [project @ 2004-08-14 14:30:10 by joty]
Removed a chunk of Norcroft compiler warnings.  Re-ident some pieces.

svn path=/import/netsurf/; revision=1231
2004-08-14 14:30:12 +00:00
John Mark Bell
c9bd6fa9fc [project @ 2004-08-09 16:11:58 by jmb]
Rework the interface of the URL handing module to allow for multiple error types.
Modify save_complete URL rewriting appropriately.

svn path=/import/netsurf/; revision=1206
2004-08-09 16:11:58 +00:00
James Bursa
534b464bec [project @ 2004-07-27 15:49:28 by bursa]
Implement proxy authentication. Bring templates in line with style guide (icon sizes and fill display fields), and remove dead icons. Clean up choices dialog code. Fix persistent dialog code. Make browser choices the default pane.

svn path=/import/netsurf/; revision=1153
2004-07-27 15:49:28 +00:00
James Bursa
48130c473b [project @ 2004-07-10 10:05:16 by bursa]
Fix crash related to aborting fetches.

svn path=/import/netsurf/; revision=1067
2004-07-10 10:05:16 +00:00
John Mark Bell
ba22b4e753 [project @ 2004-07-10 02:35:30 by jmb]
Use libcurl's progress callback functionality to display fetch status.
This will update the status line once a second, more frequently requires hacking libcurl.

svn path=/import/netsurf/; revision=1066
2004-07-10 02:35:31 +00:00
James Bursa
a1d73ecd11 [project @ 2004-06-27 23:24:11 by bursa]
Rewritten download window. Now downloads direct to file. New download window gui.

svn path=/import/netsurf/; revision=1020
2004-06-27 23:24:11 +00:00
James Bursa
fdaf72417a [project @ 2004-06-22 17:37:51 by bursa]
Fix aborting of fetches.

svn path=/import/netsurf/; revision=990
2004-06-22 17:37:51 +00:00
James Bursa
d938fe135b [project @ 2004-06-10 23:55:23 by bursa]
Split fetchcache() into fetchcache() and fetchcache_go() to solve problems of callbacks being called before fetchcache() returns. Fix <style> breakage. Error handling fixes & improvements.

svn path=/import/netsurf/; revision=957
2004-06-10 23:55:23 +00:00