Commit Graph

264 Commits

Author SHA1 Message Date
John Mark Bell
da2b1ce44d Hack around curl complaining about junk gzip streams
svn path=/trunk/netsurf/; revision=13129
2011-11-05 22:10:46 +00:00
Michael Drake
8f554bb0e5 Fix "about:".
svn path=/trunk/netsurf/; revision=13054
2011-10-15 09:47:37 +00:00
John Mark Bell
d3fce3ffc3 Fix handling of URLs without an authority
svn path=/trunk/netsurf/; revision=13053
2011-10-14 21:41:16 +00:00
Vincent Sanders
eb9c223f9d fix image cache summary information to properly use a format string
remove snprintf buffer overflow possibility
fix the about:imagecache fetcher to contain the output formatting and use summary information formatting

svn path=/trunk/netsurf/; revision=13039
2011-10-12 12:34:11 +00:00
Vincent Sanders
d628aa06d0 fix possible buffer overflow
svn path=/trunk/netsurf/; revision=13032
2011-10-11 07:19:20 +00:00
Daniel Silverstone
986720d9d7 If the about handler fails to intern its strings, return the error, cleaning up first
svn path=/trunk/netsurf/; revision=13031
2011-10-11 06:55:20 +00:00
Vincent Sanders
56e7b23415 add about:imagecache some of teh implementation needs cleaning up
svn path=/trunk/netsurf/; revision=13030
2011-10-10 22:41:31 +00:00
Vincent Sanders
e1e88e5f98 make about fetcher use lwc_strings for url lookups
svn path=/trunk/netsurf/; revision=13029
2011-10-10 19:46:46 +00:00
Vincent Sanders
5d06dbf974 prevent odd curl non completion behaviour
svn path=/trunk/netsurf/; revision=13024
2011-10-09 17:22:57 +00:00
Michael Drake
73afb1e728 Use malloc, memcpy instead of strdup since we have the length.
svn path=/trunk/netsurf/; revision=13006
2011-10-08 12:47:42 +00:00
Vincent Sanders
443dce6e80 make resource fetcher cope with missing resources gracefully
svn path=/trunk/netsurf/; revision=12994
2011-10-07 22:51:31 +00:00
Vincent Sanders
4c829ae481 use resource:favicon.ico instead of resource:favicon.png
svn path=/trunk/netsurf/; revision=12982
2011-10-07 21:03:45 +00:00
Vincent Sanders
8a293636fb fix favicon default selection for non http(s) urls
svn path=/trunk/netsurf/; revision=12981
2011-10-07 20:47:34 +00:00
John Mark Bell
d5bb9d98e7 Build resource: map at start time
Remove stylesheet url globals -- the core will always use resource:{default,quirks,adblock}.css as appropriate

svn path=/trunk/netsurf/; revision=12949
2011-10-05 08:32:52 +00:00
Michael Drake
36eff6da2b Port more internals to nsurl. Front ends may need updating.
svn path=/trunk/netsurf/; revision=12926
2011-10-03 15:56:47 +00:00
Michael Drake
3a319b4829 Store host as lwc_string in curl fetcher and obtain it via nsurl. Clean up cached hosts on exit.
svn path=/trunk/netsurf/; revision=12918
2011-10-01 16:53:45 +00:00
Michael Drake
4b2730e407 Keep url as nsurl.
svn path=/trunk/netsurf/; revision=12917
2011-10-01 16:02:49 +00:00
Michael Drake
af6c81b042 Convert nsurl to string for log.
svn path=/trunk/netsurf/; revision=12916
2011-10-01 15:40:16 +00:00
Michael Drake
6cfd37e60f Convert fetchers to nsurl.
svn path=/trunk/netsurf/; revision=12910
2011-09-29 15:31:54 +00:00
Michael Drake
c94271edf5 Fetchers register with an lwc_string, rather than a string.
svn path=/trunk/netsurf/; revision=12891
2011-09-26 21:07:19 +00:00
John Mark Bell
c25d7890e7 Cope with zero-length files
svn path=/trunk/netsurf/; revision=12777
2011-09-08 08:25:48 +00:00
John Mark Bell
a5dc6b9e66 Less broken WWW-Authenticate header parser
svn path=/trunk/netsurf/; revision=12542
2011-06-29 23:32:53 +00:00
John Mark Bell
381588bece Log CA options
svn path=/trunk/netsurf/; revision=12436
2011-05-19 22:35:41 +00:00
John Mark Bell
54f6b3bffc Handle case where configured curl handle cache limit is 0
svn path=/trunk/netsurf/; revision=12420
2011-05-16 22:05:47 +00:00
François Revel
955896ac46 Revert r12412, the doi: URI scheme hasn't been kept around it seems, even ACM doesn't use it. And kyllikki was jumping on his seat anyway... It's still in the svn history if needed. I have some stuff for view-source: and man: but I'm not use they would be appreciated either :-D
svn path=/trunk/netsurf/; revision=12414
2011-05-16 19:28:40 +00:00
François Revel
d25e5568bb Add a redirecting fetcher for the doi: URI scheme. Not much used but just in case...
svn path=/trunk/netsurf/; revision=12412
2011-05-15 20:01:25 +00:00
François Revel
069a501c4d Add an about:logo redirect to resource:netsurf.png.
svn path=/trunk/netsurf/; revision=12407
2011-05-14 00:17:25 +00:00
Michael Drake
48b374736a Various changes to about: content markup. Added me to LibNSFB bit.
svn path=/trunk/netsurf/; revision=12105
2011-03-18 12:10:15 +00:00
Michael Drake
d2f8eee65b Fix comment.
svn path=/trunk/netsurf/; revision=12094
2011-03-17 12:26:59 +00:00
Vincent Sanders
50b9de4864 set hidden flag for more about: pages
svn path=/trunk/netsurf/; revision=12093
2011-03-17 12:24:40 +00:00
Michael Drake
49f1d58938 Less bloated "license" spelling handler.
svn path=/trunk/netsurf/; revision=12092
2011-03-17 12:17:25 +00:00
Daniel Silverstone
293c2b026b Retabify about.c since I had previously moosed it
svn path=/trunk/netsurf/; revision=12091
2011-03-17 12:12:55 +00:00
Michael Drake
4de9349fb8 Skip duplicates on about:about.
svn path=/trunk/netsurf/; revision=12090
2011-03-17 12:09:48 +00:00
Michael Drake
a70c32422a gui_find_resource --> gui_get_resource_url.
svn path=/trunk/netsurf/; revision=12089
2011-03-17 11:57:09 +00:00
Vincent Sanders
7390a6805e implement about:about
svn path=/trunk/netsurf/; revision=12085
2011-03-17 10:20:04 +00:00
Vincent Sanders
d90ca6181b Improve options output
improve options:config styling

svn path=/trunk/netsurf/; revision=12084
2011-03-17 09:21:02 +00:00
Vincent Sanders
bd39eb4267 make resource header documentation more explicit
svn path=/trunk/netsurf/; revision=12083
2011-03-17 08:11:54 +00:00
Michael Drake
7f454dff4b Add about:license --> about:licence redirect.
svn path=/trunk/netsurf/; revision=12076
2011-03-16 14:17:22 +00:00
John Mark Bell
408f78b737 Move schedule.h to utils/
svn path=/trunk/netsurf/; revision=12039
2011-03-13 22:50:54 +00:00
Daniel Silverstone
93e9bfe323 Shunt the schedule function definitions to desktop/schedule.h. Shunt the hlcache/llcache to using schedule to get their cleanups run.
svn path=/trunk/netsurf/; revision=12029
2011-03-13 18:26:46 +00:00
Daniel Silverstone
514b94b1f4 Add hostname and path
svn path=/trunk/netsurf/; revision=12023
2011-03-13 12:17:18 +00:00
Daniel Silverstone
8695000b83 Slightly better out-of-svn build info
svn path=/trunk/netsurf/; revision=12022
2011-03-13 12:09:01 +00:00
Daniel Silverstone
cc18b5f21e Initial testament functionality
svn path=/trunk/netsurf/; revision=12020
2011-03-13 11:59:20 +00:00
Daniel Silverstone
314124d3ef Instead of deleting the early scheduled poll, simply always add another poll if cURL thinks there might be work to do later
svn path=/trunk/netsurf/; revision=12003
2011-03-12 23:59:37 +00:00
Michael Drake
66ac795f04 Make about:config a bit nicer, if resource: is implemented on your front end.
svn path=/trunk/netsurf/; revision=11999
2011-03-12 23:00:19 +00:00
Vincent Sanders
01d4be82c4 fix about:licence
svn path=/trunk/netsurf/; revision=11996
2011-03-12 22:38:00 +00:00
Daniel Silverstone
d558aae0bb I hate this, but it helps to prevent stalling during early fetch stages with curl
svn path=/trunk/netsurf/; revision=11986
2011-03-12 19:25:40 +00:00
John Mark Bell
997e829b3d Stop pretending to support anything other than HTTP(s)
svn path=/trunk/netsurf/; revision=11966
2011-03-12 00:06:15 +00:00
John Mark Bell
2e04c57929 Fix WWW-Authenticate parsing to leave f->realm alone if there is no realm specified in the header
svn path=/trunk/netsurf/; revision=11793
2011-02-25 00:41:16 +00:00
John Mark Bell
308f549f78 Fix backend fetcher poll loops
svn path=/trunk/netsurf/; revision=11775
2011-02-23 22:41:17 +00:00
Vincent Sanders
0231675abf add resource handling
move gtk and framebuffer to use generic resource handling

svn path=/trunk/netsurf/; revision=11772
2011-02-23 22:27:55 +00:00
John Mark Bell
1b5d9f07c8 Squash warnings
svn path=/trunk/netsurf/; revision=11745
2011-02-20 23:41:50 +00:00
Michael Drake
f54fc080c2 Add module namespace to options_snoptionf()
svn path=/trunk/netsurf/; revision=11739
2011-02-20 22:38:43 +00:00
Vincent Sanders
675a07083a add options display to about: scheme
svn path=/trunk/netsurf/; revision=11733
2011-02-20 21:04:53 +00:00
Vincent Sanders
fa98e3d76a add about: fetcher
rename fetchers to be more sensible 

svn path=/trunk/netsurf/; revision=11692
2011-02-15 23:18:10 +00:00
John Mark Bell
3fcb02afa1 Revert r10650/1. We no longer require noisy fetch/cache logging.
svn path=/trunk/netsurf/; revision=11223
2011-01-06 00:31:54 +00:00
John Mark Bell
cf30b8b074 Use ETag to validate file: URLs, returning 304 if there's no change.
Force directories to be uncacheable.
Don't emit a Last-Modified header for files, as it may result in a cached object being considered fresh (c.f. RFC2616 $13.2.3/4)
Fix leak of file descriptor on failure to allocate buffer.

svn path=/trunk/netsurf/; revision=10782
2010-09-15 20:32:14 +00:00
John Mark Bell
051dc72a56 Purge last vestiges of file: cruft from curl binding
svn path=/trunk/netsurf/; revision=10781
2010-09-15 19:40:17 +00:00
John Mark Bell
1075214307 Improve error reporting, and reflow to 80 columns.
svn path=/trunk/netsurf/; revision=10768
2010-09-13 18:34:11 +00:00
Chris Young
4972283771 Ensure the file that is read for info is in the platform's native path format, whilst
the path used for any links is in the correct URL format.

svn path=/trunk/netsurf/; revision=10761
2010-09-12 11:42:37 +00:00
Vincent Sanders
faca1687ce hell with it, heres a version which should result in fewer portability complaints
svn path=/trunk/netsurf/; revision=10756
2010-09-11 18:47:36 +00:00
John Mark Bell
2077918805 Merge branches/vince/netsurf-file-fetcher to trunk
r=jmb

svn path=/trunk/netsurf/; revision=10750
2010-09-09 21:45:59 +00:00
John Mark Bell
1052f19075 Don't suppress curl debug output, either
svn path=/trunk/netsurf/; revision=10651
2010-07-17 09:13:44 +00:00
Vincent Sanders
03bb3e8636 remove _GNU_SOURCE define from everywhere.
strndup is not standard so provide an implementation.

svn path=/trunk/netsurf/; revision=10474
2010-04-24 15:22:17 +00:00
John Mark Bell
014cbd63cc Purge spurious filetype computation
svn path=/trunk/netsurf/; revision=10431
2010-04-18 16:08:32 +00:00
John Mark Bell
06145e570d Fix mismatched free call
svn path=/trunk/netsurf/; revision=10420
2010-04-17 09:05:15 +00:00
Vincent Sanders
448b0275ae Fix file: handling on risc os, gtk, windows and framebuffer frontends
svn path=/trunk/netsurf/; revision=10419
2010-04-16 23:56:53 +00:00
John Mark Bell
b0f1b7b170 Improve const-correctness of fetch_start
svn path=/trunk/netsurf/; revision=10342
2010-04-10 17:13:53 +00:00
Daniel Silverstone
6c9e4bea8e Purge FETCH_TYPE which is unused throughout NetSurf. Clearing a todo in llcachec.
svn path=/trunk/netsurf/; revision=10300
2010-04-08 12:16:52 +00:00
Michael Drake
bbd757c34a Fix fetch_curl_post_convert() for filename_from_path() returning NULL.
svn path=/trunk/netsurf/; revision=10265
2010-04-07 11:10:55 +00:00
Vincent Sanders
95e4a737de rationalise the usage of the file scheme
svn path=/trunk/netsurf/; revision=10221
2010-03-31 13:59:57 +00:00
Daniel Silverstone
270ef59a98 Merge jmb/new-cache; r=dsilvers,rs=vince
svn path=/trunk/netsurf/; revision=10180
2010-03-28 12:56:39 +00:00
Chris Young
033b5d815a Move code which extracts the filename from a given path into frontend.
svn path=/trunk/netsurf/; revision=10139
2010-03-21 13:32:59 +00:00
John Mark Bell
ee833877c5 Rate limit fetch progress notifications.
svn path=/trunk/netsurf/; revision=9962
2010-02-10 00:32:12 +00:00
Vincent Sanders
6990c2cd04 remove uneccissary includes
svn path=/trunk/netsurf/; revision=9931
2010-01-29 11:32:53 +00:00
James Bursa
914eedc11d Don't set proxy option for file: URLs to workaround some versions of libcurl attempting to use it (fixed in latest libcurl). Also unset curl proxy option if it's disabled so the previous xfer's setting isn't retained.
svn path=/trunk/netsurf/; revision=9759
2009-12-25 19:27:22 +00:00
John Mark Bell
355799ce0b Merge branches/MarkieB/gtkmain to trunk.
svn path=/trunk/netsurf/; revision=9729
2009-12-17 23:55:02 +00:00
John Tytgat
2261b616f6 - content/urldb.c(auth_data): Removed;
(prot_space_data): Added, it lives linked in the leaf host_part
  struct and together with its scheme and port (which defins canonical root
  url) and realm this defines a protection space.
  (path_data): Removed auth_data field and replaced by a prot_space_data
  pointer.
  (host_part::prot_space): Added linked list of protection space data
  structs.
  (urldb_get_auth_details): Given an URL fetch fetches its auth.
  (urldb_set_auth_details): Creates or updates the contents of a
  protection space to which given URL belongs.
  (urldb_destroy_host_tree): Delete protection data space structures
  using urldb_destroy_prot_space.
  (urldb_destroy_prot_space): Added.
- content/urldb.h(urldb_get_auth_details): Added realm parameter.
- content/fetchers/fetch_curl.c(fetch_curl_set_options): Update
  urldb_get_auth_details call (we don't know realm at this point).
- content/fetchcache.c(fetchcache_callback, fetchcache_auth): At FETCH_AUTH,
  use realm to determine if we really don't know auth data and if so,
  refetch content.
- content/content.h(struct content): Add content::tried_with_auth.
- content/content.c(content_create): Initialize content::tried_with_auth.
- riscos/401login.c(ro_gui_401login_open): Show known authentication
  data in dialogue so user can see what was wrong with it and correct it.

Solves bug #2830829.

svn path=/trunk/netsurf/; revision=9045
2009-08-04 23:02:23 +00:00
John Mark Bell
243c1ba03f Limit scope of manually-defined NDEBUG. Purge a bunch of redundant #undef NDEBUG.
svn path=/trunk/netsurf/; revision=7593
2009-05-28 10:39:53 +00:00
John Mark Bell
42e24e8099 s/abort/abort_fetch/
svn path=/trunk/netsurf/; revision=7512
2009-05-14 10:13:20 +00:00
John Mark Bell
b98f105b24 Purge WITH_AUTH
svn path=/trunk/netsurf/; revision=6600
2009-02-22 15:13:10 +00:00
John Mark Bell
847b776285 Purge NETSURF_USE_SSL and WITH_SSL
svn path=/trunk/netsurf/; revision=6599
2009-02-22 15:07:50 +00:00
John Mark Bell
16c01d97e9 Log detected state of openssl availability
svn path=/trunk/netsurf/; revision=6595
2009-02-21 23:04:49 +00:00
John Mark Bell
756c393abb Given that the build-time SSL detection can be somewhat inaccurate, attempt to determine whether we can use the SSL_CTX stuff at runtime.
svn path=/trunk/netsurf/; revision=6586
2009-02-21 02:46:47 +00:00
John Mark Bell
f7f19ab0ed Squash strict-aliasing warning in the most disgusting way possible.
svn path=/trunk/netsurf/; revision=6070
2009-01-15 16:00:16 +00:00
Daniel Silverstone
4e3951660f Silence a bunch of warnings caused by cURL's typecheck-gcc stuff being strict and in one case cURL simply being daft (whoever heard of a signed HTTP response code?).
svn path=/trunk/netsurf/; revision=5991
2009-01-08 10:36:11 +00:00
Daniel Silverstone
5c01ca3306 Better, safer fix for the type punning issue
svn path=/trunk/netsurf/; revision=5562
2008-10-13 16:39:54 +00:00
Daniel Silverstone
1108d1d1c5 Fix type-punning warning in fetch_data.c
svn path=/trunk/netsurf/; revision=5561
2008-10-13 16:27:45 +00:00
Daniel Silverstone
ac427e05e5 Add additional debug to fetch errors from cURL
svn path=/trunk/netsurf/; revision=5174
2008-08-22 11:10:33 +00:00
François Revel
f28f4cc760 First round of C89 fixes. Those should be obvious, hope I didn't introduce any side effect.
svn path=/trunk/netsurf/; revision=4788
2008-07-29 09:05:36 +00:00
James Bursa
3d590b1efa Handle errors returned by curl_multi_perform(), such as out of memory, instead of exiting.
svn path=/trunk/netsurf/; revision=4329
2008-06-14 21:43:52 +00:00
James Bursa
ef6690ad72 Stop processing headers and sending FETCH_HEADER if fetch was aborted.
svn path=/trunk/netsurf/; revision=4231
2008-05-31 16:52:37 +00:00
John Mark Bell
f4be61553c Fix handling of cache control data
svn path=/trunk/netsurf/; revision=4227
2008-05-30 11:09:50 +00:00
James Bursa
ae992eab4d Add new fetch callback FETCH_HEADER for headers and move as much header parsing as possible from fetch_curl.c to fetchcache.c. This simplifies fetch_curl.c and will make it possible to store response headers in future.
svn path=/trunk/netsurf/; revision=4226
2008-05-30 04:11:16 +00:00
John Tytgat
793d466edf - introduction of SLEN()
- enforce result of url_normalize() being NULL on failure
- plug memory leak when url_normalize() fails

svn path=/trunk/netsurf/; revision=4200
2008-05-25 15:51:30 +00:00
John Mark Bell
c24b426881 No longer suppress the default Accept: header.
This fixes all those sites that brokenly assume that it's required and thus break when the client doesn't send one (here's looking at you royalmail.com).
libcurl's default Accept header is "*/*", which is semantically equivalent to not sending a header at all (no header implies the client accepts all content types).

svn path=/trunk/netsurf/; revision=4196
2008-05-24 10:09:02 +00:00
John Mark Bell
74a1095cd6 Remove unnecessary includes
svn path=/trunk/netsurf/; revision=4144
2008-05-11 10:09:44 +00:00
John Mark Bell
d260420249 Squash warning
svn path=/trunk/netsurf/; revision=3983
2008-03-19 17:36:07 +00:00
Rob Kendrick
fed99c2cb5 jmb's rework of data: URL handling
svn path=/trunk/netsurf/; revision=3890
2008-03-06 11:28:00 +00:00
Rob Kendrick
58ee433b67 Add another check for malformed data: URLs
svn path=/trunk/netsurf/; revision=3889
2008-03-05 23:09:36 +00:00
Rob Kendrick
b168d636d0 Many data: fixes. Acid2 no longer crashes NetSurf.
svn path=/trunk/netsurf/; revision=3888
2008-03-05 22:57:46 +00:00
Rob Kendrick
c9bd38f5dd Check for calloc() failing
svn path=/trunk/netsurf/; revision=3885
2008-03-05 15:01:42 +00:00
Rob Kendrick
0e43415013 Use curl_free() when freeing data: URLs that are URL-encoded instead of using free()
svn path=/trunk/netsurf/; revision=3884
2008-03-05 14:57:33 +00:00
Rob Kendrick
8e79665248 Initial attempt at implementing data: URLs. Needs cleaning and verifying that I'm using the fetcher API correctly. Appears to work, though.
svn path=/trunk/netsurf/; revision=3882
2008-03-05 14:21:29 +00:00
John Mark Bell
cc3481a2ff Make core fetching code handle verifiability of transactions
Make core fetching code be responsible for inserting cookies into the urldb
Provide accessor to a fetch's parent url (this is defined as being the URL of the verifiable fetch which caused this one to occur)
Make fetchcache's 3xx handling use the parent url when spawning new fetches

svn path=/trunk/netsurf/; revision=3809
2008-01-30 19:56:41 +00:00
Vincent Sanders
2f864eeecc Update all source code file headers to reflect GPL version 2 only and contain appropriate licence text
svn path=/trunk/netsurf/; revision=3486
2007-08-08 16:16:03 +00:00
John Mark Bell
40b1c80896 Ensure option strings aren't blank before setting headers: previously, a blank option string could result in "Accept-Language: , *;q=0.1" being sent, for example.
svn path=/trunk/netsurf/; revision=3450
2007-07-21 08:53:00 +00:00
James Bursa
594eb74c4c Fix deadlock when fetching stylesheets when max_fetchers_per_host is reached by splitting fetch_can_be_freed() into fetch_remove_from_queues() and fetch_free().
svn path=/trunk/netsurf/; revision=3380
2007-07-04 18:44:13 +00:00
James Bursa
960c11de1a Rename register_curl_fetchers() to fetch_curl_register(). Add declarations for static functions and reorder functions.
svn path=/trunk/netsurf/; revision=3379
2007-07-04 18:05:16 +00:00
Daniel Silverstone
e2d0019119 Add option_suppress_curl_debug and default it to true
svn path=/trunk/netsurf/; revision=3373
2007-06-27 16:27:52 +00:00
Daniel Silverstone
82c4458689 Only pass option_ca_bundle and option_ca_path to cURL if they are non-empty.
svn path=/trunk/netsurf/; revision=3372
2007-06-27 16:13:54 +00:00
Daniel Silverstone
e91cfc8e47 Add ca_path option which sets CURLOPT_CAPATH.
svn path=/trunk/netsurf/; revision=3369
2007-06-27 15:39:56 +00:00
John Mark Bell
87ca206e13 Warning fixes.
svn path=/trunk/netsurf/; revision=3332
2007-06-10 18:08:22 +00:00
Daniel Silverstone
1dd7e97eb3 Merge scheme switcher branch in.
svn path=/trunk/netsurf/; revision=3330
2007-06-10 17:46:44 +00:00