Commit Graph

145 Commits

Author SHA1 Message Date
Michael Drake
ba9769bc8c Pass fetch redirect info up to content layer as content_msg. Mark redirect origin URLs as visited in browser window content callback. Note this doesn't mean we track redirects, it just lets us get the :visited link style on links that redirect. 2013-05-27 14:51:00 +01:00
Vincent Sanders
848f3d13ca ensure parameter contract is met 2013-05-04 08:52:12 +01:00
Vincent Sanders
82e2fc57d7 free memory on error path 2013-05-01 22:15:24 +01:00
Vincent Sanders
87ee963ff2 Initial changes ready to improve caching decisions 2013-01-07 18:45:29 +00:00
John-Mark Bell
3dff750ae2 Downgrade TLS version support if it turns out the server can't cope with TLSv1.1. 2013-01-04 23:13:23 +00:00
Michael Drake
af89fdb38a Fixup for nsurl urldb. 2012-10-09 11:20:21 +01:00
Vincent Sanders
6f86862edb cleanup and reorganise low level cache source ready for extending with disc cache
svn path=/trunk/netsurf/; revision=13893
2012-04-28 13:24:09 +00:00
John Mark Bell
c22038bed8 Fix bogus cache validation decision
svn path=/trunk/netsurf/; revision=13261
2011-12-08 22:44:19 +00:00
Michael Drake
714f07c48b Rename nsurl_enquire to nsurl_has_component.
svn path=/trunk/netsurf/; revision=13183
2011-11-27 18:32:06 +00:00
John Mark Bell
dcbafe6b87 Clean up fetch callback API. Inject some long-needed type safety.
svn path=/trunk/netsurf/; revision=13137
2011-11-08 21:51:42 +00:00
Michael Drake
17525fbf25 Avoid making a new nsurl to remove fragment if there is no fragment.
svn path=/trunk/netsurf/; revision=12958
2011-10-05 18:05:41 +00:00
Michael Drake
57547f7b1e Make llcache_handle_get_url return a nsurl.
svn path=/trunk/netsurf/; revision=12905
2011-09-28 14:21:35 +00:00
Michael Drake
861f88aa43 Port llcache to nsurl.
svn path=/trunk/netsurf/; revision=12904
2011-09-28 11:26:30 +00:00
Michael Drake
e1f7a37f15 Port fetch layer to nsurl. Remove unused fetch_get_referer function.
svn path=/trunk/netsurf/; revision=12899
2011-09-27 11:07:32 +00:00
Vincent Sanders
2d33a8f85a Make high level cache, low level cache and image cache all be initialised from passed parameters
Calculate all cache sizes from single memory cache size option and sanity check
have a single global struct to hold all parameters instead of several individual variables

svn path=/trunk/netsurf/; revision=12784
2011-09-10 00:55:39 +00:00
John Mark Bell
f6ac059a05 Handle zero-length documents correctly
svn path=/trunk/netsurf/; revision=12778
2011-09-08 08:26:16 +00:00
John Mark Bell
6807fa854d Sniff content types where appropriate.
We never sniff for CSS, nor for non-page artefacts (e.g. treeview icons)

svn path=/trunk/netsurf/; revision=12707
2011-09-04 06:28:09 +00:00
John Mark Bell
da7c3c10bc Implement todo
svn path=/trunk/netsurf/; revision=12418
2011-05-16 21:47:36 +00:00
John Mark Bell
35cd1a79f7 Squash static analyser warning
svn path=/trunk/netsurf/; revision=12416
2011-05-16 21:42:53 +00:00
Daniel Silverstone
93e9bfe323 Shunt the schedule function definitions to desktop/schedule.h. Shunt the hlcache/llcache to using schedule to get their cleanups run.
svn path=/trunk/netsurf/; revision=12029
2011-03-13 18:26:46 +00:00
John Mark Bell
60b5039c33 Protect against object being cleaned while waiting for a query response
svn path=/trunk/netsurf/; revision=11894
2011-03-03 00:40:50 +00:00
John Mark Bell
d25a88145b Make no-cache a tristate. In the case of a forced reload, we want to revalidate once only.
svn path=/trunk/netsurf/; revision=11791
2011-02-24 21:22:24 +00:00
John Mark Bell
00bf0fbd24 Revert the functional change from r11789: the no-cache directive forces us to revalidate a cached object, not reject it entirely.
svn path=/trunk/netsurf/; revision=11790
2011-02-24 21:06:45 +00:00
John Mark Bell
4f740f5a32 Ensure cache hits after a forced reload -- don't even attempt to reuse objects with the no_cache flag set (as they'll never be considered fresh, thus requiring a round trip to the server every time they're used). Instead, we'll fetch a new object, which will be considered fresh for caching purposes.
svn path=/trunk/netsurf/; revision=11789
2011-02-24 20:51:35 +00:00
John Mark Bell
6b26f77bb1 Use cache control data invalidation function everywhere we want the cache control data reset.
Fix cache control data invalidation to reset the age and max_age fields to INVALID_AGE.
Fix cache control data reset on encountering an HTTP response line to occur before we fill in the response time, rather than after it.

svn path=/trunk/netsurf/; revision=11783
2011-02-24 10:14:50 +00:00
John Mark Bell
b967824575 Improve commentary.
Stop leaking object_scheme in error cases.

svn path=/trunk/netsurf/; revision=11778
2011-02-23 23:05:36 +00:00
Vincent Sanders
0231675abf add resource handling
move gtk and framebuffer to use generic resource handling

svn path=/trunk/netsurf/; revision=11772
2011-02-23 22:27:55 +00:00
John Mark Bell
d8d0353a73 Fix bug #3184972: cope with server sending a 304 in response to an unconditional request.
svn path=/trunk/netsurf/; revision=11710
2011-02-17 17:50:14 +00:00
John Mark Bell
3fcb02afa1 Revert r10650/1. We no longer require noisy fetch/cache logging.
svn path=/trunk/netsurf/; revision=11223
2011-01-06 00:31:54 +00:00
Michael Drake
0eb65d3344 Add some assert()s.
svn path=/trunk/netsurf/; revision=11144
2010-12-29 20:42:00 +00:00
John Mark Bell
bcfb508f86 User may have no handle if it was the target of an iterator when the handle was aborted
svn path=/trunk/netsurf/; revision=11071
2010-12-16 00:18:41 +00:00
John Mark Bell
c675c3552d Ensure that we clean up after the client even if they report an error from their notification callback.
svn path=/trunk/netsurf/; revision=11067
2010-12-15 22:59:19 +00:00
John Mark Bell
d745dae903 Make llcache_object_user objects distinct from llcache_handles.
This allows us to be robust against clients completely changing the contents of the user list underneath us during iteration.
Fix llcache_send_event_to_users to also be robust against the user list changing under it.

svn path=/trunk/netsurf/; revision=11063
2010-12-15 21:15:49 +00:00
John Mark Bell
67556f54fe Always invalidate next/prev pointers
svn path=/trunk/netsurf/; revision=11052
2010-12-13 08:36:27 +00:00
John Mark Bell
c0f49a2785 Revert r11047-11050: if we remove a user from an object while it's the iterator target, the effect will be that the iteration terminates early.
Subsequent users that require notifications will simply have their notifications delayed

svn path=/trunk/netsurf/; revision=11051
2010-12-13 08:34:01 +00:00
John Mark Bell
f919b689ba Take 2: make callers of llcache_object_user_destroy invalidate next/prev pointers
svn path=/trunk/netsurf/; revision=11050
2010-12-13 08:28:22 +00:00
John Mark Bell
a5221248c2 Revert last change
svn path=/trunk/netsurf/; revision=11049
2010-12-13 08:25:54 +00:00
John Mark Bell
991472d264 Remove bogus assertions
svn path=/trunk/netsurf/; revision=11048
2010-12-13 08:24:04 +00:00
John Mark Bell
f40da966ad We cannot invalidate a user's next/prev pointers when removing it from an object, as it may be the target of iteration
svn path=/trunk/netsurf/; revision=11047
2010-12-13 08:16:32 +00:00
John Mark Bell
702c94bb30 Fix assertion: user->next may be NULL, so there's no point asserting that it isn't
svn path=/trunk/netsurf/; revision=11046
2010-12-13 08:12:31 +00:00
John Mark Bell
d9f8b8949e Fix assertion to compare, and not assign
svn path=/trunk/netsurf/; revision=11045
2010-12-13 08:10:29 +00:00
John Mark Bell
650f884527 Make llcache_object_notify_users robust to the object's user list changing underneath it
svn path=/trunk/netsurf/; revision=11044
2010-12-12 23:37:21 +00:00
John Mark Bell
e5bc525030 Fix bug 3096811: discard HTTP headers for all but the last response
svn path=/trunk/netsurf/; revision=10912
2010-10-27 22:12:00 +00:00
John Mark Bell
a10c4d3a7b Fix crash on receipt of a 304 response: it turns out that FETCH_HEADER may be called before any decision about how to process the response is made.
Move FETCH_HEADER handler to start of switch statement to reflect this.

svn path=/trunk/netsurf/; revision=10792
2010-09-16 22:46:40 +00:00
John Mark Bell
a7dd752046 Ensure that candidate count is reduced if a conditional request results in a non-304 response
svn path=/trunk/netsurf/; revision=10791
2010-09-16 22:04:06 +00:00
Vincent Sanders
1391ebe5cc perpetrate a neatness to centralise all the low level cache cache control data invalidation
svn path=/trunk/netsurf/; revision=10787
2010-09-16 19:47:15 +00:00
John Mark Bell
db736f8378 Fix heap corruption
svn path=/trunk/netsurf/; revision=10786
2010-09-16 19:33:18 +00:00
John Mark Bell
dafccf21d5 Fix validation of invalidated cache entries.
svn path=/trunk/netsurf/; revision=10785
2010-09-16 18:58:32 +00:00
Daniel Silverstone
be59d5870b Ensure we free etags before invalidating cache control data
svn path=/trunk/netsurf/; revision=10784
2010-09-16 16:21:58 +00:00
John Mark Bell
c88044c8cb Forcibly invalidate contents with response codes that aren't 200 or 203.
svn path=/trunk/netsurf/; revision=10780
2010-09-15 19:33:18 +00:00
John Mark Bell
78f3bbaed3 Fix bug #3055480: properly compare URLs, instead of erroneously assuming that strcasecmp is the correct answer.
svn path=/trunk/netsurf/; revision=10721
2010-08-29 17:41:54 +00:00
John Mark Bell
00aeb20334 Less spam
svn path=/trunk/netsurf/; revision=10687
2010-08-10 21:46:45 +00:00
John Mark Bell
48b17a5aea Rate-limit cache clean attempts
svn path=/trunk/netsurf/; revision=10686
2010-08-10 19:58:39 +00:00
John Mark Bell
183974b437 Enable noisy logging
svn path=/trunk/netsurf/; revision=10650
2010-07-17 08:17:42 +00:00
John Mark Bell
0de2013f25 Clean up caches on exit
svn path=/trunk/netsurf/; revision=10494
2010-04-27 06:48:09 +00:00
Vincent Sanders
03bb3e8636 remove _GNU_SOURCE define from everywhere.
strndup is not standard so provide an implementation.

svn path=/trunk/netsurf/; revision=10474
2010-04-24 15:22:17 +00:00
John Mark Bell
248a53f6ba Avoid double free when shrink-wrapping source data on fetch completion.
svn path=/trunk/netsurf/; revision=10423
2010-04-17 21:17:08 +00:00
John Mark Bell
01eb197f56 When the fetch system informs its clients that certificate verification has failed, it has already stopped and cleaned up the fetch.
Stop llcache_fetch_cert_error attempting to abort the fetch, as this will result in a double free.

svn path=/trunk/netsurf/; revision=10418
2010-04-16 23:15:22 +00:00
John Mark Bell
44552e7492 Provide API to invalidate llcache object cache control data
svn path=/trunk/netsurf/; revision=10411
2010-04-16 20:42:10 +00:00
John Mark Bell
ca0daa1a66 Reset fetch state when refetching object.
Ensure that cache control data is invalidated for 401/certificate failure so that conditional headers are not emitted when object is refetched.

svn path=/trunk/netsurf/; revision=10408
2010-04-16 20:08:27 +00:00
John Mark Bell
b008327de1 Improve 401 response handling
svn path=/trunk/netsurf/; revision=10407
2010-04-16 20:00:10 +00:00
John Mark Bell
1c42c0569f Ensure we invalidate cache data and mark objects as complete on failure to fetch.
svn path=/trunk/netsurf/; revision=10402
2010-04-14 20:42:03 +00:00
Daniel Silverstone
2ba2dab3a7 Eat any #fragment in llcache_object_retrieve -- fixing double-fragment issues at the top level, and preventing fragment(ed) redirects from confusing the issue
svn path=/trunk/netsurf/; revision=10399
2010-04-14 14:53:21 +00:00
Daniel Silverstone
b56886d0ee Abstract a number of unsafe iterations of object users into a single safe function call
svn path=/trunk/netsurf/; revision=10398
2010-04-14 10:56:44 +00:00
John Mark Bell
4b8c720ac5 Shrink source buffer to final length on fetch completion
svn path=/trunk/netsurf/; revision=10373
2010-04-11 21:26:20 +00:00
John Mark Bell
6a88eaf67a Pedantic whitespace changes
svn path=/trunk/netsurf/; revision=10356
2010-04-11 14:39:26 +00:00
John Mark Bell
fe87225ccf Complete low-level cache cleaner
svn path=/trunk/netsurf/; revision=10355
2010-04-11 14:32:39 +00:00
John Mark Bell
9666787dfa Remove redundant todo
svn path=/trunk/netsurf/; revision=10354
2010-04-11 14:12:13 +00:00
John Mark Bell
a70c67fa35 Support 307 redirects for GET requests
svn path=/trunk/netsurf/; revision=10353
2010-04-11 14:11:33 +00:00
John Mark Bell
863d5c77c3 Forbid redirects to file:/// URLs
svn path=/trunk/netsurf/; revision=10352
2010-04-11 14:06:02 +00:00
John Mark Bell
40bac34ffd Bail out if we're redirected to an URL we can't handle
svn path=/trunk/netsurf/; revision=10351
2010-04-11 14:01:11 +00:00
John Mark Bell
758190f37b Reintroduce redirect limit
svn path=/trunk/netsurf/; revision=10350
2010-04-11 13:58:12 +00:00
John Mark Bell
3470764041 Remove redundant todos
svn path=/trunk/netsurf/; revision=10349
2010-04-11 13:29:02 +00:00
John Mark Bell
b0f1b7b170 Improve const-correctness of fetch_start
svn path=/trunk/netsurf/; revision=10342
2010-04-10 17:13:53 +00:00
John Mark Bell
ee2a5b40ce Use computed POST data when redirecting, instead of original.
Certain redirect methods result in POST being converted to GET.

svn path=/trunk/netsurf/; revision=10339
2010-04-10 11:56:14 +00:00
Daniel Silverstone
dfae5c33d2 Some more asserts to try and help spot multiple-release of llcache handles
svn path=/trunk/netsurf/; revision=10327
2010-04-09 00:06:36 +00:00
Daniel Silverstone
4262232956 Fix missing 'break' introduced by 10300
svn path=/trunk/netsurf/; revision=10304
2010-04-08 13:18:50 +00:00
Daniel Silverstone
92bd0d4ce0 Add an assert to clarify a segfault I'm investigating
svn path=/trunk/netsurf/; revision=10303
2010-04-08 13:12:30 +00:00
Daniel Silverstone
5c55264217 Remove todo -- no longer appropriate
svn path=/trunk/netsurf/; revision=10302
2010-04-08 12:24:29 +00:00
Daniel Silverstone
45faa483e5 Ensure an error'd fetch's llcache object is implicitly stale so it is not reused
svn path=/trunk/netsurf/; revision=10301
2010-04-08 12:20:20 +00:00
Daniel Silverstone
6c9e4bea8e Purge FETCH_TYPE which is unused throughout NetSurf. Clearing a todo in llcachec.
svn path=/trunk/netsurf/; revision=10300
2010-04-08 12:16:52 +00:00
Daniel Silverstone
2b1146bcb3 Remove parent from fetches, clearing a todo from llcache.c
svn path=/trunk/netsurf/; revision=10299
2010-04-08 11:47:05 +00:00
John Mark Bell
4682f0ccd8 When we've snapshotted an object, make sure we add the *new* object into the list of uncacheable objects, and not the old one. Otherwise, our cache chains end up being corrupted.
svn path=/trunk/netsurf/; revision=10295
2010-04-08 07:08:16 +00:00
John Mark Bell
2748fe4f64 Make downloads work again.
svn path=/trunk/netsurf/; revision=10243
2010-04-05 21:35:38 +00:00
John Mark Bell
79ce683b4e Most of a stop implementation.
Remaining work:
  1) Clone content_html_data
  2) Cloning content_css_data requires the charset of the old content
  3) Calling hlcache_handle_abort() before a content has been created must clean up the retrieval context.

svn path=/trunk/netsurf/; revision=10236
2010-04-04 12:41:19 +00:00
John Mark Bell
9f575c590b Do not abort fetch on receipt of FETCH_ERROR: the fetch has already been destroyed.
Implement LLCACHE_EVENT_ERROR case of hlcache_llcache_callback().

svn path=/trunk/netsurf/; revision=10218
2010-03-30 21:36:39 +00:00
John Mark Bell
99dee5e3d0 Disable llcache tracing
svn path=/trunk/netsurf/; revision=10201
2010-03-29 06:29:56 +00:00
John Mark Bell
b6f6bc5fdd Fix broken linked list removal.
svn path=/trunk/netsurf/; revision=10200
2010-03-29 06:29:25 +00:00
John Mark Bell
311b458d56 Forbid contents with active fetches from being cleaned.
svn path=/trunk/netsurf/; revision=10199
2010-03-29 06:28:46 +00:00
John Mark Bell
4e2c3a4c39 Exceptionally verbose tracing of llcache operation.
svn path=/trunk/netsurf/; revision=10198
2010-03-29 06:27:37 +00:00
John Mark Bell
57253cdd58 Fixup per review
svn path=/trunk/netsurf/; revision=10196
2010-03-28 23:25:21 +00:00
Daniel Silverstone
66d1418506 Allow us to cache foo?bar when it explicitly allows it. as per rfc2616 13.9
svn path=/trunk/netsurf/; revision=10190
2010-03-28 16:51:47 +00:00
Daniel Silverstone
8effd1cb98 Ensure that finished redirection objects in llcache do not get found by subsequent fetch attempts.
svn path=/trunk/netsurf/; revision=10188
2010-03-28 16:28:28 +00:00
Daniel Silverstone
cb7412181f Replace fetch_poll with llcache_poll throughout. Allow CONTENT_OTHER to complete properly.
svn path=/trunk/netsurf/; revision=10187
2010-03-28 16:00:54 +00:00
Daniel Silverstone
270ef59a98 Merge jmb/new-cache; r=dsilvers,rs=vince
svn path=/trunk/netsurf/; revision=10180
2010-03-28 12:56:39 +00:00