mirrors/netsurf
1
0
Fork 0
mirror of https://github.com/netsurf-browser/netsurf synced 2024-12-19 10:42:36 +03:00
Code Issues Packages Projects Releases Wiki Activity
14,533 Commits 6 Branches 12 Tags 33 MiB
26b189af67
Commit Graph

187 Commits

Author SHA1 Message Date
Daniel Silverstone
4d1ef3bac4 Add support for retrying timed-out cURL fetches. 
This is an attempt to amelioriate the situation found in #2384 where
we see the cURL connect() failing to complete.  Based on the pcap
from the bug log, we believe that RISC OS is likely failing to signal
the completion of the connection to cURL.  As such, cURL times out.

This change permits retries of timed out connections in the hope that
a fresh socket FD might subsequently function correctly.  The defaults
chosen mean that the previous behaviour of 30 seconds before timeout
is reported will remain the same, but in that time we will make 3 separate
attempts to connect the socket.
 2015-11-10 21:51:54 +00:00
Vincent Sanders
73e45ff024 Fix error reporting from fetch_start 
Any fetch start error was being reported as "out of memory" which was
clearly insufficient. Foe example bad urls (reported was file:// with
a missing /) were causing a warn_user with out of memory. This change
now at least causes a "bad url" message.
 2015-06-24 10:31:13 +01:00
Vincent Sanders
0d811963c1 fixup all the remaining logging macro callsites 
The semantic patch tool appears to have missed some difficult to
reason about callsites which have now been manually cleaned up
 2015-05-28 17:06:18 +01:00
Vincent Sanders
c105738fa3 Change LOG() macro to be varadic 
This changes the LOG macro to be varadic removing the need for all
callsites to have double bracketing and allows for future improvement
on how we use the logging macros.

The callsites were changed with coccinelle and the changes checked by
hand. Compile tested for several frontends but not all.

A formatting annotation has also been added which allows the compiler
to check the parameters and types passed to the logging.
 2015-05-28 16:08:46 +01:00
Michael Drake
8f1c9673d7 Fix format string type missmatches on 32-bit systems. 
Use C99 inttypes header for format string macros.
 2015-05-12 10:19:38 +01:00
Michael Drake
82237d7070 Avoid generating confusing warning. 
Now the disc cache will be disabled silently if its too slow.
 2015-05-12 10:01:06 +01:00
Vincent Sanders
37aad9c749 Improve the logge dinformation about written cache data 2015-05-08 15:42:49 +01:00
Vincent Sanders
2aef095f27 Ensure bandwidth minimum check is only performed when enough data has 
been written.
 2015-04-30 14:28:18 +01:00
Vincent Sanders
9927232a2d Improve metadata deserialisation to only update object on success 
The low level cache deserialisation was leaving bad data in an low
level cache object in the error case. This fixes it so the object
state only gets modified on successful deserialisation of all the
metadata.
 2015-01-28 22:39:39 +00:00
Vincent Sanders
5464dc7778 Ensure no division by zero occours if no time has ever been spent writing llcache 2015-01-03 15:13:40 +00:00
Vincent Sanders
6b645664fe add missing includes 2014-12-27 22:48:37 +00:00
Vincent Sanders
45f98ffd6e improve comments in low level cache and remove completed todo 2014-12-04 21:09:42 +00:00
Vincent Sanders
5bf4b3ff2b move the low bandwidth check to allow for an average over a much longer time 2014-12-03 20:13:10 +00:00
Vincent Sanders
57cd5c77b2 Use translated messgae for low bandwidth error 2014-12-02 10:03:27 +00:00
Vincent Sanders
c9ee49baa8 cope with backing store writeout making no progress 2014-12-01 11:32:33 +00:00
Vincent Sanders
ba5ade87b2 ensure the backing store write reports at least a minimal amount of time 2014-12-01 09:18:31 +00:00
Vincent Sanders
61ab2777da Improve writeout rescheduling if list of objects is completed quickly 2014-11-30 00:27:05 +00:00
Vincent Sanders
d08acbc5ae correctly calculate writeout bandwidth and properly impose limits 2014-11-29 23:57:15 +00:00
Vincent Sanders
6b2676d33f Improve llcache writeout stratagy 2014-11-28 19:17:42 +00:00
Vincent Sanders
8d51dd4879 update the backing store to allow for invalidating entries which have an allocation 2014-11-27 13:31:43 +00:00
Vincent Sanders
79d5b1bf3b stop using gettimeofday and use the monotonic time interface fron nsutils 2014-11-26 21:29:13 +00:00
Vincent Sanders
81417ae4a5 caclulate the total amount of time spent writing an object out 
In order to calculate the writeout bandwidth we need to know how long
it took to write the data to peristant storage in addition to how much
was written.
 2014-11-25 23:11:41 +00:00
Vincent Sanders
5a78cd810f correctly release source data from llcache 2014-11-23 18:21:56 +00:00
Vincent Sanders
01b2bc5a57 Improve doxygen comments and add todo for cache and backing store 2014-11-23 15:43:03 +00:00
Vincent Sanders
8b810ee4a1 change the persistant data store to owning the allocations 2014-11-22 23:56:43 +00:00
Vincent Sanders
56bb1b03c5 Doxygen cleanups 2014-11-09 12:50:30 +00:00
Vincent Sanders
d06df231bc Update the core to use the split operations table headers 
Second in the series to decouple the core API usage from the frontends.
 2014-10-16 22:33:43 +01:00
Vincent Sanders
4b2101ba6a clean up the fetcher factory and improve its API 2014-06-19 18:27:24 +01:00
Vincent Sanders
5899f6c16e turn off debugging and add some comments and todo from dsilvers 2014-06-03 22:30:04 +01:00
Daniel Silverstone
b5461c8313 Adjust the llcache behaviour to use scheduler for user notification. 
This change updates the llcache to use the scheduler to notify users of the
llcache of events.  This should be just as safe as before and is part of an
effort to remove hlcache_poll and llcache_poll eventually because fetchers
should schedule themselves if need-be.

This is a big change despite the diminutive nature of the patch.  Please report
issues promptly if they turn up after this and are not visible before it.

Signed-off-by: Daniel Silverstone <dsilvers@netsurf-browser.org>
Reviewed-by: Vincent Sanders <vince@netsurf-browser.org>
 2014-06-03 17:00:23 +01:00
Vincent Sanders
644fc616b3 attempt to purge low level cache on out of memory during fetch 2014-05-29 14:00:13 +01:00
Michael Drake
7ba459f1b5 Fix leak of url schemes. 2014-05-16 18:58:06 +01:00
Vincent Sanders
ef00272e2f add helpers for time_t reading/writing 2014-05-15 01:18:52 +01:00
Vincent Sanders
1a3ee6090d add strptime compatability 2014-05-13 16:37:49 +01:00
Vincent Sanders
4a49ff5266 Extend low level source data cache with persistant storage 2014-05-13 15:52:58 +01:00
Vincent Sanders
da0b969f25 Improve llcache header processing 
By skipping empty headers and correctly dealing with whitespace around
header names we store fewer entries with better adherance to allowed
values in http responses.
 2014-03-09 16:22:28 +00:00
Vincent Sanders
25ce52ee64 only try and cache http and https urls 2014-03-09 16:20:53 +00:00
Vincent Sanders
8d883978c6 remove unnecessary caching of whether a url has a query element 2014-03-09 16:20:53 +00:00
Vincent Sanders
20b3c40816 make nsurl_defragment() API more obvious and remove duplicated parameter checking 2014-03-09 16:20:53 +00:00
Vincent Sanders
60973fe5d1 Make llcache debugging less invasive 2014-02-21 11:43:48 +00:00
Vincent Sanders
ff8c37a6ee Witespace cleanup to prepare for backing store addition 2014-02-21 11:31:39 +00:00
Michael Drake
2075ad7ff4 Use corestrings. 2014-01-24 20:19:20 +00:00
Michael Drake
ba9769bc8c Pass fetch redirect info up to content layer as content_msg. Mark redirect origin URLs as visited in browser window content callback. Note this doesn't mean we track redirects, it just lets us get the :visited link style on links that redirect. 2013-05-27 14:51:00 +01:00
Vincent Sanders
848f3d13ca ensure parameter contract is met 2013-05-04 08:52:12 +01:00
Vincent Sanders
82e2fc57d7 free memory on error path 2013-05-01 22:15:24 +01:00
Vincent Sanders
87ee963ff2 Initial changes ready to improve caching decisions 2013-01-07 18:45:29 +00:00
John-Mark Bell
3dff750ae2 Downgrade TLS version support if it turns out the server can't cope with TLSv1.1. 2013-01-04 23:13:23 +00:00
Michael Drake
af89fdb38a Fixup for nsurl urldb. 2012-10-09 11:20:21 +01:00
Vincent Sanders
6f86862edb cleanup and reorganise low level cache source ready for extending with disc cache 
svn path=/trunk/netsurf/; revision=13893
 2012-04-28 13:24:09 +00:00
John Mark Bell
c22038bed8 Fix bogus cache validation decision 
svn path=/trunk/netsurf/; revision=13261
 2011-12-08 22:44:19 +00:00
Michael Drake
714f07c48b Rename nsurl_enquire to nsurl_has_component. 
svn path=/trunk/netsurf/; revision=13183
 2011-11-27 18:32:06 +00:00
John Mark Bell
dcbafe6b87 Clean up fetch callback API. Inject some long-needed type safety. 
svn path=/trunk/netsurf/; revision=13137
 2011-11-08 21:51:42 +00:00
Michael Drake
17525fbf25 Avoid making a new nsurl to remove fragment if there is no fragment. 
svn path=/trunk/netsurf/; revision=12958
 2011-10-05 18:05:41 +00:00
Michael Drake
57547f7b1e Make llcache_handle_get_url return a nsurl. 
svn path=/trunk/netsurf/; revision=12905
 2011-09-28 14:21:35 +00:00
Michael Drake
861f88aa43 Port llcache to nsurl. 
svn path=/trunk/netsurf/; revision=12904
 2011-09-28 11:26:30 +00:00
Michael Drake
e1f7a37f15 Port fetch layer to nsurl. Remove unused fetch_get_referer function. 
svn path=/trunk/netsurf/; revision=12899
 2011-09-27 11:07:32 +00:00
Vincent Sanders
2d33a8f85a Make high level cache, low level cache and image cache all be initialised from passed parameters 
Calculate all cache sizes from single memory cache size option and sanity check
have a single global struct to hold all parameters instead of several individual variables

svn path=/trunk/netsurf/; revision=12784
 2011-09-10 00:55:39 +00:00
John Mark Bell
f6ac059a05 Handle zero-length documents correctly 
svn path=/trunk/netsurf/; revision=12778
 2011-09-08 08:26:16 +00:00
John Mark Bell
6807fa854d Sniff content types where appropriate. 
We never sniff for CSS, nor for non-page artefacts (e.g. treeview icons)

svn path=/trunk/netsurf/; revision=12707
 2011-09-04 06:28:09 +00:00
John Mark Bell
da7c3c10bc Implement todo 
svn path=/trunk/netsurf/; revision=12418
 2011-05-16 21:47:36 +00:00
John Mark Bell
35cd1a79f7 Squash static analyser warning 
svn path=/trunk/netsurf/; revision=12416
 2011-05-16 21:42:53 +00:00
Daniel Silverstone
93e9bfe323 Shunt the schedule function definitions to desktop/schedule.h. Shunt the hlcache/llcache to using schedule to get their cleanups run. 
svn path=/trunk/netsurf/; revision=12029
 2011-03-13 18:26:46 +00:00
John Mark Bell
60b5039c33 Protect against object being cleaned while waiting for a query response 
svn path=/trunk/netsurf/; revision=11894
 2011-03-03 00:40:50 +00:00
John Mark Bell
d25a88145b Make no-cache a tristate. In the case of a forced reload, we want to revalidate once only. 
svn path=/trunk/netsurf/; revision=11791
 2011-02-24 21:22:24 +00:00
John Mark Bell
00bf0fbd24 Revert the functional change from r11789: the no-cache directive forces us to revalidate a cached object, not reject it entirely. 
svn path=/trunk/netsurf/; revision=11790
 2011-02-24 21:06:45 +00:00
John Mark Bell
4f740f5a32 Ensure cache hits after a forced reload -- don't even attempt to reuse objects with the no_cache flag set (as they'll never be considered fresh, thus requiring a round trip to the server every time they're used). Instead, we'll fetch a new object, which will be considered fresh for caching purposes. 
svn path=/trunk/netsurf/; revision=11789
 2011-02-24 20:51:35 +00:00
John Mark Bell
6b26f77bb1 Use cache control data invalidation function everywhere we want the cache control data reset. 
Fix cache control data invalidation to reset the age and max_age fields to INVALID_AGE.
Fix cache control data reset on encountering an HTTP response line to occur before we fill in the response time, rather than after it.

svn path=/trunk/netsurf/; revision=11783
 2011-02-24 10:14:50 +00:00
John Mark Bell
b967824575 Improve commentary. 
Stop leaking object_scheme in error cases.

svn path=/trunk/netsurf/; revision=11778
 2011-02-23 23:05:36 +00:00
Vincent Sanders
0231675abf add resource handling 
move gtk and framebuffer to use generic resource handling

svn path=/trunk/netsurf/; revision=11772
 2011-02-23 22:27:55 +00:00
John Mark Bell
d8d0353a73 Fix bug #3184972: cope with server sending a 304 in response to an unconditional request. 
svn path=/trunk/netsurf/; revision=11710
 2011-02-17 17:50:14 +00:00
John Mark Bell
3fcb02afa1 Revert r10650/1. We no longer require noisy fetch/cache logging. 
svn path=/trunk/netsurf/; revision=11223
 2011-01-06 00:31:54 +00:00
Michael Drake
0eb65d3344 Add some assert()s. 
svn path=/trunk/netsurf/; revision=11144
 2010-12-29 20:42:00 +00:00
John Mark Bell
bcfb508f86 User may have no handle if it was the target of an iterator when the handle was aborted 
svn path=/trunk/netsurf/; revision=11071
 2010-12-16 00:18:41 +00:00
John Mark Bell
c675c3552d Ensure that we clean up after the client even if they report an error from their notification callback. 
svn path=/trunk/netsurf/; revision=11067
 2010-12-15 22:59:19 +00:00
John Mark Bell
d745dae903 Make llcache_object_user objects distinct from llcache_handles. 
This allows us to be robust against clients completely changing the contents of the user list underneath us during iteration.
Fix llcache_send_event_to_users to also be robust against the user list changing under it.

svn path=/trunk/netsurf/; revision=11063
 2010-12-15 21:15:49 +00:00
John Mark Bell
67556f54fe Always invalidate next/prev pointers 
svn path=/trunk/netsurf/; revision=11052
 2010-12-13 08:36:27 +00:00
John Mark Bell
c0f49a2785 Revert r11047-11050: if we remove a user from an object while it's the iterator target, the effect will be that the iteration terminates early. 
Subsequent users that require notifications will simply have their notifications delayed

svn path=/trunk/netsurf/; revision=11051
 2010-12-13 08:34:01 +00:00
John Mark Bell
f919b689ba Take 2: make callers of llcache_object_user_destroy invalidate next/prev pointers 
svn path=/trunk/netsurf/; revision=11050
 2010-12-13 08:28:22 +00:00
John Mark Bell
a5221248c2 Revert last change 
svn path=/trunk/netsurf/; revision=11049
 2010-12-13 08:25:54 +00:00
John Mark Bell
991472d264 Remove bogus assertions 
svn path=/trunk/netsurf/; revision=11048
 2010-12-13 08:24:04 +00:00
John Mark Bell
f40da966ad We cannot invalidate a user's next/prev pointers when removing it from an object, as it may be the target of iteration 
svn path=/trunk/netsurf/; revision=11047
 2010-12-13 08:16:32 +00:00
John Mark Bell
702c94bb30 Fix assertion: user->next may be NULL, so there's no point asserting that it isn't 
svn path=/trunk/netsurf/; revision=11046
 2010-12-13 08:12:31 +00:00
John Mark Bell
d9f8b8949e Fix assertion to compare, and not assign 
svn path=/trunk/netsurf/; revision=11045
 2010-12-13 08:10:29 +00:00
John Mark Bell
650f884527 Make llcache_object_notify_users robust to the object's user list changing underneath it 
svn path=/trunk/netsurf/; revision=11044
 2010-12-12 23:37:21 +00:00
John Mark Bell
e5bc525030 Fix bug 3096811: discard HTTP headers for all but the last response 
svn path=/trunk/netsurf/; revision=10912
 2010-10-27 22:12:00 +00:00
John Mark Bell
a10c4d3a7b Fix crash on receipt of a 304 response: it turns out that FETCH_HEADER may be called before any decision about how to process the response is made. 
Move FETCH_HEADER handler to start of switch statement to reflect this.

svn path=/trunk/netsurf/; revision=10792
 2010-09-16 22:46:40 +00:00
John Mark Bell
a7dd752046 Ensure that candidate count is reduced if a conditional request results in a non-304 response 
svn path=/trunk/netsurf/; revision=10791
 2010-09-16 22:04:06 +00:00
Vincent Sanders
1391ebe5cc perpetrate a neatness to centralise all the low level cache cache control data invalidation 
svn path=/trunk/netsurf/; revision=10787
 2010-09-16 19:47:15 +00:00
John Mark Bell
db736f8378 Fix heap corruption 
svn path=/trunk/netsurf/; revision=10786
 2010-09-16 19:33:18 +00:00
John Mark Bell
dafccf21d5 Fix validation of invalidated cache entries. 
svn path=/trunk/netsurf/; revision=10785
 2010-09-16 18:58:32 +00:00
Daniel Silverstone
be59d5870b Ensure we free etags before invalidating cache control data 
svn path=/trunk/netsurf/; revision=10784
 2010-09-16 16:21:58 +00:00
John Mark Bell
c88044c8cb Forcibly invalidate contents with response codes that aren't 200 or 203. 
svn path=/trunk/netsurf/; revision=10780
 2010-09-15 19:33:18 +00:00
John Mark Bell
78f3bbaed3 Fix bug #3055480: properly compare URLs, instead of erroneously assuming that strcasecmp is the correct answer. 
svn path=/trunk/netsurf/; revision=10721
 2010-08-29 17:41:54 +00:00
John Mark Bell
00aeb20334 Less spam 
svn path=/trunk/netsurf/; revision=10687
 2010-08-10 21:46:45 +00:00
John Mark Bell
48b17a5aea Rate-limit cache clean attempts 
svn path=/trunk/netsurf/; revision=10686
 2010-08-10 19:58:39 +00:00
John Mark Bell
183974b437 Enable noisy logging 
svn path=/trunk/netsurf/; revision=10650
 2010-07-17 08:17:42 +00:00
John Mark Bell
0de2013f25 Clean up caches on exit 
svn path=/trunk/netsurf/; revision=10494
 2010-04-27 06:48:09 +00:00
Vincent Sanders
03bb3e8636 remove _GNU_SOURCE define from everywhere. 
strndup is not standard so provide an implementation.

svn path=/trunk/netsurf/; revision=10474
 2010-04-24 15:22:17 +00:00
John Mark Bell
248a53f6ba Avoid double free when shrink-wrapping source data on fetch completion. 
svn path=/trunk/netsurf/; revision=10423
 2010-04-17 21:17:08 +00:00
John Mark Bell
01eb197f56 When the fetch system informs its clients that certificate verification has failed, it has already stopped and cleaned up the fetch. 
Stop llcache_fetch_cert_error attempting to abort the fetch, as this will result in a double free.

svn path=/trunk/netsurf/; revision=10418
 2010-04-16 23:15:22 +00:00