Commit Graph

62 Commits

Author SHA1 Message Date
Vincent Sanders
51dc59ecc9 remove unused constants 2020-03-26 23:38:50 +00:00
Daniel Silverstone
14286b381b
fs_backing_store: Remove cache on failure to init
If we fail to init the control file for reasons other than it
not being found, we blow away the cache in its entirety and then
try again.  We warn if the removal fails, but carry on regardless
since right now the worst that'll happen is that we'll end up
with more on disk than we know about in the cache.

Signed-off-by: Daniel Silverstone <dsilvers@digital-scurf.org>
2020-02-24 21:01:58 +00:00
Daniel Silverstone
02c5bde5a9
fs_backing_store: Fix some comparison types
Signed-off-by: Daniel Silverstone <dsilvers@digital-scurf.org>
2020-02-24 17:43:14 +00:00
Daniel Silverstone
fa75e8e4fd
fs_backing_store: Use correct format specifier for 64bit int
Signed-off-by: Daniel Silverstone <dsilvers@digital-scurf.org>
2020-02-24 17:19:45 +00:00
Daniel Silverstone
5c205fbff0
fs_backing_store: Fix up comments to be in-line with values
Signed-off-by: Daniel Silverstone <dsilvers@digital-scurf.org>
2020-02-24 15:56:55 +00:00
Daniel Silverstone
133e00edbe
fs_backing_store: 8K metadata blocks, 1K blocks, bump version
Signed-off-by: Daniel Silverstone <dsilvers@digital-scurf.org>
2020-02-24 15:37:12 +00:00
Michael Drake
63dd59bd2e Cache backing store: Add missing calloc check.
Found by coverity:

CID 1419831:  Null pointer dereferences  (NULL_RETURNS)
>>>     Dereferencing a pointer that might be "NULL" "url" when calling "read".
2020-02-24 14:34:37 +00:00
Daniel Silverstone
360997c265
fs_backing_store: Clear entry memory flags on read
Signed-off-by: Daniel Silverstone <dsilvers@digital-scurf.org>
2020-02-23 23:04:49 +00:00
Daniel Silverstone
859972df71
llcache: Rework fs_backing_store to use hashmap
As a result, we no longer waste a bunch of RAM on the entries
tables.  This ought to be no slower, and more memory efficient.

Signed-off-by: Daniel Silverstone <dsilvers@digital-scurf.org>
2020-02-23 20:59:40 +00:00
Michael Drake
2290c208ba Disc cache: Squash armhf warnings. 2018-05-08 12:26:04 +01:00
John-Mark Bell
17ae38771d Backing store: clean up resources properly.
Patch from Felix S. Fixes #2579.
2018-01-20 16:18:51 +00:00
Vincent Sanders
5d6f189d8b Fixup everything the semantic patch missed 2017-09-07 18:12:09 +01:00
Vincent Sanders
75018632a9 Use coccinelle to change logging macro calls in c files
for F in $(git ls-files '*.c');do spatch --sp-file foo.cocci --in-place ${F};done

@@ expression E; @@
-LOG(E);
+NSLOG(netsurf, INFO, E);
@@ expression E, E1; @@
-LOG(E, E1);
+NSLOG(netsurf, INFO, E, E1);
@@ expression E, E1, E2; @@
-LOG(E, E1, E2);
+NSLOG(netsurf, INFO, E, E1, E2);
@@ expression E, E1, E2, E3; @@
-LOG(E, E1, E2, E3);
+NSLOG(netsurf, INFO, E, E1, E2, E3);
@@ expression E, E1, E2, E3, E4; @@
-LOG(E, E1, E2, E3, E4);
+NSLOG(netsurf, INFO, E, E1, E2, E3, E4);
@@ expression E, E1, E2, E3, E4, E5; @@
-LOG(E, E1, E2, E3, E4, E5);
+NSLOG(netsurf, INFO, E, E1, E2, E3, E4, E5);
@@ expression E, E1, E2, E3, E4, E5, E6; @@
-LOG(E, E1, E2, E3, E4, E5, E6);
+NSLOG(netsurf, INFO, E, E1, E2, E3, E4, E5, E6);
@@ expression E, E1, E2, E3, E4, E5, E6, E7; @@
-LOG(E, E1, E2, E3, E4, E5, E6, E7);
+NSLOG(netsurf, INFO, E, E1, E2, E3, E4, E5, E6, E7);
2017-09-06 18:45:27 +01:00
Vincent Sanders
3217c82cdb attempt to remove unintended sign extension warning in fs backing store 2017-03-05 13:02:47 +00:00
Vincent Sanders
e96bf01339 make backing store use inttypes format specifiers 2017-01-21 16:48:41 +00:00
Vincent Sanders
6075feb487 create netsurf inttypes header to have portable integer formatting macros 2017-01-21 14:20:55 +00:00
Vincent Sanders
8828bdc702 fix comedy spelling in fs backing store code 2016-11-20 15:22:04 +00:00
Vincent Sanders
fe7eb85614 move misc header into public API 2016-05-30 11:23:32 +01:00
Vincent Sanders
33c7df0c40 complete the rename of the gui browser table
When the operations tables were created the browser table was renamed
to miscellaneous except the actual rename patch was never applied,
this fixes that situation.
2016-04-16 23:50:21 +01:00
Vincent Sanders
0d811963c1 fixup all the remaining logging macro callsites
The semantic patch tool appears to have missed some difficult to
reason about callsites which have now been manually cleaned up
2015-05-28 17:06:18 +01:00
Vincent Sanders
c105738fa3 Change LOG() macro to be varadic
This changes the LOG macro to be varadic removing the need for all
callsites to have double bracketing and allows for future improvement
on how we use the logging macros.

The callsites were changed with coccinelle and the changes checked by
hand. Compile tested for several frontends but not all.

A formatting annotation has also been added which allows the compiler
to check the parameters and types passed to the logging.
2015-05-28 16:08:46 +01:00
Vincent Sanders
35751f3faa Move the setting of block file extents to background maintinance.
On some OS the ftruncate operation can take some time so move it to
occour in the background maintinance operations instead of when data
blocks are initialy opened. This should improve browsing responsiveness.
2015-05-04 09:54:03 +01:00
Vincent Sanders
05538c0264 Ensure small block cache files allocate their entire extent at open.
It seems many filesystems are greatly more efficient if the block file
is allocated its entire extent once rather than trying to
continuously grown the file later.

The size of the block files is known at their creation time so this
change ensures they are grown to the full possible extent hence removing
future inefficient writes.
2015-04-30 13:49:25 +01:00
Vincent Sanders
ea9a2ff01d Fix up some doxygen errors 2015-04-03 00:28:22 +01:00
Vincent Sanders
77ffda1e46 Use nsutils unistd operations for pread and pwrite to get consistant interface 2015-04-02 17:28:55 +01:00
Vincent Sanders
47d08b6506 Revert "Fix RISC OS not having a pread/pwrite implementation."
This reverts commit 82c7a7a4ba.

Conflicts:
	content/fs_backing_store.c
2015-04-02 16:37:04 +01:00
Vincent Sanders
576b1c55bf extend backing store logging to show errno values ofter operations 2015-04-01 09:26:39 +01:00
Vincent Sanders
82c7a7a4ba Fix RISC OS not having a pread/pwrite implementation. 2015-03-31 22:41:37 +01:00
Vincent Sanders
87a38ca5c1 ensure entry invalidation copes with entries in blocks 2015-03-31 22:02:07 +01:00
Vincent Sanders
9511e9c4e9 Improve file naming inside cache 2015-03-31 20:57:00 +01:00
Vincent Sanders
f1d4094566 allocate small block using block use bitmaps to find free entries. 2015-03-31 20:56:55 +01:00
Vincent Sanders
706a77170d implement block read and write operations. 2015-03-31 20:56:55 +01:00
Vincent Sanders
c965fa8739 Block file usage bitmaps (de)serialising 2015-03-31 20:56:55 +01:00
Vincent Sanders
a8e501a663 define block store parameters 2015-03-31 20:56:55 +01:00
Vincent Sanders
980e28d916 Change backing store fetch operation to always perform the allocations
The fetch API previously allowed for the caller to supply the storage,
this was never used and was preventing the refactoring necessary for
small black storage to be available.
2015-03-31 20:56:55 +01:00
Vincent Sanders
1d03f34d62 Compute the element index once and use that as a parameter.
Change to computing the element index from the flags passed to store
and fetch methods instead of passing the flags around and calculating
everywhere.

Additionally split out writing element of entry to file into distinct
function to make code clearer.
2015-03-31 20:56:55 +01:00
Vincent Sanders
c2eda75710 Change element layout and cope with index collisions. 2015-03-31 20:56:55 +01:00
Michael Drake
ee75f5b3ad Fix to support non-POSIX behaviour when renaming cache entries on disc. 2015-02-03 10:22:28 +00:00
Vincent Sanders
1c735140b6 fix resource leak on error (Coverity 1256245) 2014-11-30 11:21:34 +00:00
Vincent Sanders
d08acbc5ae correctly calculate writeout bandwidth and properly impose limits 2014-11-29 23:57:15 +00:00
Vincent Sanders
6b2676d33f Improve llcache writeout stratagy 2014-11-28 19:17:42 +00:00
Vincent Sanders
8d51dd4879 update the backing store to allow for invalidating entries which have an allocation 2014-11-27 13:31:43 +00:00
Vincent Sanders
5a78cd810f correctly release source data from llcache 2014-11-23 18:21:56 +00:00
Vincent Sanders
01b2bc5a57 Improve doxygen comments and add todo for cache and backing store 2014-11-23 15:43:03 +00:00
Vincent Sanders
246a1fec05 remove constness from backing stores store method 2014-11-23 01:37:24 +00:00
Vincent Sanders
63c35da66e annotate element struct with packing attributes to reduce entry size 2014-11-23 01:25:14 +00:00
Vincent Sanders
08c7ef6bed Do not use anonymous union as it is not portable
unnamed unions are not portable to old compilers. When reloading data
ensure the heap allocation status is discarded.
2014-11-23 00:35:11 +00:00
Vincent Sanders
8b810ee4a1 change the persistant data store to owning the allocations 2014-11-22 23:56:43 +00:00
Vincent Sanders
eb87406758 update entry points to backing store ready for allowing differing object lifetimes 2014-11-20 22:28:19 +00:00
Vincent Sanders
69f4397fe4 Limit creation of store path directories.
Only create all elements of a path in the store when it is being
opened for creation instead of causing reads to create directories.
2014-11-19 16:26:29 +00:00