mirrors/netsurf
1
0
Fork 0
mirror of https://github.com/netsurf-browser/netsurf synced 2024-12-25 13:37:02 +03:00
Code Issues Packages Projects Releases Wiki Activity
12,708 Commits 6 Branches 12 Tags 33 MiB
51062e55b0
Commit Graph

1050 Commits

Author SHA1 Message Date
Vincent Sanders
2aef095f27 Ensure bandwidth minimum check is only performed when enough data has 
been written.
 2015-04-30 14:28:18 +01:00
Vincent Sanders
05538c0264 Ensure small block cache files allocate their entire extent at open. 
It seems many filesystems are greatly more efficient if the block file
is allocated its entire extent once rather than trying to
continuously grown the file later.

The size of the block files is known at their creation time so this
change ensures they are grown to the full possible extent hence removing
future inefficient writes.
 2015-04-30 13:49:25 +01:00
Vincent Sanders
ea2e1c4d98 Add scaled content redraw interface. 
Add a new interface to the content to allow automaticaly scaled
content redraws. This is intended to replace the thumbnail_redraw
interface with something more generic.
 2015-04-23 15:47:28 +01:00
Vincent Sanders
f37e52c394 Move bitmap operations into an operation table. 
The generic bitmap handlers provided by each frontend are called back
from the core and therefore should be in an operation table. This was
one of the very few remaining interfaces stopping the core code from
being split into a library.
 2015-04-13 23:19:04 +01:00
Vincent Sanders
ea9a2ff01d Fix up some doxygen errors 2015-04-03 00:28:22 +01:00
Vincent Sanders
77ffda1e46 Use nsutils unistd operations for pread and pwrite to get consistant interface 2015-04-02 17:28:55 +01:00
Vincent Sanders
47d08b6506 Revert "Fix RISC OS not having a pread/pwrite implementation." 
This reverts commit 82c7a7a4ba.

Conflicts:
	content/fs_backing_store.c
 2015-04-02 16:37:04 +01:00
Vincent Sanders
576b1c55bf extend backing store logging to show errno values ofter operations 2015-04-01 09:26:39 +01:00
Vincent Sanders
82c7a7a4ba Fix RISC OS not having a pread/pwrite implementation. 2015-03-31 22:41:37 +01:00
Vincent Sanders
87a38ca5c1 ensure entry invalidation copes with entries in blocks 2015-03-31 22:02:07 +01:00
Vincent Sanders
9511e9c4e9 Improve file naming inside cache 2015-03-31 20:57:00 +01:00
Vincent Sanders
f1d4094566 allocate small block using block use bitmaps to find free entries. 2015-03-31 20:56:55 +01:00
Vincent Sanders
706a77170d implement block read and write operations. 2015-03-31 20:56:55 +01:00
Vincent Sanders
c965fa8739 Block file usage bitmaps (de)serialising 2015-03-31 20:56:55 +01:00
Vincent Sanders
a8e501a663 define block store parameters 2015-03-31 20:56:55 +01:00
Vincent Sanders
980e28d916 Change backing store fetch operation to always perform the allocations 
The fetch API previously allowed for the caller to supply the storage,
this was never used and was preventing the refactoring necessary for
small black storage to be available.
 2015-03-31 20:56:55 +01:00
Vincent Sanders
1d03f34d62 Compute the element index once and use that as a parameter. 
Change to computing the element index from the flags passed to store
and fetch methods instead of passing the flags around and calculating
everywhere.

Additionally split out writing element of entry to file into distinct
function to make code clearer.
 2015-03-31 20:56:55 +01:00
Vincent Sanders
c2eda75710 Change element layout and cope with index collisions. 2015-03-31 20:56:55 +01:00
Vincent Sanders
8a99b045bc Remove url from content thumbnailers API 
The content thumbnailers for each frontend were being provided the
contents url. This was only ever used to call the urldb thumbnail
setting API.

This changes it so the single callsite that passed a valid url adds
the bitmap to that url itself in desktop_history.c instead of forcing
every frontend to require the urldb API.

Additionally the old API could pass the url as NULL which was causing
asserts where this was not an expected parameter value. Because of
this this fixes bug #2286 which was also present in the monkey
frontend as both called nsurl_access() on the url without the NULL
check and caused an assertion.
 2015-03-15 00:00:45 +00:00
Vincent Sanders
a487f7e611 Move win32 window operations into their own module 
This splits up a great deal of the win32 window code out from other
gui code. It also remove large quantities of unused and junk
variables and functions.
 2015-03-12 23:48:53 +00:00
Michael Drake
ac636100e8 Remove include of nsurl from corestrings.h 2015-02-26 16:58:41 +00:00
Michael Drake
cbd9d710a3 Use forward declaration instead of #include for nsurl. 2015-02-26 16:14:23 +00:00
Michael Drake
ee75f5b3ad Fix to support non-POSIX behaviour when renaming cache entries on disc. 2015-02-03 10:22:28 +00:00
Vincent Sanders
9927232a2d Improve metadata deserialisation to only update object on success 
The low level cache deserialisation was leaving bad data in an low
level cache object in the error case. This fixes it so the object
state only gets modified on successful deserialisation of all the
metadata.
 2015-01-28 22:39:39 +00:00
Vincent Sanders
5464dc7778 Ensure no division by zero occours if no time has ever been spent writing llcache 2015-01-03 15:13:40 +00:00
Vincent Sanders
6b645664fe add missing includes 2014-12-27 22:48:37 +00:00
Vincent Sanders
45f98ffd6e improve comments in low level cache and remove completed todo 2014-12-04 21:09:42 +00:00
Vincent Sanders
5bf4b3ff2b move the low bandwidth check to allow for an average over a much longer time 2014-12-03 20:13:10 +00:00
Vincent Sanders
57cd5c77b2 Use translated messgae for low bandwidth error 2014-12-02 10:03:27 +00:00
Vincent Sanders
c9ee49baa8 cope with backing store writeout making no progress 2014-12-01 11:32:33 +00:00
Vincent Sanders
ba5ade87b2 ensure the backing store write reports at least a minimal amount of time 2014-12-01 09:18:31 +00:00
Vincent Sanders
1c735140b6 fix resource leak on error (Coverity 1256245) 2014-11-30 11:21:34 +00:00
Vincent Sanders
61ab2777da Improve writeout rescheduling if list of objects is completed quickly 2014-11-30 00:27:05 +00:00
Vincent Sanders
d08acbc5ae correctly calculate writeout bandwidth and properly impose limits 2014-11-29 23:57:15 +00:00
Vincent Sanders
6b2676d33f Improve llcache writeout stratagy 2014-11-28 19:17:42 +00:00
Vincent Sanders
8d51dd4879 update the backing store to allow for invalidating entries which have an allocation 2014-11-27 13:31:43 +00:00
Vincent Sanders
79d5b1bf3b stop using gettimeofday and use the monotonic time interface fron nsutils 2014-11-26 21:29:13 +00:00
Vincent Sanders
81417ae4a5 caclulate the total amount of time spent writing an object out 
In order to calculate the writeout bandwidth we need to know how long
it took to write the data to peristant storage in addition to how much
was written.
 2014-11-25 23:11:41 +00:00
Vincent Sanders
5a78cd810f correctly release source data from llcache 2014-11-23 18:21:56 +00:00
Vincent Sanders
01b2bc5a57 Improve doxygen comments and add todo for cache and backing store 2014-11-23 15:43:03 +00:00
Vincent Sanders
246a1fec05 remove constness from backing stores store method 2014-11-23 01:37:24 +00:00
Vincent Sanders
63c35da66e annotate element struct with packing attributes to reduce entry size 2014-11-23 01:25:14 +00:00
Vincent Sanders
08c7ef6bed Do not use anonymous union as it is not portable 
unnamed unions are not portable to old compilers. When reloading data
ensure the heap allocation status is discarded.
 2014-11-23 00:35:11 +00:00
Vincent Sanders
8b810ee4a1 change the persistant data store to owning the allocations 2014-11-22 23:56:43 +00:00
Vincent Sanders
eb87406758 update entry points to backing store ready for allowing differing object lifetimes 2014-11-20 22:28:19 +00:00
Vincent Sanders
69f4397fe4 Limit creation of store path directories. 
Only create all elements of a path in the store when it is being
opened for creation instead of causing reads to create directories.
 2014-11-19 16:26:29 +00:00
Vincent Sanders
92a1ac4fbb Improve backing store control data serialisation 
By scheduling the control data to be maintained (entries index written
and headers updated) once activity occurs to update these control
structures rather than a single serialisation at browser exit the data
is more likely to be up to date and not lost on a crash.
 2014-11-19 16:22:06 +00:00
Vincent Sanders
a79b30bd6f make backing store use rename to atomicly replace entries file 2014-11-19 11:26:27 +00:00
Vincent Sanders
669df172ec Fix my spelling of teh everywhere 2014-11-18 17:16:26 +00:00
Vincent Sanders
d4c7a0ff77 Improve data scheme fetcher allocation 
The data scheme fetcher was over allocating the space for decoded
base64 encoded urls and not using the base64 API that allocated the
correct size storage.
 2014-11-17 16:05:17 +00:00
Vincent Sanders
71d016aba5 Remove unecessary form debug 
Additionaly we make the curl fetcher report errors instead of
uninitialised buffer.
 2014-11-15 18:23:44 +00:00
Vincent Sanders
d1f2d29e10 Doxygen cleanups and documentation additions 2014-11-10 17:00:15 +00:00
Vincent Sanders
e32a2ad46e Doxygen fixes 2014-11-09 15:28:03 +00:00
Vincent Sanders
56bb1b03c5 Doxygen cleanups 2014-11-09 12:50:30 +00:00
Vincent Sanders
5562c9a553 Continue doxygen error cleanup. 2014-11-08 22:08:29 +00:00
Vincent Sanders
05a64bfde9 fix up more doxygen errors 2014-11-08 16:49:32 +00:00
Vincent Sanders
8f64d5d223 Improve Doxygen documentation 2014-11-08 12:35:11 +00:00
Vincent Sanders
3405803280 Improve content encoding information API 
Extend the content_get_encoding() API to retrieve the source of
the encoding as well as the actual encoding.
 2014-11-07 12:33:34 +00:00
Vincent Sanders
8c2cfecfb5 Allow content handlers to have debug values set through API 
Previously content handler debugging features were accessed by global
variables. This allows the setting of debugging parameters via a
content API giving per content control over debugging features.
Currently only used by the html content handler to toggle global
redraw debugging.
 2014-11-06 22:51:46 +00:00
Vincent Sanders
8ec7ad053a Make the fetching of a contents encoding generic. 
The frontends previously had to use an html renderer API to get the
encoding of a content. This also required the explicit checking of the
contents type rather than using the existing content API to abstract
this knowledge.
 2014-11-05 23:44:31 +00:00
Vincent Sanders
bdfedb686b replace save_link operation table entry usage of textural url with nsurl 2014-11-03 23:05:59 +00:00
Vincent Sanders
c31c4babe1 Change contextual content retrieval to browser features. 
Update the API which allows frontends to acquire the page features
(images, link urls or form elements) present at the given coordinates
within a browser window.

By making this an explicit browser_window API and using the browser.h
header for the associated data structure with a more appropriate API
naming the usage is much more obvious and contained.

Additionally the link url is now passed around as a nsurl stopping it
being converted from nsurl to text and back again several times.
 2014-11-02 15:46:42 +00:00
Vincent Sanders
cf7abb4a0a restructure urldb source 
remove forward declarations and restructure. exported functions are
also now documented in the urldb.h header.
 2014-11-01 23:27:29 +00:00
Michael Drake
98b6d02df6 Move host_is_ip_address into urldb, as that's the only thing that cares. 2014-10-31 14:31:06 +00:00
Vincent Sanders
14e2829489 remove the die API from the core. 
The die() API for abnormal termination does not belong within the core
of netsurf and instead errors are propogated back to the callers.

This is the final part of this change and the API is now only used within
some parts of the frontends
 2014-10-26 12:42:53 +00:00
Vincent Sanders
53f0f432dd update fetcher registration API to return an error code. 
By using an error code return we can gracefully handle fetcher
registration faliures instead of just immediately aborting.

The curl handler was also cleaned up and documentation improved
as a side effect.
 2014-10-25 17:15:23 +01:00
Michael Drake
b27f3e5ac4 Open select menu via content msg, instead of breaking encapsulation. 
Fixes bw deref and browser_private.h #include in render/
 2014-10-24 17:57:15 +01:00
Michael Drake
6b846c363d Add trivial maps facilitator at about:maps 
This is a simple HTML front end for the Google Static Maps API, since
we don't yet have enough JS support for the full Google Maps.

See:
  https://developers.google.com/maps/documentation/staticmaps/
 2014-10-22 20:42:21 +01:00
Vincent Sanders
d06df231bc Update the core to use the split operations table headers 
Second in the series to decouple the core API usage from the frontends.
 2014-10-16 22:33:43 +01:00
John-Mark Bell
b2242c57e1 HTTPS: disable all SSL versions; emit fallback SCSV on downgrade. 
This removes all support for SSL and (with help from servers that
support the fallback SCSV) protects against inappropriate protocol
downgrade.
 2014-10-15 12:02:25 +01:00
Vincent Sanders
02bbaa3e88 split version info into its own header and remove unnecessary nesurf.h includes 
The netsurf.h header should *only* contain the registration, core
initialisation and finalisation methods. Version information is best
placed in its own header.

Also remove any unneeded inclusion of this header limiting it to
solely the places the relevant API is required.
 2014-10-13 15:08:16 +01:00
Vincent Sanders
1a2040bc72 Trivial sepelling, documentation and whitespace cleanups 2014-10-10 13:01:20 +01:00
Vincent Sanders
b99ba0998b fixup haiku rsrc fetcher initialisation 2014-09-05 00:40:52 +01:00
Vincent Sanders
c695d3d007 memcpy and terminate from openssl buffers 
The memcpy is used instead of snprintf as the source data may not be null terminated and was
 causing OpenBSD to segfault.
 2014-09-03 14:27:25 +01:00
Vincent Sanders
5492e96793 terminate buffers from X509_NAME_print_ex() 2014-09-02 23:30:37 +01:00
Vincent Sanders
37755fb135 add incomplete scheme fetcher for javascript urls 2014-08-25 15:59:52 -07:00
Vincent Sanders
af16c38d2d fix DOM tree dump debug 2014-08-04 00:37:21 +01:00
Vincent Sanders
db01637343 remove content include from urldb header 2014-07-17 00:18:44 +01:00
Vincent Sanders
0694f345da fix leak on error path (coverity 1224429) 2014-07-08 15:57:48 +01:00
Vincent Sanders
219f505a65 version 1.1 of the backing store disc layout using base32 encoded 
filenames to allow for case insensitive filenames.
 2014-07-04 17:09:28 +01:00
Vincent Sanders
a71865b328 turn off verbose fetch debugging 2014-07-02 17:07:29 +01:00
Vincent Sanders
29e29cccd7 fix dumb error in previous commit and get scheme name element correct 2014-07-02 15:01:24 +01:00
Vincent Sanders
516dcb96e1 fix unhelpful assert at quit 2014-07-02 14:51:11 +01:00
Vincent Sanders
06eb81235d fix fetchers include on windows 2014-07-01 09:17:22 +01:00
Vincent Sanders
8944edd649 convert all frontends to scheduled fetch operation 2014-06-26 19:05:59 +01:00
Vincent Sanders
1b7aa7ffe5 make fetchers scheduled 2014-06-26 19:05:59 +01:00
Vincent Sanders
4b2101ba6a clean up the fetcher factory and improve its API 2014-06-19 18:27:24 +01:00
Vincent Sanders
f1c2dde13b extend file table with mkdir all and make fs backing store use it. 
enable fs backing store for RISC OS.
 2014-06-05 12:06:47 +01:00
Rob Kendrick (fatigue)
7b1332c5c5 Include the date in the testament 2014-06-04 12:10:42 +01:00
Michael Drake
bd804d3fe4 Avoid signed vs unsigned comparason warning. 2014-06-04 07:48:05 +01:00
Vincent Sanders
5899f6c16e turn off debugging and add some comments and todo from dsilvers 2014-06-03 22:30:04 +01:00
Daniel Silverstone
b5461c8313 Adjust the llcache behaviour to use scheduler for user notification. 
This change updates the llcache to use the scheduler to notify users of the
llcache of events.  This should be just as safe as before and is part of an
effort to remove hlcache_poll and llcache_poll eventually because fetchers
should schedule themselves if need-be.

This is a big change despite the diminutive nature of the patch.  Please report
issues promptly if they turn up after this and are not visible before it.

Signed-off-by: Daniel Silverstone <dsilvers@netsurf-browser.org>
Reviewed-by: Vincent Sanders <vince@netsurf-browser.org>
 2014-06-03 17:00:23 +01:00
Vincent Sanders
e9d53022a4 write a cache tag file as an aid to backup software 2014-05-30 00:19:20 +01:00
Vincent Sanders
644fc616b3 attempt to purge low level cache on out of memory during fetch 2014-05-29 14:00:13 +01:00
Vincent Sanders
00b6cfc57e rework path to url mapping functions to convert from and to nsurl 2014-05-26 23:52:36 +01:00
Vincent Sanders
a6d3ceae0e Completely re-write web search provider handling 2014-05-25 01:01:59 +01:00
Michael Drake
fb5af8a1b6 Check return value from write() to avoid build warnings. 2014-05-24 15:19:01 +01:00
Michael Drake
7ba459f1b5 Fix leak of url schemes. 2014-05-16 18:58:06 +01:00
Michael Drake
e4ba144fe0 Remove unused #include 
Persistent thumbnail code was ripped out of urldb ages ago.
 2014-05-15 21:47:40 +01:00
Vincent Sanders
ef00272e2f add helpers for time_t reading/writing 2014-05-15 01:18:52 +01:00
Vincent Sanders
1a3ee6090d add strptime compatability 2014-05-13 16:37:49 +01:00
Vincent Sanders
05932293f6 Add filesystem based backing store 2014-05-13 15:53:02 +01:00
Vincent Sanders
4a49ff5266 Extend low level source data cache with persistant storage 2014-05-13 15:52:58 +01:00
Vincent Sanders
657abbd245 low level source data cache backing store interface. 2014-05-12 23:47:43 +01:00
Vincent Sanders
2326d3c503 remove unecessary utils/url.h includes 2014-05-09 10:15:42 +01:00
Vincent Sanders
c56642819e add file operations table and make all frontends use it. 
This rationalises the path construction and basename file
operations. The default implementation is POSIX which works for all
frontends except windows, riscos and amiga which have differeing path
separators and rules.

These implementations are significantly more robust than the previous
nine implementations and also do not use unsafe strncpy or buffers
with arbitrary length limits.

These implementations also carry full documentation comments.
 2014-05-07 16:24:51 +01:00
Vincent Sanders
b7b3496a78 move testament output to object directory instead of modifying source 2014-04-08 11:41:58 +01:00
Vincent Sanders
4d4d74c8cd move page search gui callbacks to their own operations table 2014-03-18 22:32:52 +00:00
Vincent Sanders
da0b969f25 Improve llcache header processing 
By skipping empty headers and correctly dealing with whitespace around
header names we store fewer entries with better adherance to allowed
values in http responses.
 2014-03-09 16:22:28 +00:00
Vincent Sanders
25ce52ee64 only try and cache http and https urls 2014-03-09 16:20:53 +00:00
Vincent Sanders
8d883978c6 remove unnecessary caching of whether a url has a query element 2014-03-09 16:20:53 +00:00
Vincent Sanders
20b3c40816 make nsurl_defragment() API more obvious and remove duplicated parameter checking 2014-03-09 16:20:53 +00:00
Vincent Sanders
87f6314dab move scheduleing into browser operation table 2014-03-09 15:37:40 +00:00
Vincent Sanders
60973fe5d1 Make llcache debugging less invasive 2014-02-21 11:43:48 +00:00
Vincent Sanders
ff8c37a6ee Witespace cleanup to prepare for backing store addition 2014-02-21 11:31:39 +00:00
John-Mark Bell
823aad5ddf Use libcurl's cache if it's new enough. Fixes #2064. 2014-02-06 01:19:22 +00:00
Vincent Sanders
fba58ad4b0 remove unecessary include 2014-02-04 15:19:46 +00:00
Vincent Sanders
7b380d0982 fix sign extension issue on 64bit platforms (coverity 1109898) 2014-02-04 13:24:25 +00:00
Vincent Sanders
db33103fa3 clean up desktop/gui.h include usage 2014-01-29 21:35:53 +00:00
Vincent Sanders
8ce0a10670 move path_to_url and url_to_path to fetch operation table 2014-01-25 23:01:32 +00:00
Michael Drake
2075ad7ff4 Use corestrings. 2014-01-24 20:19:20 +00:00
Michael Drake
14238615a4 Use corestrings for "about", "data", and "resource". 2014-01-24 20:08:41 +00:00
Michael Drake
e32dc31b04 Use corestring ref. 2014-01-24 19:58:51 +00:00
Michael Drake
178f3945b8 Just use corestring refs. 2014-01-24 19:31:50 +00:00
Michael Drake
c4e2fff5a4 Use corestrings for fetch module's lwc strings. 2014-01-24 19:25:07 +00:00
Vincent Sanders
0475693394 create table for fetcher operations and move all operations into it 2014-01-23 00:00:18 +00:00
John-Mark Bell
4a014b61be Enable TLS 1.2 support again. 2014-01-19 22:57:16 +00:00
Vincent Sanders
4987a3a8a7 remove forward refs from content/fetch.c and cleanup doc comments 2014-01-19 21:18:47 +00:00
Vincent Sanders
977ae8efc6 remove all forward declarations from hlcache.c 2014-01-19 21:18:47 +00:00
Vincent Sanders
38cb39339a move filename_from_path and path_add_part into gui operation tables 2014-01-16 00:01:25 +00:00
Vincent Sanders
bd065d4a43 split browser gui operations up 2014-01-15 19:37:05 +00:00
Vincent Sanders
0075eab194 move remaining gui operations to table 2014-01-14 23:31:54 +00:00
Vincent Sanders
303a29a89c cleanups to fix new clang warnings 2014-01-10 15:55:29 +00:00
Daniel Silverstone
9a9cb399f9 Merge branch 'rjek/click-file-gadget' 2014-01-05 17:03:17 +00:00
Rob Kendrick
0d7f1cfc93 Add infrastructure for calling front ends to set file gadget filenames via clicking in addition to drag-and-drop 2014-01-05 16:04:35 +00:00
Daniel Silverstone
b4f9860848 Remove logging now that file upload *seems* to work 2014-01-04 22:05:23 +00:00
Daniel Silverstone
33f94c265e Support cloning rawfile data 2014-01-04 21:49:03 +00:00
Daniel Silverstone
de42f8880e Add a bunch of logging to try and help track down the issue with rawfile handling, sorry guys 2014-01-04 21:20:17 +00:00
Daniel Silverstone
581d877576 In theory, store raw filenames and pass them through for file upload. Untested due to no file-upload in GTK frontend just yet 2014-01-04 19:34:18 +00:00
Michael Drake
c56fb9259b Enable access to hotlist icons through resource URLs. 2013-11-20 22:03:07 +00:00
Michael Drake
c80229e256 Get referer host from referer url. 2013-11-18 11:49:51 +00:00
Michael Drake
c933b0aff9 Sort non zero-padded numerical filename parts correctly. 2013-11-14 21:01:51 +00:00
Michael Drake
5e4e32923d Can just use scandir now. 2013-11-14 19:02:35 +00:00
Michael Drake
7a920ace39 Marginally cleaner handling of scandir absence. 2013-11-11 11:38:48 +00:00
Michael Drake
74d430f44a Dirty special casing of Windows, to handle mingw dirent.h's lack of scandir and alphasort. 2013-11-11 10:58:55 +00:00
Michael Drake
8a72d5be84 Sort directory contents. 2013-11-10 22:02:12 +00:00
Michael Drake
61b5a5e98f Visual indication of hotlist's folder for unsorted entries, with special icon. 2013-11-08 18:53:34 +00:00
Michael Drake
0d03a342d9 Clean up cached fetch handles. 2013-10-24 18:12:18 +01:00
Michael Drake
2754918e25 Fix leak of path_query, when there is no host and scheme isn't file. 2013-10-23 18:55:00 +01:00
Michael Drake
f2aa7202ba Fix Coverity #1109872. 2013-10-23 16:25:48 +01:00