Commit Graph

982 Commits

Author SHA1 Message Date
Vincent Sanders
8a99b045bc Remove url from content thumbnailers API
The content thumbnailers for each frontend were being provided the
contents url. This was only ever used to call the urldb thumbnail
setting API.

This changes it so the single callsite that passed a valid url adds
the bitmap to that url itself in desktop_history.c instead of forcing
every frontend to require the urldb API.

Additionally the old API could pass the url as NULL which was causing
asserts where this was not an expected parameter value. Because of
this this fixes bug #2286 which was also present in the monkey
frontend as both called nsurl_access() on the url without the NULL
check and caused an assertion.
2015-03-15 00:00:45 +00:00
Vincent Sanders
a487f7e611 Move win32 window operations into their own module
This splits up a great deal of the win32 window code out from other
gui code. It also remove large quantities of unused and junk
variables and functions.
2015-03-12 23:48:53 +00:00
Michael Drake
ac636100e8 Remove include of nsurl from corestrings.h 2015-02-26 16:58:41 +00:00
Michael Drake
cbd9d710a3 Use forward declaration instead of #include for nsurl. 2015-02-26 16:14:23 +00:00
Michael Drake
ee75f5b3ad Fix to support non-POSIX behaviour when renaming cache entries on disc. 2015-02-03 10:22:28 +00:00
Vincent Sanders
9927232a2d Improve metadata deserialisation to only update object on success
The low level cache deserialisation was leaving bad data in an low
level cache object in the error case. This fixes it so the object
state only gets modified on successful deserialisation of all the
metadata.
2015-01-28 22:39:39 +00:00
Vincent Sanders
5464dc7778 Ensure no division by zero occours if no time has ever been spent writing llcache 2015-01-03 15:13:40 +00:00
Vincent Sanders
6b645664fe add missing includes 2014-12-27 22:48:37 +00:00
Vincent Sanders
45f98ffd6e improve comments in low level cache and remove completed todo 2014-12-04 21:09:42 +00:00
Vincent Sanders
5bf4b3ff2b move the low bandwidth check to allow for an average over a much longer time 2014-12-03 20:13:10 +00:00
Vincent Sanders
57cd5c77b2 Use translated messgae for low bandwidth error 2014-12-02 10:03:27 +00:00
Vincent Sanders
c9ee49baa8 cope with backing store writeout making no progress 2014-12-01 11:32:33 +00:00
Vincent Sanders
ba5ade87b2 ensure the backing store write reports at least a minimal amount of time 2014-12-01 09:18:31 +00:00
Vincent Sanders
1c735140b6 fix resource leak on error (Coverity 1256245) 2014-11-30 11:21:34 +00:00
Vincent Sanders
61ab2777da Improve writeout rescheduling if list of objects is completed quickly 2014-11-30 00:27:05 +00:00
Vincent Sanders
d08acbc5ae correctly calculate writeout bandwidth and properly impose limits 2014-11-29 23:57:15 +00:00
Vincent Sanders
6b2676d33f Improve llcache writeout stratagy 2014-11-28 19:17:42 +00:00
Vincent Sanders
8d51dd4879 update the backing store to allow for invalidating entries which have an allocation 2014-11-27 13:31:43 +00:00
Vincent Sanders
79d5b1bf3b stop using gettimeofday and use the monotonic time interface fron nsutils 2014-11-26 21:29:13 +00:00
Vincent Sanders
81417ae4a5 caclulate the total amount of time spent writing an object out
In order to calculate the writeout bandwidth we need to know how long
it took to write the data to peristant storage in addition to how much
was written.
2014-11-25 23:11:41 +00:00
Vincent Sanders
5a78cd810f correctly release source data from llcache 2014-11-23 18:21:56 +00:00
Vincent Sanders
01b2bc5a57 Improve doxygen comments and add todo for cache and backing store 2014-11-23 15:43:03 +00:00
Vincent Sanders
246a1fec05 remove constness from backing stores store method 2014-11-23 01:37:24 +00:00
Vincent Sanders
63c35da66e annotate element struct with packing attributes to reduce entry size 2014-11-23 01:25:14 +00:00
Vincent Sanders
08c7ef6bed Do not use anonymous union as it is not portable
unnamed unions are not portable to old compilers. When reloading data
ensure the heap allocation status is discarded.
2014-11-23 00:35:11 +00:00
Vincent Sanders
8b810ee4a1 change the persistant data store to owning the allocations 2014-11-22 23:56:43 +00:00
Vincent Sanders
eb87406758 update entry points to backing store ready for allowing differing object lifetimes 2014-11-20 22:28:19 +00:00
Vincent Sanders
69f4397fe4 Limit creation of store path directories.
Only create all elements of a path in the store when it is being
opened for creation instead of causing reads to create directories.
2014-11-19 16:26:29 +00:00
Vincent Sanders
92a1ac4fbb Improve backing store control data serialisation
By scheduling the control data to be maintained (entries index written
and headers updated) once activity occurs to update these control
structures rather than a single serialisation at browser exit the data
is more likely to be up to date and not lost on a crash.
2014-11-19 16:22:06 +00:00
Vincent Sanders
a79b30bd6f make backing store use rename to atomicly replace entries file 2014-11-19 11:26:27 +00:00
Vincent Sanders
669df172ec Fix my spelling of teh everywhere 2014-11-18 17:16:26 +00:00
Vincent Sanders
d4c7a0ff77 Improve data scheme fetcher allocation
The data scheme fetcher was over allocating the space for decoded
base64 encoded urls and not using the base64 API that allocated the
correct size storage.
2014-11-17 16:05:17 +00:00
Vincent Sanders
71d016aba5 Remove unecessary form debug
Additionaly we make the curl fetcher report errors instead of
uninitialised buffer.
2014-11-15 18:23:44 +00:00
Vincent Sanders
d1f2d29e10 Doxygen cleanups and documentation additions 2014-11-10 17:00:15 +00:00
Vincent Sanders
e32a2ad46e Doxygen fixes 2014-11-09 15:28:03 +00:00
Vincent Sanders
56bb1b03c5 Doxygen cleanups 2014-11-09 12:50:30 +00:00
Vincent Sanders
5562c9a553 Continue doxygen error cleanup. 2014-11-08 22:08:29 +00:00
Vincent Sanders
05a64bfde9 fix up more doxygen errors 2014-11-08 16:49:32 +00:00
Vincent Sanders
8f64d5d223 Improve Doxygen documentation 2014-11-08 12:35:11 +00:00
Vincent Sanders
3405803280 Improve content encoding information API
Extend the content_get_encoding() API to retrieve the source of
the encoding as well as the actual encoding.
2014-11-07 12:33:34 +00:00
Vincent Sanders
8c2cfecfb5 Allow content handlers to have debug values set through API
Previously content handler debugging features were accessed by global
variables. This allows the setting of debugging parameters via a
content API giving per content control over debugging features.
Currently only used by the html content handler to toggle global
redraw debugging.
2014-11-06 22:51:46 +00:00
Vincent Sanders
8ec7ad053a Make the fetching of a contents encoding generic.
The frontends previously had to use an html renderer API to get the
encoding of a content. This also required the explicit checking of the
contents type rather than using the existing content API to abstract
this knowledge.
2014-11-05 23:44:31 +00:00
Vincent Sanders
bdfedb686b replace save_link operation table entry usage of textural url with nsurl 2014-11-03 23:05:59 +00:00
Vincent Sanders
c31c4babe1 Change contextual content retrieval to browser features.
Update the API which allows frontends to acquire the page features
(images, link urls or form elements) present at the given coordinates
within a browser window.

By making this an explicit browser_window API and using the browser.h
header for the associated data structure with a more appropriate API
naming the usage is much more obvious and contained.

Additionally the link url is now passed around as a nsurl stopping it
being converted from nsurl to text and back again several times.
2014-11-02 15:46:42 +00:00
Vincent Sanders
cf7abb4a0a restructure urldb source
remove forward declarations and restructure. exported functions are
also now documented in the urldb.h header.
2014-11-01 23:27:29 +00:00
Michael Drake
98b6d02df6 Move host_is_ip_address into urldb, as that's the only thing that cares. 2014-10-31 14:31:06 +00:00
Vincent Sanders
14e2829489 remove the die API from the core.
The die() API for abnormal termination does not belong within the core
of netsurf and instead errors are propogated back to the callers.

This is the final part of this change and the API is now only used within
some parts of the frontends
2014-10-26 12:42:53 +00:00
Vincent Sanders
53f0f432dd update fetcher registration API to return an error code.
By using an error code return we can gracefully handle fetcher
registration faliures instead of just immediately aborting.

The curl handler was also cleaned up and documentation improved
as a side effect.
2014-10-25 17:15:23 +01:00
Michael Drake
b27f3e5ac4 Open select menu via content msg, instead of breaking encapsulation.
Fixes bw deref and browser_private.h #include in render/
2014-10-24 17:57:15 +01:00
Michael Drake
6b846c363d Add trivial maps facilitator at about:maps
This is a simple HTML front end for the Google Static Maps API, since
we don't yet have enough JS support for the full Google Maps.

See:
  https://developers.google.com/maps/documentation/staticmaps/
2014-10-22 20:42:21 +01:00
Vincent Sanders
d06df231bc Update the core to use the split operations table headers
Second in the series to decouple the core API usage from the frontends.
2014-10-16 22:33:43 +01:00
John-Mark Bell
b2242c57e1 HTTPS: disable all SSL versions; emit fallback SCSV on downgrade.
This removes all support for SSL and (with help from servers that
support the fallback SCSV) protects against inappropriate protocol
downgrade.
2014-10-15 12:02:25 +01:00
Vincent Sanders
02bbaa3e88 split version info into its own header and remove unnecessary nesurf.h includes
The netsurf.h header should *only* contain the registration, core
initialisation and finalisation methods. Version information is best
placed in its own header.

Also remove any unneeded inclusion of this header limiting it to
solely the places the relevant API is required.
2014-10-13 15:08:16 +01:00
Vincent Sanders
1a2040bc72 Trivial sepelling, documentation and whitespace cleanups 2014-10-10 13:01:20 +01:00
Vincent Sanders
b99ba0998b fixup haiku rsrc fetcher initialisation 2014-09-05 00:40:52 +01:00
Vincent Sanders
c695d3d007 memcpy and terminate from openssl buffers
The memcpy is used instead of snprintf as the source data may not be null terminated and was
 causing OpenBSD to segfault.
2014-09-03 14:27:25 +01:00
Vincent Sanders
5492e96793 terminate buffers from X509_NAME_print_ex() 2014-09-02 23:30:37 +01:00
Vincent Sanders
37755fb135 add incomplete scheme fetcher for javascript urls 2014-08-25 15:59:52 -07:00
Vincent Sanders
af16c38d2d fix DOM tree dump debug 2014-08-04 00:37:21 +01:00
Vincent Sanders
db01637343 remove content include from urldb header 2014-07-17 00:18:44 +01:00
Vincent Sanders
0694f345da fix leak on error path (coverity 1224429) 2014-07-08 15:57:48 +01:00
Vincent Sanders
219f505a65 version 1.1 of the backing store disc layout using base32 encoded
filenames to allow for case insensitive filenames.
2014-07-04 17:09:28 +01:00
Vincent Sanders
a71865b328 turn off verbose fetch debugging 2014-07-02 17:07:29 +01:00
Vincent Sanders
29e29cccd7 fix dumb error in previous commit and get scheme name element correct 2014-07-02 15:01:24 +01:00
Vincent Sanders
516dcb96e1 fix unhelpful assert at quit 2014-07-02 14:51:11 +01:00
Vincent Sanders
06eb81235d fix fetchers include on windows 2014-07-01 09:17:22 +01:00
Vincent Sanders
8944edd649 convert all frontends to scheduled fetch operation 2014-06-26 19:05:59 +01:00
Vincent Sanders
1b7aa7ffe5 make fetchers scheduled 2014-06-26 19:05:59 +01:00
Vincent Sanders
4b2101ba6a clean up the fetcher factory and improve its API 2014-06-19 18:27:24 +01:00
Vincent Sanders
f1c2dde13b extend file table with mkdir all and make fs backing store use it.
enable fs backing store for RISC OS.
2014-06-05 12:06:47 +01:00
Rob Kendrick (fatigue)
7b1332c5c5 Include the date in the testament 2014-06-04 12:10:42 +01:00
Michael Drake
bd804d3fe4 Avoid signed vs unsigned comparason warning. 2014-06-04 07:48:05 +01:00
Vincent Sanders
5899f6c16e turn off debugging and add some comments and todo from dsilvers 2014-06-03 22:30:04 +01:00
Daniel Silverstone
b5461c8313 Adjust the llcache behaviour to use scheduler for user notification.
This change updates the llcache to use the scheduler to notify users of the
llcache of events.  This should be just as safe as before and is part of an
effort to remove hlcache_poll and llcache_poll eventually because fetchers
should schedule themselves if need-be.

This is a big change despite the diminutive nature of the patch.  Please report
issues promptly if they turn up after this and are not visible before it.

Signed-off-by: Daniel Silverstone <dsilvers@netsurf-browser.org>
Reviewed-by: Vincent Sanders <vince@netsurf-browser.org>
2014-06-03 17:00:23 +01:00
Vincent Sanders
e9d53022a4 write a cache tag file as an aid to backup software 2014-05-30 00:19:20 +01:00
Vincent Sanders
644fc616b3 attempt to purge low level cache on out of memory during fetch 2014-05-29 14:00:13 +01:00
Vincent Sanders
00b6cfc57e rework path to url mapping functions to convert from and to nsurl 2014-05-26 23:52:36 +01:00
Vincent Sanders
a6d3ceae0e Completely re-write web search provider handling 2014-05-25 01:01:59 +01:00
Michael Drake
fb5af8a1b6 Check return value from write() to avoid build warnings. 2014-05-24 15:19:01 +01:00
Michael Drake
7ba459f1b5 Fix leak of url schemes. 2014-05-16 18:58:06 +01:00
Michael Drake
e4ba144fe0 Remove unused #include
Persistent thumbnail code was ripped out of urldb ages ago.
2014-05-15 21:47:40 +01:00
Vincent Sanders
ef00272e2f add helpers for time_t reading/writing 2014-05-15 01:18:52 +01:00
Vincent Sanders
1a3ee6090d add strptime compatability 2014-05-13 16:37:49 +01:00
Vincent Sanders
05932293f6 Add filesystem based backing store 2014-05-13 15:53:02 +01:00
Vincent Sanders
4a49ff5266 Extend low level source data cache with persistant storage 2014-05-13 15:52:58 +01:00
Vincent Sanders
657abbd245 low level source data cache backing store interface. 2014-05-12 23:47:43 +01:00
Vincent Sanders
2326d3c503 remove unecessary utils/url.h includes 2014-05-09 10:15:42 +01:00
Vincent Sanders
c56642819e add file operations table and make all frontends use it.
This rationalises the path construction and basename file
operations. The default implementation is POSIX which works for all
frontends except windows, riscos and amiga which have differeing path
separators and rules.

These implementations are significantly more robust than the previous
nine implementations and also do not use unsafe strncpy or buffers
with arbitrary length limits.

These implementations also carry full documentation comments.
2014-05-07 16:24:51 +01:00
Vincent Sanders
b7b3496a78 move testament output to object directory instead of modifying source 2014-04-08 11:41:58 +01:00
Vincent Sanders
4d4d74c8cd move page search gui callbacks to their own operations table 2014-03-18 22:32:52 +00:00
Vincent Sanders
da0b969f25 Improve llcache header processing
By skipping empty headers and correctly dealing with whitespace around
header names we store fewer entries with better adherance to allowed
values in http responses.
2014-03-09 16:22:28 +00:00
Vincent Sanders
25ce52ee64 only try and cache http and https urls 2014-03-09 16:20:53 +00:00
Vincent Sanders
8d883978c6 remove unnecessary caching of whether a url has a query element 2014-03-09 16:20:53 +00:00
Vincent Sanders
20b3c40816 make nsurl_defragment() API more obvious and remove duplicated parameter checking 2014-03-09 16:20:53 +00:00
Vincent Sanders
87f6314dab move scheduleing into browser operation table 2014-03-09 15:37:40 +00:00
Vincent Sanders
60973fe5d1 Make llcache debugging less invasive 2014-02-21 11:43:48 +00:00
Vincent Sanders
ff8c37a6ee Witespace cleanup to prepare for backing store addition 2014-02-21 11:31:39 +00:00
John-Mark Bell
823aad5ddf Use libcurl's cache if it's new enough. Fixes #2064. 2014-02-06 01:19:22 +00:00
Vincent Sanders
fba58ad4b0 remove unecessary include 2014-02-04 15:19:46 +00:00
Vincent Sanders
7b380d0982 fix sign extension issue on 64bit platforms (coverity 1109898) 2014-02-04 13:24:25 +00:00