Commit Graph

87 Commits

Author SHA1 Message Date
John Mark Bell ad6fcea6b0 Add url_fragment to extract fragment from URL
Optionally allow url_compare to ignore fragments in comparison
Fix handling of url_compare result in a few places
Fix redirects which contain fragments in the Location header

svn path=/trunk/netsurf/; revision=3826
2008-02-03 12:04:48 +00:00
John Mark Bell cc3481a2ff Make core fetching code handle verifiability of transactions
Make core fetching code be responsible for inserting cookies into the urldb
Provide accessor to a fetch's parent url (this is defined as being the URL of the verifiable fetch which caused this one to occur)
Make fetchcache's 3xx handling use the parent url when spawning new fetches

svn path=/trunk/netsurf/; revision=3809
2008-01-30 19:56:41 +00:00
John Mark Bell 9164f247c2 Ensure plq is terminated when looking for an URL path.
Ensure fetchcache_redirect() normalizes the redirect destination.

svn path=/trunk/netsurf/; revision=3807
2008-01-30 01:44:57 +00:00
John Mark Bell 78d194cb77 Rework handling of HTTP redirects -- we now count the number of redirects followed for a given item and abort if a fixed limit is reached. This fixes sites which have pages that redirect to themselves.
Redirect handling is now transparent to clients of fetchcache.

The new scheme works as follows:

1) Request content for URL (fetchcache()
2) Start fetch of content (fetchcache_go()
3) If no redirect, continue through LOADING, READY, DONE etc. states as before
   If redirect, receive NEWPTR for each redirect that occurs, then continue
   through LOADING, READY, DONE etc. states as before.

The upshot of this is that redirects result in extra contents being created. It also means that, until LOADING has been received, the content (and thus the URL being fetched) may change. Therefore, fetchcache clients should expect to have to deal with transient data prior to LOADING occurring.

As a necessary side-effect of this, the HTML object URLs and CSS @import URLs are no longer stored alongside the relevant contents. These URLs can be accessed by interrogating the url member of struct content anyway, so it was a rather redundant scheme before.

svn path=/trunk/netsurf/; revision=3787
2008-01-28 01:35:00 +00:00
Vincent Sanders 2f864eeecc Update all source code file headers to reflect GPL version 2 only and contain appropriate licence text
svn path=/trunk/netsurf/; revision=3486
2007-08-08 16:16:03 +00:00
Daniel Silverstone 6807b4208a Remove the netsurf/ from the include paths and rationalise use of <> vs "" in includes
NetSurf includes are now done with ""s and other system includes with <>s as C intended.
The scandeps tool has been updated to only look for ""ed includes, and to verify that the
files exist in the tree before adding them to the dependency lines. The depend rule has
therefore been augmented to make sure the autogenerated files are built before it is run.

This is untested under self-hosted RISC OS builds. All else tested and works.


svn path=/trunk/netsurf/; revision=3307
2007-05-30 22:39:54 +00:00
John Mark Bell 25213dca24 Fix handling of cookies in unverifiable transactions caused by a redirect from a fetch into a browser window which was varifiable.
svn path=/trunk/netsurf/; revision=3165
2007-02-02 23:08:13 +00:00
James Bursa a0b6661eb6 Make GTK build compile on FreeBSD.
svn path=/trunk/netsurf/; revision=3154
2007-01-29 22:27:15 +00:00
John Mark Bell df4dbaf4cf Handle cookies in unverifiable transactions
svn path=/trunk/netsurf/; revision=3151
2007-01-27 20:58:20 +00:00
James Bursa bda01b3135 Fix parsing error when an empty HTML data is returned. Add HTTP status and other information to status bar.
svn path=/trunk/netsurf/; revision=3140
2007-01-13 00:21:15 +00:00
Michael Drake ba23e4b693 Update project URL.
svn path=/trunk/netsurf/; revision=3073
2006-11-27 15:35:18 +00:00
John Mark Bell 89f760a3e7 Detect attempted fetches using protocols we can't handle.
Mark content in error in a couple of cases that I'd missed ages ago.

svn path=/trunk/netsurf/; revision=2816
2006-08-06 17:51:23 +00:00
John Mark Bell b7af14b591 Implement component-wise URL comparison
svn path=/trunk/netsurf/; revision=2697
2006-07-02 22:34:04 +00:00
John Mark Bell bc2d96368f Fix conversion of URLs beginning "file://". Additionally,
file://host/... isn't likely, so assume file://path & convert to 
file:///path

svn path=/trunk/netsurf/; revision=2685
2006-07-01 22:27:10 +00:00
John Mark Bell ef301b8619 Fix URL file loading and add support for file:/// URLs to urldb
Convert file:/... to file:///... (the former isn't a valid URL)

svn path=/trunk/netsurf/; revision=2524
2006-04-12 08:09:27 +00:00
John Mark Bell c09eb457df Unify information databases
svn path=/trunk/netsurf/; revision=2519
2006-04-09 23:21:13 +00:00
John Mark Bell d4d3e5ee1c [project @ 2006-02-23 15:06:53 by jmb]
Handle invalid SSL certificates better - UI still needs work.
Modify fetch callback data parameter type to remove compiler warnings.
Constify things.
Lose global ssl_verify_certificates option.
Fix issue when closing a dialog without input focus.

svn path=/import/netsurf/; revision=2092
2006-02-23 15:06:54 +00:00
John Mark Bell e724672302 [project @ 2006-02-08 00:35:05 by jmb]
Handle case where no cache expiry headers are sent; use (now - last_modified) / 10. This should reduce the frequency of cache entry validation.

svn path=/import/netsurf/; revision=2064
2006-02-08 00:35:05 +00:00
John Mark Bell 0f228ada91 [project @ 2006-02-06 00:10:09 by jmb]
Implement HTTP caching algorithm; this should avoid stale cache entries being used.

svn path=/import/netsurf/; revision=2059
2006-02-06 00:10:09 +00:00
John Mark Bell aa76fb1754 [project @ 2006-01-03 23:26:58 by jmb]
Detect and prevent self-referential redirects.
This is not a complete solution as it does not handle the likes of page1 -> page2 -> page1 -> ...
To solve that probably requires counting the number of redirects a given fetch has made and then bailing once it exceeds some fixed value. From what I can see at present, this will have to be done at the top-level (i.e. browser_window_callback and suchlike).

svn path=/import/netsurf/; revision=1978
2006-01-03 23:26:58 +00:00
Richard Wilson e6f311f97c [project @ 2005-12-31 04:30:49 by rjw]
Updates to work with new url_store system.

svn path=/import/netsurf/; revision=1909
2005-12-31 04:30:49 +00:00
James Bursa 666cdaf8dd [project @ 2005-12-11 21:54:30 by bursa]
Disable some log messages.

svn path=/import/netsurf/; revision=1896
2005-12-11 21:54:30 +00:00
James Bursa 08e8ee3ed1 [project @ 2005-08-21 12:04:17 by bursa]
Change void * parameters to intptr_t to make them correct for storing integers or pointers.

svn path=/import/netsurf/; revision=1852
2005-08-21 12:04:18 +00:00
John Mark Bell 79c3c60a19 [project @ 2005-05-01 22:20:40 by jmb]
Work around invalid Content-Type headers

svn path=/import/netsurf/; revision=1707
2005-05-01 22:20:40 +00:00
James Bursa d9575d3bd5 [project @ 2005-04-18 21:40:29 by bursa]
Fix bug with redirected contents being reused (as triggered by looping redirects).

svn path=/import/netsurf/; revision=1670
2005-04-18 21:40:29 +00:00
John Mark Bell 92fa1133c9 [project @ 2005-04-01 02:25:11 by jmb]
application/xhtml+xml -> CONTENT_HTML
Fix minor issue with mimetype matching regexp.

svn path=/import/netsurf/; revision=1587
2005-04-01 02:25:11 +00:00
Richard Wilson 2affb76944 [project @ 2005-02-03 13:18:22 by rjw]
Implementation of URL suggestion

svn path=/import/netsurf/; revision=1488
2005-02-03 13:18:22 +00:00
James Bursa 04f0525fbf [project @ 2005-01-24 23:02:37 by bursa]
Reformat pages loaded from memory cache to window width.

svn path=/import/netsurf/; revision=1467
2005-01-24 23:02:37 +00:00
James Bursa a76404dfd0 [project @ 2005-01-13 20:28:50 by bursa]
Fix some compilation warnings in the debug build. Update nsgtk_plot_rectangle() with the new parameters.

svn path=/import/netsurf/; revision=1447
2005-01-13 20:29:24 +00:00
John Mark Bell 5b57548a6c [project @ 2005-01-03 16:09:11 by jmb]
Fix race condition caused by introduction of downloads

svn path=/import/netsurf/; revision=1431
2005-01-03 16:09:11 +00:00
John Mark Bell 77de9c1336 [project @ 2005-01-03 02:09:20 by jmb]
Shift + Click == download

svn path=/import/netsurf/; revision=1429
2005-01-03 02:09:20 +00:00
John Mark Bell 8334683068 [project @ 2005-01-02 03:58:20 by jmb]
xcalloc/xrealloc/xstrdup-purge - Lose remaining calls (and purge the relevant functions from utils.c)

svn path=/import/netsurf/; revision=1419
2005-01-02 03:58:21 +00:00
James Bursa 143d756fcf [project @ 2005-01-01 22:26:34 by bursa]
xcalloc/xrealloc/xstrdup-purge week.

svn path=/import/netsurf/; revision=1418
2005-01-01 22:27:05 +00:00
John Mark Bell e6de2f8b1d [project @ 2004-12-31 15:54:15 by jmb]
Fix scrollbar issue when fetching cached HTML contents.

svn path=/import/netsurf/; revision=1414
2004-12-31 15:54:15 +00:00
Richard Wilson 32db7e04d0 [project @ 2004-12-09 10:30:43 by rjw]
Re-implementation of hotlist via general tree code. Animations can be stopped once more. Purged a few xcalloc() calls.

svn path=/import/netsurf/; revision=1394
2004-12-09 10:30:44 +00:00
John Tytgat 402c4ca66c [project @ 2004-08-14 14:30:10 by joty]
Removed a chunk of Norcroft compiler warnings.  Re-ident some pieces.

svn path=/import/netsurf/; revision=1231
2004-08-14 14:30:12 +00:00
John Mark Bell c9bd6fa9fc [project @ 2004-08-09 16:11:58 by jmb]
Rework the interface of the URL handing module to allow for multiple error types.
Modify save_complete URL rewriting appropriately.

svn path=/import/netsurf/; revision=1206
2004-08-09 16:11:58 +00:00
James Bursa f0c6cb6d1a [project @ 2004-07-30 16:14:43 by bursa]
Plug some minor leaks.

svn path=/import/netsurf/; revision=1163
2004-07-30 16:14:44 +00:00
John Mark Bell ba22b4e753 [project @ 2004-07-10 02:35:30 by jmb]
Use libcurl's progress callback functionality to display fetch status.
This will update the status line once a second, more frequently requires hacking libcurl.

svn path=/import/netsurf/; revision=1066
2004-07-10 02:35:31 +00:00
James Bursa 6720beb2c1 [project @ 2004-06-23 15:41:50 by bursa]
Fix fetchcache_go() treatment of contents which the initiator loses interest in (eg. wrong content-type).

svn path=/import/netsurf/; revision=997
2004-06-23 15:41:50 +00:00
James Bursa ecc3e59fdc [project @ 2004-06-23 11:40:29 by bursa]
Fix destroying of contents after redirects.

svn path=/import/netsurf/; revision=996
2004-06-23 11:40:29 +00:00
James Bursa fdaf72417a [project @ 2004-06-22 17:37:51 by bursa]
Fix aborting of fetches.

svn path=/import/netsurf/; revision=990
2004-06-22 17:37:51 +00:00
James Bursa b4723c1d05 [project @ 2004-06-21 15:09:58 by bursa]
Merge memory cache into content module.

svn path=/import/netsurf/; revision=986
2004-06-21 15:09:59 +00:00
James Bursa d938fe135b [project @ 2004-06-10 23:55:23 by bursa]
Split fetchcache() into fetchcache() and fetchcache_go() to solve problems of callbacks being called before fetchcache() returns. Fix <style> breakage. Error handling fixes & improvements.

svn path=/import/netsurf/; revision=957
2004-06-10 23:55:23 +00:00
James Bursa 08177fa581 [project @ 2004-06-10 20:41:26 by bursa]
Add global content list. Better error handling in content code. Improved code documentation. Remove some obsolete functions. Implement debug window listing contents.

svn path=/import/netsurf/; revision=951
2004-06-10 20:41:26 +00:00
John Tytgat 6790d477d5 [project @ 2004-06-08 20:25:04 by joty]
- fetchcache_error_page() : small optimisation.
- removed compiler warning

svn path=/import/netsurf/; revision=936
2004-06-08 20:25:04 +00:00
John Tytgat dc65374aa6 [project @ 2004-06-05 15:03:59 by joty]
Created content_set_status() : one centralised place where status_message
in the content struct is updated in a fail safe way.

svn path=/import/netsurf/; revision=928
2004-06-05 15:03:59 +00:00
James Bursa bb41604045 [project @ 2004-05-05 16:33:15 by bursa]
Fix some compiler warnings. Ignore size attribute for file inputs.

svn path=/import/netsurf/; revision=828
2004-05-05 16:33:15 +00:00
John Mark Bell 8cb321ec78 [project @ 2004-05-02 00:21:40 by jmb]
Minimal fixes to allow compilation with Norcroft

svn path=/import/netsurf/; revision=813
2004-05-02 00:21:41 +00:00
Matthew Hambley d44ab700d0 [project @ 2004-04-25 11:40:05 by matthewh]
A more human face to download bytecounts.  Needs someone who can speak French to correct the fr messages file.

svn path=/import/netsurf/; revision=803
2004-04-25 11:40:05 +00:00