* Remove unneeded field fOutputHeaders and convert it to a local for the
only method that uses it,
* Don't return EOVERFLOW when flushing data from ZLib (the ZLib
decompressor returns this, but zlib docs states that this is NOT an
error condition).
* Replace unneeded temporary BNetBuffer of fixed size with BStackOrHeapArray.
* receiveEnd is set in a different place in case of chunked transfers,
which would cause the decompressor to never be flushed.
* In the case of chunked transfers, we call Flush() without any input
data (to flush only whatever is remaining in the decompression buffer).
This causes ZLib to return Z_BUF_ERROR which is translated to
B_BUFFER_OVERFLOW. This is a non-fatal error and is expected behavior in
that case. Don't handle this as an error, and do use the extracted data.
Fixes various cases of missing the last chunk of a page (pastie.org,
Google search results, and more).
* Each BHttpAuthentication object is locked on all field accesses,
* They are owned by the BUrlContext and never deleted, so there is no
need for reference-counting them,
* The BUrlContext itself is now reference counted, and all BUrlRequests
hold a reference to it.
This makes sure using the BHttpAuthentication objects from requests is
thread-safe.
* Change the semantics of the iterators copy constructor and assignment
operator: they now return a new iterator for the same cookie jar (and
same url for the UrlIterator). They don't try to point to the same
position as the copied iterator. The only purpose of these is to write
code such as:
Iterator it = jar.GetIterator();
so having a full copy isn't that useful.
* The per-domain cookie lists are now protected with a read-write lock.
The iterators retain a read lock while they are handling cookies from
that list. They get a write lock when doing Remove. Adding a cookie to
the jar also gets the write lock for the matching list
* Fix a memory leak when adding a new domain-list to the jar failed
* Simplify the declaration of the PrivateHashMap type (it would be
even simpler if HashMap was a public API)
* The domain hashmap is now a SynchronizedHashMap. It is locked as long
as an Iterator or UrlIterator exists, which may be a problem as these
are public APIs. Writing safe iterators for an hashmap with concurrent
accesses is not easy, so the API could be modified to return a list of
domains and a list of cookies for a given domain or URL instead. This
would suit the intended uses just as well.
* The jar now store const cookies, so there is no need to lock them for
access/modification. Updating a cookie is done by replacing it with
another one in the jar (with the same domain and value). There is still
the problem of deleting a cookie while other threads may still access
it, this will be fixed by making cookies BReferenceable.
These were getting out of sync and causing trouble, and they are easy to
compute from existing information.
Fixes some problems detected by the testsuite where the user/password or
the host would sometime disappear from the URL.
* The DataReceived hook gets a position argument, making it possible for
listeners to handle out-of-order data (from two range requests at
different positions, for example)
* Adjust HaikuDepot (only user of the API in our sources)
* Add a copy constructor to HTTPRequest that copies the relevant
parameters from an existing request. Makes it easy to repeat a request
with a different range. Could be useful for restarting downloads, or
paralellizing them.
* Add SetRangeStart, SetRangeEnd calls to HTTPRequest, no implementation
yet. I'm putting all the API changes in this commit as it needs to be
synced with a matching haikuwebkit release.
* All archs must update to HaikuWebkit 1.3.0. Previous versions are
broken by this.
* BSecureSocket::CertificateVerificationFailed() took a BCertificate
instance by value as parameter.
BCertificate deletes internal data in its destructor. Passing an
object by value creates a copy, so the copy attempted to delete
the internal data again during its destruction.
This caused mail_daemon to crash here when it came across a failed
certificate.
* Fix: pass BCertificate object as reference.
* Instead of creating an OpenSSL context ofor each socket, use a global
one and initialize it lazily when the first SecureSocket is created
* Load the certificates from our certificate list so SSL certificates
sent by servers can be validated.
* Add a callback for signalling that certificate validation failed, the
default implementation proceeds with the connection anyway (to keep the
old behavior).
* Introduce BCertificate class, that provides some information about a
certificate. Currently it's only used by the callback mentionned above,
but it will be possible to get the leaf certificate for the connection
after it's established.
Review of the API and implementation is welcome, before I start making
use of this in HttpRequest and WebKit to allow the user to accept new
certificates.
Use standard error codes instead.
This allows using error code returned by the underlying functions
directly, and makes it possible to use strerror for debugging. So, we
can also remove StatusString() from the various *Request classes.
When calling Stop(), we expect the request thread to exit as soon as
possible. Closing the connection unlocks it from any blocking read() or
write(), avoiding some lockup situations.
* BUrlResult is back, with ContentType and Length methods.
* BHttpResult subclasses it and use HTTP header fields to implement
those
* Introduce BDataRequest for "data" URIs. These embed the data inside
the URI, either as plaintext or base64 encoded.
We can send the data directly to the output socket instead of copying it
into a BString first, at the cost of very slightly less information in
debug output.
When using the copy constructor of BNetEndpoint the socket of the
original endpoint gets dup'ed. The Accept() method later directly reset
the fSocket member of the newly created BNetEndpoint to the socket
returned by accept(). The socket dup'ed by the copy constructor was
therefore leaked.
Of course dup'ing the socket and copying the local and remote addresses
is superfluous in the accept case, as these members all get set to new
values. To reduce that overhead there is now a new private constructor
that directly gets the final socket and remote and local address.
Implement BNetworkRoster::GetRoutes() and BNetworkInterface::GetRoutes().
Also implement BNetworkInterface::GetDefaultGateway().
There is code duplication at the moment, and the api only supports IPV4.
* Remove useless dummy protocol loop in UrlRequest
* Stop HTTP requests before deleting the socket and other things the
loop may still be using
* Deletion of items from the authentication map wasn't working
* Remove some debug traces
The authentication state is stored (in a hash map, using the domain+path
as a key) in the UrlContext class. It can then be reused for multiple
requests to the same place. We also lookup stored authentications for
parent directories and stop at the first we find.
Authentication state is not stored on disk (unlike cookies), and there
can only be one for each domain+path.
This change is needed for implementing cookie persistence in Web+ using
the network kit backend.
The current implementation requires the user to unarchive the cookie
jar, then hand it over to the BUrlContext which will copy it to its own
field. This makes the code simpler, but maybe doing a complete copy
(with all the cookies) is an heavy operation and could be avoided.
* Remove the fRawData field, as handling it is too complicated (it's
not easy to have proper copy semantics on a BDataIO) and it's not used
anyway, as the listener DataReceived call is enough to get the data and
handle it.
* All the remaining fields are HTTP-only, so rename the class to
HttpResult and attach it to HttpRequest instead of UrlRequest.
CID 1108353, 1108335: memory leak.
CID 610473: unused variable.
CID 1108446, 1108433, 1108432, 1108419, 1108400, 991710, 991713, 991712,
610098, 610097, 610096, 610095: uninitialized field
CID 1108421: unused field
Change the ownership of the result for Url/HttpRequests. The request now
owns its result and you either access it by reference while the request
is live, or copy it to keep it after the request destruction. To help
with that, get BUrlResult copy constructor and assignment operator to
work.
Performance issue: copying the BUrlResult also copies the underlying
BMallocIO data. This should be shared between the BUrlResult objects to
make the copy lighter. The case of BUrlSynchronousRequest is now
particularly inefficient, with at least 2 copies needed to get at the
result.
* Now takes ownership of headers, form data and input data
* Split Set* and Adopt* methods to help with proper use of this (Set
does a copy)
* Write documentation.
* The RFC provide a regular expression for URI parsing, so just use it.
* Allows parsing URIs with missing components (no scheme or authority)
* This allows to parse relative URLs as expected
* Can also handle things such as data: or mailto:
* Also more fixes to handling of incomplete URIs, some flags weren't
always set to the right values.
This gets Windows Live Mail (or is it called Outlook?) working, with
some other fixes on WebKit side.
* With so long class names, there's no way I'm going to follow the 64
char limit on commit headlines.
* The class share the same API, so having them separate is not very
useful.
* This makes it possible to use the same listener in either synchronous
or asycnhronous mode (or both, for different requests)
* Add some error handling in NetworkCookie and don't add broken cookies
(or should I say crumbs?) to the cookie jar
* More control on the path and domain, as well as the expiration time
We now pass Opera cookie testsuite functionality tests, as well as some
of the negative tests (we even do better than curl). Not going further
right now as this works well enough for positive cases and most
security/privacy issues are fixed (cross domain and cross path cookie
setting or spying).
* This takes a relative path as a parameter, and modifies the object to
point to the given location.
* '..' is not handled yet, and will be sent as-is to the server.
* Makes it possible to follow more types of 302 redirects
In particular, I can now run the tests from Opera's testsuite
(testsuite.opera.com), which shows I have more work to do on cookie
handling.
* Some fields weren't initialized, leading to random crashes later on
* Remove the enum that was used for protocol options
* Use a single field to track the request state, instead of separate
booleans.
* The cookie jar iterator now use a BObjectList instead of a BList
* Add a convenience method to the cookie jar to add a cookie by BUrl
and raw cookie string.
* Remove some methods in BNetworkCookie that could lead to invalid
cookies (cross-domain or with no domain at all).
* Make the cookie parsing able to report errors
* Fix off-by-one error in domain cookies validation.
* Remove the BUrlRequest class, which was only delegating work to
BUrlProtocol and subclasses
* Rename BUrlProtocol to BUrlRequest, and BUrlRequestHttp to BHttpRequest
* Creating a request is now done through the BUrlProtocolRoster. For
now there is just a static MakeRequest method, this will be completed
when we get to actually allowing add-ons to provide different request
handlers.
This allows cleanup of the API for requests:
* Remove the universal SetOption method with constants, and have
dedicated setters for each protocol option.
* Setters can now have multiple parameters, for example you can give
BHTTPRequest a BDataIO and a known size
* In this case, the BHttpRequest will not use HTTP chunked transfers,
which were always used before and made most servers unhappy (tested and
failed with lighttpd, google accounts and github).
