Commit Graph

130 Commits

Author SHA1 Message Date
Adrien Destugues
ed31589c37 URL Disaptching/Async listeners: forward debug messages
This makes it possible for the Asynchronous listener to get the
messages. It can then process them in a more fancy way.

The default implementation will still log the messages to the console
(if debug is enabled), but it will do so from the Async listener for
asynchronous requests now. This means they will probably be logged from
the same thread, and show up in a more readable way.

This also makes it possible to listen to several requests and log them
in a nice way (in a status window or whatever).
2016-10-31 22:14:39 +01:00
Andrew Lindesay
cf65729463 Url: implement same URL parsing logic in C/C++ code
- Remove uses of group matching regular expression, not available on all
  build hosts,
- Parsing is faster than our old regexp engine.

Signed-off-by: Adrien Destugues <pulkomandy@pulkomandy.tk>

- Fixes #13002
- Fixed some indentation (tabs vs space), please configure your editor
  properly.
2016-10-31 09:14:44 +01:00
Mark Hellegers
e1c98ceaf7 Add support for TLS SNI
Signed-off-by: Augustin Cavalier <waddlesplash@gmail.com>
2016-06-20 19:42:24 -04:00
Andrew Lindesay
fa2dd9c45f BPackageInfo::Parser: Validate URL strings.
Fixes #12710.

Signed-off-by: Augustin Cavalier <waddlesplash@gmail.com>
I fixed the modifications to the Jamfiles in src/bin, they were all wrong
in the patch.
2016-06-18 18:40:07 -04:00
Augustin Cavalier
f0a5e33a55 HttpAuthentication: Add explicit copy & assignment constructors.
So that we don't copy the BLocker. Fixes part of the build.
2016-05-13 16:49:52 -04:00
Rene Gollent
c9dd7d0ddf libbnetapi: Extend socket classes.
B{Abstract,Datagram,Secure}Socket:
- Add functionality to listen for and accept new connections, thus allowing
  one to use the socket classes for server functionality as well.

BSecureSocket:
- Adjust to take into account differences between how SSL needs to be called
  when accepting an incoming connection vs initiating an outbound one.
  The handshake on the accepted connection stills fails for unknown reasons
  at the moment though.

Note that these changes break the ABI, and thus any packages making use of
them directly will need a rebuild.
2016-04-28 22:30:40 -04:00
Julian Harnath
310238937c Network Kit: add BNetworkDevice::Control
* Allows to do an arbitrary ioctl on the network device

* For ifreq and ieee80211req requests
2016-03-15 21:59:03 +01:00
Adrien Destugues
c614961364 Implement CONNECT pass-through for HTTPS proxy
* When using a proxy, HTTPS connexion must still go directly to the
  target website. The proxy can then act as a TCP stream relay and just
  transmit the raw SSL stream between the client and website.
* For this, we ask the proxy sending an HTTP request with the CONNECT
  method. If the proxy supports this, we can then send anything as the
  payload and it will be forwarded.
* Untested, as the network here in Dusseldorf doesn't let me use a
  proxy.

ticket : #10973
2015-11-11 01:16:38 +01:00
Adrien Destugues
4849ab6c8b BHttpRequest: add SSL certificate exception management.
When an HTTPS request uses an SSL certificate that OpenSSL considers
untrusted, and the user decides to continue anyway, add the certificate
to an exception list. Match certificates against this list and don't ask
the user again if they are already there.

Fixes #12004. Thanks to markh for the initial patch and peeking into the
WebKit code!
2015-11-09 10:46:58 +01:00
Adrien Destugues
6c32f50a64 BCertificate: fixup the API
* Add an operator== and a copy constructor
* Make the getters const so they are easier to use
2015-11-08 23:47:29 +01:00
Michael Lotz
d64c7086d5 Various whitespace cleanup only. 2015-09-05 16:09:33 +02:00
Jérôme Duval
b4665f9a59 HttpForm needs another friend class for GCC6. 2015-07-30 22:25:19 +02:00
Adrien Destugues
f972422c66 Add a simple DNS caching system to BNetworkAddress
netresolv (and libbind) won't cache DNS requests, which can result in a
lot of DNS requests being made for the same host. Implement a simple
cache in RAM (local to each application) which will keep the most
recently requested addresses cached. This can speed up loading of an
HTTP page a lot, by saving a DNS request for each resource stored on the
same server as the main page.
2015-06-14 15:47:13 +02:00
Michael Lotz
3b7b927dd0 libbnetapi: Add BNetworkRoute to replace use of route_entry.
The BNetworkRoute class manages a route_entry and the sockaddr's
associated with it. It replaces the direct use of route_entry in the
BNetworkInterface API.

Using route_entry is fragile and inconvenient as it only holds pointers
to the sockaddr's. When getting a list of routes from the kernel, each
route_entry is set up so that its pointers point into the single flat
buffer that is passed around. Creating a copy of the route_entry and
then deleting the flat buffer makes the pointers in the copy stale.
Returning these route entries therefore always lead to a use-after-free
when they were eventually used.

BNetworkRoute also takes over the code and functionallity of getting
routes from RouteSupport. The corresponding method in BNetworkRoster is
replaced by a static method in BNetworkRoute.

Also distinguish between the default route and gateway of an interface.
GetDefaultRoute() now gets the default BNetworkRoute for the interface
while GetDefaultGateway() gets the associated gateway address within
that default route. Adjust network preferences panel to this change.

Note that we currently only seem to have per interface default routes
and not an actual global default route. This was already the case before
these changes and I did not further investigate what this means.
2015-04-12 18:50:00 +02:00
Axel Dörfler
db6d25ceba NetworkInterface.h: honor 80 character limit. 2015-03-27 13:21:47 +01:00
Axel Dörfler
7720614300 Revert "Move getifaddrs to libnetwork again."
This reverts commit 31ea76548a.

Adrien, please try again without clobbering the otherwise nice
BNetworkInterface API!

Conflicts:
	src/kits/network/getifaddrs.cpp
2015-02-05 11:07:53 +01:00
Adrien Destugues
31ea76548a Move getifaddrs to libnetwork again.
* BNetworkInterfaceAddress is moved to libnetwork. It is modified to not
use BNetworkAddress (which is in libbnetapi) and instead use sockaddr
and sockaddr_storage directly. All callers are adjusted to this.
* Some support code is shared between BNetworkInterface and
BNetworkInterfaceAddress, move it to libnetwork but in the BPrivate
namespace.
2015-01-21 10:07:03 +01:00
Adrien Destugues
5ee2151e2c BHttpRequest: propagate SSL errors to listener
This way it's possible to handle them in applications.
2014-11-06 15:02:00 +01:00
Adrien Destugues
c86ad7f93b Add more features in BCertificate class
* Make it possible to extract more useful data from the certificate
* Also get the OpenSSL error message when a certificate can't be
validated. Send it to the verification failure callback so it can be
shown to the user.
2014-11-06 15:01:59 +01:00
Adrien Destugues
9bf4e99477 BUrl: IDNA ToUnicode and ToASCII conversions.
* Since DNS are normally restricted to ASCII, the use of UTF-8 in domain
names is implemented using a "punycode" encoding.
* The request to the DNS server must be sent with the ASCII
representation of the domain name, however the Unicode one should be
used for user-visible parts.
* ICU provides an implementation of the conversion, which we use here.
* Conversion is currently done in-place and modifies the BUrl object
(this is similar to UrlEncode/UrlDecode).
* Adjust existing IDN test to make use of these methods. It's passing
now.
2014-10-21 14:43:13 +02:00
Adrien Destugues
06f436b3ac Use strict mode when url-decoding file requests.
We don't want '+' to become a space here.
Fixes #11109.
2014-10-21 11:31:48 +02:00
Adrien Destugues
2f36873105 UrlRequest: add empty SetTimeout for ease of use.
* Requests that support this can implement it, NetworkRequest does
* Build fix: no need to return status_t.
2014-09-26 15:51:36 +02:00
Adrien Destugues
654ae69aa6 NetworkRequest: add a SetTimeout method. 2014-09-26 15:21:54 +02:00
Adrien Destugues
c98378e51a Add HTTP proxy support.
* Move default context management to BUrlRequest since some code
(including the testsuite) bypass the BUrlProtocolRoster.
* Introduce proxy host and port in BUrlContext
* Have BHttpRequest use the proxy when making requests
2014-09-15 14:24:37 +02:00
François Revol
295ded8c41 Fix gopher
fInputBuffer was shadowing the base class member...
2014-08-08 00:35:20 +02:00
Adrien Destugues
89b4e98a8f Move signal hack to BNetworkRequest
* This is used to unlock sockets when a read is pending after a close
* It is not needed on requests that don't use a socket.
2014-08-04 16:25:48 +02:00
Adrien Destugues
2f9b187497 Factor out a BNetworkRequest
* Shares common behavior between the Gopher and HTTP request handlers.
* Most of this can be used when implemeting other protocols.
2014-08-04 15:59:53 +02:00
Adrien Destugues
a1cce97050 HttpRequest: more small fixes and cleanups
* Remove unneeded field fOutputHeaders and convert it to a local for the
only method that uses it,
* Don't return EOVERFLOW when flushing data from ZLib (the ZLib
decompressor returns this, but zlib docs states that this is NOT an
error condition).
* Replace unneeded temporary BNetBuffer of fixed size with BStackOrHeapArray.
2014-07-28 15:02:17 +02:00
Adrien Destugues
021ebc2f8c Add the port to the HTTP Host header when needed.
* When the port is not the default one, it must be added to the "Host"
header so the server knows what we're connecting to.

Fixes #11070.
2014-07-28 15:02:13 +02:00
François Revol
0c1a4ebf8b Preliminary support for Gopher
Currently parses information and text items and retrives files.
2014-07-26 01:42:36 +02:00
Adrien Destugues
9f7d29b05e Fix two problems with chunked gzipped HTTP replies.
* receiveEnd is set in a different place in case of chunked transfers,
which would cause the decompressor to never be flushed.
* In the case of chunked transfers, we call Flush() without any input
data (to flush only whatever is remaining in the decompression buffer).
This causes ZLib to return Z_BUF_ERROR which is translated to
B_BUFFER_OVERFLOW. This is a non-fatal error and is expected behavior in
that case. Don't handle this as an error, and do use the extracted data.

Fixes various cases of missing the last chunk of a page (pastie.org,
Google search results, and more).
2014-07-21 11:49:42 +02:00
Ingo Weinhold
72f6b787cf BUrl: Add missing functionality from support kit BUrl 2014-07-03 17:39:59 +02:00
Adrien Destugues
158ae74373 Escape reserved characters when converting paths to urls
* Introduce and use BUrl::BUrl(const BPath&)
* The path is url-encoded, and the protocol is set to "file"

Fixes #10964.
2014-06-25 10:45:03 +02:00
Adrien Destugues
def1a05788 Made the wrong constructor private. 2014-06-11 19:27:03 +02:00
Adrien Destugues
895fa41e0b Make handling of Http Authentication thread safe
* Each BHttpAuthentication object is locked on all field accesses,
* They are owned by the BUrlContext and never deleted, so there is no
need for reference-counting them,
* The BUrlContext itself is now reference counted, and all BUrlRequests
hold a reference to it.

This makes sure using the BHttpAuthentication objects from requests is
thread-safe.
2014-06-11 14:11:01 +02:00
Adrien Destugues
463ffbfde4 First steps towards cookie jar thread-safety
* Change the semantics of the iterators copy constructor and assignment
operator: they now return a new iterator for the same cookie jar (and
same url for the UrlIterator). They don't try to point to the same
position as the copied iterator. The only purpose of these is to write
code such as:

Iterator it = jar.GetIterator();

so having a full copy isn't that useful.

* The per-domain cookie lists are now protected with a read-write lock.
The iterators retain a read lock while they are handling cookies from
that list. They get a write lock when doing Remove. Adding a cookie to
the jar also gets the write lock for the matching list

* Fix a memory leak when adding a new domain-list to the jar failed

* Simplify the declaration of the PrivateHashMap type (it would be
even simpler if HashMap was a public API)

* The domain hashmap is now a SynchronizedHashMap. It is locked as long
as an Iterator or UrlIterator exists, which may be a problem as these
are public APIs. Writing safe iterators for an hashmap with concurrent
accesses is not easy, so the API could be modified to return a list of
domains and a list of cookies for a given domain or URL instead. This
would suit the intended uses just as well.

* The jar now store const cookies, so there is no need to lock them for
access/modification. Updating a cookie is done by replacing it with
another one in the jar (with the same domain and value). There is still
the problem of deleting a cookie while other threads may still access
it, this will be fixed by making cookies BReferenceable.
2014-06-11 12:59:33 +02:00
Adrien Destugues
1cbab031fd More relaxing of cookie-setting rules
* Allow non-secure page to set (but not read) secure cookies
* Allow pages to set cookies for subdomains (but not access them)
2014-06-09 11:17:49 +02:00
Adrien Destugues
7f1f341e5f Forgot to commit changes to the header. 2014-06-06 19:11:03 +02:00
Stephan Aßmus
3df9235571 HttpRequest.h: Fixed some formatting 2014-06-06 00:23:48 +02:00
Stephan Aßmus
cb1a99c5f0 HttpHeaders: Small code refactorings
Also check BList::Add() for success when adding a BHttpHeader.
2014-06-06 00:23:47 +02:00
Adrien Destugues
cd805f6793 Remove some redundant fields
These were getting out of sync and causing trouble, and they are easy to
compute from existing information.

Fixes some problems detected by the testsuite where the user/password or
the host would sometime disappear from the URL.
2014-06-04 11:56:23 +02:00
Adrien Destugues
cfc4b62367 Network Kit: Prepare for HTTP range requests
* The DataReceived hook gets a position argument, making it possible for
listeners to handle out-of-order data (from two range requests at
different positions, for example)
* Adjust HaikuDepot (only user of the API in our sources)
* Add a copy constructor to HTTPRequest that copies the relevant
parameters from an existing request. Makes it easy to repeat a request
with a different range. Could be useful for restarting downloads, or
paralellizing them.
* Add SetRangeStart, SetRangeEnd calls to HTTPRequest, no implementation
yet. I'm putting all the API changes in this commit as it needs to be
synced with a matching haikuwebkit release.
* All archs must update to HaikuWebkit 1.3.0. Previous versions are
broken by this.
2014-04-12 08:57:26 +02:00
Julian Harnath
d10ecc2c41 Style fix: add parameter name
* As pointed out by Axel.
2014-02-22 13:51:55 +00:00
Julian Harnath
dad49763d8 Disallow copy-construction of BCertificate.
* Prevents accidential creation of multiple instances with
  ownership of same fPrivate.
2014-02-22 13:51:52 +00:00
Julian Harnath
c99d7ea45c Fix double-free crash in BSecureSocket when cert. verification fails
* BSecureSocket::CertificateVerificationFailed() took a BCertificate
  instance by value as parameter.
  BCertificate deletes internal data in its destructor. Passing an
  object by value creates a copy, so the copy attempted to delete
  the internal data again during its destruction.
  This caused mail_daemon to crash here when it came across a failed
  certificate.

* Fix: pass BCertificate object as reference.
2014-02-22 02:09:55 +00:00
Adrien Destugues
67af469ef0 Fix time_t/bigtime_t mixup.
Thanks stippi for noticing!
2014-01-16 09:41:01 +01:00
Adrien Destugues
5ebdc79955 SecureSocket: add some certificate support
* Instead of creating an OpenSSL context ofor each socket, use a global
one and initialize it lazily when the first SecureSocket is created
* Load the certificates from our certificate list so SSL certificates
sent by servers can be validated.
* Add a callback for signalling that certificate validation failed, the
default implementation proceeds with the connection anyway (to keep the
old behavior).
* Introduce BCertificate class, that provides some information about a
certificate. Currently it's only used by the callback mentionned above,
but it will be possible to get the leaf certificate for the connection
after it's established.

Review of the API and implementation is welcome, before I start making
use of this in HttpRequest and WebKit to allow the user to accept new
certificates.
2014-01-15 17:45:21 +01:00
Adrien Destugues
3d864cd870 Remove B_PROT_* and related code
Use standard error codes instead.
This allows using error code returned by the underlying functions
directly, and makes it possible to use strerror for debugging. So, we
can also remove StatusString() from the various *Request classes.
2014-01-13 08:05:32 +01:00
Adrien Destugues
5b53e2e516 HttpRequest: close the connection on Stop()
When calling Stop(), we expect the request thread to exit as soon as
possible. Closing the connection unlocks it from any blocking read() or
write(), avoiding some lockup situations.
2014-01-02 08:38:03 +01:00
Adrien Destugues
824dd0a834 Reintroduce BUrlResult and add BDataRequest
* BUrlResult is back, with ContentType and Length methods.
* BHttpResult subclasses it and use HTTP header fields to implement
those
* Introduce BDataRequest for "data" URIs. These embed the data inside
the URI, either as plaintext or base64 encoded.
2013-12-11 17:29:25 +01:00