Aren/haiku
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity

docs/develop: add some documentation for userlandfs and FUSE

Browse Source 
Change-Id: Ic8ee1bb5b3ca3e926251db766a120c48b96af45c
Reviewed-on: https://review.haiku-os.org/c/haiku/+/5488
Reviewed-by: waddlesplash <waddlesplash@gmail.com>
This commit is contained in:
PulkoMandy 2022-07-20 22:13:03 +02:00 committed by waddlesplash
parent ea698ddcbc
commit 3c1a967efb
2 changed files with 90 additions and 4 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

12
docs/develop/file_systems/overview.rst
View File

@ -1,5 +1,5 @@
Filesystem drivers Filesystem drivers
==================================== ==================
Filesystem drivers are in src/add-ons/kernel/file_system Filesystem drivers are in src/add-ons/kernel/file_system
@ -42,7 +42,8 @@ contents.
Development tools Development tools
----------------- -----------------
# fs_shell fs_shell
........
It is not convenient to test a filesystem by reloading its driver into a It is not convenient to test a filesystem by reloading its driver into a
running Haiku system (kernel debugging is often not as easy as userland). running Haiku system (kernel debugging is often not as easy as userland).
@ -70,7 +71,8 @@ argument. You need some tool to create one. It is possible to work using an
actual disk volume (but be careful, it's risky to use one with useful data in it), actual disk volume (but be careful, it's risky to use one with useful data in it),
a file, or a RAM disk, depending on what you are doing. a file, or a RAM disk, depending on what you are doing.
# userlandfs userlandfs
..........
As a second step, it's possible to use the filesystem as part of a runing As a second step, it's possible to use the filesystem as part of a runing
system, while still running it in userland. This allows use of Debugger, system, while still running it in userland. This allows use of Debugger,
@ -81,7 +83,8 @@ Userlandfs can run the filesystem code using the same interface as the kernel,
therefore, once everything is working with userlandfs, running the filesystem therefore, once everything is working with userlandfs, running the filesystem
as kernel code is usually quite easy (and provides a performance boost) as kernel code is usually quite easy (and provides a performance boost)
# Torture and performance tests Torture and performance tests
.............................
Once the basic operations are working fine, it is a good idea to perform more Once the basic operations are working fine, it is a good idea to perform more
agressive testing. Examples of scripts doing this are available in agressive testing. Examples of scripts doing this are available in
@ -89,5 +92,6 @@ src/tests/add-ons/kernel/file_systems/ for the fat and ext2 filesystems.
.. toctree:: .. toctree::
/file_systems/userlandfs
/file_systems/ufs2 /file_systems/ufs2
/file_systems/befs/resources /file_systems/befs/resources

82
docs/develop/file_systems/userlandfs.rst Normal file
View File

@ -0,0 +1,82 @@
UserlandFS: filesystems in userspace
####################################
UserlandFS was initially designed as a development tool allowing to run existing filesystems in
userspace. However, it turns out being able to do that is pretty useful outside of debugging
sessions too.
UserlandFS can load 3 types of filesystems, which are loaded as add-ons in an userlandfs_server
process. The process communicates with the kernel side of userlandfs to get the filesystem requests
and send back the responses.
UserlandFS exposes three different APIs to filesystem add-ons: one compatible with the Haiku kernel,
one compatible with the BeOS one, and one compatible with FUSE.
The communication between the kernel and userlandfs_server is identical no matter which filesystem
API is being used. The only differences are in the server itself, which will create an instance of
the correct subclass of Volume and forward the requests to it.
The FUSE API
============
FUSE is a similar tool to UserlandFS. It was initially designed for Linux, but, because there are
many filesystems written for it, it was later ported to MacOS and FreeBSD. The FUSE project in
itself defines a network-like communication protocol between the kernel and the filesystem, however,
this is not what is implemented in UserlandFS (which already has its own way to do this part).
Instead, the provided API is compatible with libfuse (version 2.9.9, since most filesystems
available for FUSE have not migrated to libfuse 3.x yet).
This means only filesystems using libfuse can easily be ported. Those implementing the FUSE protocol
in other ways (for example because they are not written in C or C++ and need a different library)
will not be ported to Haiku as easily.
The libfuse API is actually two different APIs. One is called "fuse_operations". It is a quite
high level API, where all functions to access files receive a path to the file to operate on, and
the functions are synchronous. The other API (called "low level") is asynchronous and the files
are identified by their inode number. Of course, the low level API allows to get better performance,
because the kernel and userlandfs already work with inode numbers internally. With the high level
API, the inode number has to be converted back to a filepath everytime a function is called.
Each FUSE filesystem uses one of these two APIs. UserlandFS implements both of them and will
automatically detect which one to use.
Because of the Linux FUSE design, normally each filesystem is a standalone application, that
directly establishes communication with the kernel. When built for UserlandFS, the filesystems are
add-ons instead, so their main() function is called from userlandfs_server after loading the add-on.
We have found that this works reasonably well and it will be possible to run most filesystems
without any changes to their sourcecode.
When they initialize, FUSE filesystems provide userlandfs with a struct containing function pointers
for the various FUSE operations they implement. From this table, userlandfs_server detects which
features are supported, and reports this back to the kernel. Then the kernel-side of userlandfs
knows to not forward to userspace requests that wouldn't be handled by the filesystem anyway.
FUSE filesystems also usually need some command line options (both custom ones, and standard ones
forwarded to FUSE initialization options). In Haiku, these are passed as mount options which are
forwarded to the filesystem add-on when starting it. The add-on main function will be called only
when a filesystem of the corresponding type is being mounted.
Extensions to the FUSE API
--------------------------
The FUSE API is missing some things that are possible in Haiku native filesystems. Specifically,
there is no possibility to implement the "get_fs_info" call, and in particular use it to set
volume flags. This is especially annoying for networked filesystems, where it is important to mark
the filesystem as "shared".
Therefore, an extra FUSE "capability" has been added to mark support for this. Capabilities are the
way FUSE negociates features between the kernel and the filesystem. The kernel tells the filesystem
which capabilities are supported, and the filesystem tells the kernel which one it wants to use.
If the FUSE_CAP_HAIKU_FUSE_EXTENSIONS capability is enabled, userlandfs will query the filesystem
info using the ioctl hook of the filesystem, with a command value FUSE_HAIKU_GET_DRIVE_INFO. It
expects the filesystem to then fill the fs_info structure passed in the ioctl pointer.
Debugging userlandfs
====================
Because a lot of the code is in userspace, you can simply start userlandfs_server in a terminal to
get its output, or attach a Debugger to it. There is no timeout for userlandfs requests, so if you
are debugging something in userlandfs_server, all calls to the filesystem mounted through it will
simply block until the request thread is running again.