haiku/docs/develop/file_systems/overview.rst

94 lines
3.7 KiB
ReStructuredText
Raw Normal View History

Filesystem drivers
====================================
Filesystem drivers are in src/add-ons/kernel/file_system
A filesystem usually relies on an underlying block device, but that's not
required. For example, NFS is a network filesystem, so it doesn't need one.
storage/SymLink: Fix Be API regression in ReadLink After this patch, "UnitTester BSymLink" passes. BSymLink::ReadLink() in BeOS would always return the length of the link unless an error occurred. Before this patch, Haiku instead seemed to emulate posix readlink() behavior, returning the number of bytes copied into the output buffer. BeOS also did not guarantee that the string written into the output buffer is NULL terminated if the output buffer cannot contain the entire link contents, but the Haiku implementation does since it is is a basic safety issue. This patch fixes this and updates the Haiku API docs to describe the behavior explicitly. Fixing this required changing behavior in bfs_read_link, which required changes in many more places. docs/user/storage/SymLink.dox: src/kits/storage/SymLink.cpp: * Don't return B_BUFFER_OVERFLOW if the provided buffer is not large enough to hold the link contents. * Update documentation to clearly describe behavior. src/add-ons/kernel/file_systems/bfs/kernel_interface.cpp: * Change bfs_read_link() to always return the link length. This is called by common_read_link in the VFS, which is called by _kern_read_link(). src/add-ons/kernel/file_systems/btrfs/kernel_interface.cpp: src/add-ons/kernel/file_systems/exfat/kernel_interface.cpp: src/add-ons/kernel/file_systems/ext2/kernel_interface.cpp: src/add-ons/kernel/file_systems/iso9660/kernel_interface.cpp: src/add-ons/kernel/file_systems/netfs/client/netfs.cpp: src/add-ons/kernel/file_systems/nfs/nfs_add_on.c: src/add-ons/kernel/file_systems/ramfs/kernel_interface.cpp: src/add-ons/kernel/file_systems/reiserfs/Iterators.cpp: src/add-ons/kernel/file_systems/reiserfs/Iterators.h: src/add-ons/kernel/file_systems/reiserfs/Volume.cpp: src/add-ons/kernel/file_systems/reiserfs/Volume.h: * Update the implementation of read_link for these filesystems. Some of them were incorrect, and some had just copied the posix behavior of bfs from before this patch. * Use user_memcpy in ext2_read_link() * Use user_memcpy in nfs fs_read_link() * Use user_memcpy in reiserfs StreamReader::_ReadIndirectItem and StreamReader::_ReadDirectItem * Remove unused method Volume::ReadObject in reiserfs. src/add-ons/kernel/file_systems/packagefs/nodes/UnpackingLeafNode.cpp: src/add-ons/kernel/file_systems/packagefs/package_links/PackageLinkSymlink.cpp: * Update UnpackingLeafNode::ReadSymlink and PackageSymLink::ReadSymLink() to set the bufferSize out parameter to the symlink length. Both of these are called by packagefs_read_symlink. * Use user_memcpy src/add-ons/kernel/file_systems/netfs/client/netfs.cpp: * netfs seems mostly unimplemented. Added a FIXME note for future implementers so that they know to implement the correct behavior. src/system/libroot/posix/unistd/link.c: * readlinkat() was just wrapping _kern_read_link() because before this patch it had expected posix behavior. But now it does not, so we need to return the number of bytes written to the output buffer. src/build/libroot/fs.cpp: * Update _kern_read_link() in the compatibility code to emulate the Haiku behavior on the host system. This is done by using an intermediate buffer that is guaranteed to fit the link contents and returning its length. The intermediate buffer is copied into the output buffer until there is no more room. src/tests/kits/storage/SymLinkTest.cpp: * This patch also resolves some test failures similar to those resolved in ee8cf35f0 which fixed tests for BNode. The tests were failing because Haiku's error checking is just better. BeOS allowed constructing a BSymLink with BSymLink(BDirectory*, const char*) with the entry name of "". The same is true of the equivilant SetTo() method. The BSymLink object will appear valid until you attempt to use it by, for example, calling the ReadLink method, which will return B_BAD_VALUE. Haiku does a more appropriate thing and returns B_ENTRY_NOT_FOUND, for this constructor and the equivilant SetTo(BDirectory*, const char*) method. This patch fixes these test assertions to match Haiku behavior. docs/develop/file_systems/overview.txt: * Add notes for future filesystem driver implementers to call this mistake when implementing fs_vnode_ops::read_symlink. docs/user/drivers/fs_interface.dox: * Fix documentation for fs_vnode_ops::read_symlink Change-Id: I8bcb8b2a0c9333059c84ace15844c32d4efeed9d Reviewed-on: https://review.haiku-os.org/c/haiku/+/2502 Reviewed-by: waddlesplash <waddlesplash@gmail.com> Reviewed-by: Axel Dörfler <axeld@pinc-software.de>
2020-04-20 08:02:41 +03:00
Implementation notes
--------------------
storage/SymLink: Fix Be API regression in ReadLink After this patch, "UnitTester BSymLink" passes. BSymLink::ReadLink() in BeOS would always return the length of the link unless an error occurred. Before this patch, Haiku instead seemed to emulate posix readlink() behavior, returning the number of bytes copied into the output buffer. BeOS also did not guarantee that the string written into the output buffer is NULL terminated if the output buffer cannot contain the entire link contents, but the Haiku implementation does since it is is a basic safety issue. This patch fixes this and updates the Haiku API docs to describe the behavior explicitly. Fixing this required changing behavior in bfs_read_link, which required changes in many more places. docs/user/storage/SymLink.dox: src/kits/storage/SymLink.cpp: * Don't return B_BUFFER_OVERFLOW if the provided buffer is not large enough to hold the link contents. * Update documentation to clearly describe behavior. src/add-ons/kernel/file_systems/bfs/kernel_interface.cpp: * Change bfs_read_link() to always return the link length. This is called by common_read_link in the VFS, which is called by _kern_read_link(). src/add-ons/kernel/file_systems/btrfs/kernel_interface.cpp: src/add-ons/kernel/file_systems/exfat/kernel_interface.cpp: src/add-ons/kernel/file_systems/ext2/kernel_interface.cpp: src/add-ons/kernel/file_systems/iso9660/kernel_interface.cpp: src/add-ons/kernel/file_systems/netfs/client/netfs.cpp: src/add-ons/kernel/file_systems/nfs/nfs_add_on.c: src/add-ons/kernel/file_systems/ramfs/kernel_interface.cpp: src/add-ons/kernel/file_systems/reiserfs/Iterators.cpp: src/add-ons/kernel/file_systems/reiserfs/Iterators.h: src/add-ons/kernel/file_systems/reiserfs/Volume.cpp: src/add-ons/kernel/file_systems/reiserfs/Volume.h: * Update the implementation of read_link for these filesystems. Some of them were incorrect, and some had just copied the posix behavior of bfs from before this patch. * Use user_memcpy in ext2_read_link() * Use user_memcpy in nfs fs_read_link() * Use user_memcpy in reiserfs StreamReader::_ReadIndirectItem and StreamReader::_ReadDirectItem * Remove unused method Volume::ReadObject in reiserfs. src/add-ons/kernel/file_systems/packagefs/nodes/UnpackingLeafNode.cpp: src/add-ons/kernel/file_systems/packagefs/package_links/PackageLinkSymlink.cpp: * Update UnpackingLeafNode::ReadSymlink and PackageSymLink::ReadSymLink() to set the bufferSize out parameter to the symlink length. Both of these are called by packagefs_read_symlink. * Use user_memcpy src/add-ons/kernel/file_systems/netfs/client/netfs.cpp: * netfs seems mostly unimplemented. Added a FIXME note for future implementers so that they know to implement the correct behavior. src/system/libroot/posix/unistd/link.c: * readlinkat() was just wrapping _kern_read_link() because before this patch it had expected posix behavior. But now it does not, so we need to return the number of bytes written to the output buffer. src/build/libroot/fs.cpp: * Update _kern_read_link() in the compatibility code to emulate the Haiku behavior on the host system. This is done by using an intermediate buffer that is guaranteed to fit the link contents and returning its length. The intermediate buffer is copied into the output buffer until there is no more room. src/tests/kits/storage/SymLinkTest.cpp: * This patch also resolves some test failures similar to those resolved in ee8cf35f0 which fixed tests for BNode. The tests were failing because Haiku's error checking is just better. BeOS allowed constructing a BSymLink with BSymLink(BDirectory*, const char*) with the entry name of "". The same is true of the equivilant SetTo() method. The BSymLink object will appear valid until you attempt to use it by, for example, calling the ReadLink method, which will return B_BAD_VALUE. Haiku does a more appropriate thing and returns B_ENTRY_NOT_FOUND, for this constructor and the equivilant SetTo(BDirectory*, const char*) method. This patch fixes these test assertions to match Haiku behavior. docs/develop/file_systems/overview.txt: * Add notes for future filesystem driver implementers to call this mistake when implementing fs_vnode_ops::read_symlink. docs/user/drivers/fs_interface.dox: * Fix documentation for fs_vnode_ops::read_symlink Change-Id: I8bcb8b2a0c9333059c84ace15844c32d4efeed9d Reviewed-on: https://review.haiku-os.org/c/haiku/+/2502 Reviewed-by: waddlesplash <waddlesplash@gmail.com> Reviewed-by: Axel Dörfler <axeld@pinc-software.de>
2020-04-20 08:02:41 +03:00
Each filesystem driver must define a few structures which act as the
interface between the VFS and the filesystem implementation. These
structures contain function pointers, some of which are optional,
which should point to functions defined by the implementer that
perform the appropriate filesystem peration as defined by the
documentation.
See docs/user/drivers/fs_interface.dox for more detailed documentation
of this interface.
It's important to note that while there may be some similarities in
the interface with that of other operations systems, one should not
make any assumptions about the desired behavior based soley on the
function prototypes defined in fs_interface.h.
The following is a list of notes calling out some potential mistakes.
# fs_vnode_ops.read_symlink
Defining this function means that the filesystem driver supports
symbolic links, and the function that f_vnode_ops.read_symlink points
to should read the contents of a symlink from the specified node.
This may seem similar to the posix function readlink(), but it is
slightly different. Unlike readlink(), which returns the number of
bytes copied into the output buffer, fs_vnode_ops.read_symlink is
expected to always return the length of the symlink contents, even if
the provided buffer is not large enough to contain the entire symlink
contents.
Development tools
-----------------
# fs_shell
It is not convenient to test a filesystem by reloading its driver into a
running Haiku system (kernel debugging is often not as easy as userland).
Moreover, the filesystem interacts with other components of the system
(file cache, block cache, but also any application reading or writing files).
For the early development steps, it is much easier to run the filesystem code
in a more controlled environment. This can be achieved through the use of
a "filesystem shell": a simple application that runs the filesystem code, and
allows performing specific operations through a command line interface.
Example of fs_shell implementations are available under src/tests/add-ons/kernel/file_systems/
for the bfs and btrfs filesystems.
For example, to build the fs_shell for btrfs, use
jam -q "<build>btrfs_shell"
To run it, use
jam run objects/haiku_host/x86_gcc2/release/tests/add-ons/kernel/file_systems/btrfs/btrfs_shell/btrfs_shell [arguments]
You need to pass at least a file or device containing a filesystem image as an
argument. You need some tool to create one. It is possible to work using an
actual disk volume (but be careful, it's risky to use one with useful data in it),
a file, or a RAM disk, depending on what you are doing.
# userlandfs
As a second step, it's possible to use the filesystem as part of a runing
system, while still running it in userland. This allows use of Debugger,
memory protection, and in general any kind of userland debugging or tracing
tool. When the filesystem crashes, it does not bring down the whole system.
Userlandfs can run the filesystem code using the same interface as the kernel,
therefore, once everything is working with userlandfs, running the filesystem
as kernel code is usually quite easy (and provides a performance boost)
# Torture and performance tests
Once the basic operations are working fine, it is a good idea to perform more
agressive testing. Examples of scripts doing this are available in
src/tests/add-ons/kernel/file_systems/ for the fat and ext2 filesystems.
.. toctree::
/file_systems/ufs2
/file_systems/befs/resources