* Added block_writer thread that continuously writes back blocks for all current
block caches.
* A block cache now maintains the number of dirty blocks if it doesn't use
transactions. That knowledge could also be used in block_cache_sync()...
git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@23698 a95241bf-73f2-0310-859d-f6bbb57e9c96
* This should speed up writing a transaction considerably.
git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@23694 a95241bf-73f2-0310-859d-f6bbb57e9c96
* Removed the public hash_grow() function again (at least for now, it's only
private).
* Removed the newSize argument from hash_grow(); it will compute the new size
automatically.
* The block cache is now using hash_insert_grow() instead of hash_insert()
which should make hash lookups much faster with some 10 thousand blocks,
also increased the initial table size from 32 to 1024...
git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@23692 a95241bf-73f2-0310-859d-f6bbb57e9c96
So now there's a central place to enable tracing in general and for
individual components.
git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@23645 a95241bf-73f2-0310-859d-f6bbb57e9c96
"block_cache" to dump info about transactions.
* Improved tracing.
* Now uses the new add_debugger_command_etc().
git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@23644 a95241bf-73f2-0310-859d-f6bbb57e9c96
blocks into the new transaction, but it would set that transaction on all
blocks of the old transaction, too. Also, it did not correctly update the
num_blocks/sub_num_blocks fields of the old transaction. Even worse, it did
return B_OK instead of the ID of the new transaction...
* get_writable_cached_block() did not correctly maintain the number of blocks
in the sub transaction.
* write_cached_block() did not free the original_data of a block when it wrote
it back as part of a previous transaction.
* Changed "cookie" for cache_next_block_in_transaction() to "long", so it will
be 64 bits when needed.
* Improved the API for detaching sub transactions: you can now get the blocks
of only the main (parent) transaction as well, added new
cache_block_in_main_transaction() function.
* BFS now flushes the log when there is no space left for the current
transaction.
* _WriteTransactionToLog() allocated a "vecs" array, but never freed it.
* _WriteTransactionToLog() now also supports detaching the current sub
transaction if the whole thing is getting too large (it will now also panic
if that doesn't work out).
* Removed a useless optimization: making the blocks available in the cache
isn't really needed, as all blocks in a transaction are locked into the
cache, anyway.
* Implemented Transaction::WriteBlocks().
* Minor cleanup, removed some dead code, fixed warnings in the fs_shell's
block_cache when compiled with debug output on.
git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@23610 a95241bf-73f2-0310-859d-f6bbb57e9c96
(so we can't allocate more physical pages than virtual address space
anymore, but so what?).
* Used the new CACHE_LARGE_SLAB flag as a temporary work-around; else
the slab would easily create several thousands of areas, which our
area code (and kernel heap) can't really handle that well (gets
awfully slow).
* Block caches with the same size could share the same slab, but we
don't do that yet.
git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@23567 a95241bf-73f2-0310-859d-f6bbb57e9c96
* Made the block_cache KDL command dump a bit more useful info (number of
referenced and dirty blocks).
* Minor cleanup.
git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@23540 a95241bf-73f2-0310-859d-f6bbb57e9c96
* improved the "block_cache" KDL command: it can now also dump blocks,
added support for the "--help" argument.
git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@23532 a95241bf-73f2-0310-859d-f6bbb57e9c96
transaction ends or has been aborted.
* BFS now listens for transactions when it created an inode to see if
the transaction will be aborted without freeing the inode (in which
case it will panic for now).
* Started implementing tracing support, but it's not working yet.
* Minor cleanup.
git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@23492 a95241bf-73f2-0310-859d-f6bbb57e9c96
set when the first chunk of the file could be read in directly, causing it
to read data to a wrong place in the buffer.
* Reading in the first chunk directly would have also only worked if vecIndex
and vecOffset was 0 when calling the function.
* Applied the fs_shell changes in file_map to the kernel version as well (the
constructor already worked correctly, though).
git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@23265 a95241bf-73f2-0310-859d-f6bbb57e9c96
the inode's internal last modified and size copies, causing in inconsistent
data. This fixes#1643.
* The block being replayed are now dumped to the debug output.
git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@23050 a95241bf-73f2-0310-859d-f6bbb57e9c96
the complete extent info or not.
* file_map_translate() now cuts down the request to the file bounds.
* Adjusted BFS and FAT to the API changes.
git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@22913 a95241bf-73f2-0310-859d-f6bbb57e9c96
that can be used by file systems.
* Changed the way the file cache works: instead of reading/writing to the
underlying device directly, it can now be used for any data source, ie.
also network file systems.
* As a result, the former pages_io() moved to the VFS layer, and can now be
called by a file system via {read|write}_file_io_vec_pages() (naming
suggestions are always welcomed :-)). It now gets an FD, and uses that to
communicate with the device (via its fs_{read|write}_pages() hooks).
* The file_cache_{read|write}() functions must now be called without holding
an I/O relevant file system lock. That allows the file cache to prepare the
pages without colliding with the page writer, IOW the "mayBlock" flag can
go into the attic again (yay!).
* This also results in a much better performance when the system does I/O and
is low on memory, as the page writer can now finally write back some pages,
and that even without maxing out the CPU :)
* The API changes put slightly more burden on the fs_{read|write}_pages()
hooks, but in combination with the file_map it's still pretty straight
forward. It just will have to dispatch the call to the underlying device
directly, usually it will just call its fs_{read|write}_pages() hooks
via the above mentioned calls.
* Ported BFS and FAT to the new API, the latter has not been tested, though.
* Also ported the API changes to the fs_shell. I also completely removed its
file cache level page handling - the downside is that device access is no
longer cached (ie. depends on the host OS now), the upside is that the code
is greatly simplified.
git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@22886 a95241bf-73f2-0310-859d-f6bbb57e9c96
and write_to_cache() before, IOW the cache could reserve too few pages.
git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@22859 a95241bf-73f2-0310-859d-f6bbb57e9c96
needed pages correctly, and would also not read/write enough in case the
offset didn't start at 0 resulting in undetected short reads/writes. It's
amazing how many bugs can be hidden in a few lines of code.
* Fixed a bug that might have been the cause for bug #1601: when the last part
of the write did not end on a page boundary, the last page had to be read
first, but that was done from the wrong offset. Also, if only parts of that
page could be read (because the file size didn't span over the whole page)
the remaining parts needed to be cleared.
* The cache_funcs were always called with the same value for numBytes and
bufferSize so I've eliminated the former.
* Large reads now also bypass the cache in case of low memory, large writes now
also only bypass the cache in that case, following Ingo's suggestion.
* Fixed compilation with debugging turned on.
git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@22858 a95241bf-73f2-0310-859d-f6bbb57e9c96
than 64KB. Reads should probably get a similar logic, at least if memory is
tight.
git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@22857 a95241bf-73f2-0310-859d-f6bbb57e9c96
sequential access.
* This is now used to let the cache free some pages (or schedule them to be
written back) before vm_reserve_pages() is called, but only if that cache
is not mapped, and there is a memory shortage.
git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@22731 a95241bf-73f2-0310-859d-f6bbb57e9c96
pages to reserve, which could result in to few being reserved in certain
situations.
* Use MutexLocker where appropriate.
* Reordered includes following the new rules.
git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@22494 a95241bf-73f2-0310-859d-f6bbb57e9c96
busy vnodes.
* dir_create_entry_ref() used get_vnode() incorrectly (and could therefore
potentially prevent a file system from doing proper locking when called
from the kernel).
* The vnode_store now uses this for its acquire_unreferenced_ref()
implementation (and therefore for the page writer).
* read_into_cache() and write_to_cache() were still marked inline.
* The system will now wait 10 secs for a busy vnode before returning an error.
* It will also no longer panic in that case.
git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@22485 a95241bf-73f2-0310-859d-f6bbb57e9c96
allocate with the vm_cache locked - this is necessary to be able to steal pages
from itself (large files...).
The system doesn't actually lock up anymore, but it still renders itself unusable;
obviously the page thief does not work correctly, yet. The rest of the experience
is created by our current scheduler (the page thief runs and runs, but it doesn't
free any pages anymore).
git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@22466 a95241bf-73f2-0310-859d-f6bbb57e9c96
twice into the same cache: cache_io() called read_into_cache() (or
write_to_cache()), and that broke down the request into smaller parts.
It then called read_chunk_into_cache() (or write_chunk_to_cache() resp.)
to actually allocate pages and fulfill the request.
However, it needed to unlock the cache for each chunk, and in the mean
time someone else could insert pages into the remaining chunks.
* Now, cache_io() already takes care of chunking the data which makes this
approach safe, and also simplified the code a bit - read_into_cache()/
write_to_cache() are gone now. I've renamed read_chunk_into_cache() to
read_into_cache() (same for the write function).
* Also got rid of that goto in that function while I was on it.
* Disabled cache_prefetch_vnode() for now (it's similar to cache_io(), but
since it's currently not used [since no cache module is installed yet],
I didn't want to go through updating it now, too).
git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@22465 a95241bf-73f2-0310-859d-f6bbb57e9c96
* his has to be used by the page writer to make sure the vnode is still valid.
* This should have been the final nail on the Luposian bug - I haven't tested
it yet, but we'll certainly see :-)
git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@22462 a95241bf-73f2-0310-859d-f6bbb57e9c96
* the page writer don't allow to block, while all other writers do. This fixes
bug #1509. The reason the page writer needs this is because it marks several
pages from different caches as busy.
* Fixed a warning about ASSERT being defined already in BFS, since
util/DoublyLinkedList.h now includes debug.h.
git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@22434 a95241bf-73f2-0310-859d-f6bbb57e9c96
to the private VM types are including vm_types.h now.
* Removed vm_page, vm_area, vm_cache, and vm_address_space typedefs; it's
cleaner this way, and the actual types are only used in C++ files now,
anyway.
* And that caused changes in many files...
* Made commpage.h self-containing.
* Minor cleanup.
git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@22329 a95241bf-73f2-0310-859d-f6bbb57e9c96
is tracked while the page scanner runs much slower. Now, we just set it to a
fixed positive, so that they can easily age in case they are actually unused.
git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@22320 a95241bf-73f2-0310-859d-f6bbb57e9c96
are locked, there is now a vm_page_reserve_pages() call to ensure upfront that
there is a page for me when I need it, and may have locked some caches.
* The vm_soft_fault() routine now makes use of that feature.
* vm_page_allocate_page() now resets the vm_page::usage_count, so that the file
cache does not need to do this in read_chunk_into_cache() and
write_chunk_to_cache().
* In cache_io() however, it need to update the usage_count - and it does that
now. Since non-mapped caches don't have mappings, the page scanner will punish
the cache pages stronger than other pages which is accidently just what we
want.
git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@22319 a95241bf-73f2-0310-859d-f6bbb57e9c96
* Removed the vm_cache/vm_store ref_count duality that besides being a bit ugly
also created the page dameon cache retrieval problem: now, only areas (and
cache consumers) retrieve a reference to the store (and therefore, the vnode).
The page daemon doesn't need to care about this at all anymore, and the pseudo
references of the vm_cache could be removed again.
* Rearranged deletion of vnodes such that its ID can be reused directly after
fs_remove_vnode() has been called.
* vm_page_allocate_page() no longer panics when it runs out of pages, but just
waits for new pages to become available using the new sFreeCondition condition
variable - to make sure this happens in an acceptable time frame, it'll
trigger a run of the low memory handlers.
* Implemented a page_thief() that steals inactive pages from caches and puts
them into the free queue. It runs as a low memory handler.
* The file cache now sets the usage count on the pages it inserts into the
cache (needs some rework though, cache_io() doesn't do it yet).
* Instead of panicking, the kernel will currently dead lock in low memory
situations, since BFS does a bit too much in bfs_release_vnode().
* Some minor cleanup.
git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@22315 a95241bf-73f2-0310-859d-f6bbb57e9c96
correct values (resolving a TODO); however, the latter doesn't do anything
with those yet.
* Cleanup.
git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@22154 a95241bf-73f2-0310-859d-f6bbb57e9c96
since fault_find_page() does no longer insert a dummy page into a cache
that has a store from which it can read the page.
git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@21887 a95241bf-73f2-0310-859d-f6bbb57e9c96
* Removed a few instances where the page state was set busy directly after
allocating it. This is a no-op, since a page is always busy after
allocation.
git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@21875 a95241bf-73f2-0310-859d-f6bbb57e9c96
removed in several cases.
* vfs_get_vnode_cache() now always gives out a reference to the cache it
returns; if it needs to allocate a new one, the vnode owns one reference,
and the caller another.
* therefore, file_cache_create() now owns a reference to its vm_cache_ref, and
frees it in file_cache_delete().
git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@21533 a95241bf-73f2-0310-859d-f6bbb57e9c96
