* Renamed page_queue to VMPageQueue and made it a proper C++ class. Use
DoublyLinkedList instead of own list code.
git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@34874 a95241bf-73f2-0310-859d-f6bbb57e9c96
playing with the unused list manually. This also clears the vnode's unused
flag, which wasn't done before and would thus cause corruption of the
unused list a bit later.
* fs_unmount():
- Fixed an iteration bug I introduced previously. The iterator would be
advanced twice per iteration, leading to NULL pointer dereferencing
when the vnode count was odd and skipping the checks for every other
vnode.
- All vnodes are going to be freed, so vnode_to_be_freed() has to invoked
for every one of them. The code wasn't adjusted correctly when
introducing the hot vnodes handling.
* Adjusted/improved some comments.
git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@34871 a95241bf-73f2-0310-859d-f6bbb57e9c96
used for it that is.
* The main cause for the heavy contention of the unused vnodes mutex was that
relatively few vnodes are actually used for a longer time. Mainly those are
the volume roots, mmap()ed files, and the files opened by programs. A good
deal of nodes -- particularly directories -- are just referenced for a very
short time, e.g. to resolve a path to a contained entry. This caused those
nodes to be added to and removed from the unused vnodes list very
frequently, thus resulting in a high contention of the mutex guarding it.
To address the problem I've introduced an approximation of a set of "hot"
vnodes, i.e. vnodes that have recently been marked unused. They are stored
in an array that by means of an r/w locker and atomic operations can most
of the time be accessed concurrently. Whenever it gets full, it is flushed
to the actual unused vnodes list.
* dec_vnode_ref_count(): No longer check the unused vnode count every time.
The called new vnode_unused() does only from time to time and returns when
the caller is expected to free some of the unused vnodes. As a side effect
this also fixes a bug I previously introduced: The unused vnode to be freed
was marked busy without being locked first.
The -j8 Haiku image test build shows that the changes reduce the contention
of the unused vnode list mutex to virtually zero without introducing any
significant contention of the new r/w lock. The VMCache lock contention also
seems to be decreased somewhat, which is probably not that surprising
considering that the page writer acquires/releases vnode references with the
cache lock held. The "pages" lock takes over even more contention, now
causing more than 100000 waits per second.
The total build time reduction is about 4.5%. Kernel time drops more than
10%.
git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@34866 a95241bf-73f2-0310-859d-f6bbb57e9c96
its own header/source files.
* Changed vnode's bit fields to a single, atomically changeable int32 using
flags instead. Added respective accessor methods.
* Added a per-vnode mutex-like lock, which uses 2 bits of the structure and
32 global "buckets" which are used for waiter lists for the vnode locks.
* Reorganized the VFS locking a bit:
Renamed sVnodeMutex to sVnodeLock and made it an r/w lock. In most situations
it is now only read-locked to reduce its contention. The per-vnode locks guard
the fields of the vnode structure and the newly introduced sUnusedVnodesLock
has taken over the job to guard the unused vnodes list.
The main intent of the changes was to reduce the contention of the sVnodeMutex,
which was partially successful. In my standard -j8 Haiku image build test the
new sUnusedVnodesLock took over about a fourth of the former sVnodeMutex
contention, but the sVnodeLock and the vnode locks have virtually no contention
to speak of, now. A lot of contention migrated to the unrelated "pages" mutex
(another bottleneck). The overall build time dropped about 10 %.
git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@34865 a95241bf-73f2-0310-859d-f6bbb57e9c96
* Fix a warning in VM tracing output, which prevented the compilation since
warnings are treated as errors.
Thanks!
git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@34855 a95241bf-73f2-0310-859d-f6bbb57e9c96
in VMCache, but, if anything, that makes a -j8 build marginally slower. I
guess busy vnodes are encountered so rarely that the additional overhead for
a more intelligent algorithm isn't really worth it. Reduced the wait time,
though.
git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@34845 a95241bf-73f2-0310-859d-f6bbb57e9c96
duplicating firmware names in firmware_get() of the freebsd compat layer.
git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@34844 a95241bf-73f2-0310-859d-f6bbb57e9c96
stores the value right-shifted by 12 bits, now, since those bits are not
relevant. This saves some bits and also resolves a TODO.
git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@34842 a95241bf-73f2-0310-859d-f6bbb57e9c96
destruction and VMVnodeCache::AcquireUnreferencedStoreRef(). Solved by
adding a flag to VMVnodeCache and letting AcquireUnreferencedStoreRef()
fail, if set.
* Added TODO regarding replacing the snooze() waiting for busy vnodes.
* get_vnode(): Unlock sVnodeMutex while calling the put_vnode() hook on
error.
git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@34841 a95241bf-73f2-0310-859d-f6bbb57e9c96
sure that the kernel's frame buffer console points to the right data.
git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@34835 a95241bf-73f2-0310-859d-f6bbb57e9c96
* When interrupts are disabled, it is still safe to capture the kernel stack
trace. The respective TODO preceded the introduction of the "kernelOnly"
flag.
* Actually made "kernelOnly" work. The wrong flag was passed to
arch_debug_get_stack_trace() in case it was false, so we never captured
user stack traces.
git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@34832 a95241bf-73f2-0310-859d-f6bbb57e9c96
directory iteration code, a mutex to protect the iteration cookie and one
to protect the cookie list have been introduced.
Overall this reduces the contention of the rootfs lock significantly. The
Haiku image -j8 build gets only marginally faster though.
git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@34831 a95241bf-73f2-0310-859d-f6bbb57e9c96
* Changed the rw_lock_{read,write}_unlock() return values to void. They
returned a value != B_OK only in case of user error and no-one checked them
anyway.
* Optimized rw_lock_read_[un]lock(). They are inline now and as long as
there's no contending write locker, they will only perform an atomic_add().
* Changed the semantics of nested locking after acquiring a write lock: Read
and write locks are counted separately, so read locks no longer implicitly
become write locks. This does e.g. make degrading a write lock to a read
lock by way of read_lock + write_unlock (as used in the VM) actually work.
These changes speed up the -j8 Haiku image build on my machine by a few
percent, but more interestingly they reduce the total kernel time by 25 %.
Apparently we get more contention on other locks, now.
git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@34830 a95241bf-73f2-0310-859d-f6bbb57e9c96
the function shall nevertheless return the length of the string that would
be written, if the buffer were large enough.
Added a touch of C++ while doing that. :-)
* Fixed the instances in boot loader, kernel, and kernel modules where the
wrong semantics were expected. The majority of uses actually.
git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@34826 a95241bf-73f2-0310-859d-f6bbb57e9c96
* Added VMCache::MovePage() and MoveAllPages() to move pages between caches.
* VMAnonymousCache:
- _MergeSwapPages(): Avoid doing anything, if neither cache has swapped out
pages.
- _MergeSwapPages() does now also remove source cache pages that are
shadowed by consumer swap pages. This allows us to call _MergeSwapPages()
before _MergePagesSmallerSource(), save the swap page shadowing check
there and get rid of the vm_page::merge_swap flag. This is an
optimization based on the assumption that usually none or only few pages
are swapped out, so we save a lot of checks.
- Implemented _MergePagesSmallerConsumer() as an alternative to
_MergePagesSmallerSource(). The former is used when the source cache has
more pages than the consumer cache. It iterates over the consumer cache's
pages, moves them to the source and finally moves all pages back to the
consumer. The final move is relatively cheap (though unfortunately we
still have to update all pages' vm_page::cache field), so that overall we
save iterations of the main loop with the more expensive checks.
The optimizations particularly improve the common fork()+exec*() situations.
fork() uses CoW, which is implemented by putting two new empty caches between
the to be copied area and its cache. exec*() destroys one copy of the area,
its cache and thus causes merging of the other new cache with the old cache.
Since this usually happens in a very short time, the old cache does still
contain many pages and the new cache only few. Previously the many pages were
all checked and moved individually. Now we do that for the few pages instead.
A very extreme example of this situation is the Haiku image build. jam has a
huge heap (> 200 MB) and it fork()s+exec*()s for every action to be executed.
Since during the cache merging the cache is locked, any write access to a
heap page causes jam to block until the cache merging is done. Formerly that
took so long that it killed a lot of parallelism in multi-job builds. That
could be observed particularly well when lots of small actions where executed
(like the Link, XRes, Mimeset, SetType, SetVersion combos when building
executables/libraries/add-ons). Those look dramatically better now.
The overall speed improvement for a -j8 image build on my machine is only
about 15%, though.
git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@34784 a95241bf-73f2-0310-859d-f6bbb57e9c96
* Pulled the code moving the pages out of Merge() into a separate method.
git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@34778 a95241bf-73f2-0310-859d-f6bbb57e9c96
another. The code originates from vm_copy_on_write_area(). We now generate
the VM cache tracing entries, though.
* count_writable_areas() -> VMCache::CountWritableAreas()
* Added debugger command "cache_stack" which is enabled when VM cache tracing
is enabled. It prints the source caches of a given cache or area at the
time of a specified tracing entry.
git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@34751 a95241bf-73f2-0310-859d-f6bbb57e9c96
- Replaced the "userOnly" parameter by a "flags" parameter, that allows to
specify kernel and userland stack traces individually.
- x86, m68k: Don't always skip the first frame as that prevents the caller
from being able to record its own address.
* capture_tracing_stack_trace(): Replaced the "userOnly" parameter by
"kernelOnly", since one is probably always interested in the kernel stack
trace, but might not want the userland stack trace.
* Added stack trace support for VM cache kernel tracing.
git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@34742 a95241bf-73f2-0310-859d-f6bbb57e9c96
* Lock the notification service and check HasListeners(), so we don't prepare
an event message needlessly.
* The on-stack buffer for the event message was too small for I/O operation
related events. Now a larger buffer belonging to the roster object is used.
git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@34737 a95241bf-73f2-0310-859d-f6bbb57e9c96
* Added Lock()/Unlock() for explicit locking by a service user.
* Added NotifyLocked() and made Notify() inline.
* Added HasListeners() so one can check whether there is a listener at all
before preparing the event message.
git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@34736 a95241bf-73f2-0310-859d-f6bbb57e9c96
be run again or generated/build/BuildConfig needs to be adjusted manually.
* Removed bochs debug hack.
git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@34721 a95241bf-73f2-0310-859d-f6bbb57e9c96
Fixes#5152.
* _get_port_message_info_etc(): Check whether the port still exists and is not
closed and empty in the loop. Though actually it shouldn't be necessary
(same in the other functions), since Wait() would return an error, if the
port was closed/deleted. Well, paranoia... :-)
git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@34713 a95241bf-73f2-0310-859d-f6bbb57e9c96
weren't terminated orderly.
* IOScheduler now stores its name and gets a unique ID.
* Added IOSchedulerRoster singleton which registers all IOSchedulers. It also
provides a notification service. We generate interesting events for
IOSchedulers, IORequests, and IOOperations.
git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@34702 a95241bf-73f2-0310-859d-f6bbb57e9c96
* Also check whether the port is closed after waiting.
* Don't increment read_count after waiting failed. That should have been the
cause of #5119.
git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@34687 a95241bf-73f2-0310-859d-f6bbb57e9c96
as soon as a second thread got into the game: if a thread was notified that
a message is ready, another thread could call read_port() and steal it before
the previous thread could claim it. The "Extensions" menu still doesn't seem
to work, but I would guess that is unrelated.
* The threads of the test app never exited, as read_port() returns the number
of bytes it read, not just a status.
git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@34681 a95241bf-73f2-0310-859d-f6bbb57e9c96
- enqueue_in_run_queue() no longer returns whether rescheduling is supposed
to happen. Instead is sets cpu_ent::invoke_scheduler on the current CPU.
- reschedule() does now handle cpu_ent::invoke_scheduler_if_idle(). No need
to let all callers do that.
* thread_unblock[_locked]() no longer return whether rescheduling is supposed
to happen.
* Got rid of the B_INVOKE_SCHEDULER handling. The interrupt hooks really
can't know, when it makes sense to reschedule or not.
* Introduced scheduler_reschedule_if_necessary[_locked]() functions for
checking+invoking the scheduler.
* Some semaphore functions (e.g. delete_sem()) invoke the scheduler now, if
they wake up anything with greater priority.
I've also tried to add scheduler invocations in the condition variable and
mutex/rw_lock code, but that actually has a negative impact on performance,
probably because it causes too much ping-ponging between threads when
multiple locking primitives are involved.
git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@34657 a95241bf-73f2-0310-859d-f6bbb57e9c96
IORequest::Wait(). Wait() immediately returned when IsFinished() returned
true, but this is the case as soon as the last IOOperation has finished. The
I/O scheduler is not done with the request at this point, though, since it
will still be sitting in at least one of three doubly linked lists. Since the
usual procedure to issue synchronous I/O requests is to create an IORequest
on the stack, pass it to the I/O scheduler, and Wait() on it, Wait()
returning early might cause the IORequest object to be destroyed while it is
still in use, leading to invalid memory access in the I/O scheduler,
corruption of its list structures, as well as later corruption of the issuing
thread's stack.
Related tickets:
* #4431: The request issuing thread returned and already deleted the area the
request was writing to before NotifyFinished() was called.
* #3048, #4883: Caused by the on stack IORequest being overwritten with other
data while being handled by the I/O scheduler thread.
* #4517: Hard to say, but I've seen a such a problem too, after a thread
scheduling related change. An explanation would be a list structure
corruption in the I/O scheduler causing an infinite loop with disabled
interrupts.
* #2845, #3428, #3429: The block notifier/writer is I/O heavy and as such
quite likely to run into the stack corruption issue.
git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@34655 a95241bf-73f2-0310-859d-f6bbb57e9c96
* Moved the CPU selection conde into a separate function.
* Re-added the support for disabling CPUs, which I accidentally removed.
* Got rid of idle CPU tracking. We don't need it anymore -- we iterate
through all CPUs and check the priority of the running thread.
* Added a kind of round-robin component to the CPU selection loop, so that
we pick CPUs more evenly. And indeed, the ProcessController display is
less skewed, now. There doesn't seem to be a measurable performance gain,
though.
git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@34637 a95241bf-73f2-0310-859d-f6bbb57e9c96
* simple_smp scheduler: Rewrote the interesting part of
enqueue_in_run_queue(). It always selects a target CPU for the inserted
thread, now. If no CPU is idle, the CPU running the thread with the lowest
priority is chosen. If the thread running on the target CPU has a lower
priority than the inserted one, it will be asked to reschedule. If that's
the current CPU, we'll return the correct value (wasn't done before at
all).
These changes help reducing latencies. On my machine in an idle system
playing music DebugAnalyzer shows maximum latencies of about 1 us. I still
find that a bit much, but it's several orders of magnitude better than
before. The -j8 Haiku image build time dropped about 10%.
git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@34635 a95241bf-73f2-0310-859d-f6bbb57e9c96
