Commit Graph

55 Commits

Author SHA1 Message Date
Ingo Weinhold
c1be1e0761 * VMTranslationMap::Map()/Protect(): Added "memoryType" parameter. Not
implemented for any architecture yet.
* vm_set_area_memory_type(): Call VMTranslationMap::ProtectArea() to change the
  memory type for the already mapped pages.



git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@36574 a95241bf-73f2-0310-859d-f6bbb57e9c96
2010-05-01 21:33:12 +00:00
Ingo Weinhold
3b0c1b5227 * VMArea: Made memory_type private and added setter and getter methods.
* Don't set the VMArea's memory type in arch_vm_set_memory_type(), but let the
  callers do that.
* vm_set_area_memory_type(): Does nothing, if the memory type doesn't change.



git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@36573 a95241bf-73f2-0310-859d-f6bbb57e9c96
2010-05-01 20:41:57 +00:00
Ingo Weinhold
907886143f * Changed some parameters of VM syscalls from int to uint32, mostly for sake
of consistency.
* Moved the B_OVERCOMMITTING_AREA flag from B_KERNEL_AREA_FLAGS to
  B_USER_AREA_FLAGS, since we really allow it to be passed from userland.
* Most VM syscalls check the provided protection against B_USER_AREA_FLAGS
  instead of B_USER_PROTECTION, now. This way they allow for
  B_OVERCOMMITTING_AREA as well.
* _user_map_file(), _user_set_memory_protection(): Check the protection like
  the other syscalls do and use fix_protection() instead of doing that
  manually.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@36572 a95241bf-73f2-0310-859d-f6bbb57e9c96
2010-05-01 19:47:39 +00:00
Ingo Weinhold
b1e06d23ce * The "page_stats" command does now also print the longest contiguous runs of
free respective free and cached pages.
* Removed the unused vm_page_allocate_page_run_no_base().
* vm_page_allocate_page_run() (and allocate_page_run()):
  - Use vm_page_reserve_pages() instead of vm_page_try_reserve_pages(), i.e.
    wait until the reservation succeeds.
  - Now we iterates two times through the pages to find a suitable page run. In
    the first iteration it only looks for free/clear pages, in the second
    iteration it also considers cached pages. This increases the chance of the
    function to succeed, when a lot of caching is going on.
    This reduces the amount of memory required to use the IOCache when booting
    off the anyboot Live CD to around 160 MB in qemu. It also seems to work with
    128 MB, but the syslog indicates that some memory allocations fail, which
    is not exactly inspiring confidence.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@36489 a95241bf-73f2-0310-859d-f6bbb57e9c96
2010-04-26 13:56:09 +00:00
Ingo Weinhold
efeca209a1 Made VMCache::Resize() virtual and let VMAnonymousCache override it to free
swap space when the cache shrinks. Currently the implementation stil leaks
swap space of busy pages.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@36373 a95241bf-73f2-0310-859d-f6bbb57e9c96
2010-04-20 14:04:18 +00:00
Ingo Weinhold
c3676b54bf * Added vm_debug_copy_page_memory() which copies memory from a potentially not
mapped page.
* debug_{mem,strl}cpy():
  - Added "team" parameter for specifying the address space the address are
    to be interpreted in.
  - When the standard memcpy() (with fault handler) fails, fall back to
    vm_debug_copy_page_memory().
* Added debug_is_debugged_team(): Predicate returning true, if the supplied
  team_id refers to the same team debug_get_debugged_thread() belongs to.
* Added DebuggedThreadSetter class for scope-based debug_set_debugged_thread().
  Made use of it in several debugger functions.
* print_demangled_call() (x86): Fixed unsafe memory access.

Allows KDL stack traces to work correctly again, even if the page daemon has
already unmapped the concerned pages.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@36230 a95241bf-73f2-0310-859d-f6bbb57e9c96
2010-04-13 17:40:15 +00:00
Ingo Weinhold
86875ad9d1 Added VMCache::DebugHasPage() and DebugLookupPage() for use in the kernel
debugger.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@36228 a95241bf-73f2-0310-859d-f6bbb57e9c96
2010-04-13 17:18:57 +00:00
Ingo Weinhold
09418c869b Added DebugGet() method for use in the kernel debugger.
git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@36227 a95241bf-73f2-0310-859d-f6bbb57e9c96
2010-04-13 17:16:42 +00:00
Ingo Weinhold
349039ff2e Added vm_[un]wire_page(), which are essentially versions of
[un]lock_memory_etc() optimized for a single page.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@36156 a95241bf-73f2-0310-859d-f6bbb57e9c96
2010-04-11 15:07:06 +00:00
Ingo Weinhold
4f774c503c * VMArea::Unwire(addr_t, size_t, bool): Don't delete the removed range, but
return it.
* lock_memory_etc(): On error the VMAreaWiredRange object could be leaked.
* [un]lock_memory_etc(): Call VMArea::Unwire() with the cache locked and
  explicitly delete the range object after unlocking the cache to avoid
  potential deadlocks.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@36035 a95241bf-73f2-0310-859d-f6bbb57e9c96
2010-04-05 12:12:36 +00:00
Ingo Weinhold
369111e741 * Removed the VMArea::Wire() version that has to allocate a VMAreaWiredRange.
Since the requirement is that the area's top cache is locked, allocating
  memory isn't allowed.
* lock_memory_etc(): Create the VMAreaWiredRange object explicitly before
  locking the area's top cache.

Fixes #5680 (deadlocks when using the slab as malloc() backend).


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@36033 a95241bf-73f2-0310-859d-f6bbb57e9c96
2010-04-05 11:08:01 +00:00
Ingo Weinhold
550376ffb8 * vm_delete_areas(): Changed return type to void (was status_t and not used).
* _user_map_file(), _user_unmap_memory(): Verify that the address (if given) is
  page aligned.
* Reworked memory locking (wiring):
  - VMArea does now have a list of wired memory ranges and supports waiting for
    a range to be removed.
  - vm_soft_fault():
    - Added "wirePage" parameter that, if given, makes the function wire the
      page and return it.
    - Added "wiredRange" parameter (for calls from lock_memory_etc()) and made
      sure we never unmap wired pages. This could e.g. happen when a page from a
      lower cache was read-mapped and a write fault occurred. Now in such a
      situation the function waits for the page to be unwired and restarts.
  - All functions that manipulate areas in a way that could affect wired ranges
    do now either require the caller to make sure there are no wired ranges in
    the way or do that themselves. Added a few wait_if_*_is_wired() helper
    functions for that purpose.
  - lock_memory_etc():
    - Does now also work correctly when the range spans more than one area.
    - Adds VMAreaWiredRanges to the affected VMAreas and retains an address
      space reference (so that the address space won't be deleted as long as a
      wired range exists).
    - Resolved TODO: The area's caches are now locked when
      increment_page_wired_count() is called.
    - Resolved TODO: The race condition due to missing locking after looking up
      the page mapping is now prevented. We hold the cache locks (in case the
      page is already mapped) and the new vm_soft_fault() parameter allows us
      to get the page wired.
  - unlock_memory_etc(): Changes symmetrical to those in lock_memory_etc() and
    resolved all TODOs.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@36030 a95241bf-73f2-0310-859d-f6bbb57e9c96
2010-04-03 18:01:29 +00:00
Ingo Weinhold
fe0963a835 Added commands to the page access debugging panic()s.
git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@35904 a95241bf-73f2-0310-859d-f6bbb57e9c96
2010-03-18 16:19:13 +00:00
Ingo Weinhold
bd7645a12a * Made vm_page::state private and added accessor methods.
* Added kernel tracing for page state transitions.



git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@35538 a95241bf-73f2-0310-859d-f6bbb57e9c96
2010-02-20 15:57:44 +00:00
Ingo Weinhold
d36b9aef3e Added vm_page::IsMapped() for convenience.
git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@35531 a95241bf-73f2-0310-859d-f6bbb57e9c96
2010-02-20 12:29:42 +00:00
Ingo Weinhold
4bb4f79355 Added assert.
git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@35497 a95241bf-73f2-0310-859d-f6bbb57e9c96
2010-02-16 16:49:52 +00:00
Ingo Weinhold
40bb94819e * Removed useless return parameter from vm_remove_all_page_mappings().
* Added vm_clear_page_mapping_accessed_flags() and
  vm_remove_all_page_mappings_if_unaccessed(), which combine the functionality
  of vm_test_map_activation(), vm_clear_map_flags(), and
  vm_remove_all_page_mappings(), thus saving lots of calls to translation map
  methods. The backend is the new method
  VMTranslationMap::ClearAccessedAndModified().
* Started to make use of the cached page queue and changed the meaning of the
  other non-free queues slightly:
  - Active queue: Contains mapped pages that have been used recently.
  - Inactive queue: Contains mapped pages that have not been used recently. Also
    contains unmapped temporary pages.
  - Modified queue: Contains unmapped modified pages.
  - Cached queue: Contains unmapped unmodified pages (LRU sorted).
  Unless we're actually low on memory and actively do paging, modified and
  cached queues only contain non-temporary pages. Cached pages are considered
  quasi free. They still belong to a cache, but since they are unmodified and
  unmapped, they can be freed immediately. And this is what
  vm_page_[try_]reserve_pages() do now when there are no more actually free
  pages at hand. Essentially this means that pages storing cached file data,
  unless mmap()ped, no longer are considered used and don't contribute to page
  pressure. Paging will not happen as long there are enough free + cached pages
  available.
* Reimplemented the page daemon. It no longer scans all pages, but instead works
  the page queues. As long as the free pages situation is harmless, it only
  iterates through the active queue and deactivates pages that have not been
  used recently. When paging occurs it additionally scans the inactive queue and
  frees pages that have not been used recently.
* Changed the page reservation/allocation interface:
  vm_page_[try_]reserve_pages(), vm_page_unreserve_pages(), and
  vm_page_allocate_page() now take a vm_page_reservation structure pointer.
  The reservation functions initialize the structure -- currently consisting
  only of a count member for the number of still reserved pages.
  vm_page_allocate_page() decrements the count and vm_page_unreserve_pages()
  unreserves the remaining pages (if any). Advantages are that reservation/
  unreservation mismatches cannot occur anymore, that vm_page_allocate_page()
  can verify that the caller has indeed a reserved page left, and that there's
  no unnecessary pressure on the free page pool anymore. The only disadvantage
  is that the vm_page_reservation object needs to be passed around a bit.
* Reworked the page reservation implementation:
  - Got rid of sSystemReservedPages and sPageDeficit. Instead
    sUnreservedFreePages now actually contains the number of free pages that
    have not yet been reserved (it cannot become negative anymore) and the new
    sUnsatisfiedPageReservations contains the number of pages that are still
    needed for reservation.
  - Threads waiting for reservations do now add themselves to a waiter queue,
    which is ordered by descending priority (VM priority and thread priority).
    High priority waiters are served first when pages become available.
  Fixes #5328.
* cache_prefetch_vnode(): Would reserve one less page than allocated later, if
  the size wasn't page aligned.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@35393 a95241bf-73f2-0310-859d-f6bbb57e9c96
2010-02-03 18:53:52 +00:00
Ingo Weinhold
cf99b9ab60 Added VMCache::CacheRef() accessor.
git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@35391 a95241bf-73f2-0310-859d-f6bbb57e9c96
2010-02-03 18:46:29 +00:00
Ingo Weinhold
e65c400299 * Replaced the vm_page_allocate_page*() "pageState" parameter by a more
general "flags" parameter. It encodes the target state of the page -- so
  that the page isn't unnecessarily put in the wrong page queue first -- a
  flag whether the page should be cleared, and one to indicate whether the
  page should be marked busy.
* Added page state PAGE_STATE_CACHED. Not used yet.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@35333 a95241bf-73f2-0310-859d-f6bbb57e9c96
2010-01-29 15:54:40 +00:00
Ingo Weinhold
72382fa629 * Removed the page state PAGE_STATE_BUSY and instead introduced a vm_page::busy
flag. The obvious advantage is that one can still see what state a page is in
  and even move it between states while being marked busy.
* Removed the vm_page::is_dummy flag. Instead we mark marker pages busy, which
  in all cases has the same effect. Introduced a vm_page_is_dummy() that can
  still check whether a given page is a dummy page.
* vm_page_unreserve_pages(): Before adding to the system reserve make sure
  sUnreservedFreePages is non-negative. Otherwise we'd make nonexisting pages
  available for allocation. steal_pages() still has the same problem and it
  can't be solved that easily.
* map_page(): No longer changes the page state/mark the page unbusy. That's the
  caller's responsibility.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@35331 a95241bf-73f2-0310-859d-f6bbb57e9c96
2010-01-29 10:00:45 +00:00
Ingo Weinhold
deee8524b7 * Introduced {malloc,memalign,free}_etc() which take an additional "flags"
argument. They replace the previous special-purpose allocation functions
  (malloc_nogrow(), vip_io_request_malloc()).
* Moved the I/O VIP heap to heap.cpp accordingly.
* Added quite a bit of passing around of allocation flags in the VM,
  particularly in the VM*AddressSpace classes.
* Fixed IOBuffer::GetNextVirtualVec(): It was ignoring the VIP flag and always
  allocated on the normal heap.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@35316 a95241bf-73f2-0310-859d-f6bbb57e9c96
2010-01-27 12:45:53 +00:00
Ingo Weinhold
cff6e9e406 * The system now holds back a small reserve of committable memory and pages. The
memory and page reservation functions have a new "priority" parameter that
  indicates how deep the function may tap into that reserve. The currently
  existing priority levels are "user", "system", and "VIP". The idea is that
  user programs should never be able to cause a state that gets the kernel into
  trouble due to heavy battling for memory. The "VIP" level (not really used
  yet) is intended for allocations that are required to free memory eventually
  (in the page writer). More levels are thinkable in the future, like "user real
  time" or "user system server".
* Added "priority" parameters to several VMCache methods.
* Replaced the map_backing_store() "unmapAddressRange" parameter by a "flags"
  parameter.
* Added area creation flag CREATE_AREA_PRIORITY_VIP and slab allocator flag
  CACHE_PRIORITY_VIP indicating the importance of the request.
* Changed most code to pass the right priorities/flags.

These changes already significantly improve the behavior in low memory
situations. I've tested a bit with 64 MB (virtual) RAM and, while not
particularly fast and responsive, the system remains at least usable under high
memory pressure.
As a side effect the slab allocator can now be used as general memory allocator.
Not done by default yet, though.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@35295 a95241bf-73f2-0310-859d-f6bbb57e9c96
2010-01-26 14:44:58 +00:00
Ingo Weinhold
b4e5e49823 MemoryManager:
* Added support to do larger raw allocations (up to one large chunk (128 pages))
  in the slab areas. For an even larger allocation an area is created (haven't
  seen that happen yet, though).
* Added kernel tracing (SLAB_MEMORY_MANAGER_TRACING).
* _FreeArea(): Copy and paste bug: The meta chunks of the to be freed area
  would be added to the free lists instead of being removed from them. This
  would corrupt the lists and also lead to all kinds of misuse of meta chunks.

object caches:
* Implemented CACHE_ALIGN_ON_SIZE. It is no longer set for all small object
  caches, but the block allocator sets it on all power of two size caches.
* object_cache_reserve_internal(): Detect recursion and don't wait in such a
  case. The function could deadlock itself, since
  HashedObjectCache::CreateSlab() does allocate memory, thus potentially
  reentering.
* object_cache_low_memory():
  - I missed some returns when reworking that one in r35254, so the function
    might stop early and also leave the cache in maintenance mode, which would
    cause it to be ignored by object cache resizer and low memory handler from
    that point on.
  - Since ReturnSlab() potentially unlocks, the conditions weren't quite correct
    and too many slabs could be freed.
  - Simplified things a bit.
* object_cache_alloc(): Since object_cache_reserve_internal() does potentially
  unlock the cache, the situation might have changed and their might not be an
  empty slab available, but a partial one. The function would crash.
* Renamed the object cache tracing variable to SLAB_OBJECT_CACHE_TRACING.
* Renamed debugger command "cache_info" to "slab_cache" to avoid confusion with
  the VMCache commands.
* ObjectCache::usage was not maintained anymore since I introduced the
  MemoryManager. object_cache_get_usage() would thus always return 0 and the
  block cache would not be considered cached memory. This was only of
  informational relevance, though.

slab allocator misc.:
* Disable the object depots of block allocator caches for object sizes > 2 KB.
  Allocations of those sizes aren't so common that the object depots yield any
  benefit.
* The slab allocator is now fully self-sufficient. It allocates its bootstrap
  memory from the MemoryManager, and the hash tables for HashedObjectCaches use
  the block allocator instead of the heap, now.
* Added option to use the slab allocator for malloc() and friends
  (USE_SLAB_ALLOCATOR_FOR_MALLOC). Currently disabled. Works in principle and
  has virtually no lock contention. Handling for low memory situations is yet
  missing, though.
* Improved the output of some debugger commands.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@35283 a95241bf-73f2-0310-859d-f6bbb57e9c96
2010-01-25 13:46:58 +00:00
Ingo Weinhold
86c794e5c1 slab allocator:
* Implemented a more elaborated raw memory allocation backend (MemoryManager).
  We allocate 8 MB areas whose pages we allocate and map when needed. An area is
  divided into equally-sized chunks which form the basic units of allocation. We
  have areas with three possible chunk sizes (small, medium, large), which is
  basically what the ObjectCache implementations were using anyway.
* Added "uint32 flags" parameter to several of the slab allocator's object
  cache and object depot functions. E.g. object_depot_store() potentially wants
  to allocate memory for a magazine. But also in pure freeing functions it
  might eventually become useful to have those flags, since they could end up
  deleting an area, which might not be allowable in all situations. We should
  introduce specific flags to indicate that.
* Reworked the block allocator. Since the MemoryManager allocates block-aligned
  areas, maintains a hash table for lookup, and maps chunks to object caches,
  we can quickly find out which object cache a to be freed allocation belongs
  to and thus don't need the boundary tags anymore.
* Reworked the slab boot strap process. We allocate from the initial area only
  when really necessary, i.e. when the object cache for the respective
  allocation size has not been created yet. A single page is thus sufficient.

other:
* vm_allocate_early(): Added boolean "blockAlign" parameter. If true, the
  semantics is the same as for B_ANY_KERNEL_BLOCK_ADDRESS.
* Use an object cache for page mappings. This significantly reduces the
  contention on the heap bin locks.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@35232 a95241bf-73f2-0310-859d-f6bbb57e9c96
2010-01-21 23:10:52 +00:00
Ingo Weinhold
6379e53e2d vm_page no longer points directly to its containing cache, but rather to a
VMCacheRef object which points to the cache. This allows to optimize
VMCache::MoveAllPages(), since it no longer needs to iterate over all pages
to adjust their cache pointer. It can simple swap the cache refs of the two
caches instead.

Reduces the total -j8 Haiku image build time only marginally. The kernel time
drops almost 10%, though.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@35155 a95241bf-73f2-0310-859d-f6bbb57e9c96
2010-01-19 08:34:14 +00:00
Ingo Weinhold
3632eeedb9 * VMCache: Added a UserData attribute which can be used by the lock holder.
* Added "bool consumerLocked" parameter to VMCache::Unlock() and
  ReleaseRefAndUnlock(). Since Unlock() may cause the cache to be merged with
  a consumer cache, the flag is needed to prevent a deadlock in case the
  caller still holds a lock to the consumer. Hasn't been a problem yet, since
  that situation never occurred.
* VMCacheChainLocker: Reversed unlocking order to bottom-up. The other
  direction could cause a deadlock in case caches would be merged, since the
  locking order would be reversed. The way VMCacheChainLocker was used this
  didn't happen, though.
* fault_get_page(): While copying a page from a lower cache to the top cache,
  we do now unlock all caches but the top one, so we don't unnecessarily
  kill concurrency.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@35153 a95241bf-73f2-0310-859d-f6bbb57e9c96
2010-01-19 03:02:11 +00:00
Ingo Weinhold
f082f7f019 * Added vm_page::accessed flag. Works analogously to vm_page::modified.
* Reorganized the code for [un]mapping pages:
  - Added new VMTranslationMap::Unmap{Area,Page[s]}() which essentially do what
    vm_unmap_page[s]() did before, just in the architecture specific code, which
    allows for specific optimizations. UnmapArea() is for the special case that
    the complete area is unmapped. Particularly in case the address space is
    deleted, some work can be saved. Several TODOs could be slain.
  - Since they are only used within vm.cpp vm_map_page() and vm_unmap_page[s]()
    are now static and have lost their prefix (and the "preserveModified"
    parameter).
* Added VMTranslationMap::Protect{Page,Area}(). They are just inline wrappers
  for Protect().
* X86VMTranslationMap::Protect(): Make sure not to accidentally clear the
  accessed/dirty flags.
* X86VMTranslationMap::Unmap()/Protect(): Make page table skipping actually
  work. It was only skipping to the next page.
* Adjusted the PPC code to at least compile.

No measurable effect for the -j8 Haiku image build time, though the kernel time
drops minimally.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@35089 a95241bf-73f2-0310-859d-f6bbb57e9c96
2010-01-15 22:32:51 +00:00
Ingo Weinhold
c6aa013564 Changed VMTranslationMap::Lock()/Unlock() return types to the usual.
git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@35075 a95241bf-73f2-0310-859d-f6bbb57e9c96
2010-01-14 20:44:29 +00:00
Ingo Weinhold
bcc2c157a1 Refactored vm_translation_map:
* Pulled the physical page mapping functions out of vm_translation_map into
  a new interface VMPhysicalPageMapper.
* Renamed vm_translation_map to VMTranslationMap and made it a proper C++
  class. The functions in the operations vector have become methods.
* Added class GenericVMPhysicalPageMapper implementing VMPhysicalPageMapper
  as far as possible (without actually writing new code).
* Adjusted the x86 and the PPC specifics accordingly (untested for the
  latter). For the other architectures the build is, I'm afraid, seriously
  broken.

The next steps will modify and extend the VMTranslationMap interface, so that
it will be possible to fix the bugs in vm_unmap_page[s]() and employ
architecture specific optimizations.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@35066 a95241bf-73f2-0310-859d-f6bbb57e9c96
2010-01-14 03:26:12 +00:00
Ingo Weinhold
946325051b * Added boolean "alreadyWired" parameter to vm_map_physical_memory().
* ioapic_init(): map_physical_memory() was called for already mapped
  addresses. This worked fine, but only because the x86 page mapping code
  didn't mind.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@35059 a95241bf-73f2-0310-859d-f6bbb57e9c96
2010-01-13 22:02:21 +00:00
Ingo Weinhold
3b8c056da0 Should have been part of r35023: Introduction of the vm_page::modified flag.
git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@35024 a95241bf-73f2-0310-859d-f6bbb57e9c96
2010-01-12 01:24:40 +00:00
Ingo Weinhold
451ca8b4b4 PAGE_TYPE_GUARD was unused and for the other two types a simple one bit flag
suffices. Therefore replaced vm_page::type by vm_page::is_dummy.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@35013 a95241bf-73f2-0310-859d-f6bbb57e9c96
2010-01-11 19:13:14 +00:00
Ingo Weinhold
d7455de2f6 allocate_page_run(): Use temporary lists to store the pages we're allocating.
This makes appending the pages to the active queue more efficient and we
don't need the vm_page::is_cleared bit anymore.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@35011 a95241bf-73f2-0310-859d-f6bbb57e9c96
2010-01-11 16:36:31 +00:00
Ingo Weinhold
509f1174ce * Added Size() method.
* Added Debug{First,Next}() methods to allow easy iteration through the
  address spaces in kernel debugger commands.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@34978 a95241bf-73f2-0310-859d-f6bbb57e9c96
2010-01-09 20:21:43 +00:00
Ingo Weinhold
77690f288e Added VMCache::SwitchFromReadLock(), atomically unlocking a read lock and
starting to lock the cache.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@34936 a95241bf-73f2-0310-859d-f6bbb57e9c96
2010-01-07 15:32:28 +00:00
Ingo Weinhold
8ab820f076 VMAddressSpace::Put() is too hot to always write lock the address spaces
table. It is now inline and uses double-checked locking. This reduces the
contention on the lock to insignificant. Total -j8 Haiku image build speedup
is marginal, but the total kernel time drops 12%.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@34934 a95241bf-73f2-0310-859d-f6bbb57e9c96
2010-01-07 14:09:56 +00:00
Ingo Weinhold
3cd2094396 * Added new debug feature (DEBUG_PAGE_ACCESS) to detect invalid concurrent
access to a vm_page. It is basically an atomically accessed thread ID field
  in the vm_page structure, which is explicitly set by macros marking the
  critical sections. As a first positive effect I had to review quite a bit of
  code and found several issues.
* Added several TODOs and comments. Some harmless ones, but also a few
  troublesome ones in vm.cpp regarding page unmapping.
* file_cache: PrecacheIO::Prepare()/read_into_cache: Removed superfluous
  vm_page_allocate_page() return value checks. It cannot fail anymore.
* Removed the heavily contended "pages" lock. We use different policies now:
  - sModifiedTemporaryPages is accessed atomically.
  - sPageDeficitLock and sFreePageCondition are protected by a new mutex.
  - The page queues have individual locks (mutexes).
  - Renamed set_page_state_nolock() to set_page_state(). Unless the caller says
    otherwise, it does now lock the affected pages queues itself. Also changed
    the return value to void -- we panic() anyway.
* set_page_state(): Add free/clear pages to the beginning of their respective
  queues as this is more cache-friendly.
* Pages with the states PAGE_STATE_WIRED or PAGE_STATE_UNUSED are no longer
  in any queue. They were in the "active" queue, but there's no good reason
  to have them there. In case we decide to let the page daemon work the queues
  (like FreeBSD) they would just be in the way.
* Pulled the common part of vm_page_allocate_page_run[_no_base]() into a helper
  function. Also fixed a bug I introduced previously: The functions must not
  vm_page_unreserve_pages() on success, since they remove the pages from the
  free/clear queue without decrementing sUnreservedFreePages.
* vm_page_set_state(): Changed return type to void. The function cannot really
  fail and no-one was checking it anyway.
* vm_page_free(), vm_page_set_state(): Added assertion: The page must not be
  free/clear before. This is implied by the policy that no-one is allowed to
  access free/clear pages without holding the respective queue's lock, which is
  not the case at this point. This found the bug fixed in r34912.
* vm_page_requeue(): Added general assertions. panic() when requeuing of
  free/clear pages is requested. Same reason as above.
* vm_clone_area(), B_FULL_LOCK case: Don't map busy pages. The implementation is
  still not correct, though.

My usual -j8 Haiku build test runs another 10% faster, now. The total kernel
time drops about 18%. As hoped the new locks have only a fraction of the old
"pages" lock contention. Other locks lead the "most wanted list" now.



git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@34933 a95241bf-73f2-0310-859d-f6bbb57e9c96
2010-01-07 02:37:05 +00:00
Ingo Weinhold
5800e8a486 * Moved the page management functionality into its own file.
* Renamed page_queue to VMPageQueue and made it a proper C++ class. Use
  DoublyLinkedList instead of own list code.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@34874 a95241bf-73f2-0310-859d-f6bbb57e9c96
2010-01-03 17:10:32 +00:00
Ingo Weinhold
355dc6bef4 Inlined several VMCache methods.
git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@34837 a95241bf-73f2-0310-859d-f6bbb57e9c96
2010-01-01 17:09:23 +00:00
Ingo Weinhold
1021fd2826 * agp_gart(): Use vm_page_[un]reserve_pages().
* Removed unused vm_page_allocate_pages().
* Removed now unused (always true) "reserved" parameter from
  vm_page_allocate_page().
* Removed unused (always false) "stealActive" parameter from steal_page().


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@34836 a95241bf-73f2-0310-859d-f6bbb57e9c96
2010-01-01 16:37:14 +00:00
Ingo Weinhold
eb8dc1ebfb * Removed DEBUG_PAGE_CACHE_TRANSITIONS debugging.
* Added VMCache::MovePage() and MoveAllPages() to move pages between caches.
* VMAnonymousCache:
  - _MergeSwapPages(): Avoid doing anything, if neither cache has swapped out
    pages.
  - _MergeSwapPages() does now also remove source cache pages that are
    shadowed by consumer swap pages. This allows us to call _MergeSwapPages()
    before _MergePagesSmallerSource(), save the swap page shadowing check
    there and get rid of the vm_page::merge_swap flag. This is an
    optimization based on the assumption that usually none or only few pages
    are swapped out, so we save a lot of checks.
  - Implemented _MergePagesSmallerConsumer() as an alternative to
    _MergePagesSmallerSource(). The former is used when the source cache has
    more pages than the consumer cache. It iterates over the consumer cache's
    pages, moves them to the source and finally moves all pages back to the
    consumer. The final move is relatively cheap (though unfortunately we
    still have to update all pages' vm_page::cache field), so that overall we
    save iterations of the main loop with the more expensive checks.

The optimizations particularly improve the common fork()+exec*() situations.
fork() uses CoW, which is implemented by putting two new empty caches between
the to be copied area and its cache. exec*() destroys one copy of the area,
its cache and thus causes merging of the other new cache with the old cache.
Since this usually happens in a very short time, the old cache does still
contain many pages and the new cache only few. Previously the many pages were
all checked and moved individually. Now we do that for the few pages instead.

A very extreme example of this situation is the Haiku image build. jam has a
huge heap (> 200 MB) and it fork()s+exec*()s for every action to be executed.
Since during the cache merging the cache is locked, any write access to a
heap page causes jam to block until the cache merging is done. Formerly that
took so long that it killed a lot of parallelism in multi-job builds. That
could be observed particularly well when lots of small actions where executed
(like the Link, XRes, Mimeset, SetType, SetVersion combos when building
executables/libraries/add-ons). Those look dramatically better now.
The overall speed improvement for a -j8 image build on my machine is only
about 15%, though.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@34784 a95241bf-73f2-0310-859d-f6bbb57e9c96
2009-12-27 16:14:13 +00:00
Ingo Weinhold
2e74d74f4f * Added method VMCache::TransferAreas() moving areas from one cache to
another. The code originates from vm_copy_on_write_area(). We now generate
  the VM cache tracing entries, though.
* count_writable_areas() -> VMCache::CountWritableAreas()
* Added debugger command "cache_stack" which is enabled when VM cache tracing
  is enabled. It prints the source caches of a given cache or area at the
  time of a specified tracing entry.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@34751 a95241bf-73f2-0310-859d-f6bbb57e9c96
2009-12-22 22:00:35 +00:00
Ingo Weinhold
522c2f19d4 * Added a simple mechanism to wait for events to VMCache. WaitForPageEvents()
waits for certain events on a given page, NotifyPageEvents() wakes up
  waiting threads respectively.
* Used the new feature instead of condition variables for waiting on busy
  pages. We save publishing and unpublishing of a condition variable whenever
  a page is marked busy. There's only something to do, if there's at least
  one thread waiting in the list of the respective cache. The general
  assumption is that this is only rarely the case and even if it happens,
  there should be only very few threads.
* Added an apparently missing notification in cache_io(). At least I didn't
  see the reason for it not being there.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@34537 a95241bf-73f2-0310-859d-f6bbb57e9c96
2009-12-07 15:42:08 +00:00
Ingo Weinhold
6440406a59 Style changes
git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@34536 a95241bf-73f2-0310-859d-f6bbb57e9c96
2009-12-07 14:28:56 +00:00
Ingo Weinhold
be7328a9f6 Moved VMCache related definitions to <vm/VMCache.h>.
git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@34535 a95241bf-73f2-0310-859d-f6bbb57e9c96
2009-12-07 14:14:21 +00:00
Ingo Weinhold
e55886c3a3 Make iteration safe. VMKernelAddressSpace::Next() doesn't like NULL pointers.
git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@34531 a95241bf-73f2-0310-859d-f6bbb57e9c96
2009-12-07 01:56:01 +00:00
Ingo Weinhold
40cd019ea0 * Renamed VMAddressSpace::ResizeArea{Head,Tail}() to ShrinkArea{Head,Tail}()
to clarify that they never enlarge the area.
* Reimplemented VMKernelAddressSpace. It is somewhat inspired by Bonwick's
  vmem resource allocator (though we have different requirements):
  - We consider the complete address space to be divided into contiguous
    ranges of type free, reserved, or area, each range being represented by
    a VMKernelAddressRange object.
  - The range objects are managed in an AVL tree and a doubly linked list
    (the latter only for faster iteration) sorted by address. This provides
    O(log(n)) lookup, insertion and removal.
  - For each power of two size we maintain a list of free ranges of at least
    that size. Thus for the most common case of B_ANY*_ADDRESS area
    allocation, we find a free range in constant time (the rest of the
    processing being O(log(n))) with a rather good fit. This should also
    help avoiding address space fragmentation.
  While the new implementation should be faster, particularly with an
  increasing number of areas, I couldn't measure any difference in the -j2
  haiku build. From a cursory test the -j8 build hasn't tangibly benefitted
  either.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@34528 a95241bf-73f2-0310-859d-f6bbb57e9c96
2009-12-06 17:18:04 +00:00
Ingo Weinhold
2c1886aeae * Added VMArea subclasses VM{Kernel,User}Area and moved the address space list
link to them.
* VM{Kernel,User}AddressSpace manage the respective VMArea subclass now, and
  VMAddressSpace has grown factory methods {Create,Delete}Area.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@34493 a95241bf-73f2-0310-859d-f6bbb57e9c96
2009-12-04 17:07:16 +00:00
Ingo Weinhold
e2518ddbb1 Made VMAddressSpace an abstract base class and moved the area management into
new derived classes VM{Kernel,User}AddressSpace. Currently those are
identical, but that will change.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@34492 a95241bf-73f2-0310-859d-f6bbb57e9c96
2009-12-04 14:45:08 +00:00
Ingo Weinhold
38a97b2c36 Moved all knowledge of reserved areas from vm.cpp to VMAddressSpace. It's a
pure address space feature, so it should be handled there.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@34491 a95241bf-73f2-0310-859d-f6bbb57e9c96
2009-12-04 13:33:25 +00:00