- Don't need to count anonpages+filepages any more; clean+unknown+dirty for
each kind of page can be summed to get the totals.
- Track the number of free pages with a counter so that it's one less thing
for the allocator to do, which opens up further options there.
- Remove cpu_count_sync_one(). It has no users and doesn't save a whole lot.
For the cheap option, give cpu_count_sync() a boolean parameter indicating
that a cached value is okay, and rate limit the updates for cached values
to hz.
cached value will do, or if the very latest total must be fetched. It can
be called thousands of times a second and fetching the totals impacts not
only the calling LWP but other CPUs doing unrelated activity in the VM
system.
Use the dedicated reference counting routines.
Change the type of struct vmspace::vm_refcnt and struct vm_map::ref_count
to volatile.
Remove the unnecessary vm->vm_map.misc_lock locking in process_domem().
Reviewed by <ad>
parameters can't change part way through a search: move the "uobj" and
"flags" arguments over to uvm_page_array_init() and store those with the
array.
- With that, detect when it's not possible to find any more pages in the
tree with the given search parameters, and avoid repeated tree lookups if
the caller loops over uvm_page_array_fill_and_peek().
it works basically the same way as !direct minus temporary mappings, and
there are no concurrency issues.
- ubc_alloc_direct(): In the PGO_OVERWRITE case blocks are allocated
beforehand. Avoid waking or activating pages unless needed.
or one too few pages can be mapped.
- In ubc_release() with UBC_FAULTBUSY, chances are that pages are newly
allocated and freshly enqueued, so avoid uvm_pageactivate() if possible
- Keep track of the pages mapped in ubc_alloc() in an array on the stack,
and use this to avoid calling pmap_extract() in ubc_release().
Not entirely sure the rest of the swap system is MP-safe, but let's
not make it worse!
XXX Why is swap_syscall_lock an rwlock? We don't seem to take the
reader lock ever.
Also problems with tmpfs+nfs noted by hannken@.
Don't pass PGO_ALLPAGES to pgo_get, and ignore PGO_DONTCARE in the
!PGO_LOCKED case. In uao_get() have uvm_pagealloc() take care of page
zeroing and release busy pages on error.
- Make PGO_LOCKED getpages imply PGO_NOBUSY and remove the latter. Mark
pages busy only when there's actually I/O to do.
- When doing COW on a uvm_object, don't mess with neighbouring pages. In
all likelyhood they're already entered.
- Don't mess with neighbouring VAs that have existing mappings as replacing
those mappings with same can be quite costly.
- Don't enqueue pages for neighbour faults unless not enqueued already, and
don't activate centre pages unless uvmpdpol says its useful.
Also:
- Make PGO_LOCKED getpages on UAOs work more like vnodes: do gang lookup in
the radix tree, and don't allocate new pages.
- Fix many assertion failures around faults/loans with tmpfs.
the allocator's buckets, instead of doing round robin distribution. There
are open questions here but this is better than doing nothing.
- Kernel reserve pages are for the kernel not realtime threads.
No functional change intended, except that the symbol that was
previously `uvm_swap_encryption' is now `uvm_swap_encrypt', backing
the sysctl knob `vm.swap_encrypt'.
Enabled by sysctl -w vm.swap_encrypt=1. Key is generated lazily when
we first need to swap a page. Key is chosen independently for each
swap device. The ith swap page is encrypted with AES256-CBC using
AES256_k(le32enc(i) || 0^96) as the initialization vector. Can be
changed at any time; no need for compatibility with on-disk formats.
Costs one bit of memory per page in each swapdev, plus a few hundred
bytes per swapdev to store the expanded AES key.
Shoulda done this decades ago! Plan to enable this by default;
performance impact is unlikely to matter because it only happens when
you're already swapping anyway. Much easier to set up than cgd, so
we can rip out all the documentation about carefully setting up
random-keyed cgd at the right time.
- In uvm_voaddr_release(), if the anon ref count drops to 0, call
uvm_anfree() rather than uvm_anon_release(). Unconditionally drop
the anon lock, and release the extra hold on the anon lock obj.
Fixes a panic that occurs if the backing store for a futex backed by
an anon memory location is unmapped while a thread is waiting in the
futex.
Add a test case that reproduced the panic to verify that it's fixed.
- Add new flag UBC_ISMAPPED which tells ubc_uiomove() the object is mmap()ed
somewhere. Use it to decide whether to do direct-mapped copy, rather than
poking around directly in the vnode in ubc_uiomove(), which is ugly and
doesn't work for tmpfs. It would be nicer to contain all this in UVM but
the filesystem provides the needed locking here (VV_MAPPED) and to
reinvent that would suck more.
- Rename UBC_UNMAP_FLAG() to UBC_VNODE_FLAGS(). Pass in UBC_ISMAPPED where
appropriate.
