Commit Graph

75 Commits

Author SHA1 Message Date
enami 694a0fec54 When loaned page become ownerless as a result of freeing, it should be
dequeue'ed from pageq.  Fix provided by chuq.
2002-05-15 00:19:12 +00:00
enami bb41d19bca In the function uvm_page_own(), clear owner_tag after assertion so that
we can see the owner when assertion failed.  Some indentation fix while
I'm here.
2002-02-20 07:06:56 +00:00
chs ef57a67ca1 fix locking for loaning. in general we should be looking at the page's
uobject and uanon pointers rather than at the PQ_ANON flag to determine
which lock to hold, since PQ_ANON can be clear even when the anon's lock
is the one which we should hold (if the page was loaned from an object
and then freed by the object).
2001-12-31 19:21:36 +00:00
chs 849c9b2bfd add {anon,file,exec}max as a upper bound on the amount of memory that
will be allocated for the respective usage types when there is contention
for memory.

replace "vnode" and "vtext" with "file" and "exec" in uvmexp field names
and sysctl names.
2001-12-09 03:07:19 +00:00
lukem b616d1ca1d add RCSIDs, and in some cases, slightly cleanup #include order 2001-11-10 07:36:59 +00:00
chs 2ed88fe090 several changes prompted by loaning problems:
- fix the loaned case in uvm_pagefree().
 - redo uvmexp.swpgonly accounting to work with page loaning.
   add an assertion before each place we adjust uvmexp.swpgonly.
 - fix uvm_km_pgremove() to always free any swap space associated with
   the range being removed.
 - get rid of UVM_LOAN_WIRED flag.  instead, we just make sure that
   pages loaned to the kernel are never on the page queues.
   this allows us to assert that pages are not loaned and wired
   at the same time.
 - add yet more assertions.
2001-11-06 08:07:49 +00:00
simonb 82649768b7 Change some unsigned int variables and parameters to plain ints so
that all usages of those agree on unsigned vs. signed.
2001-11-06 06:31:06 +00:00
chs 80373b7e54 don't depend on other headers to include sys/proc.h for us. 2001-09-28 11:59:51 +00:00
chs 64c6d1d2dc a whole bunch of changes to improve performance and robustness under load:
- remove special treatment of pager_map mappings in pmaps.  this is
   required now, since I've removed the globals that expose the address range.
   pager_map now uses pmap_kenter_pa() instead of pmap_enter(), so there's
   no longer any need to special-case it.
 - eliminate struct uvm_vnode by moving its fields into struct vnode.
 - rewrite the pageout path.  the pager is now responsible for handling the
   high-level requests instead of only getting control after a bunch of work
   has already been done on its behalf.  this will allow us to UBCify LFS,
   which needs tighter control over its pages than other filesystems do.
   writing a page to disk no longer requires making it read-only, which
   allows us to write wired pages without causing all kinds of havoc.
 - use a new PG_PAGEOUT flag to indicate that a page should be freed
   on behalf of the pagedaemon when it's unlocked.  this flag is very similar
   to PG_RELEASED, but unlike PG_RELEASED, PG_PAGEOUT can be cleared if the
   pageout fails due to eg. an indirect-block buffer being locked.
   this allows us to remove the "version" field from struct vm_page,
   and together with shrinking "loan_count" from 32 bits to 16,
   struct vm_page is now 4 bytes smaller.
 - no longer use PG_RELEASED for swap-backed pages.  if the page is busy
   because it's being paged out, we can't release the swap slot to be
   reallocated until that write is complete, but unlike with vnodes we
   don't keep a count of in-progress writes so there's no good way to
   know when the write is done.  instead, when we need to free a busy
   swap-backed page, just sleep until we can get it busy ourselves.
 - implement a fast-path for extending writes which allows us to avoid
   zeroing new pages.  this substantially reduces cpu usage.
 - encapsulate the data used by the genfs code in a struct genfs_node,
   which must be the first element of the filesystem-specific vnode data
   for filesystems which use genfs_{get,put}pages().
 - eliminate many of the UVM pagerops, since they aren't needed anymore
   now that the pager "put" operation is a higher-level operation.
 - enhance the genfs code to allow NFS to use the genfs_{get,put}pages
   instead of a modified copy.
 - clean up struct vnode by removing all the fields that used to be used by
   the vfs_cluster.c code (which we don't use anymore with UBC).
 - remove kmem_object and mb_object since they were useless.
   instead of allocating pages to these objects, we now just allocate
   pages with no object.  such pages are mapped in the kernel until they
   are freed, so we can use the mapping to find the page to free it.
   this allows us to remove splvm() protection in several places.

The sum of all these changes improves write throughput on my
decstation 5000/200 to within 1% of the rate of NetBSD 1.5
and reduces the elapsed time for "make release" of a NetBSD 1.5
source tree on my 128MB pc to 10% less than a 1.5 kernel took.
2001-09-15 20:36:31 +00:00
chris 0e7661f023 Update pmap_update to now take the updated pmap as an argument.
This will allow improvements to the pmaps so that they can more easily defer expensive operations, eg tlb/cache flush, til the last possible moment.

Currently this is a no-op on most platforms, so they should see no difference.

Reviewed by Jason.
2001-09-10 21:19:08 +00:00
thorpej 8217ab697c Since a page can be on only one of ACTIVE or INACTIVE queues at
any given time, turn two consecutive if statements into an if-else-if
construct.
2001-06-27 23:57:16 +00:00
thorpej 27f66d3bf7 Macro'ize the code that checks the free and inactive thresholds and
wakes the pagedaemon.
2001-06-27 21:18:34 +00:00
chs 11a9651c8f replace vm_page_t with struct vm_page *. 2001-05-26 21:27:10 +00:00
chs 3845302904 remove trailing whitespace. 2001-05-25 04:06:11 +00:00
ross 892627dd05 Merge the swap-backed and object-backed inactive lists. 2001-05-22 00:44:44 +00:00
thorpej 31fafb678f Support dynamic sizing of the page color bins. We also support
dynamically re-coloring pages; as machine-dependent code discovers
the size of the system's caches, it may call uvm_page_recolor() with
the new number of colors to use.  If the new mumber of colors is
smaller (or equal to) the current number of colors, then uvm_page_recolor()
is a no-op.

The system defaults to one bucket if machine-dependent code does not
initialize uvmexp.ncolors before uvm_page_init() is called.

Note that the number of color bins should be initialized to something
reasonable as early as possible -- for many early memory allocations,
we live with the consequences of the page choice for the lifetime of
the boot.
2001-05-02 01:22:19 +00:00
thorpej 01e2971ba2 Add the number of page colors to uvmexp. 2001-05-01 19:36:56 +00:00
enami 1132ef7f20 Use simple do {} while () loop instead of for {} loop + extra test/variable. 2001-05-01 14:02:56 +00:00
enami d211385f8a Fix second level indentation in recent commit. 2001-05-01 13:42:34 +00:00
thorpej cf67ac7122 Per discussion w/ chuck and chuck, restructure the md page stuff
to use a structure called "vm_page_md", and use __HAVE_VM_PAGE_MD
and __HAVE_PMAP_PHYSSEG.
2001-05-01 02:19:13 +00:00
thorpej 2b27ac7a99 Add a VM_MDPAGE_MEMBERS macro that defines pmap-specific data for
each vm_page structure.  Add a VM_MDPAGE_INIT() macro to init this
data when pages are initialized by UVM.  These macros are mandatory,
but ports may #define them to nothing if they are not needed/used.

This deprecates struct pmap_physseg.  As a transitional measure,
allow a port to #define PMAP_PHYSSEG so that it can continue to
use it until its pmap is converted to use VM_MDPAGE_MEMBERS.

Use all this stuff to eliminate a lot of extra work in the Alpha
pmap module (it's smaller and faster now).  Changes to other pmap
modules will follow.
2001-04-29 22:44:31 +00:00
thorpej cda7baa0d5 Implement page coloring, using a round-robin bucket selection
algorithm (Solaris calls this "Bin Hopping").

This implementation currently relies on MD code to define a
constant defining the number of buckets.  This will change
reasonably soon (MD code will be able to dynamically size
the bucket array).
2001-04-29 04:23:20 +00:00
thorpej 1c3a62e066 Sprinkle pmap_update() calls after calls to:
- pmap_enter()
- pmap_remove()
- pmap_protect()
- pmap_kenter_pa()
- pmap_kremove()
as described in pmap(9).

These calls are relatively conservative.  It may be possible to
optimize these a little more.
2001-04-24 04:30:50 +00:00
thorpej cfb5c7ed9f Make pmap_virtual_space() a required pmap function, even on platforms
which have pmap_steal_memory().  This is to reduce the API differences
between pmaps that implement pmap_steal_memory() and pmaps which do
not.

Note that pmap_steal_memory() needs to adjust *vstartp and/or
*vendp only if it used addresses within the range provided to UVM
via the pmap_virtual_space() call.  I.e. it is not necessary to do
so in any current pmap_steal_memory() implementation.
2001-04-22 17:22:57 +00:00
chs 83d071a318 add UBC memory-usage balancing. we track the number of pages in use for
each of the basic types (anonymous data, executable image, cached files)
and prevent the pagedaemon from reusing a given page if that would reduce
the count of that type of page below a sysctl-setable minimum threshold.
the thresholds are controlled via three new sysctl tunables:
vm.anonmin, vm.vnodemin, and vm.vtextmin.  these tunables are the
percentages of pageable memory reserved for each usage, and we do not allow
the sum of the minimums to be more than 95% so that there's always some
memory that can be reused.
2001-03-09 01:02:10 +00:00
thorpej 1cdff48674 Put the extern decl of uvm_vnodeops in uvm_object.h 2001-01-28 22:23:04 +00:00
thorpej 5849b86934 Use UVM_OBJ_IS_VNODE(). 2001-01-28 22:14:52 +00:00
thorpej 13759f5310 Sprinkle some assertions:
amap_free(): Assert that the amap is locked.
amap_share_protect(): Assert that the amap is locked.
amap_wipeout(): Assert that the amap is locked.
uvm_anfree(): Assert that the anon has a reference count of 0 and is
              not locked.
uvm_anon_lockloanpg(): Assert that the anon is locked.
anon_pagein(): Assert that the anon is locked.
uvmfault_anonget(): Assert that the anon is locked.
uvm_pagealloc_strat(): Assert that the uobj or the anon is locked

And fix the problems these have uncovered:
amap_cow_now(): Lock the new anon after allocating it, and unref and
                unlock it (rather than lock!) before freeing it in case
                of an error condition.  This should fix a problem reported
		by Dan Carosone using cdrecord on an i386 MP kernel.
uvm_fault(): Case1B -- Lock the new anon afer allocating it, and unlock
             it later when we unlock the old anon.
	     Case2 -- Lock the new anon after allocating it, and unlock
	     it later by passing it to uvmfault_unlockall() (we set anon
	     to NULL if we're not doing a promote fault).
2001-01-23 01:56:16 +00:00
thorpej f4395a4eae splimp() -> splvm() 2001-01-14 02:10:01 +00:00
chs eeabe3f90d make sure that pages are on an paging queue before unlocking them. 2000-12-01 09:54:42 +00:00
simonb 33999a4224 Move uvm_pgcnt_vnode and uvm_pgcnt_anon into uvmexp (as vnodepages and
anonpages), and add vtextpages which is currently unused but will be
used to trace the number of pages used by vtext vnodes.
2000-11-30 11:04:43 +00:00
chs aeda8d3b77 Initial integration of the Unified Buffer Cache project. 2000-11-27 08:39:39 +00:00
christos 413d7641a1 Give a hint to the user on why we failed. 2000-11-09 19:15:28 +00:00
mrg 4f75145ec1 s/vm/uvm/ in a bunch of error messages. 2000-10-05 00:37:50 +00:00
thorpej b008f5f25a Make PMAP_PAGEIDLEZERO() return a boolean value. FALSE indidcates
that the page being zero'd was not completed and that page zeroing
should be aborted.  This may be used by machine-dependent code doing
slow page access to reduce the latency of running a process that has
become runnable while in the middle of doing a slow page zero.
2000-09-21 17:46:04 +00:00
thorpej d85e53c7cc MALLOC() is not to be used for variable-sized allocations. 2000-08-02 20:25:11 +00:00
mrg dea44a9ec4 remove include of <vm/vm.h> 2000-06-27 17:29:17 +00:00
mrg 2f159a1bac remove/move more mach vm header files:
<vm/pglist.h> -> <uvm/uvm_pglist.h>
	<vm/vm_inherit.h> -> <uvm/uvm_inherit.h>
	<vm/vm_kern.h> -> into <uvm/uvm_extern.h>
	<vm/vm_object.h> -> nothing
	<vm/vm_pager.h> -> into <uvm/uvm_pager.h>

also includes a bunch of <vm/vm_page.h> include removals (due to redudancy
with <vm/vm.h>), and a scattering of other similar headers.
2000-06-26 14:20:25 +00:00
soda d5b3fb3ce1 fix printf format mismatch, when paddr_t becomes (long long) on arc port. 2000-06-09 04:43:19 +00:00
thorpej 65184f2470 Change the comment before the vm_page_zero_enable global to indicate
what it will now be used for.
2000-05-29 19:25:56 +00:00
thorpej a7d0570e67 First sweep at scheduler state cleanup. Collect MI scheduler
state into global and per-CPU scheduler state:

	- Global state: sched_qs (run queues), sched_whichqs (bitmap
	  of non-empty run queues), sched_slpque (sleep queues).
	  NOTE: These may collectively move into a struct schedstate
	  at some point in the future.

	- Per-CPU state, struct schedstate_percpu: spc_runtime
	  (time process on this CPU started running), spc_flags
	  (replaces struct proc's p_schedflags), and
	  spc_curpriority (usrpri of processes on this CPU).

	- Every platform must now supply a struct cpu_info and
	  a curcpu() macro.  Simplify existing cpu_info declarations
	  where appropriate.

	- All references to per-CPU scheduler state now made through
	  curcpu().  NOTE: this will likely be adjusted in the future
	  after further changes to struct proc are made.

Tested on i386 and Alpha.  Changes are mostly mechanical, but apologies
in advance if it doesn't compile on a particular platform.
2000-05-26 21:19:19 +00:00
thorpej 9ec517a68e Changes necessary to implement pre-zero'ing of pages in the idle loop:
- Make page free lists have two actual queues: known-zero pages and
  pages with unknown contents.
- Implement uvm_pageidlezero().  This function attempts to zero up to
  the target number of pages until the target has been reached (currently
  target is `all free pages') or until whichqs becomes non-zero (indicating
  that a process is ready to run).
- Define a new hook for the pmap module for pre-zero'ing pages.  This is
  used to zero the pages using uncached access.  This allows us to zero
  as many pages as we want without polluting the cache.

In order to use this feature, each platform must add the appropropriate
glue in their idle loop.
2000-04-24 17:12:00 +00:00
thorpej 2c48131727 Add UVM_PGA_ZERO which instructs uvm_pagealloc{,_strat}() to return a
zero'd, ! PG_CLEAN page, as if it were uvm_pagezero()'d.
2000-04-10 00:28:05 +00:00
thorpej 2bc5adb20e Instead of checking vm_physmem[<physseg>].pgs to determine if
uvm_page_init() has completed, add a boolean uvm.page_init_done,
and test against that.  Use this same boolean (rather than
pmap_initialized) in pmap_growkernel() to determine if we are
being called via uvm_page_init() to grow the kernel address space.

This fixes a problem on some i386 configurations where pmap_init()
itself was needing to have the kernel page table grown, and since
pmap_initialized was not yet set to TRUE, pmap_growkernel() was
choosing the wrong code path.

Fix tested by Havard Eidnes.
2000-04-02 20:39:14 +00:00
kleink 6e5b64c8a0 Merge parts of chs-ubc2 into the trunk:
Add a new type voff_t (defined as a synonym for off_t) to describe offsets
into uvm objects, and update the appropriate interfaces to use it, the
most visible effect being the ability to mmap() file offsets beyond
the range of a vaddr_t.

Originally by Chuck Silvers; blame me for problems caused by merging this
into non-UBC.
2000-03-26 20:54:45 +00:00
thorpej 9671588a30 Allocate the page buckets out of kernel_map, not kmem_map. Saves 16
or so kmem_map pages on a 32MB SPARCstation 2.
2000-02-13 03:34:40 +00:00
eeh c0ac678704 I should have made uvm_page_physload() take paddr_t's instead of vaddr_t's.
Also, add uvm_coredump32().
1999-12-30 16:09:47 +00:00
drochner 38e73c0c99 in uvm_page_physget(), try the vm_physmem[] chunks in the order of their
"free_list" attributes, to save DMA memory
1999-12-01 16:08:32 +00:00
thorpej 63494b0b50 Avoid an integer overflow on systems w/ more than 2G of RAM. 1999-11-30 18:34:23 +00:00
drochner b1f2453dee add a diagnostic panic to catch illegal memory ranges passed to
uvm_page_physload()
1999-11-24 18:28:49 +00:00