Commit Graph

59 Commits

Author SHA1 Message Date
thorpej f1c14428bb Do the reentrancy checking if POOL_DIAGNOSTIC, not DIAGNOSTIC. Prevents
ABI change for diagnostic vs. non-diagnostic kernels.
2001-06-05 18:51:03 +00:00
thorpej 47c29f3cbe Assert that no locks are held if we're called with PR_WAITOK.
From Bill Sommerfeld.
2001-06-05 04:40:39 +00:00
sommerfeld b4c4ecb455 Make this build again ifdef DIAGNOSTIC (oops) 2001-05-13 17:17:34 +00:00
sommerfeld 5b2241ad77 Remove pool reentrancy testing overhead unless DIAGNOSTIC is defined.
Previously, we passed __FILE__ and __LINE__ on all pool_get/pool_set calls.

This change results in a measured 1.2% performance improvement in
ping-flood packets-per-second as reported by ping(8).
2001-05-13 17:06:58 +00:00
thorpej 2f58df9a8f Rearrange the code that adds pages of objects to the pool; require
that the caller allocate the pool_item_header when it allocates the
pool page, so we can avoid a locking pitfall (sleeping with a simple
lock held).

Also revive pool_prime(), as there are some letigimate uses of it,
but in doing so, eliminate some of the bogosities of the old version
(i.e. don't do an implicit "setlowat", just prime the pool, and incr
the minpages for each additional page we add, and compute the number
of pages to prime in a way that callers would expect).
2001-05-10 04:51:41 +00:00
thorpej 6ec67f871a Use POOL_NEEDS_CATCHUP() in one more place. 2001-05-10 02:19:32 +00:00
thorpej db26619a5e Encapsulate the test for a pool needing a pool_catchup() in a macro. 2001-05-10 01:37:40 +00:00
thorpej 627edfcb50 Remove pool_create() and pool_prime(). Nothing except pool_create()
used pool_prime(), and no one uses pool_create() anymore.

This makes it easier to fix a locking pitfall.
2001-05-09 23:46:02 +00:00
thorpej da1f26d65f Add pool_cache_destruct_object(), used to force destruction of
an object and release back into the pool.
2001-05-04 19:41:25 +00:00
enami 6dcc7e3de4 Don't use PR_URGENT to allocate page header. We don't want to just panic
on memory shortage.  Instead, use the same wait/nowait condition with the
item requested, and just cleanup and return failure if we can't allocate
page header while we aren't allowed to wait.
2001-01-29 02:38:02 +00:00
thorpej b5104c1ca5 Change some low-hanging splimp() calls to splvm(). 2001-01-14 02:06:21 +00:00
thorpej 105cf38061 Add some basic statistics to pool_cache. 2000-12-11 05:22:55 +00:00
thorpej ce866cd0e8 Don't hold a pool cache lock across any call to pool_get() or pool_put().
This allows us to change a try-lock into a normal lock in the reclaim
case.
2000-12-10 17:03:34 +00:00
thorpej 21ca07bb5c ...and when freeing cache groups, clear `freeto' if that's the one
we're freeing.
2000-12-07 21:30:07 +00:00
thorpej 3e9d93b241 When we invalidate a pool cache, make sure to clear `allocfrom' if
we empty out that cache group.
2000-12-07 20:16:56 +00:00
thorpej 1a658211bc Add a /c modifier to "show pool" to display pool caches. 2000-12-07 19:30:31 +00:00
thorpej 7553470722 This is a first-cut implementation of support for caching of
constructed objects in the pool allocator, similar to caching
of constructed objects in the Solaris SLAB allocator.

This implementation is a separate API (pool_cache_*()) layered
on top of pools to keep the caching complexity out of the way
of pools that won't benefit from it.

While we're here, allow pool items to be as large as the pool
page size.
2000-12-07 05:45:57 +00:00
thorpej 581799e1f8 ANSI'ify. 2000-12-06 18:20:52 +00:00
sommerfeld f276315e1c In pool_setlowat(), only call pool_catchup() if the pool is under the
low water mark.  (Avoids annoying warning when you setlowat a static
pool).
2000-11-19 00:29:51 +00:00
sommerfeld 01233143b9 Use ltsleep instead of simple_unlock/tsleep/simple_lock 2000-08-12 16:28:30 +00:00
mrg 32aa199ccf remove include of <vm/vm.h> 2000-06-27 17:41:07 +00:00
mrg 2f159a1bac remove/move more mach vm header files:
<vm/pglist.h> -> <uvm/uvm_pglist.h>
	<vm/vm_inherit.h> -> <uvm/uvm_inherit.h>
	<vm/vm_kern.h> -> into <uvm/uvm_extern.h>
	<vm/vm_object.h> -> nothing
	<vm/vm_pager.h> -> into <uvm/uvm_pager.h>

also includes a bunch of <vm/vm_page.h> include removals (due to redudancy
with <vm/vm.h>), and a scattering of other similar headers.
2000-06-26 14:20:25 +00:00
sommerfeld e964d558a7 Fix assorted bugs around shutdown/reboot/panic time.
- add a new global variable, doing_shutdown, which is nonzero if
vfs_shutdown() or panic() have been called.
- in panic, set RB_NOSYNC if doing_shutdown is already set on entry
so we don't reenter vfs_shutdown if we panic'ed there.
 - in vfs_shutdown, don't use proc0's process for sys_sync unless
curproc is NULL.
 - in lockmgr, attribute successful locks to proc0 if doing_shutdown
&& curproc==NULL, and  panic if we can't get the lock right away; avoids the
spurious lockmgr DIAGNOSTIC panic from the ddb reboot command.
 - in subr_pool, deal with curproc==NULL in the doing_shutdown case.
 - in mfs_strategy, bitbucket writes if doing_shutdown, so we don't
wedge waiting for the mfs process.
 - in ltsleep, treat ((curproc == NULL) && doing_shutdown) like the
panicstr case.

Appears to fix: kern/9239, kern/10187, kern/9367.
May also fix kern/10122.
2000-06-10 18:44:43 +00:00
pk b74050d677 Allow a pool's pagesz to larger than the VM page size.
Enforce the required page alignment restriction in pool_prime_page().
2000-05-31 15:29:42 +00:00
pk b60b3fb997 Assert that the pool item size does not exceed the page size. 2000-05-31 15:03:54 +00:00
thorpej c7bff4127b __predict_false() the DIAGNOSTIC and other error condition checks. 2000-05-08 20:09:44 +00:00
chs c81c7aa367 always define PI_MAGIC so this compiles in all cases. 2000-04-13 00:44:19 +00:00
chs 9faf1824c8 in pool_put(), fill the entire object with PI_MAGIC instead of just the
first element.
2000-04-10 02:17:42 +00:00
thorpej 2649d26c0e Use ratecheck(). 2000-02-14 19:28:19 +00:00
thorpej c2ce79c0c9 In _pool_put(), panic if we're put'ing with nout == 0. This will help us
detect a little earlier if we've dup-put'd.  Otherwise, underflow occurs,
and subsequent allocations simply hang or fail (it thinks the hardlimit
has been reached).
1999-08-29 00:26:01 +00:00
sommerfeld b8e4538f80 Create new pool flag PR_LIMITFAIL, indicating that even PR_WAIT
allocations should fail if the pool is at its hard limit.
Document flag in pool(9).
Use it in mbuf.h for the first allocate call for M_GET, M_GETHDR, and
MCLGET, so that m_reclaim gets called even for blocking allocations.
1999-08-05 04:00:03 +00:00
thorpej cd992b17df In _pool_put(), call simple_lock_freecheck() if we're LOCKDEBUG before
we put the item on the free list.
1999-07-27 21:31:17 +00:00
pk 62cb666f4a Guard our global resource `phpool' against all interrupts. 1999-06-06 22:20:15 +00:00
thorpej 6c37e2b392 Make sure page allocations are counted everywhere that they need to be. 1999-05-10 21:15:42 +00:00
thorpej 4b6d8943c2 Improve the pool allocator's diagnostic helpers, adding the ability to
log on a per-pool basis, reentrancy checking, and dumping various pool
information from DDB.
1999-05-10 21:13:05 +00:00
scottr 3d5c979e43 Pull in opt_poollog.h for POOL_LOGSIZE. 1999-04-29 17:47:19 +00:00
thorpej b2741be06e More locking protocol fixes. Protect pool_head with a spin lock (statically
initialized).  This lock also protects the "next drain candidate" pointer.

XXX There is still one locking protocol problem, which should not be
a problem in practice, but is still marked as an issue in the code anyhow.
1999-04-06 23:32:44 +00:00
chs c109816333 Undo the part of the last revision about pr_rmpage() referencing
a data structure after it was freed.  This wasn't actually a problem,
and the change caused the wrong pool_item_header to be freed
in the non-PR_PHINPAGE case.
1999-04-04 17:17:31 +00:00
thorpej 278e7ae222 Yet more fixes to the pool allocator:
- Protect userspace from unnecessary header inclusions (as noted on
current-users).

- Some const poisioning.

- GREATLY simplify the locking protocol, and fix potential deadlock
scenarios.  In particular, assume that the back-end page allocator
provides its own locking mechanism (this is currently true for all
such allocators in the NetBSD kernel).  Doing so allows us to simply
use one spin lock for serialized access to all r/w members of the pool
descriptor.  The spin lock is released before calling the back-end
allocator, and re-acquired upon return from it.

- Fix a problem in pr_rmpage() where a data structure was referenced
after it was freed.

- Minor tweak to page manaement.  Migrate both idle and empty pages
to the end of the page list.  As soon as a page becomes un-empty
(by a pool_put()), place it at the head of the page list, and set
curpage to point to it.  This reduces fragmentation as well as the
time required to find a non-empty page as soon as curpage becomes
empty again.

- Use mono_time throughout, and protect access to it w/ splclock().

- In pool_reclaim(), if freeing an idle page would reduce the number
of allocatable items to below the low water mark, don't.
1999-03-31 23:23:47 +00:00
thorpej d4d4e314e9 Fix several bugs/deficiencies in the pool allocator:
- Add support for hard limits, with optional rate-limited logging of
a warning message when the pool limit is reached.  (This will be used
to fix a bug in mbuf cluster allocation on the MIPS and Alpha ports.)

- Fix some locking protocol errors.  This required splitting pr_flags
into pr_flags (which is protected by the spin lock) and pr_roflags (which
are `read only' flags, set when the pool is initialized, and never changed
again; these do not need to be protected by a mutex).

- Make the low water support actually mean something.  When a low water
mark is set, add free items to the pool until the low water mark is
reached.  When an item allocation causes the number of free items to
drop below the low water mark, make the pool catch up to it.  This can
make the pool allocator more useful for several applications (e.g.
pmap `pv entry' management) and more robust for others (for e.g. mbuf
and mbuf cluster allocation, so that the pagedaemon can use NFS to clean
pages on diskless systems without completely running dry on buffers to
receive packets in during extreme memory shoratages).

- Add a comment where we sleep waiting for more pages for the back-end
page allocator.  Specifically, instead of sleeping potentially forever,
perhaps we should just wake up once a second to try allocating a page
again.  XXX Revisit this soon.
1999-03-31 01:14:06 +00:00
mrg d2397ac5f7 completely remove Mach VM support. all that is left is the all the
header files as UVM still uses (most of) these.
1999-03-24 05:50:49 +00:00
thorpej 9614a68c70 Fix the order of arguments to roundup(). 1999-03-23 02:49:03 +00:00
thorpej e1315a2447 Make this compile with POOL_DIAGNOSTIC, and add a POOL_LOGSIZE option.
Defopt these.
1998-12-27 21:13:43 +00:00
briggs 4a01b776e5 Prototype pool_print() and pool_chk() if DEBUG.
Initialize pool hash table with PR_HASHTABSIZE (i.e., 8) LIST_INIT()s
instead of one memset().
Only check for page != ph->ph_page if PR_PHINPAGE is set (in pool_chk()).
Print pool base pointer when reporting page inconsistency in pool_chk().
1998-12-16 04:28:23 +00:00
pk 25e37f3b97 In addition to the spinlock, use the lockmgr() to serialize access to
the back-end page allocator. This allows the back-end to sleep since we
now relinquish the spin lock after acquiring the long-term lock.
1998-09-29 18:09:29 +00:00
thorpej c0dd0b8353 Make sure the size is large enough to hold a pool_item. 1998-09-22 03:01:29 +00:00
christos 34c5a58bb4 Make copyrights consistent; fix weird/trailing spaces add missing (c) etc. 1998-09-12 17:20:02 +00:00
thorpej f1f6ec6afe Add an alternate pool page allocator that can be used if the pool is
never accessed in interrupt context.  In the UVM case, this uses the
kernel_map, to reduce usage of the previous kmem_map resource.
1998-08-28 21:18:37 +00:00
thorpej 77d0a69569 Add a waitok boolean argument to the VM system's pool page allocator backend. 1998-08-28 20:05:48 +00:00
eeh a2dd74ed79 Merge paddr_t changes into the main branch. 1998-08-13 02:10:37 +00:00