Commit Graph

36 Commits

Author SHA1 Message Date
rmind ccfaf6e414 Optimise mutex_onproc() and rw_onproc() by making them O(1), instead
of O(ncpu) for adaptive paths.  Add an LWP destructor, lwp_dtor() with
a comment describing the principle of this barrier.

Reviewed by yamt@ and ad@.
2011-03-20 23:19:16 +00:00
skrll c53ddcfff2 Re-apply:
Invert the sense of the bit to mark if LOCKDEBUG is enabled to
	disabled.

	This will help my fellow developers spot "use before initialised"
	problems that hppa picks up very well.

but fix the !LOCKDEBUG case by defining the "no debug" bits to zero so
they have no effect on lock stubs.
2010-02-08 09:54:27 +00:00
cube 5ba423200b Revert commit from Fri Feb 5 06:43:17 UTC 2010 by skrll:
Invert the sense of the bit to mark if LOCKDEBUG is enabled to disabled.

      This will help my fellow developers spot "use before initialised" problems
      that hppa picks up very well.

It has to be done differently, because the semantics of mtx_owner in the non-
LOCKDEBUG case can vary significantly between archs, and thus it is not
possible to simply flip a bit to 1.

Ok core@, as at least i386 is unbootable right now.
2010-02-06 04:50:19 +00:00
skrll 60b795dc0a Invert the sense of the bit to mark if LOCKDEBUG is enabled to disabled.
This will help my fellow developers spot "use before initialised" problems
that hppa picks up very well.
2010-02-05 06:43:16 +00:00
pooka 83685e650c Heave-ho mutex/rwlock object routines into separate modules -- they
don't have anything to do with the lock internals.
2009-11-04 13:29:45 +00:00
yamt 805df27570 rw_vector_exit: remove a redundant condition. 2009-05-16 08:36:32 +00:00
yamt bed2400e59 lockdebug fixes for rw_tryupgrade/rw_downgrade. 2009-05-09 03:33:10 +00:00
ad 9b7896c50a A workaround for a bug with some Opteron revisions where locked operations
sometimes do not serve as memory barriers, allowing memory references to
bleed outside of critical sections.  It's possible that this is the
reason for pkgbuild's longstanding crashiness.

For rwlocks, always enable the explicit membars. They were disabled only
on x86, and since they are not in the fast-path it's not a big deal.
TODO: convert these to an atomic_membar_foo() or similar that does ordering
between regular data references and atomic references.
2009-04-24 17:53:06 +00:00
ad a8bd3c39aa Add rw_obj_*() functions to mirror the existing mutex functions.
Proposed on tech-kern quite some time ago.
2009-04-19 08:36:04 +00:00
thorpej 261002c777 rw_vector_exit(): When doing direct hand-off to a writer, only set
RW_WRITE_WANTED if there are additional writers waiting.
2008-07-29 16:13:39 +00:00
rmind a7fc471680 PR/39155: Wolfgang Solfrank: Invalid parameter to turnstile_exit in rw_downgrade. 2008-07-17 23:00:01 +00:00
ad e10320350c Use __noinline. 2008-05-31 13:31:25 +00:00
ad 7b8f512433 LOCKDEBUG:
- Tweak it so it can also catch common errors with condition variables.
  The change to kern_condvar.c is not included in this commit and will
  come later.

- Don't call kmem_alloc() if operating in interrupt context, just fail
  the allocation and disable debugging for the object. Makes it safe
  to do mutex_init/rw_init/cv_init in interrupt context, when running
  a LOCKDEBUG kernel.
2008-05-31 13:15:21 +00:00
ad 245f0726ac Reduce ifdefs due to MULTIPROCESSOR slightly. 2008-05-19 17:06:02 +00:00
ad 81194e34f1 Allow rw_tryenter(&lock, RW_READER) to recurse, for vfs_busy(). 2008-05-06 17:11:45 +00:00
martin ce099b4099 Remove clause 3 and 4 from TNF licenses 2008-04-28 20:22:51 +00:00
ad ebca8ee832 mutex_owned, rw_read_held, rw_write_held, rw_lock_held: check for a NULL
pointer.
2008-04-22 14:46:35 +00:00
ad 4598f15028 rwlock changes, discussed on tech-kern:
- Use atomic ops directly, since rwlocks work the same way on all platforms.
- Try to make it a bit more cache efficient, and use branch hints.
- Fix a bug in rw_downgrade() where the turnstile lock was not released.
- Remove a couple of redundant assertions.
- Use atomic_swap instead of atomic_cas where it's safe to do so.
- After acquiring the turnstile lock in rw_vector_enter, check if the
  owner is running again and spin if so.
- Introduce and use rw_onproc() instead of abusing mutex_onproc().
- Change the handoff/release algorithm to reduce the window when a rwlock
  can held, but the owner not on a CPU.
2008-04-11 14:55:51 +00:00
ad 15efd9ad99 Do adaptive spinning for rwlocks, but only if the lock is write held and
there are no waiters. This gives a major boost to build.sh on larger
systems as directory vnode locks are exclusive for lookup, but are often
only held for a very short period of time.

This change has the potential to more readily expose lock order reversals
and other types of deadlock.
2008-04-04 17:25:09 +00:00
ad 40fb254ec2 Fix error in previous. Spotted by yamt@. 2008-01-28 19:58:32 +00:00
ad 2763af2195 rw_write_held: return true only if held by the current LWP. 2008-01-26 14:13:37 +00:00
ad 501eb05cc2 - Prevent LOCKDEBUG from complaining if we do rw_tryenter() on a lock
that we already own.
- Add an assembly stub for rw_tryenter().
2008-01-25 19:02:31 +00:00
ad 8a9b6ca32a Need sys/lock.h for kernel_lock. 2008-01-04 21:54:49 +00:00
ad 070e98eb7b Pull in sys/atomic.h for memory barriers. 2008-01-04 21:52:48 +00:00
ad bb7ddbf003 rw_vector_enter: assert !cpu_intr_p(). 2007-12-04 09:13:59 +00:00
yamt 38d5e34116 make kmutex_t and krwlock_t smaller by killing lock id.
ok'ed by Andrew Doran.
2007-11-21 10:19:06 +00:00
ad 11dc639958 Merge from vmlocking:
- G/C spinlockmgr() and simple_lock debugging.
- Always include the kernel_lock functions, for LKMs.
- Slightly improved subr_lockdebug code.
- Keep sizeof(struct lock) the same if LOCKDEBUG.
2007-10-11 19:45:24 +00:00
dsl 0ca3d21b5b Include sys/cdefs.h first. 2007-09-21 19:14:12 +00:00
ad 5b3ae27b0d __FUNCTION__ -> __func__ 2007-09-17 21:33:34 +00:00
yamt f03010953f merge yamt-idlelwp branch. asked by core@. some ports still needs work.
from doc/BRANCHES:

	idle lwp, and some changes depending on it.

	1. separate context switching and thread scheduling.
	   (cf. gmcgarry_ctxsw)
	2. implement idle lwp.
	3. clean up related MD/MI interfaces.
	4. make scheduler(s) modular.
2007-05-17 14:51:11 +00:00
ad 0ac3a31ea3 Cosmetic changes. 2007-03-30 11:06:58 +00:00
itohy d2585a8c91 Remove extra space so that symbol renaming works properly. 2007-03-03 10:08:18 +00:00
ad f944608c11 Add a comment. 2007-02-26 19:06:10 +00:00
yamt e781af39bd implement priority inheritance. 2007-02-26 09:20:52 +00:00
ad 6bcf70b518 - Add/correct comments.
- Print correct function name when an assertion triggers.
2007-02-10 21:07:52 +00:00
ad b07ec3fc38 Merge newlock2 to head. 2007-02-09 21:55:00 +00:00