Commit Graph

1675 Commits

Author SHA1 Message Date
reinoud
9e3fb3728b Redo uvm_map_setattr() to never fail and remove the possible panic. The
possibility of failure was a C&P error.
2011-12-22 13:12:50 +00:00
reinoud
39568060d0 If we need to set the PK_CHKNOSYSCALL flag in struct proc be so nice to first
take the mutex. Tnx for pointing it out to me.
2011-12-20 19:49:36 +00:00
reinoud
ea698f7362 Ooops forgot the uvm_map.h 2011-12-20 15:41:01 +00:00
reinoud
d131102a3b Add a MAP_NOSYSCALLS flag to mmap. This flag prohibits executing of system
calls from the mapped region. This can be used for emulation perposed or for
extra security in the case of generated code.

Its implemented by adding mapping-attributes to each uvm_map_entry. These can
then be queried when needed.

Currently the MAP_NOSYSCALLS is only implemented for x86 but other
architectures are easy to adapt; see the sys/arch/x86/x86/syscall.c patch.
Port maintainers are encouraged to add them for their processor ports too.
When this feature is not yet implemented for an architecture the
MAP_NOSYSCALLS is simply ignored with virtually no cpu cost..
2011-12-20 15:39:35 +00:00
yamt
49cf653167 comment and assertion 2011-12-20 13:47:38 +00:00
mrg
9a4a666ff3 implement bdev_size(9) wrapper around d_psize() routine, so we can take
the device lock in relevant places.  avoid doing so while actually dumping.

tested i386 crash dumps still work, and that all touched files compile.

fixes PR#45705.
2011-12-12 19:03:07 +00:00
matt
a617230f3e Redefine ptoa() to be the inverse of atop. If you were using a 32-bit vaddr_t
with 64-bit paddr_t and using managed addresses > 4GB, uvm_page_init would
silently discard the upper 32-bits of the physical address possibly double
mapping pages.
2011-11-29 07:43:54 +00:00
yamt
ef9d35249e comments 2011-11-28 14:06:59 +00:00
matt
4b00b594fa When allocating a page for a kernel stack and PMAP_ALLOC_POOLPAGE is
defined, use it.  (allows a MIPS N32 kernel to boot when there is memory
outside of KSEG0).
2011-11-23 01:07:50 +00:00
matt
c1be7fc1fe When allocating pages for kernel map entries and PMAP_ALLOC_POOLPAGE is
defined, use it.  (allows a MIPS N32 kernel to boot when there is memory
outside of KSEG0).
2011-11-23 01:00:52 +00:00
christos
cff1dac372 if you are going to dereference a variable, check the variable itself, not
it cousin.
2011-11-13 02:10:40 +00:00
hannken
2cc7a01f10 Change the vnode locking protocol of VOP_GETATTR() to request at least
a shared lock.  Make all calls outside of file systems respect it.

The calls from file systems need review.

No objections from tech-kern.
2011-10-14 09:23:28 +00:00
yamt
4e27ff24fc fix an integer promotion bug on 64 bit ports.
(signed + unsigned = unsigned)
2011-10-12 00:03:47 +00:00
yamt
8ddce5912c assertion 2011-10-11 23:57:50 +00:00
yamt
f8a2cce5e6 comment 2011-10-11 23:57:07 +00:00
uebayasi
5590d34021 Correct pagermap emergva allocation. From yamt@.
Tested by building i386 kernel with DTRACE defined which died 100%.
2011-10-06 12:26:03 +00:00
mrg
ad1c111812 re-arrange the end of uvm_page_recolor() to avoid the multiple exit
points.  move the call to uvm_pager_realloc_emerg() to after we
drop the uvm_fpageqlock, since it may be taken again in uvm_km_alloc().

fixes LOCKDEBUG crashes with the previous change.
2011-09-30 05:29:12 +00:00
matt
d289edb2b2 Reallocate emergency pager va when ncolors is increased. (modication of
patch from mrg).
2011-09-28 22:52:15 +00:00
jym
325494fe33 Modify *ASSERTMSG() so they are now used as variadic macros. The main goal
is to provide routines that do as KASSERT(9) says: append a message
to the panic format string when the assertion triggers, with optional
arguments.

Fix call sites to reflect the new definition.

Discussed on tech-kern@. See
http://mail-index.netbsd.org/tech-kern/2011/09/07/msg011427.html
2011-09-27 01:02:33 +00:00
matt
3d901f1f5e Allocate color appropriate pages. 2011-09-06 16:41:55 +00:00
dyoung
78b0e18345 Report vmem(9) errors out-of-band so that we can use vmem(9) to manage
ranges that include the least and the greatest vmem_addr_t.  Update
vmem(9) uses throughout the kernel.  Slightly expand on the tests in
subr_vmem.c, which still pass.  I've been running a kernel with this
patch without any trouble.
2011-09-02 22:25:08 +00:00
matt
207bff18bc Forward some UVM from matt-nb5-mips64. Add UVM_KMF_COLORMATCH flag.
When uvm_map gets passed UVM_FLAG_COLORMATCH, the align argument contains
the color of the starting address to be allocated (0..colormask).
When uvm_km_alloc is passed UVM_KMF_COLORMATCH (which can only be used with
UVM_KMF_VAONLY), the align argument contain the color of the starting address
to be allocated.
Change uvm_pagermapin to use this.  When mapping user pages in the kernel,
if colormatch is used with the color of the starting user page then the kernel
mapping will be congruent with the existing user mappings.
2011-09-01 06:40:28 +00:00
christos
262ad41ded Add an optional pglist argument to uvm_obj_wirepages, to be
filled with the list of pages that were wired.
2011-08-27 09:11:52 +00:00
oki
32a22fa000 make compile without VMSWAP. no functional change. 2011-08-23 03:00:35 +00:00
yamt
a7b89fad2a uvm_aio_aiodone_pages: check disposed anon correctly. 2011-08-18 14:17:08 +00:00
yamt
2b339dd699 uvm_anon_release:
- don't forget to call uvm_anon_dispose.
- simplify code a little.
2011-08-18 14:13:59 +00:00
yamt
535ffbdf82 uvm_anon_freelst:
- clear an_link/an_ref when deferring anon disposal.  otherwise others can
  see bogus an_ref.
- fix the code to remove anon from the list.
2011-08-18 14:13:02 +00:00
rmind
e52f8e8779 amap_cow_now: just free the fresh anon on error, no need to dispose it. 2011-08-17 20:46:27 +00:00
rmind
eeaaf989e3 uvm_anon_freelst: do not free PG_RELEASED pages (change uvm_anon_dispose()
to indicate them with a return value).
2011-08-14 01:20:33 +00:00
rmind
39445b05bc - Rework uvm_anfree() into uvm_anon_freelst(), which always drops the lock.
- Free anons in uvm_anon_freelst() without lock held.
- Mechanic sync to unused loaning code.
2011-08-06 17:25:03 +00:00
martin
eef17f7b6c Make uvmspace_exec() deal with procs that have no vmspace (yet) at all.
Greatly simplifies the upcoming posix_spawn implementation.
2011-07-30 20:05:46 +00:00
martin
95377927e5 Get rid of #ifdef __sparc__ in uvm code - as noted by cgd back 1996,
now that we have __HAVE_CPU_VMSPACE_EXEC/cpu_vmspace_exec().
2011-07-30 19:29:12 +00:00
yamt
b456aa6a56 - fix a use-after-free bug in uvm_km_free.
(after uvm_km_pgremove frees pages, the following pmap_remove touches them.)
- acquire the object lock for operations on pmap_kernel as it can actually be
  raced with P->V operations.  eg. pagedaemon.
2011-07-05 14:03:06 +00:00
yamt
7eec89b991 reduce the number of atomic ops in common cases. it's exceptional for
anons to remain longer than amap.
2011-07-05 13:47:24 +00:00
matt
dd6af2c565 Allow the MD code to decide to panic if cpu_uarea_alloc would return NULL.
If NULL is returned, just allocate the standard way.
2011-07-02 01:26:29 +00:00
matt
ebc746ad25 Move PMAP_* cache defines to before inclusion of <machine/pmap.h> 2011-06-30 00:49:14 +00:00
hannken
ea8870b58b Remove dead uvm_vnp_zerorange() after bump to 5.99.54. 2011-06-29 19:51:12 +00:00
hannken
79568783ba amap_copy(): Keep the source amap locked until its lock has been copied.
Kernel assertion "anon->an_lock == amap->am_lock" no longer fails.

Ok: Mindaugas Rasiukevicius <rmind@netbsd.org>
2011-06-27 15:56:36 +00:00
rmind
c59e31527c amap_copy: fix one more regression, thanks to enami@. 2011-06-24 01:48:43 +00:00
rmind
7be1d60258 Fix uvmplock regression - a lock against oneself case in amap_swap_off().
Happens since amap is NULL in uvmfault_anonget(), so uvmfault_unlockall()
keeps anon locked, when it should unlock it.
2011-06-24 01:39:22 +00:00
yamt
fa43ee9ddc uvm_anon_release: fix a locking error after the rmind-uvmplock merge 2011-06-24 01:23:05 +00:00
rmind
538c69f6e6 amap_pp_adjref: fix regression, spotted by nonaka@. 2011-06-24 01:03:08 +00:00
matt
65bd0920b3 Allow PAX_ASLR to be used by itself. 2011-06-23 23:42:43 +00:00
rmind
6398a253c4 Clean-up, add asserts, slightly simplify. 2011-06-23 18:15:30 +00:00
rmind
df10270871 uvmfault_anonget: clean-up, improve some comments, misc. 2011-06-23 17:36:59 +00:00
yamt
05d8362d92 band-aid fix after the merge of rmind-uvmplock branch. 2011-06-20 23:18:58 +00:00
rmind
7083a919fc - Fix a silly bug: remove umap from uobj in ubc_release() UBC_UNMAP case.
- Use UBC_WANT_UNMAP() consistently.

ARM (PMAP_CACHE_VIVT case) works again.
2011-06-19 02:42:53 +00:00
rmind
7a15ad245d - Move pre-check from uvm_obj_destroy() to ubc_purge(), keep it abstracted.
- Add comments noting the race between ubc_alloc() and ubc_purge().
2011-06-18 21:14:43 +00:00
rmind
10583b12ce Clean up, sprinkle asserts, consify, use unsigned, use kmem_zalloc instead
of memset, reduce the scope of some variables, improve some comments.

No functional change intended.
2011-06-18 21:13:29 +00:00
rmind
1885100a7e amap_add/amap_unadd: clean up slightly, use unsigned, add asserts. 2011-06-18 20:51:22 +00:00
rmind
223aec6f50 Add amap_adjref_anons() helper and simplify amap_ref()/amap_unref(). 2011-06-18 20:29:56 +00:00
hannken
772633be86 When ubc_alloc() reuses a cached mapping window remove the object from
the lists AFTER clearing its mapping.

Removes a race where uvm_obj_destroy() sees an empty uo_ubc list and
destroys the object before ubc_alloc() gets the objects lock to clear
the mapping.
2011-06-17 09:50:52 +00:00
rmind
3aad734e54 Improve comments on uvm_anon.c, tidy up slightly.
No functional changes.
2011-06-17 02:12:35 +00:00
rmind
71341c3c42 amap_lookup{s}: add assert, clean-up slightly. 2011-06-16 19:42:20 +00:00
hannken
d296304e60 Rename uvm_vnp_zerorange(struct vnode *, off_t, size_t) to
ubc_zerorange(struct uvm_object *, off_t, size_t, int) changing
the first argument to an uvm_object and adding a flags argument.

Modify tmpfs_reg_resize() to zero the backing store (aobj) instead
of the vnode.  Ubc_purge() no longer panics when unmounting tmpfs.

Keep uvm_vnp_zerorange() until the next kernel version bump.
2011-06-16 09:21:02 +00:00
rmind
035445691f uvm_pagealloc_strat: fix diagnostic assert. Reported by drochner@. 2011-06-15 19:46:11 +00:00
rmind
eb51bf5884 uvm_map_lock_entry: fix the order of locking. Spotted by yamt@.
Also, keep uvm_map_unlock_entry() symmetric.
2011-06-13 23:19:40 +00:00
mrg
a14dae9853 include uvm_object.c in the rump kernel for the new uvm_obj* functions.
don't build the uvm_object.c uvm_object_printit() for _RUMPKERNEL. (XXX)
add empty panic() stubs for uvm_loanbreak() and ubc_purge().

fixes some more 5.99.53 rump build issues.
2011-06-12 06:36:38 +00:00
rmind
e225b7bd09 Welcome to 5.99.53! Merge rmind-uvmplock branch:
- Reorganize locking in UVM and provide extra serialisation for pmap(9).
  New lock order: [vmpage-owner-lock] -> pmap-lock.

- Simplify locking in some pmap(9) modules by removing P->V locking.

- Use lock object on vmobjlock (and thus vnode_t::v_interlock) to share
  the locks amongst UVM objects where necessary (tmpfs, layerfs, unionfs).

- Rewrite and optimise x86 TLB shootdown code, make it simpler and cleaner.
  Add TLBSTATS option for x86 to collect statistics about TLB shootdowns.

- Unify /dev/mem et al in MI code and provide required locking (removes
  kernel-lock on some ports).  Also, avoid cache-aliasing issues.

Thanks to Andrew Doran and Joerg Sonnenberger, as their initial patches
formed the core changes of this branch.
2011-06-12 03:35:36 +00:00
matt
8704f281b7 Fix fencepost error. 2011-06-05 16:58:00 +00:00
tsutsui
2d7fb158b6 No need to pass UVM_FLAG_COLORMATCH to uvm_pagealloc()
if no valid vaddr is specified.
2011-05-21 11:49:34 +00:00
yamt
3b8a580f96 g/c unused function prototypes 2011-05-19 09:57:20 +00:00
rmind
4b0ec60601 ubc_release: use voff_t for offsets, rather than int. Constify.
Reviewed by matt@.
2011-05-19 03:44:19 +00:00
mrg
a852848eef fix the ordering and make UVMHIST enable KERNHIST automatically. 2011-05-17 05:32:31 +00:00
mrg
8169e46991 move and rename the uvm history code out of uvm_stat to "kernhist".
rename "UVMHIST" option to enable the uvm histories.

TODO:
- make UVMHIST properly depend upon KERNHIST
- enable dynamic registration of histories.  this is mostly just
  allocating something in a bitmap, and is only for viewing multiple
  histories in a merged form.


tested on amd64 and sparc64.
2011-05-17 04:18:05 +00:00
rmind
fe41737f5c Remove public uvm_swap_stats() routine, keep it internal. 2011-04-27 00:35:52 +00:00
rmind
520d1ecfd3 Initialize UVM loaning subsystem a bit later, after kmem(9).
Makes UVMHIST work again.
2011-04-24 03:56:50 +00:00
rmind
c22a36981f Replace "malloc" in comments, remove unnecessary header inclusions. 2011-04-23 18:14:12 +00:00
yamt
500e2272be - ensure that the promoted page is on the queue even when later pmap_enter
failed.
- don't activate a page twice.
- remove an argument which is used only for an assertion.
- assertions and comments.
2011-04-08 10:42:51 +00:00
yamt
09f62f3554 comment 2011-04-08 10:36:58 +00:00
rmind
5940ea9676 uvm_pageidlezero: use try-lock to not occupy uvm_fpageqlock, which may
be on demand by other CPUs.  Reduces lock contention in some workloads
on many CPU (8+) systems.

Tested by tls@.
2011-04-01 00:47:11 +00:00
drochner
f376d5576a make this build w/o HAVE_CPU_UAREA_ROUTINES 2011-02-18 10:43:52 +00:00
matt
4f38307d62 Add support for cpu-specific uarea allocation routines. Allows different
allocation for user and system lwps.  MIPS will use this to map uareas of
system lwp used direct-mapped addresses (to reduce the overhead of
switching to kernel threads).  ibm4xx could use to map uareas via direct
mapped addresses and avoid the problem of having the kernel stack not in
the TLB.
2011-02-17 19:27:13 +00:00
jmcneill
50ba38fe0a need uvm_pmap.h for pmap_mmap_flags definition 2011-02-12 14:45:31 +00:00
jmcneill
ee6551845f add optional MD pmap_mmap_flags macro for passing flags between cdev_mmap
and pmap_enter, ok matt@
2011-02-11 23:05:55 +00:00
rmind
cdc76ff97e Replace uvm_aobj_cache with kmem(9). 2011-02-11 00:21:18 +00:00
skrll
b5b02ca20c Spell uvm_fault_lower_neighbor correctly in UVMHIST_FUNC by using
__func__
2011-02-10 21:05:52 +00:00
pooka
22c822c7f3 Make vmapbuf() return success/error and make physio deal with a
failure.
2011-02-10 14:46:44 +00:00
yamt
c6c7ed993c pageobj: remove a wrong assertion. 2011-02-05 13:33:47 +00:00
chuck
afca0358a6 udpate license clauses on my code to match the new-style BSD licenses.
verified with Mike Hibler it is ok to remove clause 3 on utah copyright,
as per UCB.
based on diff that rmind@ sent me.

no functional change with this commit.
2011-02-02 20:07:25 +00:00
chuck
beb929a933 udpate license clauses on my code to match the new-style BSD licenses.
based on diff that rmind@ sent me.

no functional change with this commit.
2011-02-02 17:53:41 +00:00
chuck
f9d8cc1a37 udpate license clauses on chuck^2 code to match the new-style BSD licenses.
based on diff that rmind@ sent me (and confirmed with chs@ via email).

no functional change with this commit.
2011-02-02 15:28:38 +00:00
chuck
40ec801a13 udpate license clauses on my code to match the new-style BSD licenses.
based on second diff that rmind@ sent me.

no functional change with this commit.
2011-02-02 15:25:27 +00:00
chuck
3ba477b154 udpate license clauses on my code to match the new-style BSD licenses.
based on diff that rmind@ sent me.

no functional change with this commit.
2011-02-02 15:13:33 +00:00
enami
40713a94c3 Introducing inner loop prevent us from exiting from the original loop. 2011-01-26 08:49:48 +00:00
matt
d32b258b78 When starting the second pass, don't continue the for loop but instead
just test try exceeding limit.
2011-01-25 17:22:43 +00:00
enami
7f4cb8b53c Remove nop code; the code is moved to uao_dropswap_range1() when it is
introduced in rev. 1.75.
2011-01-25 03:34:29 +00:00
matt
cb4ebd8be4 Use the (new) KDASSERTMSG 2011-01-24 22:54:01 +00:00
matt
12708a66ee Fix start_hint in "simple" alloc (fencepost error).
When restarting the loop, make sure end is not above current limit.
Do a quick test to see if the physseg is within the range of desired addresses.
2011-01-24 19:13:55 +00:00
he
e9075e068f DEBUG does not imply DIAGNOSTIC; make sure we have a non-null
KASSERTMSG implementation (DIAGNOSTIC) so that the variable inside
the DEBUG section gets used.
2011-01-23 21:29:52 +00:00
matt
5f98725eda Fix the corruption of ps->start_hint. 2011-01-22 01:36:27 +00:00
matt
651bde3939 Cleanup/add some asserts. no functional change. 2011-01-21 19:27:09 +00:00
cegger
c6a6ce3477 buildfix: use PRIxPADDR for type paddr_t 2011-01-21 16:56:38 +00:00
matt
37726e8583 Improve the efficiency of searching for a contiguous set of free pages. 2011-01-18 21:43:29 +00:00
rmind
7146b2f61d Retire struct user, remove sys/user.h inclusions. Note sys/user.h header
as obsolete.  Remove USER_TO_UAREA/UAREA_TO_USER macros.

Various #include fixes and review by matt@.
2011-01-14 02:06:22 +00:00
enami
eaefa873fe Fix bugs introduced by previous commit; allocated page needs to be bound
with the anon, and uvmfault_anonget may be called with ufi NULL.
2011-01-06 05:51:57 +00:00
enami
7b2a66bec9 Fix format string; use PRIu64 for uint64_t. 2011-01-05 21:20:44 +00:00
matt
4d8e8a7e36 Add better color matching selecting free pages. KM pages will now allocated
so that VA and PA have the same color.  On a page fault, choose a physical
page that has the same color as the virtual address.

When allocating kernel memory pages, allow the MD to specify a preferred
VM_FREELIST from which to choose pages.  For machines with large amounts
of memory (> 4GB), all kernel memory to come from <4GB to reduce the amount
of bounce buffering needed with 32bit DMA devices.
2011-01-04 08:26:33 +00:00
matt
5d3db402fb Add a MD hook to indicate a change of vmspace due to exec. (This is useful
to update any cpu flag due to a change to/from a 64bit and a 32bit address
space).  This can set the state needed for copyout/copyin before setregs
is invoked.
2011-01-04 08:21:18 +00:00
matt
b3322481fa Print the number of page colors in use with db> show uvm 2011-01-04 08:17:01 +00:00
matt
6a66466f0c Move counting of faults, traps, intrs, soft[intr]s, syscalls, and nswtch
from uvmexp to per-cpu cpu_data and move them to 64bits.  Remove unneeded
includes of <uvm/uvm_extern.h> and/or <uvm/uvm.h>.
2010-12-20 00:25:23 +00:00
yamt
473b1715a5 cosmetics. no functional changes.
- constify
- wrap long lines
- assertions
- comments
2010-12-17 22:00:43 +00:00
pooka
fe0a6aa142 Remove duplicate asserts from when uvm_fault_lower1() was merged
into uvm_fault_lower() (the duplicates were there already before,
just in different functions).

reported by Alexander Nasonov on tech-kern
2010-12-15 13:44:17 +00:00
matt
9898b7c4fd When panicing due a non-power of 2 pagesize, include the pagesize in the
panic message.
2010-12-11 22:34:03 +00:00
uebayasi
565a3d3094 Make UVM_PAGE_TRKOWN a real flag. 2010-12-09 01:48:05 +00:00
hannken
bd8f6f0b8f Always take the object lock before changing vmpage flags. Fixes a deadlock
where a thread is waiting on "genput" but the page in question is neither
BUSY nor WANTED.

No objections from tech-kern@.
2010-11-30 10:55:25 +00:00
mrg
05061c6ad9 put the kernel-only externs back before <machine/pmap.h>. fixes ofppc build. 2010-11-29 09:49:33 +00:00
christos
a9bdee4f4d don't leak kernel variables to userland! 2010-11-26 18:51:19 +00:00
uebayasi
f4ae5ecf6e Put back VM_PAGE_TO_MD(); pointed out by skrll@, thanks. 2010-11-26 00:45:27 +00:00
uebayasi
e3b768e416 Revert vm_physseg allocation changes. A report says that it causes
panics when used with mplayer in heavy load.
2010-11-25 04:45:30 +00:00
dholland
8f6ed30d57 Introduce struct pathbuf. This is an abstraction to hold a pathname
and the metadata required to interpret it. Callers of namei must now
create a pathbuf and pass it to NDINIT (instead of a string and a
uio_seg), then destroy the pathbuf after the namei session is
complete.

Update all namei call sites accordingly. Add a pathbuf(9) man page and
update namei(9).

The pathbuf interface also now appears in a couple of related
additional places that were passing string/uio_seg pairs that were
later fed into NDINIT. Update other call sites accordingly.
2010-11-19 06:44:33 +00:00
cegger
7f56ec2e89 build fix: vm_physmem_index is only used with DEBUG.
Fix build when DIAGNOSTIC is enabled but not DEBUG
2010-11-18 11:49:41 +00:00
uebayasi
5cdd3cec1f Optimize DIAGNOSTIC check code. 2010-11-18 08:41:11 +00:00
uebayasi
56df070b0c Fix DIAGNOSTIC physseg find check. 2010-11-18 08:18:31 +00:00
enami
2c376812ef Nowadays, comparing priority against PZERO doesn't make any sense.
Instead, see if a process waits uninterruptibly like ps does,
so that the second column (`b') of default vmstat output prints
some useful value (-t is still broken though).
2010-11-16 03:49:53 +00:00
uebayasi
2bc2c4def2 ... and another. 2010-11-14 15:18:07 +00:00
uebayasi
12d4db54c0 Fix build caused by a last minute change. 2010-11-14 15:16:53 +00:00
uebayasi
26dd1e598f Be a little more friendly to dynamic physical segment registration.
Maintain an array of pointer to struct vm_physseg, instead of struct
array.  So that VM subsystem can take its pointer safely.  Pointer
to this struct will replace raw paddr_t usage in the future.

Dynamic removal is not supported yet.

Only MD data structure changes, no kernel bump needed.

Tested on i386, amd64, powerpc/ibm40x, arm11.
2010-11-14 15:06:34 +00:00
uebayasi
1674b65491 Oops. Fix thinko. 2010-11-14 04:31:02 +00:00
uebayasi
c9ff0b160b Platforms that dynamically set PAGE_{SIZE,MASK,SHIFT}, those values are
saved in struct uvmexp.  Expose only the relevant part for symbol users,
so that they don't need to include the whole uvm(9) API.
2010-11-14 04:25:16 +00:00
uebayasi
e048abfd86 UVM constants should not rely on sys/lock.h. 2010-11-13 12:50:09 +00:00
uebayasi
6d3191d5b6 Hide uvm/uvm_page.h again to ensure its internal structures are MD.
GENERIC or at least one kernel compile tested for:
	acorn26, acorn32, algor, all, alpha, amd64, amiga, amigappc,
	arc, bebox, bighill, cats, cobalt, dreamcast, ews4800mips,
	hp300, hp700, hpcarm, hpcmips, hpcsh, i386, ibmnws,
	integrator, ixm1200, iyonix, landisk, luna68k, mac68k,
	macppc, mipsco, mmeye, mvme68k, mvmeppc, netwinder, news68k,
	newsmips, next68k, obs266a, ofppc, pmax, pmppc, prep,
	rs6000, sandpoint, sbmips, shark, sidebeach, sparc, sparc64,
	sun2, sun3, usermode, vax, x68k, zaurus
2010-11-13 05:52:55 +00:00
uebayasi
3ebf409bf5 Put back uvm_page.h for now. Sorry for mess. 2010-11-12 12:02:35 +00:00
uebayasi
4f1dd4067f Put VM_PAGE_TO_MD() definition in one place. No functional changes. 2010-11-12 07:59:24 +00:00
uebayasi
77d80f38cd Abstraction fix; move physical address -> per-page metadata (struct
vm_page *) "reverse" lookup code from uvm_page.h to uvm_page.c, to
help migration to not do that.

Likewise move per-page metadata (struct vm_page *) -> physical
address "forward" conversion code into *.c too.  This is called
only low-layer VM and MD code.
2010-11-12 05:23:41 +00:00
uebayasi
aa803dbb9d Abstraction fix; move physical address -> physical segment "reverse"
lookup code from uvm_page.h to uvm_page.c.

This code is used by some pmaps to lookup per-page state (PV) from
per-segment metadata (struct vm_physseg).  This is not needed if
UVM looks up physical segment once in fault handler, then directly
passes it to pmap.  This change helps transition to that model.

The only users of vm_physseg_find() are pmap_motorola.c and
powerpc/ibm4xx/pmap.c.

Tested By:	Compiling and running powerpc/ibm4xx/pmap.c
		(evbppc/conf/OPENBLOCKS266)
2010-11-12 03:21:04 +00:00
uebayasi
186b58eb54 Abstraction fix; don't pull in physical segment/page definitions
in UVM external API, uvm_extern.h.  Because most users care only
virtual memory.

Device drivers use bus_dma(9) to manage physical memory.  Device
drivers pull in bus_dma(9) API, bus_dma.h.  bus_dma(9) implementations
pull in UVM internal API, uvm.h.

Tested By:	Compiling i386 ALL kernel
2010-11-12 02:36:02 +00:00
uebayasi
7e090d7604 C style; make a sentinel pointer have an exclusive value; no
functional changes.
2010-11-11 15:59:27 +00:00
uebayasi
4e73439bf8 Typo in a comment. 2010-11-11 15:51:05 +00:00
uebayasi
2d8b28a059 Minor clean up. 2010-11-11 15:47:43 +00:00
uebayasi
e2e0c29553 Minor clean up. 2010-11-11 14:50:54 +00:00
uebayasi
04cf143fd6 Use more VM_PHYSMEM_*() accessors. No functional changes. 2010-11-10 09:27:21 +00:00
uebayasi
e367a5a854 Prepare vm_physmem[] -> (*vm_physmem)[] migration, so that physical
segments can be changed at run-time.  Pointers are easier to update.
2010-11-10 01:24:46 +00:00
uebayasi
40a5185389 Provide a forward declaration of "struct vm_page", whose internal
is opaque to uvm_pglist.h users.  Users don't need to pull in
uvm_page.h.
2010-11-06 15:48:00 +00:00
uebayasi
41e5df6d3e Remove incomplete, never worked dynamic run-time memory registration
(uvm_page_physload(9)).  This functionality will be re-added later.
2010-11-06 15:42:43 +00:00
uebayasi
b73ba829c6 Include uvm/uvm_pglist.h for struct pglist. 2010-11-06 12:19:14 +00:00
uebayasi
8e3c57e6cc Include uvm/uvm.h because this is part of UVM. 2010-11-06 12:18:17 +00:00
skrll
162991256a Spell immediately correctly. 2010-11-02 20:49:47 +00:00
tsutsui
8e16588b0b Make common kernel module binaries work on both sun3 and sun3x.
Tested on 3/160 (on TME) and (real) 3/80.

XXX: module files can be loaded only on single user?
2010-10-15 15:55:52 +00:00
matt
19e6c76b2d Rename rb.h to rbtree.h, as it is more appropriate (c.f. ptree.h). Also
helps find code that hasn't been updated to use the new rbtree API.
2010-09-25 01:42:38 +00:00
rmind
879d5dfb5e Fixes/improvements to RB-tree implementation:
1. Fix inverted node order, so that negative value from comparison operator
   would represent lower (left) node, and positive - higher (right) node.
2. Add an argument (i.e. "context"), passed to comparison operators.
3. Change rb_tree_insert_node() to return a node - either inserted one or
   already existing one.
4. Amend the interface to manipulate the actual object, instead of the
   rb_node (in a similar way as Patricia-tree interface does).
5. Update all RB-tree users accordingly.

XXX: Perhaps rename rb.h to rbtree.h, since cleaning-up..

1-3 address the PR/43488 by Jeremy Huddleston.

Passes RB-tree regression tests.
Reviewed by: matt@, christos@
2010-09-24 22:51:50 +00:00
pooka
04d191ff71 Make "no options VMSWAP" kernels compile again. 2010-09-07 04:45:22 +00:00
chs
fca58884f4 replace the earlier workaround for PR 40389 with a better fix.
the earlier change caused data corruption by freeing pages
without invaliding their mappings.  instead of the trylock/retry,
just take the genfs-node lock before calling VOP_GETPAGES()
and pass a new flag to tell it that we're already holding this lock.
2010-09-01 16:56:19 +00:00
yamt
86b3a16608 sys_mremap: unwrap a short line 2010-08-16 01:21:10 +00:00
hannken
c84e81cad1 Add vm page flag PG_MARKER and use it to tag dummy marker pages
in genfs_do_putpages() and uao_put().
Use 'v_uobj.uo_npages' to check for an empty memq.
Put some assertions where these marker pages may not appear.

Ok: YAMAMOTO Takashi <yamt@netbsd.org>
2010-07-29 10:54:50 +00:00
hannken
7296ba383a Replace vget() with vref()/vn_lock(), this node already has a reference. 2010-07-09 08:10:50 +00:00
chs
4c14d1923f switch the UVMHIST counters from mutexes to atomic ops
to avoid a bad interaction with DIAGNOSTIC.
2010-07-07 01:08:51 +00:00
cegger
71e8cdce36 Turn PMAP_NOCACHE into MI flag.
Add MI flags PMAP_WRITE_COMBINE, PMAP_WRITE_BACK, PMAP_NOCACHE_OVR.
Update pmap(9) manpage.

hppa: Remove MD PMAP_NOCACHE flag as it exists as MI flag
mips: Rename MD PMAP_NOCACHE to PGC_NOCACHE.

x86: Implement new MI flags using Page-Attribute Tables.
x86: Implement BUS_SPACE_MAP_PREFETCHABLE.

Patch presented on tech-kern@:
http://mail-index.netbsd.org/tech-kern/2010/06/30/msg008458.html

No comments on this last version.
Forgot to commit this in previous.
2010-07-06 21:11:22 +00:00
hannken
1423e65b26 Clean up vnode lock operations pass 2:
VOP_UNLOCK(vp, flags) -> VOP_UNLOCK(vp): Remove the unneeded flags argument.

Welcome to 5.99.32.

Discussed on tech-kern.
2010-06-24 12:58:48 +00:00
rmind
3e68c94985 Keep the lock around pmap_update() where required. While fixing this
in ubc_fault(), rework logic to "remember" the last object of page and
reduce locking overhead, since in common case pages belong to one and
the same UVM object (but not always, therefore add a comment).

Unlocks before pmap_update(), on removal of mappings, might cause TLB
coherency issues, since on architectures like x86 and mips64 invalidation
IPIs are deferred to pmap_update().  Hence, VA space might be globally
visible before IPIs are sent or while they are still in-flight.

OK ad@.
2010-06-22 18:34:50 +00:00
mrg
72ceb9b885 disable some DEBUG code uvm_pglist_add() that has severe performance
problems with large mappings.  i've seen my system hang for a total
of 45 seconds when radeondrm is opened by X11, and it is the checks
in this function that take so long.
2010-06-17 03:13:58 +00:00
pooka
cbed8ae4b7 it's a wonderful static 2010-06-02 15:48:49 +00:00
rmind
7bf5bfa1e6 ubc_fault: split-off code part handling a single page into ubc_fault_page(). 2010-05-29 23:17:53 +00:00
rmind
22d67cdea8 uvm_fault_{upper,lower}_done: move drop-swap outside the page-queues lock.
Assert for object lock being held (or ref count 0) in uao_set_swslot().
2010-05-28 23:41:14 +00:00
cegger
07ebeefdd8 Move PMAP_KMPAGE to be used in pmap_kenter_pa flags argument.
'Looks good to me' gimpy@

Forgot to commit this piece in previous commit
2010-05-14 05:32:06 +00:00
cegger
dfa8eb2f81 Move PMAP_KMPAGE to be used in pmap_kenter_pa flags argument.
'Looks good to me' gimpy@
2010-05-14 05:02:05 +00:00
ad
b746352090 Reduce memory spent on bookkeeping for large values of MAXCPUS. 2010-04-25 15:54:14 +00:00
rmind
5f0ac9a4fa - Merge sched_pstats() and uvm_meter()/uvm_loadav(). Avoids double loop
through all LWPs and duplicate locking overhead.

- Move sched_pstats() from soft-interrupt context to process 0 main loop.
  Avoids blocking effect on real-time threads.  Mostly fixes PR/38792.

Note: it might be worth to move the loop above PRI_PGDAEMON.  Also,
sched_pstats() might be cleaned-up slightly.
2010-04-16 03:21:49 +00:00
mrg
665bf35b1e now that CTLTYPE_BOOL actually works, use it to export vm_page_zero_enable
as vm.idlezero in a way that actually works on big endian systems.
2010-04-11 01:53:03 +00:00
pooka
b6d93fa026 For the nfs throttling kludge, test against v_tag == VT_NFS instead
of v_op (the latter imposes linkage).
2010-03-02 21:32:29 +00:00
jym
2c546c52cc Change RSS (resident set size) limit. Instead of setting it arbitrarily
to the total free memory available to the system, use the smallest value
between VM_MAXUSER_ADDRESS and total free memory (having a RSS limit
bigger than VM_MAXUSER_ADDRESS has no real meaning).

Fix a possible int overflow when ptoa(uvmexp.free) is bigger than 4GB
with a 32 bits vaddr_t.

Reviewed by bouyer@.

See also http://mail-index.netbsd.org/tech-kern/2010/02/24/msg007395.html
2010-02-25 23:10:49 +00:00
uebayasi
af0ced746a Merge more indirect functions. Some comments. 2010-02-24 15:58:26 +00:00
uebayasi
f736e76fa5 uvm_fault_upper_lookup, uvm_fault_upper_neighbor: There is no point to call
pmap_update() without calling pmap_enter().

(Probably calling only once after loop (as done in uvm_fault_lower_lookup())
is enough.  If done so, other threads see entered neighbor pages as reflected
a little latter.)
2010-02-24 06:18:19 +00:00
uebayasi
efa838c11f Minor clean up. 2010-02-24 05:26:28 +00:00
uebayasi
a9659a6dfb Revert a thinko. 2010-02-24 05:00:55 +00:00
uebayasi
3bf5a87956 Slightly clean up uvm_fault() code path after pmap_enter(). Now tasks
needed for page cache are concentrated in own functions (uvm_fault_*_done()).
2010-02-24 04:32:58 +00:00
uebayasi
79440fb3fd Record if "promote" is done in UVMHIST. Do it for "upper" fault too. 2010-02-24 04:20:45 +00:00
uebayasi
b8168569e9 Merge some indirect "lower" fault handlers back. Prompted by rmind@. 2010-02-24 04:18:09 +00:00
jym
7bf36164a7 - Use ctob() instead of ptoa() to obtain physical addresses from frame
numbers. Using ptoa() will cast to vaddr_t, which might no be adequate
for architectures where sizeof(paddr_t) > sizeof(vaddr_t) (like i386 PAE).

- small fix inside AGP heuristics to avoid masking high order bits for
systems with more than 4GB.

Reviewed by bouyer@.

See also http://mail-index.netbsd.org/tech-kern/2010/02/22/msg007373.html
2010-02-24 00:01:11 +00:00
drochner
9783f258ba rename the va0_disabled option and cpp conditional to "disable" as well,
for consistency, and document option and sysctl flag
2010-02-21 13:17:50 +00:00
drochner
1a101ef360 rename the new sysctl to "vm.user_va0_disable", for consistency
with the majority of existing sysctl flags, suggested by yamt
2010-02-20 13:21:58 +00:00
drochner
dda500d0d8 Disable mapping of virtual address 0 by user programs per default.
This blocks an easy exploit of kernel bugs leading to dereference
of a NULL pointer on some architectures (eg i386).
The check can be disabled in various ways:
-by CPP definitions in machine/types.h (portmaster's choice)
-by a kernel config option USER_VA0_DISABLED_DEFAULT=0
-at runtime by sysctl vm.user_va0_disabled (cannot be cleared
 at securelevel>0)
2010-02-18 14:57:01 +00:00
joerg
d621e29eca Remove separate mb_map. The nmbclusters is computed at boot time based
on the amount of physical memory and limited by NMBCLUSTERS if present.
Architectures without direct mapping also limit it based on the kmem_map
size, which is used as backing store. On i386 and ARM, the maximum KVA
used for mbuf clusters is limited to 64MB by default.

The old default limits and limits based on GATEWAY have been removed.
key_registered_sb_max is hard-wired to a value derived from 2048
clusters.
2010-02-08 19:02:25 +00:00
mlelstv
3e39181b49 pgo_get needs the page array to be initialized. 2010-02-08 00:02:50 +00:00
mlelstv
5158187dbc Move assertion to make check more clear. 2010-02-08 00:01:39 +00:00
mlelstv
676d68d9ae Make UVMHIST build again. 2010-02-07 23:25:07 +00:00
mlelstv
dfd86ea3a8 Use filesystem blocks to address filesystem objects. f_iosize just
happens to be the same for current filesystems.
2010-02-07 15:51:28 +00:00
uebayasi
2903e6d834 __inline -> inline 2010-02-06 12:10:59 +00:00
uebayasi
6f78b42add Make vm_physseg lookup routines take the target vm_physseg. This is for the
coming "managed" device segments.
2010-02-06 02:56:17 +00:00
uebayasi
f0cafea3a4 vnode.h is not used here. 2010-02-05 03:49:11 +00:00
uebayasi
31973b36e6 Cosmetic. Shorten some long names. 2010-02-05 02:27:15 +00:00
uebayasi
2b1c626381 Fix !DIAGNOSTIC build. Reported by Geoff Wing. 2010-02-05 00:55:31 +00:00
uebayasi
e1b26edd82 Reduce diff between upper/lower neighbor handlers. 2010-02-04 05:48:26 +00:00
uebayasi
58625d0ac1 Merge "obfuscating layers" for readability. Inline some functions.
Requested by rmind@.
2010-02-04 03:32:21 +00:00
uebayasi
c0d0f35c5f Move uvm_fault_* static func decls in one place. 2010-02-04 03:19:08 +00:00
uebayasi
d0c590668f A few assertions & comments. 2010-02-03 14:02:49 +00:00
uebayasi
cfde757678 uvm_fault_lower_generic_io: Reduce diff from uvm_loanuobj(). 2010-02-03 12:40:39 +00:00
uebayasi
cd14256881 uvm_fault_lower_generic_io: One missing mutex_exit(vmobjlock). Found while
comparing this function with uvm_loanuobj().  (Part of) these should be
merged.
2010-02-03 07:48:18 +00:00
uebayasi
5f0a89783a uobj->pgops->pgo_get doing PGO_SYNCIO returns a uobjpage whose uobj backpointer
refers to another "uobj" used to call pgo_get.  Revert the wrong assertion
I made.  My bad.

(This and pgo_get's possible ERESTART return value check is the only 2 behavioral
changes I made.)

Reported by drochner@, thanks.
2010-02-02 18:49:23 +00:00
uebayasi
0dceadd765 Don't pass an unnecessary reference to uvm_loanbreak_anon().
Requested by rmind@.
2010-02-02 17:40:43 +00:00
wiz
27b261aabe Missing 'if defined COMPAT13 or COMPAT50' in uvm_swap.c found by cppcheck
and reported by Henning Petersen in PR 42721.
2010-02-02 15:00:34 +00:00
uebayasi
5526267c10 Be consistent to decide if PMAP_WIRED or not. 2010-02-02 06:52:59 +00:00
uebayasi
19fbe1698d Move A->K loan break code to uvm_loan.c. 2010-02-02 06:06:02 +00:00
uebayasi
be06afbe24 Indent. 2010-02-02 05:58:16 +00:00
uebayasi
a79520c14e uvm_fault: Split "neighbor" fault and loan handling into functions. 2010-02-02 04:35:35 +00:00
uebayasi
49bcc1198b Sort struct uvm_faultctx members for better alignment. 2010-02-02 01:54:48 +00:00
uebayasi
689dab24ea Indent. 2010-02-01 16:12:36 +00:00
uebayasi
d99dd23d72 More split. 2010-02-01 16:08:27 +00:00
uebayasi
49c8580e07 Fix build without DIAGNOSTIC. 2010-02-01 11:58:39 +00:00
uebayasi
ef11535fc6 uvm_fault: Clarify when to wire what. 2010-02-01 10:22:40 +00:00