when mmapping a file, permissions are checked as it should be. When
mprotect()-ing the address range afterwards, no protection was checked
regarding the protection of the file originally opened. So
when you open /usr/bin/su RDONLY and SHARED you could afterwards change
the mmapped region to READ|WRITE. This gave the possibility to obtain
root privs obviously.
Here are some fixes I derived from the mach 3.0 VM system a couple of months
ago. At the time, I was giving the memory object routines a good looking
at, trying to fix the long-standing problem where vm_object_collapse()
sometimes fails to collapse objects left over from the exit of a forked
child. As bde has noted, the problem seems to occur when portions of the
parent are paged out. These "lost" memory objects, which can eat up a huge
amount of swap space, are reclaimed when the parent responsible for the
fork()s is killed.
it seems to keep the vm system from deadlocking the system when it runs
out of swap + physical memory.
prevents the system from giving the last page(s) to anything but the
referenced "processes" (especially important is the pager process,
which should never have to wait for a free page).
mycroft