* No need for the atomically changed variables to be declared as
volatile.
* Drop support for atomically getting and setting unaligned data.
* Introduce atomic_get_and_set[64]() which works the same as
atomic_set[64]() used to. atomic_set[64]() does not return the
previous value anymore.
Since ICI arguments are used to send addresses in some places, uint32 is
not sufficient on x86_64. addr_t still refers to the same type as uint32
(unsigned long) on other platforms, so this change only really affects
x86_64.
boot CPU wait until all other CPUs are ready to wait. This solves a
theoretical problem in main(): The boot CPU could run fully through the early
initialization and reset sCpuRendezvous2 before the other CPUs left
smp_cpu_rendezvous(). It's very unlikely on real hardware that the non-boot
CPUs are so much slower, but it might be a concern in emulation.
git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@36558 a95241bf-73f2-0310-859d-f6bbb57e9c96
* Use atomic_{and,or}() instead of atomic_set(), as there are no built-ins
for the latter.
git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@35021 a95241bf-73f2-0310-859d-f6bbb57e9c96
implemented by means of an additional member in cpu_ent.
* Removed thread::keep_scheduled and the related functions. The feature
wasn't used yet and wouldn't have worked as implemented anyway.
* Resurrected an older, SMP aware version of our simple scheduler and made it
the default instead of the affine scheduler. The latter is in no state to
be used yet. It causes enormous latencies (I've seen up to 0.1s) even when
six or seven CPUs were idle at the same time, totally killing parallelism.
That's also the reason why a -j8 build was slower than a -j2. This is no
longer the case. On my machine the -j2 build takes about 10% less time now
and the -j8 build saves another 20%. The latter is not particularly
impressive (compared with Linux), but that seems to be due to lock
contention.
git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@34615 a95241bf-73f2-0310-859d-f6bbb57e9c96
* Keep track of the currently running threads.
* Make use of that info to decide if a thread that becomes ready should preempt
the running thread.
* If we should preempt we send the target CPU a reschedule message.
* This preemption strategy makes keeping track of idle CPUs by means of a bitmap
superflous and it is therefore removed.
* Right now only other CPUs are preempted though, not the current one.
* Add missing initialization of the quantum tracking code.
* Do not extend the quantum of the idle thread based quantum tracking as we want
it to not run longer than necessary. Once the preemption works completely
adding a quantum timer for the idle thread will become unnecessary though.
* Fix thread stealing code, it did missed the last thread in the run queue.
* When stealing, try to steal the highest priority thread that is currently
waiting by taking priorities into account when finding the target run queue.
* Simplify stealing code a bit as well.
* Minor cleanups.
git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@32503 a95241bf-73f2-0310-859d-f6bbb57e9c96
* SMP:
- Added smp_send_broadcast_ici_interrupts_disabled(), which is basically
equivalent to smp_send_broadcast_ici(), but is only called with interrupts
disabled and gets the CPU index, so it doesn't have to use
smp_get_current_cpu() (which dereferences the current thread).
- Added cpu index parameter to smp_intercpu_int_handler().
* x86:
- arch_int.c -> arch_int.cpp
- Set up an IDT per CPU. We were using a single IDT for all CPUs, but that
can't work, since we need different tasks for the double fault interrupt
vector.
- Set the per CPU double fault task gates correctly.
- Renamed set_intr_gate() to set_interrupt_gate and set_system_gate() to
set_trap_gate() and documented them a bit.
- Renamed double_fault_exception() x86_double_fault_exception() and fixed
it not to use smp_get_current_cpu(). Instead we have the new
x86_double_fault_get_cpu() that deducts the CPU index from the used stack.
- Fixed the double_fault interrupt handler: It no longer calls int_bottom to
avoid accessing the current thread.
* debug.cpp:
- Introduced explicit debug_double_fault() to enter the kernel debugger from
a double fault handler.
- Avoid using smp_get_current_cpu().
- Don't use kprintf() before sDebuggerOnCPU is set. Otherwise
acquire_spinlock() is invoked by arch_debug_serial_puts().
Things look a bit better when the current thread pointer is broken -- we run
into kernel_debugger_loop() and successfully print the "Welcome to KDL"
message -- but we still dereference the thread pointer afterwards, so that we
don't get a usable kernel debugger yet.
git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@32050 a95241bf-73f2-0310-859d-f6bbb57e9c96
* Introduced flag "invoke_scheduler" in the per CPU structure. It is
evaluated in hardware_interrupt() (x86 only ATM).
* Introduced SMP_MSG_RESCHEDULE_IF_IDLE message, which enters the
scheduler when the CPU currently runs an idle thread.
* Don't do dprintf() "CPU x halted!" when handling a SMP_MSG_CPU_HALT
ICI message. It uses nested spinlocks and could thus potentially
deadlock itself (acquire_spinlock() processes ICI messages, so it
could already hold one of the locks). This is a pretty likely scenario
on machines with more than two CPUs, but is also possible when the
panic()ing thread holds the threads spinlock. Probably fixes#2572.
* Reworked the way the kernel debugger is entered and added a "cpu"
command that allows switching the CPU once in KDL. It is thus possible
to get a stack trace of the thread not on the panic()ing CPU.
* When a thread is added to the run queue, we do now check, if another
CPU is idle and ask it to reschedule, if it is. Before this change, the
CPU was continuing to idle until the quantum of the idle thread
expired. Speeds up the libbe.so build about 8% on my machine (haven't
tested the full Haiku image build yet).
* When spinlock debugging is enabled (DEBUG_SPINLOCKS) we also record
the spinlock acquirer on non-smp machines. Added "spinlock" debugger
command to get the info.
* Added debugger commands "ici" and "ici_message", printing info on
pending ICI message respectively on a given one.
* Process not only a single ICI message in acquire_spinlock() and other
places, but all pending ones.
* Also process ICI messages when waiting for a free one -- avoids a
potential deadlock.
* Mask out non-existing CPUs in send_multicast_ici(). panic() instead of
just returning when there's no target CPU left.
git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@28223 a95241bf-73f2-0310-859d-f6bbb57e9c96
* Added smp_send_multicast_ici(), which sends the message to all CPUs
specified via a mask.
git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@27910 a95241bf-73f2-0310-859d-f6bbb57e9c96
B_DEBUG_SPINLOCK_CONTENTION is defined to 1. It typedefs spinlock to a
structure (thus breaking BeOS binary compatibility), containing a
counter which is incremented whenever a thread has to wait for the
spinlock.
* Added macros for spinlock initialization and access and changed
code using spinlocks accordingly. This breaks compilation for BeOS --
the macros should be defined in the respective compatibility wrappers.
* Added generic syscall to get the spinlock counters for the thread and
the team spinlocks.
git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@25752 a95241bf-73f2-0310-859d-f6bbb57e9c96
Basically, there was a pretty subtle race between the cpus in main where if the main cpu released the AP cpus and then before the AP cpus had a chance to run the boot cpu started creating the main thread (which causes smp ici messages to be created) the system would livelock, where the boot cpu waited forever for the AP cpu to acknowledge the ICI (for a TLB flush when creating the kernel stack).
Added smp_cpu_rendezvous(), used to synchronize all the cpus to a particular point, and used it a few times in main().
While i was at it i fixed another race that'll probably never happen, but what the hey. Make sure the kernel args are copied into kernel space by the main cpu before letting any other ones use it.
git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@20269 a95241bf-73f2-0310-859d-f6bbb57e9c96
kernel TLBs from being flushed on context switch.
* new arch_cpu_user_TLB_invalidate() that now does what arch_cpu_global_TLB_invalidate()
did before.
* arch_cpu_global_TLB_invalidate() will now flush all TLBs, even those from the
kernel.
* some cleanups.
git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@15535 a95241bf-73f2-0310-859d-f6bbb57e9c96
The "data_ptr" parameter is now only freed if you specify SMP_MSG_FLAG_FREE_ARG
when sending the ICI message.
git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@14519 a95241bf-73f2-0310-859d-f6bbb57e9c96
Extracted scheduler_init() from start_scheduler() (which is now called scheduler_start()).
Moved scheduler related function prototypes from thread.h to the new scheduler.h.
Cleanup.
git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@14518 a95241bf-73f2-0310-859d-f6bbb57e9c96
some broken C++ export definitions, added missing licenses etc.
git-svn-id: file:///srv/svn/repos/haiku/trunk/current@3152 a95241bf-73f2-0310-859d-f6bbb57e9c96
