Commit Graph

48165 Commits

Author SHA1 Message Date
Pawel Dziepak
273f2f38cd kernel: Improve spinlock implementation
atomic_or() and atomic_and() are not supported by x86 are need to be
emulated using CAS. Use atomic_get_and_set() and atomic_set() instead.
2013-11-05 22:47:18 +01:00
Pawel Dziepak
077c84eb27 kernel: atomic_*() functions rework
* No need for the atomically changed variables to be declared as
   volatile.
 * Drop support for atomically getting and setting unaligned data.
 * Introduce atomic_get_and_set[64]() which works the same as
   atomic_set[64]() used to. atomic_set[64]() does not return the
   previous value anymore.
2013-11-05 22:32:59 +01:00
Pawel Dziepak
e7dba861fd kernel: User{Event, Timer}: Use atomic access where necessary 2013-11-05 20:28:25 +01:00
Pawel Dziepak
f4b088a992 kernel: Protect UserTimers with sUserTimerLock 2013-11-05 05:36:05 +01:00
Pawel Dziepak
4824f7630b kernel: Add sequential lock implementation 2013-11-05 04:16:13 +01:00
Pawel Dziepak
958f6d00aa kernel: Make UserEvent::Fire() work without gSchedulerLock held 2013-11-04 23:53:20 +01:00
Pawel Dziepak
3c819aaa72 kernel: DPC: remove schedulerLocked argument 2013-11-04 23:51:18 +01:00
Pawel Dziepak
11cacd0c13 kernel: Remove thread_block_with_timeout_locked() 2013-11-04 23:45:14 +01:00
Pawel Dziepak
c2763aaffb kernel: Add spinlock for undertaker data 2013-10-31 02:34:09 +01:00
Pawel Dziepak
d8fcc8a825 kernel: Remove B_TIMER_ACQUIRE_SCHEDULER_LOCK flag
The flag main purpose is to avoid race conditions between event handler
and cancel_timer(). However, cancel_timer() is safe even without
using gSchedulerLock.

If the event is scheduled to happen on other CPU than the CPU that
invokes cancel_timer() then cancel_timer() either disables the event
before its handler starts executing or waits until the event handler
is done.

If the event is scheduled on the same CPU that calls cancel_timer()
then, since cancel_timer() disables interrupts, the event is either
executed before cancel_timer() or when the timer interrupt handler
starts running the event is already disabled.
2013-10-31 01:49:43 +01:00
Pawel Dziepak
c8dd9f7780 kernel: Add thread_unblock() and use it where possible 2013-10-30 03:58:36 +01:00
Pawel Dziepak
d70728f54d kernel/lock: Do not use *_locked() functions when not needed 2013-10-30 03:26:13 +01:00
Pawel Dziepak
d54a9e0a41 kernel: Do not use gSchedulerLock when accesing UID and GID
Reads and writes to uid_t and gid_t are atomic anyway. The only real
problem that may happen here is inconsistent state of triples
effective_{u, g}id, saved_set_{u, g}id, real_{u, g}id, but team locks
protect us against that.
2013-10-30 02:57:45 +01:00
Pawel Dziepak
9c0ff0eed1 kernel: Add cpufreq module for Intel P-states
Since Sandy Bridge managing P-states on Intel processors is much easier
and more powerful than when using previous versions of EIST.
2013-10-30 00:55:03 +01:00
Pawel Dziepak
1e3cf82d85 scheduler: Manage CPU performance 2013-10-30 00:49:24 +01:00
Pawel Dziepak
22d8248267 kernel: Add support and interface for cpufreq modules 2013-10-30 00:48:07 +01:00
Pawel Dziepak
6d96f462dc scheduler: Use load information to migrate threads 2013-10-28 02:44:46 +01:00
Pawel Dziepak
5e2701a2b5 scheduler: Keep track of the load each thread produces 2013-10-28 01:38:54 +01:00
Pawel Dziepak
dc38e6ca87 scheduler: Use core load to distribute threads 2013-10-28 00:39:16 +01:00
Pawel Dziepak
d80cdf504f scheduler: Keep track of core and logical CPU load 2013-10-27 22:39:56 +01:00
Pawel Dziepak
890ba7415c scheduler: Decide whether to cancel thread penalty 2013-10-27 20:05:20 +01:00
Pawel Dziepak
1df2e75540 scheduler: Increase penalty of waiting threads
The fact that thread is waiting doesn't mean that it is nice to the others.
If the thread, indeed, waits for a longer time its penalty will be cancelled
anyway, however if the thread waits for a very short time do not count that
as being nice since lower priority threads didn't have much chance to run.
2013-10-27 18:14:48 +01:00
Pawel Dziepak
45ff530069 scheduler: Be more demanding when cancelling penalties 2013-10-25 04:12:13 +02:00
Pawel Dziepak
9d7e2acf34 scheduler: Update scheduler_set_operation_mode() 2013-10-24 03:24:53 +02:00
Pawel Dziepak
978fc08065 scheduler: Remove support for running different schedulers
Simple scheduler behaves exactly the same as affine scheduler with a
single core. Obviously, affine scheduler is more complicated thus
introduces greater overhead but quite a lot of multicore logic has been
disabled on single core systems in the previous commit.
2013-10-24 02:04:03 +02:00
Pawel Dziepak
e927edd376 scheduler_affine: Disable logic not needed on current topology 2013-10-24 01:33:12 +02:00
Pawel Dziepak
ed8627e535 kernel/util: Fix MinMaxHeap::_GrowHeap() 2013-10-24 00:59:58 +02:00
Pawel Dziepak
7e1ecb9315 kernel: Protect scheduler_set_thread_priority() with lock 2013-10-24 00:59:10 +02:00
Pawel Dziepak
31a75d402f kernel: Protect lock internals with per-lock spinlock 2013-10-24 00:01:18 +02:00
Pawel Dziepak
d6efe8ee75 kernel: Update cpu_ent::active_time atomically 2013-10-23 21:56:14 +02:00
Pawel Dziepak
453bf75027 scheduler_affine: Try not to get overloaded by small tasks 2013-10-23 21:32:27 +02:00
Pawel Dziepak
2df11d8a80 scheduler_affine: Put small tasks on a single core 2013-10-23 01:59:25 +02:00
Pawel Dziepak
4c4994435d kernel/util: Fixes in [MinMax]Heap implementation 2013-10-22 23:56:31 +02:00
Pawel Dziepak
8d471bc3d9 scheduler_affine: Store cores with CPU bound threads separately
This is preparation for small task packing. We want to have as many idle
cores as possible. To achieve that we put all threads on the most heavily
loaded core (so the other ones can become idle). However, we don't really
want to do that if there are CPU bound tasks and if any of the cores
becomes overloaded.
2013-10-22 23:52:40 +02:00
Pawel Dziepak
f823aacf59 scheduler_affine: Remove old code 2013-10-22 01:21:51 +02:00
Pawel Dziepak
2e0ee59462 scheduler_affine: Migrate threads from overloaded cores
* Keep number of CPU bound threads on cores balanced.
 * If possible migrate normal threads from cores with cpu bound ones to
   the less busy cores.
2013-10-22 01:18:03 +02:00
Pawel Dziepak
7aba623f52 scheduler_affine: Balance number of threads assigned to CPUs
When the thread cannot be run immediately assign it to the core with
lowest number of CPU bound threads and assigned threads.
2013-10-21 21:39:40 +02:00
Pawel Dziepak
5cf9b69b49 kernel/util: Minor improvements in Heap and MinMaxHeap
* [MinMax]Heap::ModifyKey(): Do not attempt to move node if the key
   actually hasn't changed.
 * Allow allocating initial array at construction.
2013-10-21 21:24:05 +02:00
Pawel Dziepak
afe1735d7d scheduler_affine: Expire old cache affinities
Performance mode:
 If there have been a lot of activity on the core since the thread went
 sleep its data in cache probably has been overwritten.

Power saving mode:
 If the thread went to sleep a long time ago either there has been a
 lot of activity on its core or the core has been idle and it may
 be more efficient to wake another one.
2013-10-21 19:19:28 +02:00
Pawel Dziepak
7ea42e7add kernel: Remove invoke_scheduler_if_idle 2013-10-21 02:38:57 +02:00
Pawel Dziepak
ea79da9500 kernel: Remove support for thread_queue 2013-10-21 02:30:20 +02:00
Pawel Dziepak
74192fd984 scheduler_affine: Fix compilation warning 2013-10-21 02:21:14 +02:00
Pawel Dziepak
84812e6033 scheduler_affine: Correctly assign CPUs to idle threads 2013-10-21 02:20:09 +02:00
Pawel Dziepak
cd8d4e39fd kernel: Introduce scheduler modes of operation 2013-10-21 02:17:00 +02:00
Pawel Dziepak
4b446279e6 scheduler_affine: Use CPU topology tree to create ID mappings 2013-10-21 01:34:31 +02:00
Pawel Dziepak
343c489689 kernel: Create CPU topology tree 2013-10-21 01:33:35 +02:00
Pawel Dziepak
5cbf227236 kernel/util: Allocate only one array in MinMaxHeap 2013-10-20 23:33:55 +02:00
Pawel Dziepak
a6d4233e59 scheduler_affine: Choose wisely which core to wake up
The longer core is idle the deeper idle state it has entered. That's
why the scheduler should always choose the core that has gone idle
most recently (both for performance and power saving reasons).

Moreover, if there are more than one package the scheduler should
minimize the number of packages with at least one core active when
power saving is the priority. Contrary, as many packages as possible
should be used when aiming for high performance.
2013-10-20 23:26:32 +02:00
Pawel Dziepak
da3a48f4a8 scheduler_affine: Use min-max heap as per-core CPU heap 2013-10-17 19:23:27 +02:00
Pawel Dziepak
18c0d163ed kernel/util: Add MinMaxHeap implementation 2013-10-17 19:22:29 +02:00