Commit Graph

2188 Commits

Author SHA1 Message Date
Pawel Dziepak
d0f2d8282f Merge branch 'scheduler'
Conflicts:
	build/jam/packages/Haiku
	headers/os/kernel/OS.h
	headers/os/opengl/GLRenderer.h
	headers/private/shared/cpu_type.h
	src/add-ons/kernel/drivers/power/acpi_battery/acpi_battery.h
	src/bin/sysinfo.cpp
	src/bin/top.c
	src/system/kernel/arch/x86/arch_system_info.cpp
	src/system/kernel/port.cpp
2014-01-17 04:06:15 +01:00
Pawel Dziepak
a2634874ed scheduler: Estimate the load thread is able to produce
Previous implementation based on the actual load of each core and share
each thread has in that load turned up to be very problematic when
balancing load on very heavily loaded systems (i.e. more threads
consuming all available CPU time than there is logical CPUs).

The new approach is to estimate how much load would a thread produce
if it had all CPU time only for itself. Summing such load estimations
of each thread assigned to a given core we get a rank that contains
much more information than just simple actual core load.
2014-01-09 03:50:54 +01:00
Pawel Dziepak
4ca31ac964 kernel: Fix ABA problem in try_acquire_read_spinlock() 2014-01-06 21:03:42 +01:00
Pawel Dziepak
8cf8e53774 kernel/x86: Inline atomic functions and memory barriers 2014-01-06 09:08:53 +01:00
Pawel Dziepak
cb66faef24 scheduler: Work around GCC2 limitations in function inlining
GCC2 won't inline a function if it is used before its definition.
2014-01-06 01:22:53 +01:00
Pawel Dziepak
81e04d7b97 kernel: Remove cpu_info::load
This field forces kernel to track each CPU load all the time. It is not
a problem with the current scheduler on a multicore systems, but on
single core machnies or with any other future scheduler this field may
become just an unnecessary burden. It isn't difficult for an application
to compute CPU load by itself when it needs it.
2014-01-03 19:44:57 +01:00
Pawel Dziepak
135bb9c959 kernel: Do not attempt to interrupt a thread that is not waiting 2013-12-31 03:53:56 +01:00
Pawel Dziepak
15a7f2046a kernel: Return CPU load in cpu_info 2013-12-29 22:47:06 +01:00
Pawel Dziepak
e46f284adc kernel/util: MinMaxHeap: Fix unused variable warnings 2013-12-29 22:47:05 +01:00
Pawel Dziepak
a47974dd06 scheduler: Use heap to determine highest priority thread 2013-12-29 19:24:01 +01:00
Pawel Dziepak
3309bf33c7 kernel/util: Make MinMaxHeap::Peek*() const 2013-12-23 03:52:13 +01:00
Pawel Dziepak
b89b5d3826 x86: Make arch/smp.h a C++ only header 2013-12-20 22:05:26 +01:00
Pawel Dziepak
b258298c70 kernel: Protect cpu_ent::active_time with sequential lock
atomic_{get, set}64() are problematic on architectures without 64 bit
compare and swap.
Also, using sequential lock instead of atomic access ensures that
any reads from cpu_ent::active_time won't require any writes to shared
memory.
2013-12-20 01:31:32 +01:00
Pawel Dziepak
6639514437 x86[_64]: Support assigning MSI IRQs to arbitrary CPU 2013-12-20 01:07:08 +01:00
Pawel Dziepak
e3d001ff02 x86: Implement multicast ICIs 2013-12-19 19:35:44 +01:00
Pawel Dziepak
1b06228f13 kernel: Propagate scheduler modes to cpu{freq, idle} modules 2013-12-17 23:26:37 +01:00
Pawel Dziepak
4fcbac58de kernel: Make get_cpu_topology() return a pointer to const
The client code is not supposed to change the topology info.
It would be also nice if cpu_topology_node::children was an array of
pointers to const but that would require several const_casts in the
topology tree generation code so it's probably not worth it.
2013-12-17 22:08:18 +01:00
Pawel Dziepak
735f67481f x86: Debugger can now use dr3 2013-12-17 04:31:29 +01:00
Pawel Dziepak
a5b070f1fa x86: Store pointer to the current thread in gs:0
Apparently, reading from dr3 is slower than reading from memory
with cache hit.

Also, depending on hypervisor configuration, accessing dr3 may cause
a VM exit (and, at least on kvm, it does), what makes it much slower
than a memory access even when there is a cache miss.
2013-12-17 04:08:51 +01:00
Pawel Dziepak
611376fef7 x86: Let each CPU have its own GDT 2013-12-17 03:57:20 +01:00
Pawel Dziepak
7b9668170c kernel, boot: Increase CPU limit to 64 2013-12-16 05:00:02 +01:00
Pawel Dziepak
d02aaee17e kernel, libroot: Add more memory info in system_info
system_info now contains all information previously available only
through __get_system_info_etc(B_MEMORY_INFO, ...).
2013-12-16 04:53:46 +01:00
Pawel Dziepak
1bc7045fdf kernel, libroot: Introduce new API for obtaining system info 2013-12-16 03:58:43 +01:00
Ingo Weinhold
9bdb5e0a92 kernel safemode: Add functions to get options early
Add get_safemode_option_early() and get_safemode_boolean_early() to get
safemode options before the kernel heap has been initialized. They use a
simplified parser.
2013-12-10 19:05:47 +01:00
Ingo Weinhold
7a1e706df7 safemode.h: Update coding style 2013-12-10 17:07:20 +01:00
Pawel Dziepak
3e0e3be760 boot, kernel: Replace MAX_BOOT_CPUS with SMP_MAX_CPUS 2013-12-06 19:43:08 +01:00
Pawel Dziepak
f0fe981769 kernel: Fix gcc2 build 2013-12-06 03:31:20 +01:00
Pawel Dziepak
2b04d8ab31 x86/paging: Use CPUSet instead of cpu_mask_t 2013-12-06 03:27:48 +01:00
Pawel Dziepak
7629d527c5 kernel: Use CPUSet in ICI code instead of cpu_mask_t 2013-12-06 03:08:39 +01:00
Pawel Dziepak
730882d615 kernel: Remove RunQueueLink.h 2013-12-05 23:28:42 +01:00
Pawel Dziepak
52b442a687 kernel: smp_cpu_rendezvous(): Use counter instead of bitmap 2013-12-05 23:10:04 +01:00
Pawel Dziepak
d287274dce scheduler: Code refactoring 2013-12-05 22:47:30 +01:00
Ingo Weinhold
7b83ce1142 Add KDL command "mapping"
* VMTranslationMap:
  - Add DebugPrintMappingInfo(): Given a virtual address it is supposed
    to print the paging structure information for that address. To be
    implemented by derived classes.
  - Add DebugGetReverseMappingInfo(): Given a physical addresss it is
    supposed to find all virtual addresses mapped to it. To be
    implemented by derived classes.
* X86VMTranslationMapPAE: Implement the new methods
  DebugPrintMappingInfo() and DebugGetReverseMappingInfo().
* Add KDL command "mapping". It supports both virtual address lookups
  and reverse lookups.
2013-12-05 05:13:21 +01:00
Ingo Weinhold
7bf85edf58 Allow disabling ASLR via DISABLE_ASLR environment variable
* VMAddressSpace: Add randomizingEnabled property.
* VMUserAddressSpace: Randomize addresses only when randomizingEnabled
  property is set.
* create_team_arg(): Check, if the team's environment contains
  "DISABLE_ASLR=1". Set the team's address space property
  randomizingEnabled accordingly in load_image_internal() and
  exec_team().
2013-12-01 02:51:50 +01:00
Ingo Weinhold
e2c5cf691d boot loader: Menu: Add SetTitle() 2013-11-30 19:56:26 +01:00
Ingo Weinhold
cbfc38c62d boot loader: MenuItem: Add SetLabel() 2013-11-30 19:39:57 +01:00
Pawel Dziepak
2b7ea4cddf kernel: Remove Thread::next_state 2013-11-29 19:31:10 +01:00
Pawel Dziepak
673f08a995 kernel: Make thread_block_locked() private 2013-11-29 05:30:50 +01:00
Pawel Dziepak
7f8cf14b71 scheduler, kernel/util: Fix style issues 2013-11-29 04:51:38 +01:00
Pawel Dziepak
3514fd77f7 kernel: Reduce lock contention when processing ICIs 2013-11-29 03:36:44 +01:00
Pawel Dziepak
286b341a40 kernel: Merge two occurences of thread resume code 2013-11-28 14:03:57 +01:00
Pawel Dziepak
7db89e8dc3 kernel: Rework cpuidle module
* Create new interface for cpuidle modules (similar to the cpufreq
   interface)
 * Generic cpuidle module is no longer needed
 * Fix and update Intel C-State module
2013-11-25 23:50:27 +01:00
Ingo Weinhold
dd11460a49 Add a hex dump utility for use in KDL commands 2013-11-25 12:45:36 +01:00
Pawel Dziepak
0e94a12f8e kernel: Make CACHE_LINE_ALIGN visible in the whole kernel 2013-11-25 00:35:15 +01:00
Pawel Dziepak
03f7d3d1db kernel: Restore logical processor disabling 2013-11-24 22:51:07 +01:00
Pawel Dziepak
14e36af649 kernel[/util]: Fix debug builds 2013-11-21 03:24:54 +01:00
Pawel Dziepak
024541a4c8 kernel: Improve rw_spinlock implementation
* Add more debug checks
 * Reduce the number of executed instructions that lock the bus.
2013-11-21 02:25:03 +01:00
Pawel Dziepak
308f594e2a kernel, libroot: Make scheduler modes interface public 2013-11-20 23:32:40 +01:00
Ingo Weinhold
c04f3a625a boot loader: Add safe mode blacklist submenu
It's a browser for the system package content, where entries can be
selected to blacklist them. The selected entries are removed from the
packagefs instance in the boot loader, so that e.g. selected drivers
won't be picked up. The paths are also added to the safe mode driver
settings and will be interpreted when the system packagefs instance is
mounted by the kernel.
2013-11-20 16:00:35 +01:00
Ingo Weinhold
6c7abe9829 boot loader: Menu[Item] API improvements
* Make Menu and MenuItem polymorphic.
* MenuItem:
  - Make SetMarked() virtual, so it can be overridden.
  - Add SetSubmenu() and Supermenu().
  - Delete the submenu in the destructor.
* Menu:
  - Add Entered()/Exited() hooks. They frame the time the user navigates
    the menu or any of its submenus. The hooks allow for subclasses
    populating their item list dynamically.
  - Add SortItems().
* Update boot loader menu copyright text to include 2013, now that it is
  over soon. :-)
2013-11-20 16:00:34 +01:00
Ingo Weinhold
435fb01509 DoublyLinkedList: Add Sort() 2013-11-20 16:00:34 +01:00
Ingo Weinhold
7e7f482590 SinglyLinkedList: Missing include 2013-11-20 16:00:34 +01:00
Pawel Dziepak
9c2e74da04 scheduler: Move mode specific logic to separate files 2013-11-20 09:46:59 +01:00
Ingo Weinhold
2fdd1d9ef1 khash: Move string hash functions to own header/source file
Unlike khash they shouldn't be phased out (only renamed).
2013-11-19 15:08:34 +01:00
Pawel Dziepak
d897a478d7 kernel: Allow reassigning IRQs to logical processors 2013-11-18 04:55:25 +01:00
Pawel Dziepak
955c7edec2 kernel: Measure time spent in interrupt handlers 2013-11-18 01:50:37 +01:00
Pawel Dziepak
6a164daad4 kernel: Track load produced by interrupt handlers 2013-11-18 01:17:44 +01:00
Pawel Dziepak
288a2664a2 scheduler: Remove sSchedulerInternalLock
* pin idle threads to their specific CPUs
 * allow scheduler to implement SMP_MSG_RESCHEDULE handler
 * scheduler_set_thread_priority() reworked
 * at reschedule: enqueue old thread after dequeueing the new one
2013-11-13 05:31:58 +01:00
Pawel Dziepak
829f836324 scheduler: Minor cleanup 2013-11-12 04:42:12 +01:00
Ingo Weinhold
e5f6591382 VM: vm_memset_physical(): Correct length parameter type 2013-11-11 22:27:52 +01:00
Pawel Dziepak
03fb2d8868 kernel: Remove gSchedulerLock
* Thread::scheduler_lock protects thread state, priority, etc.
 * sThreadCreationLock protects thread creation and removal and list of
   threads in team.
 * Team::signal_lock and Team::time_lock protect list of threads in team
   as well.
 * Scheduler uses its own internal locking.
2013-11-08 02:41:26 +01:00
Pawel Dziepak
72addc62e0 kernel: Introduce Thread::time_lock and Team::time_lock 2013-11-07 22:16:36 +01:00
Axel Dörfler
547cd462f8 trim: Added is_called_via_syscall() function.
* And use it in get_trim_data_from_user(), formerly known as copy_*().
* This fixes differentiating between user and kernel buffers.
2013-11-07 19:06:13 +01:00
Axel Dörfler
99086aa323 trim: Target SCSI UNMAP command instead of WRITE SAME.
* The UNMAP command is theoretically much faster, as it can get many block
  ranges instead of just a single range.
* Furthermore, the ATA TRIM command resembles it much better.
* Therefore, fs_trim_data now gets an array of ranges, and we use SCSI UNMAP
  to trim.
* Updated BFS code to collect array ranges to fully support the new
  fs_trim_data possibilities.
2013-11-07 19:03:47 +01:00
Pawel Dziepak
3519eb334a kernel: Change Thread::team_lock to rw_spinlock 2013-11-07 04:20:59 +01:00
Pawel Dziepak
defee266db kernel: Add read write spinlock implementation 2013-11-07 04:20:32 +01:00
Pawel Dziepak
d3e5752b11 scheduler: Performance mode is actually low latency mode 2013-11-07 01:50:20 +01:00
Pawel Dziepak
83983eaf38 kernel: Remove Thread::alarm 2013-11-07 01:40:02 +01:00
Pawel Dziepak
aa4aca0264 kernel: Protect signal data with Team::signal_lock 2013-11-07 01:32:48 +01:00
Pawel Dziepak
73ad2473e7 Remove remaining unnecessary 'volatile' qualifiers 2013-11-06 00:03:07 +01:00
Pawel Dziepak
273f2f38cd kernel: Improve spinlock implementation
atomic_or() and atomic_and() are not supported by x86 are need to be
emulated using CAS. Use atomic_get_and_set() and atomic_set() instead.
2013-11-05 22:47:18 +01:00
Pawel Dziepak
077c84eb27 kernel: atomic_*() functions rework
* No need for the atomically changed variables to be declared as
   volatile.
 * Drop support for atomically getting and setting unaligned data.
 * Introduce atomic_get_and_set[64]() which works the same as
   atomic_set[64]() used to. atomic_set[64]() does not return the
   previous value anymore.
2013-11-05 22:32:59 +01:00
Pawel Dziepak
f4b088a992 kernel: Protect UserTimers with sUserTimerLock 2013-11-05 05:36:05 +01:00
Pawel Dziepak
4824f7630b kernel: Add sequential lock implementation 2013-11-05 04:16:13 +01:00
Pawel Dziepak
3c819aaa72 kernel: DPC: remove schedulerLocked argument 2013-11-04 23:51:18 +01:00
Pawel Dziepak
11cacd0c13 kernel: Remove thread_block_with_timeout_locked() 2013-11-04 23:45:14 +01:00
Jérôme Duval
e2183a14c4 Increased kernel stack size by another page for 64-bit
* USB boot now works on x86_64 with PM.
2013-11-04 18:53:49 +01:00
Pawel Dziepak
d8fcc8a825 kernel: Remove B_TIMER_ACQUIRE_SCHEDULER_LOCK flag
The flag main purpose is to avoid race conditions between event handler
and cancel_timer(). However, cancel_timer() is safe even without
using gSchedulerLock.

If the event is scheduled to happen on other CPU than the CPU that
invokes cancel_timer() then cancel_timer() either disables the event
before its handler starts executing or waits until the event handler
is done.

If the event is scheduled on the same CPU that calls cancel_timer()
then, since cancel_timer() disables interrupts, the event is either
executed before cancel_timer() or when the timer interrupt handler
starts running the event is already disabled.
2013-10-31 01:49:43 +01:00
Pawel Dziepak
c8dd9f7780 kernel: Add thread_unblock() and use it where possible 2013-10-30 03:58:36 +01:00
Pawel Dziepak
9c0ff0eed1 kernel: Add cpufreq module for Intel P-states
Since Sandy Bridge managing P-states on Intel processors is much easier
and more powerful than when using previous versions of EIST.
2013-10-30 00:55:03 +01:00
Pawel Dziepak
22d8248267 kernel: Add support and interface for cpufreq modules 2013-10-30 00:48:07 +01:00
Julian Harnath
7f64b301b1 Reduce lock contention in kernel port subsystem.
* Replace ports list mutex with R/W-lock.

* Move team port list protection to separate array of mutexes.
  Relieve contention on sPortsLock by removing Team::port_list from its
  protected items. With this, set_port_owner() only needs to acquire the
  sPortsLock for reading.

* Add another hash table holding the ports by name. Used by find_port()
  so it doesn't have to iterate over the list anymore.

* Use slab-based memory allocator for port messages. sPortQuotaLock was
  acquired on every message send or receive and was thus another point
  of contention. The lock is not necessary anymore.

* Lock for port hashes and Port::lock are no longer locked in a nested
  fashion to reduce chances of blocking other threads.

* Make operations concurrency-safe by adding an atomically accessed
  Port::state which provides linearization points to port creation and
  deletion. Both operations are now divided into logical and physical
  parts, the logical part just updating the state and the physical part
  adding/remove it to/from the port hash and team port list.

* set_port_owner() is the only remaining function which still locks
  Port::lock and one or two of sTeamListLock[] in a nested fashion.
  Since it needs to move the port from one team list to another and
  change Port::owner, there's no way around.

* Ports are now reference counted to make accesses to already-deleted
  ports safe.

* Should fix #8007.
2013-10-26 16:10:03 +02:00
Pawel Dziepak
978fc08065 scheduler: Remove support for running different schedulers
Simple scheduler behaves exactly the same as affine scheduler with a
single core. Obviously, affine scheduler is more complicated thus
introduces greater overhead but quite a lot of multicore logic has been
disabled on single core systems in the previous commit.
2013-10-24 02:04:03 +02:00
Pawel Dziepak
ed8627e535 kernel/util: Fix MinMaxHeap::_GrowHeap() 2013-10-24 00:59:58 +02:00
Pawel Dziepak
31a75d402f kernel: Protect lock internals with per-lock spinlock 2013-10-24 00:01:18 +02:00
Pawel Dziepak
4c4994435d kernel/util: Fixes in [MinMax]Heap implementation 2013-10-22 23:56:31 +02:00
Pawel Dziepak
5cf9b69b49 kernel/util: Minor improvements in Heap and MinMaxHeap
* [MinMax]Heap::ModifyKey(): Do not attempt to move node if the key
   actually hasn't changed.
 * Allow allocating initial array at construction.
2013-10-21 21:24:05 +02:00
Pawel Dziepak
7ea42e7add kernel: Remove invoke_scheduler_if_idle 2013-10-21 02:38:57 +02:00
Pawel Dziepak
ea79da9500 kernel: Remove support for thread_queue 2013-10-21 02:30:20 +02:00
Pawel Dziepak
cd8d4e39fd kernel: Introduce scheduler modes of operation 2013-10-21 02:17:00 +02:00
Pawel Dziepak
343c489689 kernel: Create CPU topology tree 2013-10-21 01:33:35 +02:00
Pawel Dziepak
5cbf227236 kernel/util: Allocate only one array in MinMaxHeap 2013-10-20 23:33:55 +02:00
Pawel Dziepak
18c0d163ed kernel/util: Add MinMaxHeap implementation 2013-10-17 19:22:29 +02:00
Pawel Dziepak
278c9784a1 scheduler_affine: Use global core heap and per-core CPU heaps
There is a global heap of cores, where the key is the highest priority
of threads running on that core. Moreover, for each core there is
a heap of logical processors on this core where the key is the priority
of currently running thread.

The per-core heap is used for load balancing among logical processors
on that core. The global heap is used in initial decision where to put
the thread (note that the algorithm that makes this decision is not
complete yet).
2013-10-17 02:11:28 +02:00
Pawel Dziepak
cf863a5040 kernel: Decide whether to use simple or affine scheduler
Simple scheduler is used when we do not have to worry about cache affinity
(i.e. single core with or without SMT, multicore with all cache levels
shared).

When we replace gSchedulerLock with more fine grained locking affine
scheduler should also be chosen when logical CPU count is high (regardless
of cache).
2013-10-16 18:39:25 +02:00
François Revol
bc1184c253 bootloader: Add an arguments_count field to stage2_args
Some boot platforms pass a non-NULL-terminated list of args
to the loader, so store the count here to avoid having to copy
the list itself.
2013-10-15 22:15:03 +02:00
Pawel Dziepak
f20ad54be2 kernel: Add support for SMP systems to simple scheduler
In SMP systems simple scheduler will be used only when all logical
processors share all levels of cache and the number of CPUs is low.
In such systems we do not have to care about cache affinity and
the contention on the lock protecting shared run queue is low. Single
run queue makes load balancing very simple.
2013-10-15 00:29:04 +02:00
Pawel Dziepak
29e65827fd kernel: Remove possibility to yield to all threads
Kernel support for yielding to all (including lower priority) threads
has been removed. POSIX sched_yield() remains unchanged.

If a thread really needs to yield to everyone it can reduce its priority
to the lowest possible and then yield (it will then need to manually
return to its prvious priority upon continuing).
2013-10-09 20:42:34 +02:00
Pawel Dziepak
fee8009184 kernel: Add another penalty for CPU bound threads
Each thread has its minimal priority that depends on the static priority.
However, it is still able to starve threads with even lower priority
(e.g. CPU bound threads with lower static priority). To prevent this
another penalty is introduced. When the minimal priority is reached
penalty (count mod minimal_priority) is added, where count is the number
of time slices since the thread reached its minimal priority. This prevents
starvation of lower priorirt threads (since all CPU bound threads may have
their priority temporaily reduced to 1) but preserves relation between
static priorities - when there are two CPU bound threads the one with
higher static priority would get more CPU time.
2013-10-09 20:13:47 +02:00
Pawel Dziepak
4ade765cab kernel/util: Add heap implementation 2013-10-09 04:26:09 +02:00