Haneef Mubarak
429025634e
resolve #201 with a platform-selective REP MOVSB implementation
2020-05-26 16:04:28 -07:00
daan
cb05ef9f2c
merge from dev
2020-05-19 10:43:46 -07:00
daan
a7d2bc8ad6
edit warning messages to be more consistent
2020-05-19 10:16:28 -07:00
daan
dcb3574cf0
fix assertions for huge segment free
2020-05-01 21:14:41 -07:00
daan
1f396e64a0
merge from dev
2020-03-16 16:41:21 -07:00
daan
f2ac272baa
strengthen alignment check for memalign and aligned_alloc
2020-02-17 09:59:11 -08:00
daan
a96e94f940
change TLS slot on OpenBSD
2020-02-02 22:46:38 -08:00
daan
3560e0a867
fix TLS slot number on OSX
2020-02-02 22:15:09 -08:00
daan
f3c47c7c91
improved malloc zone handling on macOSX (not working yet)
2020-02-02 21:03:09 -08:00
daan
757dcc8411
extend interpose for macOSX
2020-02-02 19:07:26 -08:00
daan
12c4108abe
update comments
2020-02-02 16:09:09 -08:00
daan
07fbe4f80f
fixes for dragonfly
2020-02-02 14:31:28 -08:00
daan
8bc20631e4
fixes for freeBSD
2020-02-02 13:25:26 -08:00
daan
d2db9f1fc2
update thread local storage
2020-02-02 13:12:22 -08:00
daan
0989562c2d
add initial fast tls for macOSX
2020-02-01 16:57:00 -08:00
daan
fea903900d
use __thread locals on linux
2020-02-01 14:33:34 -08:00
daan
a169cf0e3f
merge dev-exp; add pthread TLS support for macOSX
2020-02-01 14:10:10 -08:00
daan
edff9d4fbb
merge from dev-win (padding)
2020-02-01 12:32:59 -08:00
daan
aa68b8cbc7
improve encoding of padding canary and buffer overflow detection
2020-02-01 12:16:37 -08:00
daan
40f1e1e07b
byte-precise heap block overflow checking with encoded padding
2020-01-31 23:39:51 -08:00
daan
68112a2751
better padding implementation, more precise statistics
2020-01-31 20:34:24 -08:00
daan
f820fae858
merge from dev; add retired collect on abandon
2020-01-30 07:24:35 -08:00
daan
a7c69ccbea
fix stat accounting of segments with huge blocks
2020-01-30 06:25:42 -08:00
daan
ed1c8a203a
improve performance with tls recursion counter
2020-01-29 23:08:52 -08:00
daan
03b363a1c2
first working tls on macOS using interpose; still slow
2020-01-29 22:46:44 -08:00
daan
b50bec463d
merge from dev-exp; better abandoned reclamation
2020-01-27 22:12:23 -08:00
daan
e628fc7067
cleanup reclaim logic
2020-01-26 12:39:11 -08:00
daan
b31bc52618
add cache alignment directives for contended variables
2020-01-24 19:02:13 -08:00
daan
4a2a0c2d50
initial abandon based on fine-grained reclamation
2020-01-23 19:50:35 -08:00
daan
c9106e74a8
remove __thread attribute from mimalloc.h
2020-01-23 11:06:25 -08:00
daan
6fb434a99b
use -fvisibility=hidden on clang as well
2020-01-23 10:31:47 -08:00
daan
394a7a92ab
merge from dev
2020-01-20 19:06:08 -08:00
daan
dc58388968
Add ability to register custom error function called on various error conditions; including ENOMEM
2020-01-17 19:59:55 -08:00
daan
6b7356a10a
make mimalloc compile with the highest warning level on msvc
2020-01-16 18:12:01 -08:00
daan
7a9502973d
rearrange STL allocator code: remove pragma, ifdef for C++11
2020-01-16 15:57:11 -08:00
daan
0099707af9
use delayed free for all pages; reduce size of the page structure for improved address calculation
2020-01-15 17:19:01 -08:00
daan
783e3377f7
add output argument to stat printing
2020-01-15 10:53:54 -08:00
daan
4a27ea1643
merge from dev
2020-01-13 18:01:34 -08:00
daan
12ef2816ed
fix bug exposed by commit 59fa286 where reclaimed pages could be stuck to NEVER_DELAYED
2020-01-08 19:00:03 -08:00
daan
2b108c8748
increase retire expiration to 4
2020-01-03 21:39:18 -08:00
daan
77134e1ad0
update free list encoding to stronger formula with addition last
2019-12-28 15:17:49 -08:00
daan
e3391d9a53
stronger encoding of free lists using two keys per page
2019-12-28 00:57:42 -08:00
daan
ba87a39d9f
updated random cookie generation using OS primitives and chacha20
2019-12-22 19:37:49 -08:00
daan
a799a19136
fix non-standard line continuation
2019-11-25 14:30:12 -08:00
Daan
fe51d11e61
Merge pull request #178 from carenas/master
...
avoid deadlock with BSD systems that call malloc from the dynamic linker
2019-11-25 14:28:58 -08:00
Carlo Marcelo Arenas Belón
41ef691292
avoid deadlock with BSD systems that call malloc from the dynamic linker
...
extend the exception used for macOS to cover also OpenBSD (tested in 6.4+)
and DragonFlyBSD (tested in 5.6.2)
2019-11-24 10:00:39 -08:00
daan
321e18777e
wip: delayed decommit on segments
2019-11-21 19:53:43 -08:00
daan
1066be1594
merge from dev-exp
2019-11-21 17:03:30 -08:00
daan
6568059cc6
merge from dev
2019-11-21 16:28:28 -08:00
daan
74dbfc30be
improved security by encoding NULL values; double free mitigation on by default; more precise free list corruption detection
2019-11-21 15:21:23 -08:00
Daan Leijen
211f1aa519
remove reset delay slots; add reset tracking per page and segment
2019-11-20 14:55:12 -08:00
daan
17ade3fca5
merge from dev
2019-11-13 17:52:20 -08:00
daan
a4ed63d127
Adresses pr #165 and issue #164 by @colesbury:
...
On Mac OS, the thread-local _mi_default_heap may get reset before
_mi_thread_done is called, leaking the default heap on non-main threads.
Now the current default heap is also stored in mi_pthread_key (or mi_fls_key
on Windows). The _mi_thread_done function is called with this value.
2019-11-13 17:22:03 -08:00
Daan Leijen
d01ed42bcb
replace max_numa_nodes by use_numa_nodes (to help with wrong detection of numa nodes on WSL for example)
2019-11-13 13:35:50 -08:00
daan
ef179a6377
avoid allocation at numa node detection on linux
2019-11-12 10:16:59 -08:00
daan
181bef382c
merge dev-exp
2019-11-10 11:39:53 -08:00
Daan Leijen
b04206a9d3
add os cache to arena
2019-11-10 10:10:10 -08:00
Daan Leijen
fed0068dac
merge from dev-exp; bitmap based arena
2019-11-10 07:56:40 -08:00
daan
378716c467
refactor and improve atomic bitmap usage
2019-11-07 10:26:52 -08:00
daan
829fd872f4
initial delay slots
2019-11-04 11:48:41 -08:00
daan
62df2e2df9
merge from dev-exp
2019-11-04 08:56:42 -08:00
daan
8afd06b248
use int64 for time (instead of double)
2019-11-04 08:44:40 -08:00
daan
1a6d150687
merge from dev-exp
2019-11-03 12:21:22 -08:00
daan
e320488791
add numa nodes to stats
2019-11-03 12:18:32 -08:00
daan
08c4726043
merge from dev-exp
2019-11-01 22:04:20 -07:00
daan
2c12d7f223
optimized numa calls; better Linux support
2019-11-01 22:01:52 -07:00
daan
6b26f0cb17
merge from dev-exp (numa support)
2019-11-01 20:08:56 -07:00
daan
a6499be074
initial numa support for arenas
2019-11-01 19:53:07 -07:00
daan
62984c0a24
merge from dev-exp
2019-10-31 18:44:48 -07:00
daan
d36d04b4a6
add arena for huge page management
2019-10-31 15:35:10 -07:00
daan
f7d2c45af3
initial experiment with fixed memory arena and sliced segments
2019-10-31 00:40:41 -07:00
daan
b73beede34
merge from dev
2019-10-30 15:19:34 -07:00
daan
b052d3b731
enable double free and heap corruption detection in debug mode
2019-10-28 15:54:33 -07:00
daan
9d4f57abf3
merge from dev-win
2019-10-28 12:33:01 -07:00
daan
25246070ae
fix double free check in secure = 4 mode; inline _mi_ptr_cookie
2019-10-19 08:34:18 -07:00
daan
2affdbbd2e
stronger secure mode when defining MI_SECURE=4: checks for double free, corrupted free list, and invalid pointer frees. Performance is impacted but not too much -- more perf testing is needed
2019-10-18 18:11:04 -07:00
daan
4b15e2ed97
merge from dev
2019-10-17 18:24:35 -07:00
daan
5de851a84d
update page_flags to have more portable definition
2019-10-17 16:48:16 -07:00
daan
f3a162f095
pick better umul_overflow variant based on size_t size
2019-10-17 09:52:10 -07:00
daan
4609537b8a
pick better umul_overflow variant based on intptr size
2019-10-17 09:47:12 -07:00
daan
24777f6a91
limit aligned allocation to power-of-two alignment
2019-09-11 17:49:28 -07:00
Daan Leijen
1909cfb346
refine mi_os_good_alloc_size and use it for huge pages (to ensure realloc is bounded as in #153
2019-09-10 13:26:51 -07:00
daan
ce81af1119
use mi_is_power_of_two when possible (pr #118 )
2019-09-09 08:12:50 -07:00
daan
7a91368108
pass page to block_zero_init and extend to block_size
2019-09-03 10:11:24 -07:00
daan
d1bd1644d5
support zero-initialized memory detection
2019-09-02 13:16:52 -07:00
daan
d6e35ffd83
Merge branch 'dev' into dev-exp
2019-09-02 10:10:08 -07:00
daan
8ef8ddecc5
Merge branch 'master' into dev
2019-09-02 09:53:34 -07:00
Carlo Marcelo Arenas Belón
0e35ec3879
avoid deadlock with OpenBSD
2019-09-01 01:06:01 -07:00
daan
f35f643508
merge from dev
2019-08-29 09:52:22 -07:00
daan
7bf12c7b5f
make output function configurable; remove <stdio.h> from standard includes
2019-08-29 09:42:50 -07:00
daan
9af51506a6
track commit and is_large status more precisely
2019-08-28 11:58:30 -07:00
daan
db8d443ae6
track more precisely if memory is fixed or committed
2019-08-26 22:45:26 -07:00
daan
25dca38ef9
merge from dev-win
2019-08-26 12:47:16 -07:00
daan
a654732d99
merge from dev new atomics
2019-08-26 12:30:03 -07:00
daan
2c19388bcf
initialize mimalloc options at process load
2019-08-26 11:44:41 -07:00
daan
6f5492cef8
enable initial lazy commit and optional decommit to reduce commit charge with many threads
2019-08-24 15:00:55 -07:00
daan
612b2cc9b7
clean up segment slice handling
2019-08-24 12:20:32 -07:00
daan
082f012a91
merge from dev-win
2019-08-23 21:56:28 -07:00
daan
6c6fcad242
remove threadid from pages and keep page flags separate (cherry picked)
2019-08-23 21:42:24 -07:00
daan
acde83543f
remove threadid from pages and keep page flags separate
2019-08-23 14:08:00 -07:00
daan
5c912f16d4
merge from remote
2019-08-21 11:35:09 -07:00
daan
a3c4b1c95b
merge from dev-win
2019-08-21 11:18:05 -07:00
daan
71121a7457
Merge branch 'dev-win' into dev-exp
2019-08-21 11:12:25 -07:00
daan
e7bd8c5d14
merge with dev-win
2019-08-21 11:11:40 -07:00
Daan Leijen
12257e5bc1
fix thread-id reset on page abandonment
2019-08-21 08:09:28 -07:00
Daan Leijen
cc6d14c55b
fix thread-id reset on page abandonment
2019-08-21 08:06:40 -07:00
daan
cd52d0a6d9
merge dev-win
2019-08-20 17:31:46 -07:00
daan
86d4ded831
Merge branch 'dev' into dev-exp
2019-08-20 08:27:46 -07:00
daan
05631ebfc4
Merge branch 'dev' into dev-win
2019-08-19 21:23:04 -07:00
daan
770e752578
fix parameter order on mul_overflow to fix static analysis warnings (pr #125 )
2019-08-19 19:15:04 -07:00
daan
42dedb00ec
initial implementation of mi_os_reserve_huge_pages
2019-08-19 11:10:06 -07:00
Daan Leijen
91497e8d2d
whitespace and warning fix
2019-08-16 17:49:49 -07:00
daan
a0b4ac2f66
new segment allocation; good results with Qas service
2019-08-15 23:19:52 -07:00
daan
f2ba95bc64
first working version of new segment allocation
2019-08-15 22:00:42 -07:00
daan
6ee248b012
wip: fixing bugs in new segment allocation
2019-08-15 14:40:15 -07:00
daan
f2bafbc57f
wip: new segment allocation
2019-08-15 11:49:56 -07:00
daan
bbd81bbbd1
wip: new segment allocation with flexible large objects
2019-08-15 00:46:45 -07:00
daan
6d11e59250
fix to avoid potential linear behavior in page collect
2019-08-14 07:46:38 -07:00
daan
f35c2c5201
improve message printing from redirection module
2019-08-11 16:38:58 -07:00
daan
ed785253bf
restructure the page flags to use explicit masks
2019-08-10 17:48:00 -07:00
daan
2fee6f98d7
Merge branch 'master' into dev
2019-08-10 15:23:43 -07:00
daan
5e56b40fe6
improve page flags handling
2019-08-08 17:18:49 -07:00
daan
6596e970a5
move in_full and has_aligned into page threadid for a single test in mi_free
2019-08-08 15:23:18 -07:00
daan
55778d2fe4
improved stats
2019-08-08 11:36:13 -07:00
daan
189ad0f81d
small optimizations, use bitwise aligne
2019-07-23 15:00:13 -07:00
Jim Huang
8dba36bcec
Use checked unsigned multiplication extension of GCC/Clang
...
Most processors have carry flags which they set on addition overflow, so
it is a good idea to access them whenever possible. Most of them also
have widening multiply instructions that can be used to detect overflow
of the non-widening version. Both GCC and Clang offer a way to detect an
overflow for security critical applications.
Reference:
https://clang.llvm.org/docs/LanguageExtensions.html#checked-arithmetic-builtins
2019-07-23 14:59:49 -07:00
daan
c228ecefd8
update for new dynamic direction on windows 64-bit
2019-07-18 18:59:32 -07:00
daan
7ea9cf8d1f
add region collection
2019-07-15 14:53:03 -07:00
Jim Huang
afd981d008
Use checked unsigned multiplication extension of GCC/Clang
...
Most processors have carry flags which they set on addition overflow, so
it is a good idea to access them whenever possible. Most of them also
have widening multiply instructions that can be used to detect overflow
of the non-widening version. Both GCC and Clang offer a way to detect an
overflow for security critical applications.
Reference:
https://clang.llvm.org/docs/LanguageExtensions.html#checked-arithmetic-builtins
2019-07-16 05:28:59 +08:00
daan
8dc0b5345b
Merge branch 'dev' into dev-exp
2019-07-14 16:50:38 -07:00
daan
49ceb4d018
refactor delayed_free code
2019-07-14 16:48:53 -07:00
daan
5ad2effb39
use standard compliant compilation of the thread_free atomic field
2019-07-14 16:20:27 -07:00
daan
84a292538c
merge with medium pages
2019-07-12 20:39:29 -07:00
daan
a927c07de9
merge medium pages
2019-07-12 20:27:56 -07:00
daan
42cf5e1580
reintroduce support for medium size pages
2019-07-12 20:11:39 -07:00
daan
8a6b474fa0
merge from dev
2019-07-11 16:32:11 -07:00
daan
9e9208eae8
Merge branch 'dev' into dev-win
2019-07-11 15:22:09 -07:00
daan
1fdb4b288f
more eager handling of non-local frees
2019-07-11 15:21:57 -07:00
daan
3192ca4682
merge with dev-win
2019-07-09 23:34:17 -07:00
daan
3009a7b1fe
support non- eager_commit; on larson reduces commit charge 10x, but other benchmarks often see no improvement
2019-07-09 21:34:56 -07:00
daan
0314373d93
add reset_decommit, reset_discard options; maintain more statistics
2019-07-09 20:24:00 -07:00
daan
129149977d
Merge branch 'dev' into dev-exp
2019-07-08 17:45:04 -07:00
daan
8203f3dcfa
add full heap aligned allocation api (issue #52 ); remove rezalloc (issue #63 ); add mi_version
2019-07-08 17:17:07 -07:00
daan
493dfc4b82
merge with dev
2019-07-08 15:42:42 -07:00
daan
56bdb2c2f3
add more const declarations (pr #79 )
2019-07-08 12:00:59 -07:00
daan
983c96a238
Merge branch 'dev' into dev-exp
2019-07-04 14:46:53 -07:00
daan
6abdc5f5a3
do not enable verbose by default in the debug build; add trace messages for verbose level 2
2019-07-04 09:28:22 -07:00
daan
5c0ffe9865
merge back changes to OS module from 'dev-exp'
2019-07-02 17:17:03 -07:00
daan
8cb36c62e6
merge with dev branch
2019-07-02 16:44:35 -07:00
daan
aeff1db32b
Fixing interpose on macOS
2019-07-02 12:49:28 -07:00
daan
d35fc6cdc4
PR#66 by kickunderscore to ensure consistent small block alignment
2019-07-02 08:46:50 -07:00
daan
06bcea1761
Initial commit of separate memory region layer and improved large OS pages support, see 'memory.c'
2019-07-02 07:23:24 -07:00
daan
d6901558cd
enable large OS page support on Windows
2019-06-28 22:35:57 -07:00
daan
12d756e7b2
fix page count of heap when absorbing
2019-06-27 13:32:44 -07:00
Julian Fang
c19d3a27a8
Remove underscore prefixes to match c lanuguage standard
2019-06-26 12:57:13 +08:00
daan
fac0d4b409
fix passing void as argument
2019-06-25 20:06:34 -07:00
Frank Denis
236cd2e65c
fn() is not a valid C prototype
2019-06-25 12:16:36 +02:00
daan
c1226611c6
Merge branch 'master' into dev
2019-06-24 18:13:04 -07:00
daan
cd7aa0ce83
improve handling of huge pages in the cache
2019-06-23 23:15:42 -07:00
Jim Huang
a667971cbf
Apple rebrands "OS X" as "macOS"
...
In 2012, with the release of OS X 10.8 Mountain Lion, the name of the
system was shortened from Mac OS X to OS X. In 2016, with the release
of macOS 10.12 Sierra, the name was changed from OS X to macOS to
streamline it with the branding of Apple's other primary operating
systems: iOS, watchOS, and tvOS.
Reference: https://en.wikipedia.org/wiki/MacOS
2019-06-24 10:15:50 +08:00
Daan
fab7329c7a
Merge pull request #24 from jserv/clarify-license
...
Clarify top-level license file
2019-06-23 10:47:38 -07:00
Jim Huang
b1d299b35b
Clarify top-level license file
...
To be consistent with other Microsoft open source projects, each source
file points to the file "LICENSE" in top-level directory.
2019-06-23 19:53:34 +08:00
Jim Huang
3841b32f9d
Support ARM/Aarch64 TLS register fastpath
...
Tested with Ubuntu Linux 18.04 LTS running on Marvell/Cavium ThunderX,
which consists of Armv8 based processors.
2019-06-23 16:30:20 +08:00
Daan Leijen
79bfe52255
fix accidental delete
2019-06-22 11:50:03 -07:00
Daan Leijen
e52ca3bfa9
possible fix for ARM linux compilation
2019-06-22 11:33:18 -07:00
daan
26a874eb3f
initial checkin
2019-06-19 16:26:12 -07:00
