From 81e619b9a4ed0deee578d31977baba601060df66 Mon Sep 17 00:00:00 2001 From: ad Date: Sat, 6 Oct 2007 01:09:48 +0000 Subject: [PATCH] Update for jemalloc. --- lib/libc/stdlib/malloc.3 | 534 +++++++++++++++++---------------------- 1 file changed, 232 insertions(+), 302 deletions(-) diff --git a/lib/libc/stdlib/malloc.3 b/lib/libc/stdlib/malloc.3 index 97262a303f24..ef50b1563031 100644 --- a/lib/libc/stdlib/malloc.3 +++ b/lib/libc/stdlib/malloc.3 @@ -1,5 +1,3 @@ -.\" $NetBSD: malloc.3,v 1.24 2006/04/24 21:54:37 wiz Exp $ -.\" .\" Copyright (c) 1980, 1991, 1993 .\" The Regents of the University of California. All rights reserved. .\" @@ -32,18 +30,13 @@ .\" SUCH DAMAGE. .\" .\" @(#)malloc.3 8.1 (Berkeley) 6/4/93 -.\" From FreeBSD: Id: malloc.3,v 1.18 1999/03/28 14:16:04 phk Exp +.\" $FreeBSD: src/lib/libc/stdlib/malloc.3,v 1.73 2007/06/15 22:32:33 jasone Exp $ .\" -.Dd April 24, 2006 +.Dd June 15, 2007 .Dt MALLOC 3 .Os .Sh NAME -.Nm malloc , -.Nm calloc , -.Nm realloc , -.Nm free -.\"XXX", -.\"XXX".Nm reallocf +.Nm malloc , calloc , realloc , free .Nd general purpose memory allocation functions .Sh LIBRARY .Lb libc @@ -55,31 +48,18 @@ .Fn calloc "size_t number" "size_t size" .Ft void * .Fn realloc "void *ptr" "size_t size" -.\"XXX".Ft void * -.\"XXX".Fn reallocf "void *ptr" "size_t size" .Ft void .Fn free "void *ptr" -.Ft char * -.Va malloc_options ; +.Ft const char * +.Va _malloc_options ; .Sh DESCRIPTION The .Fn malloc function allocates .Fa size -bytes of memory. +bytes of uninitialized memory. The allocated space is suitably aligned (after possible pointer coercion) for storage of any type of object. -If the space is at least -.Em pagesize -bytes in length (see -.Xr getpagesize 3 ) , -the returned memory will be page boundary aligned as well. -If -.Fn malloc -fails, a -.Dv NULL -pointer is returned, and the errno variable is set to -.Er ENOMEM . .Pp The .Fn calloc @@ -93,7 +73,8 @@ The result is identical to calling .Fn malloc with an argument of .Dq "number * size" , -with the exception that the allocated memory is initialized to all bits zero. +with the exception that the allocated memory is explicitly initialized +to zero bytes. .Pp The .Fn realloc @@ -101,16 +82,18 @@ function changes the size of the previously allocated memory referenced by .Fa ptr to .Fa size -bytes and returns a pointer to the (possibly moved) object. +bytes. The contents of the memory are unchanged up to the lesser of the new and old sizes. If the new size is larger, the value of the newly allocated portion of the memory is undefined. -If the requested memory cannot be allocated, -.Dv NULL -is returned and the memory referenced by +Upon success, the memory referenced by .Fa ptr -is valid and unchanged. +is freed and a pointer to the newly allocated memory is returned. +Note that +.Fn realloc +may move the memory allocation, resulting in a different return value than +.Fa ptr . If .Fa ptr is @@ -121,44 +104,6 @@ function behaves identically to .Fn malloc for the specified size. .Pp -When using -.Fn realloc -one must be careful to avoid the following idiom: -.Pp -.Bd -literal -offset indent -nsize += 50; -if ((p = realloc(p, nsize)) == NULL) - return (NULL); -.Ed -.Pp -Do not adjust the variable describing how much memory has been allocated -until one knows the allocation has been successful. -This can cause aberrant program behavior if the incorrect size value is used. -In most cases, the above sample will also result in a leak of memory. -As stated earlier, a return value of -.Dv NULL -indicates that the old object still remains allocated. -Better code looks like this: -.Bd -literal -offset indent -newsize = size + 50; -if ((p2 = realloc(p, newsize)) == NULL) { - if (p) - free(p); - p = NULL; - return (NULL); -} -p = p2; -size = newsize; -.Ed -.\"XXX".Pp -.\"XXX"The -.\"XXX".Fn reallocf -.\"XXX"function call is identical to the realloc function call, except that it -.\"XXX"will free the passed pointer when the requested memory cannot be allocated. -.\"XXX"This is a FreeBSD -.\"XXX"specific API designed to ease the problems with traditional coding styles -.\"XXX"for realloc causing memory leaks in libraries. -.Pp The .Fn free function causes the allocated memory referenced by @@ -169,17 +114,20 @@ If is .Dv NULL , no action occurs. +.Pp .Sh TUNING Once, when the first call is made to one of these memory allocation routines, various flags will be set or reset, which affect the -workings of this allocation implementation. +workings of this allocator implementation. .Pp -The ``name'' of the file referenced by the symbolic link named +The +.Dq name +of the file referenced by the symbolic link named .Pa /etc/malloc.conf , the value of the environment variable .Ev MALLOC_OPTIONS , and the string pointed to by the global variable -.Va malloc_options +.Va _malloc_options will be interpreted, in that order, character by character as flags. .Pp Most flags are single letters, @@ -188,46 +136,61 @@ and lowercase means that the behavior is not set, or off. .Bl -tag -width indent .It A All warnings (except for the warning about unknown -flags being set), and failure to allocate memory become fatal. +flags being set) become fatal. The process will call -.Fn abort 3 +.Xr abort 3 in these cases. +.It H +Use +.Xr madvise 2 +when pages within a chunk are no longer in use, but the chunk as a whole cannot +yet be deallocated. +This is primarily of use when swapping is a real possibility, due to the high +overhead of the +.Fn madvise +system call. .It J Each byte of new memory allocated by -.\"XXX".Fn malloc , -.\"XXX".Fn realloc -.\"XXX"or -.\"XXX".Fn reallocf -.Fn malloc -or +.Fn malloc , .Fn realloc -as well as all memory returned by -.\"XXX".Fn free , -.\"XXX".Fn realloc -.\"XXX"or -.\"XXX"Fn reallocf -.Fn free or +.Fn reallocf +will be initialized to 0xa5. +All memory returned by +.Fn free , .Fn realloc -will be initialized to 0xd0. -This option also sets the -.Dq R -option. +or +.Fn reallocf +will be initialized to 0x5a. This is intended for debugging and will impact performance negatively. -.It H -Pass a hint to the kernel about pages unused by the allocation functions. -This will help performance if the system is paging excessively. -This option is off by default. -.It R -Causes the -.Fn realloc -.\"XXX"and -.\"XXX".Fn reallocf -.\"XXX"functions -function -to always reallocate memory even if the initial allocation was -sufficiently large. -This can substantially aid in compacting memory. +.It K +Increase/decrease the virtual memory chunk size by a factor of two. +The default chunk size is 1 MB. +This option can be specified multiple times. +.It N +Increase/decrease the number of arenas by a factor of two. +The default number of arenas is four times the number of CPUs, or one if there +is a single CPU. +This option can be specified multiple times. +.It P +Various statistics are printed at program exit via an +.Xr atexit 3 +function. +This has the potential to cause deadlock for a multi-threaded process that exits +while one or more threads are executing in the memory allocation functions. +Therefore, this option should only be used with care; it is primarily intended +as a performance tuning aid during application development. +.It Q +Increase/decrease the size of the allocation quantum by a factor of two. +The default quantum is the minimum allowed by the architecture (typically 8 or +16 bytes). +This option can be specified multiple times. +.It S +Increase/decrease the size of the maximum size class that is a multiple of the +quantum by a factor of two. +Above this size, power-of-two spacing is used for size classes. +The default value is 512 bytes. +This option can be specified multiple times. .It U Generate .Dq utrace @@ -238,36 +201,39 @@ Consult the source for details on this option. .It V Attempting to allocate zero bytes will return a .Dv NULL -pointer instead of a valid pointer. +pointer instead of +a valid pointer. (The default behavior is to make a minimal allocation and return a pointer to it.) This option is provided for System V compatibility. +This option is incompatible with the +.Dq X +option. .It X Rather than return failure for any allocation function, -display a diagnostic message on stderr and cause the program to drop +display a diagnostic message on +.Dv stderr +and cause the program to drop core (using -.Fn abort 3 ) . +.Xr abort 3 ) . This option should be set at compile time by including the following in the source code: .Bd -literal -offset indent -extern char *malloc_options; -malloc_options = "X"; +_malloc_options = "X"; .Ed .It Z -This option implicitly sets the -.Dq J +Each byte of new memory allocated by +.Fn malloc , +.Fn realloc +or +.Fn reallocf +will be initialized to 0. +Note that this initialization only happens once for each byte, so +.Fn realloc and -.Dq R -options, and then zeros out the bytes that were requested. +.Fn reallocf +calls do not zero memory that was previously allocated. This is intended for debugging and will impact performance negatively. -.It \*[Lt] -Reduce the size of the cache by a factor of two. -The default cache size is 16 pages. -This option can be specified multiple times. -.It \*[Gt] -Double the size of the cache by a factor of two. -The default cache size is 16 pages. -This option can be specified multiple times. .El .Pp The @@ -277,93 +243,68 @@ and options are intended for testing and debugging. An application which changes its behavior when these options are used is flawed. -.Sh RETURN VALUES -The -.Fn malloc -and -.Fn calloc -functions return a pointer to the allocated memory if successful; otherwise a -.Dv NULL -pointer is returned and -.Va errno -is set to -.Er ENOMEM . +.Sh IMPLEMENTATION NOTES +This allocator uses multiple arenas in order to reduce lock contention for +threaded programs on multi-processor systems. +This works well with regard to threading scalability, but incurs some costs. +There is a small fixed per-arena overhead, and additionally, arenas manage +memory completely independently of each other, which means a small fixed +increase in overall memory fragmentation. +These overheads are not generally an issue, given the number of arenas normally +used. +Note that using substantially more arenas than the default is not likely to +improve performance, mainly due to reduced cache performance. +However, it may make sense to reduce the number of arenas if an application +does not make much use of the allocation functions. .Pp -The -.Fn realloc -.\"XXX"and -.\"XXX".Fn reallocf -.\"XXX"functions return -function returns -a pointer, possibly identical to -.Fa ptr , -to the allocated memory if successful; otherwise a -.Dv NULL -pointer is returned and -.Va errno -is set to -.Er ENOMEM , -in which case the -memory referenced by -.Fa ptr -is still available and intact. +Memory is conceptually broken into equal-sized chunks, where the chunk size is +a power of two that is greater than the page size. +Chunks are always aligned to multiples of the chunk size. +This alignment makes it possible to find metadata for user objects very +quickly. .Pp -The -.Fn free -function returns no value. -.Sh ENVIRONMENT -The following environment variables affect the execution of the allocation -functions: -.Bl -tag -width MMM -.It Ev MALLOC_OPTIONS -If the environment variable -.Ev MALLOC_OPTIONS -is set, the characters it contains will be interpreted as flags to the -allocation functions. -.El -.Sh FILES -.Bl -tag -width "/etc/malloc.conf" -.It Pa /etc/malloc.conf -symbolic link to filename containing option flags -.El -.Sh EXAMPLES -To set a systemwide reduction of cache size, and to dump core whenever -a problem occurs: +User objects are broken into three categories according to size: small, large, +and huge. +Small objects are no larger than one half of a page. +Large objects are smaller than the chunk size. +Huge objects are a multiple of the chunk size. +Small and large objects are managed by arenas; huge objects are managed +separately in a single data structure that is shared by all threads. +Huge objects are used by applications infrequently enough that this single +data structure is not a scalability issue. .Pp -.Bd -literal -offset indent -ln -s 'A\*[Lt]' /etc/malloc.conf -.Ed +Each chunk that is managed by an arena tracks its contents in a page map as +runs of contiguous pages (unused, backing a set of small objects, or backing +one large object). +The combination of chunk alignment and chunk page maps makes it possible to +determine all metadata regarding small and large allocations in constant time. .Pp -To specify in the source that a program does no return value checking -on calls to these functions: -.Bd -literal -offset indent -extern char *malloc_options; -malloc_options = "X"; -.Ed +Small objects are managed in groups by page runs. +Each run maintains a bitmap that tracks which regions are in use. +Allocation requests that are no more than half the quantum (see the +.Dq Q +option) are rounded up to the nearest power of two (typically 2, 4, or 8). +Allocation requests that are more than half the quantum, but no more than the +maximum quantum-multiple size class (see the +.Dq S +option) are rounded up to the nearest multiple of the quantum. +Allocation requests that are larger than the maximum quantum-multiple size +class, but no larger than one half of a page, are rounded up to the nearest +power of two. +Allocation requests that are larger than half of a page, but small enough to +fit in an arena-managed chunk (see the +.Dq K +option), are rounded up to the nearest run size. +Allocation requests that are too large to fit in an arena-managed chunk are +rounded up to the nearest multiple of the chunk size. +.Pp +Allocations are packed tightly together, which can be an issue for +multi-threaded applications. +If you need to assure that allocations do not suffer from cache line sharing, +round your allocation requests up to the nearest multiple of the cache line +size. .Sh DEBUGGING MALLOC PROBLEMS -The major difference between this implementation and other allocation -implementations is that the free pages are not accessed unless allocated, -and are aggressively returned to the kernel for reuse. -.Bd -filled -offset indent -Most allocation implementations will store a data structure containing a -linked list in the free chunks of memory, -used to tie all the free memory together. -That can be suboptimal, -as every time the free-list is traversed, -the otherwise unused, and likely paged out, -pages are faulted into primary memory. -On systems which are paging, -this can result in a factor of five increase in the number of page-faults -done by a process. -.Ed -.Pp -A side effect of this architecture is that many minor transgressions on -the interface which would traditionally not be detected are in fact detected. -As a result, programs that have been running happily for -years may suddenly start to complain loudly, when linked with this -allocation implementation. -.Pp -The first and most important thing to do is to set the +The first thing to do is to set the .Dq A option. This option forces a coredump (if possible) at the first sign of trouble, @@ -373,15 +314,15 @@ It is probably also a good idea to recompile the program with suitable options and symbols for debugger support. .Pp If the program starts to give unusual results, coredump or generally behave -differently without emitting any of the messages listed in the next section, -it is likely because it depends on the storage being filled with nul bytes. -Try running it with +differently without emitting any of the messages mentioned in the next +section, it is likely because it depends on the storage being filled with +zero bytes. +Try running it with the .Dq Z option set; if that improves the situation, this diagnosis has been confirmed. If the program still misbehaves, -the likely problem is accessing memory outside the allocated area, -more likely after than before the allocated area. +the likely problem is accessing memory outside the allocated area. .Pp Alternatively, if the symptoms are not easy to reproduce, setting the .Dq J @@ -393,92 +334,99 @@ option, if supported by the kernel, can provide a detailed trace of all calls made to these functions. .Pp Unfortunately this implementation does not provide much detail about -the problems it detects, the performance impact for storing such information +the problems it detects; the performance impact for storing such information would be prohibitive. -There are a number of allocation implementations available on the 'Net -which focus on detecting and pinpointing problems by trading performance -for extra sanity checks and detailed diagnostics. +There are a number of allocator implementations available on the Internet +which focus on detecting and pinpointing problems by trading performance for +extra sanity checks and detailed diagnostics. .Sh DIAGNOSTIC MESSAGES -If -.Fn malloc , -.Fn calloc , -.Fn realloc -or -.Fn free -detect an error or warning condition, -a message will be printed to file descriptor STDERR_FILENO. +If any of the memory allocation/deallocation functions detect an error or +warning condition, a message will be printed to file descriptor +.Dv STDERR_FILENO . Errors will result in the process dumping core. If the .Dq A option is set, all warnings are treated as errors. .Pp -The following is a brief description of possible error messages and -their meanings: -.Pp -.Bl -tag -width indent -.It "(ES): mumble mumble mumble -The allocation functions were compiled with -.Dq EXTRA_SANITY -defined, and an error was found during the additional error checking. -Consult the source code for further information. -.It "allocation failed -If the -.Dq A -option is specified it is a fatal error for an allocation function to fail. -.It "mmap(2) failed, check limits -This most likely means that the system is dangerously overloaded or that -the process' limits are incorrectly specified. -.It "freelist is destroyed -The internal free-list has been corrupted. -.El -.Pp -.Bl -tag -width indent -The following is a brief description of possible warning messages and -their meanings: -.Pp -.It "chunk/page is already free -The process attempted to -.Fn free -memory which had already been freed. -.It "junk pointer ... -A pointer specified to one of the allocation functions points outside the -bounds of the memory of which they are aware. -.It "malloc() has never been called -No memory has been allocated, -yet something is being freed or -realloc'ed. -.It "modified (chunk-/page-) pointer -The pointer passed to -.Fn free -or -.Fn realloc -has been modified. -.It "pointer to wrong page -The pointer that -.Fn malloc -or -.Fn calloc -is trying to free does not reference a possible page. -.It "recursive call -A process has attempted to call an allocation function recursively. -This is not permitted. -In particular, signal handlers should not attempt to allocate memory. -.It "out of memory The -.Dq X -option was specified and an allocation of memory failed. -.It "unknown char in MALLOC_OPTIONS -An unknown option was specified. -Even with the -.Dq A -option set, this warning is still only a warning. +.Va _malloc_message +variable allows the programmer to override the function which emits +the text strings forming the errors and warnings if for some reason +the +.Dv stderr +file descriptor is not suitable for this. +Please note that doing anything which tries to allocate memory in +this function is likely to result in a crash or deadlock. +.Pp +All messages are prefixed by +.Dq Ao Ar progname Ac Ns Li : (malloc) . +.Sh RETURN VALUES +The +.Fn malloc +and +.Fn calloc +functions return a pointer to the allocated memory if successful; otherwise +a +.Dv NULL +pointer is returned and +.Va errno +is set to +.Er ENOMEM . +.Pp +The +.Fn realloc +function returns a pointer, possibly identical to +.Fa ptr , +to the allocated memory +if successful; otherwise a +.Dv NULL +pointer is returned, and +.Va errno +is set to +.Er ENOMEM +if the error was the result of an allocation failure. +The +.Fn realloc +function always leaves the original buffer intact +when an error occurs, whereas +.Fn reallocf +deallocates it in this case. +.Pp +The +.Fn free +function returns no value. +.Sh ENVIRONMENT +The following environment variables affect the execution of the allocation +functions: +.Bl -tag -width ".Ev MALLOC_OPTIONS" +.It Ev MALLOC_OPTIONS +If the environment variable +.Ev MALLOC_OPTIONS +is set, the characters it contains will be interpreted as flags to the +allocation functions. .El +.Sh EXAMPLES +To dump core whenever a problem occurs: +.Pp +.Bd -literal -offset indent +ln -s 'A' /etc/malloc.conf +.Ed +.Pp +To specify in the source that a program does no return value checking +on calls to these functions: +.Bd -literal -offset indent +_malloc_options = "X"; +.Ed .Sh SEE ALSO -.Xr brk 2 , +.Xr limits 1 , +.Xr madvise 2 , +.Xr mmap 2 , +.Xr sbrk 2 , .Xr alloca 3 , +.Xr atexit 3 , .Xr getpagesize 3 , -.Xr memory 3 -.\"XXX" .Pa /usr/share/doc/papers/malloc.ascii.gz +.Xr memory 3 , +.Xr posix_memalign 3 .Sh STANDARDS The .Fn malloc , @@ -487,22 +435,4 @@ The and .Fn free functions conform to -.St -ansiC . -.Sh HISTORY -The present allocation implementation started out as a filesystem for a -drum attached to a 20bit binary challenged computer which was built -with discrete germanium transistors. -It has since graduated to handle primary storage rather than secondary. -It first appeared in its new shape and ability in -.Fx 2.2 , and then in -.Nx 1.5 . -.Sh BUGS -The messages printed in case of problems provide no detail about the -actual values. -.Pp -It can be argued that returning a null pointer when asked to -allocate zero bytes is a silly response to a silly question. -.Pp -This implementation was authored by Poul-Henning Kamp. -Please report any problems to him at -.Aq phk@FreeBSD.org . +.St -isoC .