the code in __libc_start_main is now responsible for parsing auxv,
rather than duplicating the parsing all over the place. this should
shave off a few cycles and some code size. __init_libc is left as an
external-linkage function despite the fact that it could be static, to
prevent it from being inlined and permanently wasting stack space when
main is called.
a few other minor changes are included, like eliminating per-thread
ssp canaries (they were likely broken when combined with certain
dlopen usages, and completely unnecessary) and some other unnecessary
checks. since this code gets linked into every program, it should be
as small and simple as possible.
at initial program load, all libraries must be loaded before the
thread pointer can be setup, since the TP-relative addresses of all
initial TLS objects must be constant.
this is needed to ensure async-cancel-safety, i.e. to make it safe to
access TLS objects when async cancellation is enabled. otherwise, if
cancellation were acter upon after the atomic fetch/add but before the
thread saved the obtained memory, another access to the same TLS in
the cancellation handler could end up performing the atomic fetch/add
again, consuming more memory than is actually available and
overflowing into other objects on the heap.
symbol value of 0 is not "undefined" for TLS; it's the address of the
first symbol in the TLS segment. however, non-definition TLS
references also have values of 0, so check the section.
hopefully the new logic is more clear, too.
compute offsets from the thread pointer statically when loading the
library, rather than repeating the logic on each thread creation. not
only is the latter less efficient at runtime; it also fails to provide
solid guarantees that the offsets will remain the same when the
initial alignment of memory is different. the new alignment handling
is both more rigorous and simpler.
the old code was also clobbering TLS bss with random image data in
some cases due to using tls_size (size of TLS segment) instead of
tls_len (length of the TLS data image).
some libraries call dlopen from their constructors, resulting in
recursive calls to dlopen. previously, this resulted in deadlock. I'm
now unlocking the dlopen lock before running constructors (this is
especially important since the lock also blocked pthread_create and
was being held while application code runs!) and using a separate
recursive mutex protecting the ctor/dtor state instead.
in order to prevent the same ctor from being called more than once, a
module is considered "constructed" just before the ctor runs.
also, switch from using atexit to register each dtor to using a single
atexit call to register the dynamic linker's dtor processing as just
one handler. this is necessary because atexit performs allocation and
may fail, but the library has already been loaded and cannot be
backed-out at the time dtor registration is performed. this change
also ensures that all dtors run after all atexit functions, rather
than in mixed order.
unlike other implementations, this one reserves memory for new TLS in
all pre-existing threads at dlopen-time, and dlopen will fail with no
resources consumed and no new libraries loaded if memory is not
available. memory is not immediately distributed to running threads;
that would be too complex and too costly. instead, assurances are made
that threads needing the new TLS can obtain it in an async-signal-safe
way from a buffer belonging to the dynamic linker/new module (via
atomic fetch-and-add based allocator).
I've re-appropriated the lock that was previously used for __synccall
(synchronizing set*id() syscalls between threads) as a general
pthread_create lock. it's a "backwards" rwlock where the "read"
operation is safe atomic modification of the live thread count, which
multiple threads can perform at the same time, and the "write"
operation is making sure the count does not increase during an
operation that depends on it remaining bounded (__synccall or dlopen).
in static-linked programs that don't use __synccall, this lock is a
no-op and has no cost.
currently, only i386 is tested. x86_64 and arm should probably work.
the necessary relocation types for mips and microblaze have not been
added because I don't understand how they're supposed to work, and I'm
not even sure if it's defined yet on microblaze. I may be able to
reverse engineer the requirements out of gcc/binutils output.
this was an optimization to save/recover a minimal amount of extra
memory for use by malloc, that's becoming increasingly costly to keep
around. freeing this data:
1. breaks debugging with gdb (it can't find library symbols)
2. breaks thread-local storage in shared libraries
it would be possible to disable freeing when TLS is used, but in
addition to the above breakages, tracking whether dlopen/dlsym is used
adds a cost to every symbol lookup, possibly making program startup
slower for large programs. combined with the complexity, it's not
worth it. we already save/recover plenty of memory in the dynamic
linker with reclaim_gaps.
this code will not work yet because the necessary relocations are not
supported, and cannot be supported without some internal changes to
how relocation processing works (coming soon).
the design for TLS in dynamic-linked programs is mostly complete too,
but I have not yet implemented it. cost is nonzero but still low for
programs which do not use TLS and/or do not use threads (a few hundred
bytes of new code, plus dependency on memcpy). i believe it can be
made smaller at some point by merging __init_tls and __init_security
into __libc_start_main and avoiding duplicate auxv-parsing code.
at the same time, I've also slightly changed the logic pthread_create
uses to allocate guard pages to ensure that guard pages are not
counted towards commit charge.
to deal with the fact that the public headers may be used with pre-c99
compilers, __restrict is used in place of restrict, and defined
appropriately for any supported compiler. we also avoid the form
[restrict] since older versions of gcc rejected it due to a bug in the
original c99 standard, and instead use the form *restrict.
based on patches submitted by boris brezillon. this commit also fixes
the issue whereby the main application and libc don't have the address
ranges of their mappings stored, which was theoretically a problem for
RTLD_NEXT support in dlsym; it didn't actually matter because libc
never calls dlsym, and it seemed to be doing the right thing (by
chance) for symbols in the main program as well.
wrong hash was being passed; just a copy/paste error. did not affect
lookups in the global namespace; this is probably why it was not
caught in testing.
previously, this usage could lead to a crash if the thread pointer was
still uninitialized, and otherwise would just cause the canary to be
zero (less secure).
before, only the first library that failed to load or symbol that
failed to resolve was reported, and then the dynamic linker
immediately exited. when attempting to fix a library compatibility
issue, this is about the worst possible behavior. now we print all
errors as they occur and exit at the very end if errors were
encountered.
not heavily tested, but the basics are working. the basic concept is
that the dynamic linker entry point code invokes a pure-PIC (no global
accesses) C function in reloc.h to perform the early GOT relocations
needed to make the dynamic linker itself functional, then invokes
__dynlink like on other archs. since mips uses some ugly arch-specific
hacks to optimize relocating the GOT (rather than just using the
normal DT_REL[A] tables like on other archs), the dynamic linker has
been modified slightly to support calling arch-specific relocation
code in reloc.h.
most of the actual mips-specific behavior was developed by reading the
output of readelf on libc.so and simple executable files. i could not
find good reference information on which relocation types need to be
supported or their semantics, so it's possible that some legitimate
usage cases will not work yet.
changing the string printed for the dso name is not a regression; the
old code was simply using the wrong dso name (head rather than the dso
currently being relocated). this will be fixed in a later commit.
if libc.a is compiled PIC for use in static PIE code, this should not
cause the dynamic linker (which still does not support static-linked
main program) to be built into libc.a.
most importantly, the name for such libs was being set from an
uninitialized buffer. also, shortname always had an initial '/'
character, making it useless for looking up already-loaded libraries
by name, and thus causing repeated searches through the library path.
major changes now:
- shortname is the base name for library lookups with no explicit
pathname. it's initially clear for libraries loaded with an explicit
pathname (and for the main program), but will be set if the same
library (detected via inodes match) is later found by a search.
- exact name match is never used to identify libraries loaded with an
explicit pathname. in this case, there's no explicit search, so we
can just stat the file and check for inode match.
previously this was being handled the same as a library-specific,
dependency-order lookup on the next library in the global chain, which
is likely to be utterly meaningless. instead the lookup needs to be in
the global namespace, but omitting the initial portion of the global
library chain up through the calling library.
the error will propagate up and be printed to the user at program
start time; at runtime, dlopen will just fail and leave a message for
dlerror.
previously, if mprotect failed, subsequent attempts to perform
relocations would crash the program. this was resulting in an
increasing number of false bug reports on grsec systems where rwx
permission is not possible in cases where users were wrongly
attempting to use non-PIC code in shared libraries. supporting that
usage is in theory possible, but the x86_64 toolchain does not even
support textrels, and the cost of keeping around the necessary
information to handle textrels without rwx permissions is
disproportionate to the benefit (which is essentially just supporting
broken library setups on grsec machines).
also, i unified the error-out code in map_library now that there are 3
places from which munmap might have to be called.
pthread structure has been adjusted to match the glibc/GCC abi for
where the canary is stored on i386 and x86_64. it will need variants
for other archs to provide the added security of the canary's entropy,
but even without that it still works as well as the old "minimal" ssp
support. eventually such changes will be made anyway, since they are
also needed for GCC/C11 thread-local storage support (not yet
implemented).
care is taken not to attempt initializing the thread pointer unless
the program actually uses SSP (by reference to __stack_chk_fail).
the code is written to pre-init the thread pointer in static linked
programs that pull in __stack_chk_fail or dynamic-linked programs that
lookup the symbol. no explicit canary is set; the canary will be
whatever happens to be in the thread structure at the offset gcc
hard-coded. this can be improved later.
note that dlerror is specified to be non-thread-safe, so no locking is
performed on the error flag or message aside from the rwlock already
held by dlopen or dlsym. if 2 invocations of dlsym are generating
errors at the same time, they could clobber each other's results, but
the resulting string, albeit corrupt, will still be null-terminated.
any use of dlerror in such a situation could not be expected to give
meaningful results anyway.
the error status is required to be sticky after failure of dlopen or
dlsym until cleared by dlerror. applications and especially libraries
should never rely on this since it is not thread-safe and subject to
race conditions, but glib does anyway.
i'm not sure that it's "correct" for dlopen to block cancellation
when calling constructors for libraries it loads, but it sure seems
like the right thing. in any case, dlopen itself needs cancellation
blocked.
