10c5b02320
memory used by the kernel at run time, and just like kASan and kCSan, it is an excellent feature. It has already detected 38 uninitialized variables in the kernel during my testing, which I have since discreetly fixed. We use two shadows: - "shad", to track uninitialized memory with a bit granularity (1:1). Each bit set to 1 in the shad corresponds to one uninitialized bit of real kernel memory. - "orig", to track the origin of the memory with a 4-byte granularity (1:1). Each uint32_t cell in the orig indicates the origin of the associated uint32_t of real kernel memory. The memory consumption of these shadows is consequent, so at least 4GB of RAM is recommended to run kMSan. The compiler inserts calls to specific __msan_* functions on each memory access, to manage both the shad and the orig and detect uninitialized memory accesses that change the execution flow (like an "if" on an uninitialized variable). We mark as uninit several types of memory buffers (stack, pools, kmem, malloc, uvm_km), and check each buffer passed to copyout, copyoutstr, bwrite, if_transmit_lock and DMA operations, to detect uninitialized memory that leaves the system. This allows us to detect kernel info leaks in a way that is more efficient and also more user-friendly than KLEAK. Contrary to kASan, kMSan requires comprehensive coverage, ie we cannot tolerate having one non-instrumented function, because this could cause false positives. kMSan cannot instrument ASM functions, so I converted most of them to __asm__ inlines, which kMSan is able to instrument. Those that remain receive special treatment. Contrary to kASan again, kMSan uses a TLS, so we must context-switch this TLS during interrupts. We use different contexts depending on the interrupt level. The orig tracks precisely the origin of a buffer. We use a special encoding for the orig values, and pack together in each uint32_t cell of the orig: - a code designating the type of memory (Stack, Pool, etc), and - a compressed pointer, which points either (1) to a string containing the name of the variable associated with the cell, or (2) to an area in the kernel .text section which we resolve to a symbol name + offset. This encoding allows us not to consume extra memory for associating information with each cell, and produces a precise output, that can tell for example the name of an uninitialized variable on the stack, the function in which it was pushed on the stack, and the function where we accessed this uninitialized variable. kMSan is available with LLVM, but not with GCC. The code is organized in a way that is similar to kASan and kCSan, so it means that other architectures than amd64 can be supported.
236 lines
7.3 KiB
C
236 lines
7.3 KiB
C
/* $NetBSD: kern_malloc.c,v 1.158 2019/11/14 16:23:52 maxv Exp $ */
|
|
|
|
/*
|
|
* Copyright (c) 1987, 1991, 1993
|
|
* The Regents of the University of California. All rights reserved.
|
|
*
|
|
* Redistribution and use in source and binary forms, with or without
|
|
* modification, are permitted provided that the following conditions
|
|
* are met:
|
|
* 1. Redistributions of source code must retain the above copyright
|
|
* notice, this list of conditions and the following disclaimer.
|
|
* 2. Redistributions in binary form must reproduce the above copyright
|
|
* notice, this list of conditions and the following disclaimer in the
|
|
* documentation and/or other materials provided with the distribution.
|
|
* 3. Neither the name of the University nor the names of its contributors
|
|
* may be used to endorse or promote products derived from this software
|
|
* without specific prior written permission.
|
|
*
|
|
* THIS SOFTWARE IS PROVIDED BY THE REGENTS AND CONTRIBUTORS ``AS IS'' AND
|
|
* ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE
|
|
* IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE
|
|
* ARE DISCLAIMED. IN NO EVENT SHALL THE REGENTS OR CONTRIBUTORS BE LIABLE
|
|
* FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL
|
|
* DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS
|
|
* OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION)
|
|
* HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT
|
|
* LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY
|
|
* OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF
|
|
* SUCH DAMAGE.
|
|
*
|
|
* @(#)kern_malloc.c 8.4 (Berkeley) 5/20/95
|
|
*/
|
|
|
|
/*
|
|
* Copyright (c) 1996 Christopher G. Demetriou. All rights reserved.
|
|
*
|
|
* Redistribution and use in source and binary forms, with or without
|
|
* modification, are permitted provided that the following conditions
|
|
* are met:
|
|
* 1. Redistributions of source code must retain the above copyright
|
|
* notice, this list of conditions and the following disclaimer.
|
|
* 2. Redistributions in binary form must reproduce the above copyright
|
|
* notice, this list of conditions and the following disclaimer in the
|
|
* documentation and/or other materials provided with the distribution.
|
|
* 3. All advertising materials mentioning features or use of this software
|
|
* must display the following acknowledgement:
|
|
* This product includes software developed by the University of
|
|
* California, Berkeley and its contributors.
|
|
* 4. Neither the name of the University nor the names of its contributors
|
|
* may be used to endorse or promote products derived from this software
|
|
* without specific prior written permission.
|
|
*
|
|
* THIS SOFTWARE IS PROVIDED BY THE REGENTS AND CONTRIBUTORS ``AS IS'' AND
|
|
* ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE
|
|
* IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE
|
|
* ARE DISCLAIMED. IN NO EVENT SHALL THE REGENTS OR CONTRIBUTORS BE LIABLE
|
|
* FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL
|
|
* DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS
|
|
* OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION)
|
|
* HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT
|
|
* LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY
|
|
* OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF
|
|
* SUCH DAMAGE.
|
|
*
|
|
* @(#)kern_malloc.c 8.4 (Berkeley) 5/20/95
|
|
*/
|
|
|
|
/*
|
|
* Wrapper interface for obsolete malloc(9).
|
|
*/
|
|
|
|
#include <sys/cdefs.h>
|
|
__KERNEL_RCSID(0, "$NetBSD: kern_malloc.c,v 1.158 2019/11/14 16:23:52 maxv Exp $");
|
|
|
|
#include <sys/param.h>
|
|
#include <sys/malloc.h>
|
|
#include <sys/kmem.h>
|
|
#include <sys/asan.h>
|
|
#include <sys/msan.h>
|
|
|
|
/*
|
|
* Built-in malloc types. Note: ought to be removed.
|
|
*/
|
|
MALLOC_DEFINE(M_DEVBUF, "devbuf", "device driver memory");
|
|
MALLOC_DEFINE(M_DMAMAP, "DMA map", "bus_dma(9) structures");
|
|
MALLOC_DEFINE(M_FREE, "free", "should be on free list");
|
|
MALLOC_DEFINE(M_TEMP, "temp", "misc. temporary data buffers");
|
|
MALLOC_DEFINE(M_RTABLE, "routetbl", "routing tables");
|
|
MALLOC_DEFINE(M_FTABLE, "fragtbl", "fragment reassembly header");
|
|
MALLOC_DEFINE(M_UFSMNT, "UFS mount", "UFS mount structure");
|
|
MALLOC_DEFINE(M_NETADDR, "Export Host", "Export host address structure");
|
|
MALLOC_DEFINE(M_MRTABLE, "mrt", "multicast routing tables");
|
|
|
|
/*
|
|
* Header contains total size, including the header itself.
|
|
*/
|
|
struct malloc_header {
|
|
size_t mh_size;
|
|
#ifdef KASAN
|
|
size_t mh_rqsz;
|
|
#endif
|
|
} __aligned(ALIGNBYTES + 1);
|
|
|
|
void *
|
|
kern_malloc(unsigned long reqsize, int flags)
|
|
{
|
|
const int kmflags = (flags & M_NOWAIT) ? KM_NOSLEEP : KM_SLEEP;
|
|
#ifdef KASAN
|
|
const size_t origsize = reqsize;
|
|
#endif
|
|
size_t size = reqsize;
|
|
size_t allocsize, hdroffset;
|
|
struct malloc_header *mh;
|
|
void *p;
|
|
|
|
kasan_add_redzone(&size);
|
|
|
|
if (size >= PAGE_SIZE) {
|
|
if (size > (ULONG_MAX-PAGE_SIZE))
|
|
allocsize = ULONG_MAX; /* this will fail later */
|
|
else
|
|
allocsize = PAGE_SIZE + size; /* for page alignment */
|
|
hdroffset = PAGE_SIZE - sizeof(struct malloc_header);
|
|
} else {
|
|
allocsize = sizeof(struct malloc_header) + size;
|
|
hdroffset = 0;
|
|
}
|
|
|
|
p = kmem_intr_alloc(allocsize, kmflags);
|
|
if (p == NULL)
|
|
return NULL;
|
|
|
|
kmsan_mark(p, allocsize, KMSAN_STATE_UNINIT);
|
|
kmsan_orig(p, allocsize, KMSAN_TYPE_MALLOC, __RET_ADDR);
|
|
|
|
if ((flags & M_ZERO) != 0) {
|
|
memset(p, 0, allocsize);
|
|
}
|
|
mh = (void *)((char *)p + hdroffset);
|
|
mh->mh_size = allocsize - hdroffset;
|
|
#ifdef KASAN
|
|
mh->mh_rqsz = origsize;
|
|
#endif
|
|
mh++;
|
|
|
|
kasan_mark(mh, origsize, size, KASAN_MALLOC_REDZONE);
|
|
|
|
return mh;
|
|
}
|
|
|
|
void
|
|
kern_free(void *addr)
|
|
{
|
|
struct malloc_header *mh;
|
|
|
|
mh = addr;
|
|
mh--;
|
|
|
|
kasan_mark(addr, mh->mh_size - sizeof(struct malloc_header),
|
|
mh->mh_size - sizeof(struct malloc_header), KASAN_MALLOC_REDZONE);
|
|
|
|
if (mh->mh_size >= PAGE_SIZE + sizeof(struct malloc_header)) {
|
|
kmsan_mark((char *)addr - PAGE_SIZE,
|
|
mh->mh_size + PAGE_SIZE - sizeof(struct malloc_header),
|
|
KMSAN_STATE_INITED);
|
|
kmem_intr_free((char *)addr - PAGE_SIZE,
|
|
mh->mh_size + PAGE_SIZE - sizeof(struct malloc_header));
|
|
} else {
|
|
kmsan_mark(mh, mh->mh_size, KMSAN_STATE_INITED);
|
|
kmem_intr_free(mh, mh->mh_size);
|
|
}
|
|
}
|
|
|
|
void *
|
|
kern_realloc(void *curaddr, unsigned long newsize, int flags)
|
|
{
|
|
struct malloc_header *mh;
|
|
unsigned long cursize;
|
|
void *newaddr;
|
|
|
|
/*
|
|
* realloc() with a NULL pointer is the same as malloc().
|
|
*/
|
|
if (curaddr == NULL)
|
|
return malloc(newsize, ksp, flags);
|
|
|
|
/*
|
|
* realloc() with zero size is the same as free().
|
|
*/
|
|
if (newsize == 0) {
|
|
free(curaddr, ksp);
|
|
return NULL;
|
|
}
|
|
|
|
if ((flags & M_NOWAIT) == 0) {
|
|
ASSERT_SLEEPABLE();
|
|
}
|
|
|
|
mh = curaddr;
|
|
mh--;
|
|
|
|
#ifdef KASAN
|
|
cursize = mh->mh_rqsz;
|
|
#else
|
|
cursize = mh->mh_size - sizeof(struct malloc_header);
|
|
#endif
|
|
|
|
/*
|
|
* If we already actually have as much as they want, we're done.
|
|
*/
|
|
if (newsize <= cursize)
|
|
return curaddr;
|
|
|
|
/*
|
|
* Can't satisfy the allocation with the existing block.
|
|
* Allocate a new one and copy the data.
|
|
*/
|
|
newaddr = malloc(newsize, ksp, flags);
|
|
if (__predict_false(newaddr == NULL)) {
|
|
/*
|
|
* malloc() failed, because flags included M_NOWAIT.
|
|
* Return NULL to indicate that failure. The old
|
|
* pointer is still valid.
|
|
*/
|
|
return NULL;
|
|
}
|
|
memcpy(newaddr, curaddr, cursize);
|
|
|
|
/*
|
|
* We were successful: free the old allocation and return
|
|
* the new one.
|
|
*/
|
|
free(curaddr, ksp);
|
|
return newaddr;
|
|
}
|