From 7b01cb974f1093885c40bf4d0d3e78e27e531363 Mon Sep 17 00:00:00 2001
From: Alexandre Derumier <aderumier@odiso.com>
Date: Fri, 19 Jun 2015 12:56:58 +0200
Subject: [PATCH] configure: Add support for jemalloc

This adds "--enable-jemalloc" and "--disable-jemalloc" to allow linking
to jemalloc memory allocator.

We have already tcmalloc support,
but it seem to not working well with a lot of iothreads/disks.

The main problem is that tcmalloc use a shared thread cache of 16MB
by default.
With more threads, this cache is shared, and some bad garbage collections
can occur if the cache is too low.

It's possible to tcmalloc cache increase it with a env var:
TCMALLOC_MAX_TOTAL_THREAD_CACHE_BYTES=256MB

With default 16MB, performances are  really bad with more than 2 disks.
Increasing to 256MB, it's helping but still have problem with 16 disks/iothreads.

Jemalloc don't have performance problem with default configuration.

Here the benchmark results in iops of 1 qemu vm randread 4K iodepth=32,
with rbd block backend (librbd is doing a lot of memory allocation),
1 iothread by disk

glibc malloc
------------

1 disk      29052
2 disks     55878
4 disks     127899
8 disks     240566
15 disks    269976

jemalloc
--------

1 disk      41278
2 disks     75781
4 disks     195351
8 disks     294241
15 disks    298199

tcmalloc 2.2.1 default 16M cache
--------------------------------

1 disk   37911
2 disks  67698
4 disks  41076
8 disks  43312
15 disks 37569

tcmalloc : 256M cache
---------------------------

1 disk     33914
2 disks    58839
4 disks    148205
8 disks    213298
15 disks   218383

Signed-off-by: Alexandre Derumier <aderumier@odiso.com>
Message-Id: <1434711418-20429-1-git-send-email-aderumier@odiso.com>
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
---
 configure | 28 ++++++++++++++++++++++++++++
 1 file changed, 28 insertions(+)

diff --git a/configure b/configure
index 34e49b4c8c..5332482f1f 100755
--- a/configure
+++ b/configure
@@ -337,6 +337,7 @@ libssh2=""
 vhdx=""
 numa=""
 tcmalloc="no"
+jemalloc="no"
 
 # parse CC options first
 for opt do
@@ -1143,6 +1144,10 @@ for opt do
   ;;
   --enable-tcmalloc) tcmalloc="yes"
   ;;
+  --disable-jemalloc) jemalloc="no"
+  ;;
+  --enable-jemalloc) jemalloc="yes"
+  ;;
   *)
       echo "ERROR: unknown option $opt"
       echo "Try '$0 --help' for more information"
@@ -1367,6 +1372,7 @@ disabled with --disable-FEATURE, default is enabled if available:
   vhdx            support for the Microsoft VHDX image format
   numa            libnuma support
   tcmalloc        tcmalloc support
+  jemalloc        jemalloc support
 
 NOTE: The object files are built at the place where configure is launched
 EOF
@@ -3367,6 +3373,11 @@ EOF
   fi
 fi
 
+if test "$tcmalloc" = "yes" && test "$jemalloc" = "yes" ; then
+    echo "ERROR: tcmalloc && jemalloc can't be used at the same time"
+    exit 1
+fi
+
 ##########################################
 # tcmalloc probe
 
@@ -3383,6 +3394,22 @@ EOF
   fi
 fi
 
+##########################################
+# jemalloc probe
+
+if test "$jemalloc" = "yes" ; then
+  cat > $TMPC << EOF
+#include <stdlib.h>
+int main(void) { malloc(1); return 0; }
+EOF
+
+  if compile_prog "" "-ljemalloc" ; then
+    LIBS="-ljemalloc $LIBS"
+  else
+    feature_not_found "jemalloc" "install jemalloc devel"
+  fi
+fi
+
 ##########################################
 # signalfd probe
 signalfd="no"
@@ -4594,6 +4621,7 @@ echo "snappy support    $snappy"
 echo "bzip2 support     $bzip2"
 echo "NUMA host support $numa"
 echo "tcmalloc support  $tcmalloc"
+echo "jemalloc support  $jemalloc"
 
 if test "$sdl_too_old" = "yes"; then
 echo "-> Your SDL version is too old - please upgrade to have SDL support"