Tweaking Python Memory ManagementΒΆ

date:2015/3/6

Linux glibc often hesitates to return memory to the OS. The mechanism is well explained in the issue tracker discussions. As such, CPython 3.3 changed to use mmap() to allocate arenas: https://hg.python.org/cpython/rev/f8a697bc3ca8/ and https://hg.python.org/cpython/rev/e7aa72e6aad4/. The enhancement was later back-ported to 2.7: https://hg.python.org/cpython/rev/4e43e5b3f7fc/.

Alternatively, to resolve the wild glibc, one can use another memory allocator. Guys at ESN tried tcmalloc and found it is useful. For debugging memory issues tcmalloc looks like a good tool, as it offers quite nice heap analyzer.