MariaDB and jemalloc: 592 MB of RAM handed back on a live server

The server has 7.5 GB of RAM, four cores and a dozen sites on top. MariaDB was sitting at 1.1 GB resident, and roughly 1.5 GB had already gone to swap. Nothing was broken — there was simply nowhere left to grow.

Before buying more memory, I decided to test a claim that turns up on forums constantly but almost never with numbers behind it: that switching the memory allocator to jemalloc meaningfully reduces what MariaDB consumes.

So I measured it instead of taking it on trust. This article is the result — the method, real numbers from a live server, and an honest note at the end about what jemalloc does not do.

What an allocator actually does

When MariaDB needs memory for a sort buffer or a temporary table, it does not talk to the kernel. It talks to the allocator — the library sitting between the application and the operating system. On every Linux system the default is the one that ships with glibc.

The allocator has two jobs: find memory when asked, and give it back when it is no longer needed. Every allocator does the first one well. The difference is in the second.

glibc uses arenas — separate zones per thread, so threads do not contend on the same lock. That is a fast design and precisely why it was chosen. But when a thread frees its memory, the freed space stays in that thread’s arena, ready for the next request. As far as the kernel is concerned, that memory is still in use.

jemalloc uses the same arena idea but adds something glibc does not have: decay purge. At intervals it looks at which pages have not been touched and returns them to the kernel with madvise(MADV_DONTNEED). The memory stays reserved for the process, but it stops counting as resident.

Here is why this matters specifically for MariaDB: with 24 concurrent connections running sorts and temporary tables, hundreds of megabytes move in both directions every second. That churn is exactly what glibc holds on to.

How I measured it

The benchmark had to answer one question — how much memory the process holds, not how much it asked for. So the central metric is VmRSS from /proc/<pid>/status, sampled once a second.

The protocol was identical for both runs:

1. Restart MariaDB so the allocator starts from a clean state.
2. Warm the buffer pool and record the starting RSS.
3. Four rounds of load: 25 iterations each, 24 concurrent clients.
4. Sixty seconds of complete quiet afterwards.
5. Record the peak, the end value, and the difference.

Step four is the important one and it is what most benchmarks skip. The peak tells you how much was requested. The value after things go quiet tells you how much comes back — and that is where the two allocators part company.

The workload is a 300,000-row table and queries chosen to churn memory: sorts that do not fit in the buffer, GROUP BY with GROUP_CONCAT, DISTINCT over a text column, joins through the join buffer. Every query carries SQL_NO_CACHE, because the query cache is enabled on this server and would otherwise serve ready-made results.

For the load generator I used mysqlslap — it ships with MariaDB and needs nothing extra:

mysqlslap --concurrency=24 --iterations=25 \
          --create-schema=membench --delimiter=$'\n' \
          --query=/tmp/membench.sql --no-drop

And the memory sampling is literally one line in a background loop:

awk '/^VmRSS:/{print $2}' /proc/$(pgrep -x mariadbd)/status

I kept concurrency at 24 against max_connections = 50, leaving headroom for the live sites. The benchmark ran on a working server rather than an idle machine — that adds a little noise, but the numbers come from real conditions.

The results

The shape of the two lines says more than any table.

glibc climbs and stays there. The line does not come down once in 21 minutes. It reaches 1099 MB and that is where it ends — including after the sixty seconds of quiet. Freed memory never leaves the process.

jemalloc moves. The sawtooth between 500 and 630 MB is decay purge at work: between bursts of load it hands pages back to the kernel, and takes them again on the next burst. The moment the load stops it drops to 507 MB and stays there.

The numbers:

Metric	glibc	jemalloc	Delta
RSS at start	293.5 MB	300.4 MB	+2.4%
RSS at peak	1098.7 MB	712.6 MB	−35%
RSS after 60s idle	1098.7 MB	506.9 MB	−54%
Retained above start	805.3 MB	206.6 MB	−74%
mmap regions	280	251	−10%
Mean time per round	12.068 s	11.982 s	−0.7%

Why this is not the buffer pool

Here is the objection I raised against myself before believing the result. innodb_buffer_pool_size on this server is 1 GB. The pool is reserved up front but its pages are touched gradually — so the growth from 293 MB to 1099 MB could simply be a warming pool rather than an allocator.

Two checks close the question.

First — what MariaDB thinks it allocated. The Memory_used status variable reports how much memory was requested through its own accounting:

mysql -e "SHOW GLOBAL STATUS LIKE 'Memory_used'"

Memory_used    1280050288      # glibc
Memory_used    1281058720      # jemalloc

That is a difference of 0.08 percent. MariaDB asked for practically the same amount of memory in both runs — yet the resident footprint differs by 592 MB.

Second — how much of the pool is genuinely occupied. If the growth were the buffer pool, the pool would be full:

mysql -e "SHOW GLOBAL STATUS WHERE Variable_name LIKE 'Innodb_buffer_pool_pages_%'"

Innodb_buffer_pool_pages_data    12429
Innodb_buffer_pool_pages_free    52467
Innodb_buffer_pool_pages_total   64896

12,429 pages used out of 64,896 — around 194 MB of real data, with more than fifty thousand pages still free. The buffer pool cannot account for 1099 MB.

That leaves one explanation, and it is the allocator.

About the speed — honestly

Now for something that contradicts a good deal of what is written on this subject.

My first reading showed +8.9% more queries per second for jemalloc. A pleasing number that would have looked excellent in a headline. Except it was calculated from the global Questions counter, and that counter also counts traffic from the live sites on the same server. In other words it measures things that have nothing to do with the benchmark.

The correct figure is the one mysqlslap reports for its own queries: 12.068 versus 11.982 seconds on average per round. That is a difference of 0.7 percent — below the noise.

jemalloc here is a memory win, not a speed win. If somebody promises you a faster database purely from changing the allocator, ask how they measured it.

Which makes sense once you consider what decay purge does: it hands memory back to the kernel, and that is extra work, not saved work. The gain is in the footprint.

How to enable it

Three steps on AlmaLinux 9, or any other RHEL 9 system. jemalloc lives in EPEL:

dnf install -y jemalloc

Then the configuration. The important part is not to edit the service file itself — it gets overwritten on the next MariaDB update. The right place is a drop-in under /etc:

mkdir -p /etc/systemd/system/mariadb.service.d
cat > /etc/systemd/system/mariadb.service.d/jemalloc.conf <<'EOF'
[Service]
Environment="LD_PRELOAD=/usr/lib64/libjemalloc.so.2"
EOF

systemctl daemon-reload
systemctl restart mariadb

The restart is brief but it is genuine downtime — schedule it outside peak hours.

The check that actually matters

Here is the detail that saves you hours: if the path to the library is wrong, LD_PRELOAD fails silently. The dynamic loader prints a warning and starts the process anyway — on the old allocator. Which means the existence of the drop-in file proves nothing at all.

MariaDB answers the question directly:

mysql -e "SHOW GLOBAL VARIABLES LIKE 'version_malloc_library'"

version_malloc_library    jemalloc 5.2.1-0-gea6b3e973b477b8061e0076bb257dbd7f3faa756

On the default allocator the same query returns system. If you see system after the restart, something did not take effect.

For extra certainty, confirm the library is genuinely mapped into the process:

grep -c jemalloc /proc/$(pgrep -x mariadbd)/maps

5

Any number above zero means the library is in place.

When it helps and when it does not

The gain is not uniform. What was measured here, together with this server’s configuration, shows fairly clearly when it is worth doing:

It helps noticeably when:

• You have many concurrent connections — every thread brings its own arena.
• tmp_table_size and max_heap_table_size are large. On this server they are 128 MB each against max_connections = 50, and that is precisely the memory glibc was holding.
• Queries churn sorts, temporary tables and GROUP BY rather than simple index reads.
• The machine is memory-constrained and swap has started to make itself heard.

You will barely notice it when:

• The workload is mostly simple primary-key lookups.
• Connections are few and long-lived.
• Almost all the memory goes to the buffer pool. That is allocated once and stays occupied — there is nothing there to give back.
• You are after speed. See the previous section.

How to roll it back

The change is reversible in half a minute and leaves no trace:

rm /etc/systemd/system/mariadb.service.d/jemalloc.conf
systemctl daemon-reload
systemctl restart mariadb

Nothing in MariaDB’s own configuration was touched — the whole experiment lives in one three-line file.

In short

On a live MariaDB 10.11 with 24 concurrent clients and a workload of sorts and temporary tables, jemalloc holds 507 MB resident where glibc holds 1099 MB — with practically identical consumption according to MariaDB’s own accounting.

The difference does not come from the buffer pool. It comes from glibc not returning freed memory to the kernel, while jemalloc does.

Speed stays the same. Anyone claiming otherwise should show how they measured it.

On a server with memory to spare this is simply a tidy optimisation. On a server already in swap it is the difference between cramped and comfortable.

One honest note on method: I ran each allocator once, not as a series. I chose not to load a working server for another twenty minutes, because the proof is not in a single number anyway — it is in Memory_used being identical while the shape of the two lines is not. If you run this on your own server, repeat it a few times and see whether you get the same picture.

---

Photo: Macro shot photo of a computer RAM by Liam Briese on Unsplash.

If your server is in swap and you would rather not run the measurements yourself, get in touch — I do both server administration and database optimisation.