Investigate a slow memory leak in a Spring Boot service from a heap histogram
A Spring Boot service throws OutOfMemoryError roughly every 36 hours; a restart resets the clock. Heap usage climbs steadily and full GCs reclaim less and less. Two jmap -histo:live <pid> snapshots, taken 12 hours apart under identical traffic:
t0 t0 + 12h
num #instances #bytes class name num #instances #bytes class name
--------------------------------------- ----------------------------------------
1: 418,222 40.1M byte[] 1: 6,904,551 612.4M byte[]
2: 190,880 15.2M java.lang.String 2: 3,118,402 249.5M java.lang.String
3: 42,015 3.4M c.a.web.AuditEntry 3: 2,884,190 230.7M c.a.web.AuditEntry
4: 11,904 1.1M java.util.HashMap$Node 4: 2,901,663 92.9M java.util.HashMap$NodeAuditEntry is built once per HTTP request by an interceptor and pushed into a bean the team describes as "just a small in-memory buffer for the admin page".
Diagnose the cause.
The histogram shows AuditEntry — and the byte[]/String it retains — growing without bound in lockstep with request volume, held through HashMap$Node. The audit bean accumulates one entry per request and never evicts. A Spring singleton bean lives as long as the application context, so everything it collects stays reachable and the GC is correct not to free it. Confirm the retaining path in a heap dump's dominator tree, then bound the buffer with a fixed size or an eviction policy.
- ✗Treating a singleton bean's collection as short-lived, request-scoped state
- ✗Raising
-Xmxor swapping the collector to buy time instead of finding the retaining root - ✗Reading the largest histogram row (
byte[]) as the leak instead of what retains it
- →Why does a dominator tree, not the histogram, identify the retaining owner?
- →How would you bound the buffer without losing the audit records the admin page needs?
Analysis
Read the histogram as a delta, not as a snapshot.
| Class | t0 | t0 + 12h | Growth |
|---|---|---|---|
AuditEntry | 42 K | 2.88 M | ×69 |
HashMap$Node | 12 K | 2.90 M | ×244 |
byte[] | 418 K | 6.9 M | ×17 |
byte[] is the fattest row by bytes, and that is the trap: arrays do not leak on their own — something retains them. The telling signal is elsewhere: the AuditEntry count and the HashMap$Node count track each other almost one-to-one and grow in proportion to the number of requests served. One entry is added to a map per request, and nothing is ever removed.
Why the GC is powerless. -histo:live already forces a full collection — everything in the listing is reachable. The audit bean is an ordinary Spring singleton: it lives exactly as long as the ApplicationContext, i.e. the whole process. It is therefore effectively a GC root, and its map retains the AuditEntry objects, which in turn retain their Strings and byte[]s. The collector is right not to free them.
A "small in-memory buffer" with no size bound is an unbounded cache.
Confirmation.
jcmd <pid> GC.heap_dump /tmp/heap.hprofIn Eclipse MAT: Leak Suspects → dominator tree → Path to GC Roots (exclude weak/soft). The path runs through the singleton bean to its map — that is the owner. A histogram shows what occupies memory; a dominator tree shows who holds it. A leak needs the second answer.
The fix. Bound the retention: a fixed-size ring buffer, an LRU with a ceiling, TTL eviction — or move the audit records off the heap entirely (a log, a broker, a database) and have the admin page read them from there.
