How Java off-heap memory works
Trace Java memory beyond the heap through direct buffers, native allocation, pooling, memory mapping, limits, and diagnosis.
The whole story in 8 lines
See how direct allocation, Cleaner reclamation, Unsafe, Netty arenas, memory mapping, limits, and native tracking fit together.
- Process RSS includes the Java heap and several native areas, so a flat heap can coexist with rising process memory.
- allocateDirect creates a small heap wrapper and reserves a native segment tracked against the direct-memory limit.
- Direct-buffer storage is reclaimed only after its wrapper becomes unreachable, reference processing runs, and the Cleaner releases it.
- Unsafe allocations bypass GC ownership, so the caller must pair every native allocation with exactly one release.
- Netty arenas, chunks, and pages amortize allocation cost but may retain pooled capacity after request load drops.
- A mapped buffer reserves virtual address space backed by a file, and page faults bring accessed pages into RAM on demand.
- The direct-memory limit caps tracked direct buffers, while NMT categorizes only native allocations known to the JVM.
- Comparing process maps with NMT helps isolate resident native regions that JVM accounting does not explain.
Setup
When your Java program runs, the JVM claims a chunk of memory from the operating system. Most developers only think about the heap, the region managed by the garbage collector. But there is a whole world of memory outside the heap that the JVM and your code can use directly. This explainer teaches you how that off-heap world works.
The heap is the GC-managed region where most Java objects live. Off-heap (or native memory) is everything the JVM process uses outside the heap: thread stacks, JIT-compiled code, class metadata. And memory you allocate explicitly via NIO buffers or Unsafe.
A DirectByteBuffer is the main API for allocating off-heap memory from Java code. It creates a small wrapper object on the heap that points to a large native memory segment obtained via malloc. A Cleaner is the mechanism that frees that native memory when the GC collects the wrapper.
These four ideas form one ownership chain. A DirectByteBuffer remains a heap object, but its payload lives off-heap, so the Cleaner must eventually release that native segment. The roadmap now shows how later stages build outward from this chain.
Over the next eight stages, we will trace allocation, cleanup, pooling, memory-mapped files, and native diagnostics. Each new term will arrive when its mechanism is visible. Let us start with the JVM memory map and see where the heap fits inside the whole process.
JVM Memory Map
Here is the JVM process as the operating system sees it: one block of resident memory. Everything your Java program uses, whether heap objects, compiled code, or raw native buffers, lives inside this single process address space.
The Java Heap is the region most developers think about. It is where new Object() goes, where your application data lives, and where the garbage collector operates. But notice how much space remains outside it.
Metaspace holds class definitions, method metadata, and constant pools. Before Java 8 this was called PermGen and lived inside the heap. Now it uses native memory and can grow until the OS runs out.
Thread stacks consume native memory too. Each thread gets a fixed-size stack (usually 512 KB to 1 MB). An application with 500 threads can easily consume 500 MB of off-heap memory just for stacks.
The JIT compiler generates optimized machine code and stores it in the Code Cache. The GC itself needs bookkeeping structures like remembered sets and card tables. Both are off-heap.
Finally, the "Other" or "Internal" zone. This is where DirectByteBuffers, Unsafe allocations, and JNI native allocations live. This is the region we will focus on for the rest of this explainer.
The key insight: the heap is just one slice of a much larger memory footprint. When your container gets OOMKilled but the heap looks fine, the answer is almost always hiding in these off-heap regions. Next, let us see how ByteBuffer.allocateDirect taps into that native zone.
allocateDirect()
The most common off-heap API is ByteBuffer.allocateDirect(). Application code supplies a size in bytes, and the request flows through JVM accounting before producing a usable native buffer.
The first thing the JVM does is check the direct memory reservation. The class Bits.reserveMemory tracks how much direct memory is currently in use. If the new allocation would exceed MaxDirectMemorySize, it triggers a System.gc() and retries. If it still exceeds the limit, an OutOfMemoryError is thrown.
Once the reservation succeeds, the JVM calls Unsafe.allocateMemory(size) internally. This is a thin JNI wrapper around the C malloc function. The operating system returns a pointer to a block of virtual memory pages.
The stage has reached the decision that determines its next state. Where does the large payload of a DirectByteBuffer live?
The constructor also registers a Cleaner. This is a callback (a Deallocator) that will call Unsafe.freeMemory when the GC collects the DirectByteBuffer wrapper. Without this Cleaner, the native memory would leak forever. We will explore this lifecycle in detail in the next stage.
The returned ByteBuffer is ready. Your code reads and writes through the heap wrapper, but every get() and put() operates directly on native memory, no copying through the heap. This is why NIO channels prefer direct buffers for I/O: the kernel can DMA straight from that native segment without an intermediate copy.
Cleaner Lifecycle
We just saw how allocateDirect registers a Cleaner. Now let us trace what happens when that DirectByteBuffer is no longer needed. The deallocation path is asynchronous and depends entirely on the garbage collector running at the right time.
Your application drops all references to the DirectByteBuffer. The object is now unreachable, but nothing has happened yet. The native memory is still allocated. This is the critical window: the heap wrapper is garbage, but the off-heap segment persists.
A GC cycle runs. The collector discovers the DirectByteBuffer is unreachable. It does not free it immediately. Instead, it enqueues the associated PhantomReference into a ReferenceQueue. Think of this as the GC leaving a note: "this object is dead, someone should clean up after it."
The Reference Handler thread (a JVM-internal daemon) picks up the enqueued reference and passes it to the Cleaner. The Cleaner invokes the Deallocator, which is the callback that was registered during allocation.
The Deallocator calls Unsafe.freeMemory(address), which calls the C free() function. The native memory is finally returned to the OS. The Bits.reserveMemory counter is decremented, making room for new direct buffers.
Notice how the native memory stays allocated longer when GC is delayed. In a real system, if your application allocates direct buffers faster than GC collects. Them, native memory pressure builds up until an OOM or a forced System.gc() saves you.
This is the fundamental tension of direct buffers: allocation is explicit (you call allocateDirect), but deallocation is implicit (it depends on GC). If GC does not run, native memory leaks. This is why System.gc() calls inside Bits.reserveMemory exist.
Unsafe Path
★ If you remember one thing · Managed direct buffers defer cleanup to the GC, while raw Unsafe allocation makes the caller release the address explicitly.
The Cleaner lifecycle we just learned about works, but it has a fundamental cost: deallocation timing is at the mercy of the GC. Libraries that need predictable, high-throughput memory management take a different route entirely: sun.misc.Unsafe.
On the left is the managed path you already know. ByteBuffer.allocateDirect checks MaxDirectMemorySize via Bits.reserveMemory, allocates native memory, registers a Cleaner, and trusts GC for cleanup. On the right is the raw Unsafe path.
Unsafe.allocateMemory(size) is a direct JNI call to malloc. It returns a raw native address as a long. No limit checking, no Bits reservation, no Cleaner registration. The JVM has no idea this memory exists. NMT tracks it in the "Internal" category only in its own bookkeeping, not against MaxDirectMemorySize.
Both paths can reserve native memory, but their cleanup obligations are different. Which path requires the caller to invoke freeMemory directly?
Deallocation is Unsafe.freeMemory(address). You must call this yourself. Forget it and the memory leaks permanently. Call it twice and you get a double-free crash. This is C-style manual memory management inside a Java process.
Switch the Allocator control through every choice. Compare the downstream outcome while the earlier input and system boundary remain fixed.
The tradeoff is clear: the managed path is safe but couples deallocation to GC timing. The Unsafe path is fast and deterministic but shifts all responsibility to the programmer. Most production systems that handle high-throughput I/O (Kafka, Cassandra, Netty) choose the Unsafe path. Netty, in particular, builds an entire arena-based allocator on top of Unsafe. Let us see how that works next.
Netty Arenas
In the last stage we saw how Netty and similar libraries choose Unsafe for deterministic allocation. But Netty does not just call malloc and free repeatedly. It builds a sophisticated arena-based allocator on top of Unsafe that eliminates per-allocation system calls entirely.
The PooledByteBufAllocator creates a fixed number of Arenas at startup, typically two per CPU core. Each Arena pre-allocates large Chunks of native memory (16 MB each) via Unsafe.allocateMemory. These chunks are the raw material from which all buffers are carved.
Each application thread is bound to one Arena in round-robin fashion. But before touching the Arena at all, the thread checks its own PoolThreadCache. This is a thread-local free list of recently released buffers, organized by size class. A cache hit means zero contention and zero locking.
On a cache miss, the Arena locks and uses a buddy algorithm to find free pages in one of its Chunks. For a page-sized request (8 KB), it finds one free page. For larger requests, it merges adjacent pages. For small requests under 8 KB, it subdivides a page into fixed-size slots using a bitmap.
The allocated buffer is returned to the thread. The thread reads and writes directly to the native memory through the buffer handle. No heap allocation occurs beyond the thin wrapper object. No GC pressure is generated by the data itself.
When the thread releases the buffer, it does not go back to the OS. It returns to the PoolThreadCache for immediate reuse. Only when the cache exceeds its size limit does it return buffers to the Arena free list. Memory is recycled, never freed. This is why Netty handles millions of allocations per second.
The tradeoff: Netty trades memory overhead for speed and predictability. Your process RSS stays high even when traffic drops, because the Arena keeps its Chunks allocated. Monitoring Netty memory requires its own metrics (PooledByteBufAllocator.metric()) since NMT only sees the raw Unsafe allocations. Next, let us explore a completely different path: memory-mapped files.
mmap Files
We have seen three ways to get off-heap memory: DirectByteBuffer (managed), Unsafe (raw), and Netty arenas (pooled). There is a fourth path that is fundamentally different: memory-mapped files. Instead of allocating anonymous memory, you map a file from disk directly into the process address space.
FileChannel.map(MapMode.READ_ONLY, 0, size) calls the POSIX mmap system call. The OS kernel creates a virtual memory mapping: a range of addresses in your process that correspond to byte offsets in the file. At this point, no physical RAM has been consumed.
When your code reads from the MappedByteBuffer, the CPU translates the virtual address to a physical address via the page table. If the page is not in RAM yet, a page fault fires. The kernel handles the fault by reading the file page from disk into a physical RAM page, then resumes your code.
This is lazy loading at the OS level. Only the pages you actually touch consume physical memory. A 50 GB file can be mapped into a process with only 100 MB of RAM. The OS evicts cold pages under memory pressure and re-faults them if needed.
Here is the critical diagnostic blind spot: NMT does not track memory-mapped files. The mapping is a kernel-level operation, and the JVM has no visibility into it. To see mapped memory, you need OS tools like pmap or /proc/pid/smaps. Entries with a non-zero inode are file-backed mappings.
A mapped buffer's lifetime is coupled to reachability rather than a public unmap method. Sequential access tends to produce predictable page-fault locality, while random access scatters faults across the mapped file. Native-memory ownership still needs explicit operational monitoring.
Memory-mapped files are the backbone of databases (RocksDB, LMDB), search engines (Lucene), and message brokers (Kafka log segments). They trade explicit memory management for OS-level paging, which is efficient for read-heavy, sequential workloads but unpredictable under memory pressure. Now let us look at how the JVM tracks and limits all of these off-heap paths.
Limits & NMT
DirectByteBuffer, Unsafe, and mmap follow different accounting paths. The JVM's primary direct-buffer guard is MaxDirectMemorySize, which caps how much memory Bits.reserveMemory will allow for tracked direct allocations.
By default, MaxDirectMemorySize is approximately equal to the heap size. When your code calls allocateDirect and Bits.reserveMemory sees that the new allocation would breach this limit, it does something surprising: it calls System.gc(). This is a full-stop hint to the GC to run immediately.
After triggering GC, Bits.reserveMemory retries the reservation. If dead DirectByteBuffers were collected and their Cleaners freed native memory, the new allocation succeeds. If it still exceeds the limit, an OutOfMemoryError: Direct buffer memory is thrown. This is why disabling System.gc() with -XX:+DisableExplicitGC can cause mysterious direct buffer OOMs.
Native Memory Tracking is the JVM diagnostic that shows where native memory goes. Enable it with -XX:NativeMemoryTracking=summary. It breaks down usage into zones: Java Heap, Class (Metaspace), Thread, Code, GC, Internal, and Other.
DirectByteBuffers appear in the "Other" zone (Java 11+, previously Internal). Unsafe allocations show up in "Internal". But here is the key blind spot: NMT does not track memory-mapped files or allocations made by native JNI libraries. If pmap shows your process using 8 GB but NMT only accounts for 5 GB, the gap is likely mmap or native library allocations.
NMT summary mode adds little CPU overhead for most workloads. Its tracking metadata can add roughly 5-10% memory overhead. Detailed mode adds more overhead and tracks individual call sites, useful for debugging but not for production.
The diagnostic toolkit is now clear: NMT for JVM-tracked native memory, pmap or /proc/pid/smaps for the complete picture including mmap. And jcmd VM.native_memory to get live NMT reports. When your container gets OOMKilled, start with NMT to see what the JVM knows, then pmap to find what it does not. Let us look at pmap in detail next and see how to actually diagnose mmap and JNI-based OOMs.
Pmap & OOM
We just learned that NMT has blind spots: it cannot track mmap regions or memory allocated by native JNI libraries. When your container gets OOMKilled but NMT looks healthy, pmap is the tool that shows you what NMT cannot see.
pmap -x <pid> prints every memory region in the process address space. Each line shows an address range, its size in KB, resident pages (RSS), permissions, and a mapping name. File-backed regions show the file path. Anonymous regions show [anon] or [heap].
Start by identifying the NMT-tracked regions. The Java Heap is a large anonymous mapping, often the biggest single entry. Metaspace, thread stacks, code cache, and GC structures all appear as anonymous mappings that NMT accounts for. These are the blue bars in the visualization.
File-backed mappings come from mmap. They show a file path like /data/index.db or a shared library like libjvm.so. Large file-backed entries with high RSS are your mmap memory. If these are growing over time, you have a MappedByteBuffer leak or a file mapping that was never unmapped.
Anonymous mappings outside NMT's accounting can come from JNI libraries or other native allocators. Components such as RocksDB JNI and custom C libraries use their own allocation paths. Those regions can widen the gap between process RSS and the categories NMT reports.
The diagnostic workflow: run jcmd <pid> VM.native_memory summary to get the NMT total. Run pmap -x <pid> and sum the RSS column for the total process RSS. The gap between them is your investigation target. The stacked bar at the bottom breaks RSS into NMT-tracked, mmap, and JNI portions.
For mmap leaks, check /proc/<pid>/smaps for entries with high Rss and a file inode. For JNI leaks, use jemalloc profiling or async-profiler native memory mode. The key lesson: NMT is necessary but not sufficient. A complete memory audit always combines NMT, pmap, and knowledge of which native libraries your application loads. Now let us step back and see the whole picture together.
Recap
We started with the JVM Memory Map: the heap is just one slice of a much larger process footprint. Thread stacks, metaspace, code cache, GC structures, and native allocations all live outside the heap and contribute to your process RSS.
Then we traced ByteBuffer.allocateDirect through its internals: Bits.reserveMemory checks the limit, Unsafe.allocateMemory calls malloc. And the result is a small heap wrapper pointing to a large native segment. The heap object is the handle. the native memory is the payload.
We followed the Cleaner lifecycle: when GC collects the DirectByteBuffer wrapper, a PhantomReference is enqueued, the Cleaner thread invokes the Deallocator. And Unsafe.freeMemory returns the native segment to the OS. Deallocation is implicit and depends on GC timing.
We explored the Unsafe escape hatch: raw malloc and free with no GC interaction, no limit tracking, and no safety net. This is the foundation high-performance libraries build on.
We zoomed into Netty arenas: a full pooled allocator built on Unsafe with pre-allocated chunks, buddy page allocation, thread-local caches, and buffer recycling. No GC pressure, no per-request system calls, and millions of allocations per second.
We mapped memory-mapped files: mmap gives you OS-managed, lazily-loaded, file-backed memory that is invisible to both the GC and NMT. Page faults load data on demand. the OS evicts under pressure.
We covered limits and diagnostics: MaxDirectMemorySize caps managed direct buffers, System.gc() is the safety valve, and NMT tracks JVM-internal zones by subsystem.
And we closed with pmap: the OS-level tool that shows the full process memory map, reveals mmap and JNI allocations invisible to NMT. And exposes the RSS gap that explains container OOMs.
The unified takeaway: Java off-heap memory is not one thing. It is a spectrum from fully managed (DirectByteBuffer + Cleaner) through pooled (Netty arenas) and OS-managed (mmap) to fully manual (Unsafe). Understanding which path your libraries use, and knowing which tool reveals each path, is the key to diagnosing memory issues in production.
The whole story in 8 lines
See how direct allocation, Cleaner reclamation, Unsafe, Netty arenas, memory mapping, limits, and native tracking fit together.
- Process RSS includes the Java heap and several native areas, so a flat heap can coexist with rising process memory.
- allocateDirect creates a small heap wrapper and reserves a native segment tracked against the direct-memory limit.
- Direct-buffer storage is reclaimed only after its wrapper becomes unreachable, reference processing runs, and the Cleaner releases it.
- Unsafe allocations bypass GC ownership, so the caller must pair every native allocation with exactly one release.
- Netty arenas, chunks, and pages amortize allocation cost but may retain pooled capacity after request load drops.
- A mapped buffer reserves virtual address space backed by a file, and page faults bring accessed pages into RAM on demand.
- The direct-memory limit caps tracked direct buffers, while NMT categorizes only native allocations known to the JVM.
- Comparing process maps with NMT helps isolate resident native regions that JVM accounting does not explain.