Why Off-Heap? Understanding Java's Memory Model and the FFM API

1. The GC Problem

Java’s garbage collector is one of the language’s greatest strengths—until it isn’t. Modern collectors like G1, ZGC, and Shenandoah have pushed pause times impressively low, but every one of them must still find live objects before it can reclaim dead ones. The more live objects on the heap, the more work that scan requires.

Now picture a HashMap<String, String> holding one million entries. How many Java objects does that create?

• 1 M key String objects (each wrapping a byte[])
• 1 M value String objects (same story)
• 1 M HashMap.Node wrapper objects
• ~1 M backing byte[] arrays for the strings
• The internal Node[] table (plus tree nodes if chains get long)

That’s easily 4 million+ live objects—all of which the GC must traverse on every collection cycle. Even with concurrent collectors, tail latency grows. A G1 mixed-GC pause that normally takes 10 ms might spike to 50–200 ms when your live set is measured in gigabytes. ZGC keeps pauses short, but its concurrent mark phase still consumes CPU cycles proportional to the live object graph.

For latency-sensitive systems—high-frequency trading, real-time ad bidding, game servers—those spikes are unacceptable. You can’t serve a market order 80 ms late, and you can’t stutter a multiplayer game loop because the GC decided now was a good time to clean house.

Here’s the familiar culprit:

// Innocent looking, but creates ~4M+ heap objects
Map<String, String> cache = new HashMap<>(1_000_000);

for (int i = 0; i < 1_000_000; i++) {
    cache.put("key-" + i, "value-" + i);
}
// Every one of those keys, values, and Node wrappers
// is now a live object the GC must scan.

2. The Off-Heap Solution

The idea is disarmingly simple: store your data outside the Java heap. Native memory—allocated directly from the operating system—is invisible to the garbage collector. It doesn’t get scanned, doesn’t get compacted, and doesn’t contribute to pause times.

Think of native memory as a warehouse floor. The JVM heap is a managed storage unit—the facility tracks every box, rearranges them periodically, and bills you for the service. Off-heap memory is the raw warehouse floor next door: you get unlimited space, but you decide where things go, and you are responsible for cleaning up.

This trade-off is the crux of off-heap design. You gain predictable latency and freedom from GC pressure, but you accept responsibility for memory lifecycle management. Forget to release a segment and you have a native memory leak—one that won’t show up in your heap dumps.

Historical approaches

Java developers have been reaching for off-heap memory for years, but the available tools were crude:

• ByteBuffer.allocateDirect() — limited to 2 GB (signed int addressing), no deterministic deallocation, and the API was designed for I/O channels, not general-purpose storage.
• sun.misc.Unsafe — raw pointer arithmetic with zero safety net. No bounds checking. No lifecycle management. A single off-by-one write can silently corrupt memory or crash the JVM with a segfault. And it was never a supported API—every JDK release threatened to remove it.

Both approaches worked, but they forced developers to choose between “limited and awkward” or “powerful and dangerous.” The JDK needed a third option.

3. Enter Java’s FFM API (JEP 454)

The Foreign Function & Memory API was finalized in Java 22 after previewing since Java 19. It provides a safe, supported, performant API for working with native memory. No --add-opens hacks, no unsupported internals—just a first-class JDK citizen.

Three classes form the core of the API:

Here’s the API in action:

try (Arena arena = Arena.ofConfined()) {
    // Allocate 1 KB of native memory, owned by this arena
    MemorySegment segment = arena.allocate(1024);

    // Write an int at byte offset 0
    segment.set(ValueLayout.JAVA_INT, 0, 42);

    // Read it back
    int value = segment.get(ValueLayout.JAVA_INT, 0); // 42

} // Arena closes → segment is immediately freed

That try block isn’t decorative. When the arena goes out of scope, the native memory is unmapped and returned to the OS immediately. No finalizer delays, no phantom reference queues—deterministic deallocation.

4. Why FFM Beats Unsafe

If you’ve worked with sun.misc.Unsafe, FFM will feel familiar—but with guardrails.

The performance story deserves emphasis. Early preview builds of FFM had measurable overhead from bounds checks, but the JIT compiler in modern JDK releases eliminates most of it. In our JMH benchmarks, FFM-based reads and writes perform within noise of raw Unsafe operations—with the safety net included.

5. How Mapish Uses FFM

With the theory covered, let’s look at how the Off-Heap Mapish project puts FFM to work. The design goal is simple: implement java.util.Map with zero GC-visible objects for the stored data.

One giant segment

Instead of allocating an object per entry, Mapish allocates a single contiguous MemorySegment sized to hold the entire hash table:

// From OffHeapMap.java — the entire table is one allocation
this.memorySegment = arena.allocate((long) this.capacity * entrySize);

That single call replaces what would be millions of new Node<>() allocations in a standard HashMap.

Entry layout

Each entry occupies a fixed-size slot in the segment. The layout is calculated once at construction time:

// Status flags — stored as a single byte per slot
private static final byte EMPTY   = 0;
private static final byte FILLED  = 1;
private static final byte DELETED = 2;

// Offset calculations — computed once in the constructor
this.hashOffset  = 4;                                  // after 1-byte status + 3-byte pad
this.keyOffset   = 8;                                  // after the 4-byte hash
this.valueOffset = this.keyOffset + align(keySize, 8); // key size, 8-byte aligned
this.entrySize   = this.valueOffset + align(valueSize, 8);

Reads and writes use ValueLayout accessors on the single segment, jumping to the correct offset with simple arithmetic:

long offset = ((long) index) * entrySize;

// Read the status byte
byte status = memorySegment.get(ValueLayout.JAVA_BYTE, offset);

// Read the cached hash (fast rejection on mismatch)
int storedHash = memorySegment.get(ValueLayout.JAVA_INT, offset + hashOffset);

// Serialize the key directly into the segment
keySerializer.serialize(key, memorySegment, offset + keyOffset);

The Serializer interface

Because the segment stores raw bytes, Mapish needs a way to convert Java objects to and from binary. That’s the job of the Serializer<T> interface:

public interface Serializer<T> {
    void serialize(T obj, MemorySegment segment, long offset);
    T deserialize(MemorySegment segment, long offset);
    long sizeBytes();

    // Compare a Java object against raw bytes — no deserialization needed
    boolean equals(T obj, MemorySegment segment, long offset);
}

That equals() method is the performance secret sauce. During a get() or containsKey() call, the map probes through slots comparing the search key against stored bytes without ever deserializing them. The StringSerializer takes this further by using MemorySegment.mismatch()—a method that maps down to hardware-accelerated vector comparison:

// From StringSerializer.java — zero-allocation equality check
@Override
public boolean equals(String obj, MemorySegment segment, long offset) {
    int length = segment.get(ValueLayout.JAVA_INT, offset);
    byte[] bytes = obj.getBytes(StandardCharsets.UTF_8);
    if (length != bytes.length) return false;
    if (length == 0) return true;

    MemorySegment objSegment = MemorySegment.ofArray(bytes);
    return MemorySegment.mismatch(
        segment, offset + 4, offset + 4 + length,
        objSegment, 0, length
    ) == -1;
}

Arena lifecycle

Mapish gives you a choice. The default constructor uses Arena.ofAuto()—the GC will eventually reclaim the native memory when the map becomes unreachable. For strict deterministic release, use the confined factory:

// GC-managed — behaves like a normal Java object
OffHeapMap<String, String> auto = new OffHeapMap<>(
    new StringSerializer(100), new StringSerializer(100), 2_000_000
);

// Deterministic — memory freed on close()
try (OffHeapMap<String, String> confined = OffHeapMap.createConfined(
        new StringSerializer(100), new StringSerializer(100), 2_000_000)) {
    confined.put("key", "value");
    // ...
} // Native memory immediately unmapped and returned to the OS

Open addressing, not separate chaining

Standard HashMap resolves collisions with linked lists (and tree nodes at high load). Each link is another heap object, another pointer the GC must chase. Mapish uses open addressing with linear probing—colliding entries simply slide into the next available slot in the same contiguous segment. No extra allocations, and scanning adjacent memory slots is exactly what modern CPUs are optimized for (cache-line prefetching).

6. The Payoff

All of this engineering serves one outcome: predictable, low-latency map operations regardless of data size.

Because the entire hash table lives in a single MemorySegment, a Mapish instance holding two million entries contributes exactly one object to the GC’s live set (the OffHeapMap instance itself). Compare that to the 4–8 million objects a standard HashMap would create.

The best part? You don’t need to learn a new API. Mapish implements the full java.util.Map contract— put(), get(), entrySet(), remove(), iterators, all of it. Drop it into existing code, swap your serializers in, and the GC pressure vanishes.

// Swap in, enjoy the silence from the GC
Map<String, String> map = new OffHeapMap<>(
    new StringSerializer(100),
    new StringSerializer(100),
    2_000_000
);

map.put("ticker", "AAPL");
String value = map.get("ticker"); // "AAPL" — zero GC overhead

for (Map.Entry<String, String> entry : map.entrySet()) {
    // Standard iteration — works exactly as you'd expect
}

Where to Go from Here

If your application keeps large reference data in memory—caches, lookup tables, session stores—and you’re fighting GC pauses, off-heap storage with the FFM API is worth exploring. The API is stable, the performance is proven, and projects like Mapish show that you don’t have to sacrifice developer ergonomics to get there.

• Browse the Mapish source on GitHub
• Read JEP 454 — Foreign Function & Memory API
• View the JMH benchmark results