Library Comparison
A detailed comparison of three off-heap Java map libraries — each solving GC pressure with fundamentally different trade-offs.
Java's garbage collector is one of its greatest strengths — until it isn't. In latency-sensitive applications such as financial systems, real-time analytics, and high-frequency trading engines, GC pauses measured in milliseconds can translate to measurable business impact. All three libraries discussed here — Mapish, Yahoo Oak, and MapDB — were created to solve the same fundamental problem: storing large volumes of key-value data outside the Java heap so the garbage collector never has to scan or compact it. However, they approach this problem with radically different architectures and target very different use cases.
Mapish is a lean, modern, single-threaded off-heap hash map built on Java's Foreign Function &
Memory (FFM) API. It prioritizes raw throughput, minimal GC allocation, and full java.util.Map
compliance with zero runtime dependencies. Yahoo Oak takes a concurrent, sorted approach — a
thread-safe off-heap skip-list designed for multi-threaded workloads that require ordered iteration. MapDB
is the heavyweight of the three: a full embedded database engine that offers persistence, transactions, and
crash-safe writes backed by memory-mapped files.
Understanding these trade-offs is essential when choosing the right tool. A library optimized for single-threaded throughput will outperform a concurrent one in a single-threaded context, but cannot be shared across threads. A persistent engine gives you durability, but at the cost of raw speed. This page provides a structured comparison to help you make an informed decision based on your actual requirements.
| Dimension | Mapish | Yahoo Oak | MapDB |
|---|---|---|---|
| API Compliance | ✓ Full java.util.Map | Custom API | ✓ ConcurrentMap |
| Thread Safety | Single-threaded | ✓ Thread-safe | ✓ Thread-safe |
| Ordering | Unordered | ✓ Sorted | ✓ Both options |
| Persistence | In-memory only | In-memory only | ✓ Disk + Transactions |
| Memory Management | FFM API (Modern) | sun.misc.Unsafe | DirectByteBuffer / MMap |
| Java Version | Java 21+ | Java 11+ | Java 8+ |
| Dependencies | ✓ Zero | External deps | External deps |
| Key Lookup Strategy | Zero-deser binary mismatch | Comparator-based | Serializer-based |
| GC Impact | ✓ Near-zero | Higher alloc rate | Moderate |
| Complexity / Setup | ✓ Minimal | Moderate | Heavy |
| Use Case Sweet Spot | Single-threaded hot paths, zero-GC caches, lookup tables | Concurrent sorted maps, multi-threaded ordered access | Persistent storage, embedded DB, transactional workloads |
| License | Apache 2.0 | Apache 2.0 | Apache 2.0 |
Choose Mapish when…
java.util.Map compliance
Choose Oak when…
OakSerializer
Choose MapDB when…
Mapish allocates a single, contiguous off-heap memory segment using Java's
MemorySegment and
Arena APIs (introduced in Java 21).
The hash table uses open addressing with linear probing, meaning all entries — status flags,
cached hashes, keys, and values — are packed sequentially in a flat byte array.
This layout maximizes cache locality — linear probing walks adjacent memory, staying within
CPU cache lines. Key lookups use MemorySegment.mismatch()
to perform a direct binary comparison against stored key bytes, completely bypassing deserialization. The 32-bit
cached hash further short-circuits comparisons when hashes don't match. No sun.misc.Unsafe
is used — memory management is safe and well-defined.
Oak implements a custom off-heap memory allocator built on top of
sun.misc.Unsafe and
DirectByteBuffer. Internally, it maintains a
concurrent skip-list — a probabilistic sorted data structure where each node contains
off-heap pointers to key and value memory blocks.
The skip-list design enables sorted iteration and range queries, but involves
pointer chasing between non-contiguous memory regions. Thread safety is achieved through
internal concurrency-control mechanisms, which require on-heap object allocations (increasing GC pressure).
Developers must implement both OakSerializer and
OakComparator for each key/value type — a higher
integration burden than standard java.util.Map.
MapDB is architecturally a full embedded database engine. It uses
memory-mapped files (MappedByteBuffer)
to map disk-backed files into the virtual address space, allowing the OS to manage page caching. Internally,
data is organized in B-Tree pages (for sorted maps) or hash table buckets (for hash maps),
with pages flushed to disk for persistence.
This architecture gives MapDB unique capabilities — transactions, crash-safe writes, and data
durability — that the other two libraries don't offer. The trade-off is complexity and overhead:
memory-mapped I/O involves kernel-level page faults, DirectByteBuffer
objects add GC pressure, and the B-Tree page structure introduces indirection that reduces raw throughput
compared to Mapish's flat memory layout.
Qualitative analysis based on architectural properties and benchmark observations.
Mapish — Highest single-threaded
Contiguous memory layout with linear probing means lookups walk adjacent cache lines.
Zero-deserialization key matching via mismatch()
and cached hashes eliminate the most expensive part of off-heap lookups. In JMH benchmarks,
Mapish consistently delivers superior single-threaded ops/ms.
Oak — Concurrency over throughput
The concurrent skip-list involves pointer chasing between non-contiguous memory blocks and internal concurrency control overhead. Single-threaded throughput is lower, but Oak scales across cores for concurrent workloads where Mapish cannot be used.
MapDB — Durability over speed
Memory-mapped file I/O, B-Tree page traversal, and transaction journaling all add latency. MapDB trades raw speed for durability guarantees — page faults, fsync calls, and write-ahead logging are inherent costs of persistence.
Mapish — Near-zero allocation
Operations allocate virtually no on-heap objects. The FFM API's
MemorySegment slices are lightweight
value-like objects. JMH profiling shows allocation rates approaching zero bytes per operation,
making Mapish ideal for GC-sensitive hot paths.
Oak — Higher alloc rate
Concurrency control mechanisms create on-heap objects for thread coordination (versioned
references, epoch-based reclamation structures). JMH benchmarks reveal a measurably higher
gc.alloc.rate.norm compared to Mapish.
MapDB — Moderate overhead
DirectByteBuffer instances are tracked
by the GC even though their backing memory is off-heap. Memory-mapped regions use OS page cache,
but buffer management and serialization paths generate moderate heap allocations.
Mapish — Vertical (single-thread)
Scales vertically by maximizing the throughput of a single thread. In thread-per-core architectures, each core gets its own Mapish instance — no lock contention, no cache invalidation, maximum per-core efficiency.
Oak — Horizontal (multi-thread)
Scales horizontally across threads sharing a single map instance. Ideal for conventional multi-threaded server applications where multiple request-handling threads access shared state.
MapDB — Horizontal + Persistent
Scales to datasets larger than available RAM via disk-backed storage. Data survives restarts. Thread-safe access allows multi-threaded use, though not optimized for high-contention concurrent access like Oak.
Bottom line: If your workload is single-threaded and GC-sensitivity is paramount, Mapish delivers the best raw performance. If you need to share a sorted map across threads, Oak is purpose-built for that. If you need your data to survive a restart, MapDB is the only option among these three. They are complementary tools, not competitors.