Zero-Deserialization Lookups: The MemorySegment.mismatch() Trick

1. The Serialization Tax

Off-heap maps store data as raw bytes in native memory. There are no Java objects sitting on the heap—just a flat, contiguous MemorySegment full of serialized entries. That’s the entire point: the GC never sees them.

But there’s a catch. When you call map.get(key), the map needs to find the matching entry. In an open-addressing scheme with linear probing, that means walking a probe chain and comparing the lookup key against each candidate. The naive approach is obvious: deserialize the stored key back into a Java object, then call .equals().

// The naive approach — deserialize every candidate key
for (int i = 0; i < probeChainLength; i++) {
    K storedKey = keySerializer.deserialize(segment, offset + keyOffset);
    if (lookupKey.equals(storedKey)) {   // Java object comparison
        return valueSerializer.deserialize(segment, offset + valueOffset);
    }
    offset = nextProbe(offset);
}

Look at what happens on every step of that probe chain:

• A new Java object is allocated on the heap for the deserialized key
• For String keys: new String(bytes, UTF_8) allocates both a byte[] and a String
• The object is immediately garbage after the comparison—pure waste
• This happens for every get(), containsKey(), put() (to check for existing keys), and remove()

This isn’t a theoretical concern. It’s the primary reason many off-heap map implementations underperform expectations. The data may live off-heap, but the operations keep dragging objects back onto the heap.

2. The Insight: Compare Bytes, Not Objects

The solution is embarrassingly simple once you see it: don’t deserialize at all.

Instead of reconstructing a Java object from the stored bytes and then calling .equals(), compare the raw byte representations directly. If the bytes are identical, the keys are equal.

This works for any type with deterministic serialization—a property where the same logical value always serializes to the exact same byte sequence. UTF-8 encoding is deterministic. Big-endian integer encoding is deterministic. If your serializer is deterministic, byte equality implies object equality.

But comparing bytes one at a time in a Java loop would be painfully slow—certainly slower than just deserializing and calling .equals(). We need something faster. Much faster.

3. MemorySegment.mismatch() — The Secret Weapon

Java’s Foreign Function & Memory API provides a method that most developers have never heard of: MemorySegment.mismatch(). It compares two memory regions and returns -1 if they are identical, or the index of the first differing byte.

long mismatch = MemorySegment.mismatch(
    segment1, startOffset1, endOffset1,   // first region
    segment2, startOffset2, endOffset2    // second region
);
// Returns -1 if regions are identical
// Returns index of first differing byte otherwise

Simple API. But the magic is in the implementation.

MemorySegment.mismatch() is intrinsified by the JVM. On modern x86 CPUs, HotSpot replaces the Java method body with hand-tuned machine code that uses SIMD vector instructions—AVX2 or AVX-512 where available. That means comparing 32 or 64 bytes per CPU instruction instead of one byte at a time.

It’s the off-heap equivalent of Arrays.mismatch(), which has been intrinsified since Java 9. Both map down to the same underlying vectorizedMismatch stub in HotSpot.

4. How Mapish Implements Zero-Deserialization Equals

The technique is baked into Mapish’s Serializer<T> interface. Every serializer must implement four methods:

public interface Serializer<T> {
    void serialize(T obj, MemorySegment segment, long offset);
    T deserialize(MemorySegment segment, long offset);
    long sizeBytes();
    boolean equals(T obj, MemorySegment segment, long offset);  // ← the key method
}

The first three are standard. The fourth is where the trick lives: equals(T obj, MemorySegment segment, long offset) compares a Java object (the lookup key you’re searching for) against serialized bytes stored in native memory—without ever deserializing those bytes.

Let’s look at the StringSerializer implementation—the most interesting case because strings are variable-length.

@Override
public boolean equals(String obj, MemorySegment segment, long offset) {
    if (obj == null) return false;

    int length = segment.get(ValueLayout.JAVA_INT, offset);           // 1⃝
    byte[] bytes = obj.getBytes(StandardCharsets.UTF_8);              // 2⃝
    if (length != bytes.length) return false;                         // 3⃝

    if (length == 0) return true;

    MemorySegment objSegment = MemorySegment.ofArray(bytes);          // 4⃝
    return MemorySegment.mismatch(                                    // 5⃝
        segment, offset + 4, offset + 4 + length,
        objSegment, 0, length
    ) == -1;                                                          // 6⃝
}

Let’s walk through each step:

1. Read the stored string length — The first 4 bytes at offset hold the UTF-8 byte length of the stored string. This is a single native memory read—no allocation.
2. Convert the lookup key to UTF-8 bytes — We need the raw bytes of the key we’re searching for. This is the one allocation in the method, and it’s unavoidable: we need the bytes to compare against. Crucially, we are not constructing anything from the stored bytes.
3. Length mismatch → fast reject — If the stored string is 12 bytes long and our lookup key is 15 bytes, they can’t be equal. Reject immediately without touching any more native memory.
4. Wrap the lookup key’s bytes as a MemorySegment — MemorySegment.ofArray(bytes) creates a segment backed by the existing byte[]. This is a zero-copy operation—no data is moved.
5. Use mismatch() to compare — Compare the stored bytes (starting at offset + 4, skipping the length prefix) against the lookup key’s bytes. The JVM uses SIMD instructions under the hood.
6. No String object is ever constructed from native memory — The stored bytes are compared in-place. If they match, we have our entry. If they don’t, we move to the next probe position. Either way: zero deserialization.

For fixed-size types the implementation is even simpler. Here’s IntegerSerializer.equals():

@Override
public boolean equals(Integer obj, MemorySegment segment, long offset) {
    if (obj == null) return false;
    return obj.intValue() == segment.get(ValueLayout.JAVA_INT, offset);
}
// One memory read. Zero allocations. No mismatch() needed.

Primitives can be compared with a single native memory read—no byte comparison needed at all. The mismatch() technique shines for variable-length or larger fixed-size types where a simple == isn’t sufficient.

5. The Two-Layer Defense: Hash Caching + Binary Comparison

The zero-deserialization equals() is powerful, but Mapish layers two optimizations to make the probe chain even faster.

Layer 1: Hash Caching

Every entry in Mapish’s native memory layout stores its 32-bit hash code at a fixed offset (byte 4 of the entry, right after the 1-byte status flag):

During linear probing, the map reads the stored hash first—before touching any key bytes:

// Inside findEntryIndex() — the core lookup loop
for (int i = 0; i < capacity; i++) {
    long offset = ((long) index) * entrySize;
    byte status = memorySegment.get(ValueLayout.JAVA_BYTE, offset);

    if (status == EMPTY) {
        return -1;                                       // end of probe chain
    } else if (status == FILLED) {
        int storedHash = memorySegment.get(ValueLayout.JAVA_INT, offset + hashOffset);
        if (storedHash == h) {                           // Layer 1: integer compare
            if (keySerializer.equals(key, memorySegment, offset + keyOffset)) {
                return index;                            // Layer 2: binary compare
            }
        }
    }
    index = (index + 1) % capacity;                      // linear probing
}

If the hashes don’t match, the keys cannot be equal. Skip immediately. The cost? A single MemorySegment.get(JAVA_INT, …)—one 4-byte native memory read. With a well-distributed hash function, the overwhelming majority of probe-chain comparisons end right here.

Layer 2: Binary Comparison (mismatch)

Only when hashes match—which is rare with a good hash function—does the map fall through to the full binary comparison via the serializer’s equals() method. Even then, no deserialization occurs. The comparison runs directly on native memory with hardware-accelerated vector instructions.

Let’s trace through a concrete example. Suppose the map has 1,000 entries with a 50% load factor (2,048 slots), and we call get("user-42"):

1. Hash "user-42" → slot 173
2. Slot 173: status is FILLED, stored hash is 0x4A2F, our hash is 0x7B01 → skip (integer compare)
3. Slot 174: status is FILLED, stored hash matches → compare bytes with mismatch() → match!
4. Deserialize only the value and return it

Total heap allocations for the key lookup: zero. The only deserialization that occurs is for the value being returned—and that’s unavoidable because you need the Java object.

6. Performance Impact

The zero-deserialization strategy has a measurable impact in Mapish’s JMH benchmarks, particularly when compared to other off-heap map implementations like Yahoo’s Oak.

GC allocation rate

The starkest difference is in allocation rates during lookup operations. Oak’s get() and containsKey() create internal Java objects on every call—OakScopedReadBuffer instances, internal iterators, and intermediate byte arrays. These are short-lived objects, but in a tight loop they accumulate fast.

Mapish’s lookup path, by contrast, allocates near-zero bytes on the heap. The only allocation is the UTF-8 byte array for the lookup key in StringSerializer.equals()—and even that could be eliminated with a thread-local buffer for the truly allocation-obsessed.

Throughput and latency

Lower allocation rates translate directly to:

• Higher throughput — less time spent in allocation and GC means more time executing your code
• More predictable latency — fewer GC pauses means fewer tail-latency spikes
• Better scalability under load — allocation pressure compounds under high request rates

7. When This Technique Applies

The zero-deserialization equals pattern works for any type with deterministic serialization: the same logical value must always serialize to the exact same byte sequence.

For custom serializers, the rule is straightforward: if your serialize() method always produces the same bytes for equal objects, then you can implement equals() using mismatch().

Wrapping Up

The zero-deserialization lookup pattern is Mapish’s most impactful performance optimization. By combining hash caching with MemorySegment.mismatch(), the map achieves key lookups that are:

• Allocation-free on the lookup path (no deserialization, no heap objects)
• Hardware-accelerated via JVM intrinsics and SIMD vector instructions
• Short-circuited by integer hash comparison before touching key bytes

The technique is general-purpose. If you’re building any system that compares structured data in native memory—off-heap caches, memory-mapped file indices, custom network protocol parsers—the same pattern applies: serialize deterministically, compare bytes with mismatch(), skip the object entirely.

MemorySegment.mismatch() is one of those quiet gems in the JDK. Most developers will never need it. But when you do need it, nothing else comes close.