RogueMap

RogueMap is a Java embedded storage engine designed to provide persistent, off-heap data structures and an integrated AI memory layer for hybrid retrieval. It addresses the limitations of the Java Virtual Machine (JVM) memory by utilizing mmap-based off-heap storage, enabling Java applications to achieve persistence and AI-powered memory functions without being constrained by traditional heap size barriers.

The engine offers four main off-heap data structures: RogueMap, RogueList, RogueSet, and RogueQueue. These are accessible through a unified builder pattern and support features such as data persistence, transactions, time-to-live (TTL) for automatic data expiration, automatic checkpointing, and crash recovery. The platform is optimized for high concurrency and large-scale storage, employing segmented indexing, optimistic reads, automatic capacity expansion, and a LowHeap indexing approach that can reduce heap memory usage by up to 99%. Supported data capacity can reach terabyte levels.

RogueMap includes an AI memory layer called RogueMemory, which combines vector approximate nearest neighbor (ANN) search with BM25 keyword-based retrieval. This hybrid approach allows for efficient and intelligent data retrieval. The AI memory layer supports integration with popular embedding services such as OpenAI and Ollama, eliminating the need for external vector databases or search engines.

Operational observability is built in, with StorageMetrics providing monitoring of usage, fragmentation rates, and entry counts. The system can trigger compaction based on defined thresholds, and supports automatic expiration of TTL data. RogueMap is distributed as a Maven dependency for Java projects, with separate modules for the core storage engine and the AI memory layer.