Blob Direct Write moves large-value separation earlier in RocksDB’s write path. When enable_blob_files and enable_blob_direct_write are enabled, values at or above min_blob_size can be written directly to blob files during a write, while th…
The upcoming RocksDB 10.7 release includes a major revamp of parallel compression that dramatically reduces the feature’s CPU overhead by up to 65% while maintaining or improving throughput for compression-heavy workloads. We expect this to…
RocksDB performs a variety of IO operations—user reads, background compactions, flushes, database opens, and verification tasks. Treating all these operations the same makes it difficult for file system implementers to optimize performance,…
Modern RocksDB deployments often run in environments with strict memory constraints—cloud VMs, containers, or hosts with hundreds of DB instances. Unpredictable memory usage can lead to out-of-memory (OOM) errors, degraded performance, or e…
We want to share an update about the bug that allowed our bug bounty researcher to update the release note title in August 2024 involving the RocksDB open-source repository on GitHub. This issue was found and responsibly disclosed to us by
Evolved Binary has been working on several aspects of how the Java API to RocksDB can be improved. Two aspects of this which are of particular importance are performance and the developer experience. The synthetic benchmark repository conta…
By cutting the compaction output file earlier and allowing larger than targeted_file_size to align the compaction output files to the next level files, it can reduce WA (Write Amplification) by more than 10%. The feature is enabled by defau…
Since version 6.15 last year, RocksDB supports Ribbon filters, a new alternative to Bloom filters that save space, especially memory, at the cost of more CPU usage, mostly in constructing the filters in the background. Most applications wit…
Compression algorithms relying on an adaptive dictionary, such as LZ4, zstd, and zlib, struggle to achieve good compression ratios on small inputs when using the basic compress API. With the basic compress API, the compressor starts with an…
The RocksDB team is implementing support for a block cache on non-volatile media, such as a local flash device or NVM/SCM. It can be viewed as an extension of RocksDB’s current volatile block cache (LRUCache or ClockCache). The non-volatile…
BlobDB is essentially RocksDB for large-value use cases. The basic idea, which was proposed in the WiscKey paper, is key-value separation: by storing large values in dedicated blob files and storing only small pointers to them in the LSM tr…
To prevent or mitigate data corrution in RocksDB when some software or hardware issues happens, we keep adding online consistency checks and improving existing ones. We improved ColumnFamilyOptions::force_consistency_checks and enabled it b…