Skip to main content

database.conf - RocksDB Storage Engine Configuration

The database configuration file (database.conf) contains settings for the RocksDB storage engine used by hlquery. This file controls how hlquery stores and manages data on disk using RocksDB's proven storage architecture.

File Location​

run/conf/database.conf

Configuration Format​

The storage configuration uses a single <rocksdb> tag with multiple attributes:

<rocksdb
    memtable_size="16MB"
    max_memtables="2"
    sstable_size="128MB"
    ...>

Size Units​

All size values support units: KB, MB, GB, TB (case-insensitive)

Examples:

  • "16MB" - 16 megabytes
  • "256KB" - 256 kilobytes
  • "1GB" - 1 gigabyte
  • "512" - 512 bytes (if no unit specified)

MemTable Settings​

MemTables are in-memory write buffers that accumulate writes before flushing to disk.

memtable_size​

Size of each in-memory table before flushing to disk.

  • Default: 16MB
  • Range: 1MB-1GB
  • Impact:
    • Larger values = more writes before flush, but more memory usage
    • Smaller values = more frequent flushes, less memory
<rocksdb memtable_size="16MB">

max_memtables​

Maximum number of memtables allowed before forcing flush.

  • Default: 2
  • Range: 1-10
  • Impact:
    • More memtables = better write throughput, but more memory
    • Fewer memtables = less memory, but potentially slower writes
<rocksdb max_memtables="2">

SSTable Settings​

SSTables (Sorted String Tables) are the on-disk storage format.

sstable_size​

Target size for sorted string tables on disk.

  • Default: 128MB
  • Range: 1MB-1GB
  • Impact:
    • Larger SSTables = fewer files, but slower compaction
    • Smaller SSTables = more files, but faster compaction
<rocksdb sstable_size="128MB">

max_levels​

Maximum number of RocksDB levels.

  • Default: 7
  • Range: 3-10
  • Impact:
    • More levels = better space efficiency, but more compaction overhead
    • Fewer levels = less compaction, but potentially more space usage
<rocksdb max_levels="7">

level0_files​

Number of files in level 0 before triggering compaction.

  • Default: 4
  • Range: 2-20
  • Impact:
    • More files = better write performance, but slower reads
    • Fewer files = faster reads, but potentially slower writes
<rocksdb level0_files="4">

Compression Settings​

Compression reduces disk usage at the cost of CPU.

enable_compression​

Enable compression for SSTables.

  • Default: true
  • Options: true, false
  • Impact: Reduces disk usage but uses CPU for compression/decompression
<rocksdb enable_compression="true">

compression_level​

Compression level (0-9, where 9 is maximum).

  • Default: 6
  • Range: 0-9
  • Impact:
    • Higher = better compression but more CPU
    • Lower = faster but less compression
<rocksdb compression_level="6">

compression_per_level​

Different compression for each level (comma-separated).

  • Default: "" (use global compression_level)
  • Options: none, snappy, zlib, lz4, zstd
  • Example: "none,none,snappy,snappy,zlib,zlib,zlib"
<rocksdb compression_per_level="none,none,snappy,snappy,zlib,zlib,zlib">

Background Operations​

enable_compaction​

Enable automatic background compaction.

  • Default: true
  • Options: true, false
  • Impact: Compaction merges and optimizes SSTables
<rocksdb enable_compaction="true">

compaction_threads​

Number of background threads for compaction.

  • Default: 4
  • Range: 1-128
  • Impact: More threads = faster compaction, but more CPU/memory
  • Note: Set to 1 for lightweight operation. Default in database.conf is 4.
<rocksdb compaction_threads="4">

compaction_style​

Compaction algorithm to use.

  • Default: level
  • Options: level, universal, fifo
  • Impact:
    • level: Traditional RocksDB leveled compaction (best for reads)
    • universal: Merge all files at once (better write amp)
    • fifo: Delete old files (time-series data)
<rocksdb compaction_style="level">

max_background_jobs​

Total background threads (flushes + compactions). This is the total thread pool size for RocksDB background operations.

  • Default: max
  • Options: max or numeric value
  • Impact: Smartly use available thread budget when set to max.
<rocksdb max_background_jobs="max">

max_background_flushes​

Maximum concurrent memtable flushes.

  • Default: max
  • Range: 1-128
  • Impact: More = faster flush but more I/O. Set to max for auto-calculation.
<rocksdb max_background_flushes="max">

max_background_compactions​

Maximum concurrent compaction threads.

  • Default: max
  • Range: 1-128
  • Impact: More = faster compaction but more CPU/I/O. Set to max for auto-calculation.
<rocksdb max_background_compactions="max">

Write Flow Control​

RocksDB-style write flow control to prevent too many L0 files.

level0_slowdown_writes_trigger​

Slowdown writes when L0 reaches this many files.

  • Default: 20
  • Range: 4-50
  • Impact: Prevents too many L0 files from accumulating
<rocksdb level0_slowdown_writes_trigger="20">

level0_stop_writes_trigger​

Stop writes completely when L0 reaches this many files.

  • Default: 36
  • Range: 8-100
  • Impact: Emergency brake to prevent compaction from falling behind
<rocksdb level0_stop_writes_trigger="36">

min_write_buffer_number_to_merge​

Minimum memtables to merge before flush.

  • Default: 1
  • Range: 1-10
  • Impact: Higher values = larger flushes = fewer SSTables
<rocksdb min_write_buffer_number_to_merge="1">

Level Size Configuration​

max_bytes_for_level_base​

Maximum bytes for level 1.

  • Default: 256MB
  • Range: 10MB-10GB
  • Formula: Level N size = base * multiplier^(N-1)
<rocksdb max_bytes_for_level_base="256MB">

max_bytes_for_level_multiplier​

Size multiplier between levels.

  • Default: 10
  • Range: 2-20
  • Impact: Each level is this many times larger than previous (L2 = 10L1, L3 = 10L2, etc.)
<rocksdb max_bytes_for_level_multiplier="10">

Bloom Filter Settings​

Table filters reduce disk I/O for non-existent keys. Bloom filters are the default because they have low construction CPU overhead. Ribbon filters are also available for large, long-lived datasets where filter memory pressure matters more than compaction-time construction CPU.

enable_bloom_filter​

Enable table filters for faster lookups.

  • Default: true
  • Options: true, false
  • Impact: Reduces disk I/O for non-existent keys
<rocksdb enable_bloom_filter="true">

filter_policy​

Table filter implementation.

  • Default: bloom
  • Options: bloom, ribbon
  • Impact:
    • bloom: lower construction CPU and temporary memory
    • ribbon: lower filter memory for long-lived SSTables at similar false-positive rates
<rocksdb filter_policy="bloom">

ribbon_bloom_before_level​

Hybrid Ribbon threshold. When filter_policy="ribbon", RocksDB can still use Bloom filters for the newest LSM levels and Ribbon filters for longer-lived levels.

  • Default: 0
  • Impact: 0 keeps Bloom filters for memtable flushes and uses Ribbon filters for longer-lived SSTables. Higher values keep Bloom filters in more upper levels.
<rocksdb filter_policy="ribbon" ribbon_bloom_before_level="0">

bloom_filter_bits​

Bits per key target for Bloom filters, or the Bloom-equivalent bits-per-key target when filter_policy="ribbon".

  • Default: 10
  • Range: 4-20
  • Impact: More bits = fewer false positives, but more memory
<rocksdb bloom_filter_bits="8">

Block Cache Settings​

Block cache caches frequently accessed data blocks in memory.

enable_block_cache​

Enable block-level caching.

  • Default: true
  • Options: true, false
  • Impact: Caches frequently accessed data blocks
<rocksdb enable_block_cache="true">

block_cache_size​

Size of block cache in memory.

  • Default: 32MB
  • Range: 1MB-10GB
  • Impact: Larger cache = better read performance, but more memory. On NVMe/SSD installs with ≥32GB RAM, raise this to 2GB–4GB so hot SSTable blocks stay in RAM and prevent disk trips.
<rocksdb block_cache_size="32MB">

block_size​

Size of data blocks in SSTables.

  • Default: 4KB
  • Range: 1KB-64KB
  • Impact: Smaller = more granular reads, larger = better compression
<rocksdb block_size="4KB">

block_restart_interval​

Number of keys between restart points.

  • Default: 16
  • Range: 1-64
  • Impact: Lower = faster random access, higher = better compression
<rocksdb block_restart_interval="16">

WAL (Write-Ahead Log) Options​

WAL ensures durability by logging writes before applying them.

wal_sync_mode​

WAL sync behavior.

  • Default: normal
  • Options: none, normal, full
  • Impact:
    • none: Fast, risk data loss on crash
    • normal: fsync per write (balanced)
    • full: fsync + sync directory (safest, slowest)
<rocksdb wal_sync_mode="normal">

wal_bytes_per_sync​

Sync WAL to disk every N bytes.

  • Default: 0 (sync every write)
  • Range: 0-100MB
  • Impact:
    • 0 = sync every write (slowest, safest)
    • Large values = batch syncs (faster, small data loss window)
<rocksdb wal_bytes_per_sync="0">

max_total_wal_size​

Maximum total size of all WAL files.

  • Default: 0 (unlimited)
  • Range: 0-10GB
  • Impact: When exceeded, oldest memtables are flushed
<rocksdb max_total_wal_size="0">

wal_ttl_seconds​

WAL file time-to-live in seconds.

  • Default: 0 (keep until after flush)
  • Range: 0-86400
  • Impact: Old WAL files are deleted after this time
<rocksdb wal_ttl_seconds="0">

wal_size_limit_mb​

Maximum size of a single WAL file in MB.

  • Default: 0 (unlimited)
  • Range: 0-1000
  • Impact: New WAL file created when limit reached
<rocksdb wal_size_limit_mb="0">

Directory Paths​

data_dir​

Base directory for RocksDB data storage.

  • Default: HLQUERY_DATA_DIR/rocksdb
  • Example: "/var/lib/hlquery/rocksdb"
<rocksdb data_dir="/var/lib/hlquery/rocksdb">

wal_dir​

Write-Ahead Log directory.

  • Default: data_dir/wal
  • Example: "/var/lib/hlquery/rocksdb/wal"
<rocksdb wal_dir="/var/lib/hlquery/rocksdb/wal">

sstable_dir​

Directory for SSTable files.

  • Default: data_dir/sstables
  • Example: "/var/lib/hlquery/rocksdb/sstables"
<rocksdb sstable_dir="/var/lib/hlquery/rocksdb/sstables">

File Management​

max_open_files​

Maximum number of open file descriptors.

  • Default: -1 (unlimited)
  • Range: -1 or 10-10000
  • Impact: Lower values save memory but may slow reads
<rocksdb max_open_files="-1">

delete_obsolete_files_period_micros​

How often to check for obsolete files (microseconds).

  • Default: 21600000000 (6 hours)
  • Range: 1000000-86400000000
  • Impact: Lower = more frequent cleanup but more overhead
<rocksdb delete_obsolete_files_period_micros="21600000000">

table_cache_numshardbits​

Number of shards for table cache.

  • Default: 6 (64 shards)
  • Range: 0-10
  • Impact: Higher = less contention but more memory overhead
<rocksdb table_cache_numshardbits="6">

I/O and Performance Options​

use_direct_reads​

Bypass OS page cache for reads (O_DIRECT).

  • Default: false
  • Options: true, false
  • Impact: May improve performance with large datasets
<rocksdb use_direct_reads="false">

use_direct_io_for_flush_and_compaction​

Use O_DIRECT for compaction I/O.

  • Default: false
  • Options: true, false
  • Impact: Reduces OS cache pollution
<rocksdb use_direct_io_for_flush_and_compaction="false">

allow_mmap_reads​

Use mmap for reading SSTable files.

  • Default: true
  • Options: true, false
  • Impact: Can be faster but uses more virtual memory
<rocksdb allow_mmap_reads="true">

allow_mmap_writes​

Use mmap for writing SSTable files.

  • Default: false
  • Options: true, false
  • Warning: Usually not recommended (risk of corruption)
<rocksdb allow_mmap_writes="false">

advise_random_on_open​

Tell OS to expect random access pattern.

  • Default: true
  • Options: true, false
  • Impact: May improve read performance
<rocksdb advise_random_on_open="true">

Safety and Verification​

paranoid_checks​

Enable paranoid safety checks.

  • Default: true
  • Options: true, false
  • Impact: Slower but catches corruption early
<rocksdb paranoid_checks="true">

verify_checksums_in_compaction​

Verify checksums during compaction.

  • Default: true
  • Options: true, false
  • Impact: Catches corruption but adds overhead
<rocksdb verify_checksums_in_compaction="true">

Statistics and Monitoring​

enable_statistics​

Collect detailed performance statistics.

  • Default: false
  • Options: true, false
  • Impact: Small overhead but useful for monitoring
<rocksdb enable_statistics="false">

stats_dump_period_sec​

How often to dump stats to LOG file (seconds).

  • Default: 600 (10 minutes)
  • Range: 0-86400
  • Impact: 0 = never dump stats
<rocksdb stats_dump_period_sec="600">

Advanced Write Options​

enable_pipelined_write​

Pipeline writes for better throughput.

  • Default: false
  • Options: true, false
  • Impact: Experimental feature, may improve write performance
<rocksdb enable_pipelined_write="false">

unordered_write​

Allow out-of-order writes for better concurrency.

  • Default: false
  • Options: true, false
  • Impact: May improve write throughput in some workloads
<rocksdb unordered_write="false">

manual_wal_flush​

Disable automatic WAL flush (manual control).

  • Default: false
  • Options: true, false
  • Impact: For advanced users who want explicit flush control
<rocksdb manual_wal_flush="false">

Rate Limiting​

rate_limit_bytes_per_sec​

Limit compaction/flush I/O rate (bytes per second).

  • Default: 0 (unlimited)
  • Range: 0-10GB
  • Impact: Prevents compaction from overloading I/O
<rocksdb rate_limit_bytes_per_sec="0">

rate_limiter_refill_period_ms​

How often to refill rate limiter tokens (milliseconds).

  • Default: 100
  • Range: 10-1000
  • Impact: Controls rate limiter granularity
<rocksdb rate_limiter_refill_period_ms="100">

Performance Presets​

You can use these presets optimized for different use cases:

High Performance Preset​

For production with good hardware:

<rocksdb
    memtable_size="256MB"
    max_memtables="3"
    sstable_size="256MB"
    compaction_threads="2"
    block_cache_size="256MB"
    bloom_filter_bits="10"
>

Memory Optimized Preset​

For memory-constrained environments:

<rocksdb
    memtable_size="8MB"
    max_memtables="1"
    block_cache_size="8MB"
    bloom_filter_bits="6"
    compaction_threads="1"
>

Storage Optimized Preset​

For storage-heavy workloads:

<rocksdb
    memtable_size="128MB"
    max_memtables="2"
    sstable_size="256MB"
    compression_level="9"
    block_cache_size="128MB"
>

Complete Configuration Example​

# Advanced RocksDB Configuration
<rocksdb
    # MemTable Settings
    memtable_size="16MB"
    max_memtables="2"
    
    # SSTable Settings
    sstable_size="128MB"
    max_levels="7"
    level0_files="4"
    
    # Compression
    enable_compression="true"
    compression_level="6"
    
    # Background Operations
    enable_compaction="true"
    compaction_threads="4"
    compaction_style="level"
    max_background_jobs="max"
    max_background_flushes="max"
    max_background_compactions="max"
    
    # Write Flow Control
    level0_slowdown_writes_trigger="20"
    level0_stop_writes_trigger="36"
    min_write_buffer_number_to_merge="1"
    
    # Level Size
    max_bytes_for_level_base="256MB"
    max_bytes_for_level_multiplier="10"
    
    # Bloom Filter
    enable_bloom_filter="true"
    bloom_filter_bits="8"
    
    # Block Cache
    enable_block_cache="true"
    block_cache_size="32MB"
    block_size="4KB"
    block_restart_interval="16"
    
    # WAL
    wal_sync_mode="normal"
    wal_bytes_per_sync="0"
    max_total_wal_size="0"
    
    # File Management
    max_open_files="-1"
    table_cache_numshardbits="6"
    
    # Safety
    paranoid_checks="true"
    verify_checksums_in_compaction="true"
>

IP Filter / DNS Cache​

These settings control the DNS and reverse-DNS cache used by the IP filter.

dns_cache_size​

Maximum entries for the DNS and reverse DNS caches.

  • Default: 1024
  • Range: 1+
  • Impact:
    • Larger values = fewer cache flushes, more memory
    • Smaller values = more DNS lookups, less memory
<ip_filter dns_cache_size="1024">

Tuning Recommendations​

For High Write Workloads​

  • Increase memtable_size to 64MB-256MB
  • Increase max_memtables to 3-4
  • Increase level0_files to 6-8
  • Use compaction_style="level"

For High Read Workloads​

  • Increase block_cache_size to 256MB-1GB
  • Increase bloom_filter_bits to 10-12
  • Use compaction_style="level"
  • Enable enable_statistics for monitoring

For Memory-Constrained Environments​

  • Reduce memtable_size to 8MB-16MB
  • Reduce max_memtables to 1-2
  • Reduce block_cache_size to 8MB-32MB
  • Reduce bloom_filter_bits to 6-8

For Storage-Constrained Environments​

  • Increase compression_level to 7-9
  • Use compression_per_level with higher compression for lower levels
  • Reduce sstable_size to 64MB-128MB