Skip to main content

hlquery.conf - Main Server Configuration

The main configuration file (hlquery.conf) controls server behavior, networking, logging, and performance settings. This is the primary configuration file that hlquery loads on startup.

File Location​

run/conf/hlquery.conf      (dev layout)
/etc/hlquery/hlquery.conf  (system/debian/rpm layouts)

Configuration Format​

hlquery uses a custom XML-like configuration format for readability:

<log method="file" type="*" level="debug" target="/path/to/logfile.log">
<server name="hlquery.local" id="001">
<bind address="0.0.0.0" port="9200" type="http">

Logging Configuration​

Configure logging for different components and levels. Multiple log entries can be defined for different purposes.

Log Entry Syntax​

<log method="file" type="*" level="debug" target="/path/to/logfile.log"
     rotate_size="128MB" rotate_interval="daily"
     max_rotated_files="14" max_age_days="14">

Parameters​

  • method: Output method

    • file: Write to file
    • console: Write to console (stdout/stderr)
    • syslog: Write to syslog

  • type: Log type filter

    • *: All log types
    • startup: Server startup and shutdown events
    • database: Database operations and RocksDB storage
    • search: Search operations and queries
    • network: Network and connection events
    • auth: Authentication events
    • query: User search queries (for analytics)

  • level: Log verbosity level

    • debug: Detailed debugging information (most verbose)
    • normal: Normal operational messages (recommended for production)
    • verbose: Verbose logging
    • critical: Critical errors only (least verbose)

  • target: Output target

    • For method="file": Path to log file
    • For method="console": Ignored (uses stdout/stderr)
    • For method="syslog": Ignored (uses system syslog)

  • rotate_size: Rotate the file when it reaches this size

    • Supports KB, MB, GB, TB
    • Example: "128MB"
    • 0 or omitted disables size-based rotation

  • rotate_interval: Rotate on a time boundary

    • Accepts seconds, daily, or weekly
    • Example: "86400" or "daily"
    • Omit to disable time-based rotation

  • max_rotated_files: Maximum number of rotated files to keep

    • Example: "14"
    • Omit to use the default retention count

  • max_age_days: Delete rotated files older than this many days

    • Example: "7"
    • Omit or set to 0 to disable age-based retention

Example Logging Configurations​

Basic Setup - Single Log File:

# Global server log capturing all events
<log method="file" type="*" level="normal" target="/var/log/hlquery/hlquery.log"
     rotate_size="128MB" rotate_interval="daily" max_rotated_files="14" max_age_days="14">

Advanced Setup - Separate Log Files:

# Global server log
<log method="file" type="*" level="debug" target="/var/log/hlquery/hlquery.log">

# Startup and shutdown events
<log method="file" type="startup" level="normal" target="/var/log/hlquery/startup.log">

# Database operations
<log method="file" type="database" level="normal" target="/var/log/hlquery/database.log"
     rotate_size="128MB" rotate_interval="daily" max_rotated_files="7" max_age_days="7">

# Search operations
<log method="file" type="search" level="normal" target="/var/log/hlquery/search.log">

# Query log for analytics (only user queries, not debug logs)
<log method="file" type="query" level="normal" target="/var/log/hlquery/queries.log"
     rotate_size="64MB" rotate_interval="daily" max_rotated_files="3" max_age_days="3">

# Network events
<log method="file" type="network" level="normal" target="/var/log/hlquery/network.log">

# Authentication events
<log method="file" type="auth" level="normal" target="/var/log/hlquery/auth.log">

Development Setup - Console Logging:

# Development: log to console for immediate feedback
<log method="console" type="*" level="debug">

Production Setup - Syslog Integration:

# Production: integrate with system logging
<log method="syslog" type="*" level="normal">

Module Configuration​

Use the <module> tag when you want to load optional runtime modules such as the ratelimit module.

Rate Limit Module​

The ratelimit module wraps write- and admin-facing APIs with per-IP buckets so you can tone down traffic bursts while letting operators connect from trusted hosts.

<module name="ratelimit">

     <ratelimit enabled="true" limit_per_minute="180" window_seconds="60" log="true">

     <!-- Controls the default write-rate bucket (document/create/update/delete). -->
     <ratelimit_rule name="default-write" limit_per_minute="120" window_seconds="60" operations="AddDocument,UpdateDocument,DeleteDocument,BulkImportDocuments" />

     <!-- A stricter admin bucket for key/user/maintenance commands. -->
     <ratelimit_rule name="admin" limit_per_minute="10" window_seconds="60" operations="CreateUser,DeleteUser,CreateKey,DeleteKey,Flush,Repair" />

     <!-- Target only the "reports" collection for occasional bulk writes. -->
     <ratelimit_rule name="reports-bulk" limit_per_minute="30" window_seconds="120" operations="BulkImportDocuments" collections="reports" />

     <!-- Skip enforcement when requests originate from localhost. -->
     <ratelimit_exception description="Local test harness" ips="localhost" />

</ratelimit>

</module>

You can define multiple <ratelimit_rule> entries to cover different scopes (collections, operations, IP ranges) and pair them with <ratelimit_exception> entries for trusted hosts such as 127.0.0.1/::1. Use the CLI helpers (hlquery-cli module ratelimit status|rules|exceptions) to verify the runtime buckets and exception lists while the server is running.

AI Query Module​

The ai_query module wraps the natural-language query helpers that live in run/modules/m_ai_query. It now exposes multiple CLI and HTTP routes:

  • module ai_query ask <text>: smart route that infers intent and returns an answer with supporting evidence
  • module ai_query collections <text>: ranks the collections most related to the prompt
  • module ai_query search <text>: searches documents and auto-picks the most likely collection when one is not supplied
  • module ai_query status: shows runtime availability, routes, and loaded collections
  • module ai_query parse <text>: compatibility alias for ask

This means prompts like module ai_query ask which collection talks about science? now resolve into ranked collection candidates instead of only doing a raw single-collection search. The module refuses to start unless a models directory exists (default run/models) and a configured model descriptor points to a present file.

<module name="ai_query">

<llm models_dir="run/models" auto_find="yes">

</module>

auto_find="yes" selects the first .gguf file in models_dir after sorting filenames. Use model_file to pin a specific GGUF file or model_name to resolve a built-in local model catalog preset. The shipped presets match tools/download: qwen_0_5, qwen_1_5, qwen_3, qwen_14, and qwen_coder_1_5. If you need to refresh the models bundle, place the files anywhere under run/models (the run/models/ directory is ignored by git), or use tools/download. The module now loads heuristics directly from run/modules/m_ai_query and no longer needs etc/ai/query.js.

Server Identity​

Define the server's identity within the hlquery network.

Syntax​

<server name="hlquery.local" id="001">

Parameters​

  • name: Server name (FQDN)

    • Used for identification in logs and monitoring
    • Example: "hlquery.local", "search.example.com"

  • id: Unique numeric identifier

    • Used for server identification

    • Example: "001", "002"

    • When true: Only read and search operations allowed

    • All write operations (create, update, delete) return 403 Forbidden

    • Useful for demo environments where data modification should be prevented

Example​

<server name="hlquery.production.example.com" id="001">

Analytics​

Configure optional outbound usage analytics. This tag is disabled by default and is intended for SaaS operators that want periodic usage summaries without adding latency to query handling.

Syntax​

<analytics
     active="false"
     endpoint="http://127.0.0.1:8089/v1/analytics/hlquery"
     flush_interval="300"
     connect_timeout_ms="5000"
     track_reads="true"
     track_writes="true">

Parameters​

  • active: Enable or disable outbound analytics

    • false: Analytics are fully disabled
    • true: Analytics worker starts and sends startup and usage batches

  • endpoint: Remote HTTP endpoint that receives analytics payloads

    • Example: "http://127.0.0.1:8089/v1/analytics/hlquery"
    • Leave empty when analytics are disabled

  • flush_interval: Flush interval in seconds

    • 300 means 300 seconds, which is 5 minutes
    • Controls how often in-memory usage counters are exported

  • connect_timeout_ms: Timeout for the outbound analytics HTTP request

    • Value is in milliseconds
    • Example: "5000" means a 5 second timeout

  • track_reads: Track read-side activity

    • Includes search, get, list, and similar read operations
    • true enables read usage reporting

  • track_writes: Track write-side activity

    • Includes insert, update, and delete operations
    • true enables write usage reporting

Behavior​

  • Analytics are best-effort and non-blocking on the request path.
  • When enabled, hlquery sends one startup payload shortly after boot.
  • Regular usage data is aggregated in memory and flushed on the configured interval.
  • On shutdown, hlquery attempts one final flush.
  • If the analytics worker is contended, events may be dropped instead of blocking queries.

Example​

<server name="hlquery.production.example.com" id="001">

<analytics
     active="true"
     endpoint="http://127.0.0.1:8089/v1/analytics/hlquery"
     flush_interval="300"
     connect_timeout_ms="5000"
     track_reads="true"
     track_writes="true">

Network Bindings​

Configure network ports for client connections. Multiple bind entries can be defined for different interfaces or protocols.

HTTP Binding​

<bind address="0.0.0.0" port="9200" type="http">

Parameters​

  • address: IP address to bind to

    • 0.0.0.0: Bind to all network interfaces (default)
    • 127.0.0.1: Bind to localhost only (more secure)
    • Specific IP: 192.168.1.100 (bind to specific interface)

  • port: Port number

    • Default: 9200
    • Must be available and not in use by another process

  • type: Connection type

    • http: Standard HTTP (unencrypted)
    • https: HTTPS with SSL/TLS encryption

HTTPS Binding​

Enable SSL/TLS for secure connections:

<bind address="0.0.0.0" port="9443" type="https"
      ssl_cert="/path/to/server.crt"
      ssl_key="/path/to/server.key"
      ssl_protocols="TLSv1.2 TLSv1.3"
      ssl_ciphers="HIGH:!aNULL:!MD5">

HTTPS Parameters​

  • ssl_cert: Path to SSL certificate file (.crt or .pem)

    • Required for HTTPS
    • Example: "/etc/ssl/certs/hlquery.crt"

  • ssl_key: Path to SSL private key file (.key)

    • Required for HTTPS
    • Must match the certificate
    • Example: "/etc/ssl/private/hlquery.key"

  • ssl_protocols: Allowed SSL/TLS protocols (optional)

    • Default: "TLSv1.2 TLSv1.3"
    • Example: "TLSv1.3" (TLS 1.3 only)

  • ssl_ciphers: Allowed cipher suites (optional)

    • Default: "HIGH:!aNULL:!MD5"
    • Example: "ECDHE-RSA-AES256-GCM-SHA384:ECDHE-RSA-AES128-GCM-SHA256"

Generate SSL Certificates​

Use the provided script to generate self-signed certificates:

./scripts/create_ssl.sh

Certificates will be created in run/ssl/ directory:

  • server.crt: Certificate file
  • server.key: Private key file

Multiple Bindings Example​

# HTTP on all interfaces
<bind address="0.0.0.0" port="9200" type="http">

# HTTPS on all interfaces
<bind address="0.0.0.0" port="9443" type="https"
      ssl_cert="/etc/ssl/certs/hlquery.crt"
      ssl_key="/etc/ssl/private/hlquery.key">

# HTTP on localhost only (for local admin)
<bind address="127.0.0.1" port="9201" type="http">

Performance Settings​

Configure thread and performance limits to control resource usage.

Syntax​

<performance max_threads="10">

Parameters​

  • max_threads: Global maximum number of threads
    • Default: 8
    • Range: 1-256
    • Larger values help saturate multi-core CPUs, but make sure your data is fully indexed and your database is on low-latency NVMe/SSD storage with a healthy RocksDB block cache before increasing this number.
  • search_pool_threads: Threads dedicated to executing search requests (0 = auto)
    • Default: 0 (auto distribution)
    • If there is spare CPU headroom, raise this (30+) so searches can run in parallel without waiting on the HTTP or write pools.
  • http_pool_threads: Threads handling HTTP requests and API work (0 = auto)
    • Default: 0
    • Keep this proportionally smaller than the search pool when the workload is query-heavy.
  • write_pool_threads: Threads that serve ingestion and write operations (0 = auto)
    • Default: 0
    • Reduce this when there is little ingestion traffic so search and HTTP pools can grab more threads.
  • management_pool_threads: Threads for background admin tasks (0 = auto)
    • Default: 0
    • Can remain small (4-6 threads) since management tasks are infrequent.

Example​

# High-throughput configuration for idle CPUs and NVMe storage
<performance
     max_threads="64"
     queue_size="32768"
     max_candidates="1500"
     snippet_step="20"
     max_connections="2000"
     search_pool_threads="36"
     http_pool_threads="12"
     write_pool_threads="6"
     management_pool_threads="6">

Once you have confirmed that the collection is fully indexed and RocksDB is tuned (larger block cache, tuned compaction), set search_pool_threads high to keep the search pool saturated whenever queries arrive in bursts.

IP Allow Filtering​

Configure IP address filtering to restrict access to the server. Supports IP addresses, CIDR subnets, hostnames (with DNS resolution), and wildcard hostnames.

Syntax​

<allow allowed="*">

Parameters​

  • allowed: Comma-separated list of allowed IP addresses, CIDR ranges, and/or hostnames
    • Use "*" to allow all IPs
    • Use specific IPs: "192.168.1.1,10.0.0.5"
    • Use CIDR subnets: "192.168.1.0/24" (allows entire subnet)
    • Use hostnames: "api.example.com" (DNS resolved at startup)
    • Use wildcard hostnames: "*.example.com" (reverse DNS on connection)
    • Mix all types: "192.168.1.1,192.168.1.0/24,api.example.com,*.internal.com"

DNS Resolution Strategy​

The system intelligently uses DNS only when needed:

  • IP-Only Configs: No DNS resolution (zero overhead)
  • Hostnames in Config: Resolved at startup, cached
  • Wildcard Hostnames: Reverse DNS on connection (cached)
  • Client Hostnames: Resolved on connection (cached)

DNS Caching​

  • Forward DNS Cache: Hostname → IPs (up to 1024 entries)
  • Reverse DNS Cache: IP → Hostname (up to 1024 entries)
  • Automatic Flush: Cache flushed every 1 hour via TimerManager
  • Cache Eviction: FIFO (oldest first when full)

Examples​

# Allow all IPs (no DNS)
<allow allowed="*">

# Only localhost
<allow allowed="127.0.0.1">

# Specific IPs (no DNS)
<allow allowed="192.168.1.1,10.0.0.5">

# CIDR subnet (no DNS)
<allow allowed="192.168.1.0/24">

# Hostname (DNS at startup)
<allow allowed="api.example.com">

# Wildcard hostname (reverse DNS on connect)
<allow allowed="*.example.com">

# Mix everything
<allow allowed="192.168.1.0/24,*.internal.com,api.example.com,10.0.0.5">

Default Behavior​

IMPORTANT: If <allow> tag is not provided or allowed is empty, ALL connections are DENIED (secure by default).

You must explicitly configure allowed IPs or use allowed="*" to allow all.

How DNS Resolution Works​

  1. Startup Resolution:

    • Regular hostnames (e.g., api.example.com) are resolved at server startup
    • All resolved IPs are added to the allowed list
    • Results are cached for performance

  2. Connection-Time Resolution:

    • If client connects using a hostname, it's resolved using the cache
    • Wildcard patterns (e.g., *.example.com) trigger reverse DNS lookup
    • Reverse DNS results are cached

  3. Cache Management:

    • Both forward and reverse DNS caches are limited to 1024 entries
    • Cache is automatically flushed every 1 hour
    • Oldest entries are evicted when cache is full (FIFO)

Performance​

  • IP-Only Configs: Zero DNS overhead
  • Hostname Resolution: <5ms (cached after first)
  • Reverse DNS: <10ms (cached after first)
  • Connection Check: <0.1ms total
  • Cache Hit Rate: >95% for repeated lookups

Logging and DNS Resolution​

The system provides intelligent logging that automatically includes hostname information when DNS resolution is enabled:

DNS-Enabled Logging (when hostnames are configured):

  • Log entries show both IP and hostname: 192.168.1.100 (api.example.com)
  • Reverse DNS lookup performed on connection (cached)
  • Enhanced security visibility and troubleshooting

DNS-Disabled Logging (IP-only or wildcard "*" configs):

  • Log entries show only IP addresses: 192.168.1.100
  • No DNS overhead, optimal performance
  • Suitable for static IP environments

Example Log Output:

# With DNS enabled (hostnames configured)
[ip_allow] IP allow filter initialized with 3 IP(s), 1 hostname(s) (DNS resolution enabled)
[httpserver] Connection blocked from IP: 192.168.1.200 (unauthorized.example.com) (IP not allowed)
[httpserver] Accepted HTTP connection from 192.168.1.100:54321 (api.example.com)

# Without DNS (IP-only configuration)
[ip_allow] IP allow filter initialized with 2 IP(s) (no DNS resolution needed)
[httpserver] Connection blocked from IP: 192.168.1.200 (IP not allowed)
[httpserver] Accepted HTTP connection from 192.168.1.100:54321

Important: DNS resolution is only enabled when hostnames are configured in the allowed parameter. For IP-only configurations or wildcard "*", DNS is completely disabled, and logs show only IP addresses.

Security​

  • Default Deny: No config = deny all (secure by default)
  • HTTP 403 Response: Blocked connections receive proper HTTP error
  • Thread-Safe: All operations are mutex-protected
  • DNS Logging: Hostname information in logs when DNS is enabled (hostnames configured)

For detailed IP filtering documentation, see IP Filtering.

Configuration Layout​

hlquery.conf is now the primary runtime configuration and contains merged sections for:

  • authentication/users
  • distributed links
  • search settings
  • storage/performance settings

Optional split files can still be used in some deployments, but they are legacy compared to the merged layout.

Query Settings​

Use <query_settings> in hlquery.conf to control query parsing and lexical resources:

<query_settings
     enable_synonyms="true"
     enable_stopwords="true"
     lexical_scope_preference="merge">
  • lexical_scope_preference="merge" combines global and collection synonyms/stopwords.
  • lexical_scope_preference="local" lets collection synonyms override matching global synonym terms; collection stopwords replace global stopwords when present.
  • lexical_scope_preference="global" lets global synonyms override matching collection synonym terms; global stopwords replace collection stopwords when present.

For synonyms, precedence is applied per normalized term. For example, if both global and local sets define phone, local uses the collection definition for phone, while unrelated global synonym terms still apply.

For stopwords, precedence is applied to the whole stopword scope because stopwords are simple term sets. With local, a collection stopword list replaces the global list for that collection; with global, the global list replaces the collection list.

Auth/User Section Example​

<auth enabled="true">
<user name="admin" token="replace-with-strong-token" flags="admin">
<user name="service_api" token="replace-with-strong-token" flags="user">
<user name="dashboard" password="replace-with-strong-password" flags="user">

Complete Configuration Example​

Here's a complete example configuration file:

# hlquery - Main Server Configuration
# https://www.hlquery.com

# LOGGING CONFIGURATION
# Global server log
<log method="file" type="*" level="normal" target="/var/log/hlquery/hlquery.log">

# Startup events
<log method="file" type="startup" level="normal" target="/var/log/hlquery/startup.log">

# Database operations
<log method="file" type="database" level="normal" target="/var/log/hlquery/database.log">

# Query log for analytics
<log method="file" type="query" level="normal" target="/var/log/hlquery/queries.log">

# SERVER IDENTITY
<server name="hlquery.production.example.com" id="001">

# NETWORK BINDINGS
# HTTP on all interfaces
<bind address="0.0.0.0" port="9200" type="http">

# HTTPS (optional, uncomment to enable)
# <bind address="0.0.0.0" port="9443" type="https"
#       ssl_cert="/etc/ssl/certs/hlquery.crt"
#       ssl_key="/etc/ssl/private/hlquery.key">

# PERFORMANCE SETTINGS
<performance max_threads="8">

# AUTH / USERS (merged in hlquery.conf)
<auth enabled="true">
<user name="admin" token="replace-with-strong-token" flags="admin">
<user name="dashboard" password="replace-with-strong-password" flags="user">

Production Configuration Recommendations​

Security​

  1. Enable Authentication:

    <auth enabled="true">

  2. Use HTTPS:

    <bind address="0.0.0.0" port="9443" type="https"
          ssl_cert="/etc/ssl/certs/hlquery.crt"
          ssl_key="/etc/ssl/private/hlquery.key"
          ssl_protocols="TLSv1.3"
          ssl_ciphers="HIGH:!aNULL:!MD5">

  3. Restrict Network Access with IP Filtering:

    <!-- Allow only specific IPs/subnets -->
    <allow allowed="192.168.1.0/24,10.0.0.5">

    <!-- Or bind to specific interface -->
    <bind address="192.168.1.100" port="9200" type="http">

  4. Use Appropriate Log Levels:

    <!-- Production: normal level -->
    <log method="file" type="*" level="normal" target="/var/log/hlquery/hlquery.log">

Performance​

  1. Optimize Thread Count:

    <!-- Match to CPU cores, but keep low -->
    <performance max_threads="8">

  2. Separate Log Files:

    <!-- Separate logs for easier monitoring -->
    <log method="file" type="database" level="normal" target="/var/log/hlquery/database.log">
    <log method="file" type="search" level="normal" target="/var/log/hlquery/search.log">

  3. Enable Query Logging for Analytics:

    <log method="file" type="query" level="normal" target="/var/log/hlquery/queries.log">

Hot Reload​

hlquery supports hot-reloading configuration changes:

  1. Edit the configuration file
  2. Save the file
  3. The server automatically detects changes
  4. Configuration is reloaded without restarting the server

Note: Some changes require a server restart:

  • Port number changes
  • SSL certificate changes
  • Network binding changes

Environment Variables​

Some settings can be overridden with environment variables:

  • HLQUERY_DATA_DIR: Data directory path
  • HLQUERY_LOG_DIR: Log directory path
  • HLQUERY_PORT: HTTP port number (overrides config file)
  • HLQUERY_CONFIG_DIR: Configuration directory path

Configuration Validation​

hlquery validates the configuration file on startup. Invalid configurations will cause the server to fail to start with an error message indicating the problem.

Common validation errors:

  • Invalid XML syntax
  • Missing required parameters
  • Invalid parameter values
  • File permission issues

Troubleshooting​

Configuration Not Loading​

  1. Check file permissions: Ensure the configuration file is readable

    ls -l /etc/hlquery/hlquery.conf
    chmod 644 /etc/hlquery/hlquery.conf

  2. Validate syntax: Check for XML syntax errors

    # Look for unclosed tags, missing quotes, etc.

  3. Check logs: Review startup logs for configuration errors

    tail -f run/logs/startup.log

Port Already in Use​

  1. Change port in configuration:

    <bind address="0.0.0.0" port="9201" type="http">

  2. Or stop the process using the port:

    lsof -i :9200
    kill <PID>

SSL Certificate Issues​

  1. Verify certificate paths are correct

  2. Check certificate permissions:

    chmod 644 /path/to/server.crt
    chmod 600 /path/to/server.key

  3. Ensure certificates are valid and not expired:

    openssl x509 -in server.crt -text -noout

Log Files Not Created​

  1. Check directory exists and is writable:

    mkdir -p /var/log/hlquery
    chmod 755 /var/log/hlquery

  2. Check disk space:

    df -h /var/log