Skip to main content

Adaptive hybrid search architecture baseline

This note records the search control flow before adaptive ranking is introduced. It is the Phase 0 compatibility baseline. The master adaptive feature gate is off by default, and the ranking path described here remains authoritative while that gate is off.

Request control flow​

SearchAPI::HandleSearch in src/api/search.cpp performs the request-level work in this order:

  1. Validate the HTTP method.
  2. Extract the collection from the route.
  3. Merge query-string parameters and JSON body parameters. Body values win.
  4. Translate SQL input when present and apply SQL execution policy.
  5. Resolve the collection name and dispatch SQL insert, delete, drop, or show-collections operations before document search.
  6. Parse ComprehensiveSearchQuery.
  7. Apply configuration defaults and the request-cache override.
  8. Apply API-key embedded filters.
  9. Validate query text, filters, fields, wildcard policy, and vector payloads.
  10. Look up the finalized HTTP response cache.
  11. Attempt distributed execution when policy permits it.
  12. Fall back to local PerformComprehensiveSearch.
  13. Build JSON, including SQL-specific response shapes when applicable.
  14. Optionally inject maybe suggestions.
  15. Attach response metadata and emit analytics.
  16. Store the finalized response in the response cache.

The response cache key includes the HTTP method, path, canonical query parameters, canonical JSON body, embedded filters, and API-key identity. A cache hit returns the previously finalized response. Trace-enabled cached responses therefore describe the original execution and must be marked as cached snapshots.

Local search order​

SearchAPI::PerformComprehensiveSearch selects one retrieval mode:

  • hybrid when text and vector input are both present and hybrid_alpha is strictly between zero and one;
  • vector when vector input is present without an active hybrid blend;
  • lexical otherwise.

It then performs the following shared stages in order:

  1. Raw lexical, vector, or hybrid retrieval.
  2. filter_by filtering.
  3. Lexical index-completeness diagnostics.
  4. Capture found and out_of before grouping and pagination.
  5. Collection rank weighting.
  6. Runtime module weighting.
  7. Explicit sort, collection default sort, or stable effective-score sort.
  8. Optional preservation of all matched hits for SQL analytics.
  9. Faceting and aggregation over at most 10,000 post-processing hits.
  10. Grouping.
  11. Pagination or SQL offset pagination.
  12. Include/exclude field projection.
  13. Highlight generation.
  14. Final result assignment and complete search-time measurement.

Filters, weighting, sorting, facets, aggregation, grouping, pagination, projection, and highlighting are shared by all retrieval modes. Adaptive work must integrate before those shared operations and must not create a parallel post-processing pipeline.

Legacy hybrid order​

SearchAPI::ProcessHybridSearch in src/api/vector.cpp performs:

  1. Lexical retrieval.
  2. Vector retrieval by exact brute-force candidate scanning.
  3. Load legacy merge, normalization, dynamic-alpha, and rerank settings.
  4. Select query-length alpha when dynamic_alpha is enabled.
  5. Build lexical/vector score and rank maps by document ID.
  6. Optionally normalize component scores.
  7. Apply the existing weighted linear formula or standard two-list RRF.
  8. Sort candidates by descending hybrid score.
  9. Optionally normalize and rerank the configured top-K slice.
  10. Stable-sort the reranked slice by descending hybrid score.

The vector implementation is exact brute force. HNSW and IVF are not available execution backends and must never be reported as executed.

Distributed path​

The request handler attempts TryDistributedSearch before local execution when the request and server policy support distribution. A successful distributed result is serialized directly and marked with X-HLQ-Execution-Mode: distributed. Optional mode falls back to the local pipeline. Strict mode returns service unavailable instead of silently falling back.

A future distributed trace must keep coordinator activity and per-node summaries separate. Network arrival order is not a valid global stage order.

Existing response contract​

The standard JSON response contains hits, found, out_of, page, per_page, total_pages, page-navigation booleans, indexing and partial-result state, optional facets, aggregations and distributed diagnostics, and search_time_ms.

Each hit contains the projected document, text_match, _text_match, weight, and final score. Vector requests add vector_score; positive hybrid scores add hybrid_score; highlights are optional.

When execution tracing is disabled, response generation must add no new field. When enabled, the only new standard-response field is the top-level search_execution object. It is typed separately from document fields.

Hanalyzer baseline​

The current Hanalyzer search composable preserves backend hit order but maps hits into a reduced client shape, drops most top-level response metadata, applies an additional plain-query filter, and replaces the backend found count with the filtered client count. Phase 1 does not change the UI; later UI work must preserve search_execution and the authoritative backend count.

Future integration points​

Deterministic query feature extraction belongs immediately after request validation and before retrieval planning. Adaptive fusion belongs inside the existing ProcessHybridSearch merge section, after lexical and vector candidates exist and before shared filtering, weighting, sorting, grouping, and pagination. Neither feature should duplicate HandleSearch, PerformComprehensiveSearch, or the existing post-processing path.