Hybrid RAG search: chunking, reranking and evidence

Security retrieval needs to handle both exact identifiers and explanations phrased differently. A patch reference, route name and impact description may need different retrieval signals. Compare lexical and semantic methods against representative questions instead of assuming one works best everywhere.

Hybrid retrieval combines signals. It does not establish that results are applicable or authorised. The enterprise RAG pipeline must define those constraints first.

Preserve a meaningful unit

A chunk stating “this operation requires an administrator” is ambiguous when the operation name remains in the previous chunk. A whole document can have the opposite problem, mixing several products and revisions.

For procedures, try sections retaining title, product and revision. For API references, keep method, route, preconditions and expected response together. Associate every chunk with a stable source reference.

Overlap can restore continuity while adding duplicates. If the first five hits repeat one sentence, the context contains fewer distinct pieces of evidence than the hit count suggests. Measure redundancy before enlarging context.

Combine ranks deliberately

Lexical search may help with exact identifiers, while vector retrieval may help with paraphrases. Test both properties in your own corpus. Choosing an embedding alone cannot establish successful retrieval for every formulation.

Scores from different engines need not share a scale. Reciprocal rank fusion combines positions:

document score = sum of 1 / (c + rank) over lists containing the document

The constant, list depth and duplicate handling belong in the experiment record. This expression combines ranks rather than estimating a probability of relevance. Forbidden documents must not reach downstream consumers.

Reranking cannot recover a missing candidate

A reranker reorders supplied passages. If initial retrieval missed the required procedure, reranking cannot invent it. Measure candidate recall separately from final ordering.

An external reranking service also receives content. Apply the same confidentiality scope as for generation, even when the passage never reaches the final answer.

Design a discriminating experiment

Annotate expected passages for exact references, paraphrases, neighbouring revisions and unanswerable questions. Keep a separate evaluation set instead of repeatedly tuning against the same examples.

Compare variants with the same permissions and explicit budgets. Candidate recall, evidence precision, latency and abstention errors answer different questions. An aggregate score may conceal failure on every exact identifier.

Our offline demo implements access filtering and lexical overlap only. It does not implement BM25, embeddings or fusion. Reuse its isolation cases before adding a real search engine in a separate experiment.

The OpenSearch documentation describes one implementation; retain the tested version and configuration. For relationship and whole-corpus questions, consider GraphRAG. The series index connects these retrieval choices to security and evaluation.