What is semantic search and how is it different from keyword search?

Keyword search matches exact words in a query; semantic search matches meaning, so it finds relevant results even when no words overlap. Embeddings are what make semantic search possible.

AIEmbeddingsSearch

Overview

Traditional search (Elasticsearch, SQL LIKE) matches documents that contain the exact words you searched for. Semantic search converts both queries and documents to embedding vectors and finds the ones closest in meaning - regardless of the specific words used. This is why "how do I reset my login?" can return a document titled "password recovery guide" even though no words overlap.

Key Points

•Keyword search: fast, exact, works well when users know the right terminology.
•Semantic search: finds conceptual matches even with zero word overlap - critical for natural language queries.
•Hybrid search combines both: keyword recall + semantic reranking gives the best results in practice.
•BM25 is the standard keyword algorithm used by Elasticsearch, Solr, and most search engines.
•Reranking (cross-encoders like Cohere Rerank) adds a third stage: take the top-50 semantic matches and score them precisely.
•Most production search pipelines today use: BM25 (keyword) + embedding ANN (semantic) + reranker = excellent results.

Examples

Keyword vs semantic search comparison

typescript

const documents = [
  { id: 1, text: 'How to reset your password' },
  { id: 2, text: 'Billing and payment methods' },
  { id: 3, text: 'Account recovery guide' },
  { id: 4, text: 'Two-factor authentication setup' },
];

const query = 'I forgot my login credentials';

// Keyword search: looks for exact word matches
// "forgot", "login", "credentials" - none appear in doc titles
// Result: no matches found

// Semantic search: finds documents by meaning
// "forgot login credentials" is conceptually close to "reset password" and "account recovery"
// Result: doc 1 (similarity 0.91), doc 3 (similarity 0.87)

Hybrid search implementation

typescript

// Hybrid search: combine BM25 keyword scores + embedding cosine scores
async function hybridSearch(query: string, topK = 10) {
  // Run both searches in parallel
  const [keywordResults, semanticResults] = await Promise.all([
    elasticsearchClient.search({
      index: 'documents',
      body: {
        query: { match: { text: query } },  // BM25 keyword search
        size: topK * 2,
      },
    }),
    vectorDB.query({
      vector: await getEmbedding(query),    // semantic vector search
      topK: topK * 2,
      includeMetadata: true,
    }),
  ]);

  // Reciprocal Rank Fusion (RRF): combine the two result lists
  const scores = new Map<string, number>();
  const K = 60; // RRF constant

  keywordResults.hits.hits.forEach((hit, rank) => {
    const id = hit._id;
    scores.set(id, (scores.get(id) ?? 0) + 1 / (K + rank + 1));
  });

  semanticResults.matches.forEach((match, rank) => {
    const id = match.id;
    scores.set(id, (scores.get(id) ?? 0) + 1 / (K + rank + 1));
  });

  // Sort by combined RRF score and return top K
  return [...scores.entries()]
    .sort((a, b) => b[1] - a[1])
    .slice(0, topK)
    .map(([id, score]) => ({ id, score }));
}

Reranking with Cohere for final precision

typescript

import { CohereClient } from 'cohere-ai';

const cohere = new CohereClient({ token: process.env.COHERE_API_KEY });

async function searchWithRerank(query: string) {
  // Stage 1: fast retrieval (get 50 candidates cheaply)
  const candidates = await hybridSearch(query, 50);
  const texts = candidates.map(c => c.text);

  // Stage 2: precise reranking (cross-encoder scores each candidate against the query)
  const reranked = await cohere.rerank({
    model: 'rerank-english-v3.0',
    query,
    documents: texts,
    topN: 5,  // keep only the best 5
  });

  return reranked.results.map(r => ({
    text:      texts[r.index],
    score:     r.relevanceScore,
  }));
}

Pro Tip

Do not replace keyword search with semantic search - combine them. Keyword search has perfect recall for exact product names, error codes, and IDs. Semantic search handles natural language questions. Hybrid search with RRF fusion beats either alone by 15-25% on standard benchmarks.