What are embeddings in AI?

Embeddings are numerical vector representations of text (or images, audio) where semantic similarity maps to geometric closeness - the engine behind search, recommendation, and RAG.

AIEmbeddingsMachine Learning

Overview

An embedding is a dense array of floating-point numbers (a vector) that encodes the meaning of a piece of text. Texts with similar meanings end up as vectors that are close together in this high-dimensional space, even if the words are completely different. This lets computers perform semantic search ("find documents that mean the same thing as this query") rather than just keyword matching.

Key Points

•An embedding model maps text → a fixed-length vector (e.g., 1536 floats for OpenAI's text-embedding-3-small).
•Semantic similarity = geometric proximity. "Dog" and "puppy" are close; "dog" and "database" are far.
•Similarity is measured with cosine similarity (angle between vectors) or dot product.
•Embeddings power: semantic search, RAG retrieval, clustering, classification, duplicate detection, recommendation engines.
•Embedding the same text twice always produces the same vector - they are deterministic.
•Cross-modal embeddings (e.g., CLIP) can map images and text into the same space, enabling image search by text.

Examples

Generating and comparing embeddings

typescript

import OpenAI from 'openai';

const client = new OpenAI();

async function embed(text: string): Promise<number[]> {
  const result = await client.embeddings.create({
    model: 'text-embedding-3-small',
    input: text,
  });
  return result.data[0].embedding; // array of 1536 floats
}

// Cosine similarity: returns a value from -1 (opposite) to 1 (identical)
function cosineSimilarity(a: number[], b: number[]): number {
  const dot = a.reduce((sum, ai, i) => sum + ai * b[i], 0);
  const magA = Math.sqrt(a.reduce((s, ai) => s + ai * ai, 0));
  const magB = Math.sqrt(b.reduce((s, bi) => s + bi * bi, 0));
  return dot / (magA * magB);
}

const [dog, puppy, database] = await Promise.all([
  embed('a dog'),
  embed('a puppy'),
  embed('a relational database'),
]);

console.log(cosineSimilarity(dog, puppy));    // ~0.92 - very similar
console.log(cosineSimilarity(dog, database)); // ~0.18 - very different

Simple semantic search without a vector database

typescript

const documents = [
  'How to reset your password',
  'Billing and payment options',
  'Cancel your subscription',
  'Download the mobile app',
];

// Index: embed all documents once
const docEmbeddings = await Promise.all(documents.map(embed));

// Query: embed the question and find closest document
async function semanticSearch(query: string): Promise<string> {
  const queryEmb = await embed(query);
  let bestIdx = 0;
  let bestScore = -Infinity;

  docEmbeddings.forEach((docEmb, i) => {
    const score = cosineSimilarity(queryEmb, docEmb);
    if (score > bestScore) { bestScore = score; bestIdx = i; }
  });

  return documents[bestIdx];
}

await semanticSearch('I forgot my login details');
// Returns: 'How to reset your password' - matches by meaning, not keywords

Batch embedding for efficiency

typescript

// Don't call the API in a loop - batch all texts in one request
const texts = articles.map(a => a.content);

const result = await client.embeddings.create({
  model: 'text-embedding-3-small',
  input: texts,           // pass an array, not a single string
  dimensions: 256,        // optional: reduce dimensions to save storage
});

const embeddings = result.data.map(item => item.embedding);
// result.usage.total_tokens → total tokens billed

Pro Tip

Choose your embedding model based on the retrieval task. `text-embedding-3-small` (OpenAI) is fast and cheap and works well for English. For multilingual content, use `multilingual-e5-large` or `text-embedding-3-large`. Store embeddings in a vector database (Pinecone, pgvector, Chroma, Weaviate) for nearest-neighbour search at scale.