Embeddings and retrieval

a coordinate for meaning; similar meanings land close together.

Embedding

the embedding comes back as a fixed-length array of floats, and that length decides what it costs you to store and compare.

Vector / dimension

you compared a vector from one model against a vector from another and got similarity scores that meant nothing at all.

Embedding space

you pick a separate and far cheaper model whose output is a vector, and benchmark two candidates on your own documents before committing.

Embedding model

you rank results by the angle between two vectors, which is a dot product over two magnitudes and one line of code.

Cosine similarity

the index makes you choose a distance operator, and for the normalised vectors most models emit the choice barely changes the ranking.

Dot product and Euclidean distance

you compare the query against every stored vector, which was fine at ten thousand rows and hopeless at ten million.

Nearest neighbour search (k-NN)

you stop comparing the query against every vector you stored and accept a slightly worse top ten in exchange for a search that finishes in milliseconds.

Approximate nearest neighbour (ANN)

you were about to stand up a dedicated vector product and realised the database you already operate has an extension for it.

Vector database

you add a vector column to the database you already run, and the embeddings land inside your existing backups and transactions.

pgvector

you built the index, watched memory use climb and the build take a while, and got fast accurate search out of it.

HNSW

you pick the cheaper, smaller index because the machine is memory-bound, and accept lower recall at the same speed.

IVFFlat

you split the document into pieces before you store it, because embedding a whole 40-page PDF as one blob retrieves nothing useful.

Chunking

you cut at four hundred tokens and lost the sentence that straddled the boundary, so you make adjacent chunks overlap and try again.

Chunk size and overlap

you split on headings and paragraph boundaries instead of every five hundred tokens, and retrieval on your documentation gets better.

Semantic chunking

you prepend a line saying which document and section a chunk came from before embedding it, and chunks that were ambiguous alone start getting found.

Contextual retrieval

you raised k to twenty, the right chunk was definitely in there, and the model missed it among the noise anyway.

Top-k retrieval

you add a tenant predicate next to the similarity ordering, because similarity on its own happily returned another customer's document.

Metadata filtering

someone searched for an exact error code and semantic search returned five conceptually similar pages that did not contain it.

Lexical search / BM25

you run keyword search and vector search over the same corpus and merge the two result lists, because each one finds what the other misses.

Hybrid search

you fetch fifty candidates cheaply and then have a slower, better model reorder them before you show the top five.

Reranking

you embed queries and documents separately so the index can be built ahead of time, then let a slower model read the pair together for the few you actually rank.

Bi-encoder vs cross-encoder

you look up the relevant bits of your own documents first and paste them into the prompt, so the answer comes from your data instead of the model's memory.

Retrieval-augmented generation (RAG)

you tell it to answer only from the retrieved chunks and to admit when they do not cover it, then check each claim back against them.

Grounded / faithful answer

build a 50-question set with known source chunks and measure recall before touching generation; most bad RAG is bad retrieval.

Recall@k

you use the public leaderboard to shortlist three embedding models and your own corpus to pick one of them.

MTEB

you swapped embedding models and the old vectors are no longer comparable to the new ones, so the whole corpus has to be re-embedded.

Embedding model versioning