Embeddings and Vector Search: What They Give You That Keyword Search Cannot
A practical introduction to embeddings, similarity search, chunking, indexing, retrieval, and the limitations of semantic search systems.
Embeddings map data into numerical vectors where nearby vectors are intended to represent related content; vector search then retrieves items using a similarity measure over those representations. The important part is to understand what is measured, what is inferred, and what remains unknown.
The core idea
Keyword search looks for explicit terms and linguistic matches. Embedding search instead uses a learned representation where related meanings can be close even when the exact words differ. This makes semantic retrieval powerful for discovery, but it also introduces model-specific behavior and a new quality problem: choosing which representation and similarity method actually fit the task.
Treat this as a design problem before treating it as a coding problem. Write the assumptions down. A short experiment can often settle a question that a long argument cannot.
How it works
An embedding model converts an input into a fixed- or variable-sized numerical representation. Similarity can then be computed using a metric such as cosine similarity or another distance function.
Retrieval systems usually store embeddings in an index so they do not compare a query against every item. Approximate nearest-neighbor methods trade exactness for speed as the corpus grows.
The retrieved text is only as useful as the chunking, metadata, embedding model, and ranking pipeline. A semantically similar passage can still be incomplete, outdated, or wrong.
A concrete example
A campus knowledge base can embed club descriptions and student questions. A question such as 'Where can I find robotics workshops?' may retrieve pages that never contain the exact word 'workshops' but describe robotics events and labs. A hybrid system can combine semantic retrieval with keywords and filters.
Change one input or one assumption and predict the result before testing it. This is a compact way to turn passive reading into an active learning loop.
Common mistakes
- Treating the similarity score as a confidence score about truth.
- Creating chunks that are too large to retrieve precisely or too small to preserve context.
- Changing the embedding model without rebuilding the corpus and comparing retrieval behavior.
A student project that makes it stick
Create a small reproducible experiment around the mechanism. Store the dataset or fixture, the code, the measurement method, and the result. If the experiment cannot be rerun, the lesson is harder to verify later.
Where it connects
This topic connects to the surrounding engineering stack: data, networking, security, software design, and operations. The most useful concepts are the ones that explain behavior across several layers rather than only one framework.
What to remember
- Define the objective before selecting the technique.
- Make hidden assumptions explicit.
- Preserve a baseline so improvements are measurable.
- Inspect failure cases, not only averages.
- Keep the experiment small enough to understand end to end.
Limitations
No simplified guide can capture every implementation detail. Results vary with data, versions, hardware, workload, and configuration. The sources below provide the normative or technical reference; use them when a production decision depends on details omitted here.
Related Observatory reads
- evaluating ai systems with multiple measures
- how a browser loads a web page
- sql joins group by and aggregation
Primary sources
Evidence
Sources & further reading
Primary sources, official disclosures, and external research used to ground this report.
- Sentence Transformers Documentationsbert.net
Practical documentation for text embeddings and semantic search.
- FAISS Documentationfaiss.ai
Reference implementation and documentation for efficient vector similarity search.
Keep Exploring
Related observations.
Transformers and Attention: The Core Idea Without the Buzzwords
Attention lets a model compute relationships between positions in a sequence instead of processing every position as an isolated step.
Train, Validation, and Test Sets: The Simplest Way to Stop Fooling Yourself
A model can look excellent because you accidentally gave it information about the evaluation set. Train/validation/test discipline is how you separate learning from measurement.
Overfitting, Bias, and Variance: Why More Flexible Models Can Get Worse
A model can become better at the training set while becoming worse at unseen data. That gap is the core of overfitting.