AI: vector embeddings
This course explains how embedding models turn text into vectors and why those vectors are useful for retrieval. It starts with sparse representations like one-hot encoding and moves to learned dense embeddings, where distance carries meaning. It covers tokenization, contextualization and pooling, contrastive training with hard negatives, and the split between bi-encoders and cross-encoders. It then treats Matryoshka dimensions and truncation, choosing a model with your own labeled questions, and adapting to a domain through query rewriting, adapters or fine-tuning.
What you'll learn
- ✓Why dense embeddings place related meanings close together while one-hot vectors cannot
- ✓How tokenization, attention and pooling turn a sentence into one vector
- ✓How contrastive training with hard negatives shapes the embedding space
- ✓Why bi-encoders retrieve and cross-encoders rerank in the same pipeline
- ✓How Matryoshka training allows truncation and how to pick a dimension count by measuring
- ✓How to choose a model with recall on your own data and when fine-tuning is worth it
What a vector holds
Two courses have now treated the embedding as a given: text goes in, 1024 floats come out, and near vectors mean related text. That worked, and it left the interesting question unasked.
Read the full lesson free →Curriculum
7 lessonsWhat a vector holds
Compares one-hot and bag-of-words vectors with learned dense embeddings, where distance encodes topic but not truth.
From text to one vector: tokens and pooling
Walks through tokenization, contextualization and pooling, and how each stage explains long-chunk, error-code and truncation behavior.
How embedding models are trained
Explains contrastive learning, mined training pairs, hard negatives and the query-passage asymmetry behind the input type parameter.
Bi-encoders and cross-encoders
Shows why bi-encoder vectors are precomputable, cross-encoder scores are not, and how the retrieve-then-rerank funnel combines them.
Dimensions, Matryoshka, and truncation
Covers Matryoshka training, renormalizing truncated vectors and measuring where recall drops before fixing the dimension count.
Choosing an embedding model
Explains how leaderboards mislead and how to choose a model using recall on your own labeled questions.
Adapting embeddings to your domain
Lists cheaper fixes to rule out first, then covers fine-tuning from click data, adapters and query rewriting.