What Are Text Embeddings? Visual Guide (2026)

"The cat sat on the mat" and "Dogs love playing fetch" score a cosine similarity of 0.169. Dogs versus "the stock market crashed today" comes out to negative 0.020, slightly below zero, which the math says means faintly unrelated. Those two numbers are the whole reason I built the Embedding Playground: embeddings are the technology behind semantic search and RAG, everyone talks about them, and almost nobody has actually watched the numbers move.

So I made a tool where you type sentences and watch them turn into dots that cluster on a canvas. This post is me explaining what a text embedding is using that tool, with the real similarity numbers I measured while building it. No hand-waving, no borrowed diagrams from a Google blog post. Just the values that came out of the playground and what they taught me about how these models actually see language.

What Is a Text Embedding?

A text embedding is a list of numbers that represents the meaning of a piece of text. You feed a sentence into an embedding model and it hands back a vector, an array of floating-point numbers, positioned so that text with similar meaning lands near other similar text. That's it. Meaning becomes coordinates.

The model I use is OpenAI's text-embedding-3-small. Its full output is 1536 numbers per sentence. The playground ships 256 numbers per sentence instead, and I'll get to why that's basically free later. Every dot you see on the canvas is one of those vectors, projected down to 2D so your eyes can handle it.

What Is Cosine Similarity?

Cosine similarity measures how close two embeddings point in the same direction, on a scale from 1 (identical) to -1 (opposite). It ignores how long the vectors are and only cares about angle, which is exactly what you want when comparing meaning. Hover over any dot in the playground and every line from it gets labeled with the exact cosine value.

Here is where my expectations were wrong. I assumed similar sentences would score 0.7 or higher. They don't. With text-embedding-3-small, same-topic sentence pairs average around 0.27. Cross-topic pairs average around 0.15. Nothing gets near 1.0 unless the text is nearly verbatim. The whole scale is squashed down low, and if you don't know that going in, your intuition is off by a factor of three.

The Threshold That Showed Zero Lines

My first version of the tool drew almost no lines at all, and I nearly shipped it broken. I'd set the similarity threshold default to 0.40, picked from the folklore you read everywhere that "0.45 to 0.7 means same topic." Against real text-embedding-3-small values, 0.40 is above almost everything. Sentences that were obviously about the same subject sat there unconnected.

I had to recalibrate the entire tool against measured numbers instead of folklore. The default threshold is now 0.30, and the slider runs from 0.10 to 0.50. That single fix is the difference between a canvas that looks dead and one where clusters actually form as you type. If you ever build on top of embeddings, measure your own similarity distribution before you hardcode a cutoff. The numbers people quote are usually from a different model on a different scale.

Topic Beats Emotion, Badly

The single most surprising number I measured: "She was overjoyed when she heard the news" versus "A crushing sadness settled over her after the loss" scored 0.444, one of the highest similarities in all my testing. Joy and grief, near-opposite emotions, ranked as almost the same thing.

The model sees both sentences as "emotional reaction to a life event" and that shared topic swamps the difference in feeling. I had built a whole EMOTIONS preset around joy, grief, anger, and calm, expecting four tidy corners. Instead everything clumped into one blob because the model reads them all as the same subject. I cut the preset. The lesson stuck: in embedding space, sentiment is a thin signal and topic dominates. If you're building sentiment analysis, plain embeddings will disappoint you.

Why Motion Verbs Wrecked My Clusters

"The dog chased a squirrel around the park" versus "The motorcycle sped down the empty freeway" scored 0.375, high enough to link a pet to a vehicle. The shared idea of fast motion leaked across two topics that should have stayed apart. My clean two-cluster demo kept merging into one.

The reason it merges is how the tool defines a cluster. A cluster is a connected component of the similarity graph: draw an edge wherever two dots score above the threshold, and any dots joined by a chain of edges become one group. That's single-link clustering, and it's fragile. One stray edge above threshold fuses two clusters that are otherwise clearly separate. So I rewrote every preset sentence: pets doing quiet domestic things, vehicles stuck in maintenance and traffic. The presets in the tool are engineered precisely because a single leaky verb breaks the picture.

One Word, Two Meanings: "Bank"

Type eight sentences using the word "bank," half about money and half about a river, and they start as one undifferentiated blob. The money sentences and the river sentences look similar because they share a word. But raise the threshold slider past about 0.34 and the blob splits cleanly into two clusters, one per meaning.

The margin is razor thin. In my measurements the lowest same-meaning similarity was 0.338 and the highest cross-meaning similarity was 0.324, a gap of just 0.014. That tiny gap is context doing its job: the surrounding words in each sentence pull "bank" toward the right meaning even though the word itself is identical. Watching a single blob resolve into two word senses as you drag one slider is the clearest demo I have of what these models actually understand. This is the 'BANK' preset in the playground, and it's worth thirty seconds of your time.

Why 256 Numbers Instead of 1536

The tool truncates each vector from 1536 numbers down to 256 and loses essentially nothing. That works because of Matryoshka representation learning, a training trick (named after the nesting dolls) where the model is taught to pack the most important information into the earliest numbers of the vector, so you can chop off the tail, renormalize, and keep most of the quality.

I tested it on the same 12 sentences at three sizes. Same-topic average cosine came out to 0.272 at 256 dimensions, 0.280 at 512, and 0.260 at the full 1536. Cross-topic was 0.147, 0.128, and 0.112. The separation between topics is basically identical across all three. So the tool ships 256-dimension vectors: a 6x smaller payload for the same picture. A 12-sentence batch at 256 dimensions costs a fraction of a cent. The whole thing runs on pocket change.

The 2D Map Lies a Little

The dot positions on the canvas are an approximation, and you should trust the line labels over the pixel distances. The map is a PCA projection, principal component analysis, which finds the two directions that spread the data out the most and flattens everything onto them. I compute it with the Gram-matrix trick and power iteration, no external libraries.

PCA is deterministic, but its axes have arbitrary sign and rotate whenever you add a point, so early on the whole layout would mirror-flip every time you typed a new sentence. Deeply disorienting. I fixed it with a 2D orthogonal Procrustes alignment, which rotates each new layout to best match the previous one so the map stays stable as you add dots. Even so, squashing 256 dimensions into 2 throws away information. The cosine values on the lines are the true measurement. The positions are just a helpful cartoon.

What This Powers: Search and RAG

Embeddings are what let you search by meaning instead of keywords, and this site's own chat runs on exactly the stack in the playground. RAG, retrieval-augmented generation, means embedding your documents ahead of time, embedding the user's question at query time, and pulling back the chunks whose vectors are closest by cosine similarity before an LLM answers.

The terminal chat on this site does precisely that. A build script chunks my content, embeds each chunk with text-embedding-3-small, and writes the vectors to a JSON file committed in the repo. When you ask the terminal a question, it embeds your query and grabs the top-K closest chunks. The Embedding Playground is literally that search engine with a canvas bolted on top. If you want the bigger picture of building software this way, I wrote up architecture in the age of AI agents and how I go about structuring projects for AI agents, and the complete Claude Code guide covers the tooling I use to ship it.

Go open the playground, load the BANK preset, and drag the threshold from 0.30 up past 0.34. Watch one blob become two meanings. Then try to guess, before you hover, whether your own two sentences will score above or below 0.27. You'll be wrong more often than you expect, and being wrong against a real number is the fastest way I know to actually understand this stuff.