Someone searches your internal wiki for "database backup" and gets nothing, because the page they need is filed under "recovery procedures". A customer asks your chatbot a question three words off from the script and hits a dead end. Both failures come from the same limitation: the system is matching strings, not meaning. Embeddings are how you close that gap, and they are the quiet engine under most of the AI features enterprises actually want to ship.
An embedding turns a piece of content, a sentence, a paragraph, a product description, an image, into an array of numbers that captures what it is about. Content that means similar things ends up with similar numbers, sitting close together in a mathematical space, so a winter coat lands near a thermal jacket even though the words do not overlap. Machines can then compare, rank and retrieve by meaning without ever knowing what the words denote.
We build data and AI systems at Shipshape Data, and embeddings sit under a large share of what clients ask for: search that understands intent, chatbots grounded in internal knowledge, recommendations that go beyond category tags. The concept is straightforward. Getting from a demo to something that holds up in production is where the work lives, and that is what this guide covers: how the numbers work, how to pick a model, when to call a managed API versus host your own, and the mistakes that quietly wreck results.
Why embeddings matter for enterprise AI
Most organisations sit on a large pile of unstructured data that ordinary systems cannot really reach. Documents, tickets, wikis, email threads, CRM notes. People search all of it with keyword tools, and keyword tools miss anything phrased differently from the query, so employees burn hours hunting and customers give up. Embeddings change the retrieval model from matching words to matching meaning, which turns a static archive into something you can actually query the way people think.
The real cost of keyword search
Keyword systems break in a specific, expensive way. A user searches for "database backup solutions" while your documentation says "data protection strategies", and the two never meet. A support agent answers the same question for the fortieth time because the knowledge base will not surface the article that already answers it. None of these are dramatic on their own. They just repeat, thousands of times a month, eroding the value of data you already paid for.
What you get once meaning is searchable
Put embeddings underneath and the same content behaves differently. A support chatbot answers using information spread across hundreds of documents, even when the customer phrases things in a way nobody anticipated. An internal search tool pulls the right answer out of email, chat and project docs regardless of which team used which word. And you can spot duplicate tickets, group issues by type, and flag a rising problem before it becomes a queue.
Keyword search tells you which documents contain your words. Embedding search tells you which documents answer your question.
How embeddings actually work
You do not need the maths to use embeddings well, but a working mental model saves you from bad decisions later. A model reads your content and produces an array of floating point numbers, something like [0.23, -0.45, 0.78, ...] with a few hundred to a couple of thousand entries. None of those numbers means anything you can name; there is no slot for "mentions databases" or "is about speed". The model learned these representations by working through billions of examples during training, finding patterns that capture meaning in a form we can measure and compare but cannot read directly.
Dimensions, and why they cost you
The length of the array is its dimensionality. Most models sit between 384 and 1,536 dimensions. More dimensions let a model draw finer distinctions, at the price of more storage and compute. A 768 dimensional embedding for one paragraph takes roughly 3KB, which sounds trivial until you embed a few million documents, often several versions of each as you refine the pipeline. Each piece of content becomes a point in that space, and how close two points sit tells you how related they are. Picture cities plotted by latitude, longitude and population: places near each other cluster together, and embeddings do the same with hundreds of dimensions, so support tickets sort themselves by issue type without anyone labelling them.
Measuring similarity, and where meaning comes from
To find related content you measure the distance between two vectors. The usual choice for text is cosine similarity, which treats each embedding as an arrow from the origin and measures the angle between them. It runs from -1 for opposites to 1 for identical, and most systems treat anything above 0.7 or 0.8 as a match. Euclidean distance and dot product are the other common metrics, but cosine is the sensible default for text because it ignores vector length and cares only about direction.
All of this rests on training, which nudges a model's parameters so that similar content produces similar vectors. A model learns that "database" and "SQL" belong near each other by seeing them in similar contexts across millions of documents. The result is a semantic space where mathematical distance lines up with real meaning, and everything you build depends on that alignment holding for your content, which is exactly why model choice matters.
Choosing the right embedding model
Your model decides how well the whole system understands your content, and how much infrastructure you need to run it. Before you compare anything, get three answers straight: what kind of content you are embedding, which languages matter, and how much compute you can spend. Those three constraints knock out most of the field before you run a single test.
The size trade-off
Bigger models capture finer shades of meaning and cost more to run. OpenAI's text-embedding-3-large produces 1,536 dimensional vectors with strong accuracy across mixed content at about 6KB each; a small model like gte-tiny returns 384 dimensions at roughly 1.5KB. Across ten million documents that gap is the difference between a comfortable storage footprint and an awkward one, and smaller models are faster too. The right answer is rarely the biggest model, it is the smallest one that still gets your content right.
The only way to know which that is, is to test on your own material. Pull a representative sample, embed it with two or three candidate models, run real similarity searches, and check whether the results match what someone who knows the domain would call relevant. A model that tops the public benchmarks can still fall over on your specialist vocabulary, and internal medical notes, support tickets and a product catalogue all stress a model differently.
General versus domain-specific
General purpose models train on a broad sweep of internet text and cope well with most business content. Domain-specific models train on a narrower corpus, legal text, scientific papers, code, and pull ahead when your content leans on technical vocabulary or relationships a general model never saw enough of to learn. Legal contracts turn on precedent, medical records need symptoms and treatments connected correctly, and code search works better with a model that understands syntax. But specialisation adds cost, so be honest about whether your content needs it or whether a good general model clears the bar for less.
Working with embedding APIs and tools
You generate embeddings one of two ways, through a managed API or from a model you host yourself, and the choice shapes how fast you can prototype and what you pay as usage climbs.
Managed APIs versus self-hosting
Managed services from the big providers scale on their own. You send text to a REST endpoint, get vectors back in milliseconds, and never touch a server, a model version or a failover plan. You pay per request, which stays cheap while you are building and can climb sharply at volume: a million embeddings through a managed API tends to land somewhere between £10 and £50 depending on model and provider.
Self-hosting flips that shape. You download an open source model, deploy it on your own servers or a Kubernetes cluster, and take on the upkeep in exchange for predictable cost and full control of your data. The same million embeddings might run closer to £5 in compute on hardware you already have. Self-hosting earns its keep when you process large volumes, face strict data residency rules, or need to modify the model itself. Most teams we work with start on a managed API and grow into their own hosting once volume makes the sums obvious.
Build an abstraction layer early
Whichever way you start, put a thin integration layer between your application and the embedding provider: something that handles authentication, rate limiting, retries and errors in one place, so switching models or providers later does not mean rewriting the system around it. We put this layer in from day one on client builds, because the request to swap a model always comes.
Batch the backlog, embed queries live
Embed your existing content in batches during quiet hours to keep cost and load under control. Group documents into batches of 10 to 100 and send each in a single call rather than one request per item, which cuts network overhead and lets the provider run its throughput properly. Then generate embeddings live only for what needs it in the moment, a search query, a document someone just uploaded. Precompute the bulk, embed the rest on demand, and you stay fast where users feel it.
Architectures for search and RAG
Embeddings on their own do not do anything. You assemble them into a pipeline that turns a query into a result, and the shape of that pipeline decides how fast, how accurate and how expensive the whole thing is. Two patterns cover most of what enterprises build: semantic search, which retrieves the documents most similar to a query, and retrieval-augmented generation, or RAG, which goes a step further and feeds those documents to a language model to write a grounded answer.
Semantic search, end to end
A search request starts when a user types a query. You embed it with the same model you used on your documents, then compare that vector against your stored embeddings and return the closest matches, usually the top 10 to 50, ranked by distance. You improve on that core loop by layering filters and hybrid matching on top. Metadata filters narrow the field by document type, date or department. Hybrid search runs vector similarity alongside keyword matching, so you catch both the meaning and the exact terms, which matters the moment someone searches for a product code.
RAG, and the traps in it
RAG bolts a language model onto the end of that search. After the vector search returns its documents, you build a prompt holding the user's question plus excerpts from the top hits, and the model writes an answer grounded in your organisation's real information rather than whatever it half-remembers from training.
The trap is the context window. Every model accepts a finite amount of input, commonly 4,000 to 128,000 tokens, so you cannot pour every retrieved document in. Chunk long documents into 200 to 500 word passages when you embed them, so at query time you can pull the two or three relevant sections without blowing the limit. Track which documents fed each answer so a user can check the source.
RAG grounds an AI's answers in your own data. It cuts the confident nonsense while keeping the plain-language reply that makes the thing worth using.
Scaling and caching
A production system has to hold up under hundreds of people asking things at once without response times sliding, and caching is most of the answer. Keep frequently used embeddings in memory rather than recomputing them, and cache at the query level too, because a surprising number of users ask minor variations of the same handful of questions. Every cache hit is an API call you did not make.
High-value enterprise use cases
The best return comes from pointing embeddings at problems where understanding meaning clearly beats matching words. People wasting time searching. Customers who cannot find an answer. Work that gets categorised by hand. These are the places where the improvement is measurable. Three come up again and again.
Document search and knowledge management
People lose hours in internal docs, wikis and shared drives, using search that fails whenever they do not already know the right term. An engineer cannot find a code example because another team named the pattern differently. A salesperson misses the one detail buried in a CRM note. Embeddings fix this by reading intent, so a search for "database recovery" still finds the page titled "backup restoration procedures". Roll semantic search across your knowledge base and resolution times fall, and new starters get productive sooner because they can ask in plain language instead of learning the house vocabulary first. The same embeddings also catch duplicate documents and expose the gaps where people keep searching for something that was never written down.
Support automation and chatbots
Old chatbots annoy people because they only recognise exact phrases and preset intents. Ask "why isn't my order updating" when the bot was built for "track my order", and it falls over. Embeddings let a bot handle the variations and synonyms by matching the query against your support articles, FAQs and past resolutions regardless of wording. The bot clears the routine questions and escalates the hard ones to a human who already has the context in hand, so agents spend their time on genuinely difficult problems. And because you can see which questions the bot handles confidently and which it punts, you get a running list of where the documentation needs work.
Product discovery and recommendations
Customers struggle to find products when their words do not match your catalogue. Someone hunting for a "warm winter coat" never reaches your "thermal outerwear" through keyword search, and embeddings close that gap by recognising the two phrases describe the same thing. Recommendations gain the same way: instead of suggesting whatever shares a category tag, you surface items genuinely similar by description, review and attribute, which keeps working as your range changes without anyone re-tagging the catalogue by hand.
Design choices, pitfalls and best practices
Most embedding failures are baked in at design time, not introduced later. Teams rush to build without deciding how they will chunk documents or which model suits their content, and those two calls shape accuracy more than the vector database or the similarity threshold ever will. Get them wrong and you get a system that runs fine and returns results nobody trusts, which is the expensive kind of wrong.
The mistakes that quietly ruin accuracy
The first is embedding whole documents. A 50 page manual squeezed into one vector loses the specific paragraph on page 37 that someone needed. Break documents into logical chunks of 200 to 500 words that each hold a complete idea: a product description works as a single chunk, a support article usually wants splitting at its subheadings, and code splits better by function than by any fixed line count.
The second is mixing content types in one collection. Customer emails and product specifications need different retrieval strategies even when they cover the same subject, because emails are conversational while specs are terse. Keep them in separate collections and tune each on its own terms. The rule under both mistakes is the same: design your chunking strategy around how people will actually search, not around whatever is easiest to code.
Data quality and preprocessing
Embeddings are only as good as what you feed them. Strip out boilerplate, navigation and the headers that repeat across every document before you embed, because those waste capacity representing noise instead of content. Clean up HTML artefacts, normalise whitespace, fix the obvious encoding errors; the model cannot tell genuine content from formatting cruft on its own. Metadata earns its place too: tag every chunk with its source, date, author and category, so you can filter internal documentation apart from customer-facing content even when a query matches both.
Testing and validation
You judge an embedding system by how well its results match human judgement, not by chasing a mathematical score that has nothing to do with what users want. Build a test set of 50 to 100 realistic queries with the right answers marked by people who use the system for real, run them through the pipeline, and count how often the correct information lands in the top 5 or 10. That practical hit rate tells you far more than any abstract similarity number. Then push on the weak points with misspellings, abbreviations and the different vocabularies your departments use: engineering searches for "CI/CD pipeline failures" while operations asks about "deployment problems", and both should surface the same material. Watch which queries miss, and adjust from there.
Where to start
Whether embeddings work in practice comes down to three things: choosing a model that fits your content, chunking sensibly, and testing against real queries instead of tidy metrics. So do not try to embed the whole organisation at once. Pick one narrow, high-value problem with a number attached to it, resolution time, deflection rate, conversion, and prove the value there first, running the pilot next to your existing system rather than in place of it so you can compare honestly.
If you would rather map your data and design the pipeline properly before committing to tooling, that is what we do. Talk to us and start with a clear plan for turning your unstructured data into something AI can trust.