Embeddings: how they work, APIs, and enterprise use cases

Users frequently cannot locate information when their search queries do not precisely match the titles of your internal documents. For example, searching for "database backup" may yield no results if the pertinent page is titled "recovery procedures." Likewise, a chatbot might be unable to assist a customer when the question differs slightly from a pre-written script. These problems arise because the systems compare exact text strings instead of the underlying meaning. Embeddings address this issue by enabling systems to grasp intent. They constitute the core technology behind most enterprise AI features.

Embeddings consist of numeric lists that capture the meaning of content like sentences, images, or product descriptions. This format lets related items be placed close to one another in a mathematical space. A model generates these values, and systems subsequently compare the vectors to allow search tools and chatbots to locate matches based on intent rather than specific keywords.

An embedding transforms content like a sentence, a paragraph, a product description, or an image into a numeric list that represents its meaning. Content sharing similar meanings yields similar numbers that are placed near each other in a mathematical space. For example, a winter coat and a thermal jacket will be positioned close together even if they do not contain the same words. Computers employ these numbers to compare, rank, and retrieve information based on concepts rather than specific definitions.

Shipshape Data constructs data and AI systems. Many of our client projects depend on embeddings for tasks such as intent-based search, chatbots that use internal data, and recommendation engines; the basic concept is simple. The challenge is transitioning from a demonstration to a production system. This guide describes how the mathematics operate, how to choose a model, and when to employ a managed API rather than self-hosting. It also addresses common errors that can reduce the performance of your results.

Why embeddings matter for enterprise AI

The majority of organisations sit on a massive amount of unstructured data that conventional systems cannot truly access. Documents, tickets, wikis, email threads, and CRM notes. People query the entire collection using keyword tools, but those tools overlook any content phrased differently from the query, causing employees to waste hours searching and customers to abandon the effort. Embeddings shift the retrieval model from word matching to meaning matching, converting a static archive into a resource that can be queried in the way people think.

The real cost of keyword search

Keyword systems fail in a manner that is both predictable and expensive. A user may search for "database backup solutions" whereas the documentation employs the phrase "data protection strategies", which stops the search engine from linking the two. A support agent frequently repeats the same answer because the knowledge base does not display the relevant article. Each of these individual cases is not significant on its own. Nevertheless, they happen thousands of times each month and diminish the value of the information you have already created.

What you get once meaning is searchable

Applying embeddings alters how the same content operates. A support chatbot can deliver answers drawn from hundreds of documents, even when a customer employs unexpected phrasing. Internal search tools can locate information in emails, chats, and project documents irrespective of the specific terminology different teams use. You can also spot duplicate tickets, classify issues by type, and detect emerging problems before they raise the workload.

Keyword search finds documents that contain particular words. Embedding search locates documents that supply answers to a question.

The mechanics of embeddings

You do not need to understand the mathematics to use embeddings effectively, but having a functional mental model helps you avoid mistakes. A model processes your content and generates an array of floating point numbers. This array might look like [0.23, -0.45, 0.78,...] and typically contains between several hundred and a few thousand entries. Individual numbers do not correspond to specific concepts that humans can name. There is no specific slot for "mentions databases" or "is about speed". The model developed these representations by analysing billions of examples during its training phase. It identified patterns that represent meaning in a format that can be measured and compared, even though the data cannot be read directly by people.

Dimensions and the reasons they increase costs

The length of the array is its dimensionality. Most models sit between 384 and 1,536 dimensions. More dimensions let a model draw finer distinctions, at the price of more storage and compute. A 768 dimensional embedding for one paragraph takes roughly 3KB, which sounds trivial until you embed a few million documents, often several versions of each as you refine the pipeline. Each piece of content becomes a point in that space, and how close two points sit tells you how related they are. Picture cities plotted by latitude, longitude and population: places near each other cluster together, and embeddings do the same with hundreds of dimensions, so support tickets sort themselves by issue type without anyone labelling them.

Measuring similarity and the origins of meaning

You find related content by measuring the distance between two vectors. Cosine similarity is the standard choice for text. This method treats each embedding as a vector from the origin and measures the angle between them. The values range from -1 for opposites to 1 for identical items. Most systems consider a score above 0.7 or 0.8 to be a match. Euclidean distance and dot product are other common metrics. Cosine similarity is the best default for text because it ignores the length of the vector and focuses only on its direction.

This process depends on training. Training adjusts the parameters of a model so that related content generates similar vectors. The model learns that terms like "database" and "SQL" belong together by analysing their proximity in millions of documents. This creates a semantic space where mathematical distance corresponds to actual meaning. Your projects rely on this alignment remaining consistent for your specific content. This is the reason why your choice of model is important.

How to choose an embedding model

The model you choose determines how effectively the system processes your content and the amount of infrastructure required for operation. You should define three factors before starting any comparisons. These are the type of content you are embedding, the languages you require, and your available computing budget. These three requirements will eliminate most options before you begin testing.

The size trade-off

Bigger models capture finer shades of meaning and cost more to run. OpenAI's text-embedding-3-large produces 1,536 dimensional vectors with strong accuracy across mixed content at about 6KB each; a small model like gte-tiny returns 384 dimensions at roughly 1.5KB. Across ten million documents that gap is the difference between a comfortable storage footprint and an awkward one, and smaller models are faster too. The right answer is rarely the biggest model; it's the smallest one that still gets your content right.

You can only determine the best model by testing it on your own data. You should select a representative sample of your material and embed it using two or three different models. Run actual similarity searches and verify if the results are relevant to an expert in that field. A model with high scores on public benchmarks may still perform poorly on specialised vocabulary. Internal medical notes, customer support tickets, and product catalogues each place different demands on a model.

General versus domain-specific

General purpose models are trained on vast quantities of internet text and manage most business content effectively. Domain-specific models are trained on specialised data like legal documents, scientific papers, or code. These models work better when the content depends on technical vocabulary or particular relationships that general models have rarely seen. Legal contracts rely on precedent, medical records need precise links between symptoms and treatments, and code search works better with models that grasp syntax. Specialised models cost more to use. You need to decide whether your content needs this specialisation or if a general model satisfies your requirements at a lower cost.

Using embedding APIs and software tools

You create embeddings either through a managed API or by hosting a model yourself. This decision influences both your initial development speed and your long-term operational costs as usage grows.

Managed APIs versus self-hosting

Large providers supply managed services that scale automatically. You post text to a REST endpoint and obtain vectors in milliseconds, without having to manage servers, model versions, or failover plans. These services operate on a pay-per-request model. This method is cheap during development, yet costs can rise sharply at larger volumes. Processing one million embeddings via a managed API usually costs between £10 and £50, depending on the exact model and provider used.

Self-hosting alters the operational structure. You download an open-source model and deploy it on your own servers or a Kubernetes cluster. You handle the maintenance to achieve predictable costs and full control over your data. Processing one million embeddings may cost roughly £5 in compute fees on existing hardware. Self-hosting is useful for processing high volumes, satisfying strict data residency requirements, or modifying the model. Most teams we advise begin with a managed API. They switch to their own hosting when their usage volume makes the financial benefits evident.

Build an abstraction layer early

Regardless of how you begin, place a thin integration layer between your application and the embedding provider; it should manage authentication, rate limiting, retries and errors centrally, so that changing models or providers later does not require rewriting the system. We add this layer from day one on client builds, because a request to swap a model always arrives.

Batch the backlog, embed queries live

Process your existing content in batches during off-peak hours to control costs and system load. Group documents into sets of 10 to 100 and send each set in a single request. This method cuts network overhead and lets the provider keep efficient throughput.

Create embeddings only for items that need immediate processing, like a search query or a newly uploaded document. Precompute most of the data and embed the remaining items when they are needed. This method preserves high performance for the end user.

Architectures for search and RAG

Embeddings on their own don't do anything.

You arrange these components into a pipeline that turns a query into a result. The pipeline's structure determines the speed, accuracy, and cost of the process. Most enterprise applications use one of two patterns. The first pattern is semantic search, which finds documents most similar to a query. The second pattern is retrieval-augmented generation, or RAG. This method adds a step where those documents are fed to a language model to generate a grounded response.

Semantic search, end to end

A search request starts when a user types a query. You generate an embedding for that query with the same model used for your documents. Then you compare that vector to your stored embeddings to locate the nearest matches. Usually you return the top 10 to 50 results, ordered by their distance. This process can be improved by adding filters and hybrid matching. Metadata filters limit the search by document type, date, or department. Hybrid search combines vector similarity with keyword matching. This lets you find both the conceptual meaning and exact terms, which is helpful when a user looks for a specific product code.

RAG, and the traps in it

RAG links a language model to the search process. When the vector search returns relevant documents, you build a prompt that contains the user's question and excerpts from the top results. The model then generates an answer based on your organisation's actual information rather than the data it learned during its initial training.

The context window is a limitation. Each model accepts a specific amount of input, typically between 4,000 and 128,000 tokens, meaning you cannot include every retrieved document. Split long documents into passages of 200 to 500 words when embedding them. This lets you retrieve two or three relevant sections during a query without exceeding the limit. Record which documents supplied the information for each answer so a user can verify the source.

RAG bases an AI's answers on your specific data. This process reduces incorrect information while preserving the conversational responses that make the technology useful.

Scaling and caching

A production system needs to keep response times consistent when hundreds of users operate it at the same time. Caching serves as the main technique to achieve this. Frequently used embeddings ought to be kept in memory so they do not have to be recomputed. Caching should also be applied at the query level since many users pose similar variations of the same questions. Each cache hit eliminates an unnecessary API call.

High-value enterprise use cases

The greatest value comes from using embeddings on tasks where grasping intent works better than matching keywords. This covers cases where employees waste too much time searching for information, customers cannot locate answers, or staff members manually categorise data. These areas yield measurable improvements. Three particular use cases appear most often.

Searching for documents and managing knowledge

Employees devote considerable time to searching internal documents, wikis, and shared drives. Existing search tools frequently fail when the user does not know the exact terminology used in a file. For instance, an engineer could miss a code example because another team used a different name for a pattern. A salesperson may overlook a particular detail that resides inside a CRM note. Embeddings address these problems by detecting the intent behind a query. A search for "database recovery" will correctly find the page titled "backup restoration procedures". Deploying semantic search across a knowledge base shortens resolution times. New employees become productive more quickly since they can use natural language rather than learning specific internal jargon. These embeddings also detect duplicate documents and highlight topics where users search for information that has not yet been documented.

Support automation and chatbots

Traditional chatbots frequently frustrate users since they only recognise specific phrases and predefined intents. If a user asks "why isn't my order updating" while the system was programmed for "track my order", the bot will not be able to answer. Embeddings enable a chatbot to handle varied wordings and synonyms by comparing the user's query to support articles, FAQs, and prior resolutions. The bot clears the routine questions and forwards complex issues to a human agent. This agent gets the relevant context right away, allowing staff to concentrate on difficult problems. You can also track which questions the bot answers successfully and which it refers to humans. This data gives a clear list of areas where your documentation needs improvement.

Product discovery and recommendations

Customers are unable to locate products when their search terms do not align with your catalogue. Someone searching for a "warm winter coat" will not discover your "thermal outerwear" through a standard keyword search. Embeddings address this issue by recognising that both phrases refer to the same item. Recommendations get better for the same reason. You can recommend items that are similar using descriptions, reviews, and attributes instead of depending solely on category tags. This system keeps working as your inventory changes since it does not need manual re-tagging.

Design choices, pitfalls and best practices

Most embedding failures happen during the design phase rather than in later stages of development.

Teams frequently start development before choosing a document chunking strategy or a particular model for their content. These two choices affect system accuracy more than the selection of a vector database or a similarity threshold. If you choose incorrectly at this stage, the system will operate technically yet deliver unreliable results. Such an outcome represents a costly failure.

Errors that reduce accuracy levels

Embedding whole documents is a mistake. If you convert a 50-page manual into one vector, the specific information on page 37 that a user needs will be lost. Documents should be split into logical chunks of 200 to 500 words that each hold a complete idea. A product description can function as a single chunk. A support article is usually divided at its subheadings. Code works better when divided by function instead of by a fixed line count.

Avoid mixing different content types within a single collection. Customer emails and product specifications need distinct retrieval strategies even when they address the same topics. Emails are conversational, whereas specifications are terse. Keep these types in separate collections so each can be tuned individually. Both mistakes stem from the same principle. Design your chunking strategy according to how people will search for information rather than selecting the easiest programming method.

Data quality and preprocessing

Embeddings are only as good as the input you provide. Remove boilerplate, navigation, and repeated headers from each document before embedding, as they consume capacity representing noise rather than content. Clean HTML artefacts, normalise whitespace, and correct obvious encoding errors; the model cannot distinguish genuine content from formatting cruft by itself. Metadata also matters: tag each chunk with its source, date, author, and category, enabling you to filter internal documentation from customer-facing content even when a query matches both.

Testing and validation

Assess an embedding system by how closely its results match human judgement. Avoid focusing on mathematical scores that fail to reflect user needs. Create a test set containing 50 to 100 realistic queries. Ask real users to mark the correct answers for those queries. Process them through the system and compute how often the correct information appears within the top 5 or top 10 results. This hit rate offers more useful data than abstract similarity metrics. Further test the system using misspellings, abbreviations, and department-specific terms. For example, engineering staff might search for "CI/CD pipeline failures" while operations staff ask about "deployment problems". Both queries should return the same information. Identify queries that fail to produce results and adjust the system based on those findings.

Where to start

Embedding performance relies on three factors. Select a model that fits your specific content, split text into logical segments, and evaluate results using actual user queries instead of theoretical metrics. Avoid trying to embed all of your organisation's data in a single stage.

Choose one specific problem that offers high value and has a measurable metric. The metric might be resolution time, deflection rate, or conversion. Demonstrate the value of your solution by running a pilot programme alongside your current system rather than replacing it. This approach lets you make an accurate comparison between the two systems.

If you prefer to map your data and design the pipeline before selecting your tools, we can help. Talk to us to develop a plan for converting your unstructured data into a format reliable for AI.

Are you thinking about building this for your business? Our AI developers will describe the requirements, including if your data is enough to support the project.

Frequently asked questions