A support bot tells a customer that the returns window is sixty days, although the actual policy is thirty days. No documentation accessible to the model contains a sixty-day rule. The bot produced a plausible figure by referencing similar policies it saw during its training phase. It presented this figure with the same confidence it applies to any statement, and the refund is processed. Finance departments spot the discrepancy three weeks later when accounts fail to reconcile, prompting an employee to investigate the source of the sixty-day figure.
Large language models are systems that are trained to predict the next segment of a text sequence. When scaled sufficiently, this capability appears similar to understanding and reasoning. These models have processed a huge body of text, yet they lack information about your particular business unless you supply it in the prompt. They deliver answers confidently whether the information is accurate or not.
A large language model (LLM) is a software program that is trained to predict the next word in a sequence. This training is performed at a scale that makes the software seem to understand, reason, and converse. The model has processed an extensive amount of written text. It has not read your particular returns policy unless you feed that text to the model directly. The software will not tell you when it lacks this information. It will still produce an answer because its main function is to generate a response.
Shipshape Data creates data and AI systems. We often see businesses that have bought or built tools powered by large language models without grasping their specific capabilities or operational costs. Companies frequently adopt this technology before they comprehend how it works, which usually results in problems. This guide describes the internal mechanics of these models. It discusses how tokens and context windows restrict the amount of information a model can handle at once. It also outlines the training process, the strengths and weaknesses of these systems, and the costs of operating them. Finally, it gives guidance on how to choose and assess a model for particular business requirements.
What happens when you enter a prompt
An LLM is a large statistical function that lacks its interface. When you supply a sequence of text, the model computes a probability for each possible next token. It chooses one token, appends it to the sequence, and repeats the process. After a few hundred iterations, the model generates a paragraph that looks like human writing. The system does not look up information or retrieve a stored fact such as "your returns policy". Instead, it relies on a probability distribution shaped by its training data to pick the most likely next word given the existing text.
It is neither a database nor a search engine
It is the most practical concept for a business reader to grasp. A database supplies the specific data that was stored within it. A search engine returns documents that currently exist. An LLM yields results based on the statistical probability derived from its training data. These results are frequently accurate, though they can also be incorrect. The model lacks an internal mechanism to differentiate between these two outcomes. It keeps the same level of fluency whether the information is correct or not. This behaviour is a standard feature of the technology, not a mistake that needs fixing.
The system functions as designed, yet it is being used for a question it was not built to answer accurately.
Why the process resembles reasoning
Small next-word prediction models generate nonsensical text. Modern models are trained on a large share of all written records using billions of parameters. This scale enables the same mechanism to produce text that exhibits planning, keeps logical consistency over long passages, and answers questions beyond its direct training. Experts differ on whether this reflects actual reasoning or a high-quality imitation of how reasoning looks in writing. From a business viewpoint, the practical results matter more than this distinction. The technology works very well on tasks that resemble its training data, yet its performance declines unpredictably when it faces unfamiliar problems.
Tokens: why the model doesn't read words the way you do
Large language models do not process text as individual letters or whole words. They use tokens, which are text segments that can represent a full word, a group of letters, or a single punctuation mark. These segments are defined by a fixed vocabulary set up before the training process starts. A word such as "shipshape" might be processed as a single token or split into three tokens, depending on whether that exact string is in the model's vocabulary. Common English words usually count as one token. Rare terms, invented names, non-English scripts, and words with non-standard spelling are typically broken into multiple tokens.
Where this bites
Tokenisation is a process that users often overlook until it leads to errors. Large language models frequently fail to count the letters in a word when that word has not been tokenised as a single unit. This occurs because the model processes a sequence of token identifiers instead of individual letters. This representation does not retain spelling information the way human vision does. Product codes, serial numbers, and strings with many digits or punctuation marks are often tokenised inconsistently. This inconsistency explains why models are often unreliable for exact-match tasks, like copying a long reference number without errors. If your project needs precise strings, you should test this behaviour before relying on the generated output.
Tokens matter because API calls are priced and limited according to the total number of input and output tokens. A long prompt containing extensive system instructions or large documents incurs costs before the model produces a response.
Context windows: the memory that quietly runs out
The context window is the total number of tokens a model can handle in a single request, covering both the prompt and the response. When a conversation or document exceeds this limit, the model drops the oldest content or omits it. The model does not issue a warning like "I've forgotten the first half of this document." Instead, it produces a fluent response based solely on the remaining text. This information loss typically becomes evident only when the model gives an answer that conflicts with a detail mentioned earlier in the session.
Large context windows involve significant costs
Current models feature context windows that can hold hundreds of thousands of tokens. Although this capacity is helpful, it does not solve the fundamental issue that researchers label "lost in the middle". Models work better with information placed at the beginning or the end of a long context. They are less effective with information situated in the middle. If you insert an entire knowledge base into a prompt, the model will probably prioritise the data at the edges of that text. A large context window is useful, yet it does not replace the need to give a model a specific set of facts. That is the primary purpose of retrieval-augmented generation. That process finds a small set of relevant passages so the model does not need to search through a large volume of data.
A model with a large context window will still provide confident answers if it contains the wrong document. Increasing the window size does not guarantee that the information is relevant.
Where the behaviour comes from
Training occurs in several phases. Each phase influences a specific aspect of the final model that you interact with.
Pretraining
The first stage, and the most costly, is pretraining. The model processes a very large amount of text-primarily from the public web, books, and code-and adjusts billions of internal parameters. This adjustment is done to improve the model's ability to predict the next token in a sequence, repeating this process throughout a training run. The largest models can require tens of millions of pounds in computing resources for this stage. During pretraining, the model learns grammar, facts, style, and a general understanding of how the world is represented in writing. It also learns any biases, inaccuracies, or errors present in the text it processes, as the model does not independently verify the truthfulness of information.
It absorbed both.
Fine-tuning and human feedback
A raw pretrained model does not naturally answer questions or refuse harmful requests because its primary function is to complete text. The process that turns it into an assistant is fine-tuning. This is typically followed by reinforcement learning from human feedback, where people rank the model's responses so the model can be adjusted towards the preferred options. This process creates "helpful, harmless assistant" behaviour. It also establishes the specific personality and guardrails of a vendor. This explains why the same underlying capability can feel different depending on the provider.
The knowledge cutoff
Pretraining involves collecting text up to a specific date. Every model has a knowledge cutoff, which means it lacks information about events after that point unless you include those details in your prompt. A model should state that it does not know the answer if you ask about a topic from after its cutoff date. Many models fail to do this and instead create plausible responses about periods they were not trained on. This is a significant failure mode because the phrasing does not indicate that the information is incorrect.
What LLMs are genuinely good at
The primary use cases are language tasks. These include summarising long documents, changing the tone of a text, and drafting initial emails. The technology is also effective for extracting structured data from unorganised text, translating languages, and categorising support tickets. These tasks align with the core function of the system, which is to predict logical and correct language based on a specific input. It performs these actions at a speed and volume that human teams cannot replicate.
These models are useful as flexible interfaces for other systems when they are integrated carefully. A model can convert a plain language question into a structured query or extract specific fields from a document. Building a rules-based parser to do this would take much longer, and that parser would fail if the input format changed slightly. This flexibility is a legitimate and significant advantage.
Teams achieve the most consistent results by limiting the scope of a task. A model is reliable when it is required to summarise a specific document provided to it. When a model answers general questions about the world without source material, it relies only on memory and probability. This issue is the subject of the next section.
Where they still fail
Hallucination
The example of the returns policy at the beginning of this guide illustrates hallucination. This occurs when a model presents false information with the same fluent confidence as true information. The model is not lying, because it does not know the truth and choose to ignore it. It lacks a separate mechanism to distinguish between "I am now guessing" and "I am now certain." Both types of statements are produced in the same manner. You must design your system with this limitation in mind. The most effective way to address this is to ground the model in real retrieved documents instead of requiring it to answer using only its memory.
Arithmetic, exact counting, and anything brittle
Models perform inconsistently at mathematics, especially with multi-step arithmetic involving large numbers. This happens because they predict likely sequences of tokens instead of performing actual calculations. A model might produce a correct or incorrect result based on patterns in its training data rather than mathematical logic. In a production environment, the solution is rarely to "wait for a smarter model" to arrive. Instead, developers should provide the model with a calculator, a code interpreter, or a dedicated backend. The model should be used to determine when a calculation is necessary rather than performing the arithmetic itself.
Consistency and determinism
Large language models can provide different answers to the same question, even when settings are adjusted to minimise randomness. This variability is acceptable for chat assistants, but it creates problems for tasks that require consistent outputs, such as automated decision-making. If a project requires reliable consistency, you must plan for it specifically. You should not assume that a model call will behave like a standard function call.
Cost and latency: what shows up on the invoice
Providers charge for every input and output token, typically at a rate per million tokens. Prices vary significantly, often by a factor of ten or more, between large frontier models and smaller, faster versions. Costs accumulate quickly when you use long system prompts for every call, include entire documents instead of specific paragraphs, or maintain a growing chat history. These factors lead to expenses that far exceed initial estimates. We have observed several cases where a prototype was inexpensive to test, but the cost of production volume became the most unexpected part of the project, surpassing concerns about model performance.
Latency is another constraint that people often underestimate. Larger and more capable models generally take longer to respond. If a process calls a model several times to verify its own output, the total wait time increases accordingly. This approach is practical for maintaining quality, but it impacts speed. A chat interface for customers can function with a delay of a few seconds. A background process that extracts data from ten thousand documents overnight can handle much longer delays. In that situation, using a smaller and faster model is often the correct choice rather than a compromise.
You should develop the habit of choosing a model size that fits the difficulty of your task instead of automatically using the most capable model. Many classification and extraction tasks function correctly at a much lower cost than frontier models. These savings become significant when you move from a pilot project to a full production workload.
Where to start
Begin by defining the task rather than selecting a model. Document the specific actions you want the assistant to perform, the data it may access, and the consequences of an error. The risks associated with a frustrated customer differ from those of an incorrect figure on a regulatory document. Defining these requirements often eliminates many potential models before you evaluate their costs.
Test the model using your own examples rather than the vendor's data. Create a small set of real inputs that include the difficult cases your business handles. Review these outputs manually before you use another model for evaluation. Our guide on how to evaluate large language models explains the metrics and benchmarks to use after this initial manual review. You must complete the manual review first. Benchmark scores do not show how a specific model handles the disorganised invoices of a particular client.
Ground it before you trust it. If the model needs to know something specific to your business, put that thing in front of it at query time rather than hoping training absorbed it, and treat prompt engineering as the cheap, fast lever it is before reaching for anything heavier. Most disappointing pilots we get called in to fix were asking a general-purpose model to know things it was never shown, and then being surprised when it guessed.
You do not need to select the most capable model available. You must understand the limitations of your model's context, the costs associated with your specific usage volume, and the consequences of the model providing incorrect information with high confidence. If you want an assessment of your current operational gaps, talk to us.
Are you determining how this applies to your organisation? Our AI strategy service helps you identify your first profitable use case and the sequence for developing subsequent projects.
Frequently asked questions
What are large language models?
A large language model, or LLM, is software trained to predict the next piece of text in a sequence. When this prediction skill is applied at a large scale, it can appear as understanding, reasoning, and conversation. Unlike a database or a search engine, an LLM returns text based on statistical probability derived from its training. It does not look up or verify information. The model has processed a very large volume of written material, but it does not contain information specific to your business unless you provide it directly.
How do large language models function?
A model processes text in chunks called tokens rather than whole words. These tokens are defined by a fixed vocabulary established before training begins. When provided with a sequence, the model calculates the probability for every potential next token. It selects one token, adds it to the existing sequence, and repeats this process many times to generate a fluent paragraph; training occurs in two main phases. Pretraining uses a large collection of scraped text to teach the model grammar, facts, and style. Fine-tuning with human feedback then adjusts the model to function as a helpful assistant.
What is the difference between large language models and generative AI?
Generative AI is the broader category of systems that produce new content: text, images, audio or video. It's a different job to classifying or predicting a number. A large language model is one specific kind of generative AI, focused on text, built by predicting the next token in a sequence at enormous scale. Image generators work on the same underlying idea of generation but operate over pixels or a compressed representation, not words.
How do small language models differ from large language models?
Large language models use billions of parameters and extensive training data to perform tasks that resemble reasoning and conversation. Smaller models are less capable but operate faster and at a lower cost. Many classification and extraction tasks function effectively without the high expense of a frontier model. When managing high volumes of work, you should select a model size that fits the specific difficulty of the task instead of using the largest model available.
How do large language models function within the field of natural language processing?
Large language models power most current natural language processing work. They can summarise long documents, change the tone of a text, or draft initial emails. They also extract structured data from disorganised text, translate languages, and categorise support tickets. These tasks align with the basic function of the model, which is to predict logical and grammatically correct language based on an input. The models complete this work at a speed and volume that humans cannot achieve. They perform most reliably on specific tasks that use provided source material rather than answering open questions from memory.