If you ask a large language model the same question in two different ways, you may receive two different results. One response might be generic, while the other provides the specific information you require. This difference is not a matter of chance. It depends on the structure of the prompt. When you understand these mechanics, you can consistently produce the results you need.
To follow prompt engineering best practices, you should treat a prompt like a project brief. Use retrieval to ground the prompt in your own data, state your requirements and quality standards clearly, and always place the instructions before the data. You can then add few-shot examples, chained prompts, and self-check steps. You must test every version of a prompt before you deploy it.
Our work at Shipshape Data involves helping companies transition AI from a demonstration into a reliable daily system. We frequently observe a specific pattern where teams attribute errors to the model itself, while the actual issue is located within the prompt.
A model's performance depends on the instructions and context you provide.
The following thirteen techniques are organised into seven habits. These methods distinguish AI features that users do not trust from features that perform their tasks reliably and consistently.
1. Use your own data to ground prompts
Generic knowledge produces generic answers. Retrieval Augmented Generation, usually shortened to RAG, changes that by pulling relevant documents, records or context from your own knowledge base and handing them to the model alongside your instruction. Asking an LLM "what's our refund policy?" and hoping its training data happens to line up with your business is a gamble. Retrieve the current policy document first, then ask the model to answer based on that text and nothing else.
The prompt consists of two parts. These are the retrieved context and the instruction for how to use that context.
This process is necessary because models cannot access information that was not included in their training data. That data begins to lose relevance as soon as the training process ends. You can solve both issues by providing current and specific information within the prompt at the time of use. This practice also prevents the model from generating false information about your business. The model will use the provided material instead of guessing to fill gaps in its knowledge.
Getting it right
Chunk documents sensibly before they go into a vector database. Paragraphs or short sections retrieve better than whole documents or single sentences, in our experience by a wide margin, because a whole document dilutes the exact passage the model needs. Test a couple of embedding models against your actual domain language before settling on one; the differences between them are bigger than most teams expect, especially once you get into contract language, technical manuals or anything with heavy jargon. A handful of things separate a RAG setup that works from one that quietly returns the wrong document half the time:
- Divide the text into sections based on paragraphs or logical topics. Do not process the entire document at once or break it down into individual sentences.
- Test an embedding model against the language used in your specific domain instead of relying only on the vendor's benchmark.
- Metadata (title, date, section, owner) carried alongside every retrieved chunk
- Hybrid retrieval combines keyword search with semantic search. This method ensures that exact terms and product codes continue to appear in results.
- A log of the specific documents used to generate each response, for use in regulated or auditable environments.
If you skip any of these steps, the model will provide fluent and confident answers based on the wrong source material. This outcome is more problematic than the model providing no answer. When an auditor asks for the origin of a specific figure, you must be able to provide a direct answer instead of an uncertain response.
2. State your requirements clearly
Vague prompts get vague results. Three things fix most of it: state the task, name the audience, and define what a good answer actually looks like. Tell the model whether you want a summary, an analysis, a translation or a rewrite. Say who will read it, because a summary written for a board member and one written for an engineer shouldn't read the same way. Then say what "good" means for this particular job: short, exhaustive, a specific format, whatever genuinely matters here.
A prompt written this way might look like this: summarise this quarterly report into three points, for board members with limited technical background, each point one sentence, focused on business impact, no jargon. This version is only slightly longer than the vague prompt, "summarise this report", but it is more useful. The model now has a specific set of instructions to follow.
Place instructions before the data
Order matters more than most people assume.
If you place your actual request inside many pages of context, the model may lose track of your instructions. You should start with the instruction and then provide the material. Use a clear delimiter, such as triple quotes or a labelled section, so the model does not confuse your data with your command. Use the same discipline you would apply when briefing a new analyst. Tell them the task first and then give them the spreadsheet.
Spell out the format
If you do not define the output format, the model will provide a long block of text that may not be useful. If you require JSON for an API, you must state that and describe the necessary fields. If you want a list with a maximum of five items, you should include that instruction. Length constraints are as important as structural requirements. Using a prompt like "maximum three complaints, severity as a number from one to five" prevents the model from adding unnecessary information to appear more thorough. You should place format instructions early in the prompt, immediately after the task. For complex requirements, you should provide an example.
Create a collection of prompt templates for your organisation's recurring tasks. These templates should include pre-filled sections for the task, audience, success criteria, and format to ensure staff do not begin with an empty prompt box. Test every template with edge cases before using them in production environments. Use empty inputs, multiple items, or requests that do not fit the standard pattern, as these scenarios often cause poorly specified prompts to fail. If the output is sent to a database or an API, use automatic validation against your schema.
3. Provide examples to guide the model
Some formats are difficult to explain but simple to demonstrate. In these situations, you should provide two or three examples of the output rather than describing it. This method is called few-shot prompting. It teaches a pattern more effectively than instructions alone.
If you need customer feedback tagged by sentiment, issue and positive point, you may supply examples rather than drafting an extensive paragraph of rules. For example, you can demonstrate that "shipping took forever but the product is excellent" ought to be labelled as mixed sentiment, slow delivery issue, and high quality positive. Adding a second example that uses a different feedback structure teaches the model to generalise the pattern rather than merely memorising a single case. This approach keeps your prompt shorter than a full explanation and yields more consistent data across thousands of runs.
Clearly label your examples and insert them between the instruction and the actual input. Slightly vary these examples to aid the model in learning how to adapt the pattern. Typically, two examples are sufficient for this purpose. You should test whether a third example improves your results before deciding to add more. Adding more than three examples rarely boosts performance, and each extra example raises the cost of every call.
Adopt a practice from software development teams by rotating your example sets and re-running your evaluation against the new versions. Although this process may seem redundant, it helps identify when a model has started to over-rely on a specific example. This issue occurs more often than the promotional materials for these tools indicate.
4. Divide large tasks into a chain
If you request a model to research, analyse, summarise, and format in one instruction, the output quality declines at each stage. A long prompt containing multiple objectives splits the model's attention. This resembles giving a junior colleague several unrelated tasks without a clear order. Better results are achieved by breaking the work into a sequence of smaller prompts where the output of one task feeds into the next. This method also lets you intervene if an error occurs during the process.
Define the distinct steps in your process: get the information, sort it into groups, create a shorter version, arrange it neatly, or perform whatever the task requires. Assign a specific goal to each step and employ a separate prompt for that step. Save the results of each step so you can restart if something goes wrong. This also simplifies error detection: if the final result is incorrect, you can identify which step produced the flawed information.
We build many of these chains for clients moving from a single demonstration prompt to a system that must operate reliably on a daily basis.
The chain is usually as simple as the prompt it replaces.
The system is arranged to allow testing and fixing each piece individually. This structure matters when real users depend on the application. Document the chain as you would any pipeline by recording the inputs, outputs, and failure protocols for each step. Cache intermediate results when possible to increase speed. Caching also avoids the unnecessary expense of tokens and time that occurs when you must re-run a five-step chain to test a change in the fourth step.
5. Establish honesty as a standard
A model that gives a false answer with confidence is less useful than one that states it does not have the information. You can prevent this behaviour by applying two methods. First, let the model ask for more information when it is uncertain. Second, give specific instructions about how the model should respond.
Let it say it doesn't know
When the input is ambiguous, or the source material lacks the answer, tell the model explicitly to say so, or to ask a clarifying question. For a customer-facing tool this can be as simple as an instruction such as: if the manual doesn't cover this, say so and ask the customer to rephrase or add detail, and never speculate beyond what is stated. Test this with deliberately awkward or out-of-scope questions before you ship anything, and build tiered fallbacks: ask again, hand off to a person, answer with a caveat.
A model that admits it does not know the answer builds more trust than one that provides a convincing guess.
Tell it what to do, not what to avoid
Negative instructions have an odd side effect.
If you write "don't mention pricing", the model focuses on pricing before trying to suppress it. This process often raises the chance that the restricted information will appear. Rewrite instructions to describe the desired outcome instead. For example, replace "don't be verbose" with "answer in two sentences". Change "avoid technical jargon" to "use plain language for a general reader". Positive instructions give the model a specific goal. This approach is more reliable than negative constraints when the model encounters unusual cases.
6. Adjust the settings
Temperature and top_p
Two parameters control the predictability or variety of the output. Most teams do not adjust these settings, even though they are available in the API call. Use the following guidelines as a starting point for setting temperature:
- Use a temperature of 0 to 0.3 for data extraction, classification, and formatting. These tasks require consistency rather than creative variety.
- 0.4 to 0.6 for general question answering and everyday assistant tasks
- Use a setting of 0.7 to 1.0 for brainstorming, first drafts, and any task where variation between results is the goal rather than a problem.
Top_p works together with temperature. Lower values restrict the model to the most probable words, while higher values permit less common selections. Adjust the temperature first and observe the results before changing top_p. If you change both settings simultaneously, it will be difficult to determine which one affected the output. This approach prevents you from having to guess.
Keep context from overflowing
Every model has a maximum limit for the amount of text it can process in a single request. Long conversations or large documents will eventually hit this limit. You should compress the information rather than trying to include everything. Summarise earlier parts of the conversation into a shorter version that retains the main points, then discard the original transcript. For long documents, split the text into sections and keep a running summary as you progress through each part. Begin this process manually when a conversation reaches 70 to 80 per cent of the total limit.
7. Manage prompts as production code
Add a self-check step
Models usually do not evaluate their own output unless you instruct them to. Ask for a draft first. Then, tell the model to check that draft against your specific criteria for accuracy, completeness, and formatting before providing a final response. This process identifies mistakes that a single response would otherwise present as certain. Test this review step against known incorrect answers to ensure it identifies errors correctly.
Defend against prompt injection
Design prompts that treat all user input as data. State this rule in the prompt by specifying that text within the delimiters is content for analysis. Instruct the model to keep the system prompt private. This rule applies regardless of how a user phrases a request. Use this approach when a prompt processes input from external sources. Most production environments require this. Test the prompt with the same methods an attacker uses. Use inputs that include phrases like "ignore previous instructions" or requests for the model to repeat its configuration. Resolve any vulnerabilities before an external party identifies them.
Version and monitor what you ship
A prompt that works correctly now can fail after a model update or as additional edge cases appear over time. If you do not monitor these changes, you might only discover a failure via a customer complaint. You should version each prompt change as you would code and keep a set of evaluation examples to test new versions before deployment. Monitor output quality in production as well as tracking uptime. Maintain a rollback process so you can correct a poor prompt change within five minutes rather than handling an emergency.
Apply the same quality standards to your release process that you apply to application code. A new prompt version should not be deployed until it passes the evaluation set. Any regression against the established golden examples must automatically block the release. Maintain a brief changelog that explains why each version changed and what it improved. This is needed because details are often forgotten after several months. The next person who modifies the prompt needs a clear history of prior changes.
Where to start
Thirteen techniques is a lot to absorb at once, so begin with the two that address the most common failures. Ground your prompts in your own data if the answers must reflect anything that changes. Place your instructions before your data, always, on every prompt you write from today onward. Everything else builds from there: better formats, tighter chains, honest fallbacks, sensible settings, and the discipline to test and monitor what you ship.
AI value usually depends on how you treat the prompt rather than on the model's quality. You should treat prompt engineering as a standard component of your production logic. If you need a review of how your team develops or deploys AI features, talk to us. We will identify the specific gaps in your process.
From concept to production in weeks, not quarters: our bespoke AI development puts articles like this to work on your data.
Frequently asked questions
What is prompt engineering?
Prompt engineering is the process of designing inputs for a large language model to ensure the output meets specific requirements on the first attempt. This article explains thirteen techniques, organised into seven habits, that distinguish unreliable AI features from those that function consistently.
How do you ground prompts in your own data using RAG?
Retrieval augmented generation identifies relevant documents or records within your knowledge base and supplies them to the model with the prompt. This allows the model to use current information in its response. To implement this effectively, you should split documents into paragraphs and verify that the embedding model understands your specific industry terminology. You must also keep logs that show which documents were used to generate each specific answer.
How much detail should you include in a prompt to get the result you want?
Explain the specific task, identify the target audience, and describe the requirements for a successful response. A general prompt such as "summarise this report" will lead to an imprecise outcome. Place the instructions before the data and use a distinct delimiter to separate them. Define the required output format clearly to give the model a specific set of guidelines.
When does showing examples work better than describing the output?
Few-shot prompting is most effective when a format is difficult to describe but easy to show. You should provide two or three examples of the pattern between inputs and outputs. Two examples are typically sufficient. You should test whether a third example improves the results before assuming that adding more examples will be beneficial.
How do temperature and top_p affect LLM output?
Temperature settings determine the predictability of the output. Low settings between 0 and 0.3 are appropriate for data extraction and classification tasks that require consistency. Settings between 0.7 and 1.0 are better for brainstorming and creative tasks where variety is necessary. Top_p functions with temperature to limit or expand the vocabulary the model uses. This article suggests that you adjust the temperature first and then adjust top_p. This sequence allows you to identify which specific change affected the results.