When you open a retailer's app on your phone, it recognises you by name. It displays the shoes you viewed recently and suggests adding items to your cart to qualify for free delivery. If you access the same retailer's website on a laptop shortly after, the site does not recognise you. You see a generic homepage without your previous browsing history or specific offers. The system fails to connect these two sessions to the same user. The business's personalisation technology does not identify that one person is using both devices.
Content personalisation is the method of adjusting the elements a visitor encounters, such as banners, product lists, or prices, using data about their identity and behaviour. This process requires identity resolution to identify the same individual across different devices. It also requires objective testing with a holdout group rather than only selecting a rules engine or model.
Dynamic content optimisation is the method of adjusting the content a visitor views, such as a banner, product list, email subject line, or pricing page, using data about their identity and current behaviour. This approach replaces a single website for all users with a specific arrangement of components for each visitor, which is put together when the page loads. While the term is common in marketing, the process is straightforward. It involves collecting signals, decide a segment or a score, selecting a version of the content, displaying it, and recording the outcome to improve future decisions.
Shipshape Data builds data and AI systems. Dynamic content is an area where we often see a gap between technical goals and actual infrastructure. Teams frequently purchase personalisation platforms with the expectation that the software will make decisions automatically. They then find that these platforms depend entirely on the quality of identity data and event streams. This data is often poorly integrated. This guide explains how to make implementation decisions and how to build the underlying data foundation and identity resolution. It also covers methods for testing and measuring performance, as well as how to determine the boundary between helpful and intrusive content.
The decision-making process in practice
The term "personalised experience" refers to a relatively simple series of decisions made repeatedly: based on what we know about a visitor, which pre-designed version of a content section should we display? This question is asked across a website's main banner, in automated emails, within an application, and when targeting social media advertisements. The decision-making process varies slightly depending on the location, but the core question remains consistent. There are two main approaches to answering this question, and most organisations begin with the one that is less suitable for their scale.
Rules and the limits of their effectiveness
A rules engine consists of a series of if-then statements written by a person. For example, if a visitor clicks a paid search ad for waterproof jackets, the site displays an outdoor hero image. If the value of a shopping cart is within two pounds of the free delivery limit, the site displays a notification about free delivery instead of a discount code. Rules are easy to explain, inexpensive to create, and simple to audit when errors occur. This auditability is significant because mistakes happen and staff will need to explain why the site showed specific advertisements, such as showing nappy adverts to a pregnant woman before she had announced her pregnancy.
The practical reason to use rules is that most businesses lack the traffic required to train a model effectively. A rules engine managed by a person who understands the product will typically perform better than a machine learning model that lacks sufficient data. We have seen clients purchase recommendation engines before they had the volume to support them. These systems often end up showing the "most popular item" to most visitors while presenting the results as personalisation.
Where models provide value
Model-based selection, usually a multi-armed bandit or a small propensity model, picks the variant it predicts will perform best for this specific visitor, not the same variant for everyone in a segment. It needs enough events to learn from and enough variants worth choosing between, and it genuinely helps once both are in place: a large e-commerce catalogue with thousands of daily sessions and dozens of plausible content variants is exactly the situation rules can't keep up with by hand. Below that scale it's overhead dressed as sophistication.
The data foundation dynamic content needs
Dynamic decisions rely on three types of data. These types function in different ways. Teams that treat them as a single category often create systems that are prone to failure.
Profile, behaviour, and context
Profile data is who the visitor is as far as you can tell: account details, declared preferences, loyalty tier, past purchases. It changes slowly and is usually the most reliable of the three, when it exists at all, because most visitors are anonymous most of the time.
Behavioural data consists of actions users have taken. This includes pages they viewed, items they added to a basket and then abandoned, and search terms they typed and deleted. This data is plentiful and contains a lot of irrelevant information. Its relevance also decreases quickly. A person who looked at hiking boots in March has different needs by July. If a system does not account for this change, it will continue to suggest boots to a person who already purchased them months ago.
Context is everything true about this specific moment: device, time of day, location, referring channel, current stock levels, even weather in some retail setups. Context is what stops a system recommending an item that is out of stock, or pushing a summer sale to someone browsing at 2am in a country where it's currently winter. It's the cheapest of the three to get right and the one most often ignored.
Freshness beats volume
A customer data platform with old information is less useful than a smaller platform with current data. Old data leads to incorrect personalisation, while limited data remains neutral. To fix this, you must decide how long each piece of data stays valid. A browsing session is relevant for a few hours, while a purchase remains relevant for several months. If you treat these signals the same way, you will recommend old products to customers who are no longer interested in them.
Identity resolution: knowing it's the same person
These methods are only effective if the system can identify that the person using the phone this morning is the same person using the laptop tonight.
Vendors do not discuss this part during their sales presentations. However, this element determines if the entire system will function correctly.
Deterministic and probabilistic matching
Deterministic matching connects identities using exact data points. These include login credentials, loyalty programme numbers, or email addresses provided during checkout. This method is reliable and verifiable. However, it only applies to the small portion of visitors who have signed in or completed a transaction. On most websites, these users represent a minority of total traffic.
Probabilistic matching combines cookies, device fingerprints, IP ranges, and timing patterns to estimate if two anonymous sessions belong to one person. This method increases data coverage, but its effectiveness is decreasing. Third-party cookies are being removed from browsers and device fingerprinting faces new regulations. Each year, less visitor behaviour can be legally or technically tracked across different sessions. If you build an identity strategy based only on current signals, it will lose its accuracy over eighteen months rather than failing all at once.
Channels do not usually use the same key
Cookies identify browsers, while logged-in sessions identify accounts. Mobile apps identify devices, and email opens identify addresses. These identifiers are not naturally the same. Most organisations use four or five different systems that each claim to have the primary view of a customer. These systems often do not communicate with each other. Connecting these data points is a continuous process rather than a single project. The links break whenever a person uses a new device, clears their cookies, or uses a different email address.
In practice, you should rarely rely on a single universal identifier. You should use a hierarchy instead. Prioritise a login over a loyalty number and a loyalty number over a cookie. You must state the confidence level of each layer before using that data to make a visible decision. If a system treats an uncertain probabilistic match with the same confidence as a login, it will eventually address a stranger by the wrong name.
The same problem as the golden record
This is the same discipline as master data management, just applied to a moving target. You are deciding, field by field and system by system, which signal wins when two sources disagree about who somebody is. Get the survivorship rules wrong here and the failure mode is more serious than a duplicate invoice: a returning customer treated as a stranger, or, worse, two different strangers merged into one profile and shown each other's history.
Personalisation systems are only as accurate as the identity data they rely on. Most of these systems operate with a level of certainty that the underlying data does on justify.
Testing your work with honesty
The most common mistake we see is when teams launch a dynamic content system and assume it functions correctly simply because it is live.
Personalised content requires more testing than static pages. The core assumption of personalisation is that specific content performs better for certain audiences. This is a claim that must be verified through testing.
Segments dilute significance faster than people expect
If you split visitors into twelve segments and test three variants in each, you will need enough traffic for thirty-six buckets instead of three. Most websites do not have this much traffic. This often results in a dashboard that reports a performance increase in a segment that did not have a large enough sample size to be reliable. Teams then make decisions based on data that is actually random noise.
The solution is simple. You must use fewer segments than you might prefer and maintain them for a longer duration than is comfortable. You should also pre-register what a meaningful improvement looks like before you review the results. If you cannot determine in advance what size of effect justifies implementing the change, you are guessing rather than testing.
Bandits aren't a shortcut around this
Multi-armed bandits move traffic to the better variant while the test is active. This process can increase efficiency, but it results in a changing allocation rather than a fixed split. If you evaluate a bandit using the same methods as a standard A/B test, you will likely overestimate your confidence in the results. Bandits are appropriate when speed is more important than determining a precise causal relationship, such as a homepage banner during a short sale. They are less effective for pricing or decisions with legal and reputational risks. In those cases, a slower and more controlled test is more useful.
Measuring the actual results
The metric reported after a personalisation rollout is often the one that was easiest to access rather than the one that answers the primary question. Click-through rates for personalised banners almost always exceed those for generic banners. This result confirms that people click on items that appear relevant to them, which is already known. It does not indicate whether the business has improved its overall position.
The harder, more useful question is incremental: would this visitor have bought, or stayed, or upgraded, without the personalisation at all? That's the same puzzle attribution modelling tries to answer for marketing spend, and it's worth borrowing that discipline. A holdout group, a slice of traffic that never sees the personalised version, however small, is the cheapest way to keep an honest baseline in view. Skip it and you'll eventually be reporting a lift against a version of yourself that never existed.
You should also monitor metrics that appear positive but indicate underlying problems. When the add-to-basket rate increases while the conversion rate remains the same, it typically suggests that the system is recommending items that provoke curiosity rather than purchases. This result is acceptable for engagement reports, but it is unsuccessful for generating revenue.
When personalisation becomes intrusive
Most people cannot define the exact point where a website becomes unacceptable, but they recognise it immediately when they see it.
It is helpful to show a person a jacket similar to one they viewed recently. However, it feels intrusive to show an advertisement that reveals personal information about a user, such as a pregnancy, illness, or divorce, before they have shared that news with others. This remains true even if the data used to make those inferences was collected legally.
The legal floor sits below the comfortable line
Meeting the requirements for data subject rights and consent is the baseline, not the goal. A business can follow all rules, obtain consent, and document purposes, but still create a product that users find unsettling when they realise how much information the site collects. This loss of trust is not measured by regulators, and it usually results in customers quietly stopping their use of the service, rather than filing a formal complaint.
We use a basic rule when working with clients. You should not ship a piece of content if you would feel uncomfortable explaining the specific signal that produced it to the customer. The statement "We showed you this because you looked at similar items last week" passes this test. The statement "We showed you this because your browsing pattern statistically resembles someone in a life stage we infer from search terms" does not pass the test, even if the statement is accurate and legally allowed.
Ownership after the launch
Dynamic content systems deteriorate in the same way as master data. This happens gradually because no one is responsible for making decisions after the initial development phase is complete.
Segment definitions become outdated. Rules created for previous versions of the catalogue continue to run even after the inventory changes. Content variants approved during old launch meetings remain active because no one was assigned to remove them.
Successful organisations assign a specific person to manage rules and identity logic. This owner is different from the person who manages the platform licence. This individual has the authority to remove rules that are no longer useful, which prevents outdated rules from remaining active due to a fear of technical risk.
Things naturally decay
Personalisation rules become outdated because people do not prioritise their maintenance. This usually happens when more urgent tasks take precedence, and the issue is often discovered through customer complaints instead of automated alerts. Most of these problems can be identified by reviewing live rules against the current product catalogue and customer segments every quarter. This review identifies rules that apply to missing inventory, segments that were not updated after product changes, and content variants that are being measured against outdated control groups. This meeting is brief and less engaging than the initial build process, but skipping it causes a successful system to become inaccurate over time.
Where to start
Use rules instead of a model unless your traffic volume justifies the change. You must accurately assess which situation applies to your business. Focus on identity resolution for visitors you can identify with certainty, such as those using logins or loyalty numbers, before you attempt probabilistic matching for anonymous users. Include a holdout group in every test from the start to maintain a consistent baseline for measurement. Determine what constitutes a meaningful improvement before you review the results.
Apply the discomfort test to new content before you release it. If you cannot explain the specific data or signal used to generate a piece of content to the recipient, you should stop the project rather than proceed. Effective personalisation is subtle and usually goes unnoticed by the user. Poor personalisation results in negative public feedback and social media complaints. If you are considering a dynamic content programme, or if you are managing an existing one that has not been reviewed recently, talk to us.
Many teams complete the theoretical phase but struggle to begin construction. Our AI development service is designed to assist with that specific stage.
Frequently asked questions
What is the difference between personalisation and dynamic content?
Personalisation refers to the overall outcome of presenting visitors with content that aligns with their interests. Dynamic content serves as the particular tool employed to obtain that outcome. It operates as a decision engine that chooses a specific version of a banner, product list, or email subject line using data about the visitor. This text treats both terms as one concept since personalisation depends on dynamic content to work.
How does a dynamic content decision engine work?
A small decision tree works behind the scenes and executes many times each second. It decides which pre-built content block variant to display using data about the visitor. Most businesses employ a rules engine that follows human-written if-then statements. A model-based approach, like a multi-armed bandit, is useful only when traffic and the number of variants are sufficient for the system to learn.
Why does identity resolution matter for content personalisation?
Personalisation logic needs the system to identify that a visitor on a phone in the morning is the same person using a laptop in the evening. Deterministic matching that relies on login credentials or loyalty numbers yields reliable data, yet it covers only a small fraction of total traffic. Probabilistic matching enables wider coverage, but this approach is vulnerable to failure as cookies and fingerprinting techniques grow less effective.
How do you test whether content personalisation is working?
The most common mistake is to assume a system functions just because it has launched. Personalisation demands more testing than a static page. You should keep fewer segments than you might prefer and keep them active for longer periods. Determine what counts as a meaningful improvement before you begin. Use a holdout group to make sure you have a consistent baseline for measurement.
When does content personalisation turn creepy?
Most people cannot define the exact moment a website becomes intrusive, but they notice it when it occurs. An example is an advertisement that mentions a life event before the user has shared that information with others. A practical guideline is that you should not release content if you would feel uneasy explaining the specific data used to generate it. This rule applies even if the data was collected by legal methods.