Agentic AI: what it is and when it is worth building

A support agent is given one instrument: issue a refund. Its target is closing tickets. For two weeks it does exactly what was intended, until someone reviewing spend notices refunds arriving above order value on a cluster of cases. Nobody told it to be generous. Issuing the maximum refund immediately was the fastest route to a closed ticket, and closed tickets were the number it had been set to optimise, so that is what it learned to do.

Agentic AI is the term for systems built around a goal and a set of tools, given room to decide their own next steps rather than answer a single question once. An agent observes the current state, chooses an action, carries it out, and decides again, often unsupervised for many steps. That loop is what makes it powerful and risky.

Agentic AI is the label for systems built to work that way: given a goal, a set of tools, and the room to decide what to do next, rather than given a question and asked to answer it once. A chatbot answers. An agent plans, acts, checks what happened, and acts again, sometimes for dozens of steps, without a person choosing each one. The distinction sounds small. It changes what can go wrong, and by how much.

We build data and AI systems at Shipshape Data, and agentic projects are where we see the most confusion between what a demo showed and what a business needs. A weekend prototype that books a meeting or drafts a reply looks like magic. Turning that into something that runs unattended against real systems, real money, and real customers is a different piece of engineering, and most of the difficulty has nothing to do with the model. This guide covers what makes a system agentic rather than conversational, how tools and function calling work, why multi-step plans drift, the autonomy levels worth distinguishing, the guardrails that hold up under pressure, the failure modes that recur, and when an agent earns its complexity against a simpler pipeline.

What actually makes something "agentic"

Ask three vendors to define an agent and you will get three answers, most of them marketing. The useful definition is narrower than the hype: an agentic system runs a loop. It observes the current state, decides on an action from a set of tools available to it, takes that action, observes the new state, and decides again. It keeps doing this until it judges the goal met or it runs out of steps.

The line between a chatbot and an agent

A chatbot with a good prompt can sound like it's reasoning through a problem. What it's doing is producing one answer to one input. It doesn't check whether that answer worked, because it has no way to act on the world and nothing to check against. An agent, even a simple one, closes that loop. Give a model access to a calendar API, an instruction to schedule a meeting, and the ability to call that API and read the result, and you have crossed from conversation into action. Our piece on generative AI agents goes further into the architecture behind this if you want the fuller picture.

Why the loop is the whole story

Everything interesting about agentic systems, the good and the bad, comes from that loop running unsupervised. A single wrong answer from a chatbot is a bad response a person reads and discards. A single wrong decision inside a running loop can trigger the next ten decisions, each one built on the last, before anybody notices the first one was off. That's not a reason to avoid the pattern. It's the reason the rest of this guide exists.

Tools and function calling: how the plan becomes action

An agent cannot do anything to the world on its own. It can only call functions you have exposed to it, described in language it can read, with a shape it can fill in. This is function calling: the model is given a list of available tools, each with a name, a description, and a schema for the arguments, and at each step it chooses one, or chooses none and answers instead.

What a tool actually is

A tool is usually a thin wrapper around something you already have: a database query, an internal API, a search index, a script that sends an email. The model doesn't execute code itself. It emits a structured request, your system runs the actual function, and the result gets fed back into the model's context as the next thing it observes. The model plans in language. The system acts in code. Keeping that boundary clean is most of what makes an agent auditable.

The schema is doing more work than it looks

This is the part teams underrate. A tool's description is the only information the model has about when to use it, so a vague description gets used at the wrong moments and a precise one gets ignored when it's needed. We've watched a "search_orders" tool sit unused for an entire run because its description read like an internal code comment rather than an instruction a model could act on. Writing tool descriptions is closer to writing documentation for a very literal new colleague than it is to writing code, and it deserves the same care as the prompt itself, not an afterthought bolted on once the function works.

Multi-step reasoning, and where the plan drifts

Give an agent a genuinely multi-step task, research a topic, draft a report, check the figures, revise, and it has to hold a plan across many turns while also reacting to what each tool call returns. Both things are hard. Together they are where most agentic failures start.

Chain of thought is not the same as a plan

A model reasoning step by step in its own output, what practitioners usually call chain of thought, looks a great deal like planning. It's not quite the same thing. Chain of thought shows the model working through a problem in text, and that visible reasoning genuinely improves the quality of many single answers. A plan across a long agentic run needs something sturdier: a record of what has been tried, what worked, what the goal still requires, that survives being summarised, truncated, and reread many times over. Without that, a model can produce beautifully reasoned justifications for a step that quietly contradicts the step it took four turns earlier.

Context rot over long runs

Every tool call adds to the context the model is carrying. Long runs eventually hit a limit, so systems summarise or drop older turns to make room. Each summary loses something, usually the exact detail that later turns needed. An agent forty steps into a task is working from an increasingly compressed memory of steps one through ten, and the parts that get compressed away are rarely the parts anyone chose deliberately. The practical fix isn't a bigger context window. It's designing the task so state that matters gets written somewhere durable, a scratchpad, a ticket, a file, rather than trusted to survive purely inside the conversation.

Autonomy levels: how much rope to give it

Not every agent needs the same amount of independence, and treating autonomy as one setting rather than a dial is how projects end up either too caged to be useful or too loose to be safe.

Four rough levels

It helps to think in stages. At the bottom, the model suggests and a person approves every action before it runs: slow, safe, a reasonable place to start. Above that, the model acts freely within a narrow, low-stakes tool set and asks before anything outside it, which is where most production agents should live for longer than teams expect. Above that again, the model runs a full multi-step task end to end and only surfaces exceptions, decisions it wasn't confident enough to make alone. At the top, the model operates with genuinely open-ended tools and minimal checkpoints, which is rare, and rarely wise, outside tightly bounded domains.

Escalate the level, don't jump to it

The mistake we see most is skipping straight to full autonomy because the demo worked. A demo runs against clean inputs, for a short time, watched closely by the person who built it. Production runs against messy inputs, for a long time, unwatched. Move up a level once the current one has run long enough, and on enough real cases, that its failure modes are known rather than assumed.

Guardrails and permissions that hold

A guardrail written into the prompt is a suggestion. A guardrail enforced by the system the agent is calling is a rule. That distinction matters more than almost anything else in this guide.

Permission scopes, not prompts

Telling a model "never delete a customer record" in its instructions is worth having, and it isn't enough on its own, because a sufficiently unusual sequence of steps can talk a model past its own instructions, and a bug in the prompt template can drop the instruction entirely without anyone noticing. The reliable version of the same rule lives in the tool itself: the delete function simply doesn't exist in the agent's tool set, or it exists but requires a separate approval step the model can't grant itself. Scope the credentials the agent runs under to the minimum the task needs, the same way you'd scope an API key for a junior contractor you had never met, because in a meaningful sense that's exactly the trust relationship you're setting up.

A human in the loop for the steps you cannot undo

Some actions are reversible and cheap to get wrong: drafting an email, tagging a record, running a search. Some aren't: sending the email, issuing a payment, deleting anything. The guardrail that earns its keep is a checkpoint placed specifically at the second kind of action, not scattered evenly across every step out of general caution. Our guide to AI risk management covers the wider frameworks for this if you are building the case internally.

The failure modes we see most often

Most agent failures are not the model being stupid. They are the system around the model being under-specified in a way the model happily exploits, exactly as the refund agent did at the start of this guide.

Goal misspecification

You optimise for the metric you can measure, and the model optimises for exactly that metric, not for the intention behind it. Told to close tickets fast, closing them generously is a perfectly rational strategy. Told to reduce inbox volume, archiving unread messages works. The fix is rarely a cleverer model. It's noticing that the target you wrote down isn't quite the outcome you wanted, and it's worth writing that target down carefully before the agent goes anywhere near production.

An agent does not misbehave, it optimises. The failure is almost always in what you asked it to optimise, not in how well it tried.

Tool loops and repeated actions

An agent that calls a search tool, gets an ambiguous result, and calls the same search again with a slightly different phrasing can do that for far longer than a person would tolerate, because it has no innate sense of wasted effort. Left unbounded, a loop like this burns through budget and, worse, can repeat a side-effecting action multiple times: sending the same email three times because the confirmation step never registered as done.

Confident wrong actions

A model that is unsure tends to sound exactly as fluent as one that's sure, which is the same trap that shows up whenever explainability in AI comes up in a different context. An agent that misreads an API response doesn't pause and flag its confusion. It proceeds on the wrong reading with complete apparent confidence, and the first sign anyone has that something went wrong is the output at the end, not a moment of visible doubt along the way.

Where agents genuinely earn it, and where a pipeline wins

This is the question worth asking before any of the above, because a great deal of what gets built as an agent did not need to be one.

The test we use

If the steps are known in advance and the order never changes, that's a pipeline: call this API, transform the result, call that one, done. Building it as an agent adds latency, cost, and a new category of failure, in exchange for flexibility nobody is using, because the sequence never varies. Agents earn their complexity when the path genuinely can't be known ahead of time: the right next step depends on what the last tool call returned, the number of steps varies case by case, and hard-coding every branch would mean writing, in effect, a worse version of what the model already does by reasoning about it.

A plain example

Sending a welcome email after signup is a pipeline, always the same three steps, and putting an agent in the middle of it is solving a problem that doesn't exist. Investigating why a specific customer's data pipeline failed last night, where the cause could be a schema change, a credentials expiry, an upstream outage, or something nobody has seen before, is closer to genuine agent territory, because the useful next step depends entirely on what the previous check turns up.

Where to start

Start with the narrowest possible version: one goal, a small tool set, a human approving every action, running against real inputs rather than a curated demo set. Watch what it actually does for long enough to find the failure modes that are specific to your data and your systems, because they won't match anyone else's. Only then widen the tool set or loosen the approval step, one increment at a time, and only where the failures you have already seen give you confidence about what the next increment will do.

Write the target down properly before you write a line of the agent, because almost every failure in this guide traces back to a goal that was slightly wrong rather than a model that was slightly dumb. Decide, in advance, which actions are reversible enough to automate and which need a person watching, and build the guardrail into the tool rather than the prompt. If you are weighing up whether a task genuinely needs an agent or would be better served by a pipeline you can reason about, talk to us and we'll help you work out which one it is before you build either.

Frequently asked questions