AI agents explained: when a chatbot starts taking actions

An AI agent is a language model in a loop, with tools and a goal. A chatbot answers once and stops. An agent picks an action, runs it, reads what came back, and decides again, until it thinks the goal is met or it hits a limit you set. That is the entire idea. Everything else in the marketing is detail about which tools it was given, who authorized them, and what happens when a step goes wrong.

The loop is the whole idea

An ordinary chat turn is one pass: text in, text out, done. An agent wraps that same model in four extra pieces:

• A goal, stated once at the start, in normal language.
• Tools: search a calendar, read a file, run a query, send an email, execute code.
• A runner (the scaffold) that performs whichever tool the model asks for and feeds the result back in.
• A stopping rule: the model declares itself finished, or it hits a step cap, a spending cap, an unrecoverable error, or a human pressing stop.

Each time round the loop, the whole conversation so far, including every tool result, is sent back through the model. It does not remember the last step. It rereads the transcript and predicts the next action, which is the same operation as predicting the next word in a sentence. Agency is not a new capability inside the model, it is a wiring choice outside it.

A worked example: booking a meeting

Goal: find a 30 minute slot with Priya next week and send the invite.

1. The model reads the goal and decides it needs availability. It emits a call to calendar.list_busy with a date range and two email addresses.
2. The runner executes that call against the real calendar and returns a block of structured data: busy periods, nothing else.
3. The model reads it, finds three candidate gaps, and notices every time is stamped in one time zone. It calls directory.lookup to check where Priya works.
4. The result says her office is five hours ahead. Two of the three gaps land at 7pm for her, so the model discards them.
5. It calls calendar.create_event for the surviving slot, with a title and both attendees.
6. The tool returns an event ID. The model writes a one line summary to you and stops.

Nobody programmed that time zone check. It happened because the first result contained something that prompted it, and that responsiveness is the capability you are buying.

The failures come from the same place. There are two Priyas in the directory and it picked the wrong one. The free looking gap is a tentative hold she cares about, and busy data carries no such nuance. A person hits either of these and asks. An agent has no instinct that it should ask, unless asking is itself one of its tools.

Tool use is just structured text

This is the part most explanations skip, and it is what makes permissions make sense. Tools reach the model as a list: each has a name, a plain English description of when to use it, and a schema for its parameters. When the model wants one, it executes nothing. It produces a structured block of text naming the tool and filling in the arguments, much as it would produce any other structured format you asked for. Your code parses that, decides whether to allow it, runs it, and appends the result to the conversation.

Two things follow. The tool descriptions are prompts: the model chooses by reading them, so a vague description gets a tool used in the wrong situation and two overlapping descriptions get picked between more or less at random. And more tools is not better, because selection accuracy degrades as the list grows. A tight set of well named tools generally beats a sprawling one.

Every tool result also lands in the context window, which is finite. An agent that dumps a 40 page document into the transcript on step two has spent most of its working space and will lose the early instructions later. Those characters are billed on every subsequent loop too, which is why the token arithmetic gets away from people so quickly.

Memory and what actually carries across steps

Agents use the word memory for three different mechanisms, and confusing them causes trouble.

The transcript is the short term one: everything said and returned so far, resent on every loop. It is exact, and it is capped. When it fills, the scaffold usually summarizes the earlier part and drops the original, at which point precise details such as an account number can silently disappear.

Scratch storage is the middle one. The agent writes a plan or partial results to a file and reads them back later. This survives summarizing, which is why many agents are told to keep a running checklist.

Long term memory is a database the agent can search: past conversations, your documents, company records. Nothing is recalled automatically. Something has to fetch the relevant pieces and paste them in, which is retrieval rather than training, and it fails the way you would expect. If the retrieval misses, the agent carries on confidently without the fact.

Why small errors compound

Take a step that is right 95 percent of the time. Chain twenty of those and the arithmetic leaves roughly a one in three chance the whole run is clean. That is a caricature, since the steps are not independent and a good agent catches some of its own mistakes. But the direction is right, and it explains what people find puzzling: an agent that impresses on a four step demo falls apart on a thirty step job.

What makes it worse than plain unreliability is that a wrong step becomes an input. If step three retrieved the wrong customer record, steps four through twelve are performed competently on the wrong data and the final summary reads as a confident success. The model cannot check reality, so a fabricated detail is treated exactly like a real one for the rest of the run. Empty tool results are a classic trigger: a search returns nothing, and the model fills the gap instead of reporting it.

The mitigations are unglamorous. Keep chains short. Use real code for anything deterministic instead of asking the model to do arithmetic or sorting. Make each step verifiable, and put a check step after the steps that matter. Where you already know the sequence, do not use a loop at all: a fixed chain of prompts with defined handoffs is more reliable, cheaper and far easier to debug. Models that work through a problem before answering plan the next action better, which reduces the rate of bad steps without removing the compounding problem.

Permissions and the blast radius question

Ask one question before any agent touches a real system: what is the worst thing it can do in a single step with nobody watching? Sort its tools into read only, reversible writes, irreversible writes, spending money, and messaging another human. The last three deserve an approval step, because a correction email cannot be recalled and a deleted record may not come back.

The controls that work are ordinary ones: give the agent its own account rather than yours so its actions are distinguishable in the logs, grant the narrowest access that lets it finish, cap the steps and the spend per run, allowlist the destinations it can send to, and log every tool call and result.

One risk is specific to agents. When an agent reads a web page, an email or a shared document, that text enters the same context as its instructions, and the model has no dependable way to separate data from commands. Text planted in a document telling it to forward the contents of the inbox is a real attack, and it works because the instruction layers are all just text in the end. Wide read access plus outbound sending is the combination worth avoiding. If you are buying rather than building, access scope, log retention and liability wording are the clauses to read, which is covered in what to check in an AI vendor contract.

How to try one without regret

Start with a task whose output you can check in seconds, not one you would have to audit. Give read only tools on day one and watch the full trace rather than the summary, because the summary is written by the thing you are evaluating. Then add a single write tool with approval still switched on, and remove that approval only for actions you have watched succeed dozens of times. Set the step cap and the spend cap before the first run rather than after the first surprise. And write down who is accountable for what the agent does, because the answer is never the agent: if several people will be using one, a short written policy saves the argument later.