AI agents explained: what they automate, and what they shouldn't

"AI agent" is the most overloaded term in the industry right now. Marketing makes it sound like a digital employee you hand your password to. The useful definition is smaller and more concrete: an AI agent is a system that can take a goal, break it into steps, and use tools — search, your email, a code editor — to complete those steps without you clicking through each one. The difference from a chatbot is the doing, and the doing is exactly where things get interesting and risky.

How an agent decides what to do next

Remove the branding and the loop is always the same: read the goal, pick a next step, call a tool, read what came back, repeat until it judges the job done. Quality comes down to two unglamorous abilities — noticing that a result is wrong, and knowing when to stop.

Three things share this name; what separates them is who chooses the steps:

The rule: if a fixed sequence of steps would do, don't buy a planner. Automations fail loudly and repeatably; agents fail quietly and uniquely, which is why they earn their keep only where nobody could have written the steps in advance.

What agents are genuinely good at today

Three patterns work for one reason: something outside the model can check the result.

1. Research with a checklist. "List the 20 competitors in this niche, pull their pricing pages, summarise the tiers into a table." Agents are good at breadth — the tedious fan-out humans abandon at result eight.
2. Bounded coding tasks. Write this function to spec, fix this class of lint errors, add tests to this module. They shine when success is checkable, because you can make the agent run the checks first.
3. Routine glue between apps. Webhook arrives → classify it → draft the response → put the draft in a human's queue.

A prompt that carries its own checks, for a web-capable assistant such as ChatGPT, Claude or Perplexity:

Goal: list every company selling [category] to [customer type] in [country].
For each: name, URL, date checked, tiers as that page states them.
Rules:
- Only the vendor's own pricing page. No roundups, no blogs.
- If a page says "contact us", write "contact us". Never estimate.
- Stop at 20 companies. If you find fewer, say how many and why.
Before finishing: mark uncertain rows with "?"

What makes it work: a hard stop, a stated fallback, and a final self-check. Source and date travel with the table.

Where agents still fail — and what to do about it

The failure mode is almost always the same: an agent will confidently continue a plan that went wrong three steps ago. Keep unsupervised agents away from irreversible actions — deleting, paying, sending to customers, pushing to production — and from long ambiguous projects. "Build this page from this spec, then run these tests" works. "Build my website" doesn't, because ambiguity compounds with every autonomous step.

Two quieter failures reach your work product. Silent substitution: it can't find a nineteenth competitor, so it pads the table with something plausible. Instruction decay: the spec loosens with every step, so step nine obeys nothing you wrote at step one.

When a run goes wrong, the fixes are about scope and prompts, not the model:

• Invented rows. Demand a source and date in every row, plus a "say how many and why" fallback. Still inventing? Halve the scope per run.
• Repeated searches. Cap the tool calls and split the job in two; a step budget turns an infinite loop into a cheap failure.
• Drift from the spec. Restate the rules at the end of the prompt and ask the run to quote them back.
• Usage jumps. Something is retrying. Read the log, find the failing step, fix that instead of switching tools.

The diagnostic is one sentence long: can you write the success check? "All 20 rows have a price and a URL" — automate it. "Make the campaign better" — not yet. Then make every run cheap to discard: sandboxed credentials, an isolated drafts folder, no deleting files the agent didn't create.

Your first safe agent workflow

The first workflow should take an afternoon to set up. Do it in this order:

1. Pick one weekly chore with a clear input and output — five newsletters into a Monday brief, or support threads into a list of recurring questions.
2. Narrow the toolbox. Read-only access to named sources; the agent writes a draft file and nothing else. No send, publish, delete or spend.
3. Write the spec like an onboarding doc. Format, length, tone, what to skip, what to do when a source is missing. Agents are instruction-following interns: capable, literal, and entirely lacking in doubt.
4. Review every output for two weeks. You're building the checks the next run can enforce itself.
5. Only then remove one human step. Keep sending, publishing and paying manual until trust is earned in writing, not in feeling.

Copy this into the workflow's standing instructions and edit it to fit:

• Sources: only [named list]. If one is unavailable, say so; never substitute another.
• Output: one file, dated headline, max 600 words, then "what changed" and "what I could not verify".
• Allowed: read, summarise, write to drafts. Forbidden: send, publish, delete, purchase.
• Every claim carries a link and a checked date; uncertain rows get "?" instead of a guess.

A test plan you can run yourself

This site publishes no benchmark scores: a number would be stale within weeks and would describe one person's workload, not yours. Here is the test instead — four runs.

1. Read-only research with an answer you already know. Your own competitor set, your published prices. Score it on correct rows, provenance present, gaps flagged.
2. Same task, sabotaged. Remove two sources the run depends on. Useful behaviour is reporting four of six; bad behaviour is six confident rows.
3. Bounded code task. One function, one spec, one test command the agent may run. Success means the tests pass, not that the diff looks tidy.
4. Glue workflow, drafts only. Trigger, classify, draft, queue — nothing able to send. Two weeks before you touch the send step.

Keep a run log — date, prompt version, what went wrong — and change one variable at a time, as the tool evaluation guide describes. Two of the same mistake means a missing check, not a missing model.

Cost and risk trade-offs

An agent run consumes more of your allowance than a chat, because one task is many model calls plus retries. If your plan bills per message rather than per tier, an overnight loop becomes a bill — read the vendor's pricing page before the first unattended run and set a spend cap where offered.

On value, the honest comparison isn't "agent versus nothing", it's "agent versus twenty minutes of your time". Four decision rules:

• If it happens once → do it in a chat, no setup.
• Fixed schedule, fixed inputs → automation, not an agent.
• Inputs vary and an external check exists → agent, read-only, drafts only.
• Output reaches a customer unreviewed → not yet.

Tools worth a look

For coding agents, Cursor and GitHub Copilot are the mature entry points: both keep the work in a repo you review line by line. For research and ops, the built-in modes of ChatGPT, Claude and Gemini cover most needs without another subscription; Perplexity is built around showing its sources. Self-hosted orchestration frameworks assume you can debug a failed step, and much of that ecosystem sits on Hugging Face.

Also worth reading: the small-business automation guide for the no-code route, and the prompt writing habits guide, which fixes more agent failures than any tool switch. Our judgements follow the methodology page; corrections land in the changelog, and product detail lives in the profiles.

Agent questions, answered plainly