What Is an AI Agent? The Model, the Loop, and Everything Around It

"Agent" has become one of the most stretched words in software. A chatbot with a calendar plugin gets called an agent. So does a scheduled script that summarises email. So does a system that runs for an hour, calls forty tools, and files a finished report.

They aren't the same thing, and the difference matters when you're deciding what to build. This guide explains what an AI agent actually is, what sits around the model, and when you don't need one at all.

The short definition

The important word is decides. If every step is fixed in advance by a developer, you have a workflow. Workflows are useful (often more useful), but they aren't agents. Anthropic draws the same line in its widely cited Building effective agents guide: workflows follow predefined paths, while agents direct their own process and tool use.

The loop: think, act, observe

Every agent runs some version of the same cycle. Take a customer asking a support agent to refund a duplicate charge:

1. Think. The model reads the request and decides it first needs to find the charges.
2. Act. It calls a list_payments tool for that customer.
3. Observe. Two identical charges, four minutes apart. That looks like a genuine duplicate.
4. Think again. Policy allows automatic refunds under a set amount. This one qualifies.
5. Act. It calls issue_refund on the second charge.
6. Observe. The refund succeeded. It replies to the customer with the reference number and stops.

If the charges had been a week apart, the observation would have changed the next step: perhaps asking the customer a clarifying question, or handing the case to a person. That ability to change course based on what it finds is what makes it an agent.

The model is the brain, not the whole agent

The large language model does the reasoning: understanding the request, choosing tools, writing the reply. But a model on its own can't remember yesterday, can't see your database, and can't do anything outside its text box. Everything else is built around it.

The orchestration layer

This is the code that runs the loop. It sends the model the goal and the available tools, executes whichever tool the model picks, passes results back, enforces step and cost limits, retries failures, and decides when the run is over. Frameworks such as LangGraph, the OpenAI Agents SDK, and the Claude Agent SDK provide this layer; simple agents can use a few dozen lines of plain code.

Short-term memory

The state of the current task: what the user asked, which tools ran, what they returned. It usually lives in the conversation history or a session store, and it disappears or gets archived when the task ends.

Long-term memory

Facts worth keeping across sessions: a customer's preferences, a decision made last month, a correction a reviewer gave the agent. Long-term memory needs a write policy. An agent that remembers everything ends up retrieving noise.

Retrieval (RAG)

Search over your own content (docs, policies, tickets, product data) so the model reasons from your facts instead of its training data. Most wrong answers from agents and chatbots are really retrieval failures.

Tools

Functions the agent can call: read a record, send an email, query an API, run code. Each tool needs a clear name, a description the model can understand, and typed inputs. Increasingly, tools are exposed through a standard protocol; see MCP vs A2A.

Guardrails

The rules that keep an agent inside its job: scoped permissions, approval for irreversible actions, spending limits, input and output checks, and an audit log. They are what make an agent safe to connect to real systems.

What isn't an agent (and why that's fine)

• A single prompt that summarises a document. Useful, but there's no loop and no tools.
• A fixed chain of prompts: extract, then classify, then draft. That's a workflow.
• A scheduled automation that exports data every Monday. Predictable by design.

If you can write the steps down in advance and they rarely change, a workflow is cheaper, faster, and easier to test. Reach for an agent when the right next step genuinely depends on what the previous step found.

Signs a task needs an agent

• The path varies. Two requests that look similar need different investigations.
• It spans several systems, with decisions in between.
• There's a checkable result, so you can tell whether the agent succeeded.
• Mistakes are recoverable, or can be caught by an approval step.

How ThinkDeck builds agents

Every agent we ship for clients uses the same building blocks described above, plus two things from our own platform:

• A model gateway. Agent model calls go through AiKey, the AI gateway we built and run ourselves. It lets us route each step to the right provider (OpenAI, Anthropic, Google, Groq), set budgets per agent, and see cost and latency for every call.
• Deploy where you already run. We deploy the orchestration layer on AWS, GCP, Azure, Cloudflare, or Vercel, next to the systems the agent needs, so tools stay fast and data stays where it belongs.

Because we run AI products of our own in production (AiKey, CapAI Studio, and JobVault), the monitoring, budgets, and rollout habits we bring to client agents are ones we rely on ourselves.

Where to go next

If you're weighing an agent against a simpler bot, start with AI agent vs chatbot. For the build itself, see the AI agent development process, then context engineering, evaluation, and deployment. Or tell us the task and our AI agent development team will tell you honestly whether it needs an agent.