AE AgentEngineering.org Practical AI agent engineering All Articles
Article

What Is an AI Agent?

An AI agent is a goal-directed system that can observe state, decide what to do next, use tools, and act across multiple steps.

March 10, 2026 Updated October 13, 2026 Foundations / Beginners / Workflows / LLMs

An AI agent is a goal-directed system that can observe state, decide what to do next, use tools, and act across multiple steps.

That definition matters because the term is used too loosely. A chatbot is not automatically an agent. A workflow with a few model calls is not automatically an agent. A tool-using assistant is not automatically an agent either.

The useful line is this:

An AI agent is not just a model that generates an answer. It is a system that operates in a loop.

If the system can inspect state, choose among actions, use tools, react to results, and continue until it reaches a goal or hits a stopping condition, you are in agent territory.

The idea is older than language models. Russell and Norvig define an agent as anything that “perceives its environment through sensors and acts upon that environment through actuators” (Artificial Intelligence: A Modern Approach, 4th ed., 2020, ch. 2). Shunyu Yao’s version is the same idea pointed at software: an agent is an intelligent system interacting with an environment, whether that environment is physical, digital, or other people. What is new is that the decision procedure is now often an LLM, choosing tools and next steps at runtime.

The Five Properties of an Agent

Unpack that definition and you get five properties:

That is what separates an agent from a one-shot response system. OpenAI’s practical guide compresses the same test: “Agents are systems that independently accomplish tasks on your behalf,” and “Applications that integrate LLMs but don’t use them to control workflow execution — think simple chatbots, single-turn LLMs, or sentiment classifiers — are not agents.” Hugging Face’s agents course says it as a system that “leverages an AI model to interact with its environment in order to achieve a user-defined objective.”

Another way to say it is that an agent turns AI from a passive answer engine into an active operator. Instead of only responding to a prompt, it can work through a task, gather what it needs, choose a next step, and keep going until it either reaches the goal or hits a boundary.

A simple example helps. If you ask a model, “Summarize this report,” and it returns an answer, that is not an agent. It is a model producing a response.

If you ask a system to “figure out why revenue dropped last month, pull the right reports, compare them to the prior period, and draft a summary for finance,” and the system decides which data source to query, retrieves information, updates its plan, and asks for approval before sending anything, that starts to look like an agent.

The shift is from answering to pursuing.

Agency Is a Spectrum, Not a Badge

Yao calls agent “a notoriously broad term,” and says the field is “really a moving piece.” Andrew Ng’s advice is to stop arguing about whether something “is an agent” and talk instead about “degrees to which something is agentic.” Marketers got hold of the word. The useful question survived.

Hugging Face makes the same point as a ladder: a model whose output does not change program flow is a processor; a model that picks a branch is a router; a model that picks a function is a tool caller; a model that controls whether the program continues is a multi-step agent. Most products sit on a rung of that ladder, not at a pole.

That is why autonomy is better understood as a spectrum, which The Autonomy Spectrum treats in full:

Most real products sit somewhere in the middle, not at the extremes.

How Is an AI Agent Different From a Workflow?

This is the distinction that actually changes the architecture.

Anthropic draws it in Building effective agents with wording worth keeping exact:

Workflows are systems where LLMs and tools are orchestrated through predefined code paths. Agents, on the other hand, are systems where LLMs dynamically direct their own processes and tool usage, maintaining control over how they accomplish tasks.

Barry Zhang’s talk version of the same paper is shorter: agents are “models using tools in a loop,” and they “shouldn’t be a drop in upgrade for every use case.” Workflows are “the beginning of agentic systems.” Agents are the ones that “can decide their own trajectory.”

A workflow usually follows a predefined path. It may branch on conditions, but the designer has mostly decided the sequence in advance.

Example:

That can be useful, and it may include model calls, but it is still mainly controlled by fixed logic.

An agent has more freedom inside boundaries. Instead of following one rigid path, it can decide which step comes next based on the state of the task.

Example:

The important difference is not whether the system uses an LLM. It is whether the system has bounded discretion about how to pursue the goal. LLMs, Workflows, and Agents is the longer comparison.

How Is an AI Agent Different From an LLM?

A large language model is the reasoning engine. It can interpret language, generate text, classify information, and help choose what should happen next.

But an LLM by itself is still usually a stateless function. You give it an input. It produces an output. The surrounding system has to decide what to do with that output.

An agent is the larger system wrapped around the model.

The model may provide the reasoning, but the agent provides the operating loop:

One useful way to say it is:

That is why it is possible to have a powerful model without a capable agent, and also possible to improve a weak agent system by redesigning its loop, context, and action boundaries rather than changing the model.

LLM vs Workflow vs Agent

The shortest practical comparison looks like this:

System typeWhat it mainly doesControl flowTypical strengthTypical limit
LLMGenerates or transforms contentSingle responseFlexible reasoning in one turnDoes not manage the task beyond the response
ChatbotManages a conversationTurn by turnNatural interface for a userOwns the dialogue, not the task
WorkflowExecutes a predefined sequenceMostly fixedReliable, auditable, repeatable processWeak at adapting when the path changes
AgentPursues a goal across stepsBounded but adaptiveCan choose among actions, tools, and next stepsHarder to evaluate, control, and operate

A chatbot is worth calling out separately because it is an interface, not an architecture. A chatbot can contain an agent, but not every chatbot is an agent, and a workflow can call a model without becoming one.

This is why the term agent should be used carefully. The presence of a model does not make a system an agent, and the presence of automation does not make a system intelligent.

How Does the Agentic Control Loop Work?

The simplest way to think about an agent is as a control loop:

  1. observe the current situation
  2. interpret what matters
  3. decide the next action
  4. act through a tool or message
  5. inspect the result
  6. update state and continue or stop

This is often described as sense, think, act. The Sense-Think-Act Loop is the runtime article for that cycle. In software terms it usually looks like:

The research ancestor of that loop is ReAct (Yao et al., 2022): interleave a reasoning trace with an action, read the observation, and repeat. ReAct and the Basic Reasoning Loop is the pattern-level version.

Once you see the loop, many agent concepts become easier to understand.

Planning is part of the loop. Memory is part of the loop. Tool use is part of the loop. Evaluation and recovery are also part of the loop.

This is also where failure starts to compound. If the system misreads the state in one step, it may choose the wrong tool in the next step. If the tool returns noisy data, the next decision may drift further. That is one reason agent engineering quickly becomes a systems problem rather than a prompting problem.

The agent loop and the autonomy spectrum A spectrum strip from single call through workflow to agent, then a four-stage loop: observe state, decide the next action toward a goal, act through a tool in the environment, inspect the result, and either stop or loop. The five properties — goal, state, actions, choice, and loop — are annotated on the stages they belong to. DEGREES OF AGENCY Single call Workflow Agent fixed path → own trajectory STATE GOAL + CHOICE ACTIONS THE LOOP 1. Observe 2. Decide 3. Act via tool 4. Inspect Read the goal, the history, and what just changed Choose the next action, or decide the task is done Call a tool in the environment, or send a message Read the result. Update state. Stop, or loop. Stop condition? goal met · max steps · escalate Five properties: goal · state · actions · choice · a loop that can stop
Single call, workflow, then agent. The agent is the loop: observe, decide, act through a tool, inspect the result, and stop when the goal is met, the step budget runs out, or a human has to take over.

What Are the Core Components of an AI Agent?

OpenAI’s three-part anatomy is the right starting kit: model, tools, and instructions. Barry Zhang adds the missing noun: the environment the tools act on, plus the system prompt as the instructions. CoALA (Sumers, Yao et al.) is the research frame underneath: a memory, an action space, and a decision procedure. Put together, a modern agent has six parts.

Model

The foundation model, usually an LLM, is the decision procedure. It interprets the goal, reads the current state, and chooses the next move. Yao’s levels are useful here: a text agent produces text; an LLM agent is driven by a language model; a reasoning agent uses the model to reason and then act. Most systems people mean by “AI agent” are in that third bucket.

Tools

Tools are how the system escapes pure text generation and starts affecting the world around it. They are the action space: APIs, databases, search, code execution, browser or computer-use actions, internal business systems.

Without tools, an agent may still reason, but it cannot do much beyond producing text. With tools, it can query data, trigger workflows, update systems, execute code, or prepare actions for human approval. Tool Use: How Agents Take Action covers the call itself. Designing Tools for Agents covers the interface the model actually programs against.

Environment

The environment is whatever those tools touch: a ticket queue, a codebase, a browser, a warehouse API. The agent does not see the environment directly. It sees whatever the last tool result put back into context. Zhang’s instruction is the design test: “put yourself in the agent’s context window.” If a fact is not in that window, the agent does not have it.

Instructions

Instructions are the explicit guidelines and guardrails that define how the agent behaves. OpenAI treats them as a first-class component, not a prompt you write once and forget. They name the goal, the allowed tools, the style of work, and what to do when information is missing or a step is unsafe.

Memory and state

Memory is what lets the system stay coherent across steps and across time. CoALA splits it into working memory (the current context window), plus longer-lived stores for episodes, facts, and procedures. In production that usually means:

If the system cannot retain or recover the right state, it will repeat work, lose context, or make poor decisions. Short-Term Context, Retrieval, and Long-Term Memory is the systems article for that split.

The loop and the stop condition

The loop is what turns the other components into agentic behavior. After an agent takes an action, it inspects the result and decides what to do next. If the result is good, it can continue or stop. If the result is poor, incomplete, or unsafe, it can adjust the plan, ask for clarification, try a different tool, or escalate to a human.

That does not always mean writing a complete plan up front. Sometimes planning is lightweight and updated after every step. But some planning layer has to exist, even if it is only: what is the goal, what information is missing, and what is the best next move.

A loop without a stop condition is a runaway process. Anthropic is explicit that tasks often terminate on completion, and that it is common to include stopping conditions such as a maximum number of iterations. The three stops that show up in real systems are: the goal is met, the step or token budget is gone, or control transfers back to a human.

A Working Loop

The whole definition fits in a short, provider-neutral program: one tool, a step cap, and a stop on a final answer. decide() is the stand-in for a model call. Everything around it is the agent.

# agent_loop.py — provider-neutral agent loop. Run: python agent_loop.py
from __future__ import annotations

CATALOG = {"acme-pro": "Acme Pro ships in 3 days and costs $49."}


def lookup(sku: str) -> str:
    """The one tool. In production this is an API, a DB, or a search call."""
    return CATALOG.get(sku, f"No listing for {sku!r}.")


def decide(goal: str, observations: list[str]) -> dict:
    """Stand-in for a model call. Swap this for your provider's chat API."""
    if not observations:
        return {"tool": "lookup", "args": {"sku": "acme-pro"}}
    return {"answer": observations[-1]}


def run(goal: str, max_steps: int = 4) -> str:
    observations: list[str] = []
    for _ in range(max_steps):
        action = decide(goal, observations)
        if "answer" in action:
            return action["answer"]
        if action.get("tool") == "lookup":
            observations.append(lookup(**action["args"]))
            continue
        return "Stopped: neither a tool call nor a final answer."
    return f"Stopped after {max_steps} steps."


if __name__ == "__main__":
    print(run("What is the lead time for Acme Pro?"))

That is the five properties in code. The goal is the question. The observations list is the view of state. lookup is the action space. decide chooses the next action. The for loop updates behavior from the last result and stops on an answer or max_steps.

What Does an AI Agent Look Like in Practice?

The definition gets concrete fast once you look at the kinds of systems that actually need a loop.

A support agent might read a customer message, retrieve account details, search the knowledge base, decide whether the issue is billing or technical, draft an answer, and escalate when policy requires human approval. The valid next step depends on the situation.

A research agent might start from a question, search several sources, compare evidence, extract claims, identify gaps, and prepare a structured summary. The next search depends on what the last source revealed.

A coding agent might inspect a bug report, search the codebase, edit files, run tests, read the failures, revise the patch, and stop when the change is ready for review. The task changes as new evidence appears.

An operations agent might watch a failed job, inspect logs, identify likely causes, retry safe steps, open an incident, or ask a human before taking a risky action. Production work often requires judgment under changing conditions.

In all four cases the agent is not useful because it sounds impressive. It is useful because the path cannot be written down in advance.

Why Are Agents Harder Than They Look?

A weak chatbot gives a bad answer. A weak agent compounds a bad decision across a run.

The common failure shapes are specific:

None of these are prompting problems. They are loop design, context, and control problems, which is why the surrounding system matters more than the model choice.

Why Does Bounded Autonomy Matter?

In theory, people like to talk about autonomous agents. In practice, useful agents are almost always bounded agents.

That means their freedom is designed.

The system may decide what to do next, but only inside limits such as:

This matters because once an agent can act, mistakes stop being just bad words on a screen. They become bad tool calls, wrong state changes, wasted money, or unsafe actions.

Zhang’s budget example is the cost version of the same bound, and it is a specific one. For a high-volume customer-support system at about ten cents per task, you can afford roughly 30,000 to 50,000 tokens. That is not enough runway for an open-ended loop on the common cases, so he says to just use a workflow for those scenarios. The point is not that every cheap task must be a workflow. It is that the unit economics of high-volume support often make an open-ended agent the wrong default.

So a real-world definition of an agent should include not just autonomy, but bounded autonomy.

That is also why guardrails, permissions, and human review are not bolt-ons. They are part of the engineering shape of an agent system.

When Should You Use an Agent?

OpenAI’s test is to prefer agents where deterministic automation has already failed: complex decision-making, rulesets that have become unmaintainable, or work that lives in unstructured language. Anthropic’s test is almost the inverse of the hype: find the simplest solution that works, and only add agency when you need flexibility the predefined path cannot give you.

You should consider an agent when the task has all or most of these properties:

You should prefer a workflow when:

Ng’s observation is the one teams skip: many valuable systems are still simpler workflows. This is why the best question is often not “Can we build an agent?”

It is:

What is the smallest amount of autonomy this task actually needs?

That question usually leads to better systems. When to Use a Workflow Instead of an Agent is the decision article.

When Should You Not Use an Agent?

You should usually avoid an agent when the task is already well understood and can be represented as explicit logic.

Good examples:

In those cases, a workflow is often cheaper, easier to test, easier to audit, and easier to maintain.

You should also avoid an agent when:

Many teams reach for an agent when what they actually need is better software structure, better data access, or a narrower workflow. That is one reason the workflow-versus-agent decision is so important.

Why This Definition Matters for Everything That Comes Next

Once the definition is clear, the rest of the field becomes easier to organize.

If an agent is a goal-directed loop, then the next questions become obvious:

That is why this article comes before topics like context engineering, evals, and AgentOps. Those topics only make sense once the basic unit of the system is clear.

If you want the broader framing for why this has become a discipline, read What Is Agent Engineering?.

The Bottom Line

An AI agent is a goal-directed system that can observe state, decide what to do next, use tools, and act across multiple steps.

That does not mean every agent is highly autonomous. Most useful agents should not be. The real design problem is not maximum autonomy. It is reliable, bounded autonomy.

That is the starting point for agent engineering.

FAQ

What is the shortest useful definition of an AI agent?

An AI agent is a goal-directed system that can observe state, decide what to do next, use tools, and act across multiple steps.

Is ChatGPT an AI agent?

By itself it is a conversational interface. OpenAI is explicit that simple chatbots and single-turn LLMs are not agents, because the model does not control workflow execution. ChatGPT can become part of an agent system once it is connected to tools, memory, and multi-step task logic, but a chat window alone is not an agent.

Is a chatbot with tools an agent?

Not automatically. Tool calling is necessary for most useful agents and not sufficient on its own. The better test is whether the system can pursue a goal across a loop, choose among actions based on state, and stop when the goal is met. A chatbot that can call a search API once and then reply is a tool-using assistant. A system that can decide to search, read the result, search again, and only then answer is starting to act like an agent.

Is an AI agent the same as an LLM?

No. The LLM is usually the reasoning engine. The agent is the larger system that includes memory, tools, control logic, and action boundaries.

What is the difference between a workflow and an agent?

A workflow orchestrates models and tools through predefined code paths. An agent lets the model dynamically direct its own process and tool use, inside bounds you set. That is Anthropic’s line, and it is the one worth using.

What is the difference between “agent” and “agentic”?

“Agent” is a claim about architecture: the model is directing the loop. “Agentic” is a degree. A workflow with one model-chosen branch is more agentic than a single call, and less agentic than a system that chooses tools for as many steps as the task needs. Prefer the degree over the badge.

When should I use an agent instead of a workflow?

Use an agent when the task is multi-step, uncertain, and requires choosing among actions. Use a workflow when the path is known and can be hardcoded.

Is more autonomy always better?

No. More autonomy usually increases risk, cost, unpredictability, and evaluation difficulty. The goal is not maximum autonomy. It is appropriate autonomy.

What should I learn after understanding what an AI agent is?

The next topics are usually:

What is the next article in this learning path?

The clean next step is LLMs, Workflows, and Agents: What Actually Changes?, because that sharpens the boundary between the three system types before moving deeper into context and reliability.

Further watching

Each of these was transcript-checked; timestamps link to the moment the claim appears.

Primary Sources