What Is an Agent in Reinforcement Learning?

In reinforcement learning, almost every explanation starts with “an agent.” Yet beginners often blur three different things: the agent, the environment it acts in, and the policy it follows. Mixing them up makes algorithms hard to read and code hard to structure.

This article defines the agent precisely, shows what it contains, explains how it picks actions and learns, and ends with a small Python agent you can run.

What Is an Agent in Reinforcement Learning?

An agent in reinforcement learning is the decision-making entity. It observes the state of an environment, chooses actions, receives rewards, and uses that feedback to improve how it chooses actions in the future.

Picture a robot vacuum learning to cross a cluttered room to reach its charging dock. The decision software that looks at the robot’s position, decides where to move, and improves from experience is the agent. The room, the furniture, the floor, and the dock belong to the environment.

DockchairtableRobotMapping the analogyAgent: the robot’s decision softwareEnvironment: room, furniture, floor, dockPolicy: its rule, such as “here, turn left”Reward: +10 at dock, −1 per move,−5 for bumping into furnitureThe reward numbers are an illustrative design choice.

The Role of an Agent in an RL System

The agent has three jobs:

• Perceive. Read the current state or observation from the environment.
• Decide. Pick an action using its current decision rule.
• Learn. Update its estimates or its decision rule from the reward and the next state.

Everything else, including how states change and how rewards are produced, is the environment’s job. The agent can only influence the environment through its actions. This division is part of the larger picture in What Is Reinforcement Learning?

How an Agent Interacts With an Environment

The interaction repeats at each time step: the agent receives a state, sends an action, and gets back a reward and the next state. Only these three kinds of messages cross the boundary. A step-by-step walk-through, with numbers, is in How Does Reinforcement Learning Work?

Agent, Environment, State, Action, and Reward

Term	Meaning	Maze example
Agent	The learner and decision-maker	The maze-solving program
Environment	Everything the agent interacts with but does not directly control	The grid, its walls, the goal, the reward rules
State	Description of the current situation	The agent’s cell, such as row 2, column 0
Action	A choice available to the agent	Up, down, left, or right
Reward	Numeric feedback from the environment	−1 per move, +10 at the goal

Agent, environment, and policy are different things

• The agent is the whole learner: it holds the decision rule, the learning method, and anything it stores, such as a Q-table or a neural network.
• The environment is the world outside the agent that reacts to actions. Even a robot’s own motors and sensors are commonly modeled as part of the environment, because the boundary marks the limit of what the agent controls directly.
• The policy is one part of the agent: the mapping from states to actions. An agent contains a policy but is more than a policy.

AgentPolicypicks an action for the current stateLearning ruleuses rewards to improve the policyimprovesEnvironmentstates, rules, rewardsactionstatereward (and state)

How an Agent Selects Actions

Given a state, the agent uses its policy. Common ways to turn estimates into a choice:

• Greedy: always pick the action with the highest estimated value. Simple, but it never explores.
• Epsilon-greedy: pick a random action with probability ε, otherwise pick the greedy one.
• Sampling from a stochastic policy: the policy outputs probabilities over actions, and the agent draws one. Policy-gradient methods such as PPO work this way.
• Planning with a model: a model-based agent simulates possible futures and picks the action with the best predicted outcome.

What Is an Agent’s Policy?

A policy is the agent’s rule for choosing actions. A deterministic policy maps each state to one action, written a = π(s). A stochastic policy gives a probability for each action, written π(a | s): the probability of choosing action a in state s.

In the maze, a learned deterministic policy might say: in cell (2, 0) go up, in (1, 0) go up, in (0, 0) go right. Once training is done, you can save the policy and deploy it without the learning machinery. That is why the agent and the policy are not the same thing: the agent also includes how the policy gets improved.

Types of Agents in Reinforcement Learning

Value-basedPolicy-basedActor-criticModel-basedStoresvalue estimatesActs bychoosing the highestvalue actionStorespolicy parametersActs bysampling fromthe policyStorespolicy + value estimateActs byactor chooses,critic evaluatesStoresa model of the worldActs byplanning with themodel

Value-based agents

These learn how good each state or action is and act by choosing the best-valued option. Q-learning and DQN agents are value-based.

Policy-based agents

These learn the policy directly, adjusting its parameters to raise expected return. REINFORCE is a classic example.

Actor-critic agents

These keep both a policy (the actor) and a value estimate (the critic). The critic helps judge whether the actor’s choice was better than expected. A2C and DDPG are actor-critic methods, and PPO is usually implemented in this form.

Model-based agents

These learn or are given a model of how the environment responds, and plan using it. Dyna-Q and MuZero are examples.

Model-free agents

These learn from experience without a model of the environment. Q-learning, SARSA, and PPO are model-free. The categories overlap: an agent can be, for example, model-free and value-based at once.

How Agents Learn From Rewards

An agent compares what it expected with what it got and corrects the difference. For a value-based agent the correction is the Q-learning update:

Q(s, a) ← Q(s, a) + α [ r + γ · max_a′ Q(s′, a′) − Q(s, a) ]

Here Q(s, a) is the estimated value of action a in state s, α is the learning rate, r is the reward, γ is the discount factor, and s′ is the next state. The term in brackets is the gap between the new target and the old estimate. Policy-based agents instead adjust policy parameters so that actions followed by higher-than-expected return become more likely. The full derivation with numbers is in How Does Reinforcement Learning Work?

Exploration vs Exploitation in Agent Decision-Making

Every decision forces a trade-off. Exploiting the best-known action earns reward now. Exploring an unfamiliar action might reveal something better. An agent that never explores can lock onto a mediocre route. An agent that only explores never benefits from what it has learned. Many agents start with high exploration and reduce it over time. In the code below, a fixed ε of 0.2 keeps the example simple.

Example: Building a Maze-Solving Agent

Designing an RL agent follows a fixed recipe. For the 3×3 maze used throughout this series:

1. Define the environment. A 3×3 grid, start at the bottom-left, goal at the top-right, wall in the center.
2. Define states and actions. Nine cells (one is a wall) and four moves.
3. Define the reward. −1 per move, +10 for reaching the goal. Moving into a wall leaves the agent in place and still costs −1.
4. Choose an algorithm. Tabular Q-learning is enough for nine cells.
5. Train over episodes. Repeat: act, observe, update.
6. Evaluate. Turn off exploration and follow the learned policy.

Unlike A* search, which needs the full map and the move costs in advance, this agent learns the route from experience.

Simple Python Example of an RL Agent

The script below contains the environment (step) and a QAgent class. It runs in plain Python with no extra libraries.

import random

ACTIONS = ["up", "down", "left", "right"]
MOVES = {"up": (-1, 0), "down": (1, 0), "left": (0, -1), "right": (0, 1)}
START, GOAL, WALL = (2, 0), (0, 2), (1, 1)

def step(state, action):
    """Environment: apply the action, return (next_state, reward, done)."""
    dr, dc = MOVES[action]
    nxt = (state[0] + dr, state[1] + dc)
    if not (0 <= nxt[0] < 3 and 0 <= nxt[1] < 3) or nxt == WALL:
        nxt = state                      # blocked: stay in place
    if nxt == GOAL:
        return nxt, 10, True
    return nxt, -1, False

class QAgent:
    def __init__(self, alpha=0.5, gamma=0.9, epsilon=0.2):
        self.q = {}                      # Q-table: (state, action) -> value
        self.alpha, self.gamma, self.epsilon = alpha, gamma, epsilon

    def value(self, state, action):
        return self.q.get((state, action), 0.0)

    def select_action(self, state):
        """Policy: epsilon-greedy over the current Q-values."""
        if random.random() < self.epsilon:
            return random.choice(ACTIONS)                      # explore
        best = max(self.value(state, a) for a in ACTIONS)
        return random.choice([a for a in ACTIONS if self.value(state, a) == best])  # exploit

    def learn(self, state, action, reward, next_state, done):
        best_next = 0.0 if done else max(self.value(next_state, a) for a in ACTIONS)
        target = reward + self.gamma * best_next
        old = self.value(state, action)
        self.q[(state, action)] = old + self.alpha * (target - old)

random.seed(0)
agent = QAgent()
for episode in range(300):
    state, done, steps = START, False, 0
    while not done and steps < 50:
        action = agent.select_action(state)
        next_state, reward, done = step(state, action)
        agent.learn(state, action, reward, next_state, done)
        state, steps = next_state, steps + 1

# Follow the greedy policy after training
agent.epsilon = 0.0
state, path, total, done = START, [START], 0, False
while not done and len(path) < 20:
    action = agent.select_action(state)
    state, reward, done = step(state, action)
    path.append(state); total += reward
print("Path:", path)
print("Total reward:", total)

How the code works

• step is the environment. It takes a state and an action and returns the next state, the reward, and whether the episode ended. The agent never sees how this works inside, only the results.
• QAgent is the agent. It owns the Q-table (self.q) and the learning settings α, γ, and ε.
• select_action is the policy. With probability ε it returns a random action (exploration). Otherwise it finds the highest Q-value for the state and picks one of the actions that achieve it, breaking ties randomly (exploitation).
• learn is the learning rule. It computes the target (reward plus discounted best next value) and moves the stored value toward it by a fraction α.
• The training loop runs 300 episodes of act, observe, learn, with a cap of 50 steps per episode.
• The final loop sets ε to 0 so the agent only exploits, then records the path it follows.

When run, the script prints a four-move path from (2, 0) up to (0, 0) and across to the goal at (0, 2), with a total reward of 7. The learned Q-values for the four steps along this path settle at 4.58, 6.2, 8.0, and 10.0, matching the values derived by hand in the previous article. Another random seed can change the exact route taken, since a second shortest route also exists, but the values converge to the same numbers.

Difference Between an RL Agent and a Traditional AI System

“Traditional AI system” can mean several things, so here are the common cases:

System	How it decides	How it improves
Rule-based system	Follows rules written by people	People edit the rules
Search or planning (for example A*)	Searches a known model of the problem	Does not learn; uses a given model
Supervised model	Predicts an output from the input	Trains on labeled examples; see RL vs supervised learning
RL agent	Chooses actions with a learned policy	Learns from rewards earned by acting

The defining trait of an RL agent is the closed loop: its actions change what it observes, and it learns from that feedback. For how this differs from learning without a target, see Reinforcement Learning vs Unsupervised Learning.

One naming caution: in current language-model discussions, an “agent” often means a language model that calls tools and takes multi-step actions. That agent is not necessarily trained with reinforcement learning. For a practical example of that meaning, see How to Build a Local AI Agent With an Open-Weight LLM.

Applications of RL Agents

• Games: agents trained with RL have played Atari games and board games such as Go.
• Robotics: locomotion and manipulation skills, usually learned first in simulation.
• Resource management: scheduling, network control, and energy systems.
• Recommendation: treating a user session as a sequence of decisions.
• Language model tuning: reinforcement learning from human feedback uses a policy being optimized against a learned reward.

Common Challenges

• Reward design. The agent maximizes the reward as written, which may differ from what you meant.
• Exploration. Sparse rewards make useful behavior hard to discover.
• Sample efficiency. Learning can require a very large number of interactions.
• Partial observability. The agent may see only part of the true state.
• Safety. Mistakes made while learning can be costly outside a simulator.

Frequently Asked Questions

Is the agent the same as the policy?

No. The policy is the agent’s rule for choosing actions. The agent also includes how that rule is learned and whatever it stores, such as value estimates or a model.

Is the reward function part of the agent?

No. The reward comes from the environment. The agent cannot change it, which is what makes reward design important.

Can there be more than one agent?

Yes. Multi-agent reinforcement learning studies several agents learning together, either cooperating or competing.

Does an agent need a neural network?

No. The example above uses a plain table. Neural networks are needed when the state space is too large for a table.

Does an agent learn forever?

Not necessarily. Many systems train an agent, then freeze the policy and deploy it. Others keep learning online.

Conclusion

The agent is the decision-making entity in reinforcement learning. It perceives states, picks actions through a policy, and learns from rewards, while the environment supplies the states and the rewards. Keeping agent, environment, and policy distinct makes every RL algorithm easier to follow. Return to the complete guide to reinforcement learning to see how these pieces fit the wider field.