Introduction
Artificial intelligence is the study of agents: programs that perceive their surroundings and choose actions to reach their goals. A chess program, a spam filter, a robot vacuum and a route planner are all agents. This course builds the main techniques that let agents decide well: searching for plans, playing games, solving constraints, reasoning with logic and with probabilities, and learning from reward. It starts with the vocabulary for describing any agent.
How it works
An agent receives a percept at each step and returns an action. The rule that maps what it has perceived to what it does is its agent function. Whether it is doing well is judged by a performance measure, chosen by whoever designed the task, for example "+10 for each dirty square cleaned, −1 for each move".
A task is described with PEAS:
- Performance measure: what counts as success;
- Environment: the world the agent lives in;
- Actuators: how it acts (wheels, a suction motor, moves on a board);
- Sensors: what it can perceive (a dirt sensor, a camera, the opponent's move).
Environments differ in ways that matter:
- fully or partially observable: can the agent see everything relevant?
- deterministic or stochastic: does an action always have the same result?
- episodic or sequential: do earlier decisions affect later ones?
- static or dynamic, discrete or continuous, single-agent or multi-agent.
There are four classic designs, each more capable than the last:
- a simple reflex agent acts on the current percept only, with rules such as "if dirty, suck";
- a model-based agent keeps a model of what it cannot see, such as which squares it has already found clean;
- a goal-based agent plans a sequence of actions that reaches a goal, which needs search;
- a utility-based agent chooses between goals by how good each outcome is.
Example
def reflex_agent(location, dirty, width):
if dirty:
return "Suck"
return "Right" if location < width - 1 else "Left"
class ModelAgent:
def __init__(self, width):
self.width = width
self.known_clean = set()
def __call__(self, location, dirty, width):
if dirty:
return "Suck"
self.known_clean.add(location)
if len(self.known_clean) == self.width:
return "NoOp"
return "Right" if location < self.width - 1 else "Left"
def run(agent, world, steps):
world = [cell == "D" for cell in world]
location, actions, cleaned, moves = 0, [], 0, 0
for _ in range(steps):
action = agent(location, world[location], len(world))
actions.append(action)
if action == "Suck":
world[location] = False
cleaned += 1
elif action in ("Left", "Right"):
location += 1 if action == "Right" else -1
moves += 1
return actions, 10 * cleaned - moves
for name, agent in [("reflex agent", reflex_agent), ("model-based agent", ModelAgent(5))]:
actions, score = run(agent, "D..DD", steps=10)
print(f"{name}: {' '.join(actions)}")
print(f" score {score}")Line by line
- The world is a corridor of five squares;
Dis dirt. The score is 10 for each square cleaned minus 1 for each move. reflex_agentsees only its location and whether its square is dirty, so after cleaning it keeps walking forever.ModelAgentremembers which squares it has found clean; when it knows all five are clean it does nothing (NoOp).- Both clean the three dirty squares in the same order, because they move the same way.
- The reflex agent wastes three moves at the end, so the model-based agent scores higher: 26 against 23.
Expected result
reflex agent: Suck Right Right Right Suck Right Suck Left Right Left
score 23
model-based agent: Suck Right Right Right Suck Right Suck NoOp NoOp NoOp
score 26
In the real world
A thermostat is a simple reflex agent: too cold means heat. A robot vacuum is model-based: it builds a map of the room as it goes. A navigation app is goal-based, and a self-driving car is utility-based, trading off speed, comfort and safety. Game characters, trading systems and recommendation engines are all described in the same terms.
Common mistakes
- Judging an agent by its code instead of by its performance measure.
- Using a simple reflex agent in a partially observable world, where it cannot know when the job is done.
- Choosing a performance measure that rewards the wrong thing, such as rewarding sucking, so the agent sucks the same square forever.
- Ignoring the cost of actions, so an agent that wanders aimlessly looks as good as one that stops.
Best practices
- Write down the PEAS description before writing any code.
- Classify the environment first; it tells you which kind of agent is needed.
- Measure the agent over many runs and situations, not just one.
Summary
An agent maps percepts to actions and is judged by a performance measure. PEAS describes a task, and the properties of the environment, above all how much the agent can observe, decide which design works. Simple reflex agents react; model-based agents remember; goal-based agents plan; utility-based agents weigh outcomes.
Quick check
Key takeaways
- An agent is judged by its performance measure, not by how clever its code looks.
- A simple reflex agent reacts only to the current percept; a model-based agent also remembers.
- The kind of environment decides which kind of agent can do well.
Practice problems for this lesson
My notes
Write down what you want to remember from this lesson. Notes are saved on this device as you type, and appear on your dashboard.