- Agents
- An agent exists and performs intelligent behaviors,
e.g., makes decisions and takes actions, in a specific
environment and interacts with that environment via
a sense-understand-plan-act loop, i.e.,
- An agent perceives its environment through sensors, e.g., camera, touch, taste, keyboard input; these sensors provide percepts, which form percept sequences over time.
- Agents act in their environment through actuators, e.g., limbs, tools, allowable moves in a game; actuators perform actions that can modify the environment (including the agent's internal state and/or position in the environment).
- An agent's choice of action is specified as an
agent function and implemented as an agent
program relative to a particular architecture.
- Agent functions are also known as policies and there are various names for agent programs depending on the application, e.g., controllers (robotics).
- In general, an agent's action-choice can depend on its entire percept-history to date; however, we will in this course focus an agents whose action-choices depend on the current percept.
- There are typically many ways to implement an agent function, e.g., lookup tables, IF-THEN statement blocks, neural networks.
- An agent may have knowledge about its environment, e,g., environment model, game rules / allowable moves.
- EXAMPLE: Some example agents:
- Roomba vacuum cleaner
- Human players in a game of chess
- Self-driving vehicle
- AI personal assistant
- An agent may have multiple internal states, with percept / action pairs between these states encoded in a state-transition function.
- EXAMPLE: Three deterministic finite-state agents that
operate in a 2D grid-world (Wareham and Vardy (2018)).
- State transitions are specified as (state, percept, action, state) tuples, where an action is specified as an (environment-change, agent move) pair.
- Agent processing starts in state q0.
- What does each of these agents do?
- EXAMPLE: A stochastic agent
- Specified as a table whose x- and y-axes are percepts and table entries are probability-vectors specifying the probabilities of a set of actions relative to a set of x/y-percept combinations.
- Consider the following agent which specifies the
probabilities of
what to do (wear coat, carry umbrella, do nothing
extra) if you want to take a walk outside
depending on the weather ((rainy / cloudy / sunny)
x (cold / neutral / hot)):
- A rational agent does the best possible action
at each point wrt optimizing some pre-defined
performance measure; such an agent has an optimal
policy.
- Note that rationality is NOT omniscience, i.e., knowing the complete environment and the outcomes of all (even random) actions.
- EXAMPLE: Going across street at crosswalk to see friend vs not crossing street because of incoming meteor from space.
- Four basic types of agents:
- Simple reflex agents
- This simplest type of agent reacts without understanding or planning by using sets of encoded IF-THEN rules that directly link percepts and actions, e.g., the deterministic finite-state and stochastic agents given above.
- The intelligence encoded in such agents can be impressive, e.g., Brooks' Genghis robot, basic obstacle avoidance in self-driving vehicles, but are ultimately limited in that there is little understanding or planning in the action-selection process.
- Model-based agents
- This type of agent maintains a non-trivial internal state that incorporates knowledge of how the world works and a model of the world encountered to date (including the agent's own past actions in the world, e.g., the two-state deterministic finite-state agent given above).
- The intelligence encoded in such agents now allows understanding to be part of the action-selection process.
- Goal-based agents
- This type of agent incorporates a description of an agent's wanted goal, e.g., the destination to be reached, as well as a way of evaluating if that goal has been achieved, e.g., are we at the requested destination?
- Typically more complex than simple reflex agents but much more flexible in terms of the intelligence it can encode (in particular, planning as part of the action-selection process).
- Utility-based agents
- This type of agent incorporates a utility function that internalizes some performance measure, e.g., the distance traveled on a particular path from the start point to the destination, and is thus capable of optimizing this performance measure, e.g., choosing the shortest path from a start point to the destination.
- Such agents are more complex still but can (given an appropriate utility function) encode general rationality.
- Note that choosing the appropriate utility function is crucial to the encoded degree of rationality and this is typically very hard to do.
- Simple reflex agents
- The intelligence in the four types of agents above is hard-coded using hand-crafted algorithms. The most complex type of agent can learn from experience and both become rational and maintain this rationality in the face of changing environments. The resulting agent programs are machine-crafted and will be discussed in the final lectures of this course.
- An agent exists and performs intelligent behaviors,
e.g., makes decisions and takes actions, in a specific
environment and interacts with that environment via
a sense-understand-plan-act loop, i.e.,