Chapter 2: Intelligent Agents (3751)

Chapter 2: Intelligent Agents (3751)

1. Rational agent--one that behaves as well as possible.

   • depends on the nature of the environment

2. 2.1 Agents and environments

   • An agent is anything that can be viewed as perceiving its environment through sensors and acting upon that environment through actuators.
   • We use the term percept to refer to the agent's perceptual inputs at any given instant.
   • percept sequence is the complete history of everything the agent has ever perceived
   • an agent's choice of action at any given instant can depend on the entire percept sequence observed to date, but not on anything it hasn't perceived
   • agent's behavior is described by the agent function that maps any given percept sequence to an action.
   • Percept sequence -> History of stimulus coming from the environment
     • Agent function ->Externally a table can be created which shows each percept sequence to some action
     • Agent function -> Internally the agent program implements the function
   • The agent function is an abstract mathematical description; the agent program is a concrete implementation, running within some physical system

3. 2.2 Good Behaviour: The concept of rationality

   • A rational agent is one that does the right thing--conceptually speaking
     • by considering the consequences of the agent's behavior.
   • Actions in response to percept of environment leads to different environmental states, can be desirable or not
     • Not agent states, as the agent can always conclude that it had perfect performance
   • If the sequence is desirable, then the agent has perfonned well. This notion of desirability is captured by a performance measure that evaluates any given sequence of environment states.
   • a general rule, it is better to design performance measures according to what one actually wants in the environment, rather than according to how one thinks the agem should behave
   • 2.2.1 Rationality
     • • The performance measure that defines the criterion of success. • The agent's prior knowledge of the environment. • The actions that the agent can perform. • The agent's percept sequence to date.
     • definition of a rational agent:
       • For each possible percept sequence, a rational agent should select an action that is expected to maximize its performance measure, given the evidence provided by the percept sequence and whatever built-in knowledge the agent has.
   • 2.2.2 Omniscience, learning, and autonomy
     • An omniscient agent knows the actual outcome of its actions and can act accordingly; but omniscience is impossible in reality.
     • Rationality maximizes expected performance, while perfection maximizes actual performance.
     • Doing actions in order to modify future percepts-sometimes called information gathering-is an important part of rationality
     • A rational agent should be autonomous-it should learn what it can to compensate for partial or incorrect prior knowledge.
     • the extent that an agent relies on the prior knowledge of its designer rather than on its own percepts, we say that the agent lacks autonomy

4. 2.3 The nature of environments

   • task environments, which are essentially the "problems" to which rational agents are the "solutions."
   • 2.3.1 Specifying the task environment
     • we can this the PEAS (Performance, Environment, Actuators, Sensors) description
     • the first step must always be to specify the task environment as fully as possible.
   • 2.3.2 Properties of task environments
     • Fully observable vs. partially observable:
       • agent's sensors give it access to the complete state of the environment at each point in time, then we say that the task environment is fully observable.
         • detect all aspects that are relevant to the choice of action; relevance, in turn, depends on the perfonnance measure.
         • Fully observable environments are convenient because the agent need not maintain any internal state to keep track of the world
       • partially observable because of noisy and inaccurate sensors or because parts of the state are simply missing from the sensor data
       • If the agent has no sensors at all then the environment is unobservable
     • Single agent vs. multiagent:
       • Single -> eg. Agent working on crossword puzzle
       • Multi -> eg. More than one Agent in the environment
       • chess is a competitive multiagent environment.
       • driving is a cooperative multiagent environment.
       • communication often emerges as a rational behavior in multiagent environments; in some competitive environments, randomized behavior is rational because it avoids the pitfalls ofpredictability
     • Deterministic vs. stochastic
       • Deterministic -> If the next state of the environment is completely determined by the current state and the action executed by the agent
       • Stochastic
         • "stochastic" generally implies that uncertainty about outcomes is quantified in terms of probabilities; a nondeterministic environment is one in which actions are characterized by their possible outcomes, but no probabilities are attached to them
       • Fully observable environment -> Deterministic
       • Partially observable environments -> Appear stochastic, due to uncertainty
       • Environment is uncertain if it is not fully observable or not deterministic
     • Episodic vs. sequential:
       • Episodic -> the agent's experience is divided into atomic episodes. In each episode the agent receives a percept and then performs a single action. eg. checking car parts
         • the next episode does not depend on the actions taken in previous episodes.
       • Sequential -> the current decision could affect all future decisions.
     • Static vs. dynamic:
       • Dynamic -> environment can change while an agent is deliberating
       • Static -> Doesnt change
       • semidynamic -> Static environment, but agent performance score changes
     • Discrete vs. continuous:
       • applies to the state ofthe environment, to the way time is handled, and to the percepts and actions of the agent
     • Known vs. unknown
       • this distinction refers not to the environment itself but to the agent's (or designer's) state of knowledge about the "laws of physics" of the environment.
     • the hardest case is partially observable, multiagent, stochastic, sequential, dynamic, continuous, and unknown

5. 2.4 The structure of agents

   • agent = architecture +program .
   • Architecture -> physical hardware
   • Agent program -> Agent function -> Mapping from precepts to action
   • 2.4.1 Agent programs
     • Notice the difference between the agent program, which takes the current percept as input, and the agent function, which takes the entire percept history.
     • Figure 2.7 shows a rather trivial agent program that keeps track of the percept sequence and then uses it to index into a table of actions to decide what to do
       • The size of this table gets too large to work in complex environments
     • four basic kinds of agent programs that embody the principles underlying almost all intelligent systems:
       • • Simple reflex agents • Model-based reflex agents • Goal-based agents • Utility-based agents.
     • 2.4.5 Utility-based agents
       • Goals alone are not enough to generate high-quality behavior in most environments
       • Goals just provide a crude binary distinction between "happy" and "unhappy" states.
         • computer scientists use the term utility instead.
       • An agent's utility function is essentially an internalization of the performance measure.
       • Partial observability and stochasticity are ubiquitous in the real world, and so, therefore, is decision making w1der uncertainty.
       • a rational utility-based agent chooses the action that maximizes the expected utility of the action outcomes-that is, the utility the agent expects to derive, on average, given the probabilities and utilities of each outcome.
       • A utility-based agent has to model and keep track of its environment
   • 2.4.2 Simple reflex agents
     • The simplest kind
     • select actions on the basis of the current percept, ignoring the rest of the percept history
     • Also occurs in complex environments, in cases where percept to action is simple
       • condition-action rule
         • IF car-in-front-is-braking THEN initiate-braking
         • Also called situation-action rules, productions, or if-then rules
     • will work only if the correct decision can be made on the basis ofonly the currentpercept-that is, only if the environment is fully observable.
   • 2.4.3 Model-based reflex agents
     • The most effective way to handle partial observability is for the agent to keep track of the part ofthe world it can't see now.
     • the agent should maintain some sort of internal state that depends on the percept history and thereby reflects at least some of the unobserved aspects of the current state.
       • Updating this internal state information as time goes by requires two kinds of knowledge to be encoded in the agent program
         • how the world evolves independently of the agent-for example, that an overtaking car generally will be closer behind than it was a moment ago.
         • how the agent's own actions affect the world-for exrunple, that when the agent turns the steering wheel clockwise, the car turns to the right
       • knowledge about "how the world works" is called a model of the world.
   • 2.4.4 Goal-based agents
     • as well as a current state description, the agent needs some sort of goal information that desctibes situations that are desirable
     • Search (Chapters 3 to 5) and planning (Chapters 10 and 11) are the subfields of AI devoted to finding action sequences that achieve the agent's goals
     • it is more flexible because the knowledge that supports its decisions is represented explicitly and can be modified.
   • 2.4.6 Learning agents
     • A learning agent can be divided into four conceptual components
     • important distinction is between the learning element, which is responsible for making improvements, and the performance element, which is responsible for selecting extemal actions.
       • The performance element is what we have previously considered to be the entire agent: it takes in percepts and decides on actions.
       • The learning element uses feedback from the critic on how the agent is doing and determines how the performance element should be modified to do better in the future
       • The design ofthe learning element depends very much on the design of the performance element
     • The critic tells the learning agent how well the agent is doing in respect to a fixed performance standard.
     • The last component of the learning agent is the problem generator.
       • responsible for suggesting actions that will lead to new and informative experiences.
       • if the agent is willing to explore a little and do some perhaps suboptimal actions in the short run, it might discover much better actions for the long run.
         • The problem generator suggests these exploratory options
       • The learning element can make changes to any of the "knowledge" components shown in the agent diagrams
     • a process of modification of each component of the agent to bring the components into closer agreement with the available feedback information, thereby improving the overall performance of the agent.
   • 2.4.7 How the components of agent programs work
     • Algorithm underlying search and game-playing (Chapters 3-5), Hidden Markov models (Chapter 15), and Markov decision processes (Chapter 17) all work with atomic representations
     • Atomic - > Taxi driving from one city to the next. State only contains atomic knowledge, of the city name
     • Factored -> Taxi must track fuel, tolls, GPS, engine lights etc
       • A factored representation splits up each state into a fixed set of variables or attributes
       • Many important areas of AI are based on factored representations, including constraint satisfaction algorithms (Chapter 6), propositional logic (Chapter 7), planning (Chapters lO and 11), Bayesian networks (Chapters 13-16), and the machine learning algorithms in Chapters 18, 20, and 21.
     • Structured
       • Structured representations underlie relational databases and first-order logic (Chapters 8, 9, and 12), first-order probability models (Chapter 14), knowledge-based learning (Chapter 19) and much of natural language understanding (Chapters 22 and 23).
       • For many purposes, we need to understand the world as having things in it that are related to each other, not just variables with values.
     • intelligent systems for the real world mayneed to operate at all points along the axis simultaneously.

Images and files in the map