coopihc.agents.lqrcontrollers.IHDT_LQRController.IHDT_LQRController

class IHDT_LQRController(role, Q, R, Acontroller=None, Bcontroller=None)[source]

Bases: coopihc.agents.lqrcontrollers.LQRController.LQRController

Infinite Horizon Discrete Time LQR

An Infinite Horizon (i.e. planning for unicode:: U+221E .. steps) Discrete Time implementation of the LQR controller. The controller is computed to minimize costs :math: X^tQX + u^t R u, where X is the state of the system and u is the linear feedback command \(u = -K X\), where the feedback gain \(K\) is given by solving the discrete ARE

\[\begin{align} K = (R + B^tPB)^{-1}B^TPA \text{ (gain)}\ P = Q + A^tPA - A^tPB(R + B^tPB)^{-1}B^TPA \text{Discrete ARE} \end{align}\]

Parameters

role (string) – “user” or “assistant”
Q (numpy.ndarray) – see LQRController
R (numpy.ndarray) – see LQRController
Acontroller (numpy.ndarray) – Model of A used by the controller to compute K
Bcontroller (numpy.ndarray) – Model of B used by the controller to compute K

Methods

`finit`	Uses Discrete Algebraic Ricatti Equation to get P
`infer`	infer the agent's internal state
`observe`	produce an observation
`prepare_action`
`render`	Displays actions selected by the LQR agent.
`reset`	reset the agent --- Override this
`reset_all`	reset the agent and all its components
`take_action`	Select an action

Attributes

`action`	Last agent action
`assistant`	Connected assistant
`bundle`
`bundle_memory`
`inference_engine`	Agent inference engine
`observation`	Last agent observation
`observation_engine`	Agent observation engine
`parameters`
`policy`	Agent policy
`state`	Agent internal state
`task`	Connected task
`user`	Connected user

property action: Last agent action

property assistant: Connected assistant

finit()[source]

Uses Discrete Algebraic Ricatti Equation to get P

infer(agent_observation=None, affect_bundle=True)

infer the agent’s internal state

Infer the new agent state from the agent’s observation. By default, the agent will select the agent’s last observation. To bypass this behavior, you can provide a given agent_observation. The affect_bundle flag determines whether or not the agent’s internal state is actually updated.

Parameters

agent_observation (:py:class:State<coopihc.base.State>, optional) – last agent observation, defaults to None. If None, gets the observation from the inference engine’s buffer.
affect_bundle (bool, optional) – whether or not the agent’s state is updated with the new inferred state, defaults to True.

property inference_engine: Agent inference engine

property observation: Last agent observation

property observation_engine: Agent observation engine

observe(game_state=None, affect_bundle=True, game_info={}, task_state={}, user_state={}, assistant_state={}, user_action={}, assistant_action={})

produce an observation

Produce an observation based on state information, by querying the agent’s observation engine. By default, the agent will find the appropriate states to observe. To bypass this behavior, you can provide state information. When doing so, either provide the full game state, or provide the needed individual states. The affect_bundle flag determines whether or not the observation produces like this becomes the agent’s last observation.

Parameters

game_state (:py:class:State<coopihc.base.State>, optional) – the full game state as defined in the CoopIHC interaction model, defaults to None.
affect_bundle (bool, optional) – whether or not the observation is stored and becomes the agent’s last observation, defaults to True.
game_info (:py:class:State<coopihc.base.State>, optional) – game_info substate, see the CoopIHC interaction model, defaults to {}.
task_state (:py:class:State<coopihc.base.State>, optional) – task_state substate, see the CoopIHC interaction model, defaults to {}
user_state (:py:class:State<coopihc.base.State>, optional) – user_state substate, see the CoopIHC interaction model, defaults to {}
assistant_state (:py:class:State<coopihc.base.State>, optional) – assistant_state substate, see the CoopIHC interaction model, defaults to {}
user_action (:py:class:State<coopihc.base.State>, optional) – user_action substate, see the CoopIHC interaction model, defaults to {}
assistant_action (:py:class:State<coopihc.base.State>, optional) – assistant_action substate, see the CoopIHC interaction model, defaults to {}

property policy: Agent policy

render(*args, **kwargs): Displays actions selected by the LQR agent.

reset()

reset the agent — Override this

Override this method to specify how the components of the agent will be reset. By default, the agent will already call the reset method of all 4 components (policy, inference engine, observation engine, state). You can specify some added behavior here e.g. if you want each game to begin with a specific state value, you can specify that here. For example:

# Sets the value of state 'x' to 0
def reset(self):
    self.state["x"][...] = 123

reset_all(dic=None, random=True)

reset the agent and all its components

In addition to running the agent’s reset(), reset_all() also calls state, observation engine, inference engine and policies’ reset() method.

Parameters

dic (dictionary, optional) – reset_dictionnary, defaults to None. See the reset() method in py:class:Bundle<coopihc.bundle.Bundle> for more information.
random (bool, optional) – whether states should be randomly reset, defaults to True. See the reset() method in py:class:Bundle<coopihc.bundle.Bundle> for more information.

property state: Agent internal state

take_action(agent_observation=None, agent_state=None, increment_turn=True)

Select an action

Select an action based on agent_observation and agent_state, by querying the agent’s policy. If either of these arguments is not provided, then the argument is deduced from the agent’s internals.

Parameters

agent_observation (:py:class:State<coopihc.base.State>, optional) – last agent observation, defaults to None. If None, gets the observation from the inference engine’s buffer.
agent_state (:py:class:State<coopihc.base.State>, optional) – current value of the agent’s internal state, defaults to None. If None, gets the state from itself.
increment_turn (bool, optional) – whether to update bundle’s turn and round

property task: Connected task

property user: Connected user