coopihc.agents.lqrcontrollers.IHCT_LQGController.IHCT_LQGController

class IHCT_LQGController(role, timestep, Q, R, U, C, D, *args, noise='on', Acontroller=None, Bcontroller=None, F=None, G=None, H=None, **kwargs)[source]

Bases: coopihc.agents.BaseAgent.BaseAgent

Infinite Horizon Continuous Time LQ Gaussian Controller.

An Infinite Horizon (Steady-state) LQG controller, based on [Phillis1985] and [Qian2013].

For the a task where state ‘x’ follows a linear noisy dynamic:

\[\begin{split}\begin{align} x(+.) = (Ax(.) + Bu(.))dt + Fx(.).d\beta + G.d\omega + Hu(.)d\gamma \\ \end{align}\end{split}\]

the LQG controller produces the following observations dy and commands u minimizing cost J:

\[\begin{split}\begin{align*} dy & = Cxdt + Dd\xi \\ d\hat{x} & = (A \hat{x} + Bu) dt + K (dy - C\hat{x}dt) \\ u & = - L\hat{x} \\ \tilde{x} & = x - \hat{x} \\ J & \simeq \mathbb{E} [\tilde{x}^T U \tilde{x} + x^TQx + u^TRu] \end{align*}\end{split}\]
Phillis1985

Phillis, Y. “Controller design of systems with multiplicative noise.” IEEE Transactions on Automatic Control 30.10 (1985): 1017-1019. Link

Qian2013

Qian, Ning, et al. “Movement duration, Fitts’s law, and an infinite-horizon optimal feedback control model for biological motor systems.” Neural computation 25.3 (2013): 697-724. Link

Parameters

  • role (string) – “user” or “assistant”

  • timestep (float) – duration of timestep

  • Q (numpy.ndarray) – State cost

  • R (numpy.ndarray) – Control cost

  • U (numpy.ndarray) – Estimation error cost

  • C (numpy.ndarray) – Observation matrix

  • D (numpy.ndarray) – Observation noise matrix

  • noise (str, optional) – whether or not to have, defaults to “on”

  • Acontroller (numpy.ndarray, optional) – Representation of A for the agent. If None, the agent representation of A is equal to the task A, defaults to None.

  • Bcontroller (numpy.ndarray, optional) – Representation of B for the agent. If None, the agent representation of B is equal to the task B, defaults to None.

  • Fcontroller (numpy.ndarray, optional) – Representation of F for the agent. If None, the agent representation of F is equal to the task F, defaults to None.

  • Gcontroller (numpy.ndarray, optional) – Representation of G for the agent. If None, the agent representation of G is equal to the task G, defaults to None.

  • Hcontroller (numpy.ndarray, optional) – Representation of H for the agent. If None, the agent representation of H is equal to the task H, defaults to None.

Methods

counted_decorator

Decorator that counts the number of times function f has been called

finit

Get and compute needed matrices.

infer

infer the agent's internal state

observe

produce an observation

prepare_action

render

render the agent

reset

reset the agent --- Override this

reset_all

reset the agent and all its components

take_action

Select an action

Attributes

action

Last agent action

assistant

Connected assistant

bundle

bundle_memory

inference_engine

Agent inference engine

observation

Last agent observation

observation_engine

Agent observation engine

parameters

policy

Agent policy

state

Agent internal state

task

Connected task

user

Connected user
property action

Last agent action

property assistant

Connected assistant

counted_decorator()[source]

Decorator that counts the number of times function f has been called

Parameters

f (function) – decorated function

finit()[source]

Get and compute needed matrices.

  1. Take A, B, F, G, H from task if not provided by the end-user

  2. Create an \(\hat{x}\) state;

  3. attach the model dynamics to the inference engine if needed

  4. compute K and L;

  5. set K and L in inference engine and policy

infer(agent_observation=None, affect_bundle=True)

infer the agent’s internal state

Infer the new agent state from the agent’s observation. By default, the agent will select the agent’s last observation. To bypass this behavior, you can provide a given agent_observation. The affect_bundle flag determines whether or not the agent’s internal state is actually updated.

Parameters

  • agent_observation (:py:class:State<coopihc.base.State>, optional) – last agent observation, defaults to None. If None, gets the observation from the inference engine’s buffer.

  • affect_bundle (bool, optional) – whether or not the agent’s state is updated with the new inferred state, defaults to True.

property inference_engine

Agent inference engine

property observation

Last agent observation

property observation_engine

Agent observation engine

observe(game_state=None, affect_bundle=True, game_info={}, task_state={}, user_state={}, assistant_state={}, user_action={}, assistant_action={})

produce an observation

Produce an observation based on state information, by querying the agent’s observation engine. By default, the agent will find the appropriate states to observe. To bypass this behavior, you can provide state information. When doing so, either provide the full game state, or provide the needed individual states. The affect_bundle flag determines whether or not the observation produces like this becomes the agent’s last observation.

Parameters

  • game_state (:py:class:State<coopihc.base.State>, optional) – the full game state as defined in the CoopIHC interaction model, defaults to None.

  • affect_bundle (bool, optional) – whether or not the observation is stored and becomes the agent’s last observation, defaults to True.

  • game_info (:py:class:State<coopihc.base.State>, optional) – game_info substate, see the CoopIHC interaction model, defaults to {}.

  • task_state (:py:class:State<coopihc.base.State>, optional) – task_state substate, see the CoopIHC interaction model, defaults to {}

  • user_state (:py:class:State<coopihc.base.State>, optional) – user_state substate, see the CoopIHC interaction model, defaults to {}

  • assistant_state (:py:class:State<coopihc.base.State>, optional) – assistant_state substate, see the CoopIHC interaction model, defaults to {}

  • user_action (:py:class:State<coopihc.base.State>, optional) – user_action substate, see the CoopIHC interaction model, defaults to {}

  • assistant_action (:py:class:State<coopihc.base.State>, optional) – assistant_action substate, see the CoopIHC interaction model, defaults to {}

property policy

Agent policy

render(mode='text', ax_user=None, ax_assistant=None, ax_task=None)

render the agent

Displays agent information on the passed axes.

Parameters

  • mode (str, optional) – display mode, defaults to “text”. Also supports “plot”.

  • ax_user (Matploblib axis, optional) – user axis, defaults to None

  • ax_assistant (Matploblib axis, optional) – assistant axis, defaults to None

  • ax_task (Matploblib axis, optional) – task axis, defaults to None

reset()

reset the agent — Override this

Override this method to specify how the components of the agent will be reset. By default, the agent will already call the reset method of all 4 components (policy, inference engine, observation engine, state). You can specify some added behavior here e.g. if you want each game to begin with a specific state value, you can specify that here. For example:

# Sets the value of state 'x' to 0
def reset(self):
    self.state["x"][...] = 123
reset_all(dic=None, random=True)

reset the agent and all its components

In addition to running the agent’s reset(), reset_all() also calls state, observation engine, inference engine and policies’ reset() method.

Parameters

  • dic (dictionary, optional) – reset_dictionnary, defaults to None. See the reset() method in py:class:Bundle<coopihc.bundle.Bundle> for more information.

  • random (bool, optional) – whether states should be randomly reset, defaults to True. See the reset() method in py:class:Bundle<coopihc.bundle.Bundle> for more information.

property state

Agent internal state

take_action(agent_observation=None, agent_state=None, increment_turn=True)

Select an action

Select an action based on agent_observation and agent_state, by querying the agent’s policy. If either of these arguments is not provided, then the argument is deduced from the agent’s internals.

Parameters

  • agent_observation (:py:class:State<coopihc.base.State>, optional) – last agent observation, defaults to None. If None, gets the observation from the inference engine’s buffer.

  • agent_state (:py:class:State<coopihc.base.State>, optional) – current value of the agent’s internal state, defaults to None. If None, gets the state from itself.

  • increment_turn (bool, optional) – whether to update bundle’s turn and round

property task

Connected task

property user

Connected user