coopihc.interactiontask.ClassicControlTask.ClassicControlTask
- class ClassicControlTask(timestep, A, B, *args, F=None, G=None, H=None, discrete_dynamics=True, noise='on', timespace='discrete', end='standard', **kwargs)[source]
Bases:
coopihc.interactiontask.InteractionTask.InteractionTaskA task used for a classic control setting with signal dependent and independent noise. You can account for control-dependent noise with an appropriate noise model in the policy or the observation engine.
The task has a state x(.) which evolves according to
\[\begin{split}\begin{align} x(+.) = Ax(.) + Bu(.) + Fx(.).d\beta + G.d\omega + Hu(.)d\gamma \\ \end{align}\end{split}\]for “timespace=discrete” and
\[\begin{split}\begin{align} x(+.) = (Ax(.) + Bu(.))dt + Fx(.).d\beta + G.d\omega + Hu(.)d\gamma \\ \end{align}\end{split}\]for “timespace=continuous”.
where :math:
u(.)is the user action. The task is finised when the first component x[0,0] is close enough to 0. Currently this is implemented as the conditionabs(x[0, 0]) <= 0.01.where \(\beta, \omega \sim \mathcal{N}(0, \sqrt{dt})\) are Wiener processes.
A and B may represent continuous or discrete dynamics. A conversion is implictly made following the value of discrete_dynamics keyword:
\[\begin{split}\begin{align} A_c = \frac{1}{dt} (A - I) \\ B_c = B \frac{1}{dt} \end{align}\end{split}\]\[\begin{split}\begin{align} A_d = I + dt \cdot{} A \\ B_d = dt \cdot{} B \end{align}\end{split}\]- Parameters
timestep (float) – dt
A (numpy.ndarray) – Passive dynamics
B (numpy.ndarray) – Response to command
F (numpy.ndarray, optional) – signal dependent noise, defaults to None
G (numpy.ndarray, optional) – independent noise, defaults to None
H (numpy.ndarray, optional) – control-dependent noise, defaults to None
discrete_dynamics (bool, optional) – whether A and B are continuous or discrete, defaults to True
noise (str, optional) – whether to include noise, defaults to “on”
timespace – if the task is modeled as discrete or continuous, defaults to “discrete”
Methods
base assistant step
base user step
drawDefine whether to use continuous or discrete representation for A and B
user step
Text mode: print task state
Force all substates except the first to be null.
stopping_conditionAttributes
assistant action
parametersround_numberThe current state of the task.
Turn number.
user action
- property assistant_action
assistant action
The last action input by the assistant.
- Returns
assistant action
- Return type
- base_on_assistant_action(*args, **kwargs)
base assistant step
Wraps the assistant defined on_assistant_action() method. For now does little but provide default values, may be useful later.
- Returns
(task state, task reward, is_done flag, metadata):
- Return type
tuple(
State, float, boolean, dictionnary)
- base_on_user_action(*args, **kwargs)
base user step
Wraps the user defined on_user_action() method. For now does little but provide default values, may be useful later.
- Returns
(task state, task reward, is_done flag, metadata):
- Return type
tuple(
State, float, boolean, dictionnary)
- on_user_action(*args, user_action=None, **kwargs)[source]
user step
Takes the state from x(.) to x(+.) according to
\[\begin{split}\begin{align} x(+.) = Ax(.) + Bu(.) + Fx(.).\beta + G.\omega \\ \end{align}\end{split}\]
- render(mode='text', ax_user=None, ax_assistant=None, ax_task=None)[source]
Text mode: print task state
plot mode: Dynamically update axes with state trajectories.
- reset(dic=None)[source]
Force all substates except the first to be null.
Force all substates except the first to be null. Also stores the last state as an attribute (for rendering).
- Parameters
dic (dictionnary, optional) – reset_dic, see :py:class:
InteractionTask <coopihc.interactiontask.InteractionTask.InteractionTask>, defaults to None
- property turn_number
Turn number.
The turn number of the game
- Returns
turn number
- Return type
numpy.ndarray