# tinker_cookbook.rl.Env

## *class* [**tinker_cookbook.rl.Env**](https://github.com/thinking-machines-lab/tinker-cookbook/blob/main/tinker_cookbook/rl/types.py#L195)(*ABC*)

Stateful environment that a single agent interacts with.

Each [`Env`](https://tinker-docs.thinkingmachines.ai/cookbook/api-reference/rl/env/index.md) instance is **single-use**: create it, run one episode, then discard it. Environments are created by `EnvGroupBuilder.make_envs`.

Implementors must override `initial_observation` and `step`.

```python
class MyEnv(Env):
def __init__(self, question: str, answer: str, renderer):
self.question = question
self.answer = answer
self.renderer = renderer
async def initial_observation(self):
messages = [{"role": "user", "content": self.question}]
model_input, _ = self.renderer.build_generation_prompt(messages)
return model_input, self.renderer.get_stop_sequences()
async def step(self, action, *, extra=None):
response = self.renderer.tokenizer.decode(action)
reward = 1.0 if self.answer in response else 0.0
return StepResult(
reward=reward,
episode_done=True,
next_observation=tinker.ModelInput.from_ints([]),
next_stop_condition=[],
)
```

### [**initial_observation**](https://github.com/thinking-machines-lab/tinker-cookbook/blob/main/tinker_cookbook/rl/types.py#L228)()

Return the starting observation and stop condition for this episode.

**Returns:** tuple[Observation, StopCondition] | InitialObservationOverflow: The initial observation (model input) and the stop condition for the first generation step. Environments that enforce a token budget may instead return [`InitialObservationOverflow`](https://tinker-docs.thinkingmachines.ai/cookbook/api-reference/rl/initialobservationoverflow/index.md) when the initial prompt already exceeds it, which ends the rollout immediately and gracefully (no sampling call is made).

*Abstract method.*

### [**step**](https://github.com/thinking-machines-lab/tinker-cookbook/blob/main/tinker_cookbook/rl/types.py#L245)(*action*, *extra*)

Advance the environment by one step given the agent's action.

**Parameters:**

- [**action**](https://github.com/thinking-machines-lab/tinker-cookbook/blob/main/tinker_cookbook/rl/types.py#L245) (*Action*) – Token IDs produced by the agent.
- [**extra**](https://github.com/thinking-machines-lab/tinker-cookbook/blob/main/tinker_cookbook/rl/types.py#L245) (*[ActionExtra](https://tinker-docs.thinkingmachines.ai/cookbook/api-reference/rl/actionextra/index.md) | None*) – Optional metadata about the action, such as the stop reason.

**Returns:** *[StepResult](https://tinker-docs.thinkingmachines.ai/cookbook/api-reference/rl/stepresult/index.md)* – The reward, next observation, and whether the episode is done.

*Abstract method.*

## Referenced by

- [tinker_cookbook.rl.EnvFromMessageEnv](https://tinker-docs.thinkingmachines.ai/cookbook/api-reference/rl/envfrommessageenv/index.md)
- [tinker_cookbook.rl.EnvGroupBuilder.compute_group_rewards](https://tinker-docs.thinkingmachines.ai/cookbook/api-reference/rl/envgroupbuilder/#envgroupbuilder-compute_group_rewards)
- [tinker_cookbook.rl.EnvGroupBuilder.make_envs](https://tinker-docs.thinkingmachines.ai/cookbook/api-reference/rl/envgroupbuilder/#envgroupbuilder-make_envs)
- [tinker_cookbook.rl.ProblemEnv](https://tinker-docs.thinkingmachines.ai/cookbook/api-reference/rl/problemenv/index.md)
- [tinker_cookbook.rl.ProblemGroupBuilder.compute_group_rewards](https://tinker-docs.thinkingmachines.ai/cookbook/api-reference/rl/problemgroupbuilder/#problemgroupbuilder-compute_group_rewards)
- [tinker_cookbook.rl.ProblemGroupBuilder.make_envs](https://tinker-docs.thinkingmachines.ai/cookbook/api-reference/rl/problemgroupbuilder/#problemgroupbuilder-make_envs)
- [tinker_cookbook.rl.Trajectory](https://tinker-docs.thinkingmachines.ai/cookbook/api-reference/rl/trajectory/index.md)