# tinker_cookbook.rl

| Class                                                                                                                                 | Description                                                                      |
| ------------------------------------------------------------------------------------------------------------------------------------- | -------------------------------------------------------------------------------- |
| [`StopReason`](https://tinker-docs.thinkingmachines.ai/cookbook/api-reference/rl/stopreason/index.md)                                 | Why a trajectory ended.                                                          |
| [`InitialObservationOverflow`](https://tinker-docs.thinkingmachines.ai/cookbook/api-reference/rl/initialobservationoverflow/index.md) | Sentinel returned by `Env.initial_observation` instead of an                     |
| [`StepResult`](https://tinker-docs.thinkingmachines.ai/cookbook/api-reference/rl/stepresult/index.md)                                 | Result returned by `Env.step`.                                                   |
| [`Transition`](https://tinker-docs.thinkingmachines.ai/cookbook/api-reference/rl/transition/index.md)                                 | A single (observation, action, reward) tuple from a trajectory.                  |
| [`ActionExtra`](https://tinker-docs.thinkingmachines.ai/cookbook/api-reference/rl/actionextra/index.md)                               | Extra metadata passed alongside an action to `Env.step`.                         |
| [`Env`](https://tinker-docs.thinkingmachines.ai/cookbook/api-reference/rl/env/index.md)                                               | Stateful environment that a single agent interacts with.                         |
| [`Trajectory`](https://tinker-docs.thinkingmachines.ai/cookbook/api-reference/rl/trajectory/index.md)                                 | A complete episode: a sequence of transitions from one agent in one environment. |
| [`RolloutError`](https://tinker-docs.thinkingmachines.ai/cookbook/api-reference/rl/rollouterror/index.md)                             | A captured error from a failed trajectory rollout.                               |
| [`EnvGroupBuilder`](https://tinker-docs.thinkingmachines.ai/cookbook/api-reference/rl/envgroupbuilder/index.md)                       | Builds a group of environments. The group will be used in the following way:     |
| [`TrajectoryGroup`](https://tinker-docs.thinkingmachines.ai/cookbook/api-reference/rl/trajectorygroup/index.md)                       | A group of trajectories produced by one `EnvGroupBuilder`.                       |
| [`RLDataset`](https://tinker-docs.thinkingmachines.ai/cookbook/api-reference/rl/rldataset/index.md)                                   | A dataset that produces batches of `EnvGroupBuilder` instances.                  |
| [`RLDatasetBuilder`](https://tinker-docs.thinkingmachines.ai/cookbook/api-reference/rl/rldatasetbuilder/index.md)                     | Abstract builder that constructs training and optional test RL datasets.         |
| [`ProblemEnv`](https://tinker-docs.thinkingmachines.ai/cookbook/api-reference/rl/problemenv/index.md)                                 | A single-turn Q&A environment that rewards correct answers and valid formatting. |
| [`ProblemGroupBuilder`](https://tinker-docs.thinkingmachines.ai/cookbook/api-reference/rl/problemgroupbuilder/index.md)               | Builds a group of ProblemEnv instances from a factory callable.                  |
| [`MessageStepResult`](https://tinker-docs.thinkingmachines.ai/cookbook/api-reference/rl/messagestepresult/index.md)                   | Result of a message-level step.                                                  |
| [`MessageEnv`](https://tinker-docs.thinkingmachines.ai/cookbook/api-reference/rl/messageenv/index.md)                                 | Abstract base class for message-level environments.                              |
| [`EnvFromMessageEnv`](https://tinker-docs.thinkingmachines.ai/cookbook/api-reference/rl/envfrommessageenv/index.md)                   | Adapter that wraps a MessageEnv to implement the token-level Env interface.      |
| [`RolloutStrategy`](https://tinker-docs.thinkingmachines.ai/cookbook/api-reference/rl/rolloutstrategy/index.md)                       | Controls how trajectories are collected from a group of environments.            |
| [`FailFast`](https://tinker-docs.thinkingmachines.ai/cookbook/api-reference/rl/failfast/index.md)                                     | Default strategy: any trajectory error crashes the group.                        |
| [`RetryOnFailure`](https://tinker-docs.thinkingmachines.ai/cookbook/api-reference/rl/retryonfailure/index.md)                         | Retry failed or timed-out trajectories with fresh environments.                  |

| Function                                                                                                                        | Description                                                       |
| ------------------------------------------------------------------------------------------------------------------------------- | ----------------------------------------------------------------- |
| [`compute_advantages()`](https://tinker-docs.thinkingmachines.ai/cookbook/api-reference/rl/compute_advantages/index.md)         | Compute advantages for each trajectory, centered within groups.   |
| [`trajectory_to_data()`](https://tinker-docs.thinkingmachines.ai/cookbook/api-reference/rl/trajectory_to_data/index.md)         | Return one or more Datum objects corresponding to the trajectory. |
| [`assemble_training_data()`](https://tinker-docs.thinkingmachines.ai/cookbook/api-reference/rl/assemble_training_data/index.md) | Convert trajectories to training data format.                     |