# tinker_cookbook.rl.ProblemEnv

## *class* [**tinker_cookbook.rl.ProblemEnv**](https://github.com/thinking-machines-lab/tinker-cookbook/blob/main/tinker_cookbook/rl/problem_env.py#L26)(*[Env](https://tinker-docs.thinkingmachines.ai/cookbook/api-reference/rl/env/index.md)*)

A single-turn Q&A environment that rewards correct answers and valid formatting.

```python
class MathEnv(ProblemEnv):
def __init__(self, renderer, question, answer):
super().__init__(renderer)
self.question = question
self.answer = answer
def get_question(self):
return self.question
def check_answer(self, response):
return self.answer in response
def check_format(self, response):
return response.strip() != ""
def get_reference_answer(self):
return self.answer
```

### [**get_question**](https://github.com/thinking-machines-lab/tinker-cookbook/blob/main/tinker_cookbook/rl/problem_env.py#L73)()

Return the question text for this problem.

**Returns:** *str*

*Abstract method.*

### [**check_answer**](https://github.com/thinking-machines-lab/tinker-cookbook/blob/main/tinker_cookbook/rl/problem_env.py#L78)(*sample_str*)

Return a reward (0.0 to 1.0) for the model's response.

**Parameters:**

- [**sample_str**](https://github.com/thinking-machines-lab/tinker-cookbook/blob/main/tinker_cookbook/rl/problem_env.py#L78) (*str*) – The decoded text of the model's response.

**Returns:** *bool* – Whether the answer is correct.

*Abstract method.*

### [**check_format**](https://github.com/thinking-machines-lab/tinker-cookbook/blob/main/tinker_cookbook/rl/problem_env.py#L90)(*sample_str*)

Return a format compliance reward (0.0 to 1.0).

**Parameters:**

- [**sample_str**](https://github.com/thinking-machines-lab/tinker-cookbook/blob/main/tinker_cookbook/rl/problem_env.py#L90) (*str*) – The decoded text of the model's response.

**Returns:** *bool* – Whether the response follows the expected format.

*Abstract method.*

### [**get_reference_answer**](https://github.com/thinking-machines-lab/tinker-cookbook/blob/main/tinker_cookbook/rl/problem_env.py#L102)()

Return the reference answer for logging purposes.

**Returns:** *str*

*Abstract method.*

### [**initial_observation**](https://github.com/thinking-machines-lab/tinker-cookbook/blob/main/tinker_cookbook/rl/problem_env.py#L106)()

Build the initial prompt from the conversation prefix and question.

**Returns:** *tuple[Observation, StopCondition]*

### [**step**](https://github.com/thinking-machines-lab/tinker-cookbook/blob/main/tinker_cookbook/rl/problem_env.py#L113)(*action*, *extra*)

Score the model's response for correctness and format compliance.

**Parameters:**

- [**action**](https://github.com/thinking-machines-lab/tinker-cookbook/blob/main/tinker_cookbook/rl/problem_env.py#L113) (*Action*) – Token IDs of the model's response.
- [**extra**](https://github.com/thinking-machines-lab/tinker-cookbook/blob/main/tinker_cookbook/rl/problem_env.py#L113) (*[ActionExtra](https://tinker-docs.thinkingmachines.ai/cookbook/api-reference/rl/actionextra/index.md) | None*) – Optional action metadata (unused).

**Returns:** *[StepResult](https://tinker-docs.thinkingmachines.ai/cookbook/api-reference/rl/stepresult/index.md)*

## Referenced by

- [tinker_cookbook.distillation.PromptOnlyEnv](https://tinker-docs.thinkingmachines.ai/cookbook/api-reference/distillation/promptonlyenv/index.md)
- [tinker_cookbook.rl.ProblemGroupBuilder](https://tinker-docs.thinkingmachines.ai/cookbook/api-reference/rl/problemgroupbuilder/index.md)