# tinker_cookbook.preference.Config

## *class* [**tinker_cookbook.preference.Config**](https://github.com/thinking-machines-lab/tinker-cookbook/blob/main/tinker_cookbook/preference/train_dpo.py#L31)()

Configuration for Direct Preference Optimization (DPO) training.

This is a `chz` dataclass that holds all hyperparameters, infrastructure settings, and checkpointing options for a DPO training run.

```python
config = Config(
log_path="~/logs/dpo_run",
model_name="Qwen/Qwen3.5-9B",
renderer_name="qwen3_5_disable_thinking",
dataset_builder=my_dpo_dataset_builder,
dpo_beta=0.1,
learning_rate=1e-5,
)
main(config)
```

**Fields:**

- [**log_path**](https://github.com/thinking-machines-lab/tinker-cookbook/blob/main/tinker_cookbook/preference/train_dpo.py#L92) (*str*)
- [**model_name**](https://github.com/thinking-machines-lab/tinker-cookbook/blob/main/tinker_cookbook/preference/train_dpo.py#L93) (*str*)
- [**recipe_name**](https://github.com/thinking-machines-lab/tinker-cookbook/blob/main/tinker_cookbook/preference/train_dpo.py#L94) (*str*)
- [**dataset_builder**](https://github.com/thinking-machines-lab/tinker-cookbook/blob/main/tinker_cookbook/preference/train_dpo.py#L95) (*[ChatDatasetBuilder](https://tinker-docs.thinkingmachines.ai/cookbook/api-reference/supervised/chatdatasetbuilder/index.md)*)
- [**load_checkpoint_path**](https://github.com/thinking-machines-lab/tinker-cookbook/blob/main/tinker_cookbook/preference/train_dpo.py#L96) (*str | None*, default: `None`)
- [**renderer_name**](https://github.com/thinking-machines-lab/tinker-cookbook/blob/main/tinker_cookbook/preference/train_dpo.py#L97) (*str | None*, default: `None`)
- [**learning_rate**](https://github.com/thinking-machines-lab/tinker-cookbook/blob/main/tinker_cookbook/preference/train_dpo.py#L101) (*float*, default: `1e-05`)
- [**lr_schedule**](https://github.com/thinking-machines-lab/tinker-cookbook/blob/main/tinker_cookbook/preference/train_dpo.py#L102) (*LRSchedule*, default: `'linear'`)
- [**num_epochs**](https://github.com/thinking-machines-lab/tinker-cookbook/blob/main/tinker_cookbook/preference/train_dpo.py#L103) (*int*, default: `1`)
- [**dpo_beta**](https://github.com/thinking-machines-lab/tinker-cookbook/blob/main/tinker_cookbook/preference/train_dpo.py#L104) (*float*, default: `0.1`)
- [**lora_rank**](https://github.com/thinking-machines-lab/tinker-cookbook/blob/main/tinker_cookbook/preference/train_dpo.py#L107) (*int*, default: `32`)
- [**num_replicas**](https://github.com/thinking-machines-lab/tinker-cookbook/blob/main/tinker_cookbook/preference/train_dpo.py#L110) (*int*, default: `8`)
- [**base_url**](https://github.com/thinking-machines-lab/tinker-cookbook/blob/main/tinker_cookbook/preference/train_dpo.py#L111) (*str | None*, default: `None`)
- [**evaluator_builders**](https://github.com/thinking-machines-lab/tinker-cookbook/blob/main/tinker_cookbook/preference/train_dpo.py#L114) (*list[EvaluatorBuilder]*, default: `[]`) – Checkpointing and evaluation (0 = disabled for \*\_every fields)
- [**infrequent_evaluator_builders**](https://github.com/thinking-machines-lab/tinker-cookbook/blob/main/tinker_cookbook/preference/train_dpo.py#L115) (*list[EvaluatorBuilder]*, default: `[]`)
- [**save_every**](https://github.com/thinking-machines-lab/tinker-cookbook/blob/main/tinker_cookbook/preference/train_dpo.py#L116) (*int*, default: `20`)
- [**eval_every**](https://github.com/thinking-machines-lab/tinker-cookbook/blob/main/tinker_cookbook/preference/train_dpo.py#L117) (*int*, default: `10`)
- [**infrequent_eval_every**](https://github.com/thinking-machines-lab/tinker-cookbook/blob/main/tinker_cookbook/preference/train_dpo.py#L118) (*int*, default: `100`)
- [**ttl_seconds**](https://github.com/thinking-machines-lab/tinker-cookbook/blob/main/tinker_cookbook/preference/train_dpo.py#L119) (*int | None*, default: `604800`) – 7 days
- [**rolling_save_every**](https://github.com/thinking-machines-lab/tinker-cookbook/blob/main/tinker_cookbook/preference/train_dpo.py#L122) (*int*, default: `0`) – but skips the sampler-weight export, making it cheaper than periodic checkpoints.
- [**rolling_ttl_seconds**](https://github.com/thinking-machines-lab/tinker-cookbook/blob/main/tinker_cookbook/preference/train_dpo.py#L124) (*int*, default: `7200`) – 2 hours
- [**adam_beta1**](https://github.com/thinking-machines-lab/tinker-cookbook/blob/main/tinker_cookbook/preference/train_dpo.py#L127) (*float*, default: `0.9`)
- [**adam_beta2**](https://github.com/thinking-machines-lab/tinker-cookbook/blob/main/tinker_cookbook/preference/train_dpo.py#L128) (*float*, default: `0.95`)
- [**adam_eps**](https://github.com/thinking-machines-lab/tinker-cookbook/blob/main/tinker_cookbook/preference/train_dpo.py#L129) (*float*, default: `1e-08`)
- [**wandb_project**](https://github.com/thinking-machines-lab/tinker-cookbook/blob/main/tinker_cookbook/preference/train_dpo.py#L132) (*str | None*, default: `None`)
- [**wandb_name**](https://github.com/thinking-machines-lab/tinker-cookbook/blob/main/tinker_cookbook/preference/train_dpo.py#L133) (*str | None*, default: `None`)
- [**enable_trace**](https://github.com/thinking-machines-lab/tinker-cookbook/blob/main/tinker_cookbook/preference/train_dpo.py#L136) (*bool*, default: `False`) – Profiling
- [**span_chart_every**](https://github.com/thinking-machines-lab/tinker-cookbook/blob/main/tinker_cookbook/preference/train_dpo.py#L137) (*int*, default: `0`)
- [**reference_model_name**](https://github.com/thinking-machines-lab/tinker-cookbook/blob/main/tinker_cookbook/preference/train_dpo.py#L140) (*str | None*, default: `None`)
- [**max_steps**](https://github.com/thinking-machines-lab/tinker-cookbook/blob/main/tinker_cookbook/preference/train_dpo.py#L143) (*int | None*, default: `None`) – Maximum number of training steps. If None, train for num_epochs * n_batches.