tinker_cookbook.distillation.DatasetWithTeacher
class tinker_cookbook.distillation.DatasetWithTeacher()
Pairs a supervised dataset with its teacher model.
Each domain in multi-teacher distillation gets one of these. The teacher's highest-probability tokens are used as soft targets for training.
Fields:
- dataset_builder (SupervisedDatasetBuilder)
- teacher_config (TeacherConfig)
- weight (float, default:
1.0)