# tinker_cookbook.tokenizer_utils

| Class                                                                                                                                                  | Description                                                              |
| ------------------------------------------------------------------------------------------------------------------------------------------------------ | ------------------------------------------------------------------------ |
| [`TmlTokenizer`](https://tinker-docs.thinkingmachines.ai/cookbook/api-reference/tokenizer_utils/tmltokenizer/index.md)                                 | Structural type for the underlying `tml_renderers` tokenizer.            |
| [`SupportsTmlTokenizer`](https://tinker-docs.thinkingmachines.ai/cookbook/api-reference/tokenizer_utils/supportstmltokenizer/index.md)                 | Tokenizer facade that exposes an underlying `TmlTokenizer`.              |
| [`TmlRenderersTokenizerAdapter`](https://tinker-docs.thinkingmachines.ai/cookbook/api-reference/tokenizer_utils/tmlrendererstokenizeradapter/index.md) | Small tokenizer facade for `tml_renderers` models used through cookbook. |

| Function                                                                                                                                                       | Description                                                                  |
| -------------------------------------------------------------------------------------------------------------------------------------------------------------- | ---------------------------------------------------------------------------- |
| [`ensure_tml_renderers_importable()`](https://tinker-docs.thinkingmachines.ai/cookbook/api-reference/tokenizer_utils/ensure_tml_renderers_importable/index.md) | Make `tml_renderers` importable, falling back to `TML_RENDERERS_SOURCE_DIR`. |
| [`register_tokenizer()`](https://tinker-docs.thinkingmachines.ai/cookbook/api-reference/tokenizer_utils/register_tokenizer/index.md)                           | Register a custom tokenizer factory.                                         |
| [`get_registered_tokenizer_names()`](https://tinker-docs.thinkingmachines.ai/cookbook/api-reference/tokenizer_utils/get_registered_tokenizer_names/index.md)   | Return a list of all registered custom tokenizer names.                      |
| [`is_tokenizer_registered()`](https://tinker-docs.thinkingmachines.ai/cookbook/api-reference/tokenizer_utils/is_tokenizer_registered/index.md)                 | Check if a tokenizer name is registered.                                     |
| [`unregister_tokenizer()`](https://tinker-docs.thinkingmachines.ai/cookbook/api-reference/tokenizer_utils/unregister_tokenizer/index.md)                       | Unregister a custom tokenizer factory.                                       |
| [`get_tokenizer()`](https://tinker-docs.thinkingmachines.ai/cookbook/api-reference/tokenizer_utils/get_tokenizer/index.md)                                     | Get a tokenizer by name.                                                     |