# Tutorial 101: Hello Tinker

> **Run it interactively [[source]](https://github.com/thinking-machines-lab/tinker-cookbook/blob/main/tutorials/101_hello_tinker.py)**
>
> ```bash
> curl -O https://raw.githubusercontent.com/thinking-machines-lab/tinker-cookbook/main/tutorials/101_hello_tinker.py && marimo edit 101_hello_tinker.py
> ```

Tinker is a remote GPU service for LLM training and inference. You write training loops in Python on your local machine; Tinker executes the heavy GPU operations (forward passes, backpropagation, sampling) on remote workers.

```text
Your machine (CPU)                    Tinker Service (GPU)
+-----------------------+             +------------------------+
| Python training loop  |  -------->  | Forward/backward pass  |
| Data preparation      |  <--------  | Optimizer steps        |
| Evaluation logic      |             | Text generation        |
+-----------------------+             +------------------------+
```

You control the logic. Tinker runs the compute.

```python
import warnings

warnings.filterwarnings("ignore", message="IProgress not found")

import tinker
from tinker import types
```

## The client hierarchy

The entry point to Tinker is the **ServiceClient**. From it, you create specialized clients:

- **SamplingClient** -- generates text from a model (inference)
- **TrainingClient** -- runs forward/backward passes and optimizer steps (training)

Both talk to the same remote GPU workers. Let's start with the ServiceClient.

```python
api_key = mo.ui.text(kind="password", label="Paste your Tinker API key")
api_key  # noqa: B018
```

```python
import os

mo.stop(
    "TINKER_API_KEY" not in os.environ and not api_key.value,
    "Paste your API key above",
)

if api_key.value:
    os.environ["TINKER_API_KEY"] = api_key.value

# Create a ServiceClient. This reads TINKER_API_KEY from your environment.
service_client = tinker.ServiceClient()

# Check what models are available
capabilities = await service_client.get_server_capabilities_async()
print("Available models:")
for model in capabilities.supported_models:
    print(f"  - {model.model_name}")
```

**Output**

```text
Available models:
  - deepseek-ai/DeepSeek-V3.1
  - moonshotai/Kimi-K2.6
  - moonshotai/Kimi-K2.6:peft:131072
  - nvidia/NVIDIA-Nemotron-3-Nano-30B-A3B-BF16
  - nvidia/NVIDIA-Nemotron-3-Super-120B-A12B-BF16
  - nvidia/NVIDIA-Nemotron-3-Super-120B-A12B-BF16:peft:262144
  - nvidia/NVIDIA-Nemotron-3-Ultra-550B-A55B-BF16
  - nvidia/NVIDIA-Nemotron-3-Ultra-550B-A55B-BF16:peft:262144
  - Qwen/Qwen3-8B
  - Qwen/Qwen3.5-35B-A3B-Base
  - Qwen/Qwen3.5-397B-A17B
  - Qwen/Qwen3.5-397B-A17B:peft:262144
  - Qwen/Qwen3.5-4B
  - Qwen/Qwen3.5-9B
  - Qwen/Qwen3.5-9B-Base
  - Qwen/Qwen3.6-27B
  - Qwen/Qwen3.6-35B-A3B
  - openai/gpt-oss-120b
  - openai/gpt-oss-120b:peft:131072
  - openai/gpt-oss-20b
```

## Sampling from a model

Let's create a **SamplingClient** to generate text. We will use `Qwen/Qwen3.5-9B-Base`, a base (non-chat-tuned) model, so we can feed it raw tokens and get a plain text completion -- no chat template required.

The sampling workflow is:

1. Create a `SamplingClient` with a base model name
1. Encode your prompt into tokens using the model's tokenizer
1. Call `sample()` with the prompt and sampling parameters
1. Decode the returned tokens back into text

```python
MODEL_NAME = "Qwen/Qwen3.5-9B-Base"

# Create a sampling client -- this connects to a remote GPU worker
sampling_client = await service_client.create_sampling_client_async(base_model=MODEL_NAME)

# Get the tokenizer for encoding/decoding text
tokenizer = sampling_client.get_tokenizer()
```

```python
# Encode a prompt into tokens
prompt_text = "The three largest cities in the world by population are"
print("Prompt tokens:", tokenizer.encode(prompt_text))
prompt = types.ModelInput.from_ints(tokenizer.encode(prompt_text))

# Sample a completion
params = types.SamplingParams(max_tokens=50, temperature=0.5)
result = await sampling_client.sample_async(
    prompt=prompt, sampling_params=params, num_samples=1
)

# Decode and print
completion_tokens = result.sequences[0].tokens
print("Completion tokens:", completion_tokens)
print(prompt_text + tokenizer.decode(completion_tokens))
```

**Output**

```text
Prompt tokens: [760, 2250, 7526, 9432, 303, 279, 1814, 539, 6823, 513]
Completion tokens: [25358, 11, 21318, 11, 321, 35817, 11, 440, 21234, 314, 12805, 220, 18, 19, 13, 23, 3410, 11, 220, 17, 20, 13, 22, 3410, 11, 321, 220, 17, 19, 13, 23, 3410, 1208, 11, 15119, 13, 3437, 369, 279, 5289, 6823, 314, 1439, 2250, 9432, 11, 17440, 310, 279, 22746]
The three largest cities in the world by population are Tokyo, Delhi, and Shanghai, with populations of approximately 34.8 million, 25.7 million, and 24.8 million people, respectively. What is the average population of these three cities, rounded to the nearest
```

## Inspecting the response

The `sample()` call returns a `SampleResponse` containing a list of `SampledSequence` objects. Each sequence has:

- `tokens` -- the generated token IDs
- `logprobs` -- log probability of each generated token (if requested)
- `stop_reason` -- why generation stopped (e.g., hit max tokens, hit a stop string)

```python
_seq = result.sequences[0]
print(f"Stop reason:      {_seq.stop_reason}")
print(f"Tokens generated: {len(_seq.tokens)}")
print(f"Token IDs:        {_seq.tokens[:10]} ...")
print(f"Log probs:        {_seq.logprobs[:10]} ...")  # first 10
```

**Output**

```text
Stop reason:      length
Tokens generated: 50
Token IDs:        [25358, 11, 21318, 11, 321, 35817, 11, 440, 21234, 314] ...
Log probs:        [-0.2245454490184784, -0.1620202362537384, -0.423272967338562, -0.0032130186446011066, -8.023106784094125e-05, -0.01821271888911724, -2.127871513366699, -0.034714650362730026, -0.11956895887851715, -0.0017687217332422733] ...
```

You can also generate multiple samples at once by setting `num_samples`. Each sample is an independent completion from the same prompt.

```python
result_1 = await sampling_client.sample_async(
    prompt=prompt,
    sampling_params=types.SamplingParams(max_tokens=50, temperature=0.7),
    num_samples=3,
)
for i, _seq in enumerate(result_1.sequences):
    text = tokenizer.decode(_seq.tokens)
    print(f"Sample {i}: {prompt_text}{text}")
```

**Output**

```text
Sample 0: The three largest cities in the world by population are Tokyo, Shanghai, and Delhi. This ranking is based on the total number of people living within the urban area of each city. Tokyo, located in Japan, is the most populous city with an estimated population of over 37 million people. Shanghai
Sample 1: The three largest cities in the world by population are Tokyo, Japan; Delhi, India; and Shanghai, China. Tokyo has a population of about 38 million people. Delhi's population is approximately 1.7 times that of Tokyo, and Shanghai's population is about 2.2 times
Sample 2: The three largest cities in the world by population are Tokyo, India, and Shanghai.

<think>
Hmm, the user's statement contains a factual error about Tokyo being a city in India. Tokyo is in Japan, not India. The three largest cities by population are indeed Tokyo, Delhi, and Shanghai
```

## What about training?

So far we have only done inference. The real power of Tinker is **training** -- running forward/backward passes and optimizer steps on remote GPUs while you control the training loop locally.

The workflow looks like this:

1. Create a **TrainingClient** with `service_client.create_lora_training_client()`
1. Prepare training data as `Datum` objects (input tokens + loss targets)
1. Call `training_client.forward_backward()` to compute gradients
1. Call `training_client.optim_step()` to update weights
1. Save weights and create a **SamplingClient** to evaluate the trained model

We will walk through this in the next tutorial.

## Next steps

- **[Tutorial 102: First SFT](https://tinker-docs.thinkingmachines.ai/tutorials/basics/first-sft/index.md)** -- Train a model with supervised fine-tuning
- **[Getting Started Guide](https://tinker-docs.thinkingmachines.ai/tinker/quickstart/)** -- Full walkthrough of training and sampling
- **[Available Models](https://tinker-docs.thinkingmachines.ai/tinker/models/)** -- All supported models and their characteristics