# SamplingClient

## *class* [**tinker.SamplingClient**](https://github.com/thinking-machines-lab/tinker/blob/main/src/tinker/lib/public_interfaces/sampling_client.py#L88)(*holder*, *sampling_session_id*, *shadow=False*, *retry_config=None*, *subprocess_sampling=None*)

Client for text generation and inference from trained or base models.

The SamplingClient lets you generate text tokens from either a base model or from weights you've saved using a TrainingClient. You typically get one by calling `service_client.create_sampling_client()` or `training_client.save_weights_and_get_sampling_client()`.

**Key methods:**

- sample() - generate text completions with customizable parameters
- compute_logprobs() - get log probabilities for prompt tokens

**Create method parameters:**

- `model_path`: Path to saved model weights (starts with 'tinker://')
- `base_model`: Name of base model to use for inference (e.g., 'Qwen/Qwen3-8B')
- `retry_config`: Configuration for retrying failed requests

```python
sampling_client = service_client.create_sampling_client(base_model="Qwen/Qwen3-8B")
prompt = types.ModelInput.from_ints(tokenizer.encode("The weather today is"))
params = types.SamplingParams(max_tokens=20, temperature=0.7)
future = sampling_client.sample(prompt=prompt, sampling_params=params, num_samples=1)
result = future.result()
```

**Parameters:**

- [**holder**](https://github.com/thinking-machines-lab/tinker/blob/main/src/tinker/lib/public_interfaces/sampling_client.py#L130) (*[InternalClientHolder](https://github.com/thinking-machines-lab/tinker/blob/main/src/tinker/lib/internal_client_holder.py#L185)*)
- [**sampling_session_id**](https://github.com/thinking-machines-lab/tinker/blob/main/src/tinker/lib/public_interfaces/sampling_client.py#L132) (*str*)
- [**shadow**](https://github.com/thinking-machines-lab/tinker/blob/main/src/tinker/lib/public_interfaces/sampling_client.py#L133) (*bool*, default: `False`)
- [**retry_config**](https://github.com/thinking-machines-lab/tinker/blob/main/src/tinker/lib/public_interfaces/sampling_client.py#L134) (*[RetryConfig](https://github.com/thinking-machines-lab/tinker/blob/main/src/tinker/lib/retry_handler.py#L39) | None*, default: `None`)
- [**subprocess_sampling**](https://github.com/thinking-machines-lab/tinker/blob/main/src/tinker/lib/public_interfaces/sampling_client.py#L135) (*bool | None*, default: `None`)

### [**sample**](https://github.com/thinking-machines-lab/tinker/blob/main/src/tinker/lib/public_interfaces/sampling_client.py#L298)(*prompt*, *num_samples*, *sampling_params*, *include_prompt_logprobs=False*, *topk_prompt_logprobs=0*)

Generate text completions from the model.

**Parameters:**

- [**prompt**](https://github.com/thinking-machines-lab/tinker/blob/main/src/tinker/lib/public_interfaces/sampling_client.py#L300) (*types.[ModelInput](https://tinker-docs.thinkingmachines.ai/tinker/api-reference/types/modelinput/index.md)*) – The input tokens as ModelInput
- [**num_samples**](https://github.com/thinking-machines-lab/tinker/blob/main/src/tinker/lib/public_interfaces/sampling_client.py#L301) (*int*) – Number of independent samples to generate
- [**sampling_params**](https://github.com/thinking-machines-lab/tinker/blob/main/src/tinker/lib/public_interfaces/sampling_client.py#L302) (*types.[SamplingParams](https://tinker-docs.thinkingmachines.ai/tinker/api-reference/types/samplingparams/index.md)*) – Parameters controlling generation (temperature, max_tokens, etc.)
- [**include_prompt_logprobs**](https://github.com/thinking-machines-lab/tinker/blob/main/src/tinker/lib/public_interfaces/sampling_client.py#L303) (*bool*, default: `False`) – Whether to include log probabilities for prompt tokens
- [**topk_prompt_logprobs**](https://github.com/thinking-machines-lab/tinker/blob/main/src/tinker/lib/public_interfaces/sampling_client.py#L304) (*int*, default: `0`) – Number of top token log probabilities to return per position

**Returns:** A `Future` containing the [`SampleResponse`](https://tinker-docs.thinkingmachines.ai/tinker/api-reference/types/sampleresponse/index.md) with generated text

```python
prompt = types.ModelInput.from_ints(tokenizer.encode("The weather today is"))
params = types.SamplingParams(max_tokens=20, temperature=0.7)
future = sampling_client.sample(prompt=prompt, sampling_params=params, num_samples=1)
result = future.result()
for sample in result.samples:
print(tokenizer.decode(sample.tokens))
```

*Async variant:* `sample_async()`

### [**compute_logprobs**](https://github.com/thinking-machines-lab/tinker/blob/main/src/tinker/lib/public_interfaces/sampling_client.py#L385)(*prompt*)

Compute log probabilities for prompt tokens.

**Parameters:**

- [**prompt**](https://github.com/thinking-machines-lab/tinker/blob/main/src/tinker/lib/public_interfaces/sampling_client.py#L385) (*types.[ModelInput](https://tinker-docs.thinkingmachines.ai/tinker/api-reference/types/modelinput/index.md)*) – The input tokens as ModelInput

**Returns:** A `Future` containing a list of log probabilities for each token in the prompt. None values indicate tokens where log probabilities couldn't be computed.

```python
prompt = types.ModelInput.from_ints(tokenizer.encode("Hello world"))
future = sampling_client.compute_logprobs(prompt)
logprobs = future.result()
for i, logprob in enumerate(logprobs):
if logprob is not None:
print(f"Token {i}: logprob = {logprob:.4f}")
```

*Async variant:* `compute_logprobs_async()`

### [**get_tokenizer**](https://github.com/thinking-machines-lab/tinker/blob/main/src/tinker/lib/public_interfaces/sampling_client.py#L443)()

Get the tokenizer for the current model.

**Returns:** `PreTrainedTokenizer` compatible with the model

### [**get_base_model**](https://github.com/thinking-machines-lab/tinker/blob/main/src/tinker/lib/public_interfaces/sampling_client.py#L452)()

Get the base model name for the current sampling session.

**Returns:** *str*

*Async variant:* `get_base_model_async()`

## Referenced by

- [ServiceClient.create_sampling_client](https://tinker-docs.thinkingmachines.ai/tinker/api-reference/serviceclient/#create_sampling_client)
- [tinker_cookbook.completers.TinkerTokenCompleter](https://tinker-docs.thinkingmachines.ai/cookbook/api-reference/completers/tinkertokencompleter/index.md)
- [tinker_cookbook.eval.SamplingClientEvaluator.__call__](https://tinker-docs.thinkingmachines.ai/cookbook/api-reference/eval/samplingclientevaluator/#samplingclientevaluator-call)
- [TrainingClient.create_sampling_client](https://tinker-docs.thinkingmachines.ai/tinker/api-reference/trainingclient/#create_sampling_client)
- [TrainingClient.save_weights_and_get_sampling_client](https://tinker-docs.thinkingmachines.ai/tinker/api-reference/trainingclient/#save_weights_and_get_sampling_client)