Spaces:
Running
Running
File size: 4,639 Bytes
ae32ca4 a266504 f7b7ab6 ae32ca4 9804745 ae32ca4 9804745 ae32ca4 9804745 ae32ca4 f7b7ab6 ae32ca4 9804745 ae32ca4 f7b7ab6 ae32ca4 9804745 ae32ca4 9804745 ae32ca4 | 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 | ---
title: LLM Sampling
sidebarTitle: Sampling
description: Handle server-initiated LLM sampling requests.
icon: robot
---
import { VersionBadge } from "/snippets/version-badge.mdx";
<VersionBadge version="2.0.0" />
MCP servers can request LLM completions from clients. The client handles these requests through a sampling handler callback.
## Sampling Handler
Provide a `sampling_handler` function when creating the client:
```python
from fastmcp import Client
from fastmcp.client.sampling import (
SamplingMessage,
SamplingParams,
RequestContext,
)
async def sampling_handler(
messages: list[SamplingMessage],
params: SamplingParams,
context: RequestContext
) -> str:
# Your LLM integration logic here
# Extract text from messages and generate a response
return "Generated response based on the messages"
client = Client(
"my_mcp_server.py",
sampling_handler=sampling_handler,
)
```
### Handler Parameters
The sampling handler receives three parameters:
<Card icon="code" title="Sampling Handler Parameters">
<ResponseField name="SamplingMessage" type="Sampling Message Object">
<Expandable title="attributes">
<ResponseField name="role" type='Literal["user", "assistant"]'>
The role of the message.
</ResponseField>
<ResponseField name="content" type="TextContent | ImageContent | AudioContent">
The content of the message.
TextContent is most common, and has a `.text` attribute.
</ResponseField>
</Expandable>
</ResponseField>
<ResponseField name="SamplingParams" type="Sampling Parameters Object">
<Expandable title="attributes">
<ResponseField name="messages" type="list[SamplingMessage]">
The messages to sample from
</ResponseField>
<ResponseField name="modelPreferences" type="ModelPreferences | None">
The server's preferences for which model to select. The client MAY ignore
these preferences.
<Expandable title="attributes">
<ResponseField name="hints" type="list[ModelHint] | None">
The hints to use for model selection.
</ResponseField>
<ResponseField name="costPriority" type="float | None">
The cost priority for model selection.
</ResponseField>
<ResponseField name="speedPriority" type="float | None">
The speed priority for model selection.
</ResponseField>
<ResponseField name="intelligencePriority" type="float | None">
The intelligence priority for model selection.
</ResponseField>
</Expandable>
</ResponseField>
<ResponseField name="systemPrompt" type="str | None">
An optional system prompt the server wants to use for sampling.
</ResponseField>
<ResponseField name="includeContext" type="IncludeContext | None">
A request to include context from one or more MCP servers (including the caller), to
be attached to the prompt.
</ResponseField>
<ResponseField name="temperature" type="float | None">
The sampling temperature.
</ResponseField>
<ResponseField name="maxTokens" type="int">
The maximum number of tokens to sample.
</ResponseField>
<ResponseField name="stopSequences" type="list[str] | None">
The stop sequences to use for sampling.
</ResponseField>
<ResponseField name="metadata" type="dict[str, Any] | None">
Optional metadata to pass through to the LLM provider.
</ResponseField>
</Expandable>
</ResponseField>
<ResponseField name="RequestContext" type="Request Context Object">
<Expandable title="attributes">
<ResponseField name="request_id" type="RequestId">
Unique identifier for the MCP request
</ResponseField>
</Expandable>
</ResponseField>
</Card>
## Basic Example
```python
from fastmcp import Client
from fastmcp.client.sampling import SamplingMessage, SamplingParams, RequestContext
async def basic_sampling_handler(
messages: list[SamplingMessage],
params: SamplingParams,
context: RequestContext
) -> str:
# Extract message content
conversation = []
for message in messages:
content = message.content.text if hasattr(message.content, 'text') else str(message.content)
conversation.append(f"{message.role}: {content}")
# Use the system prompt if provided
system_prompt = params.systemPrompt or "You are a helpful assistant."
# Here you would integrate with your preferred LLM service
# This is just a placeholder response
return f"Response based on conversation: {' | '.join(conversation)}"
client = Client(
"my_mcp_server.py",
sampling_handler=basic_sampling_handler
)
```
|