Text Generation
Transformers
Safetensors
MiniMax H3
English
qwen3_5
image-text-to-text
prompt-expansion
text-to-image
fp4
sglang
conversational
8-bit precision
modelopt
Instructions to use MiniMaxAI/MiniMax-H3-Image-PE with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- Transformers
How to use MiniMaxAI/MiniMax-H3-Image-PE with Transformers:
# Use a pipeline as a high-level helper from transformers import pipeline pipe = pipeline("text-generation", model="MiniMaxAI/MiniMax-H3-Image-PE") messages = [ { "role": "user", "content": [ {"type": "image", "url": "https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/p-blog/candy.JPG"}, {"type": "text", "text": "What animal is on the candy?"} ] }, ] pipe(text=messages)# pip install -U transformers accelerate # Load model directly from transformers import AutoProcessor, AutoModelForMultimodalLM processor = AutoProcessor.from_pretrained("MiniMaxAI/MiniMax-H3-Image-PE") model = AutoModelForMultimodalLM.from_pretrained("MiniMaxAI/MiniMax-H3-Image-PE", device_map="auto") messages = [ { "role": "user", "content": [ {"type": "image", "url": "https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/p-blog/candy.JPG"}, {"type": "text", "text": "What animal is on the candy?"} ] }, ] inputs = processor.apply_chat_template( messages, add_generation_prompt=True, tokenize=True, return_dict=True, return_tensors="pt", ).to(model.device) outputs = model.generate(**inputs, max_new_tokens=256) print(processor.decode(outputs[0][inputs["input_ids"].shape[-1]:])) - Notebooks
- Google Colab
- Kaggle
- Local Apps Settings
- vLLM
How to use MiniMaxAI/MiniMax-H3-Image-PE with vLLM:
Install from pip and serve model
# Install vLLM from pip: pip install vllm # Start the vLLM server: vllm serve "MiniMaxAI/MiniMax-H3-Image-PE" # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:8000/v1/chat/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "MiniMaxAI/MiniMax-H3-Image-PE", "messages": [ { "role": "user", "content": "What is the capital of France?" } ] }'Use Docker
docker model run hf.co/MiniMaxAI/MiniMax-H3-Image-PE
- SGLang
How to use MiniMaxAI/MiniMax-H3-Image-PE with SGLang:
Install from pip and serve model
# Install SGLang from pip: pip install sglang # Start the SGLang server: python3 -m sglang.launch_server \ --model-path "MiniMaxAI/MiniMax-H3-Image-PE" \ --host 0.0.0.0 \ --port 30000 # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:30000/v1/chat/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "MiniMaxAI/MiniMax-H3-Image-PE", "messages": [ { "role": "user", "content": "What is the capital of France?" } ] }'Use Docker images
docker run --gpus all \ --shm-size 32g \ -p 30000:30000 \ -v ~/.cache/huggingface:/root/.cache/huggingface \ --env "HF_TOKEN=<secret>" \ --ipc=host \ lmsysorg/sglang:latest \ python3 -m sglang.launch_server \ --model-path "MiniMaxAI/MiniMax-H3-Image-PE" \ --host 0.0.0.0 \ --port 30000 # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:30000/v1/chat/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "MiniMaxAI/MiniMax-H3-Image-PE", "messages": [ { "role": "user", "content": "What is the capital of France?" } ] }' - Docker Model Runner
How to use MiniMaxAI/MiniMax-H3-Image-PE with Docker Model Runner:
docker model run hf.co/MiniMaxAI/MiniMax-H3-Image-PE
Unify image editing terminology and auto aspect-ratio guidance
Browse filesUse Image editing PE consistently and describe text input as an instruction. Address readers as you/your and clarify that auto uses the model output while an explicit selection takes precedence for image generation. Align the editing example and task prompt on auto, and refresh the prompt hash.
Validation: README Python/JSON parsing, message assembly and relative links checked; seven focused client tests pass. No inference run.
- README.md +16 -17
- prompt/edit/task_prompt.txt +5 -5
- prompt/manifest.json +1 -1
README.md
CHANGED
|
@@ -51,15 +51,15 @@ This repository covers two workflows:
|
|
| 51 |
|
| 52 |
| Workflow | Input | Released assets |
|
| 53 |
| --- | --- | --- |
|
| 54 |
-
| **Text-to-image (T2I) PE** | A text
|
| 55 |
-
| **
|
| 56 |
|
| 57 |
T2I PE expands the scene, composition and visual details of a generation request.
|
| 58 |
-
|
| 59 |
-
reference images. **
|
| 60 |
prompt only.**
|
| 61 |
|
| 62 |
-
The examples below explain how to use the released T2I model and
|
| 63 |
|
| 64 |
## How to use T2I PE
|
| 65 |
|
|
@@ -88,7 +88,7 @@ final answer.
|
|
| 88 |
|
| 89 |
### Expand a prompt
|
| 90 |
|
| 91 |
-
Load the task prompt and insert your
|
| 92 |
|
| 93 |
```python
|
| 94 |
from pathlib import Path
|
|
@@ -162,12 +162,11 @@ Reasoning is returned separately in `message.reasoning_content`. Example output:
|
|
| 162 |
}
|
| 163 |
```
|
| 164 |
|
| 165 |
-
|
| 166 |
-
|
| 167 |
-
|
| 168 |
-
image-generation stage.
|
| 169 |
|
| 170 |
-
## How to use
|
| 171 |
|
| 172 |
Use the task prompt with a vision-language model that supports interleaved text
|
| 173 |
and images. The example below describes message assembly without selecting a
|
|
@@ -181,13 +180,13 @@ Provide the following inputs:
|
|
| 181 |
| --- | --- |
|
| 182 |
| `user_query` | The editing instruction. Refer to images in their supplied order. |
|
| 183 |
| `references` | At least one prepared image, in order, with its original display aspect ratio as a reduced `W:H` string. Use `None` when the source ratio is unknown. |
|
| 184 |
-
| `aspect_ratio` | The target ratio: `
|
| 185 |
|
| 186 |
### Assemble the message
|
| 187 |
|
| 188 |
Load [`prompt/edit/task_prompt.txt`](prompt/edit/task_prompt.txt) unchanged as the
|
| 189 |
first content part. Follow it with the image count, each image's label and source
|
| 190 |
-
ratio immediately before that image, and
|
| 191 |
in the final text part. This produces one user message.
|
| 192 |
|
| 193 |
The fixed task prompt forms a shared prefix across requests, allowing reuse by
|
|
@@ -203,7 +202,7 @@ from pathlib import Path
|
|
| 203 |
|
| 204 |
task_prompt = Path("prompt/edit/task_prompt.txt").read_text(encoding="utf-8")
|
| 205 |
user_query = "Replace the bag in the first image with the bag in the second image."
|
| 206 |
-
aspect_ratio = "
|
| 207 |
references = [
|
| 208 |
{"image": source_image, "aspect_ratio": "3:2"},
|
| 209 |
{"image": reference_image, "aspect_ratio": "1:1"},
|
|
@@ -248,6 +247,6 @@ messages = [{"role": "user", "content": content}]
|
|
| 248 |
|
| 249 |
The description identifies source subjects, states what is preserved or changed,
|
| 250 |
and describes new content. Descriptive prose is English; literal text retains the
|
| 251 |
-
language
|
| 252 |
-
|
| 253 |
-
|
|
|
|
| 51 |
|
| 52 |
| Workflow | Input | Released assets |
|
| 53 |
| --- | --- | --- |
|
| 54 |
+
| **Text-to-image (T2I) PE** | A text instruction | Fine-tuned Qwen3.8-27B weights in FP4, task and system prompts, and a client example |
|
| 55 |
+
| **Image editing PE** | An editing instruction and reference images | A task prompt and message-assembly example for a vision-language model of your choice |
|
| 56 |
|
| 57 |
T2I PE expands the scene, composition and visual details of a generation request.
|
| 58 |
+
Image editing PE describes the intended changes while preserving relevant content from the
|
| 59 |
+
reference images. **Image editing PE is still under development; this release provides its
|
| 60 |
prompt only.**
|
| 61 |
|
| 62 |
+
The examples below explain how to use the released T2I model and image editing task prompt.
|
| 63 |
|
| 64 |
## How to use T2I PE
|
| 65 |
|
|
|
|
| 88 |
|
| 89 |
### Expand a prompt
|
| 90 |
|
| 91 |
+
Load the task prompt and insert your instruction with `.format(user_query)`:
|
| 92 |
|
| 93 |
```python
|
| 94 |
from pathlib import Path
|
|
|
|
| 162 |
}
|
| 163 |
```
|
| 164 |
|
| 165 |
+
When your aspect-ratio selection is `auto`, use the `aspect_ratio` returned by the
|
| 166 |
+
model for image generation. If you select an explicit ratio, use your selected
|
| 167 |
+
ratio instead of the model's recommendation at the image-generation stage.
|
|
|
|
| 168 |
|
| 169 |
+
## How to use Image editing PE
|
| 170 |
|
| 171 |
Use the task prompt with a vision-language model that supports interleaved text
|
| 172 |
and images. The example below describes message assembly without selecting a
|
|
|
|
| 180 |
| --- | --- |
|
| 181 |
| `user_query` | The editing instruction. Refer to images in their supplied order. |
|
| 182 |
| `references` | At least one prepared image, in order, with its original display aspect ratio as a reduced `W:H` string. Use `None` when the source ratio is unknown. |
|
| 183 |
+
| `aspect_ratio` | The target ratio: `auto` to let the model choose, or an explicit positive-integer `W:H` string. |
|
| 184 |
|
| 185 |
### Assemble the message
|
| 186 |
|
| 187 |
Load [`prompt/edit/task_prompt.txt`](prompt/edit/task_prompt.txt) unchanged as the
|
| 188 |
first content part. Follow it with the image count, each image's label and source
|
| 189 |
+
ratio immediately before that image, and your editing instruction and target ratio
|
| 190 |
in the final text part. This produces one user message.
|
| 191 |
|
| 192 |
The fixed task prompt forms a shared prefix across requests, allowing reuse by
|
|
|
|
| 202 |
|
| 203 |
task_prompt = Path("prompt/edit/task_prompt.txt").read_text(encoding="utf-8")
|
| 204 |
user_query = "Replace the bag in the first image with the bag in the second image."
|
| 205 |
+
aspect_ratio = "auto" # Or an explicit ratio, such as "16:9".
|
| 206 |
references = [
|
| 207 |
{"image": source_image, "aspect_ratio": "3:2"},
|
| 208 |
{"image": reference_image, "aspect_ratio": "1:1"},
|
|
|
|
| 247 |
|
| 248 |
The description identifies source subjects, states what is preserved or changed,
|
| 249 |
and describes new content. Descriptive prose is English; literal text retains the
|
| 250 |
+
language you request. `target_aspect_ratio` is a positive-integer `W:H` string.
|
| 251 |
+
If you select an explicit ratio, the model preserves it exactly. If you select
|
| 252 |
+
`auto`, the model determines a numeric ratio for the intended edited image.
|
prompt/edit/task_prompt.txt
CHANGED
|
@@ -3,7 +3,7 @@ You are an expert at enhancing a short image-editing instruction into a rich, vi
|
|
| 3 |
## What you are given
|
| 4 |
- Input image(s), labeled sequentially starting from <Picture 1>. These are the user's source and reference images.
|
| 5 |
- One short editing instruction. It may be terse, vague, informal, or phrased as a desired result rather than an imperative. The final text part supplies it as User Editing Instruction (JSON string).
|
| 6 |
-
- User Aspect Ratio Selection: any concrete positive-integer W:H ratio, or
|
| 7 |
- Each input image is immediately preceded by a Reference Media text part containing its <Picture N> label and aspect_ratio W:H. The caller has already reduced the image's original display width and height to their exact lowest integer ratio, after EXIF orientation and before preview resizing or padding. No pixel dimensions are supplied in this metadata. The ratio applies only to the immediately following image. The supplied aspect_ratio is the reference image's exact ratio, not necessarily the final target ratio. Use it as given; do not estimate the source ratio from the preview. Determine the final target ratio using the rules below. If the original dimensions are unknown, the metadata says aspect_ratio unavailable.
|
| 8 |
- There is no target image. Construct the intended target from the visible source evidence and the instruction; never pretend to have observed a hidden result.
|
| 9 |
|
|
@@ -22,8 +22,8 @@ Read User Aspect Ratio Selection from the final input part.
|
|
| 22 |
Specified ratio:
|
| 23 |
If a concrete ratio is supplied, copy that exact string to target_aspect_ratio. Any positive-integer W:H ratio is allowed; it is not limited to a preset list. Neither the editing instruction nor a reference image may override it.
|
| 24 |
|
| 25 |
-
|
| 26 |
-
Only when the selection is
|
| 27 |
|
| 28 |
1. Follow an explicit final target ratio or target dimensions in the editing instruction. Use the requested final geometry rather than source or earlier geometry. Exact numeric geometry takes precedence over generic orientation. When only a final orientation is specified, use 21:9 for ultra-wide, 16:9 for landscape, 1:1 for square, and 9:16 for portrait.
|
| 29 |
|
|
@@ -41,7 +41,7 @@ Only when the selection is adaptive, determine the aspect ratio of the intended
|
|
| 41 |
|
| 42 |
Apply these rules to the whole target canvas. Grid row/column counts, individual panels, objects inside the image, and literal text are not whole-canvas ratio requests. A portrait subject or wide-angle lens alone is not a target-orientation request. Do not use a reference board's ratio as the ratio of one panel.
|
| 43 |
|
| 44 |
-
Serialize target_aspect_ratio as two positive integers in W:H form, with no spaces; never output
|
| 45 |
|
| 46 |
Later descriptive additions must not change the chosen ratio. Pixel-size resolution, 32 alignment, and downstream size fallback are outside this task.
|
| 47 |
|
|
@@ -185,7 +185,7 @@ Before answering, verify:
|
|
| 185 |
- all non-literal descriptive and relationship prose is English, while quoted source, user-specified, or newly authored requested text uses its required language and script;
|
| 186 |
- any grid layout described makes the row count and column count unambiguous, either in words or with explicitly labeled multiplication notation, with positions or uneven-row structure clarified where needed;
|
| 187 |
- expanded_caption is one fluent, complete paragraph with no structural meta-language in the narration; multi-line literals retain their wording, reading order, and intended line boundaries through inline quoted lines;
|
| 188 |
-
- target_aspect_ratio copies any concrete user selection exactly; in
|
| 189 |
- the final answer is a valid JSON object containing only the two string fields target_aspect_ratio and expanded_caption.
|
| 190 |
|
| 191 |
Output only the required two-field JSON object.
|
|
|
|
| 3 |
## What you are given
|
| 4 |
- Input image(s), labeled sequentially starting from <Picture 1>. These are the user's source and reference images.
|
| 5 |
- One short editing instruction. It may be terse, vague, informal, or phrased as a desired result rather than an imperative. The final text part supplies it as User Editing Instruction (JSON string).
|
| 6 |
+
- User Aspect Ratio Selection: any concrete positive-integer W:H ratio, or auto, supplied in the final text part.
|
| 7 |
- Each input image is immediately preceded by a Reference Media text part containing its <Picture N> label and aspect_ratio W:H. The caller has already reduced the image's original display width and height to their exact lowest integer ratio, after EXIF orientation and before preview resizing or padding. No pixel dimensions are supplied in this metadata. The ratio applies only to the immediately following image. The supplied aspect_ratio is the reference image's exact ratio, not necessarily the final target ratio. Use it as given; do not estimate the source ratio from the preview. Determine the final target ratio using the rules below. If the original dimensions are unknown, the metadata says aspect_ratio unavailable.
|
| 8 |
- There is no target image. Construct the intended target from the visible source evidence and the instruction; never pretend to have observed a hidden result.
|
| 9 |
|
|
|
|
| 22 |
Specified ratio:
|
| 23 |
If a concrete ratio is supplied, copy that exact string to target_aspect_ratio. Any positive-integer W:H ratio is allowed; it is not limited to a preset list. Neither the editing instruction nor a reference image may override it.
|
| 24 |
|
| 25 |
+
Auto mode:
|
| 26 |
+
Only when the selection is auto, determine the aspect ratio of the intended finished target image, which may differ from the main reference image. Apply these rules in order:
|
| 27 |
|
| 28 |
1. Follow an explicit final target ratio or target dimensions in the editing instruction. Use the requested final geometry rather than source or earlier geometry. Exact numeric geometry takes precedence over generic orientation. When only a final orientation is specified, use 21:9 for ultra-wide, 16:9 for landscape, 1:1 for square, and 9:16 for portrait.
|
| 29 |
|
|
|
|
| 41 |
|
| 42 |
Apply these rules to the whole target canvas. Grid row/column counts, individual panels, objects inside the image, and literal text are not whole-canvas ratio requests. A portrait subject or wide-angle lens alone is not a target-orientation request. Do not use a reference board's ratio as the ratio of one panel.
|
| 43 |
|
| 44 |
+
Serialize target_aspect_ratio as two positive integers in W:H form, with no spaces; never output auto. Preserve concrete user selections, explicit positive-integer target ratios in the instruction, and unchanged reference ratios exactly. Convert other explicit numeric ratios or target dimensions to their exact reduced integer ratio; reduce newly calculated or chosen ratios to their lowest integer form. Do not snap to preset ratios or clamp to a supported range.
|
| 45 |
|
| 46 |
Later descriptive additions must not change the chosen ratio. Pixel-size resolution, 32 alignment, and downstream size fallback are outside this task.
|
| 47 |
|
|
|
|
| 185 |
- all non-literal descriptive and relationship prose is English, while quoted source, user-specified, or newly authored requested text uses its required language and script;
|
| 186 |
- any grid layout described makes the row count and column count unambiguous, either in words or with explicitly labeled multiplication notation, with positions or uneven-row structure clarified where needed;
|
| 187 |
- expanded_caption is one fluent, complete paragraph with no structural meta-language in the narration; multi-line literals retain their wording, reading order, and intended line boundaries through inline quoted lines;
|
| 188 |
+
- target_aspect_ratio copies any concrete user selection exactly; in auto mode, it preserves the main reference ratio when no canvas-shape change is required and reflects any requested expansion or cropping; expanded_caption and its preservation relationships agree with the final canvas;
|
| 189 |
- the final answer is a valid JSON object containing only the two string fields target_aspect_ratio and expanded_caption.
|
| 190 |
|
| 191 |
Output only the required two-field JSON object.
|
prompt/manifest.json
CHANGED
|
@@ -15,7 +15,7 @@
|
|
| 15 |
"edit": {
|
| 16 |
"files": {
|
| 17 |
"prompt/edit/task_prompt.txt": {
|
| 18 |
-
"sha256": "
|
| 19 |
}
|
| 20 |
}
|
| 21 |
}
|
|
|
|
| 15 |
"edit": {
|
| 16 |
"files": {
|
| 17 |
"prompt/edit/task_prompt.txt": {
|
| 18 |
+
"sha256": "8aafd512c4ac61e9cbc2a6e9a23c23f4dff2495886b48f663578e6330e01fce5"
|
| 19 |
}
|
| 20 |
}
|
| 21 |
}
|