Improve model card for Intern-S1-GGUF: Add paper, abstract, project links & enhance description
#1
by nielsr HF Staff - opened
README.md
CHANGED
|
@@ -1,35 +1,160 @@
|
|
| 1 |
---
|
| 2 |
-
|
|
|
|
| 3 |
language:
|
| 4 |
- en
|
| 5 |
-
|
| 6 |
-
- internlm/Intern-S1
|
| 7 |
-
|
| 8 |
-
base_model_relation: quantized
|
| 9 |
-
|
| 10 |
pipeline_tag: image-text-to-text
|
| 11 |
tags:
|
| 12 |
- chat
|
|
|
|
| 13 |
---
|
| 14 |
|
| 15 |
-
# Intern-S1-GGUF Model
|
| 16 |
|
| 17 |

|
| 18 |
|
| 19 |
-
|
| 20 |
<p align="center">
|
| 21 |
👋 join us on <a href="https://discord.gg/xa29JuW87d" target="_blank">Discord</a> and <a href="https://cdn.vansin.top/intern-s1.jpg" target="_blank">WeChat</a>
|
| 22 |
</p>
|
| 23 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 24 |
## Introduction
|
| 25 |
|
| 26 |
-
|
| 27 |
-
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 28 |
|
| 29 |
-
|
| 30 |
-
And finally we will illustrate the methods for model inference and service deployment through specific examples.
|
| 31 |
|
| 32 |
-
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 33 |
|
| 34 |
We recommend building `llama.cpp` from source. The following code snippet provides an example for the Linux CUDA platform. For instructions on other platforms, please refer to the [official guide](https://github.com/ggerganov/llama.cpp?tab=readme-ov-file#build).
|
| 35 |
|
|
@@ -54,17 +179,17 @@ All the built targets can be found in the sub directory `build/bin`
|
|
| 54 |
|
| 55 |
In the following sections, we assume that the working directory is at the root directory of `llama.cpp`.
|
| 56 |
|
| 57 |
-
## Download models
|
| 58 |
|
| 59 |
In the [introduction section](#introduction), we mentioned that this repository includes several models with varying levels of computational precision. You can download the appropriate model based on your requirements.
|
| 60 |
For instance, fp16 gguf files can be downloaded as below:
|
| 61 |
|
| 62 |
```shell
|
| 63 |
pip install huggingface-hub
|
| 64 |
-
huggingface-cli download internlm/Intern-S1-GGUF
|
| 65 |
```
|
| 66 |
|
| 67 |
-
## Merge model files
|
| 68 |
|
| 69 |
Run the following command to merge gguf files into one:
|
| 70 |
|
|
@@ -75,20 +200,23 @@ Intern-S1-GGUF/f16/Intern-S1-f16-00001-of-00016.gguf \
|
|
| 75 |
Intern-S1-GGUF/f16/Intern-S1-f16.gguf
|
| 76 |
```
|
| 77 |
|
| 78 |
-
## Inference
|
| 79 |
|
| 80 |
You can use `build/bin/llama-mtmd-cli` for conducting inference. For a detailed explanation of `build/bin/llama-mtmd-cli`, please refer to [this guide](https://github.com/ggerganov/llama.cpp/blob/master/examples/main/README.md)
|
| 81 |
|
| 82 |
-
###
|
| 83 |
|
| 84 |
Here is an example of using the thinking system prompt.
|
| 85 |
|
| 86 |
```shell
|
| 87 |
|
| 88 |
-
system_prompt="<|im_start|>system
|
|
|
|
|
|
|
|
|
|
| 89 |
|
| 90 |
build/bin/llama-mtmd-cli \
|
| 91 |
-
--model Intern-S1-GGUF/f16/Intern-S1-f16.gguf
|
| 92 |
--mmproj Intern-S1-GGUF/f16/mmproj-Intern-S1-f16.gguf \
|
| 93 |
--predict 2048 \
|
| 94 |
--ctx-size 8192 \
|
|
@@ -101,7 +229,7 @@ build/bin/llama-mtmd-cli \
|
|
| 101 |
|
| 102 |
Then input your question with image input as `/image xxx.jpg`.
|
| 103 |
|
| 104 |
-
## Serving
|
| 105 |
|
| 106 |
`llama.cpp` provides an OpenAI API compatible server - `llama-server`. You can deploy the model as a service like this:
|
| 107 |
|
|
@@ -139,8 +267,7 @@ response = client.chat.completions.create(
|
|
| 139 |
print(response)
|
| 140 |
```
|
| 141 |
|
| 142 |
-
|
| 143 |
-
## Ollama
|
| 144 |
|
| 145 |
```shell
|
| 146 |
# install ollama
|
|
@@ -151,3 +278,250 @@ ollama pull internlm/interns1
|
|
| 151 |
ollama run internlm/interns1
|
| 152 |
# then use openai client to call on http://localhost:11434/v1
|
| 153 |
```
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
---
|
| 2 |
+
base_model:
|
| 3 |
+
- internlm/Intern-S1
|
| 4 |
language:
|
| 5 |
- en
|
| 6 |
+
license: apache-2.0
|
|
|
|
|
|
|
|
|
|
|
|
|
| 7 |
pipeline_tag: image-text-to-text
|
| 8 |
tags:
|
| 9 |
- chat
|
| 10 |
+
base_model_relation: quantized
|
| 11 |
---
|
| 12 |
|
| 13 |
+
# Intern-S1-GGUF: A Scientific Multimodal Foundation Model
|
| 14 |
|
| 15 |

|
| 16 |
|
|
|
|
| 17 |
<p align="center">
|
| 18 |
👋 join us on <a href="https://discord.gg/xa29JuW87d" target="_blank">Discord</a> and <a href="https://cdn.vansin.top/intern-s1.jpg" target="_blank">WeChat</a>
|
| 19 |
</p>
|
| 20 |
|
| 21 |
+
## Paper
|
| 22 |
+
The model was presented in the paper [Intern-S1: A Scientific Multimodal Foundation Model](https://huggingface.co/papers/2508.15763).
|
| 23 |
+
|
| 24 |
+
## Abstract
|
| 25 |
+
In recent years, a plethora of open-source foundation models have emerged, achieving remarkable progress in some widely attended fields, with performance being quite close to that of closed-source models. However, in high-value but more challenging scientific professional fields, either the fields still rely on expert models, or the progress of general foundation models lags significantly compared to those in popular areas, far from sufficient for transforming scientific research and leaving substantial gap between open-source models and closed-source models in these scientific domains. To mitigate this gap and explore a step further toward Artificial General Intelligence (AGI), we introduce Intern-S1, a specialized generalist equipped with general understanding and reasoning capabilities with expertise to analyze multiple science modal data. Intern-S1 is a multimodal Mixture-of-Experts (MoE) model with 28 billion activated parameters and 241 billion total parameters, continually pre-trained on 5T tokens, including over 2.5T tokens from scientific domains. In the post-training stage, Intern-S1 undergoes offline and then online reinforcement learning (RL) in InternBootCamp, where we propose Mixture-of-Rewards (MoR) to synergize the RL training on more than 1000 tasks simultaneously. Through integrated innovations in algorithms, data, and training systems, Intern-S1 achieved top-tier performance in online RL training. On comprehensive evaluation benchmarks, Intern-S1 demonstrates competitive performance on general reasoning tasks among open-source models and significantly outperforms open-source models in scientific domains, surpassing closed-source state-of-the-art models in professional tasks, such as molecular synthesis planning, reaction condition prediction, predicting thermodynamic stabilities for crystals. Our models are available at this https URL .
|
| 26 |
+
|
| 27 |
+
## Links
|
| 28 |
+
* **GitHub Repository**: [InternLM/Intern-S1](https://github.com/InternLM/Intern-S1)
|
| 29 |
+
* **Online Demo**: [Intern-S1 Chat](https://chat.intern-ai.org.cn/)
|
| 30 |
+
|
| 31 |
## Introduction
|
| 32 |
|
| 33 |
+
We introduce **Intern-S1**, our **most advanced open-source multimodal reasoning model** to date. Intern-S1 combines **strong general-task capabilities with state-of-the-art performance on a wide range of scientific tasks**, rivaling leading closed-source commercial models.
|
| 34 |
+
|
| 35 |
+
Built upon a 235B MoE language model (Qwen3) and a 6B Vision encoder (InternViT), Intern-S1 has been further pretrained on **5 trillion tokens** of multimodal data, including over **2.5 trillion scientific-domain tokens**. This enables the model to retain strong general capabilities while excelling in specialized scientific domains such as **interpreting chemical structures, understanding protein sequences, and planning compound synthesis routes**, making Intern-S1 to be a capable research assistant for real-world scientific applications.
|
| 36 |
+
|
| 37 |
+
This repository offers `Intern-S1` models in GGUF format, which can be utilized by [llama.cpp](https://github.com/ggerganov/llama.cpp), a highly popular open-source framework for Large Language Model (LLM) inference, across a variety of hardware platforms, both locally and in the cloud. This repository provides `Intern-S1` models in GGUF format in both half precision and various low-bit quantized versions, including `q8_0`.
|
| 38 |
+
|
| 39 |
+
In the subsequent sections, we will first present the installation procedure, followed by an explanation of the model download process. And finally we will illustrate the methods for model inference and service deployment through specific examples.
|
| 40 |
+
|
| 41 |
+
## Features
|
| 42 |
+
|
| 43 |
+
* Strong performance across language and vision reasoning benchmarks, especially scientific tasks.
|
| 44 |
+
* Continuously pretrained on a massive 5T token dataset, with over 50% specialized scientific data, embedding deep domain expertise.
|
| 45 |
+
* Dynamic tokenizer enables native understanding of molecular formulas, protein sequences, and seismic signals.
|
| 46 |
+
|
| 47 |
+
## Model Zoo
|
| 48 |
+
|
| 49 |
+
### Intern-S1
|
| 50 |
+
|
| 51 |
+
| | BF16 | FP8 | GGUF |
|
| 52 |
+
| :----------------------------------------------------------------: | :------------------------------------------------------------------------------------------------ | :-------------------------------------------------------------------------------------------------------- | :---------------------------------------------------------------------------------------------------------- |
|
| 53 |
+
| 🤗HuggingFace | [internlm/Intern-S1](https://huggingface.co/internlm/Intern-S1) | [internlm/Intern-S1-FP8](https://huggingface.co/internlm/Intern-S1-FP8) | [internlm/Intern-S1-GGUF](https://huggingface.co/internlm/Intern-S1-GGUF) |
|
| 54 |
+
| <img src="./assets/modelscope_logo.png" width="20px" /> ModelScope | [Shanghai\_AI\_Laboratory/Intern-S1](https://modelscope.cn/models/Shanghai_AI_Laboratory/Intern-S1) | [Shanghai\_AI\_Laboratory/Intern-S1-FP8](https://modelscope.cn/models/Shanghai_AI_Laboratory/Intern-S1-FP8) | [Shanghai\_AI\_Laboratory/Intern-S1-GGUF](https://modelscope.cn/models/Shanghai_AI_Laboratory/Intern-S1-GGUF) |
|
| 55 |
+
|
| 56 |
+
### Intern-S1-mini
|
| 57 |
+
|
| 58 |
+
| | BF16 | FP8 | GGUF |
|
| 59 |
+
| :----------------------------------------------------------------: | :---------------------------------------------------------------------------------------------------------- | :------------------------------------------------------------------------------------------------------------------ | :---------------------------------------------------------------------------------- |
|
| 60 |
+
| 🤗HuggingFace | [internlm/Intern-S1-mini](https://huggingface.co/internlm/Intern-S1-mini) | [internlm/Intern-S1-mini-FP8](https://huggingface.co/internlm/Intern-S1-mini-FP8) | [internlm/Intern-S1-mini-GGUF](https://huggingface.co/internlm/Intern-S1-mini-GGUF) |
|
| 61 |
+
| <img src="./assets/modelscope_logo.png" width="20px" /> ModelScope | [Shanghai\_AI\_Laboratory/Intern-S1-mini](https://modelscope.cn/models/Shanghai_AI_Laboratory/Intern-S1-mini) | [Shanghai\_AI\_Laboratory/Intern-S1-mini-FP8](https://modelscope.cn/models/Shanghai_AI_Laboratory/Intern-S1-mini-FP8) | - |
|
| 62 |
+
|
| 63 |
+
## Performance
|
| 64 |
+
|
| 65 |
+
We evaluate the Intern-S1 on various benchmarks including general datasets and scientific datasets. We report the performance comparison with the recent VLMs and LLMs below.
|
| 66 |
+
|
| 67 |
+
### Intern-S1
|
| 68 |
+
|
| 69 |
+
<table>
|
| 70 |
+
<thead>
|
| 71 |
+
<tr>
|
| 72 |
+
<th rowspan="2">Benchmarks</th>
|
| 73 |
+
<th colspan="2">Intern-S1</th>
|
| 74 |
+
<th>InternVL3-78B</th>
|
| 75 |
+
<th>Qwen2.5-VL-72B</th>
|
| 76 |
+
<th>DS-R1-0528</th>
|
| 77 |
+
<th>Qwen3-235B-A22B</th>
|
| 78 |
+
<th>Kimi-K2-Instruct</th>
|
| 79 |
+
<th>Gemini-2.5 Pro</th>
|
| 80 |
+
<th>o3</th>
|
| 81 |
+
<th>Grok-4</th>
|
| 82 |
+
</tr>
|
| 83 |
+
</thead>
|
| 84 |
+
<tbody>
|
| 85 |
+
<tr><td>MMLU-Pro</td><td colspan="2">83.5 ✅</td><td>73.0</td><td>72.1</td><td>83.4</td><td>82.2</td><td>82.7</td><td>86.0</td><td>85.0</td><td>85.9</td></tr>
|
| 86 |
+
<tr><td>MMMU</td><td colspan="2">77.7 ✅</td><td>72.2</td><td>70.2</td><td>-</td><td>-</td><td>-</td><td>81.9</td><td>80.8</td><td>77.9</td></tr>
|
| 87 |
+
<tr><td>GPQA</td><td colspan="2">77.3</td><td>49.9</td><td>49.0</td><td>80.6</td><td>71.1</td><td>77.8</td><td>83.8</td><td>83.3</td><td>87.5</td></tr>
|
| 88 |
+
<tr><td>MMStar</td><td colspan="2">74.9 ✅</td><td>72.5</td><td>70.8</td><td>-</td><td>-</td><td>-</td><td>79.3</td><td>75.1</td><td>69.6</td></tr>
|
| 89 |
+
<tr><td>MathVista</td><td colspan="2">81.5 👑</td><td>79.0</td><td>74.8</td><td>-</td><td>-</td><td>-</td><td>80.3</td><td>77.5</td><td>72.5</td></tr>
|
| 90 |
+
<tr><td>AIME2025</td><td colspan="2">86.0</td><td>10.7</td><td>10.9</td><td>87.5</td><td>81.5</td><td>51.4</td><td>83.0</td><td>88.9</td><td>91.7</td></tr>
|
| 91 |
+
<tr><td>MathVision</td><td colspan="2">62.5 ✅</td><td>43.1</td><td>38.1</td><td>-</td><td>-</td><td>-</td><td>73.0</td><td>67.7</td><td>67.3</td></tr>
|
| 92 |
+
<tr><td>IFEval</td><td colspan="2">86.7</td><td>75.6</td><td>83.9</td><td>79.7</td><td>85.0</td><td>90.2</td><td>91.5</td><td>92.2</td><td>92.8</td></tr>
|
| 93 |
+
<tr><td>SFE</td><td colspan="2">44.3 👑</td><td>36.2</td><td>30.5</td><td>-</td><td>-</td><td>-</td><td>43.0</td><td>37.7</td><td>31.2</td></tr>
|
| 94 |
+
<tr><td>Physics</td><td colspan="2">44.0 ✅</td><td>23.1</td><td>15.7</td><td>-</td><td>-</td><td>-</td><td>40.0</td><td>47.9</td><td>42.8</td></tr>
|
| 95 |
+
<tr><td>SmolInstruct</td><td colspan="2">51.0 👑</td><td>19.4</td><td>21.0</td><td>30.7</td><td>28.7</td><td>48.1</td><td>40.4</td><td>43.9</td><td>47.3</td></tr>
|
| 96 |
+
<tr><td>ChemBench</td><td colspan="2">83.4 👑</td><td>61.3</td><td>61.6</td><td>75.6</td><td>75.8</td><td>75.3</td><td>82.8</td><td>81.6</td><td>83.3</td></tr>
|
| 97 |
+
<tr><td>MatBench</td><td colspan="2">75.0 👑</td><td>49.3</td><td>51.5</td><td>57.7</td><td>52.1</td><td>61.7</td><td>61.7</td><td>61.6</td><td>67.9</td></tr>
|
| 98 |
+
<tr><td>MicroVQA</td><td colspan="2">63.9 👑</td><td>59.1</td><td>53.0</td><td>-</td><td>-</td><td>-</td><td>63.1</td><td>58.3</td><td>59.5</td></tr>
|
| 99 |
+
<tr><td>ProteinLMBench</td><td colspan="2">63.1</td><td>61.6</td><td>61.0</td><td>61.4</td><td>59.8</td><td>66.7</td><td>62.9</td><td>67.7</td><td>66.2</td></tr>
|
| 100 |
+
<tr><td>MSEarthMCQ</td><td colspan="2">65.7 👑</td><td>57.2</td><td>37.6</td><td>-</td><td>-</td><td>-</td><td>59.9</td><td>61.0</td><td>58.0</td></tr>
|
| 101 |
+
<tr><td>XLRS-Bench</td><td colspan="2">55.0 👑</td><td>49.3</td><td>50.9</td><td>-</td><td>-</td><td>-</td><td>45.2</td><td>43.6</td><td>45.4</td></tr>
|
| 102 |
+
</tbody>
|
| 103 |
+
</table>
|
| 104 |
+
|
| 105 |
+
> **Note**: ✅ means the best performance among open-sourced models, 👑 indicates the best performance among all models.
|
| 106 |
+
|
| 107 |
+
### Intern-S1-mini
|
| 108 |
+
|
| 109 |
+
| Benchmarks | Intern-S1-mini | Qwen3-8B | GLM-4.1V | MiMo-VL-7B-RL-2508 |
|
| 110 |
+
| :------------- | :------------- | :------- | :------- | :----------------- |
|
| 111 |
+
| MMLU-Pro | **74.78** | 73.7 | 57.1 | 73.93 |
|
| 112 |
+
| MMMU | **72.33** | - | 69.9 | 70.4 |
|
| 113 |
+
| MMStar | 65.2 | - | 71.5 | 72.9 |
|
| 114 |
+
| GPQA | **65.15** | 62 | 50.32 | 60.35 |
|
| 115 |
+
| AIME2024 | **84.58** | 76 | 36.2 | 72.6 |
|
| 116 |
+
| AIME2025 | **80** | 67.3 | 32 | 64.4 |
|
| 117 |
+
| MathVision | 51.41 | - | 53.9 | 54.5 |
|
| 118 |
+
| MathVista | 70.3 | - | 80.7 | 79.4 |
|
| 119 |
+
| IFEval | 81.15 | 85 | 71.53 | 71.4 |
|
| 120 |
+
| SFE | 35.84 | - | 43.2 | 43.9 |
|
| 121 |
+
| Physics | **28.76** | - | 28.3 | 28.2 |
|
| 122 |
+
| SmolInstruct | **32.2** | 17.6 | 18.1 | 16.11 |
|
| 123 |
+
| ChemBench | **76.47** | 61.1 | 56.2 | 66.78 |
|
| 124 |
+
| MatBench | **61.55** | 45.24 | 54.3 | 46.9 |
|
| 125 |
+
| MicroVQA | **56.62** | - | 50.2 | 50.96 |
|
| 126 |
+
| ProteinLMBench | 58.47 | 59.1 | 58.3 | 59.8 |
|
| 127 |
+
| MSEarthMCQ | **58.12** | - | 50.3 | 47.3 |
|
| 128 |
+
| XLRS-Bench | **51.63** | - | 49.8 | 12.29 |
|
| 129 |
+
|
| 130 |
+
We use the [OpenCompass](https://github.com/open-compass/OpenCompass/) and [VLMEvalkit](https://github.com/open-compass/vlmevalkit) to evaluate all models.
|
| 131 |
+
Please refer to [this page](https://opencompass.readthedocs.io/en/latest/user_guides/interns1.html) to quickly start the text-only evaluation task.
|
| 132 |
+
|
| 133 |
+
## Quick Start (GGUF via llama.cpp and Ollama)
|
| 134 |
+
|
| 135 |
+
### Sampling Parameters
|
| 136 |
+
|
| 137 |
+
We recommend using the following hyperparameters to ensure better results
|
| 138 |
+
|
| 139 |
+
For Intern-S1:
|
| 140 |
+
|
| 141 |
+
```python
|
| 142 |
+
top_p = 1.0
|
| 143 |
+
top_k = 50
|
| 144 |
+
min_p = 0.0
|
| 145 |
+
temperature = 0.7
|
| 146 |
+
```
|
| 147 |
|
| 148 |
+
For Intern-S1-mini:
|
|
|
|
| 149 |
|
| 150 |
+
```python
|
| 151 |
+
top_p = 1.0
|
| 152 |
+
top_k = 50
|
| 153 |
+
min_p = 0.0
|
| 154 |
+
temperature = 0.8
|
| 155 |
+
```
|
| 156 |
+
|
| 157 |
+
### Installation (llama.cpp)
|
| 158 |
|
| 159 |
We recommend building `llama.cpp` from source. The following code snippet provides an example for the Linux CUDA platform. For instructions on other platforms, please refer to the [official guide](https://github.com/ggerganov/llama.cpp?tab=readme-ov-file#build).
|
| 160 |
|
|
|
|
| 179 |
|
| 180 |
In the following sections, we assume that the working directory is at the root directory of `llama.cpp`.
|
| 181 |
|
| 182 |
+
### Download models (GGUF)
|
| 183 |
|
| 184 |
In the [introduction section](#introduction), we mentioned that this repository includes several models with varying levels of computational precision. You can download the appropriate model based on your requirements.
|
| 185 |
For instance, fp16 gguf files can be downloaded as below:
|
| 186 |
|
| 187 |
```shell
|
| 188 |
pip install huggingface-hub
|
| 189 |
+
huggingface-cli download internlm/Intern-S1-GGUF --include *-f16-*.gguf --local-dir Intern-S1-GGUF --local-dir-use-symlinks False
|
| 190 |
```
|
| 191 |
|
| 192 |
+
### Merge model files (GGUF)
|
| 193 |
|
| 194 |
Run the following command to merge gguf files into one:
|
| 195 |
|
|
|
|
| 200 |
Intern-S1-GGUF/f16/Intern-S1-f16.gguf
|
| 201 |
```
|
| 202 |
|
| 203 |
+
### Inference (llama.cpp)
|
| 204 |
|
| 205 |
You can use `build/bin/llama-mtmd-cli` for conducting inference. For a detailed explanation of `build/bin/llama-mtmd-cli`, please refer to [this guide](https://github.com/ggerganov/llama.cpp/blob/master/examples/main/README.md)
|
| 206 |
|
| 207 |
+
#### Chat example
|
| 208 |
|
| 209 |
Here is an example of using the thinking system prompt.
|
| 210 |
|
| 211 |
```shell
|
| 212 |
|
| 213 |
+
system_prompt="<|im_start|>system
|
| 214 |
+
You are an expert reasoner with extensive experience in all areas. You approach problems through systematic thinking and rigorous reasoning. Your response should reflect deep understanding and precise logical thinking, making your solution path and reasoning clear to others. Please put your thinking process within <think>...</think> tags.
|
| 215 |
+
<|im_end|>
|
| 216 |
+
"
|
| 217 |
|
| 218 |
build/bin/llama-mtmd-cli \
|
| 219 |
+
--model Intern-S1-GGUF/f16/Intern-S1-f16.gguf \
|
| 220 |
--mmproj Intern-S1-GGUF/f16/mmproj-Intern-S1-f16.gguf \
|
| 221 |
--predict 2048 \
|
| 222 |
--ctx-size 8192 \
|
|
|
|
| 229 |
|
| 230 |
Then input your question with image input as `/image xxx.jpg`.
|
| 231 |
|
| 232 |
+
### Serving (llama.cpp)
|
| 233 |
|
| 234 |
`llama.cpp` provides an OpenAI API compatible server - `llama-server`. You can deploy the model as a service like this:
|
| 235 |
|
|
|
|
| 267 |
print(response)
|
| 268 |
```
|
| 269 |
|
| 270 |
+
### Ollama
|
|
|
|
| 271 |
|
| 272 |
```shell
|
| 273 |
# install ollama
|
|
|
|
| 278 |
ollama run internlm/interns1
|
| 279 |
# then use openai client to call on http://localhost:11434/v1
|
| 280 |
```
|
| 281 |
+
|
| 282 |
+
## Advanced Usage
|
| 283 |
+
|
| 284 |
+
### Tool Calling
|
| 285 |
+
|
| 286 |
+
Many Large Language Models (LLMs) now feature **Tool Calling**, a powerful capability that allows them to extend their functionality by interacting with external tools and APIs. This enables models to perform tasks like fetching up-to-the-minute information, running code, or calling functions within other applications.
|
| 287 |
+
|
| 288 |
+
A key advantage for developers is that a growing number of open-source LLMs are designed to be compatible with the OpenAI API. This means you can leverage the same familiar syntax and structure from the OpenAI library to implement tool calling with these open-source models. As a result, the code demonstrated in this tutorial is versatile—it works not just with OpenAI models, but with any model that follows the same interface standard.
|
| 289 |
+
|
| 290 |
+
To illustrate how this works, let's dive into a practical code example that uses tool calling to get the latest weather forecast (based on lmdeploy api server).
|
| 291 |
+
|
| 292 |
+
```python
|
| 293 |
+
|
| 294 |
+
from openai import OpenAI
|
| 295 |
+
import json
|
| 296 |
+
|
| 297 |
+
|
| 298 |
+
def get_current_temperature(location: str, unit: str = "celsius"):
|
| 299 |
+
"""Get current temperature at a location.
|
| 300 |
+
|
| 301 |
+
Args:
|
| 302 |
+
location: The location to get the temperature for, in the format "City, State, Country".
|
| 303 |
+
unit: The unit to return the temperature in. Defaults to "celsius". (choices: ["celsius", "fahrenheit"])
|
| 304 |
+
|
| 305 |
+
Returns:
|
| 306 |
+
the temperature, the location, and the unit in a dict
|
| 307 |
+
"""
|
| 308 |
+
return {
|
| 309 |
+
"temperature": 26.1,
|
| 310 |
+
"location": location,
|
| 311 |
+
"unit": unit,
|
| 312 |
+
}
|
| 313 |
+
|
| 314 |
+
|
| 315 |
+
def get_temperature_date(location: str, date: str, unit: str = "celsius"):
|
| 316 |
+
"""Get temperature at a location and date.
|
| 317 |
+
|
| 318 |
+
Args:
|
| 319 |
+
location: The location to get the temperature for, in the format "City, State, Country".
|
| 320 |
+
date: The date to get the temperature for, in the format "Year-Month-Day".
|
| 321 |
+
unit: The unit to return the temperature in. Defaults to "celsius". (choices: ["celsius", "fahrenheit"])
|
| 322 |
+
|
| 323 |
+
Returns:
|
| 324 |
+
the temperature, the location, the date and the unit in a dict
|
| 325 |
+
"""
|
| 326 |
+
return {
|
| 327 |
+
"temperature": 25.9,
|
| 328 |
+
"location": location,
|
| 329 |
+
"date": date,
|
| 330 |
+
"unit": unit,
|
| 331 |
+
}
|
| 332 |
+
|
| 333 |
+
def get_function_by_name(name):
|
| 334 |
+
if name == "get_current_temperature":
|
| 335 |
+
return get_current_temperature
|
| 336 |
+
if name == "get_temperature_date":
|
| 337 |
+
return get_temperature_date
|
| 338 |
+
|
| 339 |
+
tools = [{
|
| 340 |
+
'type': 'function',
|
| 341 |
+
'function': {
|
| 342 |
+
'name': 'get_current_temperature',
|
| 343 |
+
'description': 'Get current temperature at a location.',
|
| 344 |
+
'parameters': {
|
| 345 |
+
'type': 'object',
|
| 346 |
+
'properties': {
|
| 347 |
+
'location': {
|
| 348 |
+
'type': 'string',
|
| 349 |
+
'description': 'The location to get the temperature for, in the format \'City, State, Country\'.'
|
| 350 |
+
},
|
| 351 |
+
'unit': {
|
| 352 |
+
'type': 'string',
|
| 353 |
+
'enum': [
|
| 354 |
+
'celsius',
|
| 355 |
+
'fahrenheit'
|
| 356 |
+
],
|
| 357 |
+
'description': 'The unit to return the temperature in. Defaults to \'celsius\'.'
|
| 358 |
+
}
|
| 359 |
+
},
|
| 360 |
+
'required': [
|
| 361 |
+
'location'
|
| 362 |
+
]
|
| 363 |
+
}
|
| 364 |
+
}
|
| 365 |
+
}, {
|
| 366 |
+
'type': 'function',
|
| 367 |
+
'function': {
|
| 368 |
+
'name': 'get_temperature_date',
|
| 369 |
+
'description': 'Get temperature at a location and date.',
|
| 370 |
+
'parameters': {
|
| 371 |
+
'type': 'object',
|
| 372 |
+
'properties': {
|
| 373 |
+
'location': {
|
| 374 |
+
'type': 'string',
|
| 375 |
+
'description': 'The location to get the temperature for, in the format \'City, State, Country\'.'
|
| 376 |
+
},
|
| 377 |
+
'date': {
|
| 378 |
+
'type': 'string',
|
| 379 |
+
'description': 'The date to get the temperature for, in the format \'Year-Month-Day\'.'
|
| 380 |
+
},
|
| 381 |
+
'unit': {
|
| 382 |
+
'type': 'string',
|
| 383 |
+
'enum': [
|
| 384 |
+
'celsius',
|
| 385 |
+
'fahrenheit'
|
| 386 |
+
],
|
| 387 |
+
'description': 'The unit to return the temperature in. Defaults to \'celsius\'.'
|
| 388 |
+
}
|
| 389 |
+
},
|
| 390 |
+
'required': [
|
| 391 |
+
'location',
|
| 392 |
+
'date'
|
| 393 |
+
]
|
| 394 |
+
}
|
| 395 |
+
}
|
| 396 |
+
}]
|
| 397 |
+
|
| 398 |
+
|
| 399 |
+
|
| 400 |
+
messages = [
|
| 401 |
+
{'role': 'user', 'content': 'Today is 2024-11-14, What\'s the temperature in San Francisco now? How about tomorrow?'}
|
| 402 |
+
]
|
| 403 |
+
|
| 404 |
+
openai_api_key = "EMPTY"
|
| 405 |
+
openai_api_base = "http://0.0.0.0:23333/v1"
|
| 406 |
+
client = OpenAI(
|
| 407 |
+
api_key=openai_api_key,
|
| 408 |
+
base_url=openai_api_base,
|
| 409 |
+
)
|
| 410 |
+
model_name = client.models.list().data[0].id
|
| 411 |
+
response = client.chat.completions.create(
|
| 412 |
+
model=model_name,
|
| 413 |
+
messages=messages,
|
| 414 |
+
max_tokens=32768,
|
| 415 |
+
temperature=0.8,
|
| 416 |
+
top_p=0.8,
|
| 417 |
+
stream=False,
|
| 418 |
+
extra_body=dict(spaces_between_special_tokens=False, enable_thinking=False),
|
| 419 |
+
tools=tools)
|
| 420 |
+
print(response.choices[0].message)
|
| 421 |
+
messages.append(response.choices[0].message)
|
| 422 |
+
|
| 423 |
+
for tool_call in response.choices[0].message.tool_calls:
|
| 424 |
+
tool_call_args = json.loads(tool_call.function.arguments)
|
| 425 |
+
tool_call_result = get_function_by_name(tool_call.function.name)(**tool_call_args)
|
| 426 |
+
tool_call_result = json.dumps(tool_call_result, ensure_ascii=False)
|
| 427 |
+
messages.append({
|
| 428 |
+
'role': 'tool',
|
| 429 |
+
'name': tool_call.function.name,
|
| 430 |
+
'content': tool_call_result,
|
| 431 |
+
'tool_call_id': tool_call.id
|
| 432 |
+
})
|
| 433 |
+
|
| 434 |
+
response = client.chat.completions.create(
|
| 435 |
+
model=model_name,
|
| 436 |
+
messages=messages,
|
| 437 |
+
temperature=0.8,
|
| 438 |
+
top_p=0.8,
|
| 439 |
+
stream=False,
|
| 440 |
+
extra_body=dict(spaces_between_special_tokens=False, enable_thinking=False),
|
| 441 |
+
tools=tools)
|
| 442 |
+
print(response.choices[0].message.content)
|
| 443 |
+
```
|
| 444 |
+
|
| 445 |
+
### Switching Between Thinking and Non-Thinking Modes
|
| 446 |
+
|
| 447 |
+
Intern-S1 enables thinking mode by default, enhancing the model's reasoning capabilities to generate higher-quality responses. This feature can be disabled by setting `enable_thinking=False` in `tokenizer.apply_chat_template`
|
| 448 |
+
|
| 449 |
+
```python
|
| 450 |
+
text = tokenizer.apply_chat_template(
|
| 451 |
+
messages,
|
| 452 |
+
tokenize=False,
|
| 453 |
+
add_generation_prompt=True,
|
| 454 |
+
enable_thinking=False # think mode indicator
|
| 455 |
+
)
|
| 456 |
+
```
|
| 457 |
+
|
| 458 |
+
With LMDeploy serving Intern-S1 models, you can dynamically control the thinking mode by adjusting the `enable_thinking` parameter in your requests.
|
| 459 |
+
|
| 460 |
+
```python
|
| 461 |
+
from openai import OpenAI
|
| 462 |
+
import json
|
| 463 |
+
|
| 464 |
+
messages = [
|
| 465 |
+
{
|
| 466 |
+
'role': 'user',
|
| 467 |
+
'content': 'who are you'
|
| 468 |
+
}, {
|
| 469 |
+
'role': 'assistant',
|
| 470 |
+
'content': 'I am an AI'
|
| 471 |
+
}, {
|
| 472 |
+
'role': 'user',
|
| 473 |
+
'content': 'AGI is?'
|
| 474 |
+
}]
|
| 475 |
+
|
| 476 |
+
openai_api_key = "EMPTY"
|
| 477 |
+
openai_api_base = "http://0.0.0.0:23333/v1"
|
| 478 |
+
client = OpenAI(
|
| 479 |
+
api_key=openai_api_key,
|
| 480 |
+
base_url=openai_api_base,
|
| 481 |
+
)
|
| 482 |
+
model_name = client.models.list().data[0].id
|
| 483 |
+
|
| 484 |
+
response = client.chat.completions.create(
|
| 485 |
+
model=model_name,
|
| 486 |
+
messages=messages,
|
| 487 |
+
temperature=0.7,
|
| 488 |
+
top_p=0.8,
|
| 489 |
+
max_tokens=2048,
|
| 490 |
+
extra_body={
|
| 491 |
+
"enable_thinking": False,
|
| 492 |
+
}
|
| 493 |
+
)
|
| 494 |
+
print(json.dumps(response.model_dump(), indent=2, ensure_ascii=False))
|
| 495 |
+
```
|
| 496 |
+
|
| 497 |
+
For vllm and sglang users, configure this through,
|
| 498 |
+
|
| 499 |
+
```python
|
| 500 |
+
extra_body={
|
| 501 |
+
"chat_template_kwargs": {"enable_thinking": false}
|
| 502 |
+
}
|
| 503 |
+
```
|
| 504 |
+
|
| 505 |
+
## Fine-tuning
|
| 506 |
+
|
| 507 |
+
See this [documentation](https://github.com/InternLM/Intern-S1/blob/main/docs/sft.md) for more details.
|
| 508 |
+
|
| 509 |
+
## License
|
| 510 |
+
|
| 511 |
+
This project is released under the Apache 2.0 license.
|
| 512 |
+
|
| 513 |
+
## Citation
|
| 514 |
+
|
| 515 |
+
If you find this work useful, feel free to give us a cite.
|
| 516 |
+
|
| 517 |
+
```
|
| 518 |
+
@misc{bai2025interns1scientificmultimodalfoundation,
|
| 519 |
+
title={Intern-S1: A Scientific Multimodal Foundation Model},
|
| 520 |
+
author={Lei Bai and Zhongrui Cai and Maosong Cao and Weihan Cao and Chiyu Chen and Haojiong Chen and Kai Chen and Pengcheng Chen and Ying Chen and Yongkang Chen and Yu Cheng and Yu Cheng and Pei Chu and Tao Chu and Erfei Cui and Ganqu Cui and Long Cui and Ziyun Cui and Nianchen Deng and Ning Ding and Nanqin Dong and Peijie Dong and Shihan Dou and Sinan Du and Haodong Duan and Caihua Fan and Ben Gao and Changjiang Gao and Jianfei Gao and Songyang Gao and Yang Gao and Zhangwei Gao and Jiaye Ge and Qiming Ge and Lixin Gu and Yuzhe Gu and Aijia Guo and Qipeng Guo and Xu Guo and Conghui He and Junjun He and Yili Hong and Siyuan Hou and Caiyu Hu and Hanglei Hu and Jucheng Hu and Ming Hu and Zhouqi Hua and Haian Huang and Junhao Huang and Xu Huang and Zixian Huang and Zhe Jiang and Lingkai Kong and Linyang Li and Peiji Li and Pengze Li and Shuaibin Li and Tianbin Li and Wei Li and Yuqiang Li and Dahua Lin and Junyao Lin and Tianyi Lin and Zhishan Lin and Hongwei Liu and Jiangning Liu and Jiyao Liu and Junnan Liu and Kai Liu and Kaiwen Liu and Kuikun Liu and Shichun Liu and Shudong Liu and Wei Liu and Xinyao Liu and Yuhong Liu and Zhan Liu and Yinquan Lu and Haijun Lv and Hongxia Lv and Huijie Lv and Qidang Lv and Ying Lv and Chengqi Lyu and Chenglong Ma and Jianpeng Ma and Ren Ma and Runmin Ma and Runyuan Ma and Xinzhu Ma and Yichuan Ma and Zihan Ma and Sixuan Mi and Junzhi Ning and Wenchang Ning and Xinle Pang and Jiahui Peng and Runyu Peng and Yu Qiao and Jiantao Qiu and Xiaoye Qu and Yuan Qu and Yuchen Ren and Fukai Shang and Wenqi Shao and Junhao Shen and Shuaike Shen and Chunfeng Song and Demin Song and Diping Song and Chenlin Su and Weijie Su and Weigao Sun and Yu Sun and Qian Tan and Cheng Tang and Huanze Tang and Kexian Tang and Shixiang Tang and Jian Tong and Aoran Wang and Bin Wang and Dong Wang and Lintao Wang and Rui Wang and Weiyun Wang and Wenhai Wang and Yi Wang and Ziyi Wang and Ling-I Wu and Wen Wu and Yue Wu and Zijian Wu and Linchen Xiao and Shuhao Xing and Chao Xu and Huihui Xu and Jun Xu and Ruiliang Xu and Wanghan Xu and GanLin Yang and Yuming Yang and Haochen Ye and Jin Ye and Shenglong Ye and Jia Yu and Jiashuo Yu and Jing Yu and Fei Yuan and Bo Zhang and Chao Zhang and Chen Zhang and Hongjie Zhang and Jin Zhang and Qiaosheng Zhang and Qiuyinzhe Zhang and Songyang Zhang and Taolin Zhang and Wenlong Zhang and Wenwei Zhang and Yechen Zhang and Ziyang Zhang and Haiteng Zhao and Qian Zhao and Xiangyu Zhao and Xiangyu Zhao and Bowen Zhou and Dongzhan Zhou and Peiheng Zhou and Yuhao Zhou and Yunhua Zhou and Dongsheng Zhu and Lin Zhu and Yicheng Zou},
|
| 521 |
+
year={2025},
|
| 522 |
+
eprint={2508.15763},
|
| 523 |
+
archivePrefix={arXiv},
|
| 524 |
+
primaryClass={cs.LG},
|
| 525 |
+
url={https://arxiv.org/abs/2508.15763},
|
| 526 |
+
}
|
| 527 |
+
```
|