bingyang-lei commited on
Commit
5b9f50d
·
verified ·
1 Parent(s): c26c873

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +20 -509
README.md CHANGED
@@ -2,524 +2,35 @@
2
  library_name: transformers
3
  license: apache-2.0
4
  license_link: https://huggingface.co/internlm/Intern-S2-Preview/blob/main/LICENSE
 
5
  pipeline_tag: image-text-to-text
6
  ---
7
 
8
- ## Intern-S2-Preview
9
 
10
- <div align="center">
11
- <img src="./figs/title.png" />
12
 
13
- <div>&nbsp;</div>
 
 
14
 
15
- [💻Github Repo](https://github.com/InternLM/Intern-S1) • [🤗Model Collections](https://huggingface.co/collections/internlm/intern-s2) • [💬Online Chat](https://chat.intern-ai.org.cn/)
16
 
17
- </div>
 
 
 
18
 
19
- <p align="center">
20
- 👋 join us on <a href="https://discord.gg/xa29JuW87d" target="_blank">Discord</a> and <a href="https://cdn.vansin.top/intern-s1.jpg" target="_blank">WeChat</a>
21
- </p>
22
 
 
23
 
 
 
 
24
 
25
- ## Introduction
26
 
27
- We introduce **Intern-S2-Preview**, an efficient 35B scientific multimodal foundation model. Beyond conventional parameter and data scaling, Intern-S2-Preview explores **task scaling**: increasing the difficulty, diversity, and coverage of scientific tasks to further unlock model capabilities.
28
-
29
- By extending professional scientific tasks into a full-chain training pipeline from pre-training to reinforcement learning, Intern-S2-Preview achieves performance comparable to the trillion-scale Intern-S1-Pro on multiple core professional scientific tasks, while using only 35B parameters (continued pretrained from Qwen3.5). At the same time, it maintains strong general reasoning, multimodal understanding, and agent capabilities.
30
-
31
- ### Features
32
-
33
- - **Scientific task scaling with full-chain training.** Intern-S2-Preview scales hundreds of professional scientific tasks from pre-training to RL, enabling strong performance across multiple specialized domains at only 35B parameters. It further strengthens spatial modeling for small-molecule structures and introduces real-valued prediction modules, making it the first open-source model with both material crystal structure generation capability and strong general capabilities.
34
-
35
- - **Enhanced agent capabilities for scientific workflows.** Intern-S2-Preview significantly improves agentic abilities over the previous generation, achieving strong results on multiple scientific agent benchmarks.
36
-
37
- - **Efficient RL reasoning with MTP and CoT compression.** During RL, Intern-S2-Preview adopts shared-weight MTP with KL loss to reduce the mismatch between training and inference behavior, substantially improving MTP accept rate and token generation speed. It also introduces CoT compression techniques to shorten responses while preserving strong reasoning capability, achieving improvements in both performance and efficiency.
38
-
39
- - **Upgraded time-series Modeling** for better physical signal representation; supports long, heterogeneous time-series (10^0–10^6 points).
40
-
41
- <figure>
42
- <img src="./figs/efficiency.jpg" alt="efficient RL reasoning with MTP and CoT compression">
43
- <figcaption>Fig1: Reasoning Efficiency on Complex Math Benchmarks. Accuracy vs. Average Response Length. Intern-S2-Preview (red star) significantly outperforms trillion-scale Intern-S1-Pro (red circle), and achieving higher accuracy with better token efficiency among medium-size models.</figcaption>
44
- </figure>
45
-
46
- ### Performance
47
-
48
- We evaluate the Intern-S2-Preview on various benchmarks, including general datasets and scientific datasets. We report the performance comparison with the recent VLMs and LLMs below.
49
-
50
- ![performance](./figs/performance.png)
51
-
52
-
53
- > **Note**: <u>Underline</u> means the best performance among open-sourced models, **Bold** indicates the best performance among all models.
54
-
55
- We use the [OpenCompass](https://github.com/open-compass/OpenCompass/) and [VLMEvalKit](https://github.com/open-compass/vlmevalkit) to evaluate all models. For text reasoning benchmarks, Intern-S2-Preview is evaluated with a maximum inference length of 128K tokens, while for multimodal benchmarks, it is evaluated with a maximum inference length of 64K tokens.
56
-
57
-
58
- ## Quick Start
59
-
60
- ### Sampling Parameters
61
-
62
- We recommend using the following hyperparameters to ensure better results
63
-
64
- ```python
65
- top_p = 0.95
66
- top_k = 50
67
- min_p = 0.0
68
- temperature = 0.8
69
- ```
70
-
71
- ### Serving
72
-
73
- Intern-S2-Preview can be deployed using any of the following LLM inference frameworks:
74
-
75
- - LMDeploy
76
- - vLLM
77
- - SGLang
78
-
79
- Detailed deployment examples for these frameworks are available in the [Model Deployment Guide](./deployment_guide.md).
80
-
81
-
82
- ## Advanced Usage
83
-
84
- ### Tool Calling
85
-
86
- Tool Calling lets the model extend its capabilities by invoking external tools and APIs. The example below shows how to use it to fetch the latest weather forecast via an OpenAI-compatible API (based on lmdeploy api server).
87
-
88
- ```python
89
-
90
-
91
- from openai import OpenAI
92
- import json
93
-
94
-
95
- def get_current_temperature(location: str, unit: str = "celsius"):
96
- """Get current temperature at a location.
97
-
98
- Args:
99
- location: The location to get the temperature for, in the format "City, State, Country".
100
- unit: The unit to return the temperature in. Defaults to "celsius". (choices: ["celsius", "fahrenheit"])
101
-
102
- Returns:
103
- the temperature, the location, and the unit in a dict
104
- """
105
- return {
106
- "temperature": 26.1,
107
- "location": location,
108
- "unit": unit,
109
- }
110
-
111
-
112
- def get_temperature_date(location: str, date: str, unit: str = "celsius"):
113
- """Get temperature at a location and date.
114
-
115
- Args:
116
- location: The location to get the temperature for, in the format "City, State, Country".
117
- date: The date to get the temperature for, in the format "Year-Month-Day".
118
- unit: The unit to return the temperature in. Defaults to "celsius". (choices: ["celsius", "fahrenheit"])
119
-
120
- Returns:
121
- the temperature, the location, the date and the unit in a dict
122
- """
123
- return {
124
- "temperature": 25.9,
125
- "location": location,
126
- "date": date,
127
- "unit": unit,
128
- }
129
-
130
- def get_function_by_name(name):
131
- if name == "get_current_temperature":
132
- return get_current_temperature
133
- if name == "get_temperature_date":
134
- return get_temperature_date
135
-
136
- tools = [{
137
- 'type': 'function',
138
- 'function': {
139
- 'name': 'get_current_temperature',
140
- 'description': 'Get current temperature at a location.',
141
- 'parameters': {
142
- 'type': 'object',
143
- 'properties': {
144
- 'location': {
145
- 'type': 'string',
146
- 'description': 'The location to get the temperature for, in the format \'City, State, Country\'.'
147
- },
148
- 'unit': {
149
- 'type': 'string',
150
- 'enum': [
151
- 'celsius',
152
- 'fahrenheit'
153
- ],
154
- 'description': 'The unit to return the temperature in. Defaults to \'celsius\'.'
155
- }
156
- },
157
- 'required': [
158
- 'location'
159
- ]
160
- }
161
- }
162
- }, {
163
- 'type': 'function',
164
- 'function': {
165
- 'name': 'get_temperature_date',
166
- 'description': 'Get temperature at a location and date.',
167
- 'parameters': {
168
- 'type': 'object',
169
- 'properties': {
170
- 'location': {
171
- 'type': 'string',
172
- 'description': 'The location to get the temperature for, in the format \'City, State, Country\'.'
173
- },
174
- 'date': {
175
- 'type': 'string',
176
- 'description': 'The date to get the temperature for, in the format \'Year-Month-Day\'.'
177
- },
178
- 'unit': {
179
- 'type': 'string',
180
- 'enum': [
181
- 'celsius',
182
- 'fahrenheit'
183
- ],
184
- 'description': 'The unit to return the temperature in. Defaults to \'celsius\'.'
185
- }
186
- },
187
- 'required': [
188
- 'location',
189
- 'date'
190
- ]
191
- }
192
- }
193
- }]
194
-
195
-
196
-
197
- messages = [
198
- {'role': 'user', 'content': 'Today is 2024-11-14, What\'s the temperature in San Francisco now? How about tomorrow?'}
199
- ]
200
-
201
- openai_api_key = "EMPTY"
202
- openai_api_base = "http://0.0.0.0:23333/v1"
203
- client = OpenAI(
204
- api_key=openai_api_key,
205
- base_url=openai_api_base,
206
- )
207
- model_name = client.models.list().data[0].id
208
- response = client.chat.completions.create(
209
- model=model_name,
210
- messages=messages,
211
- max_tokens=32768,
212
- temperature=0.8,
213
- top_p=0.95,
214
- extra_body=dict(spaces_between_special_tokens=False),
215
- tools=tools)
216
- print(response.choices[0].message)
217
- messages.append(response.choices[0].message)
218
-
219
- for tool_call in response.choices[0].message.tool_calls:
220
- tool_call_args = json.loads(tool_call.function.arguments)
221
- tool_call_result = get_function_by_name(tool_call.function.name)(**tool_call_args)
222
- tool_call_result = json.dumps(tool_call_result, ensure_ascii=False)
223
- messages.append({
224
- 'role': 'tool',
225
- 'name': tool_call.function.name,
226
- 'content': tool_call_result,
227
- 'tool_call_id': tool_call.id
228
- })
229
-
230
- response = client.chat.completions.create(
231
- model=model_name,
232
- messages=messages,
233
- temperature=0.8,
234
- top_p=0.95,
235
- extra_body=dict(spaces_between_special_tokens=False),
236
- tools=tools)
237
- print(response.choices[0].message)
238
- ```
239
-
240
- ### Switching Between Thinking and Non-Thinking Modes
241
-
242
- Intern-S2-Preview enables thinking mode by default, enhancing the model's reasoning capabilities to generate higher-quality responses. This feature can be disabled by setting `enable_thinking=False` in `tokenizer.apply_chat_template`
243
-
244
- ```python
245
- text = tokenizer.apply_chat_template(
246
- messages,
247
- tokenize=False,
248
- add_generation_prompt=True,
249
- enable_thinking=False # think mode indicator
250
- )
251
- ```
252
-
253
- When serving Intern-S2-Preview models, you can dynamically control the thinking mode by adjusting the `enable_thinking` parameter in your requests.
254
-
255
- ```python
256
- from openai import OpenAI
257
- import json
258
-
259
- messages = [
260
- {
261
- 'role': 'user',
262
- 'content': 'who are you'
263
- }, {
264
- 'role': 'assistant',
265
- 'content': 'I am an AI'
266
- }, {
267
- 'role': 'user',
268
- 'content': 'AGI is?'
269
- }]
270
-
271
- openai_api_key = "EMPTY"
272
- openai_api_base = "http://0.0.0.0:23333/v1"
273
- client = OpenAI(
274
- api_key=openai_api_key,
275
- base_url=openai_api_base,
276
- )
277
- model_name = client.models.list().data[0].id
278
-
279
- response = client.chat.completions.create(
280
- model=model_name,
281
- messages=messages,
282
- temperature=0.8,
283
- top_p=0.95,
284
- max_tokens=2048,
285
- extra_body={
286
- "chat_template_kwargs": {"enable_thinking": False}
287
- }
288
- )
289
- print(json.dumps(response.model_dump(), indent=2, ensure_ascii=False))
290
- ```
291
-
292
- > Note: We do not recommend disabling thinking mode for agentic tasks.
293
-
294
-
295
- ### Time Series Demo
296
-
297
- Time series inference is currently only supported in LMDeploy. To get started, download and deploy Intern-S2-Preview with LMDeploy by following the [Model Deployment Guide](./deployment_guide.md).
298
- Below is an example of detecting earthquake events from a time series signal file. Additional data types and functionalities are also supported.
299
-
300
- **Please note**: this demo is slightly different from the one in [Intern-S1-Pro](https://huggingface.co/internlm/Intern-S1-Pro#time-series-demo). The main difference is that in the messages content, you need to provide time_series_url first, followed by the text prompt. Please adapt your implementation based on this demo.
301
-
302
- ```
303
- from openai import OpenAI
304
- from lmdeploy.vl.utils import encode_time_series_base64
305
-
306
- openai_api_key = "EMPTY"
307
- openai_api_base = "http://0.0.0.0:8000/v1"
308
- client = OpenAI(
309
- api_key=openai_api_key,
310
- base_url=openai_api_base,
311
- )
312
- model_name = client.models.list().data[0].id
313
-
314
-
315
- def send_base64(file_path: str, sampling_rate: int = 100):
316
- """base64-encoded time-series data."""
317
-
318
- # encode_time_series_base64 accepts local file paths and http urls,
319
- # encoding time-series data (.npy, .csv, .wav, .mp3, .flac, etc.) into base64 strings.
320
- base64_ts = encode_time_series_base64(file_path)
321
-
322
- messages = [
323
- {
324
- "role": "user",
325
- "content": [
326
- {
327
- "type": "time_series_url",
328
- "time_series_url": {
329
- "url": f"data:time_series/npy;base64,{base64_ts}",
330
- "sampling_rate": sampling_rate
331
- },
332
- },
333
- {
334
- "type": "text",
335
- "text": "Please determine whether an Earthquake event has occurred in the provided time-series data. If so, please specify the starting time point indices of the P-wave and S-wave in the event."
336
- },
337
- ],
338
- }
339
- ]
340
-
341
- return client.chat.completions.create(
342
- model=model_name,
343
- messages=messages,
344
- temperature=0,
345
- max_tokens=200,
346
- extra_body={
347
- "chat_template_kwargs": {"enable_thinking": False}
348
- }
349
- )
350
-
351
-
352
- def send_http_url(url: str, sampling_rate: int = 100):
353
- """http(s) url pointing to the time-series data."""
354
- messages = [
355
- {
356
- "role": "user",
357
- "content": [
358
- {
359
- "type": "time_series_url",
360
- "time_series_url": {
361
- "url": url,
362
- "sampling_rate": sampling_rate
363
- },
364
- },
365
- {
366
- "type": "text",
367
- "text": "Please determine whether an Earthquake event has occurred in the provided time-series data. If so, please specify the starting time point indices of the P-wave and S-wave in the event."
368
- },
369
- ],
370
- }
371
- ]
372
-
373
- return client.chat.completions.create(
374
- model=model_name,
375
- messages=messages,
376
- temperature=0,
377
- max_tokens=200,
378
- extra_body={
379
- "chat_template_kwargs": {"enable_thinking": False}
380
- }
381
- )
382
-
383
-
384
- def send_file_url(file_path: str, sampling_rate: int = 100):
385
- """file url pointing to the time-series data."""
386
- messages = [
387
- {
388
- "role": "user",
389
- "content": [
390
- {
391
- "type": "time_series_url",
392
- "time_series_url": {
393
- "url": f"file://{file_path}",
394
- "sampling_rate": sampling_rate
395
- },
396
- },
397
- {
398
- "type": "text",
399
- "text": "Please determine whether an Earthquake event has occurred in the provided time-series data. If so, please specify the starting time point indices of the P-wave and S-wave in the event."
400
- },
401
- ],
402
- }
403
- ]
404
-
405
- return client.chat.completions.create(
406
- model=model_name,
407
- messages=messages,
408
- temperature=0,
409
- max_tokens=200,
410
- extra_body={
411
- "chat_template_kwargs": {"enable_thinking": False}
412
- }
413
- )
414
-
415
- response = send_base64("./0092638_seism.npy")
416
- # response = send_http_url("https://huggingface.co/internlm/Intern-S1-Pro/raw/main/0092638_seism.npy")
417
- # response = send_file_url("./0092638_seism.npy")
418
-
419
- print(response.choices[0].message)
420
-
421
- ```
422
-
423
- ## Agent Integration
424
-
425
- Intern-S2-Preview can be plugged into agent frameworks in two ways: connecting to a **self-hosted deployment**, or calling the **official InternLM API**. Below we cover both, with examples for agent frameworks (OpenClaw, Hermes, etc.) and for Claude Code.
426
-
427
- ### 1. Self-hosted Deployment (LMDeploy as an example)
428
-
429
- First, serve the model with LMDeploy following the [Model Deployment Guide](./deployment_guide.md). The example below assumes the server is running at `http://0.0.0.0:23333`.
430
-
431
- #### Connecting Agent Frameworks
432
-
433
- Most agent frameworks (OpenClaw, Hermes, etc.) accept an OpenAI-compatible endpoint. Point them at the LMDeploy server base url `http://0.0.0.0:23333/v1`.
434
-
435
- You can check the connection with the following command:
436
-
437
- ```bash
438
- curl http://0.0.0.0:23333/v1/chat/completions \
439
- -H "Content-Type: application/json" \
440
- -H "Authorization: Bearer EMPTY" \
441
- -d '{
442
- "model": "internlm/Intern-S2-Preview",
443
- "messages": [
444
- {"role": "user", "content": "Hello"}
445
- ],
446
- "temperature": 0.8,
447
- "top_p": 0.95
448
- }'
449
- ```
450
-
451
- Or you can configure your agent framework with the environment variables
452
-
453
- ```bash
454
- export OPENAI_API_KEY=EMPTY
455
- export OPENAI_BASE_URL=http://0.0.0.0:23333/v1
456
- export OPENAI_MODEL=internlm/Intern-S2-Preview
457
- ```
458
-
459
- Remember to launch LMDeploy with `--tool-call-parser interns2-preview` so tool calls are parsed correctly.
460
-
461
- #### Connecting Claude Code
462
-
463
- LMDeploy exposes an Anthropic-compatible `/v1/messages` endpoint that Claude Code can talk to directly. Add the following to `~/.claude/settings.json`:
464
-
465
- ```json
466
- {
467
- "env": {
468
- "ANTHROPIC_BASE_URL": "http://127.0.0.1:23333",
469
- "ANTHROPIC_AUTH_TOKEN": "dummy",
470
- "ANTHROPIC_MODEL": "internlm/Intern-S2-Preview",
471
- "ANTHROPIC_CUSTOM_MODEL_OPTION": "internlm/Intern-S2-Preview"
472
- }
473
- }
474
- ```
475
-
476
- For a full walkthrough (curl verification, model routing, troubleshooting), see [LMDeploy × Claude Code](https://lmdeploy.readthedocs.io/en/latest/intergration/claude_code.html).
477
-
478
- ### 2. Official Intern API
479
-
480
- If you do not want to self-host, you can use the official Intern API. Register at [internlm.intern-ai.org.cn](https://internlm.intern-ai.org.cn/) and create an API token (`sk-xxxxxxxx`).
481
-
482
- #### Connecting Agent Frameworks
483
-
484
- The service is OpenAI-compatible, so any agent framework works. You can set the base url to `https://chat.intern-ai.org.cn/api/v1` and the model name to `intern-s2-preview` in the cli or config file.
485
-
486
- You can check the connection with the following command:
487
-
488
- ```bash
489
- curl https://chat.intern-ai.org.cn/api/v1/chat/completions \
490
- -H "Content-Type: application/json" \
491
- -H "Authorization: Bearer sk-xxxxxxxx" \
492
- -d '{
493
- "model": "intern-s2-preview",
494
- "messages": [
495
- {"role": "user", "content": "Hello"}
496
- ],
497
- "temperature": 0.8,
498
- "top_p": 0.95
499
- }'
500
- ```
501
-
502
- Refer to the [Intern API documentation](https://internlm.intern-ai.org.cn/api/document?lang=en) for the current endpoint, available model names, rate limits, and advanced parameters.
503
-
504
- #### Connecting Claude Code
505
-
506
- Claude Code can route to the official Intern API by pointing `ANTHROPIC_BASE_URL` at the Intern Anthropic-compatible gateway:
507
-
508
- ```json
509
- {
510
- "env": {
511
- "ANTHROPIC_BASE_URL": "https://chat.intern-ai.org.cn",
512
- "ANTHROPIC_AUTH_TOKEN": "your-api-token",
513
- "ANTHROPIC_MODEL": "intern-s2-preview",
514
- "ANTHROPIC_SMALL_FAST_MODEL": "intern-s2-preview"
515
- }
516
- }
517
- ```
518
-
519
- Then start claude code with the following command:
520
-
521
- ```bash
522
- claude --model intern-s2-preview
523
- ```
524
-
525
- For step-by-step setup, see [Intern API × Claude Code Integration](https://internlm.intern-ai.org.cn/docEn/docs/Claude-Code-Integration).
 
2
  library_name: transformers
3
  license: apache-2.0
4
  license_link: https://huggingface.co/internlm/Intern-S2-Preview/blob/main/LICENSE
5
+ base_model: internlm/Intern-S2-Preview
6
  pipeline_tag: image-text-to-text
7
  ---
8
 
9
+ # Intern-S2-Preview-OPD
10
 
11
+ **Intern-S2-Preview-OPD** is a post-trained version of
12
+ [Intern-S2-Preview](https://huggingface.co/internlm/Intern-S2-Preview), an efficient 35B scientific multimodal foundation model.
13
 
14
+ The model is post-trained using the method introduced in
15
+ [SimpleOPD: Simple Tokenizer-Agnostic On-Policy Distillation of Long-Context Reasoning](PAPER_LINK_PLACEHOLDER).
16
+ This post-training process substantially improves the model's reasoning performance, particularly on challenging proof and mathematical reasoning benchmarks.
17
 
18
+ ## Evaluation Results
19
 
20
+ | Model | ProofBench | AnswerBench | AIME25 | AMOBench |
21
+ |---|---:|---:|---:|---:|
22
+ | Intern-S2-Preview | 21.70 | 76.03 | 88.33 | 58.00 |
23
+ | **Intern-S2-Preview-OPD** | **44.50 (+22.80)** | **80.10 (+4.07)** | **95.00 (+6.67)** | **59.50 (+1.50)** |
24
 
25
+ The values in parentheses indicate absolute improvements over the base model.
 
 
26
 
27
+ ## Usage
28
 
29
+ Intern-S2-Preview-OPD uses the same model architecture and inference interface as Intern-S2-Preview. Please refer to the
30
+ [Intern-S2-Preview model card](https://huggingface.co/internlm/Intern-S2-Preview)
31
+ for deployment instructions and recommended inference settings.
32
 
33
+ ## License
34
 
35
+ This model is released under the
36
+ [Apache License 2.0](https://huggingface.co/internlm/Intern-S2-Preview/blob/main/LICENSE).