File size: 2,057 Bytes
f49e2e5 9ca9af4 f49e2e5 9ca9af4 f49e2e5 9ca9af4 f49e2e5 9ca9af4 f49e2e5 6330e4a dc77eb0 f49e2e5 4ae0ef5 f49e2e5 9ca9af4 f49e2e5 9ca9af4 f49e2e5 9ca9af4 f49e2e5 9ca9af4 dca4e8a f49e2e5 dca4e8a f49e2e5 9ca9af4 0270c7e 9ca9af4 f49e2e5 9ca9af4 f49e2e5 9ca9af4 f49e2e5 9ca9af4 f49e2e5 9ca9af4 f49e2e5 9ca9af4 f49e2e5 9ca9af4 |
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 |
---
library_name: transformers
license: mit
language:
- ja
- en
---
# Stockmark-2-100B-Instruct-beta
## Model description
**Stockmark-2-100B-Instruct-beta** is a 100-billion-parameter large language model built from scratch, with a particular focus on Japanese. It was pre-trained on approximately 1.5 trillion tokens of data, consisting of 60% English, 30% Japanese, and 10% code. Following pretraining, the model underwent post-training with synthetic data in Japanese to enhance its ability to follow instructions. This synthetic data was generated using Qwen2.5-32B-Instruct.
As a beta release, Stockmark-2-100b-Instruct-beta is still undergoing improvements and evaluations. Feedback and insights from users will help refine future versions.
See [our blog](https://stockmark-tech.hatenablog.com/entry/2025/03/06/114203) for the detail.
This project is supported by [GENIAC](https://www.meti.go.jp/policy/mono_info_service/geniac/index.html).
## How to use
```python
import torch
from transformers import AutoModelForCausalLM, AutoTokenizer
tokenizer = AutoTokenizer.from_pretrained("stockmark/Stockmark-2-100B-Instruct-beta")
model = AutoModelForCausalLM.from_pretrained(
"stockmark/Stockmark-2-100B-Instruct-beta", device_map="auto", torch_dtype=torch.bfloat16
)
instruction = "自然言語処理とは？"
input_ids = tokenizer.apply_chat_template(
[{"role": "user", "content": instruction}], add_generation_prompt=True, return_tensors="pt"
).to(model.device)
with torch.inference_mode():
tokens = model.generate(
input_ids,
max_new_tokens = 512,
do_sample = True,
temperature = 0.7,
top_p = 0.95,
repetition_penalty = 1.05
)
output = tokenizer.decode(tokens[0], skip_special_tokens=True)
print(output)
```
## License
[MIT](https://opensource.org/licenses/MIT)
## Developed by
[Stockmark Inc.](https://stockmark.co.jp/)
## Author
Takahiro Omi |