Instructions to use NovusResearch/Thestral-7B-0.1-TR-EN with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use NovusResearch/Thestral-7B-0.1-TR-EN with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("text-generation", model="NovusResearch/Thestral-7B-0.1-TR-EN")

# Load model directly
from transformers import AutoModel
model = AutoModel.from_pretrained("NovusResearch/Thestral-7B-0.1-TR-EN", dtype="auto")

Notebooks
Google Colab
Kaggle
Local Apps Settings

vLLM

How to use NovusResearch/Thestral-7B-0.1-TR-EN with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "NovusResearch/Thestral-7B-0.1-TR-EN"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "NovusResearch/Thestral-7B-0.1-TR-EN",
		"prompt": "Once upon a time,",
		"max_tokens": 512,
		"temperature": 0.5
	}'

Use Docker

docker model run hf.co/NovusResearch/Thestral-7B-0.1-TR-EN

SGLang

How to use NovusResearch/Thestral-7B-0.1-TR-EN with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "NovusResearch/Thestral-7B-0.1-TR-EN" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "NovusResearch/Thestral-7B-0.1-TR-EN",
		"prompt": "Once upon a time,",
		"max_tokens": 512,
		"temperature": 0.5
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "NovusResearch/Thestral-7B-0.1-TR-EN" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "NovusResearch/Thestral-7B-0.1-TR-EN",
		"prompt": "Once upon a time,",
		"max_tokens": 512,
		"temperature": 0.5
	}'

Docker Model Runner
How to use NovusResearch/Thestral-7B-0.1-TR-EN with Docker Model Runner:
```
docker model run hf.co/NovusResearch/Thestral-7B-0.1-TR-EN
```

Model Card: Thestral-7B-0.1-TR-EN

Version: Based on Mistral-7B-v0.1: Further pretrained on a large corpus of English-to-Turkish dataset.

The training dataset consists of approximately 490,000 pairs of translated texts. These pairs are predominantly sourced from movie subtitles, offering a diverse range of linguistic structures, idiomatic expressions, and cultural references. This rich dataset ensures the model is well-equipped to handle a variety of translation tasks within its domain.

Model Training

Details regarding the model's training procedure, architecture, and fine-tuning processes will be extensively covered in the upcoming paper.

Example Outputs (Finetuned Model)

Question: Acıbadem Mahallesi’nin sessiz ve huzurlu bir sokağından
bugünlere uzanan Erman Yaşar - Emre Özcan ikilisi Atıf Bey Sokak'ın
 yeni bölümünde; Galatasaray'ın Trabzonspor'u deplasmanda 5-1 mağlup
etmesini, Wilfried Zaha'nın 9 numara performansını, Mauro Icardi'nin
dönmesiyle birlikte oluşabilecek hücum rotasyonunu, Fenerbahçe'nin
Samsunspor ile berabere kalmasını, İsmail Kartal'ın yaptığı değişiklikleri,
Kenan Yıldız'ın Juventus'ta ilk 11'e yerleşmesini ve Euro 2024'teki
hücum gücümüzü konuştu.

Answer: From the quiet and peaceful street of Acibadem Mahallesi to today;
Erman Yaşar - Emre Özcan duo in the new section of Atif Bey Street;
Galatasaray beating Trabzonspor 5-1 in the matchday, Wilfried Zaha's
performance at number 9, Mauro Icardi's return, Fenerbahçe drawing with
Samsunspor, Ismail Kartal's changes, Kenan Yıldız's place in the starting
XI at Juventus and our power for Euro 2024 were discussed.

Example Outputs (Base Model)

Question: Acıbadem Mahallesi’nin sessiz ve huzurlu bir sokağından
bugünlere uzanan Erman Yaşar - Emre Özcan ikilisi Atıf Bey Sokak'ın
 yeni bölümünde; Galatasaray'ın Trabzonspor'u deplasmanda 5-1 mağlup
etmesini, Wilfried Zaha'nın 9 numara performansını, Mauro Icardi'nin
dönmesiyle birlikte oluşabilecek hücum rotasyonunu, Fenerbahçe'nin
Samsunspor ile berabere kalmasını, İsmail Kartal'ın yaptığı değişiklikleri,
Kenan Yıldız'ın Juventus'ta ilk 11'e yerleşmesini ve Euro 2024'teki
hücum gücümüzü konuştu.

Answer: The sentence is about a street in Acibadem neighborhood that
is quiet and peaceful. The sentence talks about the new section of Atif Bey
Street, which extends from today to the past. It also mentions the victory of
 Galatasaray over Trabzonspor in the match, the performance of Wilfried Zaha with
number 9, the return of Mauro Icardi, the draw between Fenerbahce and Samsunspor,
the changes made by Ismail Kartal, and the first eleven of Kenan Yildiz at Juventus.</s>

model = AutoModelForCausalLM.from_pretrained("NovusResearch/Thestral-7B-0.1-TR-EN",
  use_flash_attention_2=True,
  torch_dtype=torch.bfloat16,
  device_map="auto",
  trust_remote_code=True)

Downloads last month: -; Downloads are not tracked for this model. How to track