|
|
--- |
|
|
language: |
|
|
- el |
|
|
license: apache-2.0 |
|
|
pipeline_tag: text-generation |
|
|
tags: |
|
|
- finetuned |
|
|
inference: true |
|
|
--- |
|
|
|
|
|
# kkOracle v0.1 |
|
|
|
|
|
kkOracle v0.1 is a LORA fine-tuned version of [Meltemi 7B Instruct v1.5](https://huggingface.co/ilsp/Meltemi-7B-Instruct-v1.5) using a synthetic dataset based on text from the daily greek newspaper "Rizospastis" covering the timespan from 2008 to 2024. |
|
|
|
|
|
|
|
|
# Running the model with mlx on a Mac |
|
|
|
|
|
``` |
|
|
pip install mlx-lm |
|
|
``` |
|
|
|
|
|
``` |
|
|
python -m mlx_lm.generate --model model_kkOracle --prompt "Καλημέρα!" --temp 0.3 |
|
|
``` |
|
|
|
|
|
|
|
|
# Running the model on other systems |
|
|
|
|
|
```python |
|
|
from transformers import AutoModelForCausalLM, AutoTokenizer |
|
|
|
|
|
device = "cuda" # or "cpu" |
|
|
|
|
|
model = AutoModelForCausalLM.from_pretrained("model_kkOracle") |
|
|
tokenizer = AutoTokenizer.from_pretrained("model_kkOracle") |
|
|
|
|
|
model.to(device) |
|
|
|
|
|
messages = [ |
|
|
{"role": "user", "content": "Καλημέρα!"}, |
|
|
] |
|
|
|
|
|
prompt = tokenizer.apply_chat_template(messages, add_generation_prompt=True, tokenize=False) |
|
|
input_prompt = tokenizer(prompt, return_tensors='pt').to(device) |
|
|
outputs = model.generate(input_prompt['input_ids'], max_new_tokens=256, do_sample=True, temperature=0.3, use_cache=True, pad_token_id=tokenizer.eos_token_id, attention_mask=input_prompt["attention_mask"]) |
|
|
|
|
|
print(tokenizer.batch_decode(outputs)[0]) |
|
|
``` |
|
|
|
|
|
|
|
|
# Ethical Considerations |
|
|
|
|
|
This model has been aligned with human preferences, but might generate misleading, harmful, and toxic content. |
|
|
|
|
|
|
|
|
|
