File size: 3,189 Bytes
9631227 fece41d 9631227 3b879c5 9631227 04d426d fece41d 9631227 04d426d 9631227 e4c7969 9631227 e4c7969 9631227 04d426d c63dfb8 04d426d 3b879c5 04d426d 9631227 04d426d 9631227 e4c7969 9631227 e4c7969 9631227 04d426d eda28d2 9631227 e4c7969 7c89bff 9631227 e4c7969 9631227 887b6d7 | 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 | ---
library_name: transformers
datasets:
- culturalheritagenus/rumi-correction-v1.1-data-v3
language:
- en
- ms
metrics:
- bleu
base_model:
- aisingapore/Gemma-SEA-LION-v3-9B-IT
---
# Model Card for Model ID
<!-- Provide a quick summary of what the model is/does. -->
## Model Details
### Model Description
<!-- Provide a longer summary of what this model is. -->
This model is trained with QLoRA with parameters `r = lora_alpha = 4`.
- **Developed by:** hyhyhyhyyhyh
- **Model type:** Gemma 2 9B
- **Language(s) (NLP):** Malay, English
- **License:** [More Information Needed]
- **Finetuned from model** aisingapore/Gemma-SEA-LION-v3-9B-IT
### Model Sources [optional]
<!-- Provide the basic links for the model. -->
- **Repository:** [More Information Needed]
- **Paper [optional]:** [More Information Needed]
- **Demo [optional]:** [More Information Needed]
## How to Get Started with the Model
Use the code below to get started with the model:
```
from transformers import AutoModelForCausalLM, AutoTokenizer, TextStreamer
trained_model = AutoModelForCausalLM.from_pretrained(
"culturalheritagenus/rumi-correction-v1.1",
device_map="auto",
torch_dtype=torch.bfloat16
)
trained_tokenizer = AutoTokenizer.from_pretrained("culturalheritagenus/rumi-correction-v1.1")
```
To perform inference:
```
messages = [
{"role": "user", "content": "You are a Malay language spelling corrector. I will give you some text written in messy Rumi (shortened or mistyped). Rewrite it in correct Malay Rumi spelling.\naurng ank. yngdim dimn anm aurngdan"},
]
inputs = tokenizer.apply_chat_template(
messages,
tokenize = True,
add_generation_prompt = True, # Must add for generation
return_tensors = "pt",
).to("cuda")
text_streamer = TextStreamer(tokenizer)
_ = trained_model.generate(input_ids = inputs, streamer = text_streamer, max_new_tokens = 128, use_cache = True)
```
## Training Details
### Training Data
The model was trained on [culturalheritagenus/rumi-correction-v1.1-data-v3](https://huggingface.co/datasets/culturalheritagenus/rumi-correction-v1.1-data-v3)
### Training Procedure
To replicate this model, please refer to the provided script and below. Ensure that the versions of all languages and libraries are the same.
## Environmental Impact
<!-- Total emissions (in grams of CO2eq) and additional considerations, such as electricity usage, go here. Edit the suggested text below accordingly -->
Carbon emissions can be estimated using the [Machine Learning Impact calculator](https://mlco2.github.io/impact#compute) presented in [Lacoste et al. (2019)](https://arxiv.org/abs/1910.09700).
- **Hardware Type:** 1x GH200 (96 GB)
- **Hours used:** ~12
- **Cloud Provider:** Lambda
- **Compute Region:** US-East (Lambda Labs)
## Technical Specifications
### Software
- Python version: 3.10.12
- CUDA version: 12.8
- Torch version: 2.7.1+cu128
## Citation [optional]
<!-- If there is a paper or blog post introducing the model, the APA and Bibtex information for that should go in this section. -->
**BibTeX:**
[More Information Needed]
**APA:**
[More Information Needed]
## Model Card Authors [optional]
hyhyhyhyyhyh
|