Update README.md
Browse files
README.md
CHANGED
|
@@ -12,16 +12,16 @@ language:
|
|
| 12 |
- en
|
| 13 |
---
|
| 14 |
|
| 15 |
-
#
|
| 16 |
|
| 17 |
-
|
| 18 |
|
| 19 |
## Model Details
|
| 20 |
|
| 21 |
### Model Description
|
| 22 |
|
| 23 |
|
| 24 |
-
|
| 25 |
|
| 26 |
A key technical modification in this model is the decoupling of the embedding and LM head layers, allowing the output layer to be trained independently, which can improve the model's ability to generate accurate Armenian text.
|
| 27 |
|
|
@@ -42,7 +42,7 @@ Then, run this example:
|
|
| 42 |
from transformers import AutoModelForCausalLM, AutoTokenizer, GenerationConfig
|
| 43 |
import torch
|
| 44 |
|
| 45 |
-
model_path = "Gen2B/
|
| 46 |
tokenizer = AutoTokenizer.from_pretrained(model_path)
|
| 47 |
model = AutoModelForCausalLM.from_pretrained(
|
| 48 |
model_path,
|
|
@@ -77,7 +77,7 @@ for s in generation_output.sequences:
|
|
| 77 |
|
| 78 |
### Direct Use
|
| 79 |
|
| 80 |
-
|
| 81 |
- Armenian text generation
|
| 82 |
- Question answering in Armenian
|
| 83 |
- Text completion for Armenian content
|
|
@@ -129,7 +129,7 @@ The model demonstrates strong performance on Armenian language tasks, showing si
|
|
| 129 |
|
| 130 |
#### Summary
|
| 131 |
|
| 132 |
-
|
| 133 |
|
| 134 |
---
|
| 135 |
|
|
|
|
| 12 |
- en
|
| 13 |
---
|
| 14 |
|
| 15 |
+
# HyGPT-10b
|
| 16 |
|
| 17 |
+
HyGPT-10b is the first Armenian large language model that has been pretrained on corpus of Armenian text data. This model is designed to understand and generate Armenian text, making it a pioneering high-quality language model specifically created for the Armenian language.
|
| 18 |
|
| 19 |
## Model Details
|
| 20 |
|
| 21 |
### Model Description
|
| 22 |
|
| 23 |
|
| 24 |
+
HyGPT-10b is a decoder-only language model based on Google's Gemma-2-9b architecture that has been further pretrained on 10B tokens of Armenian text.
|
| 25 |
|
| 26 |
A key technical modification in this model is the decoupling of the embedding and LM head layers, allowing the output layer to be trained independently, which can improve the model's ability to generate accurate Armenian text.
|
| 27 |
|
|
|
|
| 42 |
from transformers import AutoModelForCausalLM, AutoTokenizer, GenerationConfig
|
| 43 |
import torch
|
| 44 |
|
| 45 |
+
model_path = "Gen2B/HyGPT-10b"
|
| 46 |
tokenizer = AutoTokenizer.from_pretrained(model_path)
|
| 47 |
model = AutoModelForCausalLM.from_pretrained(
|
| 48 |
model_path,
|
|
|
|
| 77 |
|
| 78 |
### Direct Use
|
| 79 |
|
| 80 |
+
HyGPT-10b can be used directly for:
|
| 81 |
- Armenian text generation
|
| 82 |
- Question answering in Armenian
|
| 83 |
- Text completion for Armenian content
|
|
|
|
| 129 |
|
| 130 |
#### Summary
|
| 131 |
|
| 132 |
+
HyGPT-10b shows promising capabilities for Armenian language understanding and generation, making it a valuable resource for Armenian NLP applications. Additionally, the model serves as an excellent foundation model for further fine-tuning on specific data and domains, allowing developers to adapt it to specialized Armenian language tasks and industry-specific applications.
|
| 133 |
|
| 134 |
---
|
| 135 |
|