cfilt/iitb-english-hindi
Viewer • Updated • 1.66M • 898 • 67
How to use monsterapi/gemma-2-2b-hindi-translator with PEFT:
from peft import PeftModel
from transformers import AutoModelForCausalLM
base_model = AutoModelForCausalLM.from_pretrained("google/gemma-2-2b-it")
model = PeftModel.from_pretrained(base_model, "monsterapi/gemma-2-2b-hindi-translator")Model Used: google/gemma-2-2b-it
Dataset: cfilt/iitb-english-hindi
The IIT Bombay English-Hindi corpus contains a parallel corpus for English-Hindi as well as a monolingual Hindi corpus collected from various sources. This corpus has been utilized in the Workshop on Asian Language Translation Shared Task since 2016 for Hindi-to-English and English-to-Hindi language pairs and as a pivot language pair for Hindi-to-Japanese and Japanese-to-Hindi translations.
With the utilization of MonsterAPI's LLM finetuner, this finetuning:
$1.91 for the entire process.<bos><start_of_turn>user
{PROMPT}<end_of_turn>
<start_of_turn>model
{OUTPUT} <end_of_turn>
<eos>
license: apache-2.0