santhosh/GRMR-2B-Instruct-openvino

This model was converted to OpenVINO from qingy2024/GRMR-2B-Instruct using optimum-intel via the export space.

First make sure you have optimum-intel installed:

pip install optimum[openvino]

To load your model you can do as follows:

from transformers import AutoTokenizer, AutoConfig, pipeline
from optimum.intel.openvino import OVModelForSeq2SeqLM
import time

mode_id = "santhosh/GRMR-2B-Instruct-openvino"
model = OVModelForSeq2SeqLM.from_pretrained(
    model_id,
    config=AutoConfig.from_pretrained(model_id),
    use_cache=True,
)
tokenizer = AutoTokenizer.from_pretrained(model_id)

# Create a pipeline
pipe = pipeline(
    "text2text-generation",
    model=model,
    tokenizer=tokenizer,
    truncation=True,
    max_length=256,
)

texts = [
    "Most of the course is about semantic or  content of language but there are also interesting topics to be learned from the servicefeatures except statistics in characters in documents.",
    "At this point, He introduces herself as his native English speaker and goes on to say that if you contine to work on social scnce",
    "He come after the event.",
    "When I grew up, I start to understand what he said is quite right",
    "Write this more formally: omg! i love that song im listening to right now",
    "Improve the grammaticality: As the number of people grows, the need of habitable environment is unquestionably essential.",
]
start_time = time.time()
for result in pipe(texts):
    print(result)
end_time = time.time()
duration = end_time - start_time
print(f"Correction completed in {duration:.2f} seconds.")