Instructions to use tonmoytalukder/Bangla-Key2Text with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use tonmoytalukder/Bangla-Key2Text with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("text-generation", model="tonmoytalukder/Bangla-Key2Text")

# Load model directly
from transformers import AutoTokenizer, AutoModelForSeq2SeqLM

tokenizer = AutoTokenizer.from_pretrained("tonmoytalukder/Bangla-Key2Text")
model = AutoModelForSeq2SeqLM.from_pretrained("tonmoytalukder/Bangla-Key2Text", device_map="auto")

Notebooks
Google Colab
Kaggle
Local Apps Settings

vLLM

How to use tonmoytalukder/Bangla-Key2Text with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "tonmoytalukder/Bangla-Key2Text"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "tonmoytalukder/Bangla-Key2Text",
		"prompt": "Once upon a time,",
		"max_tokens": 512,
		"temperature": 0.5
	}'

Use Docker

docker model run hf.co/tonmoytalukder/Bangla-Key2Text

SGLang

How to use tonmoytalukder/Bangla-Key2Text with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "tonmoytalukder/Bangla-Key2Text" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "tonmoytalukder/Bangla-Key2Text",
		"prompt": "Once upon a time,",
		"max_tokens": 512,
		"temperature": 0.5
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "tonmoytalukder/Bangla-Key2Text" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "tonmoytalukder/Bangla-Key2Text",
		"prompt": "Once upon a time,",
		"max_tokens": 512,
		"temperature": 0.5
	}'

Docker Model Runner
How to use tonmoytalukder/Bangla-Key2Text with Docker Model Runner:
```
docker model run hf.co/tonmoytalukder/Bangla-Key2Text
```

Bangla-Key2Text / README.md

tonmoytalukder

Update README.md

c788953 about 3 years ago

preview code

Raw

History Blame Contribute Delete

2.36 kB

metadata

license: mit
language:
  - bn
tags:
  - Bangla-Key2Text
pipeline_tag: text2text-generation
datasets:
  - tonmoytalukder/Bangla-Key2Text-2Million
widget:
  - text: ঠিকমতো এই অভাবের তাই শিক্ষার্থীরা কারণে
  - text: কেমন ডাটাসেট সময় ভাই বানাতে

This code imports the necessary libraries and loads tonmoytalukder/Bangla-Key2Text pre-trained model for sequence-to-sequence learning using the Hugging Face Transformers library. The model is designed to convert Bangla text from a key to a sentence.

Using this model in transformers

!pip install sentencepiece
!pip install transformers
!pip install git+https://github.com/csebuetnlp/normalizer
!pip install torch

import torch
from transformers import AutoTokenizer, AutoModelForSeq2SeqLM
from normalizer import normalize

model_dir = 'tonmoytalukder/Bangla-Key2Text'
tokenizer = AutoTokenizer.from_pretrained(model_dir)
model = AutoModelForSeq2SeqLM.from_pretrained(model_dir)
device = torch.device("cuda" if torch.cuda.is_available() else "cpu")
model.to(device)

def predict(key): # Function to generate text from given keywords
    input_ids = tokenizer.encode(key, return_tensors='pt',add_special_tokens=True).to(device)

    with torch.no_grad():
      outputs = model.generate(
          input_ids=input_ids,
          max_length =512,
          num_beams =2,
          early_stopping =True,
          num_return_sequences = 1,
          top_k= 50,
          top_p= 0.95,
          repetition_penalty= 2.5,
          length_penalty= 1.0)

    preds = [tokenizer.decode(g,skip_special_tokens=True,clean_up_tokenization_spaces=True) for g in outputs]

    generated_text = preds[0]
    return generated_text


keywords = "কেমন ডাটাসেট সময় ভাই বানাতে" # Put as কেমন ডাটাসেট সময় ভাই বানাতে in the Hosted inference API. Don't put any punctuation mark.
predict(normalize(keywords)) # "ভাই, ডাটাসেট বানাতে কেমন সময় লাগে?"

The code defines a function called predict() that takes a string of keywords as input and returns a generated sentence based on those keywords. The function uses the pre-trained model to generate the sentence.