Text Generation
Transformers
Safetensors
Russian
English
qwen2
MANTA
autocomplete
word-completion
text-generation-inference
Instructions to use average23/MANTA-20M with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- Transformers
How to use average23/MANTA-20M with Transformers:
# Use a pipeline as a high-level helper from transformers import pipeline pipe = pipeline("text-generation", model="average23/MANTA-20M")# pip install -U transformers accelerate # Load model directly from transformers import AutoTokenizer, AutoModelForCausalLM tokenizer = AutoTokenizer.from_pretrained("average23/MANTA-20M") model = AutoModelForCausalLM.from_pretrained("average23/MANTA-20M", device_map="auto") - Notebooks
- Google Colab
- Kaggle
- Local Apps Settings
- vLLM
How to use average23/MANTA-20M with vLLM:
Install from pip and serve model
# Install vLLM from pip: pip install vllm # Start the vLLM server: vllm serve "average23/MANTA-20M" # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:8000/v1/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "average23/MANTA-20M", "prompt": "Once upon a time,", "max_tokens": 512, "temperature": 0.5 }'Use Docker
docker model run hf.co/average23/MANTA-20M
- SGLang
How to use average23/MANTA-20M with SGLang:
Install from pip and serve model
# Install SGLang from pip: pip install sglang # Start the SGLang server: python3 -m sglang.launch_server \ --model-path "average23/MANTA-20M" \ --host 0.0.0.0 \ --port 30000 # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:30000/v1/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "average23/MANTA-20M", "prompt": "Once upon a time,", "max_tokens": 512, "temperature": 0.5 }'Use Docker images
docker run --gpus all \ --shm-size 32g \ -p 30000:30000 \ -v ~/.cache/huggingface:/root/.cache/huggingface \ --env "HF_TOKEN=<secret>" \ --ipc=host \ lmsysorg/sglang:latest \ python3 -m sglang.launch_server \ --model-path "average23/MANTA-20M" \ --host 0.0.0.0 \ --port 30000 # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:30000/v1/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "average23/MANTA-20M", "prompt": "Once upon a time,", "max_tokens": 512, "temperature": 0.5 }' - Docker Model Runner
How to use average23/MANTA-20M with Docker Model Runner:
docker model run hf.co/average23/MANTA-20M
|
Download README.md from average23/MANTA-20M: direct link, hf CLI and curl.
- Browser
- Download file 3.64 kB
-
https://huggingface.co/average23/MANTA-20M/resolve/main/README.md
- Command line
-
hf download hf://average23/MANTA-20M/README.md
-
curl -L -o README.md https://huggingface.co/average23/MANTA-20M/resolve/main/README.md
3.64 kB
| license: apache-2.0 | |
| language: | |
| - ru | |
| - en | |
| library_name: transformers | |
| pipeline_tag: text-generation | |
| tags: | |
| - MANTA | |
| - qwen2 | |
| - autocomplete | |
| - word-completion | |
| - safetensors | |
| model-index: | |
| - name: MANTA 20M | |
| results: [] | |
| # MANTA 20M | |
| **v0.1.0** · Compact Russian conversational language model for next-word prediction and word completion. | |
| ## Architecture | |
| | Parameter | Value | | |
| |---|---:| | |
| | Parameters | 20,084,064 | | |
| | Decoder | Qwen2-compatible, autoregressive | | |
| | Transformer layers | 8 | | |
| | Hidden size / intermediate size | 352 / 1,024 | | |
| | Attention heads / KV heads | 8 / 4 | | |
| | Vocabulary | 24,000, ByteLevel BPE | | |
| | Architecture context | 1,024 tokens | | |
| | Keyboard evaluation context | 128 tokens | | |
| | Weights | FP32 Safetensors, tied embeddings | | |
| ## Training | |
| The initial checkpoint was pretrained on Russian educational web text, general web text and Wikipedia. | |
| MANTA then received conversational adaptation: 779,333,270 target tokens in one corpus pass, | |
| followed by 41,074,231 target tokens over 2,239,218 windows. The final stage used | |
| 1,227,460 new Telegram messages and 1,009,803 replay windows from the earlier corpus. | |
| Sources cover Telegram chats, conversational dialogue, forum text, slang and profanity, | |
| RU/EN code-switching, technical vocabulary, lexical normalization and a small news component. | |
| Final stage: AdamW, learning rate 3e-5, weight decay 0.01, batch size 16, | |
| 512-token packs, 5,212 updates, seed 42, FP32 parameters with CUDA BF16 autocast. | |
| ## Evaluation | |
| | Model | Parameters | Next-word Top-1 | Next-word Top-3 | Completion Top-1 | Completion Top-3 | | |
| |---|---:|---:|---:|---:|---:| | |
| | MANTA 20M | 20,084,064 | 8.09% | 15.74% | 31.61% | 52.59% | | |
| | [Keyboard LM 57M](https://huggingface.co/ushastoek/russian-keyboard-learning) | 57,321,984 | 5.96% | 12.34% | 26.68% | 38.60% | | |
| Same Telegram cohort: 128 authors, 235 unique target positions, | |
| 940 next-word and 1,544 completion queries. Completion prefixes use one character | |
| and half the target word, deduplicating equal prefix lengths. Queries repeat across | |
| four history snapshots (0/10/100/500); MANTA uses the language model alone. | |
| Scoring uses NFC-normalized exact word match. Whole-word beam search: | |
| beam 10, 64 steps, 48-character word limit, 128 context tokens, 30-second ceiling, | |
| ten candidates before Top-3 scoring, FP32, TF32 disabled. | |
| ## Usage | |
| ```bash | |
| pip install https://huggingface.co/average23/MANTA-20M/resolve/v0.1.0/manta_keyboard-0.1.0-py3-none-any.whl | |
| ``` | |
| ```python | |
| from manta import MantaKeyboard | |
| keyboard = MantaKeyboard.from_pretrained("average23/MANTA-20M") | |
| next_words = keyboard.next_word("ну это полный ") | |
| completions = keyboard.complete("надо перезапустить ", prefix="док") | |
| print([word["text"] for word in next_words]) | |
| print([word["text"] for word in completions]) | |
| ``` | |
| Output: | |
| ```text | |
| ['пиздец', 'бред', 'набор'] | |
| ['докер', 'докерфайл', 'доку'] | |
| ``` | |
| Both methods return up to three dictionaries with `text` (a complete word) | |
| and `log_probability` (explored word-boundary path mass). Completion preserves | |
| the typed prefix; completed context ends in whitespace. The default search has | |
| fixed beam and step budgets. Pass `max_seconds=30` to use the benchmark deadline. | |
| Download once, then load the saved directory for offline inference. | |
| Standard Transformers loading: | |
| ```python | |
| from transformers import AutoModelForCausalLM, AutoTokenizer | |
| model = AutoModelForCausalLM.from_pretrained("average23/MANTA-20M", revision="v0.1.0", trust_remote_code=False) | |
| tokenizer = AutoTokenizer.from_pretrained("average23/MANTA-20M", revision="v0.1.0", trust_remote_code=False) | |
| ``` | |