| --- |
| language: |
| - en |
| license: apache-2.0 |
| pipeline_tag: text-generation |
| tags: |
| - llama |
| - causal-lm |
| - experimental |
| library_name: transformers |
| --- |
| # PingVortexLM1-20M-Base |
| A small experimental language model based on LLaMA architecture trained on custom high-quality English dataset with around 200M tokens. |
| This model is just an experiment, it is not designed for coherent text generation or logical reasoning and may produce repetitive or nonsensical outputs. |
|
|
| Built by [PingVortex Labs](https://github.com/PingVortexLabs). |
|
|
| --- |
| ## Model Details |
| + **Parameters:** 20M |
| + **Context length:** 8192 tokens |
| + **Language:** English only |
| + **License:** Apache 2.0 |
|
|
| --- |
| ## Usage |
| ```python |
| from transformers import LlamaForCausalLM, PreTrainedTokenizerFast |
| |
| model = LlamaForCausalLM.from_pretrained("pvlabs/PingVortexLM1-20M-Base") |
| tokenizer = PreTrainedTokenizerFast.from_pretrained("pvlabs/PingVortexLM1-20M-Base") |
| |
| # don't expect a coherent response |
| prompt = "The capital of France is" |
| inputs = tokenizer(prompt, return_tensors="pt") |
| outputs = model.generate(**inputs, max_new_tokens=50, repetition_penalty=1.3) |
| print(tokenizer.decode(outputs[0])) |
| ``` |
|
|
| --- |
| *Made by [PingVortex](https://pingvortex.com).* |
