| --- |
| library_name: onnxruntime |
| tags: |
| - trl |
| - onnx |
| - onnxruntime-genai |
| base_model: |
| - trl-internal-testing/tiny-GptOssForCausalLM |
| pipeline_tag: text-generation |
| --- |
| |
|
|
|
|
| # tiny-GptOssForCausalLM (ONNX) |
|
|
|
|
| This is an ONNX version of [trl-internal-testing/tiny-GptOssForCausalLM](https://huggingface.co/trl-internal-testing/tiny-GptOssForCausalLM). It was converted using the ONNX Runtime GenAI builder. |
|
|
|
|
| ## Conversion Details |
|
|
|
|
| - **Precision**: fp16 |
|
|
| - **Execution Provider**: cpu |
|
|
| - **Base Model**: [trl-internal-testing/tiny-GptOssForCausalLM](https://huggingface.co/trl-internal-testing/tiny-GptOssForCausalLM) |
|
|
|
|
| ## Usage with ONNX Runtime GenAI |
|
|
|
|
| ```python |
| |
| import onnxruntime_genai as og |
| |
| |
| |
| # Load the model |
| |
| model = og.Model("haydso/tiny-GptOssForCausalLM-ONNX") |
| |
| tokenizer = og.Tokenizer(model) |
| |
| |
| |
| # Generate text |
| |
| prompt = "Your prompt here" |
| |
| tokens = tokenizer.encode(prompt) |
| |
| params = og.GeneratorParams(model) |
| |
| params.set_search_options(max_length=200) |
| |
| params.input_ids = tokens |
| |
| |
| |
| generator = og.Generator(model, params) |
| |
| while not generator.is_done(): |
| |
| generator.compute_logits() |
| |
| generator.generate_next_token() |
| |
| |
| |
| output_tokens = generator.get_sequence(0) |
| |
| text = tokenizer.decode(output_tokens) |
| |
| print(text) |
| |
| ``` |
|
|
|
|
| --- |
|
|
|
|
| # Original Model Card |
|
|
|
|
| # Tiny GptOssForCausalLM |
|
|
| This is a minimal model built for unit tests in the [TRL](https://github.com/huggingface/trl) library. |
|
|
