Upload model and tokenizer.

README.md

@@ -1,3 +1,58 @@
 ---
 license: cc
+datasets:
+- VMware/open-instruct-v1-oasst-dolly-hhrlhf
+language:
+- en
+library_name: transformers
+pipeline_tag: text-generation
 ---
+# blackmount8/open-llama-13B-open-instruct-ct2-int8_float16
+
+Int8_float16version of  [VMware/open-llama-13b-open-instruct](https://huggingface.co/VMware/open-llama-13b-open-instruct), quantized using CTranslate2.
+
+## VMware/open-llama-13B-open-instruct
+
+Instruction-tuned version of the fully trained Open LLama 13B model. The model is open for `<b>`COMMERCIAL USE `</b>`. `<br>`
+
+`<b>` NOTE `</b>` : The model was trained using the Alpaca prompt template
+`<b>` NOTE `</b>` : Fast tokenizer results in incorrect encoding, set the ``use_fast = False`` parameter, when instantiating the tokenizer
+
+## License
+
+- `<b>`Commercially Viable `</b>`
+- Instruction dataset, [VMware/open-instruct-v1-oasst-dolly-hhrlhf](https://huggingface.co/datasets/VMware/open-instruct-v1-oasst-dolly-hhrlhf) is under cc-by-sa-3.0
+- Language Model, ([openlm-research/open_llama_13b](https://huggingface.co/openlm-research/open_llama_13b)) is under apache-2.0
+
+## Nomenclature
+
+- Model : Open-llama
+- Model Size: 13B parameters
+- Dataset: Open-instruct-v1 (oasst, dolly, hhrlhf)
+
+## Use in CTranslate2
+
+```
+import ctranslate2
+from transformers import AutoTokenizer
+
+model_name = "blackmount8/open-llama-13b-open-instruct-ct2-int8_float16"
+
+tokenizer = AutoTokenizer.from_pretrained(model_name, use_fast=False, padding_side="left", truncation_side="left")
+model = ctranslate2.Generator(model_name, device="auto", compute_type="int8_float16")
+
+input_text = ["What is the meaning of stonehenge?", "Hello mate!"]
+
+input_ids = tokenizer(input_text, return_tensors="pt", padding=True, truncation=True).input_ids
+input_tokens = [tokenizer.convert_ids_to_tokens(ele) for ele in input_ids]
+
+outputs = model.generate_batch(input_tokens, max_length=128)
+
+output_tokens = [
+    ele.sequences_ids[0] for ele in outputs
+]
+
+output = tokenizer.batch_decode(output_tokens)
+
+print(output)
+```

config.json

@@ -0,0 +1,6 @@
+{
+  "bos_token": "<s>",
+  "eos_token": "</s>",
+  "layer_norm_epsilon": null,
+  "unk_token": "<unk>"
+}

model.bin

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:fdbda702db933751d9e9bfc3b7be138c71c7d9f48f3239c905931f385872e40e
+size 13025087966

special_tokens_map.json

@@ -0,0 +1,24 @@
+{
+  "bos_token": {
+    "content": "<s>",
+    "lstrip": false,
+    "normalized": true,
+    "rstrip": false,
+    "single_word": false
+  },
+  "eos_token": {
+    "content": "</s>",
+    "lstrip": false,
+    "normalized": true,
+    "rstrip": false,
+    "single_word": false
+  },
+  "pad_token": "<unk>",
+  "unk_token": {
+    "content": "<unk>",
+    "lstrip": false,
+    "normalized": true,
+    "rstrip": false,
+    "single_word": false
+  }
+}

tokenizer.model

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:ab1b681ec7fc02fed5edd3026687d7a692a918c4dd8e150ca2e3994a6229843b
+size 534194

tokenizer_config.json

@@ -0,0 +1,34 @@
+{
+  "add_bos_token": true,
+  "add_eos_token": false,
+  "bos_token": {
+    "__type": "AddedToken",
+    "content": "<s>",
+    "lstrip": false,
+    "normalized": true,
+    "rstrip": false,
+    "single_word": false
+  },
+  "clean_up_tokenization_spaces": false,
+  "eos_token": {
+    "__type": "AddedToken",
+    "content": "</s>",
+    "lstrip": false,
+    "normalized": true,
+    "rstrip": false,
+    "single_word": false
+  },
+  "model_max_length": 2048,
+  "pad_token": null,
+  "padding_side": "right",
+  "sp_model_kwargs": {},
+  "tokenizer_class": "LlamaTokenizer",
+  "unk_token": {
+    "__type": "AddedToken",
+    "content": "<unk>",
+    "lstrip": false,
+    "normalized": true,
+    "rstrip": false,
+    "single_word": false
+  }
+}