Update README.md

README.md

@@ -30,10 +30,6 @@ img {
 
 Palmyra was primarily pretrained with English text, there is still a trace amount of non-English data present within the training corpus that was accessed through CommonCrawl. A causal language modeling (CLM) objective was utilized during the process of the model's pretraining. Similar to GPT-3, Palmyra is a member of the same family of models that only contain a decoder. As a result, it was pretrained utilizing the objective of self-supervised causal language modeling. Palmyra uses the prompts and general experimental setup from GPT-3 in order to conduct its evaluation in accordance with GPT-3. Read the official paper if you want more information about this.
 
-The model consists of 28 layers with a model dimension of 4096, and a feedforward dimension of 16384. The model
-dimension is split into 16 heads, each with a dimension of 256. Rotary Position Embedding (RoPE) is applied to 64
-dimensions of each head. The model is trained with a tokenization vocabulary of 50257, using the same set of BPEs as
-GPT-2/GPT-3.
 
 ## Training data
 
@@ -48,10 +44,24 @@ Palmyra-small learns an inner representation of the English language that can be
 This model can be easily loaded using the `AutoModelForCausalLM` functionality:
 
 ```python
-from transformers import AutoTokenizer, AutoModelForCausalLM
 
-tokenizer = AutoTokenizer.from_pretrained("EleutherAI/palmyra-small")
-model = AutoModelForCausalLM.from_pretrained("EleutherAI/palmyra-small")
+from transformers import AutoModelForCausalLM, AutoTokenizer
+import torch
+
+model = AutoModelForCausalLM.from_pretrained("Writer/palmyra-small", torch_dtype=torch.float16).cuda()
+
+# the fast tokenizer currently does not work correctly
+tokenizer = AutoTokenizer.from_pretrained("Writer/palmyra-small", use_fast=False)
+
+prompt = "What is the color of a carrot?\nA:"
+
+input_ids = tokenizer(prompt, return_tensors="pt").input_ids.cuda()
+
+generated_ids = model.generate(input_ids)
+
+tokenizer.batch_decode(generated_ids, skip_special_tokens=True)
+
+
 ```
 
 ### Limitations and Biases