Update README.md

README.md

@@ -1,3 +1,70 @@
 ---
-license: apache-2.0
+license: cc-by-4.0
 ---
+# Model Card
+
+We provide a low-rank adapter for an instruction-tuned 12B-parameter GPT3-style language model.
+
+
+
+## Prompting
+
+For instructions that **do not** require extraneous inputs, the recommended prompt is:
+
+```
+Below is an instruction that describes a task. Write a response that appropriately completes the request.
+
+
+### Instruction:
+
+< -- instruction goes here --- >
+
+### Response:
+```
+
+For instructions that **do** require extraneous inputs, the recommended prompt is:
+
+```
+Below is an instruction that describes a task. Write a response that appropriately completes the request.
+
+
+### Instruction:
+
+< -- instruction goes here -- >
+
+### Input:
+
+< -- extraneous input goes here -- >
+
+### Response:
+```
+
+Since the model performs [causal language modeling](https://huggingface.co/docs/transformers/tasks/language_modeling), the model's response to the prompt is the text completing the sequence beginning with the prompt.
+
+
+## Instruction-Tuning
+
+This model was instruction-tuned on an internally-curated Alpaca-style dataset.
+
+- **Epochs**: 3
+- **Batch size**: 128
+- **Cutoff length**: 2048
+- **Learning rate**: 3.2e-5, cosine decay
+- **LoRA _r_**: 8
+- **LoRA _alpha_**: 16
+- **LoRA _dropout_**: 0.0
+- **LoRA target modules**: 'query_key_value', 'dense_h_to_4h', 'dense_4h_to_h', 'dense'
+- **License**: The instruction-tuning data is subject to the [Creative Commons 4.0](https://creativecommons.org/licenses/by/4.0/) license.
+
+## Base Model
+
+This model was instruction-tuned from a 12B variant from the Pythia family. 
+
+- **Repository:** [EleutherAI/pythia-12b](https://huggingface.co/EleutherAI/pythia-12b)
+- **Paper:** [arxiv:2304.01373](https://arxiv.org/abs/2304.01373)
+- **License**: The base model is subject to the Apache 2.0 license.
+- **Model type**: Transformer-based Language Model
+
+## Licensing Information
+
+We release this adapter under the [Creative Commons NonCommercial (CC BY-NC 4.0)](https://creativecommons.org/licenses/by-nc/4.0/) license.