dfurman
/

Llama-2-70B-Instruct-v0.1

@@ -14,7 +14,7 @@ base_model: meta-llama/Llama-2-70b-hf
 This instruction model was built via parameter-efficient QLoRA finetuning of [llama-2-70b](https://huggingface.co/meta-llama/Llama-2-70b-hf) on the first 25k rows of [ehartford/dolphin](https://huggingface.co/datasets/ehartford/dolphin) (an open-source implementation of [Microsoft's Orca](https://www.microsoft.com/en-us/research/publication/orca-progressive-learning-from-complex-explanation-traces-of-gpt-4/)). Finetuning was executed on a single H100 (80 GB PCIe) for roughly 17 hours on the [Lambda Labs](https://cloud.lambdalabs.com/instances) platform.
-### Benchmark metrics
 | Metric                | Value |
 |-----------------------|-------|
@@ -26,7 +26,7 @@ This instruction model was built via parameter-efficient QLoRA finetuning of [ll
 We use state-of-the-art [Language Model Evaluation Harness](https://github.com/EleutherAI/lm-evaluation-harness) to run the benchmark tests above, using the same version as Hugging Face's [Open LLM Leaderboard](https://huggingface.co/spaces/HuggingFaceH4/open_llm_leaderboard).
-### Helpful links
 * Model license: Llama 2 Community License Agreement
 * Basic usage: [notebook](assets/basic_inference_llama_2_dolphin.ipynb)
@@ -40,7 +40,7 @@ We use state-of-the-art [Language Model Evaluation Harness](https://github.com/E
 The above loss curve was generated from the run's private wandb.ai log.
-### Example prompts and responses
 Example 1:
@@ -136,7 +136,7 @@ The llama-2-70b models have been modified from a standard transformer in the fol
 | sequence length | 4096 |
 | grouped-query attention | ✔️ |
-## PreTraining data
 For more details on the pretraining process, see [Llama-2-70b-hf](https://huggingface.co/meta-llama/Llama-2-70b-hf).
@@ -150,9 +150,9 @@ This model can produce factually incorrect output, and should not be relied on t
 This model was trained on various public datasets.
 While great efforts have been taken to clean the pretraining data, it is possible that this model could generate lewd, biased or otherwise offensive outputs.
-## How to use
-Basic usage: [notebook](assets/basic_inference_llama_2_dolphin.ipynb)
 ```python
 !pip install -q -U huggingface_hub peft transformers torch accelerate
@@ -221,8 +221,7 @@ with torch.autocast("cuda", dtype=torch.bfloat16):
 print(tokenizer.decode(output["sequences"][0], skip_special_tokens=True))
 ```
-### Runtime tests
 | runtime / 50 tokens (sec) | GPU             | attn | torch dtype | VRAM (GB) |
 |:-----------------------------:|:----------------------:|:---------------------:|:-------------:|:-----------------------:|
@@ -253,7 +252,7 @@ The license on this model does not constitute legal advice. We are not responsib
 ---
-### Framework versions
 - PEFT 0.5.0.dev0

 This instruction model was built via parameter-efficient QLoRA finetuning of [llama-2-70b](https://huggingface.co/meta-llama/Llama-2-70b-hf) on the first 25k rows of [ehartford/dolphin](https://huggingface.co/datasets/ehartford/dolphin) (an open-source implementation of [Microsoft's Orca](https://www.microsoft.com/en-us/research/publication/orca-progressive-learning-from-complex-explanation-traces-of-gpt-4/)). Finetuning was executed on a single H100 (80 GB PCIe) for roughly 17 hours on the [Lambda Labs](https://cloud.lambdalabs.com/instances) platform.
+## Benchmark metrics
 | Metric                | Value |
 |-----------------------|-------|
 We use state-of-the-art [Language Model Evaluation Harness](https://github.com/EleutherAI/lm-evaluation-harness) to run the benchmark tests above, using the same version as Hugging Face's [Open LLM Leaderboard](https://huggingface.co/spaces/HuggingFaceH4/open_llm_leaderboard).
+## Helpful links
 * Model license: Llama 2 Community License Agreement
 * Basic usage: [notebook](assets/basic_inference_llama_2_dolphin.ipynb)
 The above loss curve was generated from the run's private wandb.ai log.
+## Example prompts and responses
 Example 1:
 | sequence length | 4096 |
 | grouped-query attention | ✔️ |
+## Pre-training data
 For more details on the pretraining process, see [Llama-2-70b-hf](https://huggingface.co/meta-llama/Llama-2-70b-hf).
 This model was trained on various public datasets.
 While great efforts have been taken to clean the pretraining data, it is possible that this model could generate lewd, biased or otherwise offensive outputs.
+## Basic usage
+* [notebook](assets/basic_inference_llama_2_dolphin.ipynb)
 ```python
 !pip install -q -U huggingface_hub peft transformers torch accelerate
 print(tokenizer.decode(output["sequences"][0], skip_special_tokens=True))
 ```
+## Runtime tests
 | runtime / 50 tokens (sec) | GPU             | attn | torch dtype | VRAM (GB) |
 |:-----------------------------:|:----------------------:|:---------------------:|:-------------:|:-----------------------:|
 ---
+## Framework versions
 - PEFT 0.5.0.dev0