Update README.md
Browse files
README.md
CHANGED
|
@@ -40,14 +40,7 @@ The model is released under the Apache 2.0 license.
|
|
| 40 |
- [Recommendations](#recommendations)
|
| 41 |
- [Training Details](#training-details)
|
| 42 |
- [Training Data](#training-data)
|
| 43 |
-
- [Training Procedure](#training-procedure)
|
| 44 |
-
- [Preprocessing](#preprocessing)
|
| 45 |
-
- [Speeds, Sizes, Times](#speeds-sizes-times)
|
| 46 |
- [Evaluation](#evaluation)
|
| 47 |
-
- [Testing Data, Factors & Metrics](#testing-data-factors--metrics)
|
| 48 |
-
- [Testing Data](#testing-data)
|
| 49 |
-
- [Factors](#factors)
|
| 50 |
-
- [Metrics](#metrics)
|
| 51 |
- [Results](#results)
|
| 52 |
- [Environmental Impact](#environmental-impact)
|
| 53 |
- [Technical Specifications](#technical-specifications)
|
|
@@ -55,9 +48,9 @@ The model is released under the Apache 2.0 license.
|
|
| 55 |
- [Compute Infrastructure](#compute-infrastructure)
|
| 56 |
- [Hardware](#hardware)
|
| 57 |
- [Software](#software)
|
|
|
|
| 58 |
- [Citation](#citation)
|
| 59 |
- [Contact](#contact)
|
| 60 |
-
- [How to Get Started with the Model](#how-to-get-started-with-the-model)
|
| 61 |
|
| 62 |
# 🐯 Model Details
|
| 63 |
|
|
@@ -111,45 +104,18 @@ If considering LINCE-ZERO for production use, it is crucial to thoroughly evalua
|
|
| 111 |
|
| 112 |
## Training Data
|
| 113 |
|
| 114 |
-
LINCE-ZERO is based on [Falcon-7B](https://huggingface.co/tiiuae/falcon-7b) and has been fine-tuned using an augmented combination of the [Alpaca](https://huggingface.co/datasets/tatsu-lab/alpaca) and [Dolly](https://huggingface.co/datasets/databricks/databricks-dolly-15k) datasets, both translated into Spanish.
|
| 115 |
|
| 116 |
Alpaca is a 24.2 MB dataset of 52,002 instructions and demonstrations in English. It was generated by OpenAI's `text-davinci-003` engine using the data generation pipeline from the [Self-Instruct framework](https://github.com/yizhongw/self-instruct) with some modifications. For further details, refer to [Alpaca's Data Card](https://huggingface.co/datasets/tatsu-lab/alpaca).
|
| 117 |
|
| 118 |
Dolly is a 13.1 MB dataset of 15,011 instruction-following records in American English. It was generated by thousands of Databricks employees, who were requested to provide reference texts copied from Wikipedia for specific categories. To learn more, consult [Dolly’s Data Card](https://huggingface.co/datasets/databricks/databricks-dolly-15k).
|
| 119 |
|
| 120 |
-
After combining both translations, the dataset was augmented to a total of 80k examples.
|
| 121 |
-
|
| 122 |
-
## Training Procedure
|
| 123 |
-
|
| 124 |
-
For detailed information about the model architecture and compute infrastructure, please refer to the Technical Specifications section.
|
| 125 |
-
|
| 126 |
-
### Preprocessing
|
| 127 |
-
|
| 128 |
-
To prepare the training data, both the Alpaca and Dolly datasets, originally in English, were translated into Spanish using …
|
| 129 |
-
|
| 130 |
-
The data was tokenized using LINCE-ZERO’s tokenizer, which is based on the Falcon-[7B](https://huggingface.co/tiiuae/falcon-7b)/[40B](https://huggingface.co/tiiuae/falcon-40b) tokenizer.
|
| 131 |
-
|
| 132 |
-
### Training Hyperparameters
|
| 133 |
|
| 134 |
-
More information needed
|
| 135 |
-
|
| 136 |
-
### Speeds, Sizes, Times
|
| 137 |
-
|
| 138 |
-
More information needed (throughput, start/end time, checkpoint size if relevant, etc.)
|
| 139 |
|
| 140 |
# ✅ Evaluation
|
| 141 |
|
| 142 |
-
|
| 143 |
-
|
| 144 |
-
### Testing Data
|
| 145 |
-
|
| 146 |
-
The model has been tested on a X% of the augmented combination of Alpaca (24.2 MB) and Dolly (13.1 MB) translated into Spanish.
|
| 147 |
-
|
| 148 |
-
### Metrics
|
| 149 |
-
|
| 150 |
-
Since LINCE-ZERO is an instruction model, the metrics used to evaluate it are:
|
| 151 |
-
|
| 152 |
-
- X: <value>
|
| 153 |
|
| 154 |
### Results
|
| 155 |
|
|
|
|
| 40 |
- [Recommendations](#recommendations)
|
| 41 |
- [Training Details](#training-details)
|
| 42 |
- [Training Data](#training-data)
|
|
|
|
|
|
|
|
|
|
| 43 |
- [Evaluation](#evaluation)
|
|
|
|
|
|
|
|
|
|
|
|
|
| 44 |
- [Results](#results)
|
| 45 |
- [Environmental Impact](#environmental-impact)
|
| 46 |
- [Technical Specifications](#technical-specifications)
|
|
|
|
| 48 |
- [Compute Infrastructure](#compute-infrastructure)
|
| 49 |
- [Hardware](#hardware)
|
| 50 |
- [Software](#software)
|
| 51 |
+
- [How to Get Started with the Model](#how-to-get-started-with-the-model)
|
| 52 |
- [Citation](#citation)
|
| 53 |
- [Contact](#contact)
|
|
|
|
| 54 |
|
| 55 |
# 🐯 Model Details
|
| 56 |
|
|
|
|
| 104 |
|
| 105 |
## Training Data
|
| 106 |
|
| 107 |
+
LINCE-ZERO is based on [Falcon-7B](https://huggingface.co/tiiuae/falcon-7b) and has been fine-tuned using an augmented combination of the [Alpaca](https://huggingface.co/datasets/tatsu-lab/alpaca) and [Dolly](https://huggingface.co/datasets/databricks/databricks-dolly-15k) datasets, both translated with the best quality into Spanish.
|
| 108 |
|
| 109 |
Alpaca is a 24.2 MB dataset of 52,002 instructions and demonstrations in English. It was generated by OpenAI's `text-davinci-003` engine using the data generation pipeline from the [Self-Instruct framework](https://github.com/yizhongw/self-instruct) with some modifications. For further details, refer to [Alpaca's Data Card](https://huggingface.co/datasets/tatsu-lab/alpaca).
|
| 110 |
|
| 111 |
Dolly is a 13.1 MB dataset of 15,011 instruction-following records in American English. It was generated by thousands of Databricks employees, who were requested to provide reference texts copied from Wikipedia for specific categories. To learn more, consult [Dolly’s Data Card](https://huggingface.co/datasets/databricks/databricks-dolly-15k).
|
| 112 |
|
| 113 |
+
After combining both translations, the dataset was augmented to reach a total of 80k examples.
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 114 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 115 |
|
| 116 |
# ✅ Evaluation
|
| 117 |
|
| 118 |
+
This is WIP.
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 119 |
|
| 120 |
### Results
|
| 121 |
|