Super-squash branch 'main' using huggingface_hub

.gitattributes

@@ -0,0 +1,35 @@
+*.7z filter=lfs diff=lfs merge=lfs -text
+*.arrow filter=lfs diff=lfs merge=lfs -text
+*.bin filter=lfs diff=lfs merge=lfs -text
+*.bz2 filter=lfs diff=lfs merge=lfs -text
+*.ckpt filter=lfs diff=lfs merge=lfs -text
+*.ftz filter=lfs diff=lfs merge=lfs -text
+*.gz filter=lfs diff=lfs merge=lfs -text
+*.h5 filter=lfs diff=lfs merge=lfs -text
+*.joblib filter=lfs diff=lfs merge=lfs -text
+*.lfs.* filter=lfs diff=lfs merge=lfs -text
+*.mlmodel filter=lfs diff=lfs merge=lfs -text
+*.model filter=lfs diff=lfs merge=lfs -text
+*.msgpack filter=lfs diff=lfs merge=lfs -text
+*.npy filter=lfs diff=lfs merge=lfs -text
+*.npz filter=lfs diff=lfs merge=lfs -text
+*.onnx filter=lfs diff=lfs merge=lfs -text
+*.ot filter=lfs diff=lfs merge=lfs -text
+*.parquet filter=lfs diff=lfs merge=lfs -text
+*.pb filter=lfs diff=lfs merge=lfs -text
+*.pickle filter=lfs diff=lfs merge=lfs -text
+*.pkl filter=lfs diff=lfs merge=lfs -text
+*.pt filter=lfs diff=lfs merge=lfs -text
+*.pth filter=lfs diff=lfs merge=lfs -text
+*.rar filter=lfs diff=lfs merge=lfs -text
+*.safetensors filter=lfs diff=lfs merge=lfs -text
+saved_model/**/* filter=lfs diff=lfs merge=lfs -text
+*.tar.* filter=lfs diff=lfs merge=lfs -text
+*.tar filter=lfs diff=lfs merge=lfs -text
+*.tflite filter=lfs diff=lfs merge=lfs -text
+*.tgz filter=lfs diff=lfs merge=lfs -text
+*.wasm filter=lfs diff=lfs merge=lfs -text
+*.xz filter=lfs diff=lfs merge=lfs -text
+*.zip filter=lfs diff=lfs merge=lfs -text
+*.zst filter=lfs diff=lfs merge=lfs -text
+*tfevents* filter=lfs diff=lfs merge=lfs -text

README.md

@@ -0,0 +1,101 @@
+---
+license: other
+base_model: microsoft/phi-1_5
+tags:
+- bees
+- honey
+- bzz
+metrics:
+- accuracy
+datasets:
+- BEE-spoke-data/bees-internal
+language:
+- en
+pipeline_tag: text-generation
+---
+
+# phi-1bee5 🐝
+
+> Where Code Meets Beekeeping: An Unbeelievable Synergy!
+
+<a href="https://colab.research.google.com/gist/pszemraj/7ea68b3b71ee4e6c0729d2318f3f4158/we-bee-testing.ipynb">
+  <img src="https://colab.research.google.com/assets/colab-badge.svg" alt="Open In Colab"/>
+</a>
+
+Have you ever found yourself in the depths of a debugging session and thought, "I wish I could be basking in the glory of a blooming beehive right now"? Or maybe you've been donning your beekeeping suit, puffing on your smoker, and longed for the sweet aroma of freshly written code?
+
+Well, brace yourselves, hive-minded humans and syntax-loving sapiens, for `phi-1bee5`, a groundbreaking transformer model that's here to disrupt your apiary and your IDE!
+
+
+## Details
+
+This model is a fine-tuned version of [microsoft/phi-1_5](https://huggingface.co/microsoft/phi-1_5) on the `BEE-spoke-data/bees-internal` dataset.
+
+It achieves the following results on the evaluation set:
+- Loss: 2.6982
+- Accuracy: 0.4597
+
+## Usage 
+
+load model:
+
+```python
+import torch
+from transformers import AutoTokenizer, AutoModelForCausalLM
+# !pip install -U -q transformers accelerate einops
+
+checkpoint = "BEE-spoke-data/phi-1bee5"
+tokenizer = AutoTokenizer.from_pretrained(checkpoint)
+model = AutoModelForCausalLM.from_pretrained(
+    checkpoint, 
+    device_map="auto",
+    torch_dtype=torch.float16,
+    trust_remote_code=True
+)
+```
+Run inference:
+
+```python
+prompt = "Today was an amazing day because"
+inputs = tokenizer(prompt, return_tensors="pt", return_attention_mask=False).to(
+    model.device
+)
+
+outputs = model.generate(
+    **inputs, do_sample=True, max_new_tokens=128, epsilon_cutoff=7e-4
+)
+result = tokenizer.batch_decode(outputs, skip_special_tokens=True)[0]
+print(result)
+# output will probably contain a story/info about bees
+```
+
+
+### Intended Uses:
+
+1. **Educational Edification**: Are you a coding novice with a budding interest in beekeeping? Or perhaps a seasoned developer whose curiosity has been piqued by the buzzing in your backyard? phi-1bee5 aims to serve as a fun, informative bridge between these two worlds.
+2. **Casual Queries**: This model can generate code examples and beekeeping tips. It's perfect for those late-night coding sessions when you feel like taking a virtual stroll through an apiary.
+3. **Academic & Research Insights**: Interested in interdisciplinary studies that explore the intersection of technology and ecology? phi-1bee5 might offer some amusing, if not entirely accurate, insights.
+
+### Limitations:
+
+1. **Not a beekeeping expert**: For the love of all things hexagonal, please do not use phi-1bee5 to make serious beekeeping decisions. While our model is well read in the beekeeping literature, it lacks the practical experience and nuanced understanding that professional beekeepers possess.
+2. **Licensing**: This model is derived from a base model under the Microsoft Research License. Any use must comply with the terms of that license.
+3. **Infallibility:** Like any machine learning model, phi-1bee5 can make mistakes. Always double check the code and bee facts before using it in production or in your hive.
+4. **Ethical Constraints**: This model may not be used for illegal or unethical activities, including but not limited to terrorism, harassment, or spreading disinformation.
+
+## Training procedure
+
+While the full dataset is not yet complete and therefore not yet released for "safety reasons", you can check out a preliminary sample at: [bees-v0](https://huggingface.co/datasets/BEE-spoke-data/bees-v0)
+
+### Training hyperparameters
+
+The following hyperparameters were used during training:
+- learning_rate: 0.0001
+- train_batch_size: 1
+- eval_batch_size: 2
+- gradient_accumulation_steps: 32
+- total_train_batch_size: 32
+- optimizer: Adam with betas=(0.9,0.995) and epsilon=1e-08
+- lr_scheduler_type: cosine
+- lr_scheduler_warmup_ratio: 0.03
+- num_epochs: 2.0

added_tokens.json

@@ -0,0 +1,40 @@
+{
+  "\t\t": 50294,
+  "\t\t\t": 50293,
+  "\t\t\t\t": 50292,
+  "\t\t\t\t\t": 50291,
+  "\t\t\t\t\t\t": 50290,
+  "\t\t\t\t\t\t\t": 50289,
+  "\t\t\t\t\t\t\t\t": 50288,
+  "\t\t\t\t\t\t\t\t\t": 50287,
+  "  ": 50286,
+  "   ": 50285,
+  "    ": 50284,
+  "     ": 50283,
+  "      ": 50282,
+  "       ": 50281,
+  "        ": 50280,
+  "         ": 50279,
+  "          ": 50278,
+  "           ": 50277,
+  "            ": 50276,
+  "             ": 50275,
+  "              ": 50274,
+  "               ": 50273,
+  "                ": 50272,
+  "                 ": 50271,
+  "                  ": 50270,
+  "                   ": 50269,
+  "                    ": 50268,
+  "                     ": 50267,
+  "                      ": 50266,
+  "                       ": 50265,
+  "                        ": 50264,
+  "                         ": 50263,
+  "                          ": 50262,
+  "                           ": 50261,
+  "                            ": 50260,
+  "                             ": 50259,
+  "                              ": 50258,
+  "                               ": 50257
+}

config.json

@@ -0,0 +1,35 @@
+{
+  "_name_or_path": "microsoft/phi-1_5",
+  "activation_function": "gelu_new",
+  "architecture": {
+    "block_cls": "parallel",
+    "mixer": {},
+    "mlp": {
+      "mlp_cls": "mlp"
+    }
+  },
+  "architectures": [
+    "MixFormerSequentialForCausalLM"
+  ],
+  "auto_map": {
+    "AutoConfig": "microsoft/phi-1_5--configuration_mixformer_sequential.MixFormerSequentialConfig",
+    "AutoModelForCausalLM": "microsoft/phi-1_5--modeling_mixformer_sequential.MixFormerSequentialForCausalLM"
+  },
+  "embd_layer": "default",
+  "embd_pdrop": 0.0,
+  "initializer_range": 0.02,
+  "layer_norm_epsilon": 1e-05,
+  "model_type": "mixformer-sequential",
+  "n_embd": 2048,
+  "n_head": 32,
+  "n_inner": null,
+  "n_layer": 24,
+  "n_positions": 2048,
+  "phyagi_version": "0.0.4.dev",
+  "resid_pdrop": 0.0,
+  "rotary_dim": 32,
+  "tie_word_embeddings": false,
+  "torch_dtype": "float32",
+  "transformers_version": "4.34.0.dev0",
+  "vocab_size": 51200
+}

generation_config.json

@@ -0,0 +1,4 @@
+{
+  "_from_model_config": true,
+  "transformers_version": "4.34.0.dev0"
+}

model.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:e81ba22664e35156c7c3725615ee62e2f4bcc407043c6465fbd71a69e205f462
+size 5673107168

special_tokens_map.json

@@ -0,0 +1,5 @@
+{
+  "bos_token": "<|endoftext|>",
+  "eos_token": "<|endoftext|>",
+  "unk_token": "<|endoftext|>"
+}

tokenizer_config.json

@@ -0,0 +1,9 @@
+{
+  "add_prefix_space": false,
+  "bos_token": "<|endoftext|>",
+  "clean_up_tokenization_spaces": true,
+  "eos_token": "<|endoftext|>",
+  "model_max_length": 2048,
+  "tokenizer_class": "CodeGenTokenizer",
+  "unk_token": "<|endoftext|>"
+}

training_args.bin

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:e4308d51607094a4894d0e96ac0c58394eca48bd93bee3b762a22c6f291bbc7d
+size 4664