README.md

---
license: apache-2.0
library_name: transformers
tags:
- juanako
- UNA
datasets:
- fblgit/tree-of-knowledge
- Open-Orca/SlimOrca-Dedup
- HuggingFaceH4/ultrafeedback_binarized
model-index:
- name: una-cybertron-7b-v1-fp16
  results:
  - task:
      type: text-generation
      name: Text Generation
    dataset:
      name: AI2 Reasoning Challenge (25-Shot)
      type: ai2_arc
      config: ARC-Challenge
      split: test
      args:
        num_few_shot: 25
    metrics:
    - type: acc_norm
      value: 68.43
      name: normalized accuracy
    source:
      url: https://huggingface.co/spaces/HuggingFaceH4/open_llm_leaderboard?query=fblgit/una-cybertron-7b-v1-fp16
      name: Open LLM Leaderboard
  - task:
      type: text-generation
      name: Text Generation
    dataset:
      name: HellaSwag (10-Shot)
      type: hellaswag
      split: validation
      args:
        num_few_shot: 10
    metrics:
    - type: acc_norm
      value: 85.42
      name: normalized accuracy
    source:
      url: https://huggingface.co/spaces/HuggingFaceH4/open_llm_leaderboard?query=fblgit/una-cybertron-7b-v1-fp16
      name: Open LLM Leaderboard
  - task:
      type: text-generation
      name: Text Generation
    dataset:
      name: MMLU (5-Shot)
      type: cais/mmlu
      config: all
      split: test
      args:
        num_few_shot: 5
    metrics:
    - type: acc
      value: 63.34
      name: accuracy
    source:
      url: https://huggingface.co/spaces/HuggingFaceH4/open_llm_leaderboard?query=fblgit/una-cybertron-7b-v1-fp16
      name: Open LLM Leaderboard
  - task:
      type: text-generation
      name: Text Generation
    dataset:
      name: TruthfulQA (0-shot)
      type: truthful_qa
      config: multiple_choice
      split: validation
      args:
        num_few_shot: 0
    metrics:
    - type: mc2
      value: 63.28
    source:
      url: https://huggingface.co/spaces/HuggingFaceH4/open_llm_leaderboard?query=fblgit/una-cybertron-7b-v1-fp16
      name: Open LLM Leaderboard
  - task:
      type: text-generation
      name: Text Generation
    dataset:
      name: Winogrande (5-shot)
      type: winogrande
      config: winogrande_xl
      split: validation
      args:
        num_few_shot: 5
    metrics:
    - type: acc
      value: 81.37
      name: accuracy
    source:
      url: https://huggingface.co/spaces/HuggingFaceH4/open_llm_leaderboard?query=fblgit/una-cybertron-7b-v1-fp16
      name: Open LLM Leaderboard
  - task:
      type: text-generation
      name: Text Generation
    dataset:
      name: GSM8k (5-shot)
      type: gsm8k
      config: main
      split: test
      args:
        num_few_shot: 5
    metrics:
    - type: acc
      value: 55.12
      name: accuracy
    source:
      url: https://huggingface.co/spaces/HuggingFaceH4/open_llm_leaderboard?query=fblgit/una-cybertron-7b-v1-fp16
      name: Open LLM Leaderboard
---
# Model Card for una-cybertron-7b-v1 (UNA: Uniform Neural Alignment)

We strike back, introducing **Cybertron 7B v1** a 7B MistralAI based model, best on it's series. Trained on SFT, DPO and UNA (Unified Neural Alignment) on multiple datasets.
He scores **64.60**+ on HF LeaderTests (without DROP for now).

Scoring **#1** at 2 December 2023:

| Model | Average | ARC (25-s) | HellaSwag (10-s) | MMLU (5-s) | TruthfulQA (MC) (0-s) | Winogrande (5-s) | GSM8K (5-s) |
| --- | --- | --- | --- | --- | --- | --- | --- |
| [mistralai/Mistral-7B-v0.1](https://huggingface.co/mistralai/Mistral-7B-v0.1) | 60.97 | 59.98  | 83.31  | 64.16  | 42.15 | 78.37 | 37.83 |
| [perlthoughts/Chupacabra-7B-v2](https://huggingface.co/perlthoughts/Chupacabra-7B-v2) | 63.54 | 66.47 | 85.17 | 64.49  | 57.6 | 79.16 | 28.35 |
| [fblgit/una-cybertron-7b-v1](https://huggingface.co/fblgit/una-cybertron-7b-v1) | **64.60** | **68.17** | 85.14 | 62.07  | **63.98** | **80.9** | 27.34 |

The model excels in mathematics, logic, reasoning, overall very smart.

## Model Details

Adiestrated with UNA: Uniform Neural Alignment technique (paper going out soon).


### Model Description

- **Developed by:** [juanako.ai](https://juanako.ai)
- **Author:** [Xavier M.](xavi@juanako.ai)
- **Model type:** MistralAI 7B
- **Funded by Cybertron's H100's**

### Prompt
The model is very good, works well on almost any prompt but ChatML format and Alpaca System gets the best
```
<|im_start|>system
- You are a helpful assistant chatbot trained by MosaicML.
- You answer questions.
- You are excited to be able to help the user, but will refuse to do anything that could be considered harmful to the user.
- You are more than just an information source, you are also able to write poetry, short stories, and make jokes.<|im_end|>
<|im_start|>user
Explain QKV<|im_end|>
<|im_start|>assistant
```
```
### Assistant: I am StableVicuna, a large language model created by CarperAI. I am here to chat!

### Human: Explain QKV
### Assistant:
```
```
[Round <|round|>]
问：Explain QKV
答：
```
```
[Round <|round|>]
Question：Explain QKV
Answer：
```
```
Question：Explain QKV
Answer：
```

## Evaluation
```
|    Tasks     |Version|Shots | Metric |Value |   |Stderr|
|--------------|-------|------|--------|-----:|---|-----:|
|arc_challenge |       | 25   |acc_norm|0.6817|±  |0.0136|
|truthfulqa_mc2|       | 0    |acc     |0.6398|±  |0.0151|
|hellaswag     |       | 10   |acc_norm|0.8492|±  |0.0036|
|winogrande    |       | 0    |acc     |0.809 |±  |0.011 |
|gsm8k         |       | 5    |acc     |0.2733|±  |0.0137|
|mmlu          |       | 5    |acc     |0.6207|±  |0.1230|
|              |average|      |acc     |0.6456|   |      |

|      Groups      |Version|Filter|n-shot|Metric|Value |   |Stderr|
|------------------|-------|------|-----:|------|-----:|---|-----:|
|mmlu              |N/A    |none  |     0|acc   |0.6207|_  |0.1230|
| - humanities     |N/A    |none  |     5|acc   |0.5675|_  |0.1125|
| - other          |N/A    |none  |     5|acc   |0.6933|_  |0.1108|
| - social_sciences|N/A    |none  |     5|acc   |0.7270|_  |0.0666|
| - stem           |N/A    |none  |     5|acc   |0.5249|_  |0.1311|
```

### Framework versions

- Transformers 4.35.0-UNA
- Pytorch 2.1.0
- Datasets 2.14.6
- Tokenizers 0.14.1

### Citations
If you find Cybertron, Juanako or any of our models useful, specially if you use it for your big brand.. cite please:
```
@misc{unacybertron7a,
  title={Cybertron: Uniform Neural Alignment}, 
  author={Xavier Murias},
  year={2023},
  publisher = {HuggingFace},
  journal = {HuggingFace repository},
  howpublished = {\url{https://huggingface.co/fblgit/una-cybertron-7b-v1}},
}
```
# [Open LLM Leaderboard Evaluation Results](https://huggingface.co/spaces/HuggingFaceH4/open_llm_leaderboard)
Detailed results can be found [here](https://huggingface.co/datasets/open-llm-leaderboard/details_fblgit__una-cybertron-7b-v1-fp16)

|             Metric              |Value|
|---------------------------------|----:|
|Avg.                             |69.49|
|AI2 Reasoning Challenge (25-Shot)|68.43|
|HellaSwag (10-Shot)              |85.42|
|MMLU (5-Shot)                    |63.34|
|TruthfulQA (0-shot)              |63.28|
|Winogrande (5-shot)              |81.37|
|GSM8k (5-shot)                   |55.12|