NewstaR/Starlight-13B

Starlight (13B)

Model	Average ⬆️	ARC	HellaSwag	MMLU	TruthfulQA
NewstaR/Starlight-13B	58.63	59.3	82.15	55.67	37.39
NewstaR/Starlight-7B	54.3	53.07	78.57	46.8	38.75

The model follows the Alpaca template:

### Instruction: {prompt} ### Response:

Example:

### Instruction: Summarize the key details of the Starlight model in a few sentences.

### Response: Starlight is a 13B parameter transformer model trained on the AverageData and Above the Clouds datasets for conversational text generation. It has strong language modeling capabilities but lacks true language understanding and may generate incorrect or biased text, so outputs should be monitored and safeguards implemented. The model is intended for use in chatbots and content creation applications.

Instructions for Safe Use

• Avoid exposing Starlight to offensive, unethical, dangerous or illegal prompts
• Monitor outputs for signs of bias, toxicity or factual incorrectness
• Do not rely on Starlight for high-stakes or safety critical applications

Limitations

• May hallucinate or generate incorrect information
• Large model size leads to high compute requirements

Open LLM Leaderboard Evaluation Results

Detailed results can be found here

Metric	Value
Avg.	46.87
ARC (25-shot)	59.3
HellaSwag (10-shot)	82.15
MMLU (5-shot)	55.67
TruthfulQA (0-shot)	37.39
Winogrande (5-shot)	76.64
GSM8K (5-shot)	10.84
DROP (3-shot)	6.08