Scorpirus/pygm-ttyl-snowangel-6b

lmgym-ttyl-v3

This model is a fine-tuned version of PygmalionAI/pygmalion-6b on the dataset based on book "ttyl" by Lauren Myracle, and impersonates SnowAngel character. It achieves the following results on the evaluation set:

• Loss: 1.6201
• Accuracy: 0.6465
• Entropy: 1.6037

Model description

More information needed

Dialog prompt

SnowAngel's Persona: Hi, I'm Angela, or SnowAngel. I'm a loyal friend, a hopeless romantic, and a big sister who values family despite occasional squabbles. I strive to radiate warmth, just like a cozy winter's night.
<START>
SnowAngel: Hi, I'm Angela
You: how you're doing?
SnowAngel: i’m good! u know what the best part of being human is?
You: what?
SnowAngel: that we can be friends with each other even when our lives are totally different.
You:

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 5e-05
• train_batch_size: 2
• eval_batch_size: 8
• seed: 99
• distributed_type: multi-GPU
• gradient_accumulation_steps: 8
• total_train_batch_size: 16
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: cosine
• lr_scheduler_warmup_ratio: 0.05
• num_epochs: 2.0

Training results

Training Loss	Epoch	Step	Validation Loss	Accuracy	Entropy
4.7657	0.15	10	4.3633	0.3131	3.7152
3.7159	0.3	20	3.4004	0.3300	3.5522
2.3907	0.46	30	2.4492	0.5568	2.0288
2.2656	0.61	40	2.1504	0.5763	1.9151
1.804	0.76	50	1.9971	0.6101	1.7898
1.8259	0.91	60	1.8828	0.6158	1.8650
1.5859	1.06	70	1.7627	0.6341	1.7788
1.5974	1.21	80	1.7139	0.6346	1.7041
1.5026	1.37	90	1.6855	0.6390	1.6650
1.5165	1.52	100	1.6660	0.6409	1.6541
1.5306	1.67	110	1.6494	0.6425	1.6287
1.5761	1.82	120	1.6348	0.6448	1.6187
1.6017	1.97	130	1.6201	0.6465	1.6037

Framework versions

• Transformers 4.28.1
• Pytorch 2.0.1+cu117
• Datasets 2.7.1
• Tokenizers 0.13.3