---
license: mit
tags:
- generated_from_trainer
model-index:
- name: Fuxi
  results: []
---

# Fuxi

负屃是一个Python代码生成模型，训练过程主要参考了[Training a causal language model from scratch](https://huggingface.co/learn/nlp-course/en/chapter7/6)

在中国神话中，负屃是龙的第八个儿子。似龙形，平生好文。负屃十分爱好闪耀着艺术光彩的碑文，它甘愿化做图案文龙去衬托这些传世的文学珍品。

Fuxi is a Python code generation model, and the training process mainly refers to [Training a causal language model from scratch](https://huggingface.co/learn/nlp-course/en/chapter7/6).

In Chinese mythology, Fuxi is the eighth son of the Dragon. He resembles a dragon,  and has a strong affinity for literature. Fuxi is particularly fond of inscriptions shining with artistic brilliance, and is willing to transform into an ornamental dragon to set off these enduring literary treasures.

## Model description

More information needed

## Intended uses & limitations

More information needed

## Training and evaluation data

More information needed

## Training procedure

### Training hyperparameters

The following hyperparameters were used during training:
- learning_rate: 0.0005
- train_batch_size: 64
- eval_batch_size: 64
- seed: 42
- gradient_accumulation_steps: 8
- total_train_batch_size: 512
- optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
- lr_scheduler_type: cosine
- lr_scheduler_warmup_steps: 1000
- num_epochs: 1

### Training results

| Training Loss | Epoch | Step  | Validation Loss |
|:-------------:|:-----:|:-----:|:---------------:|
| 2.3403        | 0.15  | 5000  | 1.5401          |
| 1.5052        | 0.31  | 10000 | 1.3514          |
| 1.3657        | 0.46  | 15000 | 1.2464          |
| 1.2715        | 0.61  | 20000 | 1.1665          |
| 1.1977        | 0.77  | 25000 | 1.1059          |
| 1.1498        | 0.92  | 30000 | 1.0777          |


### Framework versions

- Transformers 4.30.2
- Pytorch 2.0.1+cu117
- Datasets 2.13.1
- Tokenizers 0.13.3