Hugging Face's logo

bigcode
/
starcoder

Text Generation
Transformers
PyTorch
Safetensors
gpt_bigcode
code
Eval Results (legacy)
text-generation-inference
Model card Files
xet
 Community
128

StarCoderBase->StarCoder, more details?

#52
by Bilibili - opened
Discussion
Bilibili

StarCoder is kind of like "LLaMa" for the Code LLM! Thanks for the a brilliant job!

Sicne there is a Paper about pretraining, can we have more details on the finetuning side?

For example, tha Paper says: "StarCoder is the fine-tuned version of StarCoderBase, trained on another 35B Python tokens (roughly 2 epochs).", I have such questions:

  1. Does 'another' means they are not from the Stack? Is it open?
  2. Finetuing code and settings, can we find it somewhere?

Glad to have this lovely community!
loubnabnl
 BigCode org

It's the same Python dataset that we pre-trained StarCoder on, we just did two more epochs . You can find the parameters for the training in this slurm here.
Bilibili

Thanks for response, love u~
Bilibili changed discussion status to closed

Sign up or log in to comment