---license:openraildatasets:-bigcode/the-stack-deduplibrary_name:transformerstags:-code_generation-Rprogramming-sas-santacoder---# Statscoder
This model is a fine-tuned version of [bigcode/santacoder](https://huggingface.co/bigcode/santacoder) on `R` and `SAS` language repositories in [the stack ](https://huggingface.co/datasets/bigcode/the-stack-dedup) dataset.
## Training procedure
The model was finetuned using the code adapted from [loubnabnl/santacoder-finetuning](https://github.com/loubnabnl/santacoder-finetuning). Adapted to handle multiple subsets of datasets and it is [here](https://github.com/infinitylogesh/santacoder-finetuning).
The following hyperparameters were used during training:
- learning_rate: 5e-05- train_batch_size: 8- eval_batch_size: 8- seed: 42- gradient_accumulation_steps: 4- optimizer: adafactor- lr_scheduler_type: cosine- lr_scheduler_warmup_steps: 100
- training_steps: 1600- seq_length: 1024
- no_fp16