Zaynes commited on
Commit
dcdfd82
·
verified ·
1 Parent(s): e772d14

Upload README.md with huggingface_hub

Browse files
Files changed (1) hide show
  1. README.md +26 -0
README.md ADDED
@@ -0,0 +1,26 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ # M-Olmo-7B_3args_R1-sft-sft
2
+
3
+ This model was created as part of the **Olmo-7B_3args_R1-sft** experiment using the SkillFactory experiment management system.
4
+
5
+ ## Model Details
6
+
7
+ - **Training Method**: LLaMAFactory SFT (Supervised Fine-Tuning)
8
+ - **Stage Name**: sft
9
+ - **Experiment**: Olmo-7B_3args_R1-sft
10
+
11
+ ## Training Configuration
12
+
13
+ {"model_name_or_path": "allenai/Olmo-3-7B-Instruct-SFT", "trust_remote_code": true, "stage": "sft", "do_train": true, "finetuning_type": "full", "deepspeed": "/scratch/zrs2020/skill-factory/thirdparty/LLaMA-Factory/examples/deepspeed/ds_z2_config.json", "dataset": "SkillFactory__D_SFT_C_Olmo_7B_3args_R1_sft_sft_data__sft_train", "template": "olmo", "cutoff_len": 16384, "max_samples": 1000000, "overwrite_cache": true, "preprocessing_num_workers": 1, "dataloader_num_workers": 0, "disable_tqdm": false, "output_dir": "/scratch/zrs2020/skill_inject_outputs/sf_experiments/o7b3argsourssft/llamafactory/checkpoints", "logging_steps": 10, "save_steps": 100000, "plot_loss": true, "overwrite_output_dir": true, "per_device_train_batch_size": 1, "gradient_accumulation_steps": 1, "learning_rate": 1e-06, "num_train_epochs": 2, "lr_scheduler_type": "cosine", "warmup_ratio": 0.05, "weight_decay": 0.0001, "adam_beta1": 0.9, "adam_beta2": 0.95, "bf16": true, "ddp_timeout": 180000000, "gradient_checkpointing": true, "save_only_model": true, "save_strategy": "steps", "save_total_limit": 5, "run_name": "Olmo-7B_3args_R1-sft_sft"}
14
+
15
+ ## Experiment Tracking
16
+
17
+ 🔗 **View complete experiment details**: [Experiment Tracker Dataset](https://huggingface.co/datasets/SkillFactory/D-ExpTracker__Olmo-7B_3args_R1-sft__v1)
18
+
19
+ ## Usage
20
+
21
+ ```python
22
+ from transformers import AutoTokenizer, AutoModelForCausalLM
23
+
24
+ tokenizer = AutoTokenizer.from_pretrained("SkillFactory/M-Olmo-7B_3args_R1-sft-sft")
25
+ model = AutoModelForCausalLM.from_pretrained("SkillFactory/M-Olmo-7B_3args_R1-sft-sft")
26
+ ```