duoyuncloud commited on
Commit
884613f
·
verified ·
1 Parent(s): dd11053

Update model card: remove placeholder items

Browse files
Files changed (1) hide show
  1. README.md +19 -108
README.md CHANGED
@@ -1,124 +1,35 @@
1
  ---
2
- library_name: peft
3
- license: apache-2.0
4
  base_model: Qwen/Qwen2-0.5B
5
  tags:
6
- - axolotl
7
- - base_model:adapter:Qwen/Qwen2-0.5B
8
  - lora
9
- - transformers
10
  - cli
11
- - zsh
12
  - command-completion
13
- pipeline_tag: text-generation
14
  ---
15
 
16
- <!-- This model card has been generated automatically according to the information the Trainer had access to. You
17
- should probably proofread and complete it, then remove this comment. -->
18
-
19
- [<img src="https://raw.githubusercontent.com/axolotl-ai-cloud/axolotl/main/image/axolotl-badge-web.png" alt="Built with Axolotl" width="200" height="32"/>](https://github.com/axolotl-ai-cloud/axolotl)
20
- <details><summary>See axolotl config</summary>
21
-
22
- axolotl version: `0.13.1`
23
- ```yaml
24
- base_model: Qwen/Qwen2-0.5B
25
- model_type: AutoModelForCausalLM
26
- tokenizer_type: AutoTokenizer
27
- trust_remote_code: true
28
- datasets:
29
- - path: /Users/duoyun/Desktop/zsh-llm-cli-autocomplete-tool/src/training/data_splits_axolotl/train_axolotl.jsonl
30
- type: alpaca
31
- dataset_prepared_path: /Users/duoyun/Desktop/zsh-llm-cli-autocomplete-tool/last_run_prepared
32
- val_set_size: 0.0
33
- output_dir: /Users/duoyun/Desktop/zsh-llm-cli-autocomplete-tool/zsh-lora-output
34
- adapter: lora
35
- lora_r: 8
36
- lora_alpha: 16
37
- lora_dropout: 0.05
38
- lora_target_modules:
39
- - q_proj
40
- - k_proj
41
- - v_proj
42
- - o_proj
43
- - gate_proj
44
- - up_proj
45
- - down_proj
46
- lora_modules_to_save:
47
- - embed_tokens
48
- - lm_head
49
- sequence_len: 512
50
- sample_packing: false
51
- pad_to_sequence_len: true
52
- micro_batch_size: 1
53
- gradient_accumulation_steps: 8
54
- num_epochs: 2
55
- optimizer: adamw_torch
56
- lr_scheduler: cosine
57
- learning_rate: 0.0002
58
- warmup_steps: 50
59
- load_in_8bit: false
60
- load_in_4bit: true
61
- bf16: false
62
- fp16: false
63
- tf32: false
64
- gradient_checkpointing: true
65
- group_by_length: false
66
- logging_steps: 10
67
- save_steps: 15
68
- save_safetensors: true
69
- save_total_limit: 2
70
- early_stopping_patience: null
71
- early_stopping_threshold: null
72
- wandb_mode: disabled
73
- fp8: false
74
- val_dataset:
75
- path: /Users/duoyun/Desktop/zsh-llm-cli-autocomplete-tool/src/training/data_splits_axolotl/val_axolotl.jsonl
76
- type: alpaca
77
-
78
- ```
79
-
80
- </details><br>
81
-
82
- # Users/duoyun/Desktop/zsh-llm-cli-autocomplete-tool/zsh-lora-output
83
 
84
- This model is a fine-tuned version of [Qwen/Qwen2-0.5B](https://huggingface.co/Qwen/Qwen2-0.5B) on the /Users/duoyun/Desktop/zsh-llm-cli-autocomplete-tool/src/training/data_splits_axolotl/train_axolotl.jsonl dataset.
85
 
86
- ## Model description
87
 
88
- More information needed
 
 
 
89
 
90
- ## Intended uses & limitations
91
-
92
- More information needed
93
-
94
- ## Training and evaluation data
95
-
96
- More information needed
97
-
98
- ## Training procedure
99
-
100
- ### Training hyperparameters
101
-
102
- The following hyperparameters were used during training:
103
- - learning_rate: 0.0002
104
- - train_batch_size: 1
105
- - eval_batch_size: 1
106
- - seed: 42
107
- - gradient_accumulation_steps: 8
108
- - total_train_batch_size: 8
109
- - optimizer: Use OptimizerNames.ADAMW_TORCH with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
110
- - lr_scheduler_type: cosine
111
- - lr_scheduler_warmup_steps: 50
112
- - training_steps: 49
113
-
114
- ### Training results
115
 
 
 
 
116
 
 
 
 
 
117
 
118
- ### Framework versions
119
 
120
- - PEFT 0.18.1
121
- - Transformers 4.57.6
122
- - Pytorch 2.10.0
123
- - Datasets 4.5.0
124
- - Tokenizers 0.22.1
 
1
  ---
2
+ license: mit
 
3
  base_model: Qwen/Qwen2-0.5B
4
  tags:
 
 
5
  - lora
 
6
  - cli
 
7
  - command-completion
8
+ - zsh
9
  ---
10
 
11
+ # CLI Command Completion LoRA Adapter
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
12
 
13
+ LoRA adapter fine-tuned on Qwen/Qwen2-0.5B for CLI command completion (Zsh).
14
 
15
+ ## Model Details
16
 
17
+ - **Base Model**: Qwen/Qwen2-0.5B
18
+ - **Adapter Type**: LoRA
19
+ - **Task**: CLI command completion
20
+ - **Training Data**: Zsh command completion pairs
21
 
22
+ ## Usage
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
23
 
24
+ ```python
25
+ from transformers import AutoModelForCausalLM, AutoTokenizer
26
+ from peft import PeftModel
27
 
28
+ base_model = AutoModelForCausalLM.from_pretrained("Qwen/Qwen2-0.5B")
29
+ tokenizer = AutoTokenizer.from_pretrained("Qwen/Qwen2-0.5B")
30
+ model = PeftModel.from_pretrained(base_model, "duoyuncloud/zsh-cli-lora")
31
+ ```
32
 
33
+ ## License
34
 
35
+ MIT License