Hugging Face's logo

voidful
/
llmcodec-librispeech-abl-sa

PEFT
Safetensors
qwen3
Generated from Trainer
Model card Files
xet
Community
voidful commited on
Commit
4a72c13
·
verified ·
1 Parent(s): e9bb9c6

Model save

Browse files
Files changed (7) hide show
  1. README.md +7 -5
  2. final/adapter_config.json +4 -4
  3. final/adapter_model.safetensors +2 -2
  4. final/added_tokens.json +0 -0
  5. final/tokenizer.json +2 -2
  6. final/tokenizer_config.json +0 -0
  7. final/training_args.bin +1 -1
README.md CHANGED
@@ -15,7 +15,7 @@ should probably proofread and complete it, then remove this comment. -->
15
 
16
  This model is a fine-tuned version of [voidful/llm-codec-abl-sa](https://huggingface.co/voidful/llm-codec-abl-sa) on an unknown dataset.
17
  It achieves the following results on the evaluation set:
18
- - Loss: 12.0998
19
 
20
  ## Model description
21
 
@@ -43,13 +43,15 @@ The following hyperparameters were used during training:
43
  - optimizer: Use OptimizerNames.ADAMW_TORCH with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
44
  - lr_scheduler_type: cosine
45
  - lr_scheduler_warmup_ratio: 0.1
46
- - num_epochs: 1
47
 
48
  ### Training results
49
 
50
- | Training Loss | Epoch | Step | Validation Loss |
51
- |:-------------:|:-----:|:-----:|:---------------:|
52
- | 12.1109 | 1.0 | 35156 | 12.0998 |
 
 
53
 
54
 
55
  ### Framework versions
 
15
 
16
  This model is a fine-tuned version of [voidful/llm-codec-abl-sa](https://huggingface.co/voidful/llm-codec-abl-sa) on an unknown dataset.
17
  It achieves the following results on the evaluation set:
18
+ - Loss: 8.4431
19
 
20
  ## Model description
21
 
 
43
  - optimizer: Use OptimizerNames.ADAMW_TORCH with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
44
  - lr_scheduler_type: cosine
45
  - lr_scheduler_warmup_ratio: 0.1
46
+ - num_epochs: 3
47
 
48
  ### Training results
49
 
50
+ | Training Loss | Epoch | Step | Validation Loss |
51
+ |:-------------:|:-----:|:------:|:---------------:|
52
+ | 8.5968 | 1.0 | 35156 | 8.7275 |
53
+ | 8.3714 | 2.0 | 70312 | 8.5016 |
54
+ | 8.3344 | 3.0 | 105468 | 8.4431 |
55
 
56
 
57
  ### Framework versions
final/adapter_config.json CHANGED
@@ -24,13 +24,13 @@
24
  "rank_pattern": {},
25
  "revision": null,
26
  "target_modules": [
 
27
  "k_proj",
28
  "v_proj",
29
- "o_proj",
30
- "down_proj",
31
- "gate_proj",
32
  "up_proj",
33
- "q_proj"
 
34
  ],
35
  "task_type": "CAUSAL_LM",
36
  "trainable_token_indices": null,
 
24
  "rank_pattern": {},
25
  "revision": null,
26
  "target_modules": [
27
+ "gate_proj",
28
  "k_proj",
29
  "v_proj",
30
+ "q_proj",
 
 
31
  "up_proj",
32
+ "o_proj",
33
+ "down_proj"
34
  ],
35
  "task_type": "CAUSAL_LM",
36
  "trainable_token_indices": null,
final/adapter_model.safetensors CHANGED
@@ -1,3 +1,3 @@
1
  version https://git-lfs.github.com/spec/v1
2
- oid sha256:3bd0d0deb9f6c7a072445eb419b7c36b3303a236468d8409ba78253d892e8779
3
- size 2501071448
 
1
  version https://git-lfs.github.com/spec/v1
2
+ oid sha256:d8ec90ab1c2f0e387fc92598b9763f484b90ef179cd9524e536655c735be8947
3
+ size 528550256
final/added_tokens.json CHANGED
The diff for this file is too large to render. See raw diff
 
final/tokenizer.json CHANGED
@@ -1,3 +1,3 @@
1
  version https://git-lfs.github.com/spec/v1
2
- oid sha256:73e0dd860929d8b8842232d6b15674a5c94b615d457f717f7241735a26aaeffe
3
- size 19162453
 
1
  version https://git-lfs.github.com/spec/v1
2
+ oid sha256:fdbbfab204782795ee3986e63016e44007fe18b73404bca7f3093ca7064c59c3
3
+ size 15261883
final/tokenizer_config.json CHANGED
The diff for this file is too large to render. See raw diff
 
final/training_args.bin CHANGED
@@ -1,3 +1,3 @@
1
  version https://git-lfs.github.com/spec/v1
2
- oid sha256:95d18491b18d50456464212e4933a1c39dd0ccb95b2aa0703ae0345d5193a664
3
  size 5841
 
1
  version https://git-lfs.github.com/spec/v1
2
+ oid sha256:64bc082ecbed46e65e44dd270d995a0bde5b0c6b6dd2bd9a7cde505dd566cf2f
3
  size 5841