Cheng98 commited on
Commit
1213e01
·
verified ·
1 Parent(s): 7855acc

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +3 -2
README.md CHANGED
@@ -15,7 +15,8 @@ A 60M parameter language model trained on `22 * 60M` tokens from FineWeb-Edu dat
15
 
16
  ## Model Details
17
 
18
- aixsim-60M is a transformer-based language model with approximately 60 million parameters (embedding layer params excluded). It uses RMSNorm for normalization and is trained on the FineWeb dataset.
 
19
 
20
  - **Developed by:** AICrossSim
21
  - **Funded by:** [ARIA](https://www.aria.org.uk/)
@@ -34,7 +35,7 @@ Experiment setup and training logs can be found at [wandb run](https://wandb.ai/
34
  import transformers
35
 
36
  model_name="AICrossSim/clm-60m"
37
- model = transformers.AutoModelForCausalLM.from_pretrained(model_name, torch_dtype=getattr(torch, dtype))
38
  tokenizer = transformers.AutoTokenizer.from_pretrained(model_name)
39
  ```
40
 
 
15
 
16
  ## Model Details
17
 
18
+ aixsim-60M is a transformer-based language model with approximately 60 million parameters (embedding layer params excluded).
19
+ It uses RMSNorm for normalization and is trained on the FineWeb-Edu dataset.
20
 
21
  - **Developed by:** AICrossSim
22
  - **Funded by:** [ARIA](https://www.aria.org.uk/)
 
35
  import transformers
36
 
37
  model_name="AICrossSim/clm-60m"
38
+ model = transformers.AutoModelForCausalLM.from_pretrained(model_name)
39
  tokenizer = transformers.AutoTokenizer.from_pretrained(model_name)
40
  ```
41