ApacheOne commited on
Commit
84fd786
·
verified ·
1 Parent(s): 7a6267b

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +9 -0
README.md CHANGED
@@ -5,6 +5,15 @@ base_model:
5
  base_model_relation: quantized
6
  ---
7
 
 
 
 
 
 
 
 
 
 
8
  Not sure of the best way to even run the base model yet, Should work where you offload the dense layers into ram and only leave the active layers in vram.
9
 
10
  # UNTESTED
 
5
  base_model_relation: quantized
6
  ---
7
 
8
+ # UPDATE
9
+
10
+ I am still getting OOM on a L4 24GB , 64gb ram system.
11
+
12
+ Aggressive quant is most likely a failure and will not work as is.
13
+
14
+ 2d weights only quant does load and run but still getting OOM from improper loading method.
15
+
16
+
17
  Not sure of the best way to even run the base model yet, Should work where you offload the dense layers into ram and only leave the active layers in vram.
18
 
19
  # UNTESTED