cool model !!
#3
by
gopi87
- opened
cool model i was able to run the model at 12t/sec in 3060 12gb and dual cpu with 256 gb ram but still slow while comparing to the glm 4.7 flash mxfp4.
cool model i was able to run the model at 12t/sec in 3060 12gb and dual cpu with 256 gb ram but still slow while comparing to the glm 4.7 flash mxfp4.
seems like that has more to do with the distribution of weights into system ram vs vram than the model itself.