Request: LM Studio (GGUF FP16) Versions of both PatentGPT-J 1.6B and 6B Models
Request: LM Studio (GGUF FP16) Versions of both PatentGPT-J 1.6B and 6B Models
Hello, and thank you for making the PatentGPT-J models available!
I would love to see both PatentGPT-J 1.6B and PatentGPT-J 6B made available in LM Studio-compatible GGUF format. Because of their compact size, I’d particularly like to run them at FP16 precision rather than only in quantized form. My concern is that quantization might reduce accuracy on models this small, so having unquantized FP16 versions would preserve performance while still being feasible to run locally.
Would you (or someone in the community) be willing to provide LM Studio–ready GGUF FP16 builds of these models? I’d be more than happy to test them and share feedback.
For clarity, LM Studio is based on llama.cpp, and GGUF FP16 files load directly there. This would make these models fully functional offline on Apple Silicon without sacrificing accuracy.
Thanks very much for considering, and for all the work you’ve already put into making these models accessible!