Partial answers in the model card
Hi, I'm trying to test the model card with the question below, but the answer is truncated.
Hitting "compute" again adds a few more words, but still seems incomplete.
--------Quote----------------------------------------
How does pcsk9 impact ldl receptor processing? {#s4a}
The LDLr pathway requires proteolytic cleavage of the receptor to generate the mature, non-
Hi, the model is a bit too big for the HF hosted inference API (or at least the free version) to work well. You're best off trying it with your own hardware
Thanks. I tried moving it to a dedicated instance (GPU Medium instance) and i do get longer answers, although it does still seem to get "stuck".
Any advice on the right instance size ?
Also - what's the right way to use "ChatGPT mode" ( answering questions ) ? I tried several questions and it don't seem to behave the same way
The model is not really "chat" or "instruction" tuned, yet (this is something we're interested in). So it's not going to be particularly chatty out of the box. It's mostly going to want to imitate pubmed articles or abstracts.
In our experiments so far we finetuned it for the particular tasks. We haven't released those models.
I don't really have any experience with HF's inference APIs so I couldn't really say what's expected to work... I would have thought a GPU with 30gb would be more than enough, but again I haven't used their APIs.
Thanks