No “lm_head.weight” Layer Question.
#6
by wikeeyang - opened
Hi, Kijai, Why does your model, compared to the official, haven't the "lm_head.weight" layer? Does this layer serve no purpose?
It is used only for autoregressive text generation, it serves no purpose for this model.
It is used only for autoregressive text generation, it serves no purpose for this model.
Got it, thank you very much! 👍
wikeeyang changed discussion status to closed