Convert weights to bf16; add max_position_embeddings

#2
  • Convert all 17 tensors from fp16 to bf16 using round-to-nearest-ties-to-even rounding.
  • Add max_position_embeddings=262144 to config.json.

Superseded by a re-submitted PR with corrected Signed-off-by email (yue@lightseek.org).

yueweng changed pull request status to closed

Sign up or log in to comment