codefuse-ai

/

F2LLM-v2-330M

like 8

Follow

CodeFuse AI 230

1_Pooling/config.json declares word_embedding_dimension: 1024, but this model's actual hidden_size (per config.json) is 896. This mismatch causes sentence-transformers' MSELoss (which sizes its projection layer via model.get_embedding_dimension(), trusting this file) to build a Linear(1024, ...) layer instead of Linear(896, ...), raising a shape mismatch at runtime when real 896-dim embeddings are passed in. It also affects any other pooling-dimension-derived Matryoshka dimension defaults. This PR corrects word_embedding_dimension to 896 to match config.json's hidden_size; all other pooling settings are unchanged.