codefuse-ai

/

F2LLM-v2-8B-Preview

like 3

Follow

CodeFuse AI 230

1_Pooling/config.json declares word_embedding_dimension: 1024, but this model's actual hidden_size (per config.json) is 4096. This mismatch causes sentence-transformers' MSELoss (which sizes its projection layer via model.get_embedding_dimension(), trusting this file) to build a Linear(1024, ...) layer instead of Linear(4096, ...), raising a shape mismatch at runtime when real 4096-dim embeddings are passed in. It also affects any other pooling-dimension-derived Matryoshka dimension defaults. This PR corrects word_embedding_dimension to 4096 to match config.json's hidden_size; all other pooling settings are unchanged.