infosys
/

NT-Java-1.1B

Text Generation

NarrowTransformer

text-generation-inference

Model card Files Files and versions

rajabmondal commited on Jul 4, 2024

Commit

f8bbaeb

·

verified ·

1 Parent(s): b3565f9

Update README.md

Files changed (1) hide show

README.md +1 -1

README.md CHANGED Viewed

@@ -121,7 +121,7 @@ print(tokenizer.decode(outputs[0]))
 ## Model
 - **Architecture:** GPT-2 model with Multi-Query Attention and Fill-in-the-Middle objective.
-- **Pretraining steps:** 100K
 - **Context length:** 8K tokens
 - **Pretraining tokens:** 22 billion
 - **Precision:** bfloat16

 ## Model
 - **Architecture:** GPT-2 model with Multi-Query Attention and Fill-in-the-Middle objective.
+- **Training steps:** 120K
 - **Context length:** 8K tokens
 - **Pretraining tokens:** 22 billion
 - **Precision:** bfloat16