alikLab
/

NoLBERT

@@ -101,14 +101,12 @@ print(f"Text: {text}")
 print(f"Embedding (first 10 dimensions): {cls_embedding[:10].tolist()}")
 ```
-### A discussion about model choice
 Even though NoLBERT has the advantage of no lookahead and lookback bias, researchers should carefully consider their model choice on a case-by-case basis, especially for long texts.
 In particular, there is a bias–performance trade-off between NoLBERT or other custom small models (or simpler NLP methods, e.g., BoW, Word2Vec, etc.) versus large industrial-grade language models. On one hand, a BERT-like custom information-leakage-free model avoids temporal inconsistencies by design. On the other hand, these models lack the ability to process long texts due to limited context windows, and their output text representations are often of lower quality compared to large models trained on unconstrained data.
-% and is not verifiable from the input text,
 The advantage of avoiding temporal biases is pronounced in tasks where models must predict outcomes that go beyond the information explicitly stated in the text, such as forecasting stock price reactions from earnings call transcripts, despite the tradeoff of having less precise text representations. However, for in-context information retrieval tasks such as summarization, classification, and other NLP tasks based on given precise guidelines, the risk of information leakage from the model’s out-of-context knowledge base is limited (with careful prompting and verification, or by using methods like RAG). Therefore, large, highly performant models may be preferable.
@@ -119,10 +117,9 @@ If you use this model in your research, please cite:
 ```
 @misc{nolbert,
   author = {Ali Kakhbod, Peiyao Li},
-  title = {NoLBert: A Time-Stamped Pre-Trained LLM},
   year = {2025},
-  publisher = {Hugging Face},
-  journal = {Hugging Face Model Hub},
   howpublished = {\url{https://huggingface.co/alikLab/NoLBERT}},
 }
 ```

 print(f"Embedding (first 10 dimensions): {cls_embedding[:10].tolist()}")
 ```
+### A note about model choice
 Even though NoLBERT has the advantage of no lookahead and lookback bias, researchers should carefully consider their model choice on a case-by-case basis, especially for long texts.
 In particular, there is a bias–performance trade-off between NoLBERT or other custom small models (or simpler NLP methods, e.g., BoW, Word2Vec, etc.) versus large industrial-grade language models. On one hand, a BERT-like custom information-leakage-free model avoids temporal inconsistencies by design. On the other hand, these models lack the ability to process long texts due to limited context windows, and their output text representations are often of lower quality compared to large models trained on unconstrained data.
 The advantage of avoiding temporal biases is pronounced in tasks where models must predict outcomes that go beyond the information explicitly stated in the text, such as forecasting stock price reactions from earnings call transcripts, despite the tradeoff of having less precise text representations. However, for in-context information retrieval tasks such as summarization, classification, and other NLP tasks based on given precise guidelines, the risk of information leakage from the model’s out-of-context knowledge base is limited (with careful prompting and verification, or by using methods like RAG). Therefore, large, highly performant models may be preferable.
 ```
 @misc{nolbert,
   author = {Ali Kakhbod, Peiyao Li},
+  title = {NoLBERT: A No Lookahead(back) Foundational Language Model},
   year = {2025},
+  journal = {NeurIPS 2025 (GenAI in Finance)},
   howpublished = {\url{https://huggingface.co/alikLab/NoLBERT}},
 }
 ```