Update README.md

README.md

@@ -10,4 +10,38 @@ tags:
 - language modeling
 metrics:
 - BrierLM
----
+---
+
+# Continuous Autoregressive Language Models
+
+[![Paper](https://img.shields.io/badge/Paper_📃-green)](https://arxiv.org/abs/)
+[![GitHub](https://img.shields.io/badge/GitHub_🧑‍💻-blue)](https://github.com/shaochenze/calm)
+[![HuggingFace](https://img.shields.io/badge/HuggingFace_🤗%20-CALM%20-orange)](https://huggingface.co/collections/cccczshao/calm)
+
+## Model Description
+
+Modern Large Language Models (LLMs) are constrained by a fundamental bottleneck: they generate text one token at a time. **CALM (Continuous Autoregressive Language Models)** confronts this challenge by introducing a paradigm shift in language modeling. Instead of predicting one discrete token at a time, CALM learns to predict a single continuous vector that represents an entire chunk of K tokens.
+
+This is achieved through a two-stage process:
+
+1. **A high-fidelity autoencoder** learns to compress K tokens into a single vector and reconstruct them with near-perfect accuracy.
+2. **A continuous-domain language model** then performs autoregressive prediction in this vector space.
+
+### Key Features
+
+* 🚀 **Ultra-Efficient by Design:** Dramatically improves training and inference efficiency by reducing the number of autoregressive steps by a factor of K.
+* 💡 **A New Scaling Axis:** Introduces a new scaling dimension for LLMs—semantic bandwidth (K). Instead of just scaling parameters and data, you can now scale the amount of information processed in a single step.
+* 🛠️ **A Comprehensive Likelihood-Free Toolkit:** Operating in a continuous domain requires new tools. This repository provides the full suite of algorithms that make CALM possible:
+  
+  * **A Robust Autoencoder** to learn high-fidelity continuous representations of token chunks.
+  * **Energy-Based Training**, a principled and likelihood-free method for generative modeling.
+  * **BrierLM**, a new metric for calibrated, likelihood-free evaluation of language models.
+  * **Temperature Sampling** for controlled, high-quality text generation using only a black-box sampler.
+
+## How to use
+
+See our [GitHub README](https://github.com/shaochenze/calm), where we provide scripts for training and evaluation.
+
+## Contact
+
+If you have any questions, feel free to submit an issue or contact `chenzeshao@tencent.com`.