Update README.md
Browse files
README.md
CHANGED
|
@@ -6,8 +6,6 @@ license: mit
|
|
| 6 |
|
| 7 |
Making decoder-only transformers predict state consequences instead of tokens.
|
| 8 |
|
| 9 |
-
🔗 **[View on Hugging Face](https://huggingface.co/wassemgtk/jepa_llm_prototypes)**
|
| 10 |
-
|
| 11 |
## What's This?
|
| 12 |
|
| 13 |
Three approaches to convert a standard LLM into a world model that predicts "what happens next" given a state and action — like JEPA but for language models.
|
|
@@ -23,7 +21,7 @@ Three approaches to convert a standard LLM into a world model that predicts "wha
|
|
| 23 |
## Quick Start
|
| 24 |
|
| 25 |
1. Open any notebook in [Google Colab](https://colab.research.google.com/)
|
| 26 |
-
2. Set runtime to **GPU** (Runtime → Change runtime type →
|
| 27 |
3. Run all cells
|
| 28 |
4. Watch the model learn to predict state transitions
|
| 29 |
|
|
@@ -77,7 +75,7 @@ All dependencies install automatically in the notebooks.
|
|
| 77 |
## Next Steps
|
| 78 |
|
| 79 |
- Swap synthetic data for real enterprise workflow logs
|
| 80 |
-
- Scale up base model (Llama,
|
| 81 |
- Add multi-step trajectory prediction
|
| 82 |
- Integrate with planning/search algorithms
|
| 83 |
|
|
|
|
| 6 |
|
| 7 |
Making decoder-only transformers predict state consequences instead of tokens.
|
| 8 |
|
|
|
|
|
|
|
| 9 |
## What's This?
|
| 10 |
|
| 11 |
Three approaches to convert a standard LLM into a world model that predicts "what happens next" given a state and action — like JEPA but for language models.
|
|
|
|
| 21 |
## Quick Start
|
| 22 |
|
| 23 |
1. Open any notebook in [Google Colab](https://colab.research.google.com/)
|
| 24 |
+
2. Set runtime to **GPU** (Runtime → Change runtime type → H100)
|
| 25 |
3. Run all cells
|
| 26 |
4. Watch the model learn to predict state transitions
|
| 27 |
|
|
|
|
| 75 |
## Next Steps
|
| 76 |
|
| 77 |
- Swap synthetic data for real enterprise workflow logs
|
| 78 |
+
- Scale up base model (Llama, Qwen, Palmyra)
|
| 79 |
- Add multi-step trajectory prediction
|
| 80 |
- Integrate with planning/search algorithms
|
| 81 |
|