Spaces:
Running
Running
Update README.md
Browse files
README.md
CHANGED
|
@@ -7,4 +7,53 @@ sdk: static
|
|
| 7 |
pinned: false
|
| 8 |
---
|
| 9 |
|
| 10 |
-
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 7 |
pinned: false
|
| 8 |
---
|
| 9 |
|
| 10 |
+
# Marin
|
| 11 |
+
|
| 12 |
+
Marin is an open lab for building foundation models---together.
|
| 13 |
+
We're training powerful models from scratch, and sharing and programmatically documenting every step:
|
| 14 |
+
the code, the data, the experiments, the mistakes...all in real-time.
|
| 15 |
+
We invite anyone who shares our vision of open science and open-source to join and contribute,
|
| 16 |
+
whether you want to try out a new architecture, training algorithm, dataset,
|
| 17 |
+
evaluation...there is a lot to do!
|
| 18 |
+
|
| 19 |
+
Find us here:
|
| 20 |
+
|
| 21 |
+
- [GitHub](https://github.com/marin-community/marin): where the code for reproducing all experiments live
|
| 22 |
+
- [Discord](https://discord.gg/J9CTk7pqcM): come hang out with us, ask questions, and coordinate on research directions
|
| 23 |
+
- [Documentation](https://marin.readthedocs.io/en/latest/): where you can find tutorials and explanations
|
| 24 |
+
- [DeepWiki](https://deepwiki.com/marin-community/marin): automatically generated Wiki with an integrated chatbot
|
| 25 |
+
- [HuggingFace](https://huggingface.co/marin-community): where to find our datasets and models
|
| 26 |
+
- [WandB](https://wandb.ai/marin-community): where to find our training runs
|
| 27 |
+
- [Homepage](https://marin.community): See this content on another page
|
| 28 |
+
|
| 29 |
+
Want to jump in? [Install](https://marin.readthedocs.io/en/latest/tutorials/installation/) the Marin code and
|
| 30 |
+
[run your first experiment](https://marin.readthedocs.io/en/latest/tutorials/first-experiment/)!
|
| 31 |
+
|
| 32 |
+
## Experiments
|
| 33 |
+
|
| 34 |
+
Building a foundation model requires countless experiments trying out endless variants of algorithms and datasets.
|
| 35 |
+
All the experiments we're doing are captured as [GitHub issues](https://github.com/marin-community/marin/issues?q=is%3Aissue%20label%3Aexperiment).
|
| 36 |
+
Here is a [summary](https://marin.readthedocs.io/en/latest/reports/) to get the lay of the land.
|
| 37 |
+
|
| 38 |
+
## Speedrun 🏃
|
| 39 |
+
|
| 40 |
+
Have a new architecture or training procedure that's more efficient?
|
| 41 |
+
Participate in the [Marin speedrun](https://marin.community/speedrun) competition
|
| 42 |
+
(inspired by the [nanogpt speedrun](https://github.com/KellerJordan/modded-nanogpt?tab=readme-ov-file#world-record-history))
|
| 43 |
+
and create the fastest method to train a model to a certain quality!
|
| 44 |
+
Get started [here](https://marin.readthedocs.io/en/latest/tutorials/submitting-speedrun/).
|
| 45 |
+
|
| 46 |
+
## Datashop 🛠️
|
| 47 |
+
|
| 48 |
+
Want to add new capabilities to the Marin models?
|
| 49 |
+
Go visit our datashop, where you can upload a dataset or craft a prompt to curate a relevant dataset for your task.
|
| 50 |
+
|
| 51 |
+
## Models 🌐
|
| 52 |
+
|
| 53 |
+
We have trained a strong 8B parameter model. Try it out [here](https://huggingface.co/spaces/WillHeld/soft-racoon-test)!
|
| 54 |
+
|
| 55 |
+
## Acknowledgements
|
| 56 |
+
|
| 57 |
+
Marin wouldn't be possible without the generous support of the [Google TPU Research Cloud program](https://sites.research.google/trc/about/).
|
| 58 |
+
We also benefit immensely from the broader open ecosystem, who have released numerous tools and datasets, including
|
| 59 |
+
[AI2](https://allenai.org), [Hugging Face](https://huggingface.co/), [NVIDIA](https://www.nvidia.com/en-us/research/), etc.
|