Add nano-Geneformer as a community reference implementation
#588
by nqhuya - opened
README.md
CHANGED
|
@@ -10,7 +10,15 @@ Geneformer is a foundational transformer model pretrained on a large-scale corpu
|
|
| 10 |
|
| 11 |
- See [our manuscript](https://rdcu.be/ddrx0) for details of the original model trained on ~30 million transcriptomes in June 2021 and the initial report of our in silico perturbation and cell and gene classification strategies.
|
| 12 |
- See [our manuscript](https://rdcu.be/famFk) for details of the expanded model, now trained on ~104 million transcriptomes, and our quantization implementation for resource-efficient predictions.
|
| 13 |
-
- See [our preprint](https://www.biorxiv.org/content/10.1101/2024.08.16.608180v1.full.pdf
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 14 |
- See [geneformer.readthedocs.io](https://geneformer.readthedocs.io) for documentation.
|
| 15 |
|
| 16 |
# Model Description
|
|
@@ -60,6 +68,9 @@ Example applications demonstrated in [our manuscript](https://rdcu.be/ddrx0) inc
|
|
| 60 |
- in silico perturbation to determine transcription factor targets
|
| 61 |
- in silico perturbation to determine transcription factor cooperativity
|
| 62 |
|
|
|
|
|
|
|
|
|
|
| 63 |
# Installation
|
| 64 |
In addition to the pretrained model, contained herein are functions for tokenizing and collating data specific to single cell transcriptomics, pretraining the model, fine-tuning the model, extracting and plotting cell embeddings, and performing in silico pertrubation with either the pretrained or fine-tuned models. To install (~20s):
|
| 65 |
|
|
|
|
| 10 |
|
| 11 |
- See [our manuscript](https://rdcu.be/ddrx0) for details of the original model trained on ~30 million transcriptomes in June 2021 and the initial report of our in silico perturbation and cell and gene classification strategies.
|
| 12 |
- See [our manuscript](https://rdcu.be/famFk) for details of the expanded model, now trained on ~104 million transcriptomes, and our quantization implementation for resource-efficient predictions.
|
| 13 |
+
- See [our preprint](https://www.biorxiv.org/content/10.1101/2024.08.16.608180v1.full.pdf
|
| 14 |
+
|
| 15 |
+
|
| 16 |
+
|
| 17 |
+
|
| 18 |
+
|
| 19 |
+
|
| 20 |
+
|
| 21 |
+
) for details of our continual and multitask learning strategies.
|
| 22 |
- See [geneformer.readthedocs.io](https://geneformer.readthedocs.io) for documentation.
|
| 23 |
|
| 24 |
# Model Description
|
|
|
|
| 68 |
- in silico perturbation to determine transcription factor targets
|
| 69 |
- in silico perturbation to determine transcription factor cooperativity
|
| 70 |
|
| 71 |
+
# Community Projects
|
| 72 |
+
**[nano-Geneformer](https://github.com/huynguyen250896/nano-Geneformer)** — A lightweight, faithful, and easy-to-read reimplementation of Geneformer for learning, benchmarking, rapid experimentation, and extension.
|
| 73 |
+
|
| 74 |
# Installation
|
| 75 |
In addition to the pretrained model, contained herein are functions for tokenizing and collating data specific to single cell transcriptomics, pretraining the model, fine-tuning the model, extracting and plotting cell embeddings, and performing in silico pertrubation with either the pretrained or fine-tuned models. To install (~20s):
|
| 76 |
|