more cleanup on docs

.readthedocs.yaml

@@ -1,19 +0,0 @@
-# Read the Docs configuration file
-
-# Required
-version: 2
-
-# Set the OS, Python version and other tools you might need
-build:
-  os: ubuntu-22.04
-  tools:
-    python: "3.10"
-
-# Build documentation in the "docs/" directory with Sphinx
-sphinx:
-  configuration: docs/source/conf.py
-
-# Python requirements required build your documentation
-python:
-   install:
-   - requirements: docs/requirements.txt

docs/Makefile

@@ -1,20 +0,0 @@
-# Minimal makefile for Sphinx documentation
-#
-
-# You can set these variables from the command line, and also
-# from the environment for the first two.
-SPHINXOPTS    ?=
-SPHINXBUILD   ?= sphinx-build
-SOURCEDIR     = source
-BUILDDIR      = build
-
-# Put it first so that "make" without argument is like "make help".
-help:
-	@$(SPHINXBUILD) -M help "$(SOURCEDIR)" "$(BUILDDIR)" $(SPHINXOPTS) $(O)
-
-.PHONY: help Makefile
-
-# Catch-all target: route all unknown targets to Sphinx using the new
-# "make mode" option.  $(O) is meant as a shortcut for $(SPHINXOPTS).
-%: Makefile
-	@$(SPHINXBUILD) -M $@ "$(SOURCEDIR)" "$(BUILDDIR)" $(SPHINXOPTS) $(O)

docs/make.bat

@@ -1,35 +0,0 @@
-@ECHO OFF
-
-pushd %~dp0
-
-REM Command file for Sphinx documentation
-
-if "%SPHINXBUILD%" == "" (
-	set SPHINXBUILD=sphinx-build
-)
-set SOURCEDIR=source
-set BUILDDIR=build
-
-%SPHINXBUILD% >NUL 2>NUL
-if errorlevel 9009 (
-	echo.
-	echo.The 'sphinx-build' command was not found. Make sure you have Sphinx
-	echo.installed, then set the SPHINXBUILD environment variable to point
-	echo.to the full path of the 'sphinx-build' executable. Alternatively you
-	echo.may add the Sphinx directory to PATH.
-	echo.
-	echo.If you don't have Sphinx installed, grab it from
-	echo.https://www.sphinx-doc.org/
-	exit /b 1
-)
-
-if "%1" == "" goto help
-
-%SPHINXBUILD% -M %1 %SOURCEDIR% %BUILDDIR% %SPHINXOPTS% %O%
-goto end
-
-:help
-%SPHINXBUILD% -M help %SOURCEDIR% %BUILDDIR% %SPHINXOPTS% %O%
-
-:end
-popd

docs/requirements.txt

@@ -1,3 +0,0 @@
-.
-sphinx_rtd_theme==2.0.0
-nbsphinx==0.9.3

docs/source/_static/css/custom.css

@@ -1,40 +0,0 @@
-/* top left logo */
-.wy-side-nav-search, .wy-nav-top {
-    background: linear-gradient(15deg, #13547a 0%, #80d0c7 100%);
-}
-
-
-/* unvisited link */
-.wy-nav-content a:link {
-  color: #067abd;
-}
-
-/* visited link */
-.wy-nav-content a:visited {
-  color: #4b827c;
-}
-
-/* mouse over link */
-.wy-nav-content a:hover {
-  color: #80d0c7;
-}
-
-/* selected link */
-.wy-nav-content a:active {
-  color: #4b827c;
-}
-
-/* class object */
-.sig.sig-object {
-    padding: 5px 5px 5px 5px;
-    background-color: #ececec;
-    border-style: solid;
-    border-color: black;
-    border-width: 1px 0;
-}
-
-/* parameter object */
-dt {
-    padding: 5px 5px 5px 5px;
-    background-color: #ececec;
-}

docs/source/about.rst

@@ -1,49 +0,0 @@
-About
-=====
-
-Model Description
------------------
-
-**Geneformer** is a context-aware, attention-based deep learning model pretrained on a large-scale corpus of single-cell transcriptomes to enable context-specific predictions in settings with limited data in network biology. During pretraining, Geneformer gained a fundamental understanding of network dynamics, encoding network hierarchy in the attention weights of the model in a completely self-supervised manner. With both zero-shot learning and fine-tuning with limited task-specific data, Geneformer consistently boosted predictive accuracy in a diverse panel of downstream tasks relevant to chromatin and network dynamics. In silico perturbation with zero-shot learning identified a novel transcription factor in cardiomyocytes that we experimentally validated to be critical to their ability to generate contractile force. In silico treatment with limited patient data revealed candidate therapeutic targets for cardiomyopathy that we experimentally validated to significantly improve the ability of cardiomyocytes to generate contractile force in an iPSC model of the disease. Overall, Geneformer represents a foundational deep learning model pretrained on a large-scale corpus of human single cell transcriptomes to gain a fundamental understanding of gene network dynamics that can now be democratized to a vast array of downstream tasks to accelerate discovery of key network regulators and candidate therapeutic targets.
-
-In `our manuscript <https://rdcu.be/ddrx0>`_, we report results for the original 6 layer Geneformer model pretrained on Genecorpus-30M. We additionally provide within the repository a 12 layer Geneformer model, scaled up with retained width:depth aspect ratio, also pretrained on Genecorpus-30M.
-
-Both the `6 <https://huggingface.co/ctheodoris/Geneformer/blob/main/gf-6L-30M-i2048/model.safetensors>`_ and `12 <https://huggingface.co/ctheodoris/Geneformer/blob/main/gf-12L-30M-i2048/pytorch_model.bin>`_ layer Geneformer models were pretrained in June 2021.
-
-Also see `our 2024 manuscript <https://www.biorxiv.org/content/10.1101/2024.08.16.608180v1.full.pdf>`_, for details of the `expanded model <https://huggingface.co/ctheodoris/Geneformer/blob/main/model.safetensors>`_ trained on ~95 million transcriptomes in April 2024 and our continual learning, multitask learning, and quantization strategies.
-
-Application
------------
-
-The pretrained Geneformer model can be used directly for zero-shot learning, for example for in silico perturbation analysis, or by fine-tuning towards the relevant downstream task, such as gene or cell state classification.
-
-Example applications demonstrated in `our manuscript <https://rdcu.be/ddrx0>`_ include:
-
-| *Fine-tuning*:
-| - transcription factor dosage sensitivity
-| - chromatin dynamics (bivalently marked promoters)
-| - transcription factor regulatory range
-| - gene network centrality
-| - transcription factor targets
-| - cell type annotation
-| - batch integration
-| - cell state classification across differentiation
-| - disease classification
-| - in silico perturbation to determine disease-driving genes
-| - in silico treatment to determine candidate therapeutic targets
-
-| *Zero-shot learning*:
-| - batch integration
-| - gene context specificity
-| - in silico reprogramming
-| - in silico differentiation
-| - in silico perturbation to determine impact on cell state
-| - in silico perturbation to determine transcription factor targets
-| - in silico perturbation to determine transcription factor cooperativity
-
-Citations
----------
-
-| C V Theodoris #, L Xiao, A Chopra, M D Chaffin, Z R Al Sayed, M C Hill, H Mantineo, E Brydon, Z Zeng, X S Liu, P T Ellinor #. `Transfer learning enables predictions in network biology. <https://rdcu.be/ddrx0>`_ *Nature*, 31 May 2023. (# co-corresponding authors)
-
-| H Chen \*, M S Venkatesh \*, J Gomez Ortega, S V Mahesh, T Nandi, R Madduri, K Pelka †, C V Theodoris † #. `Quantized multi-task learning for context-specific representations of gene network dynamics. <https://www.biorxiv.org/content/10.1101/2024.08.16.608180v1.full.pdf>`_ *bioRxiv*, 19 Aug 2024. (\* co-first authors, † co-senior authors, # corresponding author)

docs/source/api.rst

@@ -1,51 +0,0 @@
-API
-===
-
-Tokenizer
----------
-
-.. toctree::
-   :maxdepth: 1
-
-   geneformer.tokenizer
-
-Classifier
-----------
-
-.. toctree::
-   :maxdepth: 1
-
-   geneformer.classifier
-
-Multitask Classifier
---------------------
-
-.. toctree::
-   :maxdepth: 1
-
-   geneformer.mtl_classifier
-
-Embedding Extractor
--------------------
-
-.. toctree::
-   :maxdepth: 1
-
-   geneformer.emb_extractor
-
-In Silico Perturber
--------------------
-
-.. toctree::
-   :maxdepth: 1
-
-   geneformer.in_silico_perturber
-
-
-In Silico Perturber Stats
--------------------------
-
-.. toctree::
-   :maxdepth: 1
-
-   geneformer.in_silico_perturber_stats

docs/source/conf.py

@@ -1,80 +0,0 @@
-# Configuration file for the Sphinx documentation builder.
-#
-# For the full list of built-in configuration values, see the documentation:
-# https://www.sphinx-doc.org/en/master/usage/configuration.html
-
-import pathlib
-import re
-import sys
-
-from sphinx.ext import autodoc
-
-sys.path.insert(0, pathlib.Path(__file__).parents[2].resolve().as_posix())
-
-
-# -- Project information -----------------------------------------------------
-# https://www.sphinx-doc.org/en/master/usage/configuration.html#project-information
-
-project = "geneformer"
-copyright = "2024, Christina Theodoris"
-author = "Christina Theodoris"
-release = "0.1.0"
-repository_url = "https://huggingface.co/ctheodoris/Geneformer"
-
-# -- General configuration ---------------------------------------------------
-# https://www.sphinx-doc.org/en/master/usage/configuration.html#general-configuration
-
-extensions = [
-    "sphinx.ext.autodoc",
-    "sphinx.ext.autosummary",
-    "nbsphinx",
-    "sphinx.ext.viewcode",
-    "sphinx.ext.doctest",
-]
-
-templates_path = ["_templates"]
-exclude_patterns = [
-    "**.ipynb_checkpoints",
-]
-autoclass_content = "both"
-
-
-class MockedClassDocumenter(autodoc.ClassDocumenter):
-    def add_line(self, line: str, source: str, *lineno: int) -> None:
-        if line == "   Bases: :py:class:`object`":
-            return
-        super().add_line(line, source, *lineno)
-
-
-autodoc.ClassDocumenter = MockedClassDocumenter
-add_module_names = False
-
-
-def process_signature(app, what, name, obj, options, signature, return_annotation):
-    # loop through each line in the docstring and replace path with
-    # the generic path text
-    signature = re.sub(r"PosixPath\(.*?\)", "FILEPATH", signature)
-    return (signature, None)
-
-
-def setup(app):
-    app.connect("autodoc-process-signature", process_signature)
-
-
-# -- Options for HTML output -------------------------------------------------
-# https://www.sphinx-doc.org/en/master/usage/configuration.html#options-for-html-output
-
-html_theme = "sphinx_rtd_theme"
-html_show_sphinx = False
-html_static_path = ["_static"]
-html_logo = "_static/gf_logo.png"
-html_theme_options = {
-    "collapse_navigation": False,
-    "sticky_navigation": True,
-    "navigation_depth": 3,
-    "logo_only": True,
-}
-html_css_files = [
-    "css/custom.css",
-]
-html_show_sourcelink = False

docs/source/geneformer.classifier.rst

@@ -1,10 +0,0 @@
-geneformer.classifier
-=====================
-
-.. automodule:: geneformer.classifier
-   :members:
-   :undoc-members:
-   :show-inheritance:
-   :exclude-members:
-        valid_option_dict,
-        validate_options

docs/source/geneformer.emb_extractor.rst

@@ -1,26 +0,0 @@
-geneformer.emb\_extractor
-=========================
-
-.. automodule:: geneformer.emb_extractor
-   :members:
-   :undoc-members:
-   :show-inheritance:
-   :exclude-members:
-       accumulate_tdigests,
-       gen_heatmap_class_colors,
-       gen_heatmap_class_dict,
-       get_embs,
-       label_cell_embs,
-       label_gene_embs,
-       make_colorbar,
-       plot_heatmap,
-       plot_umap,
-       summarize_gene_embs,
-       tdigest_mean,
-       tdigest_median,
-       test_emb,
-       update_tdigest_dict,
-       update_tdigest_dict_mean,
-       update_tdigest_dict_median,
-       valid_option_dict,
-       validate_options

docs/source/geneformer.in_silico_perturber.rst

@@ -1,8 +0,0 @@
-geneformer.in\_silico\_perturber
-=======================================
-
-.. automodule:: geneformer.in_silico_perturber
-   :members:
-   :undoc-members:
-   :show-inheritance:
-   :exclude-members:  valid_option_dict, validate_options, apply_additional_filters, isp_perturb_all, isp_perturb_set, update_perturbation_dictionary

docs/source/geneformer.in_silico_perturber_stats.rst

@@ -1,25 +0,0 @@
-geneformer.in\_silico\_perturber\_stats
-==============================================
-
-.. automodule:: geneformer.in_silico_perturber_stats
-   :members:
-   :undoc-members:
-   :show-inheritance:
-   :exclude-members:
-        find,
-        get_fdr,
-        get_gene_list,
-        get_impact_component,
-        invert_dict,
-        isp_aggregate_gene_shifts,
-        isp_aggregate_grouped_perturb,
-        isp_stats_mixture_model,
-        isp_stats_to_goal_state,
-        isp_stats_vs_null,
-        n_detections,
-        read_dict,
-        read_dictionaries,
-        token_to_gene_name,
-        token_tuple_to_ensembl_ids,
-        valid_option_dict,
-        validate_options

docs/source/geneformer.mtl_classifier.rst

@@ -1,11 +0,0 @@
-geneformer.mtl\_classifier
-==========================
-
-.. automodule:: geneformer.mtl_classifier
-   :members:
-   :undoc-members:
-   :show-inheritance:
-   :exclude-members:
-        valid_option_dict,
-        validate_options,
-        validate_additional_options

docs/source/geneformer.tokenizer.rst

@@ -1,15 +0,0 @@
-geneformer.tokenizer
-====================
-
-.. automodule:: geneformer.tokenizer
-   :members:
-   :undoc-members:
-   :show-inheritance:
-   :exclude-members:
-        create_dataset,
-        tokenize_anndata,
-        tokenize_files,
-        tokenize_loom,
-        rank_genes,
-        tokenize_cell,
-        sum_ensembl_ids

docs/source/getstarted.rst

@@ -1,36 +0,0 @@
-Getting Started
-===============
-
-Installation
-------------
-
-Geneformer installation instructions.
-
-Make sure you have git-lfs installed (https://git-lfs.com).
-
-.. code-block:: bash
-
-    git lfs install
-    git clone https://huggingface.co/ctheodoris/Geneformer
-    cd Geneformer
-    pip install .
-
-
-Tutorials
----------
-
-| See `examples <https://huggingface.co/ctheodoris/Geneformer/tree/main/examples>`_ for:
-| - tokenizing transcriptomes
-| - pretraining
-| - hyperparameter tuning
-| - fine-tuning
-| - extracting and plotting cell embeddings
-| - in silico perturbation
-
-Please note that the fine-tuning examples are meant to be generally applicable and the input datasets and labels will vary dependent on the downstream task. Example input files for a few of the downstream tasks demonstrated in the manuscript are located within the `example_input_files directory <https://huggingface.co/datasets/ctheodoris/Genecorpus-30M/tree/main/example_input_files>`_ in the dataset repository, but these only represent a few example fine-tuning applications.
-
-
-Tips
-----
-
-Please note that GPU resources are required for efficient usage of Geneformer. Additionally, we strongly recommend tuning hyperparameters for each downstream fine-tuning application as this can significantly boost predictive potential in the downstream task (e.g. max learning rate, learning schedule, number of layers to freeze, etc.).

docs/source/index.rst

@@ -1,16 +0,0 @@
-Geneformer
-==========
-
-Geneformer is a foundation transformer model pretrained on a large-scale corpus of single cell transcriptomes to enable context-aware predictions in network biology.
-
-See `our manuscript <https://rdcu.be/ddrx0>`_ for details.
-
-Table of Contents
------------------
-
-.. toctree::
-   :maxdepth: 2
-
-   about
-   getstarted
-   api