Delete README.md
Browse files
README.md
DELETED
|
@@ -1,94 +0,0 @@
|
|
| 1 |
-
# CIS 6270 Course Code
|
| 2 |
-
|
| 3 |
-

|
| 4 |
-
|
| 5 |
-
Here is the code for CIS 6270, Discrete Generative Models, at the University of Pennsylvania. We
|
| 6 |
-
wrote one file per framework so that you can read it beside the lecture notes, with the corruption,
|
| 7 |
-
the loss and the sampler of each method all in that one file, in the notes' own symbols, and a
|
| 8 |
-
comment on almost every line.
|
| 9 |
-
|
| 10 |
-
You can run everything offline on a laptop CPU in seconds. For `lecture_2/flow_matching_mnist.py` we
|
| 11 |
-
download MNIST, as the name says, and `--dataset toy` keeps even that one offline. In Lecture 3 we
|
| 12 |
-
stay offline by default, and you can ask for MNIST or ESM-2 with `--dataset mnist` or
|
| 13 |
-
`--dataset esm2`.
|
| 14 |
-
|
| 15 |
-
To get started, we can clone the repository and build an environment.
|
| 16 |
-
|
| 17 |
-
```bash
|
| 18 |
-
git clone https://huggingface.co/ChatterjeeLab/CIS6270
|
| 19 |
-
cd CIS6270
|
| 20 |
-
python3 -m venv .venv && source .venv/bin/activate
|
| 21 |
-
python -m pip install -r requirements.txt
|
| 22 |
-
|
| 23 |
-
python lecture_4/mdlm.py # train masked diffusion on DNA and generate, about 20 seconds
|
| 24 |
-
python -m pytest tests -q # the notes' worked numbers, checked against the code
|
| 25 |
-
```
|
| 26 |
-
|
| 27 |
-
On Windows PowerShell, we activate with `.venv\Scripts\Activate.ps1`.
|
| 28 |
-
|
| 29 |
-
## The lectures
|
| 30 |
-
|
| 31 |
-
| | Lecture | What the code builds | Guide |
|
| 32 |
-
| --- | --- | --- | --- |
|
| 33 |
-
| 1 | Mathematical Foundations | Every worked example of the chapter, printed beside the chapter's own number | [lecture_1](lecture_1/README.md) |
|
| 34 |
-
| 2 | Flow Matching | Velocity fields and the flow map, the conditional path, the objective, MNIST | [lecture_2](lecture_2/README.md) |
|
| 35 |
-
| 3 | Score Matching and Diffusion | The forward SDE, the three score objectives, DDPM, three samplers, guidance, latent diffusion on protein embeddings | [lecture_3](lecture_3/README.md) |
|
| 36 |
-
| 4 | Discrete Diffusion | Jump processes, MDLM, uniform replacement, D3PM, SEDD, block diffusion, guidance, planners, search | [lecture_4](lecture_4/README.md) |
|
| 37 |
-
| 5 | Discrete Flow Matching | The mixture path and its velocity, correctors, kinetic-optimal paths, Edit Flows, Dirichlet, Fisher and Gumbel geometries, guidance, rectification | [lecture_5](lecture_5/README.md) |
|
| 38 |
-
| 6 | Flow Maps | The three axioms, the three residuals, MeanFlow, Shortcut, consistency, the discrete flow maps, stochastic, expanding and posterior maps | [lecture_6](lecture_6/README.md) |
|
| 39 |
-
| 7 | Optimal Transport and Schrödinger Bridges | Kantorovich and duality, Sinkhorn, displacement interpolation, static and dynamic bridges, DSB, DSBM, SF2M, the discrete bridges, branched and entangled | [lecture_7](lecture_7/README.md) |
|
| 40 |
-
|
| 41 |
-
## How a file is laid out
|
| 42 |
-
|
| 43 |
-
Every script follows the shape of `lecture_2/flow_matching_mnist.py`, so once you can read one of
|
| 44 |
-
them you can read them all. We open with a docstring naming the framework and the chapter sections
|
| 45 |
-
it implements, then number the `# %%` sections in the order the lecture develops them, and finish
|
| 46 |
-
with a `main()` that prints a report.
|
| 47 |
-
|
| 48 |
-
```
|
| 49 |
-
cis6270/ the scaffolding you read once: the data, the four small networks, the training
|
| 50 |
-
loop and the sample metrics
|
| 51 |
-
lecture_1/ ... 7/ one file per framework, standalone and runnable
|
| 52 |
-
tests/ the chapters' worked numbers, asserted against the code
|
| 53 |
-
```
|
| 54 |
-
|
| 55 |
-
Each `lecture_N/` folder stands on its own, and the only import we take from inside the repository
|
| 56 |
-
is `cis6270`, so to read a method you can open its file and stay there.
|
| 57 |
-
|
| 58 |
-
## The data
|
| 59 |
-
|
| 60 |
-
| Data | Used by | Download |
|
| 61 |
-
| --- | --- | --- |
|
| 62 |
-
| Synthetic DNA, four letters, a planted ACGT motif, a GC label | Lectures 4 to 6 | no |
|
| 63 |
-
| Four 2-D Gaussians, two moons, a checkerboard | Lectures 2, 3, 6, 7 | no |
|
| 64 |
-
| Four-token phrases such as `red circle moves left` | Lecture 6 | no |
|
| 65 |
-
| MNIST | Lectures 2, 3, with `--dataset mnist` | yes |
|
| 66 |
-
| ESM-2 residue embeddings | Lecture 3, with `--dataset esm2` | yes |
|
| 67 |
-
|
| 68 |
-
We generate the DNA ourselves, and we keep the two guidance objectives simple enough to compute in a
|
| 69 |
-
line, so a training run finishes while you watch it. As you can see from the samples a run prints,
|
| 70 |
-
you can also check by eye whether the sampler recovered the motif we planted.
|
| 71 |
-
|
| 72 |
-
## The shared flags
|
| 73 |
-
|
| 74 |
-
`--steps`, `--batch-size`, `--lr`, `--width`, `--length`, `--samples`, `--sample-steps`, `--seed`,
|
| 75 |
-
`--device`, `--out`, `--quiet`. You can learn them once and use them everywhere. Add `--out <dir>`
|
| 76 |
-
and you get `checkpoint.pt`, `config.json`, `report.json`, `losses.json` and `samples.txt`.
|
| 77 |
-
|
| 78 |
-
We set the defaults to a short budget, enough to watch a method work in well under a minute. When
|
| 79 |
-
you want a method to look its best, try `--steps 1500 --width 128`.
|
| 80 |
-
|
| 81 |
-
## Tests
|
| 82 |
-
|
| 83 |
-
```bash
|
| 84 |
-
python -m pytest tests -q
|
| 85 |
-
```
|
| 86 |
-
|
| 87 |
-
The suite checks the numbers the notes work out by hand, among them the posterior average of the
|
| 88 |
-
reverse rates on one DNA base, the Sinkhorn iterates at `epsilon = 2`, the cocycle and Jacobian of
|
| 89 |
-
the Gaussian flow, and the conditional velocity of the mixture path. Run it after any edit and you
|
| 90 |
-
can see at once whether a number in the code still agrees with the number in the book.
|
| 91 |
-
|
| 92 |
-
## License
|
| 93 |
-
|
| 94 |
-
[MIT](LICENSE). ESM-2 weights come from their original repository under its own terms.
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|