Upload folder using huggingface_hub

.gitattributes

@@ -2,34 +2,26 @@
 *.arrow filter=lfs diff=lfs merge=lfs -text
 *.bin filter=lfs diff=lfs merge=lfs -text
 *.bz2 filter=lfs diff=lfs merge=lfs -text
-*.ckpt filter=lfs diff=lfs merge=lfs -text
 *.ftz filter=lfs diff=lfs merge=lfs -text
 *.gz filter=lfs diff=lfs merge=lfs -text
 *.h5 filter=lfs diff=lfs merge=lfs -text
 *.joblib filter=lfs diff=lfs merge=lfs -text
 *.lfs.* filter=lfs diff=lfs merge=lfs -text
-*.mlmodel filter=lfs diff=lfs merge=lfs -text
 *.model filter=lfs diff=lfs merge=lfs -text
 *.msgpack filter=lfs diff=lfs merge=lfs -text
-*.npy filter=lfs diff=lfs merge=lfs -text
-*.npz filter=lfs diff=lfs merge=lfs -text
 *.onnx filter=lfs diff=lfs merge=lfs -text
 *.ot filter=lfs diff=lfs merge=lfs -text
 *.parquet filter=lfs diff=lfs merge=lfs -text
 *.pb filter=lfs diff=lfs merge=lfs -text
-*.pickle filter=lfs diff=lfs merge=lfs -text
-*.pkl filter=lfs diff=lfs merge=lfs -text
 *.pt filter=lfs diff=lfs merge=lfs -text
 *.pth filter=lfs diff=lfs merge=lfs -text
 *.rar filter=lfs diff=lfs merge=lfs -text
-*.safetensors filter=lfs diff=lfs merge=lfs -text
 saved_model/**/* filter=lfs diff=lfs merge=lfs -text
 *.tar.* filter=lfs diff=lfs merge=lfs -text
-*.tar filter=lfs diff=lfs merge=lfs -text
 *.tflite filter=lfs diff=lfs merge=lfs -text
 *.tgz filter=lfs diff=lfs merge=lfs -text
 *.wasm filter=lfs diff=lfs merge=lfs -text
 *.xz filter=lfs diff=lfs merge=lfs -text
 *.zip filter=lfs diff=lfs merge=lfs -text
-*.zst filter=lfs diff=lfs merge=lfs -text
+*.zstandard filter=lfs diff=lfs merge=lfs -text
 *tfevents* filter=lfs diff=lfs merge=lfs -text

README.md

@@ -0,0 +1,128 @@
+---
+tags:
+- pytorch
+license: mit
+---
+
+# min(DALL·E)
+
+[![Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/kuprel/min-dalle/blob/main/min_dalle.ipynb)
+[![Discord](https://img.shields.io/discord/823813159592001537?color=5865F2&logo=discord&logoColor=white)](https://discord.com/channels/823813159592001537/912729332311556136)
+
+**[GitHub](https://github.com/kuprel/min-dalle)**
+
+This is a fast, minimal port of Boris Dayma's [DALL·E Mini](https://github.com/borisdayma/dalle-mini) (with mega weights).  It has been stripped down for inference and converted to PyTorch.  The only third party dependencies are numpy, requests, pillow and torch.
+
+To generate a 4x4 grid of DALL·E Mega images it takes:
+- 89 sec with a T4 in Colab
+- 48 sec with a P100 in Colab
+- 13 sec with an A100 on Replicate
+
+Here's a more detailed breakdown of performance on an A100. Credit to [@technobird22](https://github.com/technobird22) and his [NeoGen](https://github.com/technobird22/NeoGen) discord bot for the graph.
+<br />
+<img src="https://github.com/kuprel/min-dalle/raw/main/performance.png" alt="min-dalle" width="450"/>
+<br />
+
+The flax model and code for converting it to torch can be found [here](https://github.com/kuprel/min-dalle-flax).
+
+## Install
+
+```bash
+$ pip install min-dalle
+```  
+
+## Usage
+
+Load the model parameters once and reuse the model to generate multiple images.
+
+```python
+from min_dalle import MinDalle
+
+model = MinDalle(
+    models_root='./pretrained',
+    dtype=torch.float32,
+    device='cuda',
+    is_mega=True, 
+    is_reusable=True
+)
+```
+
+The required models will be downloaded to `models_root` if they are not already there.  Set the `dtype` to `torch.float16` to save GPU memory.  If you have an Ampere architecture GPU you can use `torch.bfloat16`.  Set the `device` to either "cuda" or "cpu".  Once everything has finished initializing, call `generate_image` with some text as many times as you want.  Use a positive `seed` for reproducible results.  Higher values for `supercondition_factor` result in better agreement with the text but a narrower variety of generated images.  Every image token is sampled from the `top_k` most probable tokens.  The largest logit is subtracted from the logits to avoid infs.  The logits are then divided by the `temperature`.  If `is_seamless` is true, the image grid will be tiled in token space not pixel space.
+
+```python
+image = model.generate_image(
+    text='Nuclear explosion broccoli',
+    seed=-1,
+    grid_size=4,
+    is_seamless=False,
+    temperature=1,
+    top_k=256,
+    supercondition_factor=32,
+    is_verbose=False
+)
+
+display(image)
+```
+<img src="https://github.com/kuprel/min-dalle/raw/main/examples/nuclear_broccoli.jpg" alt="min-dalle" width="400"/>
+
+Credit to [@hardmaru](https://twitter.com/hardmaru) for the [example](https://twitter.com/hardmaru/status/1544354119527596034)
+
+
+### Saving Individual Images
+The images can also be generated as a `FloatTensor` in case you want to process them manually.
+
+```python
+images = model.generate_images(
+    text='Nuclear explosion broccoli',
+    seed=-1,
+    grid_size=3,
+    is_seamless=False,
+    temperature=1,
+    top_k=256,
+    supercondition_factor=16,
+    is_verbose=False
+)
+```
+
+To get an image into PIL format you will have to first move the images to the CPU and convert the tensor to a numpy array.
+```python
+images = images.to('cpu').numpy()
+```
+Then image $i$ can be coverted to a PIL.Image and saved
+```python
+image = Image.fromarray(images[i])
+image.save('image_{}.png'.format(i))
+```
+
+### Progressive Outputs
+
+If the model is being used interactively (e.g. in a notebook) `generate_image_stream` can be used to generate a stream of images as the model is decoding.  The detokenizer adds a slight delay for each image.  Set `progressive_outputs` to `True` to enable this.  An example is implemented in the colab.
+
+```python
+image_stream = model.generate_image_stream(
+    text='Dali painting of WALL·E',
+    seed=-1,
+    grid_size=3,
+    progressive_outputs=True,
+    is_seamless=False,
+    temperature=1,
+    top_k=256,
+    supercondition_factor=16,
+    is_verbose=False
+)
+
+for image in image_stream:
+    display(image)
+```
+<img src="https://github.com/kuprel/min-dalle/raw/main/examples/dali_walle_animated.gif" alt="min-dalle" width="300"/>
+
+### Command Line
+
+Use `image_from_text.py` to generate images from the command line.
+
+```bash
+$ python image_from_text.py --text='artificial intelligence' --no-mega
+```
+<img src="https://github.com/kuprel/min-dalle/raw/main/examples/artificial_intelligence.jpg" alt="min-dalle" width="200"/>
+
+**[❤️ Sponsor](https://github.com/sponsors/kuprel)**

config.json

@@ -0,0 +1,14 @@
+{
+  "d_model": 2048,
+  "decoder_attention_heads": 32,
+  "decoder_ffn_dim": 4096,
+  "decoder_layers": 24,
+  "decoder_start_token_id": 16384,
+  "encoder_attention_heads": 32,
+  "encoder_ffn_dim": 4096,
+  "encoder_layers": 24,
+  "encoder_vocab_size": 50272,
+  "image_length": 256,
+  "image_vocab_size": 16415,
+  "max_text_length": 64
+}

config_mini.json

@@ -0,0 +1,14 @@
+{
+  "d_model": 1024,
+  "decoder_attention_heads": 16,
+  "decoder_ffn_dim": 2730,
+  "decoder_layers": 12,
+  "decoder_start_token_id": 16384,
+  "encoder_attention_heads": 16,
+  "encoder_ffn_dim": 2730,
+  "encoder_layers": 12,
+  "encoder_vocab_size": 50264,
+  "image_length": 256,
+  "image_vocab_size": 16384,
+  "max_text_length": 64
+}

dalle_bart_mega/decoder.pt

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:fc94a4dff25b97575239eb21e1ae3d988beff95470fec9903d5dcad521ab9efa
+size 2955054938

dalle_bart_mega/encoder.pt

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:76e7797b19625122e21138556e06b78cc84686ed5f5e5c42aa53c81ffb2f4bb8
+size 2220079602

decoder.pt

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:fc94a4dff25b97575239eb21e1ae3d988beff95470fec9903d5dcad521ab9efa
+size 2955054938

decoder_mini.pt

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:f8343c2bf982cb60d463417facc4debeeaabc9f238a1f7115ac5bdf778ea3901
+size 470515674

decoder_v26.pt

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:78ce1ebd38848e6233d7fe6088d56a6754f8eeb400926854db694fbe633f6ae1
+size 2955054938

detoker.pt

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:7e1ea14cb2a45661d4599d897779673ff93e2ad4202e311edc0c837e0d86933c
+size 186899831

encoder.pt

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:76e7797b19625122e21138556e06b78cc84686ed5f5e5c42aa53c81ffb2f4bb8
+size 2220079602

encoder_mini.pt

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:158e1ff118b6e85c7b03577178758f94a24b6e1625e8f8ee6415fbde020b938b
+size 405201458

encoder_v26.pt

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:b673011bce7410da947e97a9725369f677ff79a1a0a61611a8fc818dde64ed22
+size 2220079602

vqgan/detoker.pt

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:7e1ea14cb2a45661d4599d897779673ff93e2ad4202e311edc0c837e0d86933c
+size 186899831