Update README.md
Browse files
README.md
CHANGED
|
@@ -1,10 +1,77 @@
|
|
| 1 |
---
|
| 2 |
-
license:
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 3 |
---
|
| 4 |
|
| 5 |
-
#
|
| 6 |
-
this is text 2 timage model based on ViT trained on 1M timage text pairs. backbone is T5 Encoder and Sana audio tokenizer
|
| 7 |
-
for more information: https://github.com/firdavsus/Text2Image
|
| 8 |
|
| 9 |
-
#
|
| 10 |
-
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
---
|
| 2 |
+
license: apache-2.0
|
| 3 |
+
task_categories:
|
| 4 |
+
- text-to-image
|
| 5 |
+
tags:
|
| 6 |
+
- conditional-generation
|
| 7 |
+
- diffusion-models
|
| 8 |
+
- generative-art
|
| 9 |
+
- pytorch
|
| 10 |
+
- text2image
|
| 11 |
+
- flow-matching
|
| 12 |
---
|
| 13 |
|
| 14 |
+
# Text2Image Model Card
|
|
|
|
|
|
|
| 15 |
|
| 16 |
+
## Model Description
|
| 17 |
+
|
| 18 |
+
`firdavsus/text2Image` is a generative text-to-image foundation pipeline built and trained using the custom codebase templates available in the companion [firdavsus/Text2Image GitHub repository](https://github.com/firdavsus/Text2Image).
|
| 19 |
+
|
| 20 |
+
The framework establishes a cross-attention bridging mechanism between a conditioned textual encoder (e.g., CLIP-style or T5-style transformers) and a spatial latent processor (such as a Diffusion Transformer (DiT) or standard UNet backbone). It is engineered to perform high-fidelity image synthesis from raw text prompts, prioritizing fast convergence and structured geometric layout handling.
|
| 21 |
+
|
| 22 |
+
### Model Features & Specifications
|
| 23 |
+
- **Task:** Text-to-Image Generation (Text-Conditional Image Synthesis)
|
| 24 |
+
- **Framework Native:** PyTorch
|
| 25 |
+
- **Core Components:** Text Conditioner / Prompt Encoder, Latent Spatial Generator, and an Autoencoder (VAE/VQ-VAE) for pixel-space reconstruction.
|
| 26 |
+
- **Optimizations:** Supports native attention scaling, FP16/BF16 mixed-precision training, and accelerated sample generation steps.
|
| 27 |
+
|
| 28 |
+
---
|
| 29 |
+
|
| 30 |
+
## Architectural Workflow
|
| 31 |
+
|
| 32 |
+
The model operates across standard latent spaces to lower resource overhead during generation loops:
|
| 33 |
+
|
| 34 |
+
1. **Text Encoding:** Input prompts are tokenized and mapped into deep dense contextual matrices via the text encoder.
|
| 35 |
+
2. **Latent Denoising / Flow Matching:** The core spatial backbone uses these text matrices via cross-attention layers to iteratively clean randomly initialized Gaussian noise blocks.
|
| 36 |
+
3. **Decoding:** The final structural latents are pushed through a pre-trained spatial decoder to output clean, high-resolution pixel-space images.
|
| 37 |
+
|
| 38 |
+
---
|
| 39 |
+
|
| 40 |
+
## Intended Uses & Limitations
|
| 41 |
+
|
| 42 |
+
### Target Applications
|
| 43 |
+
- **Generative Media Research:** Testing custom conditioning styles, guidance techniques (like Classifier-Free Guidance), or alternative sampling paths (e.g., DDIM, Flow Matching steps).
|
| 44 |
+
- **Localized / Domain Adaptation:** Fine-tuning on target asset styles, downstream icon/character datasets, or multi-lingual text descriptive pools.
|
| 45 |
+
|
| 46 |
+
### Limitations
|
| 47 |
+
- **Text Rendering:** Like many medium-scale generative vision layers, the pipeline may occasionally struggle to render pixel-perfect fine-grained typography or text strings inside the synthesized images.
|
| 48 |
+
- **Anatomy / Complex Composition:** Highly crowded compositions or intricate structural geometries (like multi-finger hand layouts) might exhibit synthesis anomalies depending on the sampling steps and guidance parameters chosen during inference.
|
| 49 |
+
|
| 50 |
+
---
|
| 51 |
+
|
| 52 |
+
## Quickstart Inference
|
| 53 |
+
|
| 54 |
+
You can run text-conditional image generation loops using the model evaluation scripts available in the primary GitHub repository.
|
| 55 |
+
|
| 56 |
+
```python
|
| 57 |
+
import torch
|
| 58 |
+
from model import Text2ImagePipeline # Imported from your firdavsus/Text2Image repository
|
| 59 |
+
|
| 60 |
+
# 1. Initialize the inference pipeline on target accelerator hardware
|
| 61 |
+
device = "cuda" if torch.cuda.is_available() else "cpu"
|
| 62 |
+
pipeline = Text2ImagePipeline.from_pretrained(
|
| 63 |
+
"firdavsus/text2Image",
|
| 64 |
+
torch_dtype=torch.float16
|
| 65 |
+
).to(device)
|
| 66 |
+
|
| 67 |
+
# 2. Run the generation loop
|
| 68 |
+
prompt = "A futuristic cyberpunk skyline of Tashkent with neon lights, digital art style"
|
| 69 |
+
generated_image = pipeline(
|
| 70 |
+
prompt=prompt,
|
| 71 |
+
num_inference_steps=30,
|
| 72 |
+
guidance_scale=7.5
|
| 73 |
+
)
|
| 74 |
+
|
| 75 |
+
# 3. Save the synthesized output to disk
|
| 76 |
+
generated_image.save("output_skyline.png")
|
| 77 |
+
print("Image successfully synthesized and saved to disk.")
|