firdavsus commited on
Commit
781a9ea
·
verified ·
1 Parent(s): cf6006e

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +73 -6
README.md CHANGED
@@ -1,10 +1,77 @@
1
  ---
2
- license: mit
 
 
 
 
 
 
 
 
 
3
  ---
4
 
5
- # Text 2 Image
6
- this is text 2 timage model based on ViT trained on 1M timage text pairs. backbone is T5 Encoder and Sana audio tokenizer
7
- for more information: https://github.com/firdavsus/Text2Image
8
 
9
- # VAE-VQ
10
- this model takes 128x128 image and compressed it to 16x16 codebook with decent quality reconstrcution loss 0.07 (22 epoch 1M images)!
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
  ---
2
+ license: apache-2.0
3
+ task_categories:
4
+ - text-to-image
5
+ tags:
6
+ - conditional-generation
7
+ - diffusion-models
8
+ - generative-art
9
+ - pytorch
10
+ - text2image
11
+ - flow-matching
12
  ---
13
 
14
+ # Text2Image Model Card
 
 
15
 
16
+ ## Model Description
17
+
18
+ `firdavsus/text2Image` is a generative text-to-image foundation pipeline built and trained using the custom codebase templates available in the companion [firdavsus/Text2Image GitHub repository](https://github.com/firdavsus/Text2Image).
19
+
20
+ The framework establishes a cross-attention bridging mechanism between a conditioned textual encoder (e.g., CLIP-style or T5-style transformers) and a spatial latent processor (such as a Diffusion Transformer (DiT) or standard UNet backbone). It is engineered to perform high-fidelity image synthesis from raw text prompts, prioritizing fast convergence and structured geometric layout handling.
21
+
22
+ ### Model Features & Specifications
23
+ - **Task:** Text-to-Image Generation (Text-Conditional Image Synthesis)
24
+ - **Framework Native:** PyTorch
25
+ - **Core Components:** Text Conditioner / Prompt Encoder, Latent Spatial Generator, and an Autoencoder (VAE/VQ-VAE) for pixel-space reconstruction.
26
+ - **Optimizations:** Supports native attention scaling, FP16/BF16 mixed-precision training, and accelerated sample generation steps.
27
+
28
+ ---
29
+
30
+ ## Architectural Workflow
31
+
32
+ The model operates across standard latent spaces to lower resource overhead during generation loops:
33
+
34
+ 1. **Text Encoding:** Input prompts are tokenized and mapped into deep dense contextual matrices via the text encoder.
35
+ 2. **Latent Denoising / Flow Matching:** The core spatial backbone uses these text matrices via cross-attention layers to iteratively clean randomly initialized Gaussian noise blocks.
36
+ 3. **Decoding:** The final structural latents are pushed through a pre-trained spatial decoder to output clean, high-resolution pixel-space images.
37
+
38
+ ---
39
+
40
+ ## Intended Uses & Limitations
41
+
42
+ ### Target Applications
43
+ - **Generative Media Research:** Testing custom conditioning styles, guidance techniques (like Classifier-Free Guidance), or alternative sampling paths (e.g., DDIM, Flow Matching steps).
44
+ - **Localized / Domain Adaptation:** Fine-tuning on target asset styles, downstream icon/character datasets, or multi-lingual text descriptive pools.
45
+
46
+ ### Limitations
47
+ - **Text Rendering:** Like many medium-scale generative vision layers, the pipeline may occasionally struggle to render pixel-perfect fine-grained typography or text strings inside the synthesized images.
48
+ - **Anatomy / Complex Composition:** Highly crowded compositions or intricate structural geometries (like multi-finger hand layouts) might exhibit synthesis anomalies depending on the sampling steps and guidance parameters chosen during inference.
49
+
50
+ ---
51
+
52
+ ## Quickstart Inference
53
+
54
+ You can run text-conditional image generation loops using the model evaluation scripts available in the primary GitHub repository.
55
+
56
+ ```python
57
+ import torch
58
+ from model import Text2ImagePipeline # Imported from your firdavsus/Text2Image repository
59
+
60
+ # 1. Initialize the inference pipeline on target accelerator hardware
61
+ device = "cuda" if torch.cuda.is_available() else "cpu"
62
+ pipeline = Text2ImagePipeline.from_pretrained(
63
+ "firdavsus/text2Image",
64
+ torch_dtype=torch.float16
65
+ ).to(device)
66
+
67
+ # 2. Run the generation loop
68
+ prompt = "A futuristic cyberpunk skyline of Tashkent with neon lights, digital art style"
69
+ generated_image = pipeline(
70
+ prompt=prompt,
71
+ num_inference_steps=30,
72
+ guidance_scale=7.5
73
+ )
74
+
75
+ # 3. Save the synthesized output to disk
76
+ generated_image.save("output_skyline.png")
77
+ print("Image successfully synthesized and saved to disk.")