stellaray777/1000s-websites

DeepSeek Coder 6.7B - Website Design Fine-tuned

A fine-tuned version of DeepSeek Coder 6.7B specifically trained to generate website designs based on design specifications and requirements.

Model Description

This model is a LoRA fine-tuned version of deepseek-ai/deepseek-coder-6.7b-base trained on a dataset of website designs. The model has been specialized to understand design requirements (industry, tone, layout, etc.) and generate appropriate HTML/CSS/JavaScript implementations for brand-specific websites.

• Base Model: deepseek-ai/deepseek-coder-6.7b-base
• Fine-tuning Method: LoRA (Low-Rank Adaptation)
• Training Dataset: stellaray777/1000s-websites
• Quantization: 4-bit (NF4) with BitsAndBytes

Training Details

Training Configuration

• Epochs: 3
• Batch Size: 1 (gradient accumulation: 4)
• Learning Rate: 2e-4
• LoRA Configuration:
  • r: 8
  • alpha: 16
  • target_modules: ["q_proj", "v_proj"]
  • lora_dropout: 0.05
• Max Sequence Length: 2048 tokens
• Optimizer: paged_adamw_8bit
• Learning Rate Scheduler: cosine
• Warmup Steps: 50

Training Infrastructure

• Quantization: 4-bit NF4 quantization for memory efficiency
• Gradient Checkpointing: Enabled
• Mixed Precision: FP16

Intended Use

This model is designed for:

• Generating website designs based on design specifications
• Creating HTML/CSS/JavaScript code for brand-specific websites
• Understanding design requirements (industry, tone, layout, photo usage, motion) and translating them to code
• Assisting front-end developers in rapid website prototyping

How to Use

Installation

pip install transformers torch peft bitsandbytes accelerate

Basic Usage

from transformers import AutoModelForCausalLM, AutoTokenizer
from peft import PeftModel
import torch

# Load base model and tokenizer
base_model = "deepseek-ai/deepseek-coder-6.7b-base"
model = AutoModelForCausalLM.from_pretrained(
    base_model,
    load_in_4bit=True,
    device_map="auto",
    trust_remote_code=True
)
tokenizer = AutoTokenizer.from_pretrained(base_model)

# Load LoRA weights
model = PeftModel.from_pretrained(model, "stellaray777/1000s-websites")

# Prepare input
messages = [
    {
        "role": "system",
        "content": "You are a senior creative front-end engineer who designs brand-specific websites."
    },
    {
        "role": "user",
        "content": "Industry: Healthcare\nTone: Professional, Trustworthy\nPage type: Landing page\nLayout: Grid-based\nPhoto usage: Medium\nTask: Design the website based on the provided HTML structure and styling."
    }
]

# Generate response
input_text = tokenizer.apply_chat_template(messages, tokenize=False, add_generation_prompt=True)
inputs = tokenizer(input_text, return_tensors="pt").to(model.device)

with torch.no_grad():
    outputs = model.generate(**inputs, max_new_tokens=2048, temperature=0.7)

response = tokenizer.decode(outputs[0], skip_special_tokens=True)
print(response)

Interactive Chatbot

Use the provided test script:

# From Hugging Face Hub
python src/test_trained_model.py

# From local cloned model
python src/test_trained_model.py --local

Limitations

• The model is fine-tuned on a specific dataset of website designs and may not generalize well to all design styles
• Generated code may require manual review and adjustments
• The model may not always produce production-ready code and should be used as a starting point
• Performance depends on the quality and specificity of the design requirements provided

Model Card Contact

For questions, issues, or contributions, please refer to the main project repository.

Citation

If you use this model, please cite:

@misc{deepseek-coder-website-design,
  title={DeepSeek Coder 6.7B - Website Design Fine-tuned},
  author={Stellaray777},
  year={2024},
  publisher={Hugging Face},
  howpublished={\url{https://huggingface.co/stellaray777/1000s-websites}}
}