TopAI-1/Duchifat-2-Instruct

🚀 Duchifat-V2-Instruct (דוכיפת 2) | Official Model Card

📝 Executive Summary

Duchifat-V2-Instruct is a fine-tuned, instruction-following version of the Duchifat-V2 architecture (136M parameters). Developed by TopAI, this model is specifically optimized for creative content generation, bilingual dialogue, and task-oriented text processing.

While the base model provides a massive knowledge foundation from 3.27 billion tokens, the Instruct version has undergone targeted fine-tuning to transform it from a "text completer" into a creative writer capable of following complex prompts with a unique, human-like voice.

---

🏗️ Technical Specifications

Component	Specification	Description
Parameters	136 Million	Optimized for edge deployment and real-time inference.
Architecture	Decoder-only Transformer	Enhanced for causal reasoning and fluency.
Layers / Heads	12 / 12	Deep representation for nuanced semantics.
Context Window	1024 Tokens	Supports creative long-form generation.
Tokenizer	DictaLM 2.0	High-efficiency sub-word tokenization for Hebrew/English.
Training Phase	Post-5 Epoch Instruct	Refined for instruction-following & EOS consistency.

---

🎨 Model Capabilities & "The Creative Writer"

Unlike standard small-scale models, Duchifat-V2-Instruct exhibits "Creative Personality." It excels at:

• Narrative Writing: Crafting stories and monologues with emotional depth.
• Instruction Following: Responding to specific system prompts and user constraints.
• Bilingual Versatility: Seamlessly switching between Hebrew and English based on the prompt's linguistic context.
• Marketing & Copywriting: Generating slogans, blog posts, and creative ads.

Note: Due to its training on the C4 corpus, the model retains a vast "general knowledge" base, allowing it to act as a sophisticated creative partner rather than a purely technical agent.

---

📊 Training Infrastructure

• Dataset: Curated C4 (3.27B Tokens) - 50% Hebrew, 50% English.
• Fine-Tuning: Instruction-tuning on high-quality conversational and creative datasets.
• Optimization: AdamW with a focus on preserving the pre-trained knowledge (Knowledge Retention).

---

💻 Implementation & Inference

To utilize the Instruct capabilities, use the following structure:

import torch
from transformers import AutoModelForCausalLM, AutoTokenizer

MODEL_ID = "TopAI-1/Duchifat-2-Instruct"

def run_duchifat_chat():
    print("--- Loading Duchifat-2 (Post 5-Epoch Instruct Training) ---")
    
    tokenizer = AutoTokenizer.from_pretrained(MODEL_ID, trust_remote_code=True)
    model = AutoModelForCausalLM.from_pretrained(
        MODEL_ID, 
        trust_remote_code=True, 
        torch_dtype=torch.bfloat16,
        device_map="auto"
    )
    model.eval()
    model.config.use_cache = False 

    chat_history = []
    
    print("--- Model Ready! ---")
    
    while True:
        user_input = input("\nהכנס הוראה (או 'יציאה'): ")
        if user_input.lower() in ["exit", "quit", "יציאה"]:
            break
            
        # Add current instruction to memory
        chat_history.append(f"Instruction: {user_input}")
        
        # Build prompt with history
        full_prompt = "\n".join(chat_history) + "\nContent:"
        
        inputs = tokenizer(full_prompt, return_tensors="pt").to(model.device)
        
        # Context Window Protection (Max 1024 tokens)
        if inputs.input_ids.shape[1] > 850:
            chat_history = chat_history[2:] # Trim oldest turn
            full_prompt = "\n".join(chat_history) + "\nContent:"
            inputs = tokenizer(full_prompt, return_tensors="pt").to(model.device)

        with torch.no_grad():
            output_tokens = model.generate(
                input_ids=inputs.input_ids,
                attention_mask=inputs.attention_mask,
                max_new_tokens=300, # Increased for creative writing
                do_sample=True,
                temperature=0.75, 
                top_p=0.9,
                repetition_penalty=1.15,
                pad_token_id=tokenizer.eos_token_id,
                use_cache=False
            )
        
        full_text = tokenizer.decode(output_tokens[0], skip_special_tokens=True)
        
        # Extract only the latest response
        parts = full_text.split("Content:")
        answer = parts[-1].strip()
        
        # Save response to history for context
        chat_history.append(f"Content: {answer}")
        
        print(f"\nדוכיפת-2: {answer}")

if __name__ == "__main__":
    run_duchifat_chat()