Update train.py

train.py

@@ -1,191 +1,378 @@
+"""Training - MODIFIED with conversation data"""
+
 import tensorflow as tf
 from tensorflow import keras
 import numpy as np
 import os
 import json
+
 from model import VedaProgrammingLLM
 from tokenizer import VedaTokenizer
+from config import MODEL_DIR, VOCAB_SIZE, MAX_LENGTH, BATCH_SIZE
+
+# ADDED: Conversation training data
+TRAINING_DATA = '''
+<USER> Hello!
+<ASSISTANT> Hello! Welcome to Veda Programming Assistant! I'm here to help you with coding, explain concepts, or answer any programming questions. What would you like to learn today?
+
+<USER> What is Python?
+<ASSISTANT> Python is a high-level, interpreted programming language created by Guido van Rossum in 1991. It's known for its simple, readable syntax that emphasizes code readability.
+
+Key features:
+- Easy to learn and read
+- Dynamic typing
+- Large standard library
+- Great for beginners and experts alike
+
+Python is used for web development, data science, AI, automation, and much more!
+
+<USER> How do I print hello world?
+<ASSISTANT> To print "Hello, World!" in Python, use the print() function:
+
+<CODE>
+print("Hello, World!")
+<ENDCODE>
+
+When you run this, it displays: Hello, World!
+
+The print() function outputs text to the console. You can print strings, numbers, and variables!
+
+<USER> Write a function to add two numbers
+<ASSISTANT> Here's a simple function to add two numbers:
+
+<CODE>
+def add(a, b):
+    return a + b
+
+# Example usage
+result = add(5, 3)
+print(result)  # Output: 8
+<ENDCODE>
+
+This function takes two parameters and returns their sum.
+
+<USER> Explain what a function is
+<ASSISTANT> A function is a reusable block of code that performs a specific task. Functions help organize code, avoid repetition, and make programs easier to understand.
+
+Here's the basic structure:
+
+<CODE>
+def function_name(parameters):
+    # Code block
+    return result
+<ENDCODE>
+
+Parts of a function:
+- def: keyword to define a function
+- function_name: what you call your function
+- parameters: input values (optional)
+- return: sends back a result (optional)
+
+Example:
+
+<CODE>
+def greet(name):
+    message = f"Hello, {name}!"
+    return message
+
+print(greet("Alice"))  # Output: Hello, Alice!
+<ENDCODE>
+
+<USER> What is a list?
+<ASSISTANT> A list is an ordered, mutable collection in Python that can hold items of any type.
+
+Creating lists:
+
+<CODE>
+# Empty list
+my_list = []
+
+# List with items
+numbers = [1, 2, 3, 4, 5]
+fruits = ["apple", "banana", "cherry"]
+mixed = [1, "hello", 3.14, True]
+<ENDCODE>
+
+Common operations:
+
+<CODE>
+fruits = ["apple", "banana"]
+
+# Add items
+fruits.append("cherry")     # Add to end
+fruits.insert(0, "apricot") # Insert at position
 
-SAMPLE_CODE = '''
-def hello_world():
-    print("Hello, World!")
-    return True
+# Access items
+print(fruits[0])   # First item
+print(fruits[-1])  # Last item
 
+# Remove items
+fruits.remove("banana")  # Remove by value
+fruits.pop()             # Remove last item
+
+# Length
+print(len(fruits))
+<ENDCODE>
+
+<USER> Write a fibonacci function
+<ASSISTANT> Here's a Fibonacci function with two approaches:
+
+Recursive approach:
+
+<CODE>
 def fibonacci(n):
-    if n <= 1:
-        return n
+    if n <= 0:
+        return 0
+    if n == 1:
+        return 1
     return fibonacci(n-1) + fibonacci(n-2)
 
+# Print first 10 numbers
+for i in range(10):
+    print(fibonacci(i), end=" ")
+# Output: 0 1 1 2 3 5 8 13 21 34
+<ENDCODE>
+
+Iterative approach (more efficient):
+
+<CODE>
+def fibonacci_iterative(n):
+    if n <= 0:
+        return 0
+    if n == 1:
+        return 1
+    
+    a, b = 0, 1
+    for _ in range(2, n + 1):
+        a, b = b, a + b
+    return b
+<ENDCODE>
+
+The Fibonacci sequence: each number is the sum of the two before it!
+
+<USER> What is recursion?
+<ASSISTANT> Recursion is when a function calls itself to solve a problem. It breaks complex problems into smaller, similar sub-problems.
+
+Key components:
+1. Base case - stops the recursion
+2. Recursive case - function calls itself
+
+Example - countdown:
+
+<CODE>
+def countdown(n):
+    if n <= 0:        # Base case
+        print("Done!")
+        return
+    print(n)
+    countdown(n - 1)  # Recursive call
+
+countdown(5)
+# Output: 5, 4, 3, 2, 1, Done!
+<ENDCODE>
+
+Another example - factorial:
+
+<CODE>
 def factorial(n):
-    if n == 0:
+    if n <= 1:        # Base case
         return 1
-    return n * factorial(n-1)
+    return n * factorial(n - 1)  # Recursive call
 
-class Calculator:
-    def __init__(self):
-        self.result = 0
-    
-    def add(self, a, b):
-        return a + b
-    
-    def subtract(self, a, b):
-        return a - b
-    
-    def multiply(self, a, b):
-        return a * b
+print(factorial(5))  # Output: 120
+<ENDCODE>
+
+Warning: Always have a base case to avoid infinite recursion!
 
+<USER> Write bubble sort
+<ASSISTANT> Here's the Bubble Sort algorithm:
+
+<CODE>
 def bubble_sort(arr):
     n = len(arr)
+    
     for i in range(n):
-        for j in range(0, n-i-1):
-            if arr[j] > arr[j+1]:
-                arr[j], arr[j+1] = arr[j+1], arr[j]
+        swapped = False
+        
+        for j in range(0, n - i - 1):
+            if arr[j] > arr[j + 1]:
+                arr[j], arr[j + 1] = arr[j + 1], arr[j]
+                swapped = True
+        
+        if not swapped:
+            break
+    
     return arr
 
+# Example
+numbers = [64, 34, 25, 12, 22, 11, 90]
+print("Original:", numbers)
+print("Sorted:", bubble_sort(numbers.copy()))
+<ENDCODE>
+
+Output:
+Original: [64, 34, 25, 12, 22, 11, 90]
+Sorted: [11, 12, 22, 25, 34, 64, 90]
+
+How it works: repeatedly compares adjacent elements and swaps them if in wrong order. Largest elements "bubble up" to the end.
+
+Time complexity: O(n²)
+
+<USER> Write binary search
+<ASSISTANT> Here's the Binary Search algorithm:
+
+<CODE>
 def binary_search(arr, target):
-    left, right = 0, len(arr) - 1
+    left = 0
+    right = len(arr) - 1
+    
     while left <= right:
         mid = (left + right) // 2
+        
         if arr[mid] == target:
             return mid
         elif arr[mid] < target:
             left = mid + 1
         else:
             right = mid - 1
-    return -1
+    
+    return -1  # Not found
 
-def is_palindrome(s):
-    s = s.lower()
-    return s == s[::-1]
+# Example
+numbers = [1, 3, 5, 7, 9, 11, 13, 15]
+print(binary_search(numbers, 7))   # Output: 3
+print(binary_search(numbers, 6))   # Output: -1
+<ENDCODE>
 
-def sum_list(numbers):
-    total = 0
-    for num in numbers:
-        total += num
-    return total
+Important: The array must be sorted!
 
-def find_max(arr):
-    if not arr:
-        return None
-    max_val = arr[0]
-    for val in arr:
-        if val > max_val:
-            max_val = val
-    return max_val
-'''
-
-class VedaTrainer:
-    """Trainer for Veda Programming LLM"""
-    
-    def __init__(self, data_path: str = "programming.txt", 
-                 vocab_size: int = 3000, max_length: int = 128, batch_size: int = 8):
-        self.data_path = data_path
-        self.vocab_size = vocab_size
-        self.max_length = max_length
-        self.batch_size = batch_size
-        self.tokenizer = VedaTokenizer(vocab_size=vocab_size)
-        self.model = None
-    
-    def load_data(self):
-        """Load training data"""
-        if not os.path.exists(self.data_path):
-            with open(self.data_path, 'w') as f:
-                f.write(SAMPLE_CODE)
-        
-        with open(self.data_path, 'r', encoding='utf-8') as f:
-            content = f.read()
-        
-        samples = [s.strip() for s in content.split('\n\n') if s.strip()]
-        print(f"Loaded {len(samples)} samples")
-        return samples
-    
-    def prepare_dataset(self, samples):
-        """Prepare TensorFlow dataset"""
-        self.tokenizer.fit(samples)
-        
-        all_tokens = []
-        for sample in samples:
-            tokens = self.tokenizer.encode(sample)
-            all_tokens.extend(tokens)
-        
-        sequences = []
-        step = max(1, self.max_length // 4)
-        for i in range(0, len(all_tokens) - self.max_length, step):
-            seq = all_tokens[i:i + self.max_length + 1]
-            if len(seq) == self.max_length + 1:
-                sequences.append(seq)
-        
-        if len(sequences) < 2:
-            for sample in samples:
-                tokens = self.tokenizer.encode(sample, max_length=self.max_length + 1)
-                sequences.append(tokens)
-        
-        sequences = np.array(sequences[:100])  # Limit for memory
-        X = sequences[:, :-1]
-        y = sequences[:, 1:]
-        
-        dataset = tf.data.Dataset.from_tensor_slices((X, y))
-        dataset = dataset.shuffle(100).batch(self.batch_size).prefetch(1)
-        
-        print(f"Created {len(sequences)} sequences")
-        return dataset
-    
-    def build_model(self):
-        """Build the model"""
-        self.model = VedaProgrammingLLM(
-            vocab_size=self.tokenizer.vocabulary_size,
-            max_length=self.max_length,
-            d_model=128,
-            num_heads=4,
-            num_layers=2,
-            ff_dim=256
-        )
-        
-        self.model.compile(
-            optimizer=keras.optimizers.Adam(1e-4),
-            loss=keras.losses.SparseCategoricalCrossentropy(from_logits=True),
-            metrics=['accuracy']
-        )
-        
-        # Build
-        dummy = tf.zeros((1, self.max_length), dtype=tf.int32)
-        self.model(dummy)
-        self.model.summary()
-        return self.model
+Time complexity: O(log n) - much faster than linear search!
+
+<USER> What is a class?
+<ASSISTANT> A class is a blueprint for creating objects. It bundles data (attributes) and functions (methods) together.
+
+Basic class structure:
+
+<CODE>
+class Dog:
+    def __init__(self, name, age):
+        self.name = name
+        self.age = age
     
-    def train(self, epochs: int = 5, save_path: str = "veda_model"):
-        """Train the model"""
-        samples = self.load_data()
-        dataset = self.prepare_dataset(samples)
-        self.build_model()
-        
-        os.makedirs(save_path, exist_ok=True)
-        
-        history = self.model.fit(dataset, epochs=epochs, verbose=1)
-        
-        # Save weights only (more reliable)
-        self.model.save_weights(os.path.join(save_path, "weights.h5"))
-        self.tokenizer.save(os.path.join(save_path, "tokenizer.json"))
-        
-        config = self.model.get_config()
-        with open(os.path.join(save_path, "config.json"), 'w') as f:
-            json.dump(config, f)
-        
-        print(f"Model saved to {save_path}")
-        return history
+    def bark(self):
+        return f"{self.name} says Woof!"
     
-    def generate(self, prompt: str, max_tokens: int = 50, temperature: float = 0.8):
-        """Generate code"""
-        if self.model is None:
-            raise ValueError("Model not loaded")
-        
-        tokens = self.tokenizer.encode(prompt)
-        generated = self.model.generate(tokens, max_tokens, temperature)
-        return self.tokenizer.decode(generated)
+    def get_info(self):
+        return f"{self.name} is {self.age} years old"
+
+# Create objects
+my_dog = Dog("Buddy", 3)
+print(my_dog.bark())      # Buddy says Woof!
+print(my_dog.get_info())  # Buddy is 3 years old
+<ENDCODE>
+
+Key concepts:
+- __init__: constructor, runs when object is created
+- self: refers to the current object
+- Attributes: variables inside the class
+- Methods: functions inside the class
+
+<USER> How do I read a file?
+<ASSISTANT> Here's how to read files in Python:
+
+Basic file reading:
+
+<CODE>
+# Read entire file
+with open("filename.txt", "r") as file:
+    content = file.read()
+    print(content)
+
+# Read line by line
+with open("filename.txt", "r") as file:
+    for line in file:
+        print(line.strip())
+
+# Read all lines into list
+with open("filename.txt", "r") as file:
+    lines = file.readlines()
+<ENDCODE>
+
+With error handling:
+
+<CODE>
+def read_file_safely(filename):
+    try:
+        with open(filename, "r") as file:
+            return file.read()
+    except FileNotFoundError:
+        print(f"File '{filename}' not found")
+        return None
+
+content = read_file_safely("myfile.txt")
+<ENDCODE>
+
+The 'with' statement automatically closes the file when done - always recommended!
+
+<USER> How do I handle errors?
+<ASSISTANT> Python uses try-except blocks for error handling:
+
+Basic try-except:
+
+<CODE>
+try:
+    result = 10 / 0
+except ZeroDivisionError:
+    print("Cannot divide by zero!")
+<ENDCODE>
+
+Multiple exceptions:
+
+<CODE>
+try:
+    number = int(input("Enter number: "))
+    result = 10 / number
+except ValueError:
+    print("That's not a valid number!")
+except ZeroDivisionError:
+    print("Cannot divide by zero!")
+except Exception as e:
+    print(f"Error: {e}")
+<ENDCODE>
+
+Complete structure:
+
+<CODE>
+try:
+    result = risky_operation()
+except SomeError:
+    print("Handle error")
+else:
+    print("Success!")  # Runs if no error
+finally:
+    print("Always runs")  # Cleanup code
+<ENDCODE>
+
+Common exceptions:
+- ValueError: wrong value
+- TypeError: wrong type
+- IndexError: index out of range
+- KeyError: dictionary key not found
+- FileNotFoundError: file doesn't exist
+
+<USER> Thanks!
+<ASSISTANT> You're welcome! I'm glad I could help. Remember:
+
+- Practice coding every day
+- Build small projects
+- Don't fear errors - they help you learn
+- Ask questions whenever stuck
+
+Feel free to ask me anything anytime! Happy coding! 🐍
 
+<USER> debug this: def add(a b): return a + b
+<ASSISTANT> I found the issue! You're missing a comma between parameters.
 
-if __name__ == "__main__":
-    trainer = VedaTrainer()
-    trainer.train(epochs=10)
-    print("\nTest generation:")
-    print(trainer.generate("def calculate"))
+Your code:
+```python
+def add(a b): return a + b  # ❌ Missing comma