Initial commit

.gitattributes

@@ -0,0 +1,4 @@
+*.bin filter=lfs diff=lfs merge=lfs -text
+*.pt filter=lfs diff=lfs merge=lfs -text
+*.pth filter=lfs diff=lfs merge=lfs -text
+*.safetensors filter=lfs diff=lfs merge=lfs -text

.gitignore

@@ -0,0 +1,2 @@
+.venv
+__pycache__

bitch.csv

@@ -0,0 +1,5 @@
+Company_Name,Location,Founder,Industry,Core_Focus,Mission,Vision,Team,Key_Offerings,Notable_Projects,Technology_Used,Project_Impacts,Community_Engagement
+Zeliang Codetech Private Limited,"Nagaland, India",Kangzang Zeliang,"IT, IoT, Software Development","Innovative technology solutions bridging hardware, software, and IoT",To address challenges within local communities and industries by integrating accessible technology solutions,"To empower businesses and civic organizations with smart solutions, improving operational efficiency and promoting technology-driven convenience","Composed of young, talented developers from the Naga community, committed to tech development",- IoT Solutions: Intelligent hardware-software integrations for smart applications,Smart Garbage Disposal Management System (SGDMS 1.0V): IoT-enabled waste management system with real-time tracking and notifications when bins are full,"IoT sensors, real-time data tracking, custom POS software, machine learning algorithms, data analytics, dashboards, and mobile app notifications","- SGDMS 1.0V: Improves public sanitation, reduces collection costs, and enhances environmental impact","Prioritizes local talent and supports the technical growth of its team, working closely with community organizations and fostering innovation in Northeast India"
+,,,,,,,,- Software Development: Full-stack services for web and mobile apps,Retail POS Systems: Custom POS solutions for small and medium businesses,,- POS Systems: Streamlines business operations and enhances efficiency,
+,,,,,,,,- POS Systems: Custom solutions for business needs,AI-Driven Predictive Analytics: ML-driven insights into customer trends and operational data,,- Predictive Analytics: Empowers businesses with data-driven insights,
+,,,,,,,,- AI & Machine Learning Applications: Predictive and data-driven solutions

cached_lm_GPT2Tokenizer_128_train_data.txt

@@ -0,0 +1 @@
+�]�.

csvs/company_name.csv

@@ -0,0 +1,13 @@
+Company_Name
+Zeliang codetech
+Zeliangcodetech
+zeliangcodetech
+zeliangCodetech
+zeliang-codetech
+zeliang codetech
+zeliang_code_tech
+zeliang-code-tech
+ZELIANG CODETECH
+ZELIANG-CODETECH
+ZELIANG-CODE-TECH
+ZeLiAnGCoDeTeCh

csvs/founder.csv

@@ -0,0 +1,12 @@
+Founder
+Kangzang Zeliang
+KangzangZeliang
+kangzang_zeliang
+kangzang-zeliang
+kangzang-zeliang
+kangzang-zeliang
+KangzangZeliang
+Kangzang Zeliang
+kAnGzAnG zElIaNg
+Kangzang
+Zeliang

csvs/industry.csv

@@ -0,0 +1 @@
+Industry

csvs/location.csv

@@ -0,0 +1,36 @@
+Location
+DimapurNagalandIndia
+dimapur_nagaland_india
+dimapur-nagaland-india
+DIMAPUR NAGALAND INDIA
+dimapur nagaland india
+DimapurNagalandIndia
+Dimapur Nagaland India
+dImApUr nAgAlAnD iNdIa
+Dimapur nagaland india
+Dimapur.Nagaland.India
+dImApUr_nAgAlAnD-iNdIa
+D1m4pUr N4g4l4nd 1nd14
+Diiimmaapuurr Nagggaallandd Iinddiiiaa
+im Nag Ind
+DIMAPUR NAGALAND INDIA
+Dimapur
+Nagaland 
+India
+Naga 
+Nagaland 
+Bank colony Supermarket
+BankColonySupermarket
+bank_colony_supermarket
+bank-colony-supermarket
+BANK COLONY SUPERMARKET
+BANK COLONY SUPERMARKET
+BankColonySupermarket
+Bank Colony Supermarket
+bAnK cOlOnY sUpErMaRkEt
+Bank colony Dimapur 
+Super market Dimapur 
+Bank colony
+Supermarket
+Ind
+NL

csvs/mission.csv

@@ -0,0 +1,3 @@
+Mission
+We believe in the power of the internet and  Internet of Things to transform our lives in a positive way We believe that the future is now  With the wide-spread application of artificial learning and machine learning  it's more so.
+We live a life filled with technological marvels that aid us ubiquitously

gpt2-custom/checkpoint-35/config.json

@@ -0,0 +1,39 @@
+{
+  "_name_or_path": "gpt2",
+  "activation_function": "gelu_new",
+  "architectures": [
+    "GPT2LMHeadModel"
+  ],
+  "attn_pdrop": 0.1,
+  "bos_token_id": 50256,
+  "embd_pdrop": 0.1,
+  "eos_token_id": 50256,
+  "initializer_range": 0.02,
+  "layer_norm_epsilon": 1e-05,
+  "model_type": "gpt2",
+  "n_ctx": 1024,
+  "n_embd": 768,
+  "n_head": 12,
+  "n_inner": null,
+  "n_layer": 12,
+  "n_positions": 1024,
+  "reorder_and_upcast_attn": false,
+  "resid_pdrop": 0.1,
+  "scale_attn_by_inverse_layer_idx": false,
+  "scale_attn_weights": true,
+  "summary_activation": null,
+  "summary_first_dropout": 0.1,
+  "summary_proj_to_labels": true,
+  "summary_type": "cls_index",
+  "summary_use_proj": true,
+  "task_specific_params": {
+    "text-generation": {
+      "do_sample": true,
+      "max_length": 50
+    }
+  },
+  "torch_dtype": "float32",
+  "transformers_version": "4.46.0",
+  "use_cache": true,
+  "vocab_size": 50257
+}

gpt2-custom/checkpoint-35/generation_config.json

@@ -0,0 +1,6 @@
+{
+  "_from_model_config": true,
+  "bos_token_id": 50256,
+  "eos_token_id": 50256,
+  "transformers_version": "4.46.0"
+}

gpt2-custom/checkpoint-35/model.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:fda0a5151c125b62530ce0a3bdc0c7ac15a31787ff084cc3a4bd87eb840a7cd5
+size 497774208

gpt2-custom/checkpoint-35/optimizer.pt

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:bbcc017831e14300ef4b71c319fc917bc3900c105e5e8b831eb8de574b4dd554
+size 995638202

gpt2-custom/checkpoint-35/rng_state.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:c98859a12d53c0b788e8d709c65c4d35bac9cee26017cc95fb42c6c969931e04
+size 13990

gpt2-custom/checkpoint-35/scheduler.pt

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:5d878f9de4609bd4fbbedb6ef1d1b6a282880aef389509b2fedeceab92b946e8
+size 1064

gpt2-custom/checkpoint-35/trainer_state.json

@@ -0,0 +1,32 @@
+{
+  "best_metric": null,
+  "best_model_checkpoint": null,
+  "epoch": 5.0,
+  "eval_steps": 500,
+  "global_step": 35,
+  "is_hyper_param_search": false,
+  "is_local_process_zero": true,
+  "is_world_process_zero": true,
+  "log_history": [],
+  "logging_steps": 500,
+  "max_steps": 35,
+  "num_input_tokens_seen": 0,
+  "num_train_epochs": 5,
+  "save_steps": 10000,
+  "stateful_callbacks": {
+    "TrainerControl": {
+      "args": {
+        "should_epoch_stop": false,
+        "should_evaluate": false,
+        "should_log": false,
+        "should_save": true,
+        "should_training_stop": true
+      },
+      "attributes": {}
+    }
+  },
+  "total_flos": 6532300800000.0,
+  "train_batch_size": 3,
+  "trial_name": null,
+  "trial_params": null
+}

gpt2-custom/checkpoint-35/training_args.bin

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:86e69812d76976eae1f510debcec68b20eed3acc4e6acd97c07e1f97ec59d2ab
+size 5176

gpt2-custom/checkpoint-60/config.json

@@ -0,0 +1,39 @@
+{
+  "_name_or_path": "gpt2",
+  "activation_function": "gelu_new",
+  "architectures": [
+    "GPT2LMHeadModel"
+  ],
+  "attn_pdrop": 0.1,
+  "bos_token_id": 50256,
+  "embd_pdrop": 0.1,
+  "eos_token_id": 50256,
+  "initializer_range": 0.02,
+  "layer_norm_epsilon": 1e-05,
+  "model_type": "gpt2",
+  "n_ctx": 1024,
+  "n_embd": 768,
+  "n_head": 12,
+  "n_inner": null,
+  "n_layer": 12,
+  "n_positions": 1024,
+  "reorder_and_upcast_attn": false,
+  "resid_pdrop": 0.1,
+  "scale_attn_by_inverse_layer_idx": false,
+  "scale_attn_weights": true,
+  "summary_activation": null,
+  "summary_first_dropout": 0.1,
+  "summary_proj_to_labels": true,
+  "summary_type": "cls_index",
+  "summary_use_proj": true,
+  "task_specific_params": {
+    "text-generation": {
+      "do_sample": true,
+      "max_length": 50
+    }
+  },
+  "torch_dtype": "float32",
+  "transformers_version": "4.46.0",
+  "use_cache": true,
+  "vocab_size": 50257
+}

gpt2-custom/checkpoint-60/generation_config.json

@@ -0,0 +1,6 @@
+{
+  "_from_model_config": true,
+  "bos_token_id": 50256,
+  "eos_token_id": 50256,
+  "transformers_version": "4.46.0"
+}

gpt2-custom/checkpoint-60/model.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:71d38e7bf6feb3aadca56675812363fde64a32e3033adc9791c40a58c415d3fa
+size 497774208

gpt2-custom/checkpoint-60/optimizer.pt

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:4d6618ece00a33e1bcb48c75ab9c7081362a6a675f9870c6aa7c602c249b91c9
+size 995638202

gpt2-custom/checkpoint-60/rng_state.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:fc44dde0c9d8726cdf4a901b295b7d49dd152982941d5d7b7aaf9e60f7d0cf4f
+size 13990

gpt2-custom/checkpoint-60/scheduler.pt

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:90979e82e94259c1d6f68c5ef23bb17c7019c0bbe3422cf6d46e9cdac106c874
+size 1064

gpt2-custom/checkpoint-60/trainer_state.json

@@ -0,0 +1,32 @@
+{
+  "best_metric": null,
+  "best_model_checkpoint": null,
+  "epoch": 5.0,
+  "eval_steps": 500,
+  "global_step": 60,
+  "is_hyper_param_search": false,
+  "is_local_process_zero": true,
+  "is_world_process_zero": true,
+  "log_history": [],
+  "logging_steps": 500,
+  "max_steps": 60,
+  "num_input_tokens_seen": 0,
+  "num_train_epochs": 5,
+  "save_steps": 10000,
+  "stateful_callbacks": {
+    "TrainerControl": {
+      "args": {
+        "should_epoch_stop": false,
+        "should_evaluate": false,
+        "should_log": false,
+        "should_save": true,
+        "should_training_stop": true
+      },
+      "attributes": {}
+    }
+  },
+  "total_flos": 11431526400000.0,
+  "train_batch_size": 3,
+  "trial_name": null,
+  "trial_params": null
+}

gpt2-custom/checkpoint-60/training_args.bin

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:86e69812d76976eae1f510debcec68b20eed3acc4e6acd97c07e1f97ec59d2ab
+size 5176

gpt2-custom/config.json

@@ -0,0 +1,39 @@
+{
+  "_name_or_path": "gpt2",
+  "activation_function": "gelu_new",
+  "architectures": [
+    "GPT2LMHeadModel"
+  ],
+  "attn_pdrop": 0.1,
+  "bos_token_id": 50256,
+  "embd_pdrop": 0.1,
+  "eos_token_id": 50256,
+  "initializer_range": 0.02,
+  "layer_norm_epsilon": 1e-05,
+  "model_type": "gpt2",
+  "n_ctx": 1024,
+  "n_embd": 768,
+  "n_head": 12,
+  "n_inner": null,
+  "n_layer": 12,
+  "n_positions": 1024,
+  "reorder_and_upcast_attn": false,
+  "resid_pdrop": 0.1,
+  "scale_attn_by_inverse_layer_idx": false,
+  "scale_attn_weights": true,
+  "summary_activation": null,
+  "summary_first_dropout": 0.1,
+  "summary_proj_to_labels": true,
+  "summary_type": "cls_index",
+  "summary_use_proj": true,
+  "task_specific_params": {
+    "text-generation": {
+      "do_sample": true,
+      "max_length": 50
+    }
+  },
+  "torch_dtype": "float32",
+  "transformers_version": "4.46.0",
+  "use_cache": true,
+  "vocab_size": 50257
+}

gpt2-custom/generation_config.json

@@ -0,0 +1,6 @@
+{
+  "_from_model_config": true,
+  "bos_token_id": 50256,
+  "eos_token_id": 50256,
+  "transformers_version": "4.46.0"
+}

gpt2-custom/model.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:d0cb1da30d2283f048311505fd22ce8ab96bed0f416a17593dee04505fbca16e
+size 497774208

gpt2-custom/special_tokens_map.json

@@ -0,0 +1,24 @@
+{
+  "bos_token": {
+    "content": "<|endoftext|>",
+    "lstrip": false,
+    "normalized": true,
+    "rstrip": false,
+    "single_word": false
+  },
+  "eos_token": {
+    "content": "<|endoftext|>",
+    "lstrip": false,
+    "normalized": true,
+    "rstrip": false,
+    "single_word": false
+  },
+  "pad_token": "<|endoftext|>",
+  "unk_token": {
+    "content": "<|endoftext|>",
+    "lstrip": false,
+    "normalized": true,
+    "rstrip": false,
+    "single_word": false
+  }
+}

gpt2-custom/tokenizer_config.json

@@ -0,0 +1,22 @@
+{
+  "add_bos_token": false,
+  "add_prefix_space": false,
+  "added_tokens_decoder": {
+    "50256": {
+      "content": "<|endoftext|>",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    }
+  },
+  "bos_token": "<|endoftext|>",
+  "clean_up_tokenization_spaces": false,
+  "eos_token": "<|endoftext|>",
+  "errors": "replace",
+  "model_max_length": 1024,
+  "pad_token": "<|endoftext|>",
+  "tokenizer_class": "GPT2Tokenizer",
+  "unk_token": "<|endoftext|>"
+}

main.py

@@ -0,0 +1,44 @@
+from transformers import GPT2LMHeadModel, GPT2Tokenizer
+import torch
+
+def setup_gpt2():
+    # Load the model and tokenizer
+    tokenizer = GPT2Tokenizer.from_pretrained('gpt2')
+    model = GPT2LMHeadModel.from_pretrained('gpt2')
+    
+    # Add padding token to tokenizer
+    if tokenizer.pad_token is None:
+        tokenizer.pad_token = tokenizer.eos_token
+        model.config.pad_token_id = model.config.eos_token_id
+    
+    return model, tokenizer
+
+def generate_text(prompt, model, tokenizer):
+    # Convert prompt to tokens and create attention mask
+    inputs = tokenizer(
+        prompt,
+        return_tensors='pt',
+        padding=True,
+        truncation=True,
+        max_length=512  # Set appropriate max length for your use case
+    )
+    
+    # Generate text with attention mask
+    outputs = model.generate(
+        input_ids=inputs['input_ids'],
+        attention_mask=inputs['attention_mask'],
+        max_length=100,
+        num_return_sequences=1,
+        no_repeat_ngram_size=2,
+        pad_token_id=tokenizer.pad_token_id,
+    )
+    
+    # Convert back to text
+    generated_text = tokenizer.decode(outputs[0], skip_special_tokens=True)
+    return generated_text
+
+# How to use it:
+model, tokenizer = setup_gpt2()
+prompt = "what is earth"
+result = generate_text(prompt, model, tokenizer)
+print(result)

testModel

@@ -0,0 +1 @@
+Subproject commit 2ac6c923ec697de160050ddea146973ee1536128

train.py

@@ -0,0 +1,121 @@
+import torch
+from transformers import GPT2LMHeadModel, GPT2Tokenizer, DataCollatorForLanguageModeling
+from transformers import Trainer, TrainingArguments
+from datasets import load_dataset
+import pandas as pd
+import os
+
+def check_device():
+    """Check and return the available device"""
+    if torch.cuda.is_available():
+        print("Using GPU:", torch.cuda.get_device_name(0))
+        return "cuda"
+    else:
+        print("No GPU available, using CPU")
+        return "cpu"
+
+def prepare_data(csv_file, text_column):
+    """Prepare training data, filtering out empty rows."""
+    df = pd.read_csv(csv_file)
+    
+    # Filter out rows where the specified column is NaN
+    df = df.dropna(subset=[text_column])
+    
+    # Save texts to a file
+    with open('train_data.txt', 'w', encoding='utf-8') as f:
+        for text in df[text_column]:
+            f.write(str(text) + '\n')
+    
+    return 'train_data.txt'
+
+def tokenize_function(examples, tokenizer):
+    """Tokenize the text using the GPT-2 tokenizer"""
+    # Turn each sentence into codes (tokens) the robot can understand
+    return tokenizer(examples['text'], padding="max_length", truncation=True, max_length=128)
+
+def train_model(model, dataset, tokenizer):
+    """Train the model with proper device configuration"""
+    # Set up data collator
+    data_collator = DataCollatorForLanguageModeling(
+        tokenizer=tokenizer,
+        mlm=False
+    )
+    
+    # Create output directory if it doesn't exist
+    os.makedirs("./gpt2-custom", exist_ok=True)
+    
+    # Set up training arguments with device-aware settings
+    training_args = TrainingArguments(
+        output_dir="./gpt2-custom",
+        overwrite_output_dir=True,
+        num_train_epochs=5,
+        per_device_train_batch_size=3,  # Reduced batch size for CPU
+        per_device_eval_batch_size=3,   # Reduced batch size for CPU
+        save_steps=10_000,
+        save_total_limit=2,
+        logging_dir="./logs",
+        logging_steps=500,
+        use_cpu=(check_device() == "cpu"),  # Disable CUDA if no GPU
+    )
+    
+    # Tokenize the dataset
+    dataset = dataset.map(lambda examples: tokenize_function(examples, tokenizer), batched=True)
+    dataset.set_format(type="torch", columns=["input_ids", "attention_mask"])
+
+    # Initialize trainer with tokenized dataset
+    trainer = Trainer(
+        model=model,
+        args=training_args,
+        data_collator=data_collator,
+        train_dataset=dataset['train'],
+    )
+    
+    # Train the model
+    trainer.train()
+    
+    # Save the model
+    model.save_pretrained('./gpt2-custom')
+    tokenizer.save_pretrained('./gpt2-custom')
+
+def main():
+    try:
+        # Check device first
+        device = check_device()
+        
+        # Example usage
+        csv_file = './csvs/mission.csv'  # Replace with your CSV file path
+        text_column = 'Mission'        # Replace with your text column name
+        
+        # Prepare data
+        print("Preparing data...")
+        train_file = prepare_data(csv_file, text_column)
+        
+        # Load model and tokenizer
+        print("Loading model and tokenizer...")
+        tokenizer = GPT2Tokenizer.from_pretrained('gpt2')
+        model = GPT2LMHeadModel.from_pretrained('gpt2')
+        
+        # Move model to appropriate device
+        model = model.to(device)
+        
+        # Add padding token
+        tokenizer.pad_token = tokenizer.eos_token
+        
+        # Load dataset with `datasets` library
+        print("Loading dataset...")
+        dataset = load_dataset('text', data_files={'train': train_file})
+
+        # Train model
+        print("Training model...")
+        train_model(model, dataset, tokenizer)
+        
+        print("Training completed successfully!")
+        
+    except Exception as e:
+        print(f"An error occurred: {str(e)}")
+        print("Stack trace:")
+        import traceback
+        traceback.print_exc()
+
+if __name__ == "__main__":
+    main()

train_data.txt

@@ -0,0 +1,2 @@
+We believe in the power of the internet and  Internet of Things to transform our lives in a positive way We believe that the future is now  With the wide-spread application of artificial learning and machine learning  it's more so.
+We live a life filled with technological marvels that aid us ubiquitously

zeliang_codetech_structured_details.csv

@@ -0,0 +1,21 @@
+Category,Details
+Company_Name,Zeliang Codetech Private Limited
+Location,"Nagaland, India"
+Founder,Kangzang Zeliang
+Industry,"  IT IoT Software Development SAAS"
+Core_Focus,"Innovative technology solutions bridging hardware, software, and IoT"
+Mission,To address challenges within local communities and industries by integrating accessible technology solutions
+Vision,"To empower businesses and civic organizations with smart solutions, improving operational efficiency and promoting technology-driven convenience"
+Team,"Composed of young, talented developers from the Naga community, committed to tech development"
+Key_Offerings,"- IoT Solutions: Intelligent hardware-software integrations for smart applications
+- Software Development: Full-stack services for web and mobile apps
+- POS Systems: Custom solutions for business needs
+- AI & Machine Learning Applications: Predictive and data-driven solutions"
+Notable_Projects,"Smart Garbage Disposal Management System (SGDMS 1.0V): IoT-enabled waste management system with real-time tracking and notifications when bins are full
+Retail_POS_Systems: Custom POS solutions for small and medium businesses
+AI-Driven_Predictive Analytics: ML-driven insights into customer trends and operational data"
+Technology_Used,"IoT sensors, real-time data tracking, custom POS software, machine learning algorithms, data analytics, dashboards, and mobile app notifications"
+Project_Impacts,"SGDMS 1.0V: Improves public sanitation, reduces collection costs, and enhances environmental impact
+POS_Systems: Streamlines business operations and enhances efficiency
+Predictive_Analytics: Empowers businesses with data-driven insights"
+Community_Engagement,"Prioritizes local talent and supports the technical growth of its team, working closely with community organizations and fostering innovation in Northeast India"