Initial commit

.gitattributes

@@ -0,0 +1 @@
+*.bin filter=lfs diff=lfs merge=lfs -text

README.md

@@ -0,0 +1,177 @@
+# Hello World Model
+
+A minimal "Hello World" transformer model for demonstration purposes on Hugging Face.
+
+## Model Description
+
+This is a simple transformer-based language model that serves as a basic example for uploading models to Hugging Face. It demonstrates the minimum required files and structure for a custom model.
+
+### Architecture Details
+- **Model Type**: Custom Transformer (hello_world)
+- **Vocabulary Size**: 13 tokens
+- **Hidden Size**: 64 dimensions
+- **Number of Layers**: 1 transformer encoder layer
+- **Attention Heads**: 1
+- **Intermediate Size**: 128
+- **Max Position Embeddings**: 512
+- **Activation Function**: GELU
+
+## Files Included
+
+- `config.json` - Model configuration
+- `pytorch_model.bin` - Model weights (PyTorch format)
+- `tokenizer.json` - Tokenizer vocabulary and settings
+- `tokenizer_config.json` - Tokenizer configuration
+- `model.py` - Model implementation (HelloWorldModel class)
+- `test_model.py` - Test script for local validation
+
+## Installation
+
+### Using Virtual Environment (Recommended)
+
+It's recommended to use a virtual environment to manage dependencies:
+
+```bash
+# Create a virtual environment
+python -m venv venv
+
+# Activate the virtual environment
+# On macOS/Linux:
+source venv/bin/activate
+# On Windows:
+# venv\Scripts\activate
+
+# Install required packages
+pip install torch transformers
+```
+
+### Direct Installation
+
+If you prefer to install directly:
+
+```bash
+pip install torch transformers
+```
+
+## Usage
+
+### Basic Usage
+
+```python
+from transformers import PreTrainedTokenizerFast
+from model import HelloWorldModel, HelloWorldConfig
+import torch
+
+# Load configuration and model
+config = HelloWorldConfig.from_pretrained("chiedo/chaydos")
+model = HelloWorldModel.from_pretrained("chiedo/chaydos")
+
+# Load tokenizer
+tokenizer = PreTrainedTokenizerFast.from_pretrained("chiedo/chaydos")
+
+# Generate Hello World
+output = model.generate_hello_world()
+print(output)  # "Hello World!"
+```
+
+### Tokenization Example
+
+```python
+# Tokenize text
+text = "Hello World"
+tokens = tokenizer.encode(text)
+print(f"Tokens: {tokens}")
+
+# Decode tokens back to text
+decoded = tokenizer.decode(tokens)
+print(f"Decoded: {decoded}")
+```
+
+### Forward Pass Example
+
+```python
+# Prepare input
+input_text = "Hello"
+inputs = tokenizer(input_text, return_tensors="pt")
+
+# Forward pass
+with torch.no_grad():
+    outputs = model(**inputs)
+    logits = outputs.logits
+```
+
+## Model Vocabulary
+
+The model includes a minimal vocabulary:
+- Special tokens: `[PAD]`, `[UNK]`, `[CLS]`, `[SEP]`, `[MASK]`
+- Content tokens: `Hello`, `World`, `!`, `hello`, `world`, `.`, `,`, `?`
+
+## Training
+
+This is a demonstration model and has not been trained on any dataset. The weights are randomly initialized using a normal distribution with standard deviation of 0.02.
+
+## Testing
+
+Run the included test script to verify the model works correctly:
+
+```bash
+# Make sure your virtual environment is activated if using one
+# source venv/bin/activate  # On macOS/Linux
+# venv\Scripts\activate     # On Windows
+
+python test_model.py
+```
+
+## Uploading to Hugging Face
+
+To upload this model to your Hugging Face account:
+
+```bash
+# Install huggingface-hub
+pip install huggingface-hub
+
+# Login to Hugging Face
+huggingface-cli login
+
+# Create a new model repository (if it doesn't exist)
+huggingface-cli repo create hello-world-model --type model
+
+# Upload all model files
+huggingface-cli upload your-username/hello-world-model . --repo-type model
+```
+
+## Technical Details
+
+- **Framework**: PyTorch
+- **Transformers Version**: 4.36.0+
+- **Python Version**: 3.6+
+- **License**: MIT
+
+## Limitations
+
+- This model is for demonstration and educational purposes only
+- Not trained on any real data
+- Should not be used for production applications
+- Limited vocabulary of 13 tokens
+- Single layer architecture is too simple for real NLP tasks
+
+## Citation
+
+If you use this model as a template:
+
+```bibtex
+@misc{hello-world-model,
+  title={Hello World Model - A Minimal Hugging Face Model Example},
+  author={Your Name},
+  year={2024},
+  publisher={Hugging Face}
+}
+```
+
+## License
+
+MIT License - This model is open source and available for any use.
+
+## Contact
+
+For questions or issues with this demonstration model, please open an issue on the repository.

config.json

@@ -0,0 +1,16 @@
+{
+  "model_type": "hello_world",
+  "architectures": ["HelloWorldModel"],
+  "vocab_size": 13,
+  "hidden_size": 64,
+  "num_hidden_layers": 1,
+  "num_attention_heads": 1,
+  "intermediate_size": 128,
+  "hidden_act": "gelu",
+  "max_position_embeddings": 512,
+  "type_vocab_size": 1,
+  "initializer_range": 0.02,
+  "layer_norm_eps": 1e-12,
+  "pad_token_id": 0,
+  "transformers_version": "4.36.0"
+}

model.py

@@ -0,0 +1,127 @@
+import torch
+import torch.nn as nn
+from transformers import PreTrainedModel, PretrainedConfig
+from transformers.modeling_outputs import CausalLMOutputWithPast
+
+
+class HelloWorldConfig(PretrainedConfig):
+    model_type = "hello_world"
+    
+    def __init__(
+        self,
+        vocab_size=13,
+        hidden_size=64,
+        num_hidden_layers=1,
+        num_attention_heads=1,
+        intermediate_size=128,
+        hidden_act="gelu",
+        max_position_embeddings=512,
+        type_vocab_size=1,
+        initializer_range=0.02,
+        layer_norm_eps=1e-12,
+        pad_token_id=0,
+        **kwargs
+    ):
+        super().__init__(pad_token_id=pad_token_id, **kwargs)
+        self.vocab_size = vocab_size
+        self.hidden_size = hidden_size
+        self.num_hidden_layers = num_hidden_layers
+        self.num_attention_heads = num_attention_heads
+        self.intermediate_size = intermediate_size
+        self.hidden_act = hidden_act
+        self.max_position_embeddings = max_position_embeddings
+        self.type_vocab_size = type_vocab_size
+        self.initializer_range = initializer_range
+        self.layer_norm_eps = layer_norm_eps
+
+
+class HelloWorldModel(PreTrainedModel):
+    config_class = HelloWorldConfig
+    
+    def __init__(self, config):
+        super().__init__(config)
+        self.config = config
+        
+        self.embeddings = nn.Embedding(config.vocab_size, config.hidden_size)
+        self.position_embeddings = nn.Embedding(config.max_position_embeddings, config.hidden_size)
+        
+        self.layer = nn.TransformerEncoderLayer(
+            d_model=config.hidden_size,
+            nhead=config.num_attention_heads,
+            dim_feedforward=config.intermediate_size,
+            batch_first=True
+        )
+        
+        self.lm_head = nn.Linear(config.hidden_size, config.vocab_size)
+        
+        self.init_weights()
+    
+    def _init_weights(self, module):
+        if isinstance(module, nn.Linear):
+            module.weight.data.normal_(mean=0.0, std=self.config.initializer_range)
+            if module.bias is not None:
+                module.bias.data.zero_()
+        elif isinstance(module, nn.Embedding):
+            module.weight.data.normal_(mean=0.0, std=self.config.initializer_range)
+            if module.padding_idx is not None:
+                module.weight.data[module.padding_idx].zero_()
+    
+    def forward(
+        self,
+        input_ids=None,
+        attention_mask=None,
+        position_ids=None,
+        past_key_values=None,
+        labels=None,
+        use_cache=False,
+        output_attentions=False,
+        output_hidden_states=False,
+        return_dict=True,
+    ):
+        if input_ids is not None:
+            batch_size, seq_length = input_ids.shape
+        else:
+            raise ValueError("You have to specify input_ids")
+        
+        if position_ids is None:
+            position_ids = torch.arange(seq_length, dtype=torch.long, device=input_ids.device)
+            position_ids = position_ids.unsqueeze(0).expand(batch_size, -1)
+        
+        inputs_embeds = self.embeddings(input_ids)
+        position_embeds = self.position_embeddings(position_ids)
+        
+        hidden_states = inputs_embeds + position_embeds
+        
+        hidden_states = self.layer(hidden_states)
+        
+        logits = self.lm_head(hidden_states)
+        
+        loss = None
+        if labels is not None:
+            shift_logits = logits[..., :-1, :].contiguous()
+            shift_labels = labels[..., 1:].contiguous()
+            loss_fct = nn.CrossEntropyLoss()
+            loss = loss_fct(shift_logits.view(-1, self.config.vocab_size), shift_labels.view(-1))
+        
+        if not return_dict:
+            output = (logits,)
+            return ((loss,) + output) if loss is not None else output
+        
+        return CausalLMOutputWithPast(
+            loss=loss,
+            logits=logits,
+            past_key_values=past_key_values,
+            hidden_states=hidden_states if output_hidden_states else None,
+            attentions=None
+        )
+    
+    def generate_hello_world(self):
+        hello_token_id = 5
+        world_token_id = 6
+        
+        input_ids = torch.tensor([[hello_token_id, world_token_id]])
+        
+        with torch.no_grad():
+            outputs = self.forward(input_ids)
+        
+        return "Hello World!"

pytorch_model.bin

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:2d4b73d903ac63975c8183e6b1b727ae4e505639512375a5bcf40235021ed709
+size 277815

test_model.py

@@ -0,0 +1,28 @@
+from model import HelloWorldModel, HelloWorldConfig
+from transformers import PreTrainedTokenizerFast
+import torch
+
+print("Loading configuration...")
+config = HelloWorldConfig.from_pretrained(".")
+
+print("Loading model...")
+model = HelloWorldModel(config)
+model.load_state_dict(torch.load("pytorch_model.bin", map_location="cpu", weights_only=True))
+model.eval()
+
+print("Loading tokenizer...")
+tokenizer = PreTrainedTokenizerFast(tokenizer_file="tokenizer.json")
+
+print("\nTesting model generation...")
+output = model.generate_hello_world()
+print(f"Model output: {output}")
+
+print("\nTesting tokenization...")
+text = "Hello World"
+tokens = tokenizer.encode(text)
+print(f"Tokenized '{text}': {tokens}")
+
+decoded = tokenizer.decode(tokens)
+print(f"Decoded back: {decoded}")
+
+print("\nModel test completed successfully!")

tokenizer.json

@@ -0,0 +1,82 @@
+{
+  "version": "1.0",
+  "truncation": null,
+  "padding": null,
+  "added_tokens": [
+    {
+      "id": 0,
+      "content": "[PAD]",
+      "single_word": false,
+      "lstrip": false,
+      "rstrip": false,
+      "normalized": false,
+      "special": true
+    },
+    {
+      "id": 1,
+      "content": "[UNK]",
+      "single_word": false,
+      "lstrip": false,
+      "rstrip": false,
+      "normalized": false,
+      "special": true
+    },
+    {
+      "id": 2,
+      "content": "[CLS]",
+      "single_word": false,
+      "lstrip": false,
+      "rstrip": false,
+      "normalized": false,
+      "special": true
+    },
+    {
+      "id": 3,
+      "content": "[SEP]",
+      "single_word": false,
+      "lstrip": false,
+      "rstrip": false,
+      "normalized": false,
+      "special": true
+    },
+    {
+      "id": 4,
+      "content": "[MASK]",
+      "single_word": false,
+      "lstrip": false,
+      "rstrip": false,
+      "normalized": false,
+      "special": true
+    }
+  ],
+  "normalizer": null,
+  "pre_tokenizer": {
+    "type": "Whitespace"
+  },
+  "post_processor": null,
+  "decoder": null,
+  "model": {
+    "type": "BPE",
+    "dropout": null,
+    "unk_token": "[UNK]",
+    "continuing_subword_prefix": null,
+    "end_of_word_suffix": null,
+    "fuse_unk": false,
+    "vocab": {
+      "[PAD]": 0,
+      "[UNK]": 1,
+      "[CLS]": 2,
+      "[SEP]": 3,
+      "[MASK]": 4,
+      "Hello": 5,
+      "World": 6,
+      "!": 7,
+      "hello": 8,
+      "world": 9,
+      ".": 10,
+      ",": 11,
+      "?": 12
+    },
+    "merges": []
+  }
+}

tokenizer_config.json

@@ -0,0 +1,13 @@
+{
+  "tokenizer_class": "PreTrainedTokenizerFast",
+  "model_max_length": 512,
+  "padding_side": "right",
+  "truncation_side": "right",
+  "special_tokens_map_file": null,
+  "clean_up_tokenization_spaces": true,
+  "unk_token": "[UNK]",
+  "pad_token": "[PAD]",
+  "cls_token": "[CLS]",
+  "sep_token": "[SEP]",
+  "mask_token": "[MASK]"
+}