Upload model files

README.md

@@ -1,3 +1,50 @@
----
-license: mit
----
+---
+        language: en
+        license: mit
+        tags:
+        - pytorch
+        - causal-lm
+        - language-model
+        - flash-attention
+        ---
+
+        # PurelyUnfunctionalAI/GibberishGPT
+
+        This is a language model trained with Flash Attention. The model is based on a decoder-only transformer architecture.
+
+        ## Model Details
+
+        - **Model Type:** Causal Language Model
+        - **Embedding Size:** 512
+        - **Hidden Layers:** 8
+        - **Attention Heads:** 8
+        - **Context Length:** 512
+        - **Flash Attention:** Enabled
+
+        ## Usage
+
+        ```python
+        import tiktoken
+        from transformers import AutoModelForCausalLM
+
+        # Load the tokenizer
+        tokenizer = tiktoken.get_encoding("gpt2")
+
+        # Load the model
+        model = AutoModelForCausalLM.from_pretrained("PurelyUnfunctionalAI/GibberishGPT")
+
+        # Encode input
+        input_text = "Your prompt here"
+        input_ids = tokenizer.encode(input_text)
+        input_tensor = torch.tensor([input_ids], dtype=torch.long)
+
+        # Generate
+        output = model.generate(input_tensor, max_length=100)
+        generated_text = tokenizer.decode(output[0].tolist())
+        print(generated_text)
+        ```
+
+        ## License
+
+        This model is available under the MIT License.
+        

config.json

@@ -0,0 +1,16 @@
+{
+  "architectures": [
+    "FlashAttentionForCausalLM"
+  ],
+  "model_type": "flash_attention_lm",
+  "vocab_size": 50257,
+  "hidden_size": 512,
+  "num_hidden_layers": 8,
+  "num_attention_heads": 8,
+  "max_position_embeddings": 512,
+  "hidden_dropout_prob": 0.1,
+  "use_flash_attention": true,
+  "gradient_checkpointing": false,
+  "torch_dtype": "float32",
+  "transformers_version": "4.40.0"
+}

pytorch_model.bin

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:19e152e0a66e16497d403c629527601af9f5990fa21fc1c63d259ab041880be8
+size 408803874

tokenizer_config.json

@@ -0,0 +1,7 @@
+{
+  "model_type": "tiktoken",
+  "tokenizer_class": "TiktokenTokenizer",
+  "bos_token": "<|endoftext|>",
+  "eos_token": "<|endoftext|>",
+  "unk_token": "<|endoftext|>"
+}

upload_instructions.md

@@ -0,0 +1,45 @@
+# Upload to Hugging Face Hub
+
+            To upload your model to the Hugging Face Hub, you can use the Hugging Face CLI:
+
+            ## 1. Install the Hugging Face Hub CLI
+            ```bash
+            pip install huggingface_hub
+            ```
+
+            ## 2. Login to Hugging Face
+            ```bash
+            huggingface-cli login
+            ```
+
+            ## 3. Create a new repository
+            Go to https://huggingface.co/new and create a new model repository.
+
+            ## 4. Upload your model
+            ```bash
+            cd ./published_model/hf_model
+            git init
+            git add .
+            git commit -m "Initial model upload"
+            git remote add origin https://huggingface.co/PurelyUnfunctionalAI/GibberishGPT
+            git push -u origin main
+            ```
+
+            Alternatively, you can use the Python API:
+
+            ```python
+            from huggingface_hub import HfApi
+            api = HfApi()
+
+            # Login to Hugging Face
+            api.login()
+
+            # Upload model files
+            api.create_repo(repo_id="PurelyUnfunctionalAI/GibberishGPT", repo_type="model", exist_ok=True)
+            api.upload_folder(
+                folder_path="./published_model/hf_model",
+                repo_id="PurelyUnfunctionalAI/GibberishGPT",
+                commit_message="Upload model"
+            )
+            ```
+