Upload 6 files

README.md

@@ -0,0 +1,45 @@
+---
+language: en
+license: mit
+tags:
+- conversational
+- efficient
+- proprietary-architecture
+datasets:
+- starhopp3r/TinyChat
+---
+
+# i3 Model - Memory-Optimized Efficient Conversational Language Model
+
+## Model Description
+
+The **i3 Model** is a memory-optimized language model designed for conversational understanding. This version uses streaming tokenization to minimize RAM usage during training.
+
+**PROPRIETARY ARCHITECTURE**: The internal architecture and training methodologies are proprietary and confidential.
+
+## Model Statistics
+
+- **Vocabulary Size**: 4,466 (variable-length chunks)
+- **Hidden Dimension**: 512
+- **Number of Layers**: 24
+- **Max Sequence Length**: 256
+- **Total Parameters**: 22,640,626
+- **Tokenization**: Memory-efficient variable-length chunking (2-3 characters)
+
+### Key Features
+
+1. **Memory-Optimized**: Streaming tokenization reduces RAM usage significantly
+2. **Proprietary Hybrid Architecture**: Advanced sequence processing with linear complexity
+3. **Variable-Length Tokenization**: Smart chunking strategy for better compression
+4. **Conversational Focus**: Specialized for dialogue and emotional understanding
+
+## Training Details
+
+- **Dataset**: [TinyChat](https://huggingface.co/datasets/starhopp3r/TinyChat)
+- **Training Objective**: Next-token prediction with proprietary optimization
+- **Framework**: PyTorch
+- **Memory Optimization**: Streaming dataset processing
+
+## License
+
+**PROPRIETARY LICENSE** - All rights reserved.

config.json

@@ -0,0 +1,17 @@
+{
+  "architectures": [
+    "i3Model"
+  ],
+  "model_type": "i3",
+  "vocab_size": 4466,
+  "d_model": 512,
+  "n_layers": 24,
+  "n_heads": 16,
+  "max_seq_len": 256,
+  "rank": 128,
+  "d_state": 64,
+  "tokenizer_type": "chunk",
+  "chunk_strategy": "variable_2_3",
+  "torch_dtype": "float32",
+  "transformers_version": "4.36.0"
+}

pytorch_model.bin

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:1ccf2db41965042284ce1f7024ab4e84b2194a3efdc6152f859f90234a26ef22
+size 90818463

special_tokens_map.json

@@ -0,0 +1 @@
+{}

tokenizer_config.json

@@ -0,0 +1,8 @@
+{
+  "tokenizer_class": "ChunkTokenizer",
+  "model_max_length": 256,
+  "vocab_size": 4466,
+  "chunk_strategy": "variable_2_3",
+  "special_tokens": {},
+  "clean_up_tokenization_spaces": false
+}