Upload folder using huggingface_hub

README.md

@@ -1,62 +1,25 @@
 ---
-base_model: gpt2
-library_name: peft
-model_name: Atlantis-Pyramid-Live
+language:
+- en
+license: apache-2.0
 tags:
-- base_model:adapter:gpt2
-- lora
-- sft
-- transformers
-- trl
-licence: license
-pipeline_tag: text-generation
+- marine-biology
+- coralfill
+- nemotron
+- atlantis
+inference: false
 ---
 
-# Model Card for Atlantis-Pyramid-Live
+# Atlantis-Pyramid
 
-This model is a fine-tuned version of [gpt2](https://huggingface.co/gpt2).
-It has been trained using [TRL](https://github.com/huggingface/trl).
+Atlantis-Pyramid is a fine-tuned version of `nvidia/Nemotron-3-Nano-30B-A3B-BF16`. It is explicitly tailored for the Coralfil OS ecosystem, focusing on marine restoration, aquaculture chemistry, and dynamic oceanography modeling.
 
-## Quick start
+## Model Details
+- **Base Model:** nvidia/Nemotron-3-Nano-30B-A3B-BF16 (30B params, 3B active)
+- **Fine-Tuning:** LoRA SFT on 12 validated marine science domains.
+- **Quantization:** FP8 Supported natively via vLLM.
 
-```python
-from transformers import pipeline
-
-question = "If you had a time machine, but could only go to the past or the future once and never return, which would you choose and why?"
-generator = pipeline("text-generation", model="None", device="cuda")
-output = generator([{"role": "user", "content": question}], max_new_tokens=128, return_full_text=False)[0]
-print(output["generated_text"])
+## Usage (vLLM)
+```bash
+vllm serve Coralfil-Atlantis/Atlantis-Pyramid --quantization fp8 --dtype bfloat16 --max-model-len 1000000
 ```
-
-## Training procedure
-
- 
-
-
-
-This model was trained with SFT.
-
-### Framework versions
-
-- PEFT 0.19.1
-- TRL: 1.7.1
-- Transformers: 5.12.1
-- Pytorch: 2.11.0
-- Datasets: 5.0.0
-- Tokenizers: 0.22.2
-
-## Citations
-
-
-
-Cite TRL as:
-    
-```bibtex
-@software{vonwerra2020trl,
-  title   = {{TRL: Transformers Reinforcement Learning}},
-  author  = {von Werra, Leandro and Belkada, Younes and Tunstall, Lewis and Beeching, Edward and Thrush, Tristan and Lambert, Nathan and Huang, Shengyi and Rasul, Kashif and Gallouédec, Quentin},
-  license = {Apache-2.0},
-  url     = {https://github.com/huggingface/trl},
-  year    = {2020}
-}
-```

config.json

@@ -0,0 +1,41 @@
+{
+  "activation_function": "gelu_new",
+  "add_cross_attention": false,
+  "architectures": [
+    "GPT2LMHeadModel"
+  ],
+  "attn_pdrop": 0.1,
+  "bos_token_id": 50256,
+  "dtype": "bfloat16",
+  "embd_pdrop": 0.1,
+  "eos_token_id": 50256,
+  "initializer_range": 0.02,
+  "layer_norm_epsilon": 1e-05,
+  "model_type": "gpt2",
+  "n_ctx": 1024,
+  "n_embd": 768,
+  "n_head": 12,
+  "n_inner": null,
+  "n_layer": 12,
+  "n_positions": 1024,
+  "pad_token_id": null,
+  "reorder_and_upcast_attn": false,
+  "resid_pdrop": 0.1,
+  "scale_attn_by_inverse_layer_idx": false,
+  "scale_attn_weights": true,
+  "summary_activation": null,
+  "summary_first_dropout": 0.1,
+  "summary_proj_to_labels": true,
+  "summary_type": "cls_index",
+  "summary_use_proj": true,
+  "task_specific_params": {
+    "text-generation": {
+      "do_sample": true,
+      "max_length": 50
+    }
+  },
+  "tie_word_embeddings": true,
+  "transformers_version": "5.12.1",
+  "use_cache": true,
+  "vocab_size": 50257
+}

generation_config.json

@@ -0,0 +1,6 @@
+{
+  "_from_model_config": true,
+  "bos_token_id": 50256,
+  "eos_token_id": 50256,
+  "transformers_version": "5.12.1"
+}

model.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:3158ea0edc37418a711f2912e41fad8dc1fa04cf1cf43af201023ea30f7adc8f
+size 248894656

tokenizer_config.json

@@ -1,13 +1,13 @@
-{
-  "add_prefix_space": false,
-  "backend": "tokenizers",
-  "bos_token": "<|endoftext|>",
-  "eos_token": "<|endoftext|>",
-  "errors": "replace",
-  "is_local": false,
-  "local_files_only": false,
-  "model_max_length": 1024,
-  "pad_token": "<|endoftext|>",
-  "tokenizer_class": "GPT2Tokenizer",
-  "unk_token": "<|endoftext|>"
-}
+{
+  "add_prefix_space": false,
+  "backend": "tokenizers",
+  "bos_token": "<|endoftext|>",
+  "eos_token": "<|endoftext|>",
+  "errors": "replace",
+  "is_local": false,
+  "local_files_only": false,
+  "model_max_length": 1024,
+  "pad_token": null,
+  "tokenizer_class": "GPT2Tokenizer",
+  "unk_token": "<|endoftext|>"
+}