Instructions to use my-ai-stack/Stack-2-9-finetuned with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use my-ai-stack/Stack-2-9-finetuned with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("text-generation", model="my-ai-stack/Stack-2-9-finetuned")
messages = [
    {"role": "user", "content": "Who are you?"},
]
pipe(messages)

# Load model directly
from transformers import AutoTokenizer, AutoModelForCausalLM

tokenizer = AutoTokenizer.from_pretrained("my-ai-stack/Stack-2-9-finetuned")
model = AutoModelForCausalLM.from_pretrained("my-ai-stack/Stack-2-9-finetuned")
messages = [
    {"role": "user", "content": "Who are you?"},
]
inputs = tokenizer.apply_chat_template(
	messages,
	add_generation_prompt=True,
	tokenize=True,
	return_dict=True,
	return_tensors="pt",
).to(model.device)

outputs = model.generate(**inputs, max_new_tokens=40)
print(tokenizer.decode(outputs[0][inputs["input_ids"].shape[-1]:]))

Notebooks
Google Colab
Kaggle
Local Apps

vLLM

How to use my-ai-stack/Stack-2-9-finetuned with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "my-ai-stack/Stack-2-9-finetuned"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "my-ai-stack/Stack-2-9-finetuned",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker

docker model run hf.co/my-ai-stack/Stack-2-9-finetuned

SGLang

How to use my-ai-stack/Stack-2-9-finetuned with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "my-ai-stack/Stack-2-9-finetuned" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "my-ai-stack/Stack-2-9-finetuned",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "my-ai-stack/Stack-2-9-finetuned" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "my-ai-stack/Stack-2-9-finetuned",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Docker Model Runner
How to use my-ai-stack/Stack-2-9-finetuned with Docker Model Runner:
```
docker model run hf.co/my-ai-stack/Stack-2-9-finetuned
```

walidsobhie-code Claude Opus 4.6 commited on Apr 4

Commit

f90ca5c

1 Parent(s): 9008611

fix: define all paths in first cell for kernel restart safety

Browse files

- Define ROOT_DIR, REPO_DIR, MODEL_DIR, OUTPUT_DIR in cell 1
- All other cells now use these pre-defined variables
- Prevents NameError when kernel restarts

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

Files changed (1) hide show

colab_train_stack29.ipynb +5 -30

colab_train_stack29.ipynb CHANGED Viewed

@@ -23,39 +23,14 @@
    "execution_count": null,
    "metadata": {},
    "outputs": [],
-   "source": [
-    "# STEP 1: Setup - Mount Drive and define root directory\n",
-    "from google.colab import drive\n",
-    "drive.mount('/content/drive')\n",
-    "\n",
-    "import os\n",
-    "ROOT_DIR = \"/content/drive/MyDrive/stack-2.9\"\n",
-    "os.makedirs(ROOT_DIR, exist_ok=True)\n",
-    "os.chdir(ROOT_DIR)\n",
-    "\n",
-    "print(f\"✅ Working directory: {os.getcwd()}\")\n",
-    "!ls -la"
-   ]
   },
   {
    "cell_type": "code",
    "execution_count": null,
    "metadata": {},
    "outputs": [],
-   "source": [
-    "# STEP 2: Clone repo (fresh every time)\n",
-    "import shutil\n",
-    "\n",
-    "if os.path.exists('stack-2.9'):\n",
-    "    print(\"Removing old stack-2.9...\")\n",
-    "    shutil.rmtree('stack-2.9')\n",
-    "\n",
-    "!git clone https://github.com/my-ai-stack/stack-2.9.git\n",
-    "\n",
-    "os.chdir(os.path.join(ROOT_DIR, 'stack-2.9'))\n",
-    "print(f\"✅ In: {os.getcwd()}\")\n",
-    "!ls -la"
-   ]
   },
   {
    "cell_type": "code",
@@ -74,21 +49,21 @@
    "execution_count": null,
    "metadata": {},
    "outputs": [],
-   "source": "# STEP 4: Download Base Model (Qwen2.5-Coder-7B)\nimport os  # Make sure os is imported\nfrom transformers import AutoModelForCausalLM, AutoTokenizer\n\nMODEL_NAME = \"Qwen/Qwen2.5-Coder-7B\"\nMODEL_DIR = os.path.join(ROOT_DIR, \"stack-2.9/base_model_qwen7b\")"
   },
   {
    "cell_type": "code",
    "execution_count": null,
    "metadata": {},
    "outputs": [],
-   "source": "# STEP 5: Find or download training data\nimport os  # Make sure os is imported\nimport json  # Make sure json is imported\nREPO_DIR = os.path.join(ROOT_DIR, \"stack-2.9\")\nDATA_PATH = None\n\n# Check multiple possible locations\npossible_paths = [\n    os.path.join(REPO_DIR, \"data/final/train.jsonl\"),\n    os.path.join(REPO_DIR, \"training-data/final/train.jsonl\"),\n    os.path.join(REPO_DIR, \"data_mini/train_mini.jsonl\"),\n]\n\nfor path in possible_paths:\n    if os.path.exists(path):\n        DATA_PATH = path\n        print(f\"✅ Found data at: {path}\")\n        break"
   },
   {
    "cell_type": "code",
    "execution_count": null,
    "metadata": {},
    "outputs": [],
-   "source": "# STEP 6: Prepare Training Configuration\nimport os  # Make sure os is imported\nimport yaml\n\nconfig_path = os.path.join(REPO_DIR, \"stack/training/train_config_local.yaml\")"
   },
   {
    "cell_type": "code",

    "execution_count": null,
    "metadata": {},
    "outputs": [],
+   "source": "# STEP 1: Setup - Mount Drive and define root directory\nfrom google.colab import drive\ndrive.mount('/content/drive')\n\nimport os\nROOT_DIR = \"/content/drive/MyDrive/stack-2.9\"\nos.makedirs(ROOT_DIR, exist_ok=True)\nos.chdir(ROOT_DIR)\n\n# Define all paths once\nREPO_DIR = os.path.join(ROOT_DIR, \"stack-2.9\")\nMODEL_DIR = os.path.join(REPO_DIR, \"base_model_qwen7b\")\nOUTPUT_DIR = os.path.join(ROOT_DIR, \"training_output\")\n\nprint(f\"✅ ROOT_DIR: {ROOT_DIR}\")\nprint(f\"✅ REPO_DIR: {REPO_DIR}\")\nprint(f\"✅ MODEL_DIR: {MODEL_DIR}\")\nprint(f\"✅ OUTPUT_DIR: {OUTPUT_DIR}\")\n!ls -la"
   },
   {
    "cell_type": "code",
    "execution_count": null,
    "metadata": {},
    "outputs": [],
+   "source": "# STEP 2: Clone repo (fresh every time)\nimport shutil\n\nif os.path.exists('stack-2.9'):\n    print(\"Removing old stack-2.9...\")\n    shutil.rmtree('stack-2.9')\n\n!git clone https://github.com/my-ai-stack/stack-2.9.git\n\nos.chdir(REPO_DIR)\nprint(f\"✅ In: {os.getcwd()}\")\n!ls -la"
   },
   {
    "cell_type": "code",
    "execution_count": null,
    "metadata": {},
    "outputs": [],
+   "source": "# STEP 4: Download Base Model (Qwen2.5-Coder-7B)\nfrom transformers import AutoModelForCausalLM, AutoTokenizer\n\nMODEL_NAME = \"Qwen/Qwen2.5-Coder-7B\"\n\nif not os.path.exists(os.path.join(MODEL_DIR, \"config.json\")):\n    print(f\"Downloading {MODEL_NAME} to {MODEL_DIR}...\")\n    print(\"This will take 15-20 minutes...\")\n    tokenizer = AutoTokenizer.from_pretrained(MODEL_NAME, trust_remote_code=True)\n    tokenizer.save_pretrained(MODEL_DIR)\n    model = AutoModelForCausalLM.from_pretrained(MODEL_NAME, trust_remote_code=True)\n    model.save_pretrained(MODEL_DIR)\n    print(f\"✅ Model saved\")\nelse:\n    print(f\"✅ Model already exists\")\n\n!ls -lh {MODEL_DIR} | head -5"
   },
   {
    "cell_type": "code",
    "execution_count": null,
    "metadata": {},
    "outputs": [],
+   "source": "# STEP 5: Find or download training data\nimport json\n\nDATA_PATH = None\n\n# Check multiple possible locations\npossible_paths = [\n    os.path.join(REPO_DIR, \"data/final/train.jsonl\"),\n    os.path.join(REPO_DIR, \"training-data/final/train.jsonl\"),\n    os.path.join(REPO_DIR, \"data_mini/train_mini.jsonl\"),\n]\n\nfor path in possible_paths:\n    if os.path.exists(path):\n        DATA_PATH = path\n        print(f\"✅ Found data at: {path}\")\n        break"
   },
   {
    "cell_type": "code",
    "execution_count": null,
    "metadata": {},
    "outputs": [],
+   "source": "# STEP 6: Prepare Training Configuration\nimport yaml\n\nconfig_path = os.path.join(REPO_DIR, \"stack/training/train_config_local.yaml\")\n\nif not os.path.exists(config_path):\n    raise FileNotFoundError(f\"Config not found at: {config_path}\")\n\nwith open(config_path, 'r') as f:\n    config = yaml.safe_load(f)\n\n# Update config with absolute paths\nconfig['model']['name'] = MODEL_DIR\nconfig['data']['input_path'] = DATA_PATH\nconfig['output']['lora_dir'] = os.path.join(OUTPUT_DIR, \"lora\")\nconfig['output']['merged_dir'] = os.path.join(OUTPUT_DIR, \"merged\")\nconfig['hardware']['device'] = \"cuda\"\nconfig['hardware']['num_gpus'] = 1\n\nos.makedirs(OUTPUT_DIR, exist_ok=True)\nupdated_config_path = os.path.join(OUTPUT_DIR, \"train_config.yaml\")\n\nwith open(updated_config_path, 'w') as f:\n    yaml.dump(config, f)\n\nprint(f\"✅ Config saved to: {updated_config_path}\")\nprint(f\"   Model: {config['model']['name']}\")\nprint(f\"   Data: {config['data']['input_path']}\")"
   },
   {
    "cell_type": "code",