Install from WinGet (Windows)
winget install llama.cpp
# Start a local OpenAI-compatible server with a web UI:
llama serve -hf iamrahull/xlam-ather:Q4_K_M# Run inference directly in the terminal:
llama cli -hf iamrahull/xlam-ather:Q4_K_MBuild from source code
git clone https://github.com/ggml-org/llama.cpp.git
cd llama.cpp
cmake -B build
cmake --build build -j --target llama-app# Start a local OpenAI-compatible server with a web UI:
./build/bin/llama serve -hf iamrahull/xlam-ather:Q4_K_M# Run inference directly in the terminal:
./build/bin/llama cli -hf iamrahull/xlam-ather:Q4_K_MUse Docker
docker model run hf.co/iamrahull/xlam-ather:Q4_K_MProject AETHER: xLAM-AETHER-v1 Model Card & Technical Report
1. Executive Summary
xLAM-AETHER-v1 is a specialized, 3-Billion parameter Large Language Model (LLM) engineered to serve as the cognitive engine for autonomous agentic workflows.
Developed as the core component of Project AETHER, this model is designed to act not just as a conversational chatbot, but as a ReAct (Reasoning + Acting) agent. It has been fine-tuned to understand tool schemas, reason through a user's goal, and output highly structured, syntactically perfect JSON payloads to trigger external tools (such as shell commands or API calls).
Built upon the robust γentity-SalesforceΒ¦canonical_name=Salesforceγ/xLAM-2-3b-fc-r architecture (which is based on Qwen2.5), the model was distilled using QLoRA and quantized to a highly efficient 4-bit GGUF format. This allows it to run locally on consumer-grade laptops while retaining the complex reasoning capabilities of its parent architecture.
2. Core Capabilities
- Dual-Mode Operation: The model seamlessly switches between general conversational chat (natural language) and Agentic execution (strict JSON tool calls) without suffering from catastrophic forgetting.
- High-Fidelity Tool Calling: Outputs perfectly formatted JSON arrays exactly as requested in the system prompt, ready for immediate parsing by Python/Rust backends.
- Context Window: 32,768 tokens (native), allowing for long multi-turn agent loops, massive tool schemas, and extensive observation histories.
- Low Hardware Footprint: Quantized to
Q4_K_M(~1.93 GB), allowing it to run entirely in system RAM on standard laptops or free cloud tiers.
3. Technical Specifications
| Attribute | Detail |
|---|---|
| Base Model | Salesforce/xLAM-2-3b-fc-r (Qwen2 architecture) |
| Total Parameters | 3.09 Billion (Base) + 7.3 Million (LoRA Adapter) |
| Fine-Tuning Method | QLoRA (Quantized Low-Rank Adaptation) |
| Training Precision | 4-bit NormalFloat (NF4) with FP16 compute |
| Dataset | Salesforce/APIGen-MT-5k |
| Training Hardware | Google Colab (NVIDIA Tesla T4) |
| Final Format | GGUF (q4_k_m quantization) |
| Chat Template | ChatML |
| File Size | 1.93 GB |
| License | CC-BY-NC-4.0 |
4. End-to-End Training Pipeline
4.1 Data Synthesis & Curation
Rather than training from scratch, the model utilized the Salesforce/APIGen-MT-5k dataset.
4.2 The Distillation Protocol (QLoRA)
- Base Quantization: The base model was loaded in 4-bit NF4.
- Adapter Injection: Trainable low-rank matrices injected into attention layers.
- Hyperparameters: Rank 16, Alpha 32, LR 2e-4.
- Efficiency: Using Unsloth framework, trained for 1 epoch (625 steps), final loss
0.0003.
4.3 Merging & Quantization
Post-training, the 28MB LoRA adapter was merged and converted to GGUF Q4_K_M.
5. Benchmark & Behavior Tests
Test A: General Knowledge - Retained chat persona.
Test B: Reasoning - Correct math with LaTeX.
Test C: Agentic Tool Calling - Outputs [{"name": "shell", "arguments": {"cmd": "..."}}] with 1.12s latency.
6. Deployment & Inference
Option 1: Local Inference (Ollama / LM Studio)
FROM ./xlam-aether-q4_k_m.gguf
TEMPLATE """<|im_start|>system
{{ .System }}<|im_end|>
<|im_start|>user
{{ .Prompt }}<|im_end|>
<|im_start|>assistant
"""
PARAMETER stop "<|im_end|>"
PARAMETER temperature 0.1
- Downloads last month
- 69
4-bit
Model tree for iamrahull/xlam-ather
Base model
Salesforce/xLAM-2-3b-fc-r
Install (macOS, Linux)
# Start a local OpenAI-compatible server with a web UI: llama serve -hf iamrahull/xlam-ather:Q4_K_M# Run inference directly in the terminal: llama cli -hf iamrahull/xlam-ather:Q4_K_M