Instructions to use CreativeWorksAi/iSA-01-Mini-3B-GGUF with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use CreativeWorksAi/iSA-01-Mini-3B-GGUF with Transformers:

# Load model directly
from transformers import AutoModel
model = AutoModel.from_pretrained("CreativeWorksAi/iSA-01-Mini-3B-GGUF", dtype="auto")

llama-cpp-python

How to use CreativeWorksAi/iSA-01-Mini-3B-GGUF with llama-cpp-python:

# !pip install llama-cpp-python

from llama_cpp import Llama

llm = Llama.from_pretrained(
	repo_id="CreativeWorksAi/iSA-01-Mini-3B-GGUF",
	filename="iSA-01-Mini-3B.F16.gguf",
)

llm.create_chat_completion(
	messages = "No input example has been defined for this model task."
)

Notebooks
Google Colab
Kaggle
Local Apps

llama.cpp

How to use CreativeWorksAi/iSA-01-Mini-3B-GGUF with llama.cpp:

Install from brew

brew install llama.cpp
# Start a local OpenAI-compatible server with a web UI:
llama-server -hf CreativeWorksAi/iSA-01-Mini-3B-GGUF:Q4_K_M
# Run inference directly in the terminal:
llama-cli -hf CreativeWorksAi/iSA-01-Mini-3B-GGUF:Q4_K_M

Install from WinGet (Windows)

winget install llama.cpp
# Start a local OpenAI-compatible server with a web UI:
llama-server -hf CreativeWorksAi/iSA-01-Mini-3B-GGUF:Q4_K_M
# Run inference directly in the terminal:
llama-cli -hf CreativeWorksAi/iSA-01-Mini-3B-GGUF:Q4_K_M

Use pre-built binary

# Download pre-built binary from:
# https://github.com/ggerganov/llama.cpp/releases
# Start a local OpenAI-compatible server with a web UI:
./llama-server -hf CreativeWorksAi/iSA-01-Mini-3B-GGUF:Q4_K_M
# Run inference directly in the terminal:
./llama-cli -hf CreativeWorksAi/iSA-01-Mini-3B-GGUF:Q4_K_M

Build from source code

git clone https://github.com/ggerganov/llama.cpp.git
cd llama.cpp
cmake -B build
cmake --build build -j --target llama-server llama-cli
# Start a local OpenAI-compatible server with a web UI:
./build/bin/llama-server -hf CreativeWorksAi/iSA-01-Mini-3B-GGUF:Q4_K_M
# Run inference directly in the terminal:
./build/bin/llama-cli -hf CreativeWorksAi/iSA-01-Mini-3B-GGUF:Q4_K_M

Use Docker

docker model run hf.co/CreativeWorksAi/iSA-01-Mini-3B-GGUF:Q4_K_M

LM Studio
Jan
Ollama
How to use CreativeWorksAi/iSA-01-Mini-3B-GGUF with Ollama:
```
ollama run hf.co/CreativeWorksAi/iSA-01-Mini-3B-GGUF:Q4_K_M
```

Unsloth Studio new

How to use CreativeWorksAi/iSA-01-Mini-3B-GGUF with Unsloth Studio:

Install Unsloth Studio (macOS, Linux, WSL)

curl -fsSL https://unsloth.ai/install.sh | sh
# Run unsloth studio
unsloth studio -H 0.0.0.0 -p 8888
# Then open http://localhost:8888 in your browser
# Search for CreativeWorksAi/iSA-01-Mini-3B-GGUF to start chatting

Install Unsloth Studio (Windows)

irm https://unsloth.ai/install.ps1 | iex
# Run unsloth studio
unsloth studio -H 0.0.0.0 -p 8888
# Then open http://localhost:8888 in your browser
# Search for CreativeWorksAi/iSA-01-Mini-3B-GGUF to start chatting

Using HuggingFace Spaces for Unsloth

# No setup required
# Open https://huggingface.co/spaces/unsloth/studio in your browser
# Search for CreativeWorksAi/iSA-01-Mini-3B-GGUF to start chatting

Pi new

How to use CreativeWorksAi/iSA-01-Mini-3B-GGUF with Pi:

Start the llama.cpp server

# Install llama.cpp:
brew install llama.cpp
# Start a local OpenAI-compatible server:
llama-server -hf CreativeWorksAi/iSA-01-Mini-3B-GGUF:Q4_K_M

Configure the model in Pi

# Install Pi:
npm install -g @mariozechner/pi-coding-agent
# Add to ~/.pi/agent/models.json:
{
  "providers": {
    "llama-cpp": {
      "baseUrl": "http://localhost:8080/v1",
      "api": "openai-completions",
      "apiKey": "none",
      "models": [
        {
          "id": "CreativeWorksAi/iSA-01-Mini-3B-GGUF:Q4_K_M"
        }
      ]
    }
  }
}

Run Pi

# Start Pi in your project directory:
pi

Hermes Agent new

How to use CreativeWorksAi/iSA-01-Mini-3B-GGUF with Hermes Agent:

Start the llama.cpp server

# Install llama.cpp:
brew install llama.cpp
# Start a local OpenAI-compatible server:
llama-server -hf CreativeWorksAi/iSA-01-Mini-3B-GGUF:Q4_K_M

Configure Hermes

# Install Hermes:
curl -fsSL https://hermes-agent.nousresearch.com/install.sh | bash
hermes setup
# Point Hermes at the local server:
hermes config set model.provider custom
hermes config set model.base_url http://127.0.0.1:8080/v1
hermes config set model.default CreativeWorksAi/iSA-01-Mini-3B-GGUF:Q4_K_M

Run Hermes

hermes

Docker Model Runner
How to use CreativeWorksAi/iSA-01-Mini-3B-GGUF with Docker Model Runner:
```
docker model run hf.co/CreativeWorksAi/iSA-01-Mini-3B-GGUF:Q4_K_M
```

Lemonade

How to use CreativeWorksAi/iSA-01-Mini-3B-GGUF with Lemonade:

Pull the model

# Download Lemonade from https://lemonade-server.ai/
lemonade pull CreativeWorksAi/iSA-01-Mini-3B-GGUF:Q4_K_M

Run and chat with the model

lemonade run user.iSA-01-Mini-3B-GGUF-Q4_K_M

List all available models

lemonade list

Note: This is our first public release on Hugging Face, and the Model Card is still a work in progress. Further improvements and updates will follow.

CreativeWorksAi + NeuraLake: Designed by Earth's Creatives, Assembled by AI

Model Description

The iSA-01-Mini-3B-GGUF is a small yet advanced language model developed by CreativeWorksAi, designed to enhance text generation and reasoning capabilities. It extends the context window from 128K to 256K tokens, effectively doubling its information retention and significantly improving performance compared to its base model, meta-llama/Llama-3.2-3B-Instruct.

Hardware Requirements Estimate

Name	Quant method	Size	Memory (RAM, vRAM) required
iSA-01-Mini-3B.F16.gguf	F16	6.43 GB	12.86 GB
iSA-01-Mini-3B.Q4_K_M.gguf	Q4_K_M	2.02 GB	4.04 GB
iSA-01-Mini-3B.Q5_K_M.gguf	Q5_K_M	2.32 GB	4.64 GB

Key Features

Extended Context Window: The model's context window has been expanded from 128K to 256K tokens, enabling it to retain more information for better reasoning and logical deductions.
Enhanced Reasoning: The increased context size leads to superior performance in complex tasks like Retrieval-Augmented Generation (RAG), resulting in more precise and context-aware outputs.
Improved Information Integration: With a larger context window, the model integrates external information more effectively, producing accurate and contextually relevant responses.
Fine-tuned with NeuraLake/Megalodon: The model was fine-tuned using synthetic data generated by the state-of-the-art NeuraLake/Megalodon, enhancing its ability to process and analyze complex scenarios.
NeuraLake/Megalodon Model: This proprietary, closed-source LLM has been developed by NeuraLake over the past three years to enhance reasoning capabilities, especially for small models and agents.

Training Data

The iSA-01-Mini-3B-GGUF was trained using synthetic data generated by NeuraLake/Megalodon, focused on realistic scenarios to improve reasoning and performance in RAG tasks.

Model Details

Developed by: CreativeWorksAi
License: Apache License 2.0
Fine-tuned from: unsloth/llama-3.2-3b-instruct-bnb-4bit
NeuraLake: Learn more about NeuraLake and its advanced AI projects at NeuraLake Cloud.
NeuraLake/Megalodon Model: Discover more about the model used for synthetic data generation at NeuraLake.
Model Card Contact: Celso H A Diniz LinkedIn profile.

Usage

CreativeWorksAi's Intelligence System for Advanced Dialogue and Organized Responses Assistance (i.S.A.D.O.R.A. architecture) is designed to offer users a sophisticated tool for generating coherent, contextually rich text, making it ideal for applications that require advanced natural language understanding and generation.

🇧🇷

Downloads last month: 63

GGUF

Model size

3B params

Architecture

llama

Hardware compatibility

4-bit

5-bit

16-bit

Inference Providers NEW

This model isn't deployed by any Inference Provider. 🙋 Ask for provider support

Model tree for CreativeWorksAi/iSA-01-Mini-3B-GGUF

Base model

meta-llama/Llama-3.2-3B-Instruct

Quantized

(458)

this model

Collection including CreativeWorksAi/iSA-01-Mini-3B-GGUF

NeuraLake - Megalodon DataGen

Collection

NeuralLake's Megalodon LLM now leads in generating top-tier synthetic datasets, improving the robustness and accuracy of your Ai models • 1 item • Updated Oct 4, 2024