Instructions to use Ashok75/GAKR-model with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use Ashok75/GAKR-model with llama-cpp-python:

# !pip install llama-cpp-python

from llama_cpp import Llama

llm = Llama.from_pretrained(
	repo_id="Ashok75/GAKR-model",
	filename="gakr-model-01.gguf",
)

llm.create_chat_completion(
	messages = "No input example has been defined for this model task."
)

Notebooks
Google Colab
Kaggle
Local Apps Settings

llama.cpp

How to use Ashok75/GAKR-model with llama.cpp:

Install from brew

brew install llama.cpp
# Start a local OpenAI-compatible server with a web UI:
llama-server -hf Ashok75/GAKR-model
# Run inference directly in the terminal:
llama-cli -hf Ashok75/GAKR-model

Install from WinGet (Windows)

winget install llama.cpp
# Start a local OpenAI-compatible server with a web UI:
llama-server -hf Ashok75/GAKR-model
# Run inference directly in the terminal:
llama-cli -hf Ashok75/GAKR-model

Use pre-built binary

# Download pre-built binary from:
# https://github.com/ggerganov/llama.cpp/releases
# Start a local OpenAI-compatible server with a web UI:
./llama-server -hf Ashok75/GAKR-model
# Run inference directly in the terminal:
./llama-cli -hf Ashok75/GAKR-model

Build from source code

git clone https://github.com/ggerganov/llama.cpp.git
cd llama.cpp
cmake -B build
cmake --build build -j --target llama-server llama-cli
# Start a local OpenAI-compatible server with a web UI:
./build/bin/llama-server -hf Ashok75/GAKR-model
# Run inference directly in the terminal:
./build/bin/llama-cli -hf Ashok75/GAKR-model

Use Docker

docker model run hf.co/Ashok75/GAKR-model

LM Studio
Jan
Ollama
How to use Ashok75/GAKR-model with Ollama:
```
ollama run hf.co/Ashok75/GAKR-model
```

Unsloth Studio

How to use Ashok75/GAKR-model with Unsloth Studio:

Install Unsloth Studio (macOS, Linux, WSL)

curl -fsSL https://unsloth.ai/install.sh | sh
# Run unsloth studio
unsloth studio -H 0.0.0.0 -p 8888
# Then open http://localhost:8888 in your browser
# Search for Ashok75/GAKR-model to start chatting

Install Unsloth Studio (Windows)

irm https://unsloth.ai/install.ps1 | iex
# Run unsloth studio
unsloth studio -H 0.0.0.0 -p 8888
# Then open http://localhost:8888 in your browser
# Search for Ashok75/GAKR-model to start chatting

Using HuggingFace Spaces for Unsloth

# No setup required
# Open https://huggingface.co/spaces/unsloth/studio in your browser
# Search for Ashok75/GAKR-model to start chatting

How to use Ashok75/GAKR-model with Pi:

Start the llama.cpp server

# Install llama.cpp:
brew install llama.cpp
# Start a local OpenAI-compatible server:
llama-server -hf Ashok75/GAKR-model

Configure the model in Pi

# Install Pi:
npm install -g @mariozechner/pi-coding-agent
# Add to ~/.pi/agent/models.json:
{
  "providers": {
    "llama-cpp": {
      "baseUrl": "http://localhost:8080/v1",
      "api": "openai-completions",
      "apiKey": "none",
      "models": [
        {
          "id": "Ashok75/GAKR-model"
        }
      ]
    }
  }
}

Run Pi

# Start Pi in your project directory:
pi

Hermes Agent new

How to use Ashok75/GAKR-model with Hermes Agent:

Start the llama.cpp server

# Install llama.cpp:
brew install llama.cpp
# Start a local OpenAI-compatible server:
llama-server -hf Ashok75/GAKR-model

Configure Hermes

# Install Hermes:
curl -fsSL https://hermes-agent.nousresearch.com/install.sh | bash
hermes setup
# Point Hermes at the local server:
hermes config set model.provider custom
hermes config set model.base_url http://127.0.0.1:8080/v1
hermes config set model.default Ashok75/GAKR-model

Run Hermes

hermes

Docker Model Runner
How to use Ashok75/GAKR-model with Docker Model Runner:
```
docker model run hf.co/Ashok75/GAKR-model
```

Lemonade

How to use Ashok75/GAKR-model with Lemonade:

Pull the model

# Download Lemonade from https://lemonade-server.ai/
lemonade pull Ashok75/GAKR-model

Run and chat with the model

lemonade run user.GAKR-model-{{QUANT_TAG}}

List all available models

lemonade list

YAML Metadata Warning:empty or missing yaml metadata in repo card

Check out the documentation for more information.

refer the space - ashok75-gakr.hf.space

GAKR AI – Local File‑Aware Chat Assistant

GAKR AI is a local, privacy‑friendly chat assistant that runs entirely on your machine.
It combines a FastAPI backend, a modern web chat UI, and a file‑intelligence pipeline that can read and summarize many file types before generating natural‑language responses.

The assistant itself is text‑only. It never directly sees raw PDFs, images, audio, or videos.
Instead, specialized tools convert files into structured text summaries, and the language model reasons over that text.

✨ Features

🌐 Web Chat Interface

Clean dark UI with message bubbles and typing indicator
Auto‑growing input box
Attach files from camera, gallery, or filesystem
Works in any modern browser at http://localhost:8080

🧠 Text + File Understanding

Prompt only → general assistant (explanations, coding help, reasoning)
Prompt + files → full analysis pipeline:
- Detects file type
- Stores uploads in dataupload/
- Extracts structured facts
- Feeds extracted context + question to the model

📂 Multi‑File, Multi‑Type Uploads

Upload multiple files at once:

Documents: PDF, DOCX, TXT
Tabular data: CSV, Excel, JSON
Images: OCR via Tesseract
Audio: Speech‑to‑text via Whisper
Video: Audio extraction via ffmpeg → Whisper

💾 Persistent Uploads

Files saved under dataupload/ by type
Timestamped, safe filenames
Automatic directory creation

🔐 Simple Login Reminder UX

After 5 guest messages, a popup encourages login
Logged‑in users are not interrupted
Login state stored in localStorage

🗂 Project Structure

project_root/
├── run.py                # FastAPI backend + template serving
├── load_model.py         # Loads the language model once
├── generate.py           # generate_response() wrapper
├── file_pipeline.py      # File detection, storage, and summarization
├── templates/
│   ├── chat.html         # Main chat interface
│   └── auth.html         # Login / signup UI
├── dataupload/           # Created at runtime for uploads
│   ├── images/
│   ├── videos/
│   ├── audio/
│   ├── documents/
│   ├── tabular/
│   └── other/
└── requirements.txt

⚙️ Installation

1️⃣ Create & Activate Virtual Environment (Recommended)

python -m venv .venv
source .venv/bin/activate        # Linux / macOS
# or
.\.venv\Scripts\activate      # Windows

2️⃣ Install Python Dependencies

pip install -r requirements.txt

requirements.txt

fastapi
uvicorn[standard]
python-multipart

torch
transformers
accelerate
safetensors

pandas
numpy

pdfplumber
pymupdf
python-docx

Pillow
pytesseract

openai-whisper
ffmpeg-python

3️⃣ Install System Tools

Tesseract OCR (for image text extraction)
ffmpeg (for audio extraction and Whisper)

Install via OS package manager (apt, brew, choco) or official installers.

▶️ Running GAKR AI

Start the Backend

python run.py

Expected output:

🚀 Starting GAKR AI Backend...
✅ Model initialized successfully

🌐 SERVER & CHAT LOCATION
🚀 CHAT INTERFACE:     http://localhost:8080
🔧 API DOCUMENTATION:  http://localhost:8080/docs
✅ CHAT.HTML SERVED:   templates/chat.html

Open the Chat UI

Navigate to:

http://localhost:8080

🔌 API Overview

POST `/api/analyze`

Request (multipart/form-data)

api_key (string, required)
prompt (string, required)
files (optional, multiple)

Behavior

No files → General assistant mode
With files → File‑analysis mode using structured summaries

Response

{
  "response": "natural-language answer here",
  "context": {
    "files": [
      {
        "original_name": "report.pdf",
        "stored_path": "dataupload/documents/20241214_report.pdf",
        "kind": "document",
        "summary": {
          "type": "document",
          "char_count": 12345,
          "preview": "First 4000 characters..."
        }
      }
    ]
  },
  "status": "success"
}

🧪 File Intelligence Pipeline

Handled by file_pipeline.py

Type Detection

Tabular → CSV, XLSX, JSON
Documents → PDF, DOCX, TXT
Images → PNG, JPG
Audio → MP3, WAV
Video → MP4, MKV

Summaries

Tabular: rows, columns, missing values, stats
Documents: character count + preview
Images: dimensions + OCR text
Audio: duration + transcript preview
Video: extracted audio analysis

Errors are stored per‑file and never crash the whole request.

🎨 Frontend UX Highlights

Auto‑growing textarea
Attachment chips with remove buttons
Typing indicator
URL prefill: ?q=your+question
Generic error message for all backend failures

🔐 Security Notes

API key is currently a fixed string (for local use)
For production:
- Use environment variables
- Add real authentication (JWT / sessions)
- Restrict CORS
- Apply upload size limits and cleanup policies

🚀 Extending GAKR AI

Ideas:

Per‑user chat & file history (database)
Search across uploaded documents
External API integrations
HTTPS + reverse proxy deployment

🧠 Philosophy

GAKR AI is an intelligence layer.
Tools translate reality (files, media, data) into structured language.
The language model turns that language into insight, reasoning, and action.

Downloads last month: 4

GGUF

Model size

8B params

Architecture

qwen2

Hardware compatibility

We're not able to determine the quantization variants.

View all variants

Inference Providers NEW

This model isn't deployed by any Inference Provider. 🙋 Ask for provider support