AppleMind-1.0-Mini-GGUF

Instructions to use AppleMind-AI/AppleMind-1.0-Mini-GGUF with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use AppleMind-AI/AppleMind-1.0-Mini-GGUF with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("text-generation", model="AppleMind-AI/AppleMind-1.0-Mini-GGUF")
messages = [
    {"role": "user", "content": "Who are you?"},
]
pipe(messages)

# Load model directly
from transformers import AutoModel
model = AutoModel.from_pretrained("AppleMind-AI/AppleMind-1.0-Mini-GGUF", device_map="auto")

Notebooks
Google Colab
Kaggle
Local Apps Settings

llama.cpp

How to use AppleMind-AI/AppleMind-1.0-Mini-GGUF with llama.cpp:

Install (macOS, Linux)

curl -LsSf https://llama.app/install.sh | sh
# Start a local OpenAI-compatible server with a web UI:
llama serve -hf AppleMind-AI/AppleMind-1.0-Mini-GGUF:BF16
# Run inference directly in the terminal:
llama cli -hf AppleMind-AI/AppleMind-1.0-Mini-GGUF:BF16

Install from WinGet (Windows)

winget install llama.cpp
# Start a local OpenAI-compatible server with a web UI:
llama serve -hf AppleMind-AI/AppleMind-1.0-Mini-GGUF:BF16
# Run inference directly in the terminal:
llama cli -hf AppleMind-AI/AppleMind-1.0-Mini-GGUF:BF16

Use pre-built binary

# Download pre-built binary from:
# https://github.com/ggerganov/llama.cpp/releases
# Start a local OpenAI-compatible server with a web UI:
./llama-server -hf AppleMind-AI/AppleMind-1.0-Mini-GGUF:BF16
# Run inference directly in the terminal:
./llama-cli -hf AppleMind-AI/AppleMind-1.0-Mini-GGUF:BF16

Build from source code

git clone https://github.com/ggerganov/llama.cpp.git
cd llama.cpp
cmake -B build
cmake --build build -j --target llama-server llama-cli
# Start a local OpenAI-compatible server with a web UI:
./build/bin/llama-server -hf AppleMind-AI/AppleMind-1.0-Mini-GGUF:BF16
# Run inference directly in the terminal:
./build/bin/llama-cli -hf AppleMind-AI/AppleMind-1.0-Mini-GGUF:BF16

Use Docker

docker model run hf.co/AppleMind-AI/AppleMind-1.0-Mini-GGUF:BF16

LM Studio
Jan

vLLM

How to use AppleMind-AI/AppleMind-1.0-Mini-GGUF with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "AppleMind-AI/AppleMind-1.0-Mini-GGUF"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "AppleMind-AI/AppleMind-1.0-Mini-GGUF",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker

docker model run hf.co/AppleMind-AI/AppleMind-1.0-Mini-GGUF:BF16

SGLang

How to use AppleMind-AI/AppleMind-1.0-Mini-GGUF with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "AppleMind-AI/AppleMind-1.0-Mini-GGUF" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "AppleMind-AI/AppleMind-1.0-Mini-GGUF",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "AppleMind-AI/AppleMind-1.0-Mini-GGUF" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "AppleMind-AI/AppleMind-1.0-Mini-GGUF",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Ollama
How to use AppleMind-AI/AppleMind-1.0-Mini-GGUF with Ollama:
```
ollama run hf.co/AppleMind-AI/AppleMind-1.0-Mini-GGUF:BF16
```

Unsloth Studio

How to use AppleMind-AI/AppleMind-1.0-Mini-GGUF with Unsloth Studio:

Install Unsloth Studio (macOS, Linux, WSL)

curl -fsSL https://unsloth.ai/install.sh | sh
# Run unsloth studio
unsloth studio -H 0.0.0.0 -p 8888
# Then open http://localhost:8888 in your browser
# Search for AppleMind-AI/AppleMind-1.0-Mini-GGUF to start chatting

Install Unsloth Studio (Windows)

irm https://unsloth.ai/install.ps1 | iex
# Run unsloth studio
unsloth studio -H 0.0.0.0 -p 8888
# Then open http://localhost:8888 in your browser
# Search for AppleMind-AI/AppleMind-1.0-Mini-GGUF to start chatting

Using HuggingFace Spaces for Unsloth

# No setup required
# Open https://huggingface.co/spaces/unsloth/studio in your browser
# Search for AppleMind-AI/AppleMind-1.0-Mini-GGUF to start chatting

Docker Model Runner
How to use AppleMind-AI/AppleMind-1.0-Mini-GGUF with Docker Model Runner:
```
docker model run hf.co/AppleMind-AI/AppleMind-1.0-Mini-GGUF:BF16
```

Lemonade

How to use AppleMind-AI/AppleMind-1.0-Mini-GGUF with Lemonade:

Pull the model

# Download Lemonade from https://lemonade-server.ai/
lemonade pull AppleMind-AI/AppleMind-1.0-Mini-GGUF:BF16

Run and chat with the model

lemonade run user.AppleMind-1.0-Mini-GGUF-BF16

List all available models

lemonade list

Atomic Chat

plsss

by AxionLab-official - opened 5 days ago

Discussion

AxionLab-official

5 days ago

Please 1Bit quantization

GGUFGuy

AppleMind AI org 5 days ago

sadly impossible, the model is too small :(
logs:

llama_model_loader: - kv 9: general.dataset.1.name str = FineWeb HQ
llama_model_loader: - kv 10: general.dataset.1.organization str = Epfml
llama_model_loader: - kv 11: general.dataset.1.repo_url str = https://huggingface.co/epfml/FineWeb-HQ
llama_model_loader: - kv 12: general.dataset.2.name str = Smollm Corpus
llama_model_loader: - kv 13: general.dataset.2.organization str = HuggingFaceTB
llama_model_loader: - kv 14: general.dataset.2.repo_url str = https://huggingface.co/HuggingFaceTB/...
llama_model_loader: - kv 15: general.tags arr[str,21] = ["lm", "language-model", "causal-lm",...
llama_model_loader: - kv 16: general.languages arr[str,1] = ["en"]
llama_model_loader: - kv 17: gpt2.block_count u32 = 2
llama_model_loader: - kv 18: gpt2.context_length u32 = 256
llama_model_loader: - kv 19: gpt2.embedding_length u32 = 20
llama_model_loader: - kv 20: gpt2.feed_forward_length u32 = 80
llama_model_loader: - kv 21: gpt2.attention.head_count u32 = 2
llama_model_loader: - kv 22: gpt2.attention.layer_norm_epsilon f32 = 0.000010
llama_model_loader: - kv 23: general.file_type u32 = 1
llama_model_loader: - kv 24: general.quantization_version u32 = 2
llama_model_loader: - kv 25: tokenizer.ggml.model str = gpt2
llama_model_loader: - kv 26: tokenizer.ggml.pre str = gpt-2
llama_model_loader: - kv 27: tokenizer.ggml.tokens arr[str,50260] = ["!", """, "#", "$", "%", "&", "'", ...
llama_model_loader: - kv 28: tokenizer.ggml.token_type arr[i32,50260] = [1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, ...
llama_model_loader: - kv 29: tokenizer.ggml.merges arr[str,50000] = ["Ġ t", "Ġ a", "h e", "i n", "r e",...
llama_model_loader: - kv 30: tokenizer.ggml.bos_token_id u32 = 50256
llama_model_loader: - kv 31: tokenizer.ggml.eos_token_id u32 = 50256
llama_model_loader: - kv 32: tokenizer.ggml.unknown_token_id u32 = 50256
llama_model_loader: - kv 33: tokenizer.ggml.padding_token_id u32 = 50256
llama_model_loader: - kv 34: tokenizer.chat_template str = {% for message in messages %}{{ '<|' ...
llama_model_loader: - type f32: 19 tensors
llama_model_loader: - type f16: 9 tensors
warning: token_embd.weight - ncols 20 not divisible by 32 (required for type q8_0) llama_model_quantize: failed to quantize: no tensor type fallback is defined for type q8_0
llama_quantize: failed to quantize model from '/home/japugalt/llama.cpp/quantize/AppleMind-1.0-Mini/AppleMind-1.0-Mini-F16.gguf'

Finished in 0.0 seconds.

✗ FAILED: Q5_K_M
Exit code: 1

################################################################################

Q6_K

################################################################################

================================================================================
RUNNING

/home/japugalt/llama.cpp/build/bin/llama-quantize /home/japugalt/llama.cpp/quantize/AppleMind-1.0-Mini/AppleMind-1.0-Mini-F16.gguf /home/japugalt/llama.cpp/quantize/AppleMind-1.0-Mini/AppleMind-1.0-Mini-Q6_K.gguf Q6_K

version: 0.1.0-dev (build 10398, commit 8e7f22b67)
built with GNU 16.1.1 for Linux x86_64
llama_quantize: quantizing '/home/japugalt/llama.cpp/quantize/AppleMind-1.0-Mini/AppleMind-1.0-Mini-F16.gguf' to '/home/japugalt/llama.cpp/quantize/AppleMind-1.0-Mini/AppleMind-1.0-Mini-Q6_K.gguf' as Q6_K
llama_model_loader: loaded meta data with 35 key-value pairs and 28 tensors from /home/japugalt/llama.cpp/quantize/AppleMind-1.0-Mini/AppleMind-1.0-Mini-F16.gguf (version GGUF V3 (latest))
llama_model_loader: Dumping metadata keys/values. Note: KV overrides do not apply in this output.
llama_model_loader: - kv 0: general.architecture str = gpt2
llama_model_loader: - kv 1: general.type str = model
llama_model_loader: - kv 2: general.name str = Edd1E6E300Aca8Dc2E387Eaa780B944659De5C6C
llama_model_loader: - kv 3: general.size_label str = 1.0M
llama_model_loader: - kv 4: general.license str = apache-2.0
llama_model_loader: - kv 5: general.dataset.count u32 = 3
llama_model_loader: - kv 6: general.dataset.0.name str = Fineweb Edu
llama_model_loader: - kv 7: general.dataset.0.organization str = HuggingFaceFW
llama_model_loader: - kv 8: general.dataset.0.repo_url str = https://huggingface.co/HuggingFaceFW/...
llama_model_loader: - kv 9: general.dataset.1.name str = FineWeb HQ
llama_model_loader: - kv 10: general.dataset.1.organization str = Epfml
llama_model_loader: - kv 11: general.dataset.1.repo_url str = https://huggingface.co/epfml/FineWeb-HQ
llama_model_loader: - kv 12: general.dataset.2.name str = Smollm Corpus
llama_model_loader: - kv 13: general.dataset.2.organization str = HuggingFaceTB
llama_model_loader: - kv 14: general.dataset.2.repo_url str = https://huggingface.co/HuggingFaceTB/...
llama_model_loader: - kv 15: general.tags arr[str,21] = ["lm", "language-model", "causal-lm",...
llama_model_loader: - kv 16: general.languages arr[str,1] = ["en"]
llama_model_loader: - kv 17: gpt2.block_count u32 = 2
llama_model_loader: - kv 18: gpt2.context_length u32 = 256
llama_model_loader: - kv 19: gpt2.embedding_length u32 = 20
llama_model_loader: - kv 20: gpt2.feed_forward_length u32 = 80
llama_model_loader: - kv 21: gpt2.attention.head_count u32 = 2
llama_model_loader: - kv 22: gpt2.attention.layer_norm_epsilon f32 = 0.000010
llama_model_loader: - kv 23: general.file_type u32 = 1
llama_model_loader: - kv 24: general.quantization_version u32 = 2
llama_model_loader: - kv 25: tokenizer.ggml.model str = gpt2
llama_model_loader: - kv 26: tokenizer.ggml.pre str = gpt-2
llama_model_loader: - kv 27: tokenizer.ggml.tokens arr[str,50260] = ["!", """, "#", "$", "%", "&", "'", ...
llama_model_loader: - kv 28: tokenizer.ggml.token_type arr[i32,50260] = [1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, ...
llama_model_loader: - kv 29: tokenizer.ggml.merges arr[str,50000] = ["Ġ t", "Ġ a", "h e", "i n", "r e",...
llama_model_loader: - kv 30: tokenizer.ggml.bos_token_id u32 = 50256
llama_model_loader: - kv 31: tokenizer.ggml.eos_token_id u32 = 50256
llama_model_loader: - kv 32: tokenizer.ggml.unknown_token_id u32 = 50256
llama_model_loader: - kv 33: tokenizer.ggml.padding_token_id u32 = 50256
llama_model_loader: - kv 34: tokenizer.chat_template str = {% for message in messages %}{{ '<|' ...
llama_model_loader: - type f32: 19 tensors
llama_model_loader: - type f16: 9 tensors
warning: token_embd.weight - ncols 20 not divisible by 32 (required for type q8_0) llama_model_quantize: failed to quantize: no tensor type fallback is defined for type q8_0
llama_quantize: failed to quantize model from '/home/japugalt/llama.cpp/quantize/AppleMind-1.0-Mini/AppleMind-1.0-Mini-F16.gguf'

Finished in 0.0 seconds.

✗ FAILED: Q6_K
Exit code: 1

################################################################################

Q8_0

################################################################################

================================================================================
RUNNING

version: 0.1.0-dev (build 10398, commit 8e7f22b67)
built with GNU 16.1.1 for Linux x86_64
llama_quantize: quantizing '/home/japugalt/llama.cpp/quantize/AppleMind-1.0-Mini/AppleMind-1.0-Mini-F16.gguf' to '/home/japugalt/llama.cpp/quantize/AppleMind-1.0-Mini/AppleMind-1.0-Mini-Q8_0.gguf' as Q8_0
llama_model_loader: loaded meta data with 35 key-value pairs and 28 tensors from /home/japugalt/llama.cpp/quantize/AppleMind-1.0-Mini/AppleMind-1.0-Mini-F16.gguf (version GGUF V3 (latest))
llama_model_loader: Dumping metadata keys/values. Note: KV overrides do not apply in this output.
llama_model_loader: - kv 0: general.architecture str = gpt2
llama_model_loader: - kv 1: general.type str = model
llama_model_loader: - kv 2: general.name str = Edd1E6E300Aca8Dc2E387Eaa780B944659De5C6C
llama_model_loader: - kv 3: general.size_label str = 1.0M
llama_model_loader: - kv 4: general.license str = apache-2.0
llama_model_loader: - kv 5: general.dataset.count u32 = 3
llama_model_loader: - kv 6: general.dataset.0.name str = Fineweb Edu
llama_model_loader: - kv 7: general.dataset.0.organization str = HuggingFaceFW
llama_model_loader: - kv 8: general.dataset.0.repo_url str = https://huggingface.co/HuggingFaceFW/...
llama_model_loader: - kv 9: general.dataset.1.name str = FineWeb HQ
llama_model_loader: - kv 10: general.dataset.1.organization str = Epfml
llama_model_loader: - kv 11: general.dataset.1.repo_url str = https://huggingface.co/epfml/FineWeb-HQ
llama_model_loader: - kv 12: general.dataset.2.name str = Smollm Corpus
llama_model_loader: - kv 13: general.dataset.2.organization str = HuggingFaceTB
llama_model_loader: - kv 14: general.dataset.2.repo_url str = https://huggingface.co/HuggingFaceTB/...
llama_model_loader: - kv 15: general.tags arr[str,21] = ["lm", "language-model", "causal-lm",...
llama_model_loader: - kv 16: general.languages arr[str,1] = ["en"]
llama_model_loader: - kv 17: gpt2.block_count u32 = 2
llama_model_loader: - kv 18: gpt2.context_length u32 = 256
llama_model_loader: - kv 19: gpt2.embedding_length u32 = 20
llama_model_loader: - kv 20: gpt2.feed_forward_length u32 = 80
llama_model_loader: - kv 21: gpt2.attention.head_count u32 = 2
llama_model_loader: - kv 22: gpt2.attention.layer_norm_epsilon f32 = 0.000010
llama_model_loader: - kv 23: general.file_type u32 = 1
llama_model_loader: - kv 24: general.quantization_version u32 = 2
llama_model_loader: - kv 25: tokenizer.ggml.model str = gpt2
llama_model_loader: - kv 26: tokenizer.ggml.pre str = gpt-2
llama_model_loader: - kv 27: tokenizer.ggml.tokens arr[str,50260] = ["!", """, "#", "$", "%", "&", "'", ...
llama_model_loader: - kv 28: tokenizer.ggml.token_type arr[i32,50260] = [1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, ...
llama_model_loader: - kv 29: tokenizer.ggml.merges arr[str,50000] = ["Ġ t", "Ġ a", "h e", "i n", "r e",...
llama_model_loader: - kv 30: tokenizer.ggml.bos_token_id u32 = 50256
llama_model_loader: - kv 31: tokenizer.ggml.eos_token_id u32 = 50256
llama_model_loader: - kv 32: tokenizer.ggml.unknown_token_id u32 = 50256
llama_model_loader: - kv 33: tokenizer.ggml.padding_token_id u32 = 50256
llama_model_loader: - kv 34: tokenizer.chat_template str = {% for message in messages %}{{ '<|' ...
llama_model_loader: - type f32: 19 tensors
llama_model_loader: - type f16: 9 tensors
warning: token_embd.weight - ncols 20 not divisible by 32 (required for type q8_0) llama_model_quantize: failed to quantize: no tensor type fallback is defined for type q8_0
llama_quantize: failed to quantize model from '/home/japugalt/llama.cpp/quantize/AppleMind-1.0-Mini/AppleMind-1.0-Mini-F16.gguf'

Finished in 0.0 seconds.

✗ FAILED: Q8_0
Exit code: 1

================================================================================
FINISHED

Successful: 0

Skipped: 0

Failed: 31
✗ Q1_0
✗ Q2_0
✗ Q4_0
✗ Q4_1
✗ Q5_0
✗ Q5_1
✗ IQ1_S
✗ IQ1_M
✗ IQ2_XXS
✗ IQ2_XS
✗ IQ2_S
✗ IQ2_M
✗ TQ1_0
✗ TQ2_0
✗ Q2_K
✗ Q2_K_S
✗ IQ3_XXS
✗ IQ3_XS
✗ IQ3_S
✗ IQ3_M
✗ Q3_K_S
✗ Q3_K_M
✗ Q3_K_L
✗ IQ4_NL
✗ IQ4_XS
✗ Q4_K_S
✗ Q4_K_M
✗ Q5_K_S
✗ Q5_K_M
✗ Q6_K
✗ Q8_0

================================================================================
GGUF FILES

AppleMind-1.0-Mini-BF16.gguf 3.65 MiB
AppleMind-1.0-Mini-F16.gguf 3.65 MiB
AppleMind-1.0-Mini-F32.gguf 5.58 MiB

Total storage: 12.87 MiB

GGUF directory:
/home/japugalt/llama.cpp/quantize/AppleMind-1.0-Mini

================================================================================
FAILED FORMATS

Q1_0
Q2_0
Q4_0
Q4_1
Q5_0
Q5_1
IQ1_S
IQ1_M
IQ2_XXS
IQ2_XS
IQ2_S
IQ2_M
TQ1_0
TQ2_0
Q2_K
Q2_K_S
IQ3_XXS
IQ3_XS
IQ3_S
IQ3_M
Q3_K_S
Q3_K_M
Q3_K_L
IQ4_NL
IQ4_XS
Q4_K_S
Q4_K_M
Q5_K_S
Q5_K_M
Q6_K
Q8_0

The other formats were still generated.
Download complete: : █████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 7.45MB, 1.80MB/s
Reconstruction complete: 100%|███████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 7.77MB / 7.77MB, 1.39MB/s

GGUFGuy

AppleMind AI org 5 days ago

not even q8 is possible

AxionLab-official

5 days ago

Oh, ok :(

AxionLab-official changed discussion status to closed 5 days ago

GGUFGuy

AppleMind AI org 5 days ago

you will have to wait until applemind 1.0 medium releases (5M params)

Banaxi-Tech

AppleMind AI org about 8 hours ago

This comment has been hidden

Upload images, audio, and videos by dragging in the text input, pasting, or clicking here.

Tap or paste here to upload images

· Sign up or log in to comment

plsss

Q6_K

================================================================================RUNNING

Q8_0

================================================================================RUNNING

================================================================================FINISHED

================================================================================GGUF FILES

================================================================================FAILED FORMATS

================================================================================
RUNNING

================================================================================
RUNNING

================================================================================
FINISHED

================================================================================
GGUF FILES

================================================================================
FAILED FORMATS