Instructions to use antirez/glm-5.3-flash-gguf with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Notebooks
Google Colab
Kaggle
Local Apps Settings

How to use antirez/glm-5.3-flash-gguf with llama.cpp:

Install (macOS, Linux)

curl -LsSf https://llama.app/install.sh | sh
# Start a local OpenAI-compatible server with a web UI:
llama serve -hf antirez/glm-5.3-flash-gguf
# Run inference directly in the terminal:
llama cli -hf antirez/glm-5.3-flash-gguf

Install from WinGet (Windows)

winget install llama.cpp
# Start a local OpenAI-compatible server with a web UI:
llama serve -hf antirez/glm-5.3-flash-gguf
# Run inference directly in the terminal:
llama cli -hf antirez/glm-5.3-flash-gguf

Use pre-built binary

# Download pre-built binary from:
# https://github.com/ggerganov/llama.cpp/releases
# Start a local OpenAI-compatible server with a web UI:
./llama-server -hf antirez/glm-5.3-flash-gguf
# Run inference directly in the terminal:
./llama-cli -hf antirez/glm-5.3-flash-gguf

Build from source code

git clone https://github.com/ggerganov/llama.cpp.git
cd llama.cpp
cmake -B build
cmake --build build -j --target llama-server llama-cli
# Start a local OpenAI-compatible server with a web UI:
./build/bin/llama-server -hf antirez/glm-5.3-flash-gguf
# Run inference directly in the terminal:
./build/bin/llama-cli -hf antirez/glm-5.3-flash-gguf

Use Docker

docker model run hf.co/antirez/glm-5.3-flash-gguf

LM Studio
Jan
Ollama
How to use antirez/glm-5.3-flash-gguf with Ollama:
```
ollama run hf.co/antirez/glm-5.3-flash-gguf
```
Unsloth Desktop

How to use antirez/glm-5.3-flash-gguf with Pi:

Start the llama.cpp server

# Install llama.cpp:
brew install llama.cpp
# Start a local OpenAI-compatible server:
llama serve -hf antirez/glm-5.3-flash-gguf

Configure the model in Pi

# Install Pi:
npm install -g @earendil-works/pi-coding-agent
# Add to ~/.pi/agent/models.json:
{
  "providers": {
    "llama-cpp": {
      "baseUrl": "http://localhost:8080/v1",
      "api": "openai-completions",
      "apiKey": "none",
      "models": [
        {
          "id": "antirez/glm-5.3-flash-gguf"
        }
      ]
    }
  }
}

Run Pi

# Start Pi in your project directory:
pi

Docker Model Runner
How to use antirez/glm-5.3-flash-gguf with Docker Model Runner:
```
docker model run hf.co/antirez/glm-5.3-flash-gguf
```

Lemonade

How to use antirez/glm-5.3-flash-gguf with Lemonade:

Pull the model

# Download Lemonade from https://lemonade-server.ai/
lemonade pull antirez/glm-5.3-flash-gguf

Run and chat with the model

lemonade run user.glm-5.3-flash-gguf-{{QUANT_TAG}}

List all available models

lemonade list

Hermes Agent

How to use antirez/glm-5.3-flash-gguf with Hermes Agent:

Start the llama.cpp server

# Install llama.cpp:
brew install llama.cpp
# Start a local OpenAI-compatible server:
llama serve -hf antirez/glm-5.3-flash-gguf

Configure Hermes

# Install Hermes:
curl -fsSL https://hermes-agent.nousresearch.com/install.sh | bash
hermes setup
# Point Hermes at the local server:
hermes config set model.provider custom
hermes config set model.base_url http://127.0.0.1:8080/v1
hermes config set model.default antirez/glm-5.3-flash-gguf

Run Hermes

hermes

Atomic Chat

OpenClaw

How to use antirez/glm-5.3-flash-gguf with OpenClaw:

Start the llama.cpp server

# Install llama.cpp:
brew install llama.cpp
# Start a local OpenAI-compatible server:
llama serve -hf antirez/glm-5.3-flash-gguf

Configure OpenClaw

# Install OpenClaw:
npm install -g openclaw@latest
# Register the local server and set it as the default model:
openclaw onboard --non-interactive --mode local \
  --auth-choice custom-api-key \
  --custom-base-url http://127.0.0.1:8080/v1 \
  --custom-model-id "antirez/glm-5.3-flash-gguf" \
  --custom-provider-id llama-cpp \
  --custom-compatibility openai \
  --custom-text-input \
  --accept-risk \
  --skip-health

Run OpenClaw

openclaw agent --local --agent main --message "Hello from Hugging Face"

Loading error using unsloth/ggml lama.cpp

by WhiteDan64 - opened 5 days ago

Discussion

WhiteDan64

5 days ago

•

edited 5 days ago

When I try to load this model with last build of (unsloth) llama.cpp, I get the following error:

[57846] 0.00.525.847 I print_info: file format = GGUF V3 (latest)
[57846] 0.00.525.848 I print_info: file type   = (guessed) all F32
[57846] 0.00.525.850 I print_info: file size   = 89.87 GiB (2.41 BPW)
[57846] 0.00.529.541 E llama_model_load: error loading model: unknown model architecture: 'glm5-next'
[57846] 0.00.529.546 E llama_model_load_from_file_impl: failed to load model
[57846] 0.00.529.553 E cmn  common_init_: failed to load model 'C:\AiProjects\models\antirez\glm-5.3-flash-gguf\GLM-5.3-Flash-Q2.gguf'

I was able to load the unsloth GGUF version, because in this case

[62596] 0.05.393.193 I common_params_fit_impl:   - CUDA0 (NVIDIA GeForce RTX 5070 Ti): 46 layers (41 overflowing),  13054 MiB used,   1836 MiB free
[62596] 0.05.393.246 I common_fit_params: successfully fit params to free device memory
[62596] 0.05.393.252 I common_fit_params: fitting params to free memory took 5.16 seconds
[62596] 0.05.451.990 I llama_model_loader: additional 2 GGUFs metadata loaded.
[62596] 0.05.451.996 I llama_model_loader: loaded meta data with 72 key-value pairs and 1412 tensors from C:\AiProjects\models\unsloth\GLM-5.3-Flash-GGUF\UD-IQ1_M\GLM-5.3-Flash-UD-IQ1_M-00001-of-00003.gguf (version GGUF V3 (latest))
[62596] 0.05.452.039 I llama_model_loader: Dumping metadata keys/values. Note: KV overrides do not apply in this output.
[62596] 0.05.452.042 I llama_model_loader: - kv   0:                       general.architecture str              = glm5next
[62596] 0.05.452.042 I llama_model_loader: - kv   1:                               general.type str              = model
[62596] 0.05.452.047 I llama_model_loader: - kv   2:                     general.sampling.top_p f32              = 0.950000
[62596] 0.05.452.048 I llama_model_loader: - kv   3:                      general.sampling.temp f32              = 1.000000
[62596] 0.05.452.048 I llama_model_loader: - kv   4:                               general.name str              = GLM 5.3 Flash
[62596] 0.05.452.049 I llama_model_loader: - kv   5:                         general.size_label str              = 288x10B
[62596] 0.05.452.049 I llama_model_loader: - kv   6:                            general.license str              = mit
[62596] 0.05.452.059 I llama_model_loader: - kv   7:                               general.tags arr[str,2]       = ["conversational", "unsloth"]

It seems that the identifier of the architecture adopted by llama.cpp is "glm5next" and not "glm5-next".

I hope that you can adopt the same identifier in your GGUF

Grazie

WhiteDan64

5 days ago

Here the full unsloth layer:

[62596] 0.05.452.042 I llama_model_loader: - kv   0:                       general.architecture str              = glm5next
[62596] 0.05.452.042 I llama_model_loader: - kv   1:                               general.type str              = model
[62596] 0.05.452.047 I llama_model_loader: - kv   2:                     general.sampling.top_p f32              = 0.950000
[62596] 0.05.452.048 I llama_model_loader: - kv   3:                      general.sampling.temp f32              = 1.000000
[62596] 0.05.452.048 I llama_model_loader: - kv   4:                               general.name str              = GLM 5.3 Flash
[62596] 0.05.452.049 I llama_model_loader: - kv   5:                         general.size_label str              = 288x10B
[62596] 0.05.452.049 I llama_model_loader: - kv   6:                            general.license str              = mit
[62596] 0.05.452.059 I llama_model_loader: - kv   7:                               general.tags arr[str,2]       = ["conversational", "unsloth"]
[62596] 0.05.452.060 I llama_model_loader: - kv   8:                          general.languages arr[str,2]       = ["en", "zh"]
[62596] 0.05.452.061 I llama_model_loader: - kv   9:                       glm5next.block_count u32              = 46
[62596] 0.05.452.062 I llama_model_loader: - kv  10:                    glm5next.context_length u32              = 1048576
[62596] 0.05.452.062 I llama_model_loader: - kv  11:                  glm5next.embedding_length u32              = 4096
[62596] 0.05.452.063 I llama_model_loader: - kv  12:               glm5next.feed_forward_length u32              = 12288
[62596] 0.05.452.063 I llama_model_loader: - kv  13:              glm5next.attention.head_count u32              = 64
[62596] 0.05.452.065 I llama_model_loader: - kv  14:  glm5next.attention.layer_norm_rms_epsilon f32              = 0.000010
[62596] 0.05.452.065 I llama_model_loader: - kv  15:                      glm5next.expert_count u32              = 288
[62596] 0.05.452.066 I llama_model_loader: - kv  16:                 glm5next.expert_used_count u32              = 8
[62596] 0.05.452.066 I llama_model_loader: - kv  17:                glm5next.expert_group_count u32              = 1
[62596] 0.05.452.066 I llama_model_loader: - kv  18:           glm5next.expert_group_used_count u32              = 1
[62596] 0.05.452.067 I llama_model_loader: - kv  19:                glm5next.expert_gating_func u32              = 2
[62596] 0.05.452.067 I llama_model_loader: - kv  20:                        glm5next.vocab_size u32              = 154880
[62596] 0.05.452.075 I llama_model_loader: - kv  21:           glm5next.attention.head_count_kv arr[i32,46]      = [0, 0, 0, 1, 0, 0, 0, 1, 0, 0, 0, 1, ...
[62596] 0.05.452.076 I llama_model_loader: - kv  22:             glm5next.attention.q_lora_rank u32              = 1536
[62596] 0.05.452.076 I llama_model_loader: - kv  23:            glm5next.attention.kv_lora_rank u32              = 512
[62596] 0.05.452.077 I llama_model_loader: - kv  24:              glm5next.rope.dimension_count u32              = 0
[62596] 0.05.452.077 I llama_model_loader: - kv  25:              glm5next.attention.key_length u32              = 512
[62596] 0.05.452.077 I llama_model_loader: - kv  26:            glm5next.attention.value_length u32              = 512
[62596] 0.05.452.078 I llama_model_loader: - kv  27:          glm5next.attention.key_length_mla u32              = 256
[62596] 0.05.452.078 I llama_model_loader: - kv  28:        glm5next.attention.value_length_mla u32              = 256
[62596] 0.05.452.079 I llama_model_loader: - kv  29:      glm5next.attention.layer_norm_epsilon f32              = 0.000001
[62596] 0.05.452.080 I llama_model_loader: - kv  30:                   glm5next.ssm.conv_kernel u32              = 4
[62596] 0.05.452.080 I llama_model_loader: - kv  31:                      glm5next.kda.head_dim u32              = 128
[62596] 0.05.452.081 I llama_model_loader: - kv  32:              glm5next.kda.gate_lower_bound f32              = -5.000000
[62596] 0.05.452.082 I llama_model_loader: - kv  33:      glm5next.attention.indexer.head_count u32              = 32
[62596] 0.05.452.082 I llama_model_loader: - kv  34:      glm5next.attention.indexer.key_length u32              = 128
[62596] 0.05.452.082 I llama_model_loader: - kv  35:           glm5next.attention.indexer.top_k u32              = 2048
[62596] 0.05.452.083 I llama_model_loader: - kv  36:           glm5next.attention.indexer.kpool u32              = 4
[62596] 0.05.452.083 I llama_model_loader: - kv  37:            glm5next.hyper_connection.count u32              = 4
[62596] 0.05.452.084 I llama_model_loader: - kv  38: glm5next.hyper_connection.sinkhorn_iterations u32              = 20
[62596] 0.05.452.084 I llama_model_loader: - kv  39:          glm5next.hyper_connection.epsilon f32              = 0.000001
[62596] 0.05.452.085 I llama_model_loader: - kv  40:        glm5next.expert_feed_forward_length u32              = 2048
[62596] 0.05.452.085 I llama_model_loader: - kv  41: glm5next.expert_shared_feed_forward_length u32              = 2048
[62596] 0.05.452.086 I llama_model_loader: - kv  42:               glm5next.expert_shared_count u32              = 1
[62596] 0.05.452.086 I llama_model_loader: - kv  43:         glm5next.leading_dense_block_count u32              = 3
[62596] 0.05.452.087 I llama_model_loader: - kv  44:              glm5next.expert_weights_scale f32              = 2.500000
[62596] 0.05.452.088 I llama_model_loader: - kv  45:               glm5next.expert_weights_norm bool             = true
[62596] 0.05.452.102 I llama_model_loader: - kv  46:                  glm5next.swiglu_clamp_exp arr[f32,46]      = [10.000000, 10.000000, 10.000000, 10....
[62596] 0.05.452.115 I llama_model_loader: - kv  47:                glm5next.swiglu_clamp_shexp arr[f32,46]      = [10.000000, 10.000000, 10.000000, 10....
[62596] 0.05.452.116 I llama_model_loader: - kv  48:              glm5next.nextn_predict_layers u32              = 1
[62596] 0.05.452.116 I llama_model_loader: - kv  49:                       tokenizer.ggml.model str              = gpt2
[62596] 0.05.452.116 I llama_model_loader: - kv  50:                         tokenizer.ggml.pre str              = glm4
[62596] 0.05.480.068 I llama_model_loader: - kv  51:                      tokenizer.ggml.tokens arr[str,154880]  = ["!", "\"", "#", "$", "%", "&", "'", ...
[62596] 0.05.491.701 I llama_model_loader: - kv  52:                  tokenizer.ggml.token_type arr[i32,154880]  = [1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, ...
[62596] 0.05.549.690 I llama_model_loader: - kv  53:                      tokenizer.ggml.merges arr[str,321649]  = ["Ġ Ġ", "Ġ ĠĠĠ", "ĠĠ ĠĠ", "...
[62596] 0.05.549.697 I llama_model_loader: - kv  54:                tokenizer.ggml.eos_token_id u32              = 154820
[62596] 0.05.549.698 I llama_model_loader: - kv  55:            tokenizer.ggml.padding_token_id u32              = 154820
[62596] 0.05.549.700 I llama_model_loader: - kv  56:                tokenizer.ggml.bos_token_id u32              = 154822
[62596] 0.05.549.701 I llama_model_loader: - kv  57:                tokenizer.ggml.eot_token_id u32              = 154827
[62596] 0.05.549.701 I llama_model_loader: - kv  58:            tokenizer.ggml.unknown_token_id u32              = 154820
[62596] 0.05.549.701 I llama_model_loader: - kv  59:                tokenizer.ggml.eom_token_id u32              = 154829
[62596] 0.05.549.702 I llama_model_loader: - kv  60:               general.quantization_version u32              = 2
[62596] 0.05.549.703 I llama_model_loader: - kv  61:                          general.file_type u32              = 31
[62596] 0.05.549.704 I llama_model_loader: - kv  62:                      quantize.imatrix.file str              = GLM-5.3-Flash-GGUF/imatrix_unsloth.gguf
[62596] 0.05.549.705 I llama_model_loader: - kv  63:                   quantize.imatrix.dataset str              = unsloth_calibration_GLM-5.3-Flash.txt
[62596] 0.05.549.705 I llama_model_loader: - kv  64:             quantize.imatrix.entries_count u32              = 809
[62596] 0.05.549.706 I llama_model_loader: - kv  65:              quantize.imatrix.chunks_count u32              = 88
[62596] 0.05.549.706 I llama_model_loader: - kv  66:                                   split.no u16              = 0
[62596] 0.05.549.707 I llama_model_loader: - kv  67:                        split.tensors.count i32              = 1412
[62596] 0.05.549.708 I llama_model_loader: - kv  68:                                split.count u16              = 3
[62596] 0.05.549.714 I llama_model_loader: - kv  69:                    tokenizer.chat_template str              = [gMASK]<sop>\n{%- set effective_reason...
[62596] 0.05.549.715 I llama_model_loader: - kv  70:                       general.quantized_by str              = Unsloth
[62596] 0.05.549.715 I llama_model_loader: - kv  71:                           general.repo_url str              = https://huggingface.co/unsloth
[62596] 0.05.549.716 I llama_model_loader: - type  f32:  638 tensors
[62596] 0.05.549.716 I llama_model_loader: - type q8_0:  346 tensors
[62596] 0.05.549.717 I llama_model_loader: - type q2_K:    2 tensors
[62596] 0.05.549.717 I llama_model_loader: - type q3_K:    1 tensors
[62596] 0.05.549.717 I llama_model_loader: - type q4_K:    2 tensors
[62596] 0.05.549.718 I llama_model_loader: - type q5_K:  248 tensors
[62596] 0.05.549.718 I llama_model_loader: - type q6_K:   49 tensors
[62596] 0.05.549.718 I llama_model_loader: - type iq2_xxs:   42 tensors
[62596] 0.05.549.718 I llama_model_loader: - type iq3_xxs:   39 tensors
[62596] 0.05.549.719 I llama_model_loader: - type iq4_xs:    3 tensors
[62596] 0.05.549.719 I llama_model_loader: - type iq1_m:   42 tensors
[62596] 0.05.549.720 I print_info: file format = GGUF V3 (latest)
[62596] 0.05.549.720 I print_info: file type   = IQ1_M - 1.75 bpw
[62596] 0.05.549.724 I print_info: file size   = 90.87 GiB (2.43 BPW)
[62596] 0.05.549.816 I llama_prepare_model_devices: using device CUDA0 (NVIDIA GeForce RTX 5070 Ti) (0000:01:00.0) - 15037 MiB free
[62596] 0.05.549.871 I load_arch_hparams: indexer selection width = 2051 cells (512 pools of 4, plus a 3-wide tail)
[62596] 0.05.684.102 I load: 0 unused tokens
[62596] 0.05.704.576 W load: special_eot_id is not in special_eog_ids - the tokenizer config may be incorrect
[62596] 0.05.704.583 W load: special_eom_id is not in special_eog_ids - the tokenizer config may be incorrect
[62596] 0.05.704.583 I load: printing all EOG tokens:
[62596] 0.05.704.583 I load:   - 154820 ('<|endoftext|>')
[62596] 0.05.704.584 I load:   - 154827 ('<|user|>')
[62596] 0.05.704.584 I load:   - 154829 ('<|observation|>')
[62596] 0.05.704.894 I load: special tokens cache size = 36
[62596] 0.05.727.194 I load: token to piece cache size = 0.9811 MB
[62596] 0.05.727.213 I print_info: arch                  = glm5next
[62596] 0.05.727.213 I print_info: vocab_only            = 0
[62596] 0.05.727.214 I print_info: no_alloc              = 0
[62596] 0.05.727.214 I print_info: n_ctx_train           = 1048576
[62596] 0.05.727.215 I print_info: n_embd_inp            = 4096
[62596] 0.05.727.215 I print_info: n_embd                = 4096
[62596] 0.05.727.215 I print_info: n_embd_out            = 4096
[62596] 0.05.727.215 I print_info: n_layer               = 45
[62596] 0.05.727.216 I print_info: n_layer_all           = 46
[62596] 0.05.727.226 I print_info: n_head                = 64
[62596] 0.05.727.236 I print_info: n_head_kv             = [0, 0, 0, 1, 0, 0, 0, 1, 0, 0, 0, 1, 0, 0, 0, 1, 0, 0, 0, 1, 0, 0, 0, 1, 0, 0, 0, 1, 0, 0, 0, 1, 0, 0, 0, 1, 0, 0, 0, 1, 0, 0, 0, 1, 0, 1]
[62596] 0.05.727.237 I print_info: n_rot                 = 0
[62596] 0.05.727.237 I print_info: n_swa                 = 0
[62596] 0.05.727.237 I print_info: is_swa_any            = 0
[62596] 0.05.727.238 I print_info: n_embd_head_k         = 512
[62596] 0.05.727.238 I print_info: n_embd_head_v         = 512
[62596] 0.05.727.247 I print_info: n_gqa                 = [0, 0, 0, 64, 0, 0, 0, 64, 0, 0, 0, 64, 0, 0, 0, 64, 0, 0, 0, 64, 0, 0, 0, 64, 0, 0, 0, 64, 0, 0, 0, 64, 0, 0, 0, 64, 0, 0, 0, 64, 0, 0, 0, 64, 0, 64]
[62596] 0.05.727.255 I print_info: n_embd_k_gqa          = [0, 0, 0, 512, 0, 0, 0, 512, 0, 0, 0, 512, 0, 0, 0, 512, 0, 0, 0, 512, 0, 0, 0, 512, 0, 0, 0, 512, 0, 0, 0, 512, 0, 0, 0, 512, 0, 0, 0, 512, 0, 0, 0, 512, 0, 512]
[62596] 0.05.727.264 I print_info: n_embd_v_gqa          = [0, 0, 0, 512, 0, 0, 0, 512, 0, 0, 0, 512, 0, 0, 0, 512, 0, 0, 0, 512, 0, 0, 0, 512, 0, 0, 0, 512, 0, 0, 0, 512, 0, 0, 0, 512, 0, 0, 0, 512, 0, 0, 0, 512, 0, 512]
[62596] 0.05.727.265 I print_info: f_norm_eps            = 1.0e-06
[62596] 0.05.727.266 I print_info: f_norm_rms_eps        = 1.0e-05
[62596] 0.05.727.266 I print_info: f_clamp_kqv           = 0.0e+00
[62596] 0.05.727.267 I print_info: f_max_alibi_bias      = 0.0e+00
[62596] 0.05.727.267 I print_info: f_logit_scale         = 0.0e+00
[62596] 0.05.727.267 I print_info: f_attn_scale          = 0.0e+00
[62596] 0.05.727.268 I print_info: f_attn_value_scale    = 0.0000
[62596] 0.05.727.269 I print_info: n_ff                  = 12288
[62596] 0.05.727.270 I print_info: n_expert              = 288
[62596] 0.05.727.270 I print_info: n_expert_used         = 8
[62596] 0.05.727.270 I print_info: n_expert_groups       = 1
[62596] 0.05.727.270 I print_info: n_group_used          = 1
[62596] 0.05.727.271 I print_info: causal attn           = 1
[62596] 0.05.727.271 I print_info: pooling type          = -1
[62596] 0.05.727.271 I print_info: rope type             = -1
[62596] 0.05.727.271 I print_info: rope scaling          = linear
[62596] 0.05.727.272 I print_info: freq_base_train       = 10000.0
[62596] 0.05.727.273 I print_info: freq_scale_train      = 1
[62596] 0.05.727.273 I print_info: n_ctx_orig_yarn       = 1048576
[62596] 0.05.727.273 I print_info: rope_yarn_log_mul     = 0.0000
[62596] 0.05.727.274 I print_info: rope_finetuned        = unknown
[62596] 0.05.727.275 I print_info: model type            = 313B.A17B
[62596] 0.05.727.275 I print_info: model params          = 320.76 B
[62596] 0.05.727.275 I print_info: general.name          = GLM 5.3 Flash
[62596] 0.05.727.277 I print_info: vocab type            = BPE
[62596] 0.05.727.277 I print_info: n_vocab               = 154880
[62596] 0.05.727.277 I print_info: n_merges              = 321649
[62596] 0.05.727.277 I print_info: BOS token             = 154822 '[gMASK]'
[62596] 0.05.727.278 I print_info: EOS token             = 154820 '<|endoftext|>'
[62596] 0.05.727.278 I print_info: EOT token             = 154827 '<|user|>'
[62596] 0.05.727.278 I print_info: EOM token             = 154829 '<|observation|>'
[62596] 0.05.727.279 I print_info: UNK token             = 154820 '<|endoftext|>'
[62596] 0.05.727.279 I print_info: PAD token             = 154820 '<|endoftext|>'
[62596] 0.05.727.279 I print_info: LF token              = 198 'Ċ'
[62596] 0.05.727.280 I print_info: FIM PRE token         = 154838 '<|code_prefix|>'
[62596] 0.05.727.280 I print_info: FIM SUF token         = 154840 '<|code_suffix|>'
[62596] 0.05.727.280 I print_info: FIM MID token         = 154839 '<|code_middle|>'
[62596] 0.05.727.280 I print_info: EOG token             = 154820 '<|endoftext|>'
[62596] 0.05.727.281 I print_info: EOG token             = 154827 '<|user|>'
[62596] 0.05.727.281 I print_info: EOG token             = 154829 '<|observation|>'
[62596] 0.05.727.281 I print_info: max token length      = 1024
[62596] 0.05.727.326 I load_tensors: loading model tensors, this can take a while... (load_mode = mmap)

antirez

Owner 5 days ago

This GGUF is specific for DwarfStar, could not be compatible with other inference engines.

antirez changed discussion status to closed 5 days ago

Upload images, audio, and videos by dragging in the text input, pasting, or clicking here.

Tap or paste here to upload images

· Sign up or log in to comment