tinkersnot commited on
Commit
a7d82b8
·
verified ·
1 Parent(s): 4065de8

Upload folder using huggingface_hub

Browse files
Files changed (2) hide show
  1. Dockerfile +17 -0
  2. README.md +7 -5
Dockerfile ADDED
@@ -0,0 +1,17 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ # tinkersnot bake-off inference Space — official llama.cpp server, no compile.
2
+ # Serves an OpenAI-compatible /v1/chat/completions with native tool-calling (--jinja).
3
+ FROM ghcr.io/ggml-org/llama.cpp:server
4
+
5
+ # llama-server reads LLAMA_ARG_* env vars for its flags — no CMD/entrypoint wrestling.
6
+ # Swap LLAMA_ARG_HF_REPO to change the candidate model (repo:quant tag).
7
+ # Real candidate : unsloth/Qwen3.5-0.8B-GGUF:Q8_0 (hybrid VLM, tool-calling)
8
+ # Smoke-test/fallb: ggml-org/Qwen3-0.6B-GGUF:Q8_0 (std arch, guaranteed to load)
9
+ # If the 0.8B errors on the hybrid arch or demands a vision projector, flip to the 0.6B line.
10
+ ENV LLAMA_ARG_HOST=0.0.0.0 \
11
+ LLAMA_ARG_PORT=7860 \
12
+ LLAMA_ARG_HF_REPO=unsloth/Qwen3.5-0.8B-GGUF:Q8_0 \
13
+ LLAMA_ARG_CTX_SIZE=8192 \
14
+ LLAMA_ARG_JINJA=1 \
15
+ HF_HOME=/tmp/hf
16
+
17
+ EXPOSE 7860
README.md CHANGED
@@ -1,10 +1,12 @@
1
  ---
2
- title: Bench Server
3
- emoji: 🦀
4
- colorFrom: gray
5
- colorTo: gray
6
  sdk: docker
 
7
  pinned: false
8
  ---
9
 
10
- Check out the configuration reference at https://huggingface.co/docs/hub/spaces-config-reference
 
 
1
  ---
2
+ title: Tinkersnot Bench Server
3
+ emoji: 🦙
4
+ colorFrom: blue
5
+ colorTo: green
6
  sdk: docker
7
+ app_port: 7860
8
  pinned: false
9
  ---
10
 
11
+ OpenAI-compatible llama.cpp server for base-model bake-off eval.
12
+ Endpoint: `/v1/chat/completions`. Model set via `LLAMA_ARG_HF_REPO` in the Dockerfile.