help

usage: koboldcpp [-h] [--model [filenames] [[filenames] ...]]
                 [--port [portnumber]] [--host [ipaddr]] [--launch]
                 [--config [filename]] [--threads [threads]]
                 [--usecuda [[main GPU ID] [mmq|nommq] [rowsplit]
                 [[main GPU ID] [mmq|nommq] [rowsplit] ...]]]
                 [--usevulkan [[Device IDs] [[Device IDs] ...]]] [--usecpu]
                 [--contextsize [256 to 262144]] [--gpulayers [[GPU layers]]]
                 [--tensor_split [Ratios] [[Ratios] ...]] [--autofit]
                 [--version] [--analyze [filename]] [--maingpu [Device ID]]
                 [--batchsize {-1,16,32,64,128,256,512,1024,2048,4096}]
                 [--blasthreads [threads]] [--lora [lora_filename]
                 [[lora_filename] ...]] [--loramult [amount]] [--noshift]
                 [--nofastforward] [--useswa] [--smartcache [limit]]
                 [--ropeconfig [rope-freq-scale] [[rope-freq-base] ...]]
                 [--overridenativecontext [trained context]] [--usemmap]
                 [--usemlock] [--noavx2] [--failsafe]
                 [--debugmode [DEBUGMODE]] [--onready [shell command]]
                 [--benchmark [[filename]]] [--prompt [prompt]] [--cli]
                 [--genlimit [token limit]] [--multiuser [limit]]
                 [--multiplayer] [--websearch] [--remotetunnel]
                 [--highpriority] [--foreground] [--preloadstory [savefile]]
                 [--savedatafile [savefile]] [--quiet] [--ssl [cert_pem]
                 [[key_pem] ...]] [--nocertify] [--mmproj [filename]]
                 [--mmprojcpu] [--visionmaxres [max px]]
                 [--draftmodel [filename]] [--draftamount [tokens]]
                 [--draftgpulayers [layers]] [--draftgpusplit [Ratios]
                 [[Ratios] ...]] [--password [API key]]
                 [--ratelimit [seconds]] [--ignoremissing]
                 [--chatcompletionsadapter [filename]] [--jinja]
                 [--jinja_tools] [--noflashattention] [--lowvram]
                 [--quantkv [quantization level 0/1/2]] [--smartcontext]
                 [--unpack destination] [--exportconfig [filename]]
                 [--exporttemplate [filename]] [--nomodel]
                 [--moeexperts [num of experts]]
                 [--moecpu [[layers affected]]]
                 [--defaultgenamt DEFAULTGENAMT] [--nobostoken]
                 [--enableguidance] [--maxrequestsize [size in MB]]
                 [--overridekv [name=type:value]]
                 [--overridetensors [tensor name pattern=buffer type]]
                 [--showgui | --skiplauncher] [--singleinstance]
                 [--nopipelineparallel]
                 [--gendefaults {"parameter":"value",...}]
                 [--gendefaultsoverwrite] [--mcpfile [mcp json file]]
                 [--device <dev1,dev2,..>] [--downloaddir [directory]]
                 [--autofitpadding [padding in MB]] [--hordemodelname [name]]
                 [--hordeworkername [name]] [--hordekey [apikey]]
                 [--hordemaxctx [amount]] [--hordegenlen [amount]]
                 [--sdmodel [filename]] [--sdthreads [threads]]
                 [--sdclamped [[maxres]]] [--sdclampedsoft [maxres]]
                 [--sdt5xxl [filename]] [--sdclip1 [filename]]
                 [--sdclip2 [filename]] [--sdphotomaker [filename]]
                 [--sdupscaler [filename]] [--sdflashattention]
                 [--sdoffloadcpu] [--sdvaecpu] [--sdclipgpu]
                 [--sdconvdirect {off,vaeonly,full}] [--sdvae [filename] |
                 --sdvaeauto] [--sdquant [[quantization level 0/1/2]] |
                 --sdlora [filename] [[filename] ...]] [--sdloramult [amounts]
                 [[amounts] ...]] [--sdtiledvae [maxres]]
                 [--whispermodel [filename]] [--ttsmodel [filename]]
                 [--ttswavtokenizer [filename]] [--ttsgpu]
                 [--ttsmaxlen TTSMAXLEN] [--ttsthreads [threads]]
                 [--ttsdir [directory]] [--musicllm [filename]]
                 [--musicembeddings [filename]] [--musicdiffusion [filename]]
                 [--musicvae [filename]] [--musiclowvram]
                 [--embeddingsmodel [filename]] [--embeddingsmaxctx [amount]]
                 [--embeddingsgpu] [--admin] [--adminpassword [password]]
                 [--admindir [directory]]
                 [--adminunloadtimeout ADMINUNLOADTIMEOUT] [--routermode]
                 [model_param] [port_param]

KoboldCpp Server - Version 1.110

positional arguments:
  model_param           Model file to load (positional)
  port_param            Port to listen on (positional)

optional arguments:
  -h, --help            show this help message and exit
  --model [filenames] [[filenames] ...], -m [filenames] [[filenames] ...]
                        Model file to load. Accepts multiple values if they
                        are URLs.
  --port [portnumber]   Port to listen on. (Defaults to 5001)
  --host [ipaddr]       Host IP to listen on. If this flag is not set, all
                        routable interfaces are accepted.
  --launch              Launches a web browser when load is completed.
  --config [filename]   Load settings from a .kcpps file. Other arguments will
                        be ignored
  --threads [threads], -t [threads]
                        Use a custom number of threads if specified.
                        Otherwise, uses an amount based on CPU cores
  --usecuda [[main GPU ID] [mmq|nommq] [rowsplit] [[main GPU ID] [mmq|nommq] [rowsplit] ...]], --usecublas [[main GPU ID] [mmq|nommq] [rowsplit] [[main GPU ID] [mmq|nommq] [rowsplit] ...]], --usehipblas [[main GPU ID] [mmq|nommq] [rowsplit] [[main GPU ID] [mmq|nommq] [rowsplit] ...]]
                        Use CUDA for GPU Acceleration. Requires CUDA. Enter a
                        number afterwards to select and use 1 GPU. Leaving no
                        number will use all GPUs.
  --usevulkan [[Device IDs] [[Device IDs] ...]]
                        Use Vulkan for GPU Acceleration. Can optionally
                        specify one or more GPU Device ID (e.g. --usevulkan
                        0), leave blank to autodetect.
  --usecpu              Do not use any GPU acceleration (CPU Only)
  --contextsize [256 to 262144], --ctx-size [256 to 262144], -c [256 to 262144]
                        Controls the memory allocated for maximum context
                        size, only change if you need more RAM for big
                        contexts. (default 8192).
  --gpulayers [[GPU layers]], --gpu-layers [[GPU layers]], --n-gpu-layers [[GPU layers]], -ngl [[GPU layers]]
                        Set number of layers to offload to GPU when using GPU.
                        Requires GPU. Set to -1 to try autodetect, set to 0 to
                        disable GPU offload.
  --tensor_split [Ratios] [[Ratios] ...], --tensorsplit [Ratios] [[Ratios] ...], --tensor-split [Ratios] [[Ratios] ...], -ts [Ratios] [[Ratios] ...]
                        For CUDA and Vulkan only, ratio to split tensors
                        across multiple GPUs, space-separated list of
                        proportions, e.g. 7 3
  --autofit, --fit, -fit
                        Automatically attempt to fit the model in the best
                        possible way. Overrides everything else. Experimental.
  --showgui             Always show the GUI instead of launching the model
                        right away when loading settings from a .kcpps file.
  --skiplauncher        Doesn't display or use the GUI launcher. Overrides
                        showgui.

Advanced Commands:
  --version             Prints version and exits.
  --analyze [filename]  Reads the metadata, weight types and tensor names in
                        any GGUF file.
  --maingpu [Device ID], --main-gpu [Device ID], -mg [Device ID]
                        Only used in a multi-gpu setup. Sets the index of the
                        main GPU that will be used.
  --batchsize {-1,16,32,64,128,256,512,1024,2048,4096}, --blasbatchsize {-1,16,32,64,128,256,512,1024,2048,4096}, --batch-size {-1,16,32,64,128,256,512,1024,2048,4096}, -b {-1,16,32,64,128,256,512,1024,2048,4096}
                        Sets the batch size used in batched processing
                        (default 512). Setting it to -1 disables batched mode,
                        but keeps other benefits like GPU offload.
  --blasthreads [threads], --batchthreads [threads], --threadsbatch [threads], --threads-batch [threads]
                        Use a different number of threads during batching if
                        specified. Otherwise, has the same value as --threads
  --lora [lora_filename] [[lora_filename] ...]
                        GGUF models only, applies a lora file on top of model.
  --loramult [amount]   Multiplier for the Text LORA model to be applied.
  --noshift, --no-context-shift
                        If set, do not attempt to Trim and Shift the GGUF
                        context.
  --nofastforward       If set, do not attempt to fast forward GGUF context
                        (always reprocess). Will also enable noshift
  --useswa              If set, allows Sliding Window Attention (SWA) KV
                        Cache, which saves memory but cannot be used with
                        context shifting.
  --smartcache [limit]  Enables intelligent context switching by saving KV
                        cache snapshots to RAM. Requires fast forwarding.
  --ropeconfig [rope-freq-scale] [[rope-freq-base] ...]
                        If set, uses customized RoPE scaling from configured
                        frequency scale and frequency base (e.g. --ropeconfig
                        0.25 10000). Otherwise, uses NTK-Aware scaling set
                        automatically based on context size. For linear rope,
                        simply set the freq-scale and ignore the freq-base
  --overridenativecontext [trained context]
                        Overrides the native trained context of the loaded
                        model with a custom value to be used for Rope scaling.
  --usemmap             If set, uses mmap to load model.
  --usemlock, --mlock   Enables mlock, preventing the RAM used to load the
                        model from being paged out. Not usually recommended.
  --noavx2              Do not use AVX2 instructions, a slower compatibility
                        mode for older devices.
  --failsafe            Use failsafe mode, extremely old CPU compatibility
                        mode that should work on all devices.
  --debugmode [DEBUGMODE]
                        Shows additional debug info in the terminal.
  --onready [shell command]
                        An optional shell command to execute after the model
                        has been loaded.
  --benchmark [[filename]]
                        Do not start server, instead run benchmarks. If
                        filename is provided, appends results to provided
                        file.
  --prompt [prompt], -p [prompt]
                        Passing a prompt string triggers a direct inference,
                        loading the model, outputs the response to stdout and
                        exits. Can be used alone or with benchmark.
  --cli                 Does not launch KoboldCpp HTTP server. Instead,
                        enables KoboldCpp from the command line, accepting
                        interactive console input and displaying responses to
                        the terminal.
  --genlimit [token limit], --promptlimit [token limit]
                        Sets the maximum number of generated tokens, it will
                        restrict all generations to this or lower. Also usable
                        with --prompt or --benchmark.
  --multiuser [limit]   Runs in multiuser mode, which queues incoming requests
                        instead of blocking them.
  --multiplayer         Hosts a shared multiplayer session that others can
                        join.
  --websearch           Enable the local search engine proxy so Web Searches
                        can be done.
  --remotetunnel        Uses Cloudflare to create a remote tunnel, allowing
                        you to access koboldcpp remotely over the internet
                        even behind a firewall.
  --highpriority        Experimental flag. If set, increases the process CPU
                        priority, potentially speeding up generation. Use
                        caution.
  --foreground          Windows only. Sends the terminal to the foreground
                        every time a new prompt is generated. This helps avoid
                        some idle slowdown issues.
  --preloadstory [savefile]
                        Configures a prepared story json save file to be
                        hosted on the server, which frontends (such as
                        KoboldAI Lite) can access over the API.
  --savedatafile [savefile]
                        If enabled, creates or opens a persistent database
                        file on the server, that allows users to save and load
                        their data remotely. A new file is created if it does
                        not exist.
  --quiet               Enable quiet mode, which hides generation inputs and
                        outputs in the terminal. Quiet mode is automatically
                        enabled when running a horde worker.
  --ssl [cert_pem] [[key_pem] ...]
                        Allows all content to be served over SSL instead. A
                        valid UNENCRYPTED SSL cert and key .pem files must be
                        provided
  --nocertify           Allows insecure SSL connections. Use this if you have
                        cert errors and need to bypass certificate
                        restrictions.
  --mmproj [filename]   Select a multimodal projector file for vision models
                        like LLaVA.
  --mmprojcpu, --no-mmproj-offload
                        Force CLIP for Vision mmproj always on CPU.
  --visionmaxres [max px]
                        Clamp MMProj vision maximum allowed resolution.
                        Allowed values are between 512 to 2048 px (default
                        1024).
  --draftmodel [filename], --model-draft [filename], -md [filename]
                        Load a small draft model for speculative decoding. It
                        will be fully offloaded. Vocab must match the main
                        model.
  --draftamount [tokens], --draft-max [tokens], --draft-n [tokens]
                        How many tokens to draft per chunk before verifying
                        results
  --draftgpulayers [layers], --gpu-layers-draft [layers], --n-gpu-layers-draft [layers], -ngld [layers]
                        How many layers to offload to GPU for the draft model
                        (default=full offload)
  --draftgpusplit [Ratios] [[Ratios] ...]
                        GPU layer distribution ratio for draft model
                        (default=same as main). Only works if multi-GPUs
                        selected for MAIN model and tensor_split is set!
  --password [API key]  Enter a password required to use this instance. This
                        key will be required for all text endpoints. Image
                        endpoints are not secured.
  --ratelimit [seconds]
                        If enabled, rate limit generative request by IP
                        address. Each IP can only send a new request once per
                        X seconds.
  --ignoremissing       Ignores all missing non-essential files, just skipping
                        them instead.
  --chatcompletionsadapter [filename]
                        Select an optional ChatCompletions Adapter JSON file
                        to force custom instruct tags.
  --jinja               Enables using jinja chat template formatting for chat
                        completions endpoint. Other endpoints are unaffected.
                        Tool calls are done without jinja.
  --jinja_tools, --jinja-tools, --jinjatools
                        Enables using jinja chat template formatting for chat
                        completions endpoint. Other endpoints are unaffected.
                        Tool calls are done with jinja.
  --noflashattention, --no-flash-attn, -nofa
                        Disables flash attention.
  --lowvram, -nkvo, --no-kv-offload
                        If supported by the backend, do not offload KV to GPU
                        (lowvram mode). Not recommended, will be slow.
  --quantkv [quantization level 0/1/2]
                        Sets the KV cache data type quantization, 0=f16, 1=q8,
                        2=q4. Requires Flash Attention for full effect,
                        otherwise only K cache is quantized.
  --smartcontext        Reserving a portion of context to try processing less
                        frequently. Outdated. Not recommended.
  --unpack destination  Extracts the file contents of the KoboldCpp binary
                        into a target directory.
  --exportconfig [filename]
                        Exports the current selected arguments as a .kcpps
                        settings file
  --exporttemplate [filename]
                        Exports the current selected arguments as a .kcppt
                        template file
  --nomodel             Allows you to launch the GUI alone, without selecting
                        any model.
  --moeexperts [num of experts]
                        How many experts to use for MoE models (default=follow
                        gguf)
  --moecpu [[layers affected]], --n-cpu-moe [[layers affected]], -ncmoe [[layers affected]]
                        Keep the Mixture of Experts (MoE) weights of the first
                        N layers in the CPU. If no value is provided, applies
                        to all layers.
  --defaultgenamt DEFAULTGENAMT
                        How many tokens to generate by default, if not
                        specified. Must be smaller than context size. Usually,
                        your frontend GUI will override this.
  --nobostoken          Prevents BOS token from being added at the start of
                        any prompt. Usually NOT recommended for most models.
  --enableguidance      Enables the use of Classifier-Free-Guidance, which
                        allows the use of negative prompts. Has performance
                        and memory impact.
  --maxrequestsize [size in MB]
                        Specify a max request payload size. Any requests to
                        the server larger than this size will be dropped. Do
                        not change if unsure.
  --overridekv [name=type:value], --override-kv [name=type:value]
                        Override metadata value by key. Separate multiple
                        values with commas. Format is name=type:value. Types:
                        int, float, bool, str
  --overridetensors [tensor name pattern=buffer type], --override-tensor [tensor name pattern=buffer type], -ot [tensor name pattern=buffer type]
                        Override selected backend for specific tensors
                        matching tensor_name_regex_pattern=buffer_type, same
                        as in llama.cpp.
  --singleinstance      Allows this KoboldCpp instance to be shut down by any
                        new instance requesting the same port, preventing
                        duplicate servers from clashing on a port.
  --nopipelineparallel  Disable Pipeline Parallelism. Pipeline Parallelism
                        provides faster multigpu speeds but using more memory,
                        only active for multigpu.
  --gendefaults {"parameter":"value",...}
                        Sets extra default parameters for some fields in API
                        requests, as a JSON string.
  --gendefaultsoverwrite
                        Allow the gendefaults parameters to overwrite the
                        original value in API payloads.
  --mcpfile [mcp json file]
                        Specify path to mcp.json which contains the Cladue
                        Desktop compatible MCP server config.
  --device <dev1,dev2,..>, -dev <dev1,dev2,..>
                        Set llama.cpp compatible device selection override.
                        Comma separated. Overrides normal device choices.
  --downloaddir [directory]
                        Specify a directory that models will be downloaded to
                        or searched from, if unset uses the working directory.
  --autofitpadding [padding in MB]
                        How much spare allowance in MB should autofit reserve?
                        If it's too little, the load might fail.

Horde Worker Commands:
  --hordemodelname [name]
                        Sets your AI Horde display model name.
  --hordeworkername [name]
                        Sets your AI Horde worker name.
  --hordekey [apikey]   Sets your AI Horde API key.
  --hordemaxctx [amount]
                        Sets the maximum context length your worker will
                        accept from an AI Horde job. If 0, matches main
                        context limit.
  --hordegenlen [amount]
                        Sets the maximum number of tokens your worker will
                        generate from an AI horde job.

Image Generation Commands:
  --sdmodel [filename]  Specify an image generation safetensors or gguf model
                        to enable image generation.
  --sdthreads [threads]
                        Use a different number of threads for image generation
                        if specified. Otherwise, has the same value as
                        --threads.
  --sdclamped [[maxres]]
                        If specified, limit generation steps and image size
                        for shared use. Accepts an extra optional parameter
                        that indicates maximum resolution (eg. 768 clamps to
                        768x768, min 512px, disabled if 0).
  --sdclampedsoft [maxres]
                        If specified, limit max image size to curb memory
                        usage. Similar to --sdclamped, but less strict, allows
                        trade-offs between width and height (e.g. 640 would
                        allow 640x640, 512x768 and 768x512 images).
  --sdt5xxl [filename]  Specify a T5-XXL safetensors model. Leave blank if
                        prebaked or unused.
  --sdclip1 [filename], --sdclipl [filename]
                        Specify first safetensors Clip model (SD3 or Flux
                        Clip-L, WAN or QwenImg vision). Leave blank if
                        prebaked or unused.
  --sdclip2 [filename], --sdclipg [filename]
                        Specify second safetensors Clip model (SD3 Clip-G).
                        Leave blank if prebaked or unused.
  --sdphotomaker [filename]
                        PhotoMaker is a model that allows face cloning.
                        Specify a PhotoMaker safetensors model which will be
                        applied replacing img2img. SDXL models only. Leave
                        blank if unused.
  --sdupscaler [filename]
                        You can use ESRGAN as an upscaling model to resize
                        images. Leave blank if unused.
  --sdflashattention    Enables Flash Attention for image generation.
  --sdoffloadcpu        Offload image weights in RAM to save VRAM, swap into
                        VRAM when needed.
  --sdvaecpu            Force VAE to CPU only for image generation.
  --sdclipgpu           Put CLIP and T5 to GPU for image generation.
                        Otherwise, CLIP will use CPU.
  --sdconvdirect {off,vaeonly,full}
                        Enables Conv2D Direct. May improve performance or
                        reduce memory usage. Might crash if not supported by
                        the backend. Can be 'off' (default) to disable, 'full'
                        to turn it on for all operations, or 'vaeonly' to
                        enable only for the VAE.
  --sdvae [filename]    Specify an image generation safetensors VAE which
                        replaces the one in the model.
  --sdvaeauto           Uses a built-in tiny VAE via TAE SD, which is very
                        fast, and fixed bad VAEs.
  --sdquant [[quantization level 0/1/2]]
                        If specified, loads the model quantized to save
                        memory. 0=off, 1=q8, 2=q4
  --sdlora [filename] [[filename] ...]
                        Specify image generation LoRAs safetensors models to
                        be applied. Multiple LoRAs are accepted.
  --sdloramult [amounts] [[amounts] ...]
                        Multipliers for the image LoRA model to be applied.
  --sdtiledvae [maxres]
                        Adjust the automatic VAE tiling trigger for images
                        above this size. 0 disables vae tiling.

Whisper Transcription Commands:
  --whispermodel [filename]
                        Specify a Whisper .bin model to enable Speech-To-Text
                        transcription.

TTS Narration Commands:
  --ttsmodel [filename]
                        Specify the TTS Text-To-Speech GGUF model.
  --ttswavtokenizer [filename]
                        Specify the WavTokenizer GGUF model.
  --ttsgpu              Use the GPU for TTS.
  --ttsmaxlen TTSMAXLEN
                        Limit number of audio tokens generated with TTS.
  --ttsthreads [threads]
                        Use a different number of threads for TTS if
                        specified. Otherwise, has the same value as --threads.
  --ttsdir [directory]  Select directory containing voices for voice cloning.

Music Gen Commands:
  --musicllm [filename]
                        Select music LLM model (e.g acestep-5Hz-lm-0.6B)
  --musicembeddings [filename]
                        Select music embedding model (e.g
                        Qwen3-Embedding-0.6B)
  --musicdiffusion [filename]
                        Select music diffusion (DiT) model (e.g
                        acestep-v15-turbo)
  --musicvae [filename]
                        Select music VAE model
  --musiclowvram        Unload music models when not in use

Embeddings Model Commands:
  --embeddingsmodel [filename]
                        Specify an embeddings model to be loaded for
                        generating embedding vectors.
  --embeddingsmaxctx [amount]
                        Overrides the default maximum supported context of an
                        embeddings model (defaults to trained context).
  --embeddingsgpu       Attempts to offload layers of the embeddings model to
                        GPU. Usually not needed.

Administration Commands:
  --admin               Enables admin mode, allowing you to unload and reload
                        different configurations or models.
  --adminpassword [password]
                        Require a password to access admin functions. You are
                        strongly advised to use one for publically accessible
                        instances!
  --admindir [directory]
                        Specify a directory to look for .kcpps configs in,
                        which can be used to swap models.
  --adminunloadtimeout ADMINUNLOADTIMEOUT
                        Set an idle timeout in seconds after which KoboldCpp
                        will automatically unload the current model.
  --routermode          Router mode uses a reverse proxy router, allowing you
                        to easily hotswap models and configs within a single
                        request. Requires admin mode.