jungsin3 commited on
Commit
841f53f
·
verified ·
1 Parent(s): 5bad2b0

How to Start: Mac uses the 4B MLX 4-bit build

Browse files
Files changed (1) hide show
  1. README.md +3 -5
README.md CHANGED
@@ -95,16 +95,14 @@ The fastest way to try KORMo-Web is the **[KORMo-Web Chrome extension](https://g
95
 
96
  ### Mac (Apple silicon)
97
 
98
- Run the model with `vllm-mlx`. Keep this terminal open.
99
 
100
  ```bash
101
  uv venv --python 3.12 && source .venv/bin/activate
102
  uv pip install vllm-mlx
103
- vllm-mlx serve KORMo-VL/KORMo-Web-4B --port 8010 --mllm
104
  ```
105
 
106
- For a lighter, faster model on a Mac with less memory, use the 4-bit MLX build `KORMo-VL/KORMo-Web-4B-mlx-4bit` instead (the extension's default).
107
-
108
  ### NVIDIA GPU (local PC or server)
109
 
110
  Serve the model with vLLM. Much faster, and one server can be shared.
@@ -123,7 +121,7 @@ Check that it is up with `curl http://localhost:8010/v1/models` (Mac) or `curl h
123
 
124
  **3. Point the extension at your model.** Open the panel's **개발자 설정** (Developer settings).
125
 
126
- - **Mac:** keep the server URL `http://localhost:8010/v1` and set the model name to `KORMo-VL/KORMo-Web-4B`. `tool_choice` stays `auto`.
127
  - **NVIDIA GPU:** set the server URL to `http://<server>:8000/v1` (or `localhost`), the model name to `KORMo-Web-4B`, and `tool_choice` to `none`.
128
 
129
  **4. Discord (optional).** Run `node relay/discord-relay.mjs` in the cloned folder and follow the prompts to send tasks to the agent by mentioning a bot in a channel.
 
95
 
96
  ### Mac (Apple silicon)
97
 
98
+ Run the 4-bit MLX build of KORMo-Web-4B with `vllm-mlx`. Simple, but slower. Keep this terminal open.
99
 
100
  ```bash
101
  uv venv --python 3.12 && source .venv/bin/activate
102
  uv pip install vllm-mlx
103
+ vllm-mlx serve KORMo-VL/KORMo-Web-4B-mlx-4bit --port 8010 --mllm
104
  ```
105
 
 
 
106
  ### NVIDIA GPU (local PC or server)
107
 
108
  Serve the model with vLLM. Much faster, and one server can be shared.
 
121
 
122
  **3. Point the extension at your model.** Open the panel's **개발자 설정** (Developer settings).
123
 
124
+ - **Mac:** nothing to change. The defaults (`http://localhost:8010/v1`, `KORMo-VL/KORMo-Web-4B-mlx-4bit`, `tool_choice` `auto`) match the command above.
125
  - **NVIDIA GPU:** set the server URL to `http://<server>:8000/v1` (or `localhost`), the model name to `KORMo-Web-4B`, and `tool_choice` to `none`.
126
 
127
  **4. Discord (optional).** Run `node relay/discord-relay.mjs` in the cloned folder and follow the prompts to send tasks to the agent by mentioning a bot in a channel.