| | --- |
| | language: |
| | - en |
| | base_model: |
| | - katanemo/Arch-Function-3B |
| | - cognitivecomputations/Dolphin3.0-Qwen2.5-3b |
| | pipeline_tag: text-generation |
| | tags: |
| | - uncensored |
| | - function_calling |
| | - tool_use |
| | --- |
| | # Ramius |
| |
|
| | <!-- Provide a quick summary of what the model is/does. --> |
| |
|
| | This is Ramius, an uncensored function calling model. |
| |
|
| | ### Model Description |
| |
|
| | I needed an LLM for Home Assistant that is small and performant and I wanted one with some personality. |
| | Qwen2.5-3B is small, fast and can call functions pretty well. But it's [REDACTED], and doesn't like to roleplay. |
| | Arch-Function-3B is fantastic at calling functions, and absolutely nothing else. |
| | Dolphin3.0-Qwen2.5-3b is great at roleplay and refuses to refuse anything. But it sucks at calling functions. |
| |
|
| | So I created Ramius with MergeKit to try and get the best of both. |
| | Plus I'm GPU poor and can't train. (Intel ARC cards come with buyer's remorse at no extra charge!) |
| |
|
| | The result is... mediocre. It correctly calls functions most of the time, but it tends to hallucinate function responses instead of calling the actual function. |
| | But it does stay in character. YMMV. |
| |
|
| | The name comes from Marko Ramius, a fictional communist submarine commander who defects to the United States in Tom Clancy's The Hunt for Red October. |
| | He's a former communist and the name sounded cool. |
| |
|
| | I've included the F16 and Q4_0 weights. |
| | |
| | - **Developed by:** Other people's hard work. |
| | - **Funded by [optional]:** Also other people's hard work. |
| | - **Shared by [optional]:** Me. |
| | - **Model type:** Autoregressive transformer. |
| | - **Language(s) (NLP):** English, and others, probably. |
| | - **License:** [More Information Needed] |
| | |
| | ### Model Sources [optional] |
| | |
| | Created with MergeKit. |
| | |
| | models: |
| | - model: katanemo/Arch-Function-3B |
| | lambda: 1.0 |
| | select_topk: 0.4 |
| | weight: 0.7 |
| | - model: cognitivecomputations/Dolphin3.0-Qwen2.5-3b |
| | density: 1.0 |
| | lambda: 1.0 |
| | select_topk: 0.6 |
| | weight: 0.3 |
| | merge_method: sce |
| | base_model: katanemo/Arch-Function-3B |
| | parameters: |
| | int8_mask: true |
| | normalize: true |
| | dtype: bfloat16 |
| | |
| |
|
| | ## Bias, Risks, and Limitations |
| |
|
| | This is uncensored and does hallucinate. frequently. |
| |
|
| | [More Information Needed] |
| |
|
| | ### Recommendations |
| |
|
| | I use this with Ollama and Home Assistant via the Extended OpenAI conversation integration. |
| | Works best with a top P of around 0.95 and temperature around 0.85. |
| |
|
| | I also recommend you DO NOT put your entity states in your system prompt, and instead write functions to get the information. |
| | This will keep your system prompt static and more easily cached, which should reduce prompt processing time. |
| |
|
| |
|
