Upload 9 files

.gitattributes

@@ -33,3 +33,5 @@ saved_model/**/* filter=lfs diff=lfs merge=lfs -text
 *.zip filter=lfs diff=lfs merge=lfs -text
 *.zst filter=lfs diff=lfs merge=lfs -text
 *tfevents* filter=lfs diff=lfs merge=lfs -text
+politscanner.pdf filter=lfs diff=lfs merge=lfs -text
+Qwen3-1.7B-PolitScanner-Q5_K_S.gguf filter=lfs diff=lfs merge=lfs -text

Qwen3-1.7B-PolitScanner-Q5_K_S.gguf

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:45b90f2ed2ee072908a7d8a61f0c5a0a6ebc1e1793c72d70887c290cad396e15
+size 1230579264

README.md

@@ -1,3 +1,142 @@
----
-license: mit
----
+# PolitScanner
+
+## Abstract
+
+Swiss politicians lie.
+And they mostly get away with it.
+"One reason for this is that fact-checks, which can only be carried out retrospectively, are surprisingly ineffective. Listeners still remember the false information. The correction is forgotten." — Philipp Gerlach.
+This technical report provides a comprehensive overview of the artificial intelligence components of the PolitScanner project.
+It aims to automatically detect false narratives and fake news in the speeches of Swiss politicians while avoiding the inaccuracies inherent in Large Language Models.
+
+The training code can be found on [GitHub](https://github.com/lenamerkli/PolitScanner).
+
+This is an entry for the Swiss AI Challenge 2025.
+More information can be found at [www.ki-challenge.ch](https://www.ki-challenge.ch/).
+
+## Table of Contents
+
+0. [Abstract](#abstract)
+1. [Paper](#paper)
+2. [Installation](#installation)
+3. [Usage](#usage)
+4. [License](#license)
+5. [Citation](#citation)
+
+## Paper
+
+Read the full paper [here](https://huggingface.co/lenamerkli/PolitScanner/blob/main/PolitScanner.pdf).
+
+## Installation
+
+**Note: inference only**
+
+This installation guide is for Nobara Linux.
+Other distributions should work as well.
+
+For the full installation guide, see [the development readme](https://github.com/lenamerkli/PolitScanner/blob/main/README.md).
+
+### Increase memlock
+
+Add (or update) the following lines to `/etc/security/limits.conf`:
+```text
+* soft memlock 50331648
+* hard memlock 50331648
+```
+
+### Git
+
+Install git:
+
+```shell
+sudo dnf install git
+```
+
+Clone the PolitScanner repository:
+
+```shell
+git clone https://huggingface.co/lenamerkli/PolitScanner
+cd PolitScanner
+```
+
+### Python
+
+Install Python version 3.12.10 with the following command:
+
+```shell
+sudo dnf install python3.12-0:3.12.10-1.fc41.x86_64
+```
+
+Install the Python virtual environment package:
+
+```shell
+sudo dnf install python3-virtualenv
+```
+
+Create the virtual environment:
+
+```shell
+./create_venv.sh
+```
+
+Activate the virtual environment:
+
+```shell
+source .venv/bin/activate
+```
+
+### llama.cpp
+
+If llama.cpp is not installed, check the [development readme](https://github.com/lenamerkli/PolitScanner/blob/main/README.md) for instructions.
+
+### Download models
+
+Run the downloader:
+
+```shell
+python3 download_ggufs.py
+```
+
+Move the PolitScanner model:
+
+```shell
+mv ./Qwen3-1.7B-PolitScanner-Q5_K_S.gguf /opt/llms/Qwen3-1.7B-PolitScanner-Q5_K_S.gguf
+```
+
+## Usage
+
+Copy the political speech (preferably in swiss high german) to the `input.txt` file.
+
+Run the program:
+
+```shell
+python3 main.py
+```
+
+The output will be written to the `output.txt` file.
+
+## License
+
+[MIT License](https://github.com/lenamerkli/PolitScanner/blob/main/LICENSE)
+
+## Citation
+
+bibtex:
+```bibtex
+@misc{merkli2025politscanner,
+    title = {PolitScanner: Automatic Detection of common Incorrect Statements in Speeches of Swiss Politicians},
+    author = {Lena Merkli},
+    year = {2025},
+    month = {07},
+    url = {https://huggingface.co/lenamerkli/PolitScanner}
+}
+```
+biblatex:
+```biblatex
+@online{merkli2025politscanner,
+    title = {PolitScanner: Automatic Detection of common Incorrect Statements in Speeches of Swiss Politicians},
+    author = {Lena Merkli},
+    year = {2025},
+    month = {07},
+    url = {https://huggingface.co/lenamerkli/PolitScanner}
+}
+```

create_venv.sh

@@ -0,0 +1,7 @@
+if [ ! -d .venv ]; then
+    /usr/bin/python3.12 -m venv .venv
+fi
+
+# install packages
+.venv/bin/pip3 install --trusted-host pypi.org --trusted-host files.pythonhosted.org --upgrade pip
+.venv/bin/pip3 install wheel==0.46.1 setuptools==79.0.0 flask==3.1.0 requests==2.32.3 tqdm==4.67.1 chromadb==1.0.7 certifi==2025.6.15

download_ggufs.py

@@ -0,0 +1,53 @@
+import sys
+sys.path.append('/home/lena/Documents/python/PolitScanner/util')
+
+from pathlib import Path
+from os.path import join
+from requests import get
+from shutil import copyfile
+
+
+MODELS = {
+    'DeepSeek-R1-0528-Qwen3-8B-Q5_K_L.gguf': 'https://huggingface.co/bartowski/deepseek-ai_DeepSeek-R1-0528-Qwen3-8B-GGUF/resolve/main/deepseek-ai_DeepSeek-R1-0528-Qwen3-8B-Q5_K_L.gguf',
+    'Ministral-8B-Instruct-2410-Q4_K_S.gguf': 'https://huggingface.co/bartowski/Ministral-8B-Instruct-2410-GGUF/resolve/main/Ministral-8B-Instruct-2410-Q4_K_S.gguf',
+    'Qwen3-30B-A3B-Q5_K_M.gguf': 'https://huggingface.co/bartowski/Qwen_Qwen3-30B-A3B-GGUF/resolve/main/Qwen_Qwen3-30B-A3B-Q5_K_M.gguf',
+    'Qwen3-8B-Q5_K_M.gguf': 'https://huggingface.co/bartowski/Qwen_Qwen3-8B-GGUF/resolve/main/Qwen_Qwen3-8B-Q5_K_M.gguf',
+    'Qwen3-32B-Q4_K_S.gguf': 'https://huggingface.co/unsloth/Qwen3-32B-GGUF/resolve/main/Qwen3-32B-Q4_K_S.gguf',
+}
+
+
+def download_file(url: str, directory: str, filename: str = None):
+    Path(directory).mkdir(parents=True, exist_ok=True)
+    if filename is None:
+        filename = url.split('/')[-1].split('?')[0]
+    filepath = join(directory, filename)
+    with get(url, stream=True) as r:
+        r.raise_for_status()
+        with open(filepath, 'wb') as f:
+            for chunk in r.iter_content(chunk_size=4 * 1024 * 1024):
+                f.write(chunk)
+    return filepath
+
+
+def main() -> None:
+    Path('/opt/llms').mkdir(exist_ok=True, parents=True)
+    question = 'Which of these following models do you want to download?'
+    for i, model in enumerate(MODELS.keys()):
+        question += f"\n[{i + 1}] {model}"
+    response = input(question).replace(' ', '').replace(';', ',')
+    parsed = [int(x) for x in response.split(',')]
+    if len(parsed) > 0:
+        copyfile('./index.json', '/opt/llms/index.json')
+    for i in parsed:
+        model = list(MODELS.keys())[i - 1]
+        url = MODELS[model]
+        print(f"Downloading {model}")
+        download_file(url, '/opt/llms/', model)
+        print(f"Downloaded {model}")
+    print('Done')
+
+
+if __name__ == '__main__':
+    main()
+
+

index.json

@@ -0,0 +1,116 @@
+{
+    "Qwen3-30B-A3B": {
+        "parameters": 30532122624,
+        "context": 32768,
+        "layers": 48,
+        "thinking": true,
+        "optional_thinking": true,
+        "system_message": "",
+        "chat_template": "{%- if tools %}\n    {{- '<|im_start|>system\\n' }}\n    {%- if messages[0].role == 'system' %}\n        {{- messages[0].content + '\\n\\n' }}\n    {%- endif %}\n    {{- \"# Tools\\n\\nYou may call one or more functions to assist with the user query.\\n\\nYou are provided with function signatures within <tools></tools> XML tags:\\n<tools>\" }}\n    {%- for tool in tools %}\n        {{- \"\\n\" }}\n        {{- tool | tojson }}\n    {%- endfor %}\n    {{- \"\\n</tools>\\n\\nFor each function call, return a json object with function name and arguments within <tool_call></tool_call> XML tags:\\n<tool_call>\\n{\\\"name\\\": <function-name>, \\\"arguments\\\": <args-json-object>}\\n</tool_call><|im_end|>\\n\" }}\n{%- else %}\n    {%- if messages[0].role == 'system' %}\n        {{- '<|im_start|>system\\n' + messages[0].content + '<|im_end|>\\n' }}\n    {%- endif %}\n{%- endif %}\n{%- set ns = namespace(multi_step_tool=true, last_query_index=messages|length - 1) %}\n{%- for message in messages[::-1] %}\n    {%- set index = (messages|length - 1) - loop.index0 %}\n    {%- if ns.multi_step_tool and message.role == \"user\" and message.content is string and not(message.content.startswith('<tool_response>') and message.content.endswith('</tool_response>')) %}\n        {%- set ns.multi_step_tool = false %}\n        {%- set ns.last_query_index = index %}\n    {%- endif %}\n{%- endfor %}\n{%- for message in messages %}\n    {%- if message.content is string %}\n        {%- set content = message.content %}\n    {%- else %}\n        {%- set content = '' %}\n    {%- endif %}\n    {%- if (message.role == \"user\") or (message.role == \"system\" and not loop.first) %}\n        {{- '<|im_start|>' + message.role + '\\n' + content + '<|im_end|>' + '\\n' }}\n    {%- elif message.role == \"assistant\" %}\n        {%- set reasoning_content = '' %}\n        {%- if message.reasoning_content is string %}\n            {%- set reasoning_content = message.reasoning_content %}\n        {%- else %}\n            {%- if '</think>' in content %}\n                {%- set reasoning_content = content.split('</think>')[0].rstrip('\\n').split('<think>')[-1].lstrip('\\n') %}\n                {%- set content = content.split('</think>')[-1].lstrip('\\n') %}\n            {%- endif %}\n        {%- endif %}\n        {%- if loop.index0 > ns.last_query_index %}\n            {%- if loop.last or (not loop.last and reasoning_content) %}\n                {{- '<|im_start|>' + message.role + '\\n<think>\\n' + reasoning_content.strip('\\n') + '\\n</think>\\n\\n' + content.lstrip('\\n') }}\n            {%- else %}\n                {{- '<|im_start|>' + message.role + '\\n' + content }}\n            {%- endif %}\n        {%- else %}\n            {{- '<|im_start|>' + message.role + '\\n' + content }}\n        {%- endif %}\n        {%- if message.tool_calls %}\n            {%- for tool_call in message.tool_calls %}\n                {%- if (loop.first and content) or (not loop.first) %}\n                    {{- '\\n' }}\n                {%- endif %}\n                {%- if tool_call.function %}\n                    {%- set tool_call = tool_call.function %}\n                {%- endif %}\n                {{- '<tool_call>\\n{\"name\": \"' }}\n                {{- tool_call.name }}\n                {{- '\", \"arguments\": ' }}\n                {%- if tool_call.arguments is string %}\n                    {{- tool_call.arguments }}\n                {%- else %}\n                    {{- tool_call.arguments | tojson }}\n                {%- endif %}\n                {{- '}\\n</tool_call>' }}\n            {%- endfor %}\n        {%- endif %}\n        {{- '<|im_end|>\\n' }}\n    {%- elif message.role == \"tool\" %}\n        {%- if loop.first or (messages[loop.index0 - 1].role != \"tool\") %}\n            {{- '<|im_start|>user' }}\n        {%- endif %}\n        {{- '\\n<tool_response>\\n' }}\n        {{- content }}\n        {{- '\\n</tool_response>' }}\n        {%- if loop.last or (messages[loop.index0 + 1].role != \"tool\") %}\n            {{- '<|im_end|>\\n' }}\n        {%- endif %}\n    {%- endif %}\n{%- endfor %}\n{%- if add_generation_prompt %}\n    {{- '<|im_start|>assistant\\n' }}\n    {%- if enable_thinking is defined and enable_thinking is false %}\n        {{- '<think>\\n\\n</think>\\n\\n' }}\n    {%- endif %}\n{%- endif %}",
+        "sampling": {
+            "temperature": 0.7,
+            "top_p": 0.8,
+            "top_k": 20,
+            "min_p": 0
+        },
+        "sampling_thinking": {
+            "temperature": 0.6,
+            "top_p": 0.95,
+            "top_k": 20,
+            "min_p": 0
+        }
+    },
+    "Qwen3-1.7B": {
+        "parameters": 2031739904,
+        "context": 32768,
+        "layers": 28,
+        "thinking": true,
+        "optional_thinking": true,
+        "system_message": "",
+        "chat_template": "{%- if tools %}\n    {{- '<|im_start|>system\\n' }}\n    {%- if messages[0].role == 'system' %}\n        {{- messages[0].content + '\\n\\n' }}\n    {%- endif %}\n    {{- \"# Tools\\n\\nYou may call one or more functions to assist with the user query.\\n\\nYou are provided with function signatures within <tools></tools> XML tags:\\n<tools>\" }}\n    {%- for tool in tools %}\n        {{- \"\\n\" }}\n        {{- tool | tojson }}\n    {%- endfor %}\n    {{- \"\\n</tools>\\n\\nFor each function call, return a json object with function name and arguments within <tool_call></tool_call> XML tags:\\n<tool_call>\\n{\\\"name\\\": <function-name>, \\\"arguments\\\": <args-json-object>}\\n</tool_call><|im_end|>\\n\" }}\n{%- else %}\n    {%- if messages[0].role == 'system' %}\n        {{- '<|im_start|>system\\n' + messages[0].content + '<|im_end|>\\n' }}\n    {%- endif %}\n{%- endif %}\n{%- set ns = namespace(multi_step_tool=true, last_query_index=messages|length - 1) %}\n{%- for message in messages[::-1] %}\n    {%- set index = (messages|length - 1) - loop.index0 %}\n    {%- if ns.multi_step_tool and message.role == \"user\" and message.content is string and not(message.content.startswith('<tool_response>') and message.content.endswith('</tool_response>')) %}\n        {%- set ns.multi_step_tool = false %}\n        {%- set ns.last_query_index = index %}\n    {%- endif %}\n{%- endfor %}\n{%- for message in messages %}\n    {%- if message.content is string %}\n        {%- set content = message.content %}\n    {%- else %}\n        {%- set content = '' %}\n    {%- endif %}\n    {%- if (message.role == \"user\") or (message.role == \"system\" and not loop.first) %}\n        {{- '<|im_start|>' + message.role + '\\n' + content + '<|im_end|>' + '\\n' }}\n    {%- elif message.role == \"assistant\" %}\n        {%- set reasoning_content = '' %}\n        {%- if message.reasoning_content is string %}\n            {%- set reasoning_content = message.reasoning_content %}\n        {%- else %}\n            {%- if '</think>' in content %}\n                {%- set reasoning_content = content.split('</think>')[0].rstrip('\\n').split('<think>')[-1].lstrip('\\n') %}\n                {%- set content = content.split('</think>')[-1].lstrip('\\n') %}\n            {%- endif %}\n        {%- endif %}\n        {%- if loop.index0 > ns.last_query_index %}\n            {%- if loop.last or (not loop.last and reasoning_content) %}\n                {{- '<|im_start|>' + message.role + '\\n<think>\\n' + reasoning_content.strip('\\n') + '\\n</think>\\n\\n' + content.lstrip('\\n') }}\n            {%- else %}\n                {{- '<|im_start|>' + message.role + '\\n' + content }}\n            {%- endif %}\n        {%- else %}\n            {{- '<|im_start|>' + message.role + '\\n' + content }}\n        {%- endif %}\n        {%- if message.tool_calls %}\n            {%- for tool_call in message.tool_calls %}\n                {%- if (loop.first and content) or (not loop.first) %}\n                    {{- '\\n' }}\n                {%- endif %}\n                {%- if tool_call.function %}\n                    {%- set tool_call = tool_call.function %}\n                {%- endif %}\n                {{- '<tool_call>\\n{\"name\": \"' }}\n                {{- tool_call.name }}\n                {{- '\", \"arguments\": ' }}\n                {%- if tool_call.arguments is string %}\n                    {{- tool_call.arguments }}\n                {%- else %}\n                    {{- tool_call.arguments | tojson }}\n                {%- endif %}\n                {{- '}\\n</tool_call>' }}\n            {%- endfor %}\n        {%- endif %}\n        {{- '<|im_end|>\\n' }}\n    {%- elif message.role == \"tool\" %}\n        {%- if loop.first or (messages[loop.index0 - 1].role != \"tool\") %}\n            {{- '<|im_start|>user' }}\n        {%- endif %}\n        {{- '\\n<tool_response>\\n' }}\n        {{- content }}\n        {{- '\\n</tool_response>' }}\n        {%- if loop.last or (messages[loop.index0 + 1].role != \"tool\") %}\n            {{- '<|im_end|>\\n' }}\n        {%- endif %}\n    {%- endif %}\n{%- endfor %}\n{%- if add_generation_prompt %}\n    {{- '<|im_start|>assistant\\n' }}\n    {%- if enable_thinking is defined and enable_thinking is false %}\n        {{- '<think>\\n\\n</think>\\n\\n' }}\n    {%- endif %}\n{%- endif %}",
+        "sampling": {
+            "temperature": 0.7,
+            "top_p": 0.8,
+            "top_k": 20,
+            "min_p": 0
+        },
+        "sampling_thinking": {
+            "temperature": 0.6,
+            "top_p": 0.95,
+            "top_k": 20,
+            "min_p": 0
+        }
+    },
+    "Qwen3-8B": {
+        "parameters": 8190735360,
+        "context": 32768,
+        "layers": 36,
+        "thinking": true,
+        "optional_thinking": true,
+        "system_message": "",
+        "chat_template": "{%- if tools %}\n    {{- '<|im_start|>system\\n' }}\n    {%- if messages[0].role == 'system' %}\n        {{- messages[0].content + '\\n\\n' }}\n    {%- endif %}\n    {{- \"# Tools\\n\\nYou may call one or more functions to assist with the user query.\\n\\nYou are provided with function signatures within <tools></tools> XML tags:\\n<tools>\" }}\n    {%- for tool in tools %}\n        {{- \"\\n\" }}\n        {{- tool | tojson }}\n    {%- endfor %}\n    {{- \"\\n</tools>\\n\\nFor each function call, return a json object with function name and arguments within <tool_call></tool_call> XML tags:\\n<tool_call>\\n{\\\"name\\\": <function-name>, \\\"arguments\\\": <args-json-object>}\\n</tool_call><|im_end|>\\n\" }}\n{%- else %}\n    {%- if messages[0].role == 'system' %}\n        {{- '<|im_start|>system\\n' + messages[0].content + '<|im_end|>\\n' }}\n    {%- endif %}\n{%- endif %}\n{%- set ns = namespace(multi_step_tool=true, last_query_index=messages|length - 1) %}\n{%- for message in messages[::-1] %}\n    {%- set index = (messages|length - 1) - loop.index0 %}\n    {%- if ns.multi_step_tool and message.role == \"user\" and message.content is string and not(message.content.startswith('<tool_response>') and message.content.endswith('</tool_response>')) %}\n        {%- set ns.multi_step_tool = false %}\n        {%- set ns.last_query_index = index %}\n    {%- endif %}\n{%- endfor %}\n{%- for message in messages %}\n    {%- if message.content is string %}\n        {%- set content = message.content %}\n    {%- else %}\n        {%- set content = '' %}\n    {%- endif %}\n    {%- if (message.role == \"user\") or (message.role == \"system\" and not loop.first) %}\n        {{- '<|im_start|>' + message.role + '\\n' + content + '<|im_end|>' + '\\n' }}\n    {%- elif message.role == \"assistant\" %}\n        {%- set reasoning_content = '' %}\n        {%- if message.reasoning_content is string %}\n            {%- set reasoning_content = message.reasoning_content %}\n        {%- else %}\n            {%- if '</think>' in content %}\n                {%- set reasoning_content = content.split('</think>')[0].rstrip('\\n').split('<think>')[-1].lstrip('\\n') %}\n                {%- set content = content.split('</think>')[-1].lstrip('\\n') %}\n            {%- endif %}\n        {%- endif %}\n        {%- if loop.index0 > ns.last_query_index %}\n            {%- if loop.last or (not loop.last and reasoning_content) %}\n                {{- '<|im_start|>' + message.role + '\\n<think>\\n' + reasoning_content.strip('\\n') + '\\n</think>\\n\\n' + content.lstrip('\\n') }}\n            {%- else %}\n                {{- '<|im_start|>' + message.role + '\\n' + content }}\n            {%- endif %}\n        {%- else %}\n            {{- '<|im_start|>' + message.role + '\\n' + content }}\n        {%- endif %}\n        {%- if message.tool_calls %}\n            {%- for tool_call in message.tool_calls %}\n                {%- if (loop.first and content) or (not loop.first) %}\n                    {{- '\\n' }}\n                {%- endif %}\n                {%- if tool_call.function %}\n                    {%- set tool_call = tool_call.function %}\n                {%- endif %}\n                {{- '<tool_call>\\n{\"name\": \"' }}\n                {{- tool_call.name }}\n                {{- '\", \"arguments\": ' }}\n                {%- if tool_call.arguments is string %}\n                    {{- tool_call.arguments }}\n                {%- else %}\n                    {{- tool_call.arguments | tojson }}\n                {%- endif %}\n                {{- '}\\n</tool_call>' }}\n            {%- endfor %}\n        {%- endif %}\n        {{- '<|im_end|>\\n' }}\n    {%- elif message.role == \"tool\" %}\n        {%- if loop.first or (messages[loop.index0 - 1].role != \"tool\") %}\n            {{- '<|im_start|>user' }}\n        {%- endif %}\n        {{- '\\n<tool_response>\\n' }}\n        {{- content }}\n        {{- '\\n</tool_response>' }}\n        {%- if loop.last or (messages[loop.index0 + 1].role != \"tool\") %}\n            {{- '<|im_end|>\\n' }}\n        {%- endif %}\n    {%- endif %}\n{%- endfor %}\n{%- if add_generation_prompt %}\n    {{- '<|im_start|>assistant\\n' }}\n    {%- if enable_thinking is defined and enable_thinking is false %}\n        {{- '<think>\\n\\n</think>\\n\\n' }}\n    {%- endif %}\n{%- endif %}",
+        "sampling": {
+            "temperature": 0.7,
+            "top_p": 0.8,
+            "top_k": 20,
+            "min_p": 0
+        },
+        "sampling_thinking": {
+            "temperature": 0.6,
+            "top_p": 0.95,
+            "top_k": 20,
+            "min_p": 0
+        }
+    },
+    "Qwen3-32B": {
+        "parameters": 32762123264,
+        "context": 32768,
+        "layers": 64,
+        "thinking": true,
+        "optional_thinking": true,
+        "system_message": "",
+        "chat_template": "{%- if tools %}\n    {{- '<|im_start|>system\\n' }}\n    {%- if messages[0].role == 'system' %}\n        {{- messages[0].content + '\\n\\n' }}\n    {%- endif %}\n    {{- \"# Tools\\n\\nYou may call one or more functions to assist with the user query.\\n\\nYou are provided with function signatures within <tools></tools> XML tags:\\n<tools>\" }}\n    {%- for tool in tools %}\n        {{- \"\\n\" }}\n        {{- tool | tojson }}\n    {%- endfor %}\n    {{- \"\\n</tools>\\n\\nFor each function call, return a json object with function name and arguments within <tool_call></tool_call> XML tags:\\n<tool_call>\\n{\\\"name\\\": <function-name>, \\\"arguments\\\": <args-json-object>}\\n</tool_call><|im_end|>\\n\" }}\n{%- else %}\n    {%- if messages[0].role == 'system' %}\n        {{- '<|im_start|>system\\n' + messages[0].content + '<|im_end|>\\n' }}\n    {%- endif %}\n{%- endif %}\n{%- set ns = namespace(multi_step_tool=true, last_query_index=messages|length - 1) %}\n{%- for message in messages[::-1] %}\n    {%- set index = (messages|length - 1) - loop.index0 %}\n    {%- if ns.multi_step_tool and message.role == \"user\" and not(message.content.startswith('<tool_response>') and message.content.endswith('</tool_response>')) %}\n        {%- set ns.multi_step_tool = false %}\n        {%- set ns.last_query_index = index %}\n    {%- endif %}\n{%- endfor %}\n{%- for message in messages %}\n    {%- if (message.role == \"user\") or (message.role == \"system\" and not loop.first) %}\n        {{- '<|im_start|>' + message.role + '\\n' + message.content + '<|im_end|>' + '\\n' }}\n    {%- elif message.role == \"assistant\" %}\n        {%- set content = message.content %}\n        {%- set reasoning_content = '' %}\n        {%- if message.reasoning_content is defined and message.reasoning_content is not none %}\n            {%- set reasoning_content = message.reasoning_content %}\n        {%- else %}\n            {%- if '</think>' in message.content %}\n                {%- set content = message.content.split('</think>')[-1].lstrip('\\n') %}\n                {%- set reasoning_content = message.content.split('</think>')[0].rstrip('\\n').split('<think>')[-1].lstrip('\\n') %}\n            {%- endif %}\n        {%- endif %}\n        {%- if loop.index0 > ns.last_query_index %}\n            {%- if loop.last or (not loop.last and reasoning_content) %}\n                {{- '<|im_start|>' + message.role + '\\n<think>\\n' + reasoning_content.strip('\\n') + '\\n</think>\\n\\n' + content.lstrip('\\n') }}\n            {%- else %}\n                {{- '<|im_start|>' + message.role + '\\n' + content }}\n            {%- endif %}\n        {%- else %}\n            {{- '<|im_start|>' + message.role + '\\n' + content }}\n        {%- endif %}\n        {%- if message.tool_calls %}\n            {%- for tool_call in message.tool_calls %}\n                {%- if (loop.first and content) or (not loop.first) %}\n                    {{- '\\n' }}\n                {%- endif %}\n                {%- if tool_call.function %}\n                    {%- set tool_call = tool_call.function %}\n                {%- endif %}\n                {{- '<tool_call>\\n{\"name\": \"' }}\n                {{- tool_call.name }}\n                {{- '\", \"arguments\": ' }}\n                {%- if tool_call.arguments is string %}\n                    {{- tool_call.arguments }}\n                {%- else %}\n                    {{- tool_call.arguments | tojson }}\n                {%- endif %}\n                {{- '}\\n</tool_call>' }}\n            {%- endfor %}\n        {%- endif %}\n        {{- '<|im_end|>\\n' }}\n    {%- elif message.role == \"tool\" %}\n        {%- if loop.first or (messages[loop.index0 - 1].role != \"tool\") %}\n            {{- '<|im_start|>user' }}\n        {%- endif %}\n        {{- '\\n<tool_response>\\n' }}\n        {{- message.content }}\n        {{- '\\n</tool_response>' }}\n        {%- if loop.last or (messages[loop.index0 + 1].role != \"tool\") %}\n            {{- '<|im_end|>\\n' }}\n        {%- endif %}\n    {%- endif %}\n{%- endfor %}\n{%- if add_generation_prompt %}\n    {{- '<|im_start|>assistant\\n' }}\n    {%- if enable_thinking is defined and enable_thinking is false %}\n        {{- '<think>\\n\\n</think>\\n\\n' }}\n    {%- endif %}\n{%- endif %}",
+        "sampling": {
+            "temperature": 0.7,
+            "top_p": 0.8,
+            "top_k": 20,
+            "min_p": 0
+        },
+        "sampling_thinking": {
+            "temperature": 0.6,
+            "top_p": 0.95,
+            "top_k": 20,
+            "min_p": 0
+        }
+    },
+    "Deepseek-R1-0528-Qwen3-8B": {
+        "parameters": 8190735360,
+        "context": 32768,
+        "layers": 36,
+        "thinking": true,
+        "optional_thinking": false,
+        "system_message": "该助手为DeepSeek-R1，由深度求索公司创造。",
+        "chat_template": "{% if not add_generation_prompt is defined %}{% set add_generation_prompt = false %}{% endif %}{% set ns = namespace(is_first=false, is_tool=false, is_output_first=true, system_prompt='', is_first_sp=true, is_last_user=false) %}{%- for message in messages %}{%- if message['role'] == 'system' %}{%- if ns.is_first_sp %}{% set ns.system_prompt = ns.system_prompt + message['content'] %}{% set ns.is_first_sp = false %}{%- else %}{% set ns.system_prompt = ns.system_prompt + '\n\n' + message['content'] %}{%- endif %}{%- endif %}{%- endfor %}{{ bos_token }}{{ ns.system_prompt }}{%- for message in messages %}{% set content = message['content'] %}{%- if message['role'] == 'user' %}{%- set ns.is_tool = false -%}{%- set ns.is_first = false -%}{%- set ns.is_last_user = true -%}{{'<｜User｜>' + content + '<｜Assistant｜>'}}{%- endif %}{%- if message['role'] == 'assistant' %}{% if '</think>' in content %}{% set content = content.split('</think>')[-1] %}{% endif %}{% endif %}{%- if message['role'] == 'assistant' and message['tool_calls'] is defined and message['tool_calls'] is not none %}{%- set ns.is_last_user = false -%}{%- if ns.is_tool %}{{'<｜tool▁outputs▁end｜>'}}{%- endif %}{%- set ns.is_first = false %}{%- set ns.is_tool = false -%}{%- set ns.is_output_first = true %}{%- for tool in message['tool_calls'] %}{%- if not ns.is_first %}{%- if content is none %}{{'<｜tool▁calls▁begin｜><｜tool▁call▁begin｜>' + tool['type'] + '<｜tool▁sep｜>' + tool['function']['name'] + '\n' + '```json' + '\n' + tool['function']['arguments'] + '\n' + '```' + '<｜tool▁call▁end｜>'}}{%- else %}{{content + '<｜tool▁calls▁begin｜><｜tool▁call▁begin｜>' + tool['type'] + '<｜tool▁sep｜>' + tool['function']['name'] + '\n' + '```json' + '\n' + tool['function']['arguments'] + '\n' + '```' + '<｜tool▁call▁end｜>'}}{%- endif %}{%- set ns.is_first = true -%}{%- else %}{{'\n' + '<｜tool▁call▁begin｜>' + tool['type'] + '<｜tool▁sep｜>' + tool['function']['name'] + '\n' + '```json' + '\n' + tool['function']['arguments'] + '\n' + '```' + '<｜tool▁call▁end｜>'}}{%- endif %}{%- endfor %}{{'<｜tool▁calls▁end｜><｜end▁of▁sentence｜>'}}{%- endif %}{%- if message['role'] == 'assistant' and (message['tool_calls'] is not defined or message['tool_calls'] is none)%}{%- set ns.is_last_user = false -%}{%- if ns.is_tool %}{{'<｜tool▁outputs▁end｜>' + content + '<｜end▁of▁sentence｜>'}}{%- set ns.is_tool = false -%}{%- else %}{{content + '<｜end▁of▁sentence｜>'}}{%- endif %}{%- endif %}{%- if message['role'] == 'tool' %}{%- set ns.is_last_user = false -%}{%- set ns.is_tool = true -%}{%- if ns.is_output_first %}{{'<｜tool▁outputs▁begin｜><｜tool▁output▁begin｜>' + content + '<｜tool▁output▁end｜>'}}{%- set ns.is_output_first = false %}{%- else %}{{'\n<｜tool▁output▁begin｜>' + content + '<｜tool▁output▁end｜>'}}{%- endif %}{%- endif %}{%- endfor -%}{% if ns.is_tool %}{{'<｜tool▁outputs▁end｜>'}}{% endif %}{% if add_generation_prompt and not ns.is_last_user and not ns.is_tool %}{{'<｜Assistant｜>'}}{% endif %}",
+        "sampling_thinking": {
+            "temperature": 0.6,
+            "top_p": 0.95,
+            "top_k": 20,
+            "min_p": 0
+        }
+    },
+    "Ministral-8B-Instruct-2410": {
+        "parameters": 8019808256,
+        "context": 32768,
+        "layers": 26,
+        "thinking": false,
+        "optional_thinking": false,
+        "system_message": "",
+        "chat_template": "{%- if messages[0][\"role\"] == \"system\" %}\n    {%- set system_message = messages[0][\"content\"] %}\n    {%- set loop_messages = messages[1:] %}\n{%- else %}\n    {%- set loop_messages = messages %}\n{%- endif %}\n{%- if not tools is defined %}\n    {%- set tools = none %}\n{%- endif %}\n{%- set user_messages = loop_messages | selectattr(\"role\", \"equalto\", \"user\") | list %}\n\n{#- This block checks for alternating user/assistant messages, skipping tool calling messages #}\n{%- set ns = namespace() %}\n{%- set ns.index = 0 %}\n{%- for message in loop_messages %}\n    {%- if not (message.role == \"tool\" or message.role == \"tool_results\" or (message.tool_calls is defined and message.tool_calls is not none)) %}\n        {%- if (message[\"role\"] == \"user\") != (ns.index % 2 == 0) %}\n            {{- raise_exception(\"After the optional system message, conversation roles must alternate user/assistant/user/assistant/...\") }}\n        {%- endif %}\n        {%- set ns.index = ns.index + 1 %}\n    {%- endif %}\n{%- endfor %}\n\n{{- bos_token }}\n{%- for message in loop_messages %}\n    {%- if message[\"role\"] == \"user\" %}\n        {%- if tools is not none and (message == user_messages[-1]) %}\n            {{- \"[AVAILABLE_TOOLS][\" }}\n            {%- for tool in tools %}\n                {%- set tool = tool.function %}\n                {{- '{\"type\": \"function\", \"function\": {' }}\n                {%- for key, val in tool.items() if key != \"return\" %}\n                    {%- if val is string %}\n                        {{- '\"' + key + '\": \"' + val + '\"' }}\n                    {%- else %}\n                        {{- '\"' + key + '\": ' + val|tojson }}\n                    {%- endif %}\n                    {%- if not loop.last %}\n                        {{- \", \" }}\n                    {%- endif %}\n                {%- endfor %}\n                {{- \"}}\" }}\n                {%- if not loop.last %}\n                    {{- \", \" }}\n                {%- else %}\n                    {{- \"]\" }}\n                {%- endif %}\n            {%- endfor %}\n            {{- \"[/AVAILABLE_TOOLS]\" }}\n            {%- endif %}\n        {%- if loop.last and system_message is defined %}\n            {{- \"[INST]\" + system_message + \"\\n\\n\" + message[\"content\"] + \"[/INST]\" }}\n        {%- else %}\n            {{- \"[INST]\" + message[\"content\"] + \"[/INST]\" }}\n        {%- endif %}\n    {%- elif (message.tool_calls is defined and message.tool_calls is not none) %}\n        {{- \"[TOOL_CALLS][\" }}\n        {%- for tool_call in message.tool_calls %}\n            {%- set out = tool_call.function|tojson %}\n            {{- out[:-1] }}\n            {%- if not tool_call.id is defined or tool_call.id|length != 9 %}\n                {{- raise_exception(\"Tool call IDs should be alphanumeric strings with length 9!\") }}\n            {%- endif %}\n            {{- ', \"id\": \"' + tool_call.id + '\"}' }}\n            {%- if not loop.last %}\n                {{- \", \" }}\n            {%- else %}\n                {{- \"]\" + eos_token }}\n            {%- endif %}\n        {%- endfor %}\n    {%- elif message[\"role\"] == \"assistant\" %}\n        {{- message[\"content\"] + eos_token}}\n    {%- elif message[\"role\"] == \"tool_results\" or message[\"role\"] == \"tool\" %}\n        {%- if message.content is defined and message.content.content is defined %}\n            {%- set content = message.content.content %}\n        {%- else %}\n            {%- set content = message.content %}\n        {%- endif %}\n        {{- '[TOOL_RESULTS]{\"content\": ' + content|string + \", \" }}\n        {%- if not message.tool_call_id is defined or message.tool_call_id|length != 9 %}\n            {{- raise_exception(\"Tool call IDs should be alphanumeric strings with length 9!\") }}\n        {%- endif %}\n        {{- '\"call_id\": \"' + message.tool_call_id + '\"}[/TOOL_RESULTS]' }}\n    {%- else %}\n        {{- raise_exception(\"Only user and assistant roles are supported, with the exception of an initial optional system message!\") }}\n    {%- endif %}\n{%- endfor %}\n",
+        "sampling": {
+            "temperature": 0.5,
+            "top_p": 0.9,
+            "top_k": 20,
+            "min_p": 0
+        }
+    }
+}

input_example.txt

@@ -0,0 +1 @@
+Jetzt ist ja Klima ein grosses Thema. Sie haben sich bisher nicht gross geäussert. Jetzt ist eine Strategie in Zürich rausgekommen, man will die rot-grüne Klimapanik bekämpfen. Wie stehen Sie jetzt zu diesem Thema? Wie stehen Sie zu diesen Demonstrationen von all diesen Kindern und Schülern, die wöchentlich auf die Strasse gehen? Die, die auf die Strasse gehen und sagen, wir sind für ein gesundes Klima, da kann ich niemandem dagegen sein. Das ist ja schön, wenn sie das machen. Aber was dahinter steht, politisch bei den Grünen, das sind Sachen, die ganz verwerflich sind. Die wollen jetzt das lösen, mehr Eingriffe vom Staat, mehr Steuern, Abgaben, Gebühren, 20 ApB fürs Benzin, Heizkosten bis zu 1400 Franken pro Haushalt. Dann, was wir sollen essen und nicht sollen essen und wie wir sollen essen und wie wir sollen leben und wohnen und wie gross die Wohnungen sind, das hört nicht mehr auf. Gegen das sind wir massiv. Das ist ein Eingriff in die Freiheit und bringt dem Klima schlussendlich gar nichts. Was würde Ihrer Meinung nach etwas bringen, um den Klimawandel zu stoppen? Weiterfahren mit dem Programm, das wir jetzt haben. Wir müssen mal schauen, was wir schon alles haben. Wir haben die ganze Gewässerverschmutzung unter Kontrolle. Wir haben saubere Gewässer, sagen mir die Fischer, oder? Weil wir sogar das Meteowasser reinigen. Wir haben Rauchgasreinigung. Wir haben beim Auto Abgasvorschriften eingeführt. Wir gehen noch weiter mit dem. Muss ja technologisch möglich sein. Und dann die Innovation nicht vom Staat fördern, sondern schauen, dass die Privaten etwas machen. Wer hat ein Elektroauto? Wer hat ein Hybrid gemacht? Nicht der Staat und auch nicht die Grünen, sondern die Autoindustrie, die Wirtschaft hat das entwickelt. Kommt mit dem. Ich höre, sie arbeiten schon an den Flugzeugen. Sie wollen abgasfreie Flugzeuge. Auf der Lärmseite höre ich, das sind doch Massnahmen, die wir treffen müssen. Wir treffen sie nicht, weil das Klima sonst kaputt geht, sondern weil wir saubere Luft, reines Wasser, gesunden Boden wollen. Das war immer das Programm und bei dem müssen wir bleiben. Und dann wird es gut kommen. Kann man denn mit einer Klima-Offensive bei der SVP rechnen? Weil bisher hat man eher nur gehört, auch gerade von Herr Köppel, die Politik der anderen, die Klimapolitik der anderen, lehnen wir ab. Aber einen Vorschlag von ihnen haben wir noch nicht. Wir brauchen keine. Wir machen schon alles. Das habe ich gerade eben gesagt. Das müssen wir weiterführen. Das ist aber theoretisch nichts machen. Diese Strategie hat ja nicht wirklich ... Nichts falsches machen. Sie möchten gerne, dass wir die falschen Massnahmen machen. Jetzt wollen wir mehr Lenkungsabgaben. Jetzt wollen wir mehr für das Benzin. Das haben die in den Städten, die das Tram vor der Tür haben, die können das gut sagen. Die Leute der Agglomeration, die in die Stadt möchten, wo Züge und alles verstopft ist. Bei der Zuwanderung, die der Hauptgrund ist, machen die nichts. Da sind wir in der Offensive. Jetzt kommt ja die Begrenzungsinitiative. von Nau.ch

main.py

@@ -0,0 +1,145 @@
+import sys
+from pathlib import Path
+sys.path.append(Path(__file__).parent.absolute() + '/util')
+sys.path.append(Path(__file__).parent.absolute() + '/sentence_splitter')
+
+import chromadb
+from util.llm import LLaMaCPP
+from os.path import exists
+from json import load as json_load
+from time import sleep
+from sentence_splitter import split  # noqa
+
+
+MAX_DIFFERENCE = 1.3
+MAX_DB_RESULTS = 10
+with open('prompt.md', 'r', encoding='utf-8') as _f:
+    PROMPT = _f.read()
+GBNF_TEMPLATE = """
+root ::= "```python\\n[" list "]\\n```"
+list ::= %%
+"""
+GBNF_TEMPLATE_ITEM = '("\'%%\'")?'
+GBNF_SEPARATOR = ' (", ")? '
+
+
+def db_read(texts: list[str]):
+    """
+    Get results from ChromaDB based on vector similarity
+    :param texts: a list of strings to search for
+    :return: Query results directly from ChromaDB
+    """
+    client = chromadb.PersistentClient(path=Path(__file__).resolve().parent.parent.absolute().__str__() + '/data/database.chroma')
+    collection = client.get_collection(name='PolitScanner')
+    return collection.query(query_texts=texts, n_results=MAX_DB_RESULTS)
+
+
+def process(sentences: list, llm: LLaMaCPP) -> list:
+    """
+    Check the given sentences for topics
+    :param sentences: a list of sentences as strings
+    :param llm: LLaMaCPP instance with a loaded model (PolitScanner fine-tune preferred)
+    :return: a list of topics
+    """
+    db_results = db_read(sentences)
+    print(db_results)
+    if len(db_results['ids'][0]) == 0:
+        return []
+    topic_ids = []
+    # check if the results are below a certain threshold
+    for i, result in enumerate(db_results['ids'][0]):
+        if db_results['distances'][0][i] < MAX_DIFFERENCE:
+            id_ = result.split('-')[0]
+            if id_ not in topic_ids:
+                topic_ids.append(id_)
+    if len(topic_ids) == 0:
+        return []
+    # if there is only one topic, add 'menschengemachter Klimawandel' in order for the prompt template to make sense
+    if len(topic_ids) == 1 and topic_ids[0] != '0':
+        topic_ids.append('0')
+    topics = []
+    titles = {}
+    # Load the information about the relevant topics
+    for topic_id in topic_ids:
+        with open(Path(__file__).resolve().parent.parent.absolute().__str__() + f"/data/parsed/{topic_id}.json", 'r') as f:
+            topics.append(json_load(f))
+            titles[topics[-1]['topic']] = len(topics) - 1
+    formatted_topics = ''
+    titles_list = list(titles.keys())
+    titles_list.sort()
+    items = []
+    # create the gbnf on the fly
+    for title in titles_list:
+        items.append(GBNF_TEMPLATE_ITEM.replace('%%', title))
+    grammar = GBNF_TEMPLATE.replace('%%', GBNF_SEPARATOR.join(items))
+    topics.sort(key=lambda x: x['topic'])
+    for topic in topics:
+        if len(formatted_topics) > 0:
+            formatted_topics += '\n'
+        formatted_topics += f"'{topic['topic']}'"
+    # create the prompt
+    prompt = PROMPT.replace('{TOPICS}', formatted_topics)
+    for i, sentence in enumerate(sentences):
+        prompt = prompt.replace('{' + f'SENTENCE_{i+1}' + '}', sentence)
+    # conversation template for Qwen3
+    prompt = f"<|im_start|>user\n{prompt}\n/no_think\n<|im_end|>\n<|im_start|>assistant\n<think>\n\n</think>\n"
+    print(prompt)
+    output = llm.generate(prompt, enable_thinking=False, grammar=grammar, temperature=0.0)
+    print(output)
+    # extract the results
+    output = output.split('[')[-1].split(']')[0]
+    truths = []
+    for title in titles_list:
+        if title in output:
+            truths.append(topics[titles[title]]['fact'])  # noqa
+    return truths
+
+
+def main() -> None:
+    """
+    Check the `input.txt` file for topics and return the results in `output.txt`
+    :return: None
+    """
+    if not exists('input.txt'):
+        raise FileNotFoundError('input.txt not found')
+    with open('input.txt', 'r') as f:
+        text = f.read()
+    # Select the Large Language Model
+    llm = LLaMaCPP()
+    if exists('/opt/llms/Qwen3-1.7B-PolitScanner-Q5_K_S.gguf'):
+        llm.set_model('Qwen3-1.7B-PolitScanner-Q5_K_S.gguf')
+    else:
+        llm.set_model('Qwen3-30B-A3B-Q5_K_M.gguf')
+    # Split the file into sentences
+    sentences = split(text)
+    print(f"{len(sentences)=}")
+    chunked_sentences = []
+    # Create overlapping chunks of 3 sentences (plus two sentences of context)
+    for i in range(0, len(sentences), 3):
+        if i == 0:
+            chunk2 = ['EMPTY'] + sentences[:4]
+        elif i + 3 >= len(sentences):
+            chunk2 = sentences[-5:-1] + ['EMPTY']
+        else:
+            chunk2 = sentences[i - 1:i + 4]
+        chunked_sentences.append(chunk2)
+    print(f"{len(chunked_sentences)=}")
+    llm.load_model(print_log=True, threads=16, kv_cache_type='q8_0', context=8192)
+    while llm.is_loading() or not llm.is_running():
+        sleep(1)
+    with open('output.txt', 'w', encoding='utf-8') as f:
+        # Process the chunks
+        for chunked_sentences2 in chunked_sentences:
+            truths = process(chunked_sentences2, llm)
+            for truth in truths:
+                f.write(f"  # Hinweis: {truth}\n")
+            for i, sentence in enumerate(chunked_sentences2):
+                if i in range(1, 4):
+                    f.write(f"{sentence}\n")
+            f.write('\n')
+    print('REACHED `llm.stop()`')
+    llm.stop()
+
+
+if __name__ == '__main__':
+    main()

politscanner.pdf

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:2f65020224511c29aa93fae4e67d9975073af8e359c5d6a5db213ecc34005efc
+size 239194

prompt.md

@@ -0,0 +1,13 @@
+Hier ist eine Liste von Themen.
+```text
+{TOPICS}
+```
+Hier sind die 5 Sätze:
+```text
+1: {SENTENCE_1}
+2: {SENTENCE_2}
+3: {SENTENCE_3}
+4: {SENTENCE_4}
+5: {SENTENCE_5}
+```
+Erstelle eine python-Liste mit den Themen, die in den 5 Sätzen vorkommen. Häufig ist die leere Liste `[]` die richtige Antwort.