Instructions to use kaisser/LLM-Maroc with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use kaisser/LLM-Maroc with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("text-generation", model="kaisser/LLM-Maroc")
messages = [
    {"role": "user", "content": "Who are you?"},
]
pipe(messages)

# Load model directly
from transformers import AutoTokenizer, AutoModelForCausalLM

tokenizer = AutoTokenizer.from_pretrained("kaisser/LLM-Maroc")
model = AutoModelForCausalLM.from_pretrained("kaisser/LLM-Maroc")
messages = [
    {"role": "user", "content": "Who are you?"},
]
inputs = tokenizer.apply_chat_template(
	messages,
	add_generation_prompt=True,
	tokenize=True,
	return_dict=True,
	return_tensors="pt",
).to(model.device)

outputs = model.generate(**inputs, max_new_tokens=40)
print(tokenizer.decode(outputs[0][inputs["input_ids"].shape[-1]:]))

llama-cpp-python

How to use kaisser/LLM-Maroc with llama-cpp-python:

# !pip install llama-cpp-python

from llama_cpp import Llama

llm = Llama.from_pretrained(
	repo_id="kaisser/LLM-Maroc",
	filename="llama.cpp/models/ggml-vocab-aquila.gguf",
)

llm.create_chat_completion(
	messages = [
		{
			"role": "user",
			"content": "What is the capital of France?"
		}
	]
)

Notebooks
Google Colab
Kaggle
Local Apps

llama.cpp

How to use kaisser/LLM-Maroc with llama.cpp:

Install from brew

brew install llama.cpp
# Start a local OpenAI-compatible server with a web UI:
llama-server -hf kaisser/LLM-Maroc:BF16
# Run inference directly in the terminal:
llama-cli -hf kaisser/LLM-Maroc:BF16

Install from WinGet (Windows)

winget install llama.cpp
# Start a local OpenAI-compatible server with a web UI:
llama-server -hf kaisser/LLM-Maroc:BF16
# Run inference directly in the terminal:
llama-cli -hf kaisser/LLM-Maroc:BF16

Use pre-built binary

# Download pre-built binary from:
# https://github.com/ggerganov/llama.cpp/releases
# Start a local OpenAI-compatible server with a web UI:
./llama-server -hf kaisser/LLM-Maroc:BF16
# Run inference directly in the terminal:
./llama-cli -hf kaisser/LLM-Maroc:BF16

Build from source code

git clone https://github.com/ggerganov/llama.cpp.git
cd llama.cpp
cmake -B build
cmake --build build -j --target llama-server llama-cli
# Start a local OpenAI-compatible server with a web UI:
./build/bin/llama-server -hf kaisser/LLM-Maroc:BF16
# Run inference directly in the terminal:
./build/bin/llama-cli -hf kaisser/LLM-Maroc:BF16

Use Docker

docker model run hf.co/kaisser/LLM-Maroc:BF16

LM Studio
Jan

vLLM

How to use kaisser/LLM-Maroc with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "kaisser/LLM-Maroc"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "kaisser/LLM-Maroc",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker

docker model run hf.co/kaisser/LLM-Maroc:BF16

SGLang

How to use kaisser/LLM-Maroc with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "kaisser/LLM-Maroc" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "kaisser/LLM-Maroc",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "kaisser/LLM-Maroc" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "kaisser/LLM-Maroc",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Ollama
How to use kaisser/LLM-Maroc with Ollama:
```
ollama run hf.co/kaisser/LLM-Maroc:BF16
```

Unsloth Studio new

How to use kaisser/LLM-Maroc with Unsloth Studio:

Install Unsloth Studio (macOS, Linux, WSL)

curl -fsSL https://unsloth.ai/install.sh | sh
# Run unsloth studio
unsloth studio -H 0.0.0.0 -p 8888
# Then open http://localhost:8888 in your browser
# Search for kaisser/LLM-Maroc to start chatting

Install Unsloth Studio (Windows)

irm https://unsloth.ai/install.ps1 | iex
# Run unsloth studio
unsloth studio -H 0.0.0.0 -p 8888
# Then open http://localhost:8888 in your browser
# Search for kaisser/LLM-Maroc to start chatting

Using HuggingFace Spaces for Unsloth

# No setup required
# Open https://huggingface.co/spaces/unsloth/studio in your browser
# Search for kaisser/LLM-Maroc to start chatting

Docker Model Runner
How to use kaisser/LLM-Maroc with Docker Model Runner:
```
docker model run hf.co/kaisser/LLM-Maroc:BF16
```

Lemonade

How to use kaisser/LLM-Maroc with Lemonade:

Pull the model

# Download Lemonade from https://lemonade-server.ai/
lemonade pull kaisser/LLM-Maroc:BF16

Run and chat with the model

lemonade run user.LLM-Maroc-BF16

List all available models

lemonade list

LLM-Maroc / llama.cpp /build /examples /simple /Makefile

kaisser

Upload folder using huggingface_hub

305a42c verified 10 months ago

raw

history blame contribute delete

8.67 kB

	# CMAKE generated file: DO NOT EDIT!
	# Generated by "Unix Makefiles" Generator, CMake Version 3.31

	# Default target executed when no arguments are given to make.
	default_target: all
	.PHONY : default_target

	# Allow only one "make -f Makefile2" at a time, but pass parallelism.
	.NOTPARALLEL:

	#=============================================================================
	# Special targets provided by cmake.

	# Disable implicit rules so canonical targets will work.
	.SUFFIXES:

	# Disable VCS-based implicit rules.
	% : %,v

	# Disable VCS-based implicit rules.
	% : RCS/%

	# Disable VCS-based implicit rules.
	% : RCS/%,v

	# Disable VCS-based implicit rules.
	% : SCCS/s.%

	# Disable VCS-based implicit rules.
	% : s.%

	.SUFFIXES: .hpux_make_needs_suffix_list

	# Command-line flag to silence nested $(MAKE).
	$(VERBOSE)MAKESILENT = -s

	#Suppress display of executed commands.
	$(VERBOSE).SILENT:

	# A target that is always out of date.
	cmake_force:
	.PHONY : cmake_force

	#=============================================================================
	# Set environment variables for the build.

	# The shell in which to execute make rules.
	SHELL = /bin/sh

	# The CMake executable.
	CMAKE_COMMAND = /usr/local/lib/python3.11/dist-packages/cmake/data/bin/cmake

	# The command to remove a file.
	RM = /usr/local/lib/python3.11/dist-packages/cmake/data/bin/cmake -E rm -f

	# Escaping for special characters.
	EQUALS = =

	# The top-level source directory on which CMake was run.
	CMAKE_SOURCE_DIR = /content/llama.cpp

	# The top-level build directory on which CMake was run.
	CMAKE_BINARY_DIR = /content/llama.cpp/build

	#=============================================================================
	# Targets provided globally by CMake.

	# Special rule for the target test
	test:
	@$(CMAKE_COMMAND) -E cmake_echo_color "--switch=$(COLOR)" --cyan "Running tests..."
	/usr/local/lib/python3.11/dist-packages/cmake/data/bin/ctest --force-new-ctest-process $(ARGS)
	.PHONY : test

	# Special rule for the target test
	test/fast: test
	.PHONY : test/fast

	# Special rule for the target edit_cache
	edit_cache:
	@$(CMAKE_COMMAND) -E cmake_echo_color "--switch=$(COLOR)" --cyan "No interactive CMake dialog available..."
	/usr/local/lib/python3.11/dist-packages/cmake/data/bin/cmake -E echo No\ interactive\ CMake\ dialog\ available.
	.PHONY : edit_cache

	# Special rule for the target edit_cache
	edit_cache/fast: edit_cache
	.PHONY : edit_cache/fast

	# Special rule for the target rebuild_cache
	rebuild_cache:
	@$(CMAKE_COMMAND) -E cmake_echo_color "--switch=$(COLOR)" --cyan "Running CMake to regenerate build system..."
	/usr/local/lib/python3.11/dist-packages/cmake/data/bin/cmake --regenerate-during-build -S$(CMAKE_SOURCE_DIR) -B$(CMAKE_BINARY_DIR)
	.PHONY : rebuild_cache

	# Special rule for the target rebuild_cache
	rebuild_cache/fast: rebuild_cache
	.PHONY : rebuild_cache/fast

	# Special rule for the target list_install_components
	list_install_components:
	@$(CMAKE_COMMAND) -E cmake_echo_color "--switch=$(COLOR)" --cyan "Available install components are: \"Unspecified\""
	.PHONY : list_install_components

	# Special rule for the target list_install_components
	list_install_components/fast: list_install_components
	.PHONY : list_install_components/fast

	# Special rule for the target install
	install: preinstall
	@$(CMAKE_COMMAND) -E cmake_echo_color "--switch=$(COLOR)" --cyan "Install the project..."
	/usr/local/lib/python3.11/dist-packages/cmake/data/bin/cmake -P cmake_install.cmake
	.PHONY : install

	# Special rule for the target install
	install/fast: preinstall/fast
	@$(CMAKE_COMMAND) -E cmake_echo_color "--switch=$(COLOR)" --cyan "Install the project..."
	/usr/local/lib/python3.11/dist-packages/cmake/data/bin/cmake -P cmake_install.cmake
	.PHONY : install/fast

	# Special rule for the target install/local
	install/local: preinstall
	@$(CMAKE_COMMAND) -E cmake_echo_color "--switch=$(COLOR)" --cyan "Installing only the local directory..."
	/usr/local/lib/python3.11/dist-packages/cmake/data/bin/cmake -DCMAKE_INSTALL_LOCAL_ONLY=1 -P cmake_install.cmake
	.PHONY : install/local

	# Special rule for the target install/local
	install/local/fast: preinstall/fast
	@$(CMAKE_COMMAND) -E cmake_echo_color "--switch=$(COLOR)" --cyan "Installing only the local directory..."
	/usr/local/lib/python3.11/dist-packages/cmake/data/bin/cmake -DCMAKE_INSTALL_LOCAL_ONLY=1 -P cmake_install.cmake
	.PHONY : install/local/fast

	# Special rule for the target install/strip
	install/strip: preinstall
	@$(CMAKE_COMMAND) -E cmake_echo_color "--switch=$(COLOR)" --cyan "Installing the project stripped..."
	/usr/local/lib/python3.11/dist-packages/cmake/data/bin/cmake -DCMAKE_INSTALL_DO_STRIP=1 -P cmake_install.cmake
	.PHONY : install/strip

	# Special rule for the target install/strip
	install/strip/fast: preinstall/fast
	@$(CMAKE_COMMAND) -E cmake_echo_color "--switch=$(COLOR)" --cyan "Installing the project stripped..."
	/usr/local/lib/python3.11/dist-packages/cmake/data/bin/cmake -DCMAKE_INSTALL_DO_STRIP=1 -P cmake_install.cmake
	.PHONY : install/strip/fast

	# The main all target
	all: cmake_check_build_system
	cd /content/llama.cpp/build && $(CMAKE_COMMAND) -E cmake_progress_start /content/llama.cpp/build/CMakeFiles /content/llama.cpp/build/examples/simple//CMakeFiles/progress.marks
	cd /content/llama.cpp/build && $(MAKE) $(MAKESILENT) -f CMakeFiles/Makefile2 examples/simple/all
	$(CMAKE_COMMAND) -E cmake_progress_start /content/llama.cpp/build/CMakeFiles 0
	.PHONY : all

	# The main clean target
	clean:
	cd /content/llama.cpp/build && $(MAKE) $(MAKESILENT) -f CMakeFiles/Makefile2 examples/simple/clean
	.PHONY : clean

	# The main clean target
	clean/fast: clean
	.PHONY : clean/fast

	# Prepare targets for installation.
	preinstall: all
	cd /content/llama.cpp/build && $(MAKE) $(MAKESILENT) -f CMakeFiles/Makefile2 examples/simple/preinstall
	.PHONY : preinstall

	# Prepare targets for installation.
	preinstall/fast:
	cd /content/llama.cpp/build && $(MAKE) $(MAKESILENT) -f CMakeFiles/Makefile2 examples/simple/preinstall
	.PHONY : preinstall/fast

	# clear depends
	depend:
	cd /content/llama.cpp/build && $(CMAKE_COMMAND) -S$(CMAKE_SOURCE_DIR) -B$(CMAKE_BINARY_DIR) --check-build-system CMakeFiles/Makefile.cmake 1
	.PHONY : depend

	# Convenience name for target.
	examples/simple/CMakeFiles/llama-simple.dir/rule:
	cd /content/llama.cpp/build && $(MAKE) $(MAKESILENT) -f CMakeFiles/Makefile2 examples/simple/CMakeFiles/llama-simple.dir/rule
	.PHONY : examples/simple/CMakeFiles/llama-simple.dir/rule

	# Convenience name for target.
	llama-simple: examples/simple/CMakeFiles/llama-simple.dir/rule
	.PHONY : llama-simple

	# fast build rule for target.
	llama-simple/fast:
	cd /content/llama.cpp/build && $(MAKE) $(MAKESILENT) -f examples/simple/CMakeFiles/llama-simple.dir/build.make examples/simple/CMakeFiles/llama-simple.dir/build
	.PHONY : llama-simple/fast

	simple.o: simple.cpp.o
	.PHONY : simple.o

	# target to build an object file
	simple.cpp.o:
	cd /content/llama.cpp/build && $(MAKE) $(MAKESILENT) -f examples/simple/CMakeFiles/llama-simple.dir/build.make examples/simple/CMakeFiles/llama-simple.dir/simple.cpp.o
	.PHONY : simple.cpp.o

	simple.i: simple.cpp.i
	.PHONY : simple.i

	# target to preprocess a source file
	simple.cpp.i:
	cd /content/llama.cpp/build && $(MAKE) $(MAKESILENT) -f examples/simple/CMakeFiles/llama-simple.dir/build.make examples/simple/CMakeFiles/llama-simple.dir/simple.cpp.i
	.PHONY : simple.cpp.i

	simple.s: simple.cpp.s
	.PHONY : simple.s

	# target to generate assembly for a file
	simple.cpp.s:
	cd /content/llama.cpp/build && $(MAKE) $(MAKESILENT) -f examples/simple/CMakeFiles/llama-simple.dir/build.make examples/simple/CMakeFiles/llama-simple.dir/simple.cpp.s
	.PHONY : simple.cpp.s

	# Help Target
	help:
	@echo "The following are some of the valid targets for this Makefile:"
	@echo "... all (the default if no target is provided)"
	@echo "... clean"
	@echo "... depend"
	@echo "... edit_cache"
	@echo "... install"
	@echo "... install/local"
	@echo "... install/strip"
	@echo "... list_install_components"
	@echo "... rebuild_cache"
	@echo "... test"
	@echo "... llama-simple"
	@echo "... simple.o"
	@echo "... simple.i"
	@echo "... simple.s"
	.PHONY : help



	#=============================================================================
	# Special targets to cleanup operation of make.

	# Special rule to run CMake to check the build system integrity.
	# No rule that depends on this can have commands that come from listfiles
	# because they might be regenerated.
	cmake_check_build_system:
	cd /content/llama.cpp/build && $(CMAKE_COMMAND) -S$(CMAKE_SOURCE_DIR) -B$(CMAKE_BINARY_DIR) --check-build-system CMakeFiles/Makefile.cmake 0
	.PHONY : cmake_check_build_system