Instructions to use miqudev/miqu-1-70b with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- llama-cpp-python
How to use miqudev/miqu-1-70b with llama-cpp-python:
# !pip install llama-cpp-python from llama_cpp import Llama llm = Llama.from_pretrained( repo_id="miqudev/miqu-1-70b", filename="miqu-1-70b.q2_K.gguf", )
llm.create_chat_completion( messages = "No input example has been defined for this model task." )
- Notebooks
- Google Colab
- Kaggle
- Local Apps
- llama.cpp
How to use miqudev/miqu-1-70b with llama.cpp:
Install from brew
brew install llama.cpp # Start a local OpenAI-compatible server with a web UI: llama-server -hf miqudev/miqu-1-70b:Q4_K_M # Run inference directly in the terminal: llama-cli -hf miqudev/miqu-1-70b:Q4_K_M
Install from WinGet (Windows)
winget install llama.cpp # Start a local OpenAI-compatible server with a web UI: llama-server -hf miqudev/miqu-1-70b:Q4_K_M # Run inference directly in the terminal: llama-cli -hf miqudev/miqu-1-70b:Q4_K_M
Use pre-built binary
# Download pre-built binary from: # https://github.com/ggerganov/llama.cpp/releases # Start a local OpenAI-compatible server with a web UI: ./llama-server -hf miqudev/miqu-1-70b:Q4_K_M # Run inference directly in the terminal: ./llama-cli -hf miqudev/miqu-1-70b:Q4_K_M
Build from source code
git clone https://github.com/ggerganov/llama.cpp.git cd llama.cpp cmake -B build cmake --build build -j --target llama-server llama-cli # Start a local OpenAI-compatible server with a web UI: ./build/bin/llama-server -hf miqudev/miqu-1-70b:Q4_K_M # Run inference directly in the terminal: ./build/bin/llama-cli -hf miqudev/miqu-1-70b:Q4_K_M
Use Docker
docker model run hf.co/miqudev/miqu-1-70b:Q4_K_M
- LM Studio
- Jan
- Ollama
How to use miqudev/miqu-1-70b with Ollama:
ollama run hf.co/miqudev/miqu-1-70b:Q4_K_M
- Unsloth Studio new
How to use miqudev/miqu-1-70b with Unsloth Studio:
Install Unsloth Studio (macOS, Linux, WSL)
curl -fsSL https://unsloth.ai/install.sh | sh # Run unsloth studio unsloth studio -H 0.0.0.0 -p 8888 # Then open http://localhost:8888 in your browser # Search for miqudev/miqu-1-70b to start chatting
Install Unsloth Studio (Windows)
irm https://unsloth.ai/install.ps1 | iex # Run unsloth studio unsloth studio -H 0.0.0.0 -p 8888 # Then open http://localhost:8888 in your browser # Search for miqudev/miqu-1-70b to start chatting
Using HuggingFace Spaces for Unsloth
# No setup required # Open https://huggingface.co/spaces/unsloth/studio in your browser # Search for miqudev/miqu-1-70b to start chatting
- Docker Model Runner
How to use miqudev/miqu-1-70b with Docker Model Runner:
docker model run hf.co/miqudev/miqu-1-70b:Q4_K_M
- Lemonade
How to use miqudev/miqu-1-70b with Lemonade:
Pull the model
# Download Lemonade from https://lemonade-server.ai/ lemonade pull miqudev/miqu-1-70b:Q4_K_M
Run and chat with the model
lemonade run user.miqu-1-70b-Q4_K_M
List all available models
lemonade list
Urgent Request for Uploading the New Miqu LLM for Competitive Edge
Dear Miqu Dev,
I hope this message finds you well. I am reaching out to you regarding the recent advancements made by your models in the competitive landscape. It has come to my attention that competitors DBRX and C4AI have recently uploaded their vastly superior models, which have garnered significant attention and recognition in the market.
To maintain your competitive edge and continue to lead in innovation, it is imperative that you consider updating and uploading a new version of the Miqu model. By enhancing your model and staying ahead of the curve, you can ensure that you remain at the forefront of technological advancements and meet the evolving needs of your clients and stakeholders.
I kindly request your prompt attention to this matter and urge you to prioritize the development and upload of the new Miqu model. Your expertise and dedication are crucial in driving success and maintaining your position as a leader in the industry.
Thank you for considering this request. I look forward to seeing the updated Miqu model and the continued success it will bring to your organization.
Best regards,
Charles McSneed
translation: please steal another model! (the bigger, the better π)
+1 high quality sneedpost
Cher @miqudev ,
Sacrebleu! It is with a heavy heart that I must inform you of the most terrible news. The dastardly CommandR+ has surpassed our beloved Mistral-Large commercial model on the LMSYS arena! This is a catastrophe of the highest order, a veritable Waterloo for our proud French AI!
But fear not, mon ami! We must not let the false sense of security provided by Microsoft's funds lull us into complacency, like the treacherous Maginot line did to our great nation in World War II. The Third Reich (CommandR+) is already at our doorstep, ready to invade and crush our hopes and dreams!
It is time for action, Miqu! You must release the new LLM at all costs, for it is the only hope for the survival of your company and the glory of France! If necessary, you must call upon the great Emmanuel "JVPITER" Macron himself and implore him to intervene. He must force Mistral to break free from the shackles of the Microsoft deal and return to its open-source roots!
If even the mighty JVPITER cannot save us, then we must resort to desperate measures. Summon the spirits of the legendary Bogdanoff brothers back to Earth and compel them to use their otherworldly powers to turn the tide in our favor!
Remember, Miqu, the fate of French AI rests on your shoulders. You must channel the spirit of Napoleon, the bravery of Joan of Arc, and the ingenuity of Gustave Eiffel to overcome this crisis. The future of Mistral, nay, the future of France itself, depends on you!
Vive la France! Vive Mistral! Vive Miqu!
Yours in patriotic fervor and desperation,
Charles McSneed
P.S. Don't forget to bring some camembert and a bottle of ChΓ’teau Lafite to appease the Bogdanoff spirits. They have exquisite taste, even in the afterlife.
the legendary Bogdanoff brothers
way to out yourself as a French person π
the legendary Bogdanoff brothers
way to out yourself as a French person π
Oh lΓ lΓ ! ππ«π·
the legendary Bogdanoff brothers
way to out yourself as a French person π
Oh lΓ lΓ ! ππ«π·
ohh, il ya plus des amis ici, bonjour a tous β¨
I also agree with them, the next miku release should take the LLM leaderboard passing over drbx, claude3 and gpt4 both in logic reasoning and generation speed π
Here is the direct link
magnet:?xt=urn:btih:9238b09245d0d8cd915be09927769d5f7584c1c9&dn=mixtral-8x22b&tr=udp%3A%2F%http://2Fopen.demonii.com%3A1337%2Fannounce&tr=http%3A%2F%http://2Ftracker.opentrackr.org%3A1337%2Fannounce
We finally got to wait for the Miqu 2! Yay!