Instructions to use netcat420/MFANN3b with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- Transformers
How to use netcat420/MFANN3b with Transformers:
# Use a pipeline as a high-level helper from transformers import pipeline pipe = pipeline("text-generation", model="netcat420/MFANN3b")# Load model directly from transformers import AutoTokenizer, AutoModelForCausalLM tokenizer = AutoTokenizer.from_pretrained("netcat420/MFANN3b") model = AutoModelForCausalLM.from_pretrained("netcat420/MFANN3b") - Notebooks
- Google Colab
- Kaggle
- Local Apps
- vLLM
How to use netcat420/MFANN3b with vLLM:
Install from pip and serve model
# Install vLLM from pip: pip install vllm # Start the vLLM server: vllm serve "netcat420/MFANN3b" # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:8000/v1/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "netcat420/MFANN3b", "prompt": "Once upon a time,", "max_tokens": 512, "temperature": 0.5 }'Use Docker
docker model run hf.co/netcat420/MFANN3b
- SGLang
How to use netcat420/MFANN3b with SGLang:
Install from pip and serve model
# Install SGLang from pip: pip install sglang # Start the SGLang server: python3 -m sglang.launch_server \ --model-path "netcat420/MFANN3b" \ --host 0.0.0.0 \ --port 30000 # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:30000/v1/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "netcat420/MFANN3b", "prompt": "Once upon a time,", "max_tokens": 512, "temperature": 0.5 }'Use Docker images
docker run --gpus all \ --shm-size 32g \ -p 30000:30000 \ -v ~/.cache/huggingface:/root/.cache/huggingface \ --env "HF_TOKEN=<secret>" \ --ipc=host \ lmsysorg/sglang:latest \ python3 -m sglang.launch_server \ --model-path "netcat420/MFANN3b" \ --host 0.0.0.0 \ --port 30000 # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:30000/v1/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "netcat420/MFANN3b", "prompt": "Once upon a time,", "max_tokens": 512, "temperature": 0.5 }' - Docker Model Runner
How to use netcat420/MFANN3b with Docker Model Runner:
docker model run hf.co/netcat420/MFANN3b
# Load model directly
from transformers import AutoTokenizer, AutoModelForCausalLM
tokenizer = AutoTokenizer.from_pretrained("netcat420/MFANN3b")
model = AutoModelForCausalLM.from_pretrained("netcat420/MFANN3b")FULL RELEASE!!!
introducing MFANN (Makhi's fully autonomous neural network) a family of COT models fine-tuned on a modified Alpaca training regiment involving defining a "thought-process" in the dataset in each sample, allowing the model to generate reasoning tokens before the model generates output (yes this has been in the works MUCH longer than o1 has been out!)
this is the smaller 3b model based on phi-2
the 8b model based on an abliterated llama 3.1 8b will be coming in the next few days!! stay tuned!!
If you would like to support me and keep this project going well into the future (or even free me up from the horrors of little caesars so i can work on this more lmfao) please consider donating to my patreon!! https://www.patreon.com/c/MakhiBurroughs all members get to vote on what gets done next with MFANN, addtional perks are currently in the works!
system prompt:
Instruct: {instruction} Output:
- Downloads last month
- 70
# Use a pipeline as a high-level helper from transformers import pipeline pipe = pipeline("text-generation", model="netcat420/MFANN3b")