"Getting Started with Fine-Tuning and Running AI Models Locally"
Hi Ronan @eltorio
where to learn models tunning and these stuff would you recommend a course ? and how can I RUN this model locally on my device ?
Courses
Hugging Face offers excellent courses on LLMs, NLP, and fine-tuning. You can start with their NLP course .
Fine tuning locally
You can fine-tune models locally if you have a decent NVIDIA graphics card. For example, I created this simple notebook that produces a fined-tuned Llama 3.2B model . The dataset currently contains around 250 records, and it takes only a few minutes to train on my RTX2080/6GB.
Make sure to carefully understand all the cells in the notebook. It generates a PEFT model, which is a lightweight version of a model.
Ronan
@OsamaKhaled12
Locally for testing my notebooks I use my own docker image launched on Windows with:
docker run --gpus all --user=42420:42420 -e HF_TOKEN=hf_you_gh_token -p 8080:8080 -it sctg/roco-idefics3:0.0.9 bash -i /start.sh sleep infinity
It starts a Jupyter Lab on the port 8080 with all the required libraries.
For reference Dockerfile source is here