---
title: CONOZ DPO Project
emoji: 🦆
# red, yellow, green, blue, indigo, purple, pink, gray
colorFrom: blue
colorTo: yellow
sdk: docker
app_port: 8501
pinned: false
short_description: DPO from meta-llama/Llama-3
tags:
- DPO
- RLHF
- Finetuning
- Llama-3.2-1B
- CONOZ
---

# CONOZ DPO Project

meta-llama/Llama-3.1-8B -> choco-conoz/TwinLlama-3.2-1B-DPO