cyb3rr31a/tinyllama-data-engineering

TinyLlama Data Engineering Assistant

A TinyLlama-1.1B model fine-tuned on data engineering Q&A pairs using QLoRA. It answers questions about data engineering concepts more accurately than the base model.

Base model

TinyLlama/TinyLlama-1.1B-Chat-v1.0

Training

• Method: QLoRA (4-bit quantization + LoRA)
• Dataset: 15 custom data engineering Q&A pairs
• Epochs: 10
• LoRA rank: 16
• Hardware: NVIDIA T4 (Google Colab free tier)

Topics covered

ETL, data warehouses, data lakes, Apache Spark, dbt, Apache Airflow, DAGs, batch vs stream processing, data pipelines, partitioning, data lineage, medallion architecture, idempotency, BigQuery, dimensional modeling, RAG

How to use

from transformers import AutoModelForCausalLM, AutoTokenizer
import torch

model_name = "cyb3rr31a/tinyllama-data-engineering"
tokenizer = AutoTokenizer.from_pretrained(model_name)
model = AutoModelForCausalLM.from_pretrained(model_name, torch_dtype=torch.float16)

prompt = "### Question:\nWhat is dbt?\n\n### Answer:\n"
inputs = tokenizer(prompt, return_tensors="pt")
outputs = model.generate(**inputs, max_new_tokens=200)
print(tokenizer.decode(outputs[0], skip_special_tokens=True))

Limitations

This model was fine-tuned on a small dataset of 15 examples for demonstration purposes. It performs best on the topics covered in the training data.