SejalMutakekar/text2cypher-smollm2-135m

Text-to-Cypher: SmolLM2-135M Fine-tuned

Fine-tuned version of HuggingFaceTB/SmolLM2-135M-Instruct on the RomanTeucher/text2cypher-curated dataset for generating Cypher queries from natural language questions and graph schemas.

---

Model Details

• Base model: HuggingFaceTB/SmolLM2-135M-Instruct
• Task: Text-to-Cypher generation
• Dataset: RomanTeucher/text2cypher-curated (1000 train / 75 val / 50 test)
• Training hardware: CPU
• Training framework: HuggingFace Trainer

Checkpoints

• model_small/ — fine-tuned for 1 epoch (~24 minutes on CPU)
• model_large/ — fine-tuned for 3 epochs (~72 minutes on CPU) — best performance

---

Results

Setting	Exact Match	BLEU Score
Baseline (no fine-tuning)	0.00	0.0143
Fine-tuned (1 epoch)	0.22	0.4109
Fine-tuned (3 epochs)	0.40	0.5885

Exact Match — Checks whether the generated Cypher is character-for-character identical to the ground truth. A score of 0.40 means 20 out of 50 test queries were perfectly correct.

BLEU Score — Measures token-level overlap between the generated and ground truth query. Gives partial credit for queries that are structurally close but not identical.

---

Usage

Reproduce Evaluation Results

# Clone the repository
git clone https://github.com/sejal-0502/text2cypher.git
cd text2cypher

# Install dependencies
pip install -r requirements.txt

# Evaluate 3 epoch model directly from HuggingFace Hub
python src/evaluate.py \
  --model SejalMutakekar/text2cypher-smollm2-135m/model_large \
  --output results/hub_3epoch_results.json

# Evaluate 1 epoch model directly from HuggingFace Hub
python src/evaluate.py \
  --model SejalMutakekar/text2cypher-smollm2-135m/model_small \
  --output results/hub_1epoch_results.json