| # DistilBERT Model for Crop Recommendation Based on Environmental Parameters | |
| This repository contains a fine-tuned DistilBERT model trained for crop recommendation using structured agricultural data. By converting numerical environmental features into text format, the model leverages transformer-based NLP techniques to classify the most suitable crop type. | |
| ## 🌾 Problem Statement | |
| The goal is to recommend the best crop to cultivate based on parameters such as soil nutrients and weather conditions. Traditional ML models handle this as a tabular classification problem. Here, we explore the innovative approach of using NLP models (DistilBERT) on serialized tabular data. | |
| --- | |
| ## 📊 Dataset | |
| - **Source:** Crop Recommendation Dataset | |
| - **Features:** | |
| - N: Nitrogen content in soil | |
| - P: Phosphorus content in soil | |
| - K: Potassium content in soil | |
| - Temperature: in Celsius | |
| - Humidity: % | |
| - pH: Acidity of soil | |
| - Rainfall: mm | |
| - **Target:** Crop label (22 crop types) | |
| The dataset is preprocessed by concatenating all numeric features into a single space-separated string, making it suitable for transformer-based tokenization. | |
| --- | |
| ## 🧠 Model Details | |
| - **Architecture:** DistilBERT | |
| - **Tokenizer:** `DistilBertTokenizerFast` | |
| - **Model:** `DistilBertForSequenceClassification` | |
| - **Task Type:** Multi-Class Classification (22 classes) | |
| --- | |
| ## 🔧 Installation | |
| ```bash | |
| pip install transformers datasets pandas scikit-learn torch | |
| ``` | |
| --- | |
| ## Loading the Model | |
| ```python | |
| from transformers import DistilBertTokenizerFast, DistilBertForSequenceClassification | |
| import torch | |
| # Load model and tokenizer | |
| model_path = "model_fp32_dir" | |
| tokenizer = DistilBertTokenizerFast.from_pretrained(model_path) | |
| model = DistilBertForSequenceClassification.from_pretrained(model_path) | |
| # Sample input | |
| sample_text = "90 42 43 20.879744 82.002744 6.502985 202.935536" | |
| inputs = tokenizer(sample_text, return_tensors="pt") | |
| # Predict | |
| with torch.no_grad(): | |
| outputs = model(**inputs) | |
| predicted_class = torch.argmax(outputs.logits, dim=1).item() | |
| print("Predicted class index:", predicted_class) | |
| ``` | |
| --- | |
| ## 📈 Performance Metrics | |
| - **Accuracy:** 0.7636 | |
| - **Precision:** 0.7738 | |
| - **Recall:** 0.7636 | |
| - **F1 Score:** 0.7343 | |
| --- | |
| ## 🏋️ Fine-Tuning Details | |
| ### 📚 Dataset | |
| The dataset is sourced from the publicly available **Crop Recommendation Dataset**. It consists of structured features such as: | |
| - Nitrogen (N) | |
| - Phosphorus (P) | |
| - Potassium (K) | |
| - Temperature (°C) | |
| - Humidity (%) | |
| - pH | |
| - Rainfall (mm) | |
| All numerical features were converted into a single textual input string to be used with the DistilBERT tokenizer. Labels were factorized into class indices for training. | |
| The dataset was split using an 80/20 ratio for training and testing. | |
| --- | |
| ### 🔧 Training Configuration | |
| - **Epochs:** 3 | |
| - **Batch size:** 8 | |
| - **Learning rate:** 2e-5 | |
| - **Evaluation strategy:** `epoch` | |
| - **Model Base:** DistilBERT (`distilbert-base-uncased`) | |
| - **Framework:** Hugging Face Transformers + PyTorch | |
| --- | |
| ## 🔄 Quantization | |
| Post-training quantization was applied using PyTorch’s `half()` precision (FP16). | |
| This reduces the model size and speeds up inference with minimal impact on performance. | |
| The quantized model can be loaded with: | |
| ```python | |
| model = DistilBertForSequenceClassification.from_pretrained("quantized_model_fp16", torch_dtype=torch.float16) | |
| ``` | |
| --- | |
| ## Repository Structure | |
| ```python | |
| . | |
| ├── quantized-model/ # Contains the quantized model files | |
| │ ├── config.json | |
| │ ├── model.safetensors | |
| │ ├── tokenizer_config.json | |
| │ ├── vocab.txt | |
| │ └── special_tokens_map.json | |
| ├── README.md # Model documentation | |
| ``` | |
| --- | |
| ## Limitations | |
| - Uses text conversion of tabular data, which may miss deeper feature interactions. | |
| - Trained on a specific dataset; may not generalize to different regions or conditions. | |
| - FP16 quantization may slightly reduce accuracy in rare cases. | |
| --- | |
| ## Contributing | |
| Feel free to open issues or submit pull requests to improve the model or documentation. | |