riswanahamed commited on
Commit
bddfe27
·
verified ·
1 Parent(s): df76b12

user message updated

Browse files
Files changed (1) hide show
  1. README.md +3 -15
README.md CHANGED
@@ -33,20 +33,14 @@ print(output["generated_text"])
33
  #### **What I Did**
34
  I fine-tuned a pre-trained language model using the Hugging Face `transformers` library. The base model was adapted to perform better on specific task by training it on a domain-specific dataset.
35
 
36
- #### **Why I Did It**
37
- The pre-trained model, while powerful, was not optimized for specific domain or task. Fine-tuning was necessary to adapt the model to the specific requirements of [use case or application]. This helps improve the model's performance and ensures more accurate predictions for the intended task.
38
-
39
  #### **How I Did It**
40
- 1. **Dataset Preparation**:
41
- - Collected and preprocessed the dataset. This involved tokenization, padding, and formatting the data to ensure compatibility with the model.
42
- - Split the dataset into training and validation sets to monitor the model's performance during training.
43
 
44
- 2. **Fine-Tuning Setup**:
45
  - Configured the model training parameters, including the learning rate, batch size, and number of steps.
46
  - Used `SFTTrainer` from Hugging Face for seamless training with built-in evaluation capabilities.
47
  - Trained the model for 1 epoch to prevent overfitting, as the dataset was relatively small and hardware resources were limited.
48
 
49
- 3. **Training Environment**:
50
  - The training was performed in Google Colab using a CPU/GPU environment.
51
  - Adjusted batch sizes and learning rates to balance between performance and available resources.
52
 
@@ -61,16 +55,10 @@ The pre-trained model, while powerful, was not optimized for specific domain or
61
  1. **Use the Model**:
62
  - Load the model using the Hugging Face `transformers` library.
63
  - Tokenize your inputs and pass them to the model for inference.
64
-
65
- 2. **Adapting to New Tasks**:
66
  - If your task or domain differs, fine-tune the model further on your dataset.
67
  - Follow the same process: prepare the dataset, set training configurations, and monitor evaluation metrics.
68
 
69
- 3. **Understand Limitations**:
70
- - Be mindful that the model may not perform well on tasks outside the scope of its fine-tuning dataset.
71
- - If you encounter unexpected biases, consider augmenting the training data or further fine-tuning.
72
-
73
- 4. **Experiment with Parameters**:
74
  - If you have access to better hardware, experiment with larger batch sizes or additional epochs to improve results.
75
  - Use hyperparameter tuning to find the best configuration for your use case.
76
 
 
33
  #### **What I Did**
34
  I fine-tuned a pre-trained language model using the Hugging Face `transformers` library. The base model was adapted to perform better on specific task by training it on a domain-specific dataset.
35
 
 
 
 
36
  #### **How I Did It**
 
 
 
37
 
38
+ **Fine-Tuning Setup**:
39
  - Configured the model training parameters, including the learning rate, batch size, and number of steps.
40
  - Used `SFTTrainer` from Hugging Face for seamless training with built-in evaluation capabilities.
41
  - Trained the model for 1 epoch to prevent overfitting, as the dataset was relatively small and hardware resources were limited.
42
 
43
+ **Training Environment**:
44
  - The training was performed in Google Colab using a CPU/GPU environment.
45
  - Adjusted batch sizes and learning rates to balance between performance and available resources.
46
 
 
55
  1. **Use the Model**:
56
  - Load the model using the Hugging Face `transformers` library.
57
  - Tokenize your inputs and pass them to the model for inference.
 
 
58
  - If your task or domain differs, fine-tune the model further on your dataset.
59
  - Follow the same process: prepare the dataset, set training configurations, and monitor evaluation metrics.
60
 
61
+ 2. **Experiment with Parameters**:
 
 
 
 
62
  - If you have access to better hardware, experiment with larger batch sizes or additional epochs to improve results.
63
  - Use hyperparameter tuning to find the best configuration for your use case.
64