{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Training dataset"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "qa_pairs = [\n",
    "    (\"What is the average fuel efficiency of a car?\", \"Most cars have a fuel efficiency between twenty to thirty-five miles per gallon.\"),\n",
    "    (\"How often should I replace my car battery?\", \"Car batteries typically last three to five years, depending on usage and climate.\"),\n",
    "    (\"Why is my car not starting?\", \"Possible reasons include a dead battery, a faulty starter, or no fuel in the tank.\"),\n",
    "    (\"How can I improve my car's fuel economy?\", \"Driving smoothly, maintaining tire pressure, and reducing extra weight can help improve fuel economy.\"),\n",
    "    (\"What type of engine oil should I use?\", \"Check your owner's manual, but most modern cars use synthetic five-W-30 or zero-W-20 oil.\"),\n",
    "    (\"How do I know if my brakes need to be replaced?\", \"Signs include squeaking noises, reduced braking performance, or a vibrating brake pedal.\"),\n",
    "    (\"Why is my check engine light on?\", \"It could be due to a loose gas cap, faulty oxygen sensor, or engine issues.\"),\n",
    "    (\"How often should I check my tire pressure?\", \"It's best to check tire pressure at least once a month or before long trips.\"),\n",
    "    (\"What is the best way to keep my car's paint looking new?\", \"Regular washing, waxing, and parking in the shade can help protect the paint.\"),\n",
    "    (\"Why is my car making a loud noise when I accelerate?\", \"It might be due to an exhaust leak, engine issues, or transmission problems.\"),\n",
    "    (\"How often should I change my car's air filter?\", \"Typically, air filters should be replaced every twelve thousand to fifteen thousand miles or once a year, but check your manual.\"),\n",
    "    (\"What is the purpose of the timing belt?\", \"The timing belt synchronizes the engine's crankshaft and camshaft to ensure proper engine timing.\"),\n",
    "    (\"How do I know when my car's transmission fluid needs to be changed?\", \"Transmission fluid should be changed every thirty thousand to sixty thousand miles, depending on the car's make and model.\"),\n",
    "    (\"What should I do if my car overheats?\", \"Turn off the AC, stop the car safely, and allow the engine to cool down. If necessary, check the coolant levels.\"),\n",
    "    (\"How do I fix a flat tire?\", \"Use a jack to lift the car, remove the flat tire, and replace it with the spare. Ensure the spare tire is properly inflated.\"),\n",
    "    (\"What is the purpose of the catalytic converter?\", \"The catalytic converter reduces harmful emissions by converting exhaust gases into less harmful substances.\"),\n",
    "    (\"How do I reset my car's service light?\", \"Check the owner's manual, as the process can vary by car model. Often, it involves pressing a button or turning on the ignition.\"),\n",
    "    (\"Why is my car vibrating at high speeds?\", \"Vibrations at high speeds can be caused by unbalanced wheels, misaligned suspension, or brake issues.\"),\n",
    "    (\"How do I maintain my car's battery?\", \"To prolong battery life, clean the terminals, ensure it's properly secured, and replace it every three to five years.\"),\n",
    "    (\"What is the difference between all-season and winter tires?\", \"All-season tires are suitable for year-round use, while winter tires are designed for better traction in snow and cold weather.\"),\n",
    "    (\"How do I know when my car needs a new tire?\", \"Tires should be replaced if they are worn down to the tread wear indicators, have visible damage, or if they’re more than six years old.\"),\n",
    "    (\"What should I do if my car's air conditioning isn't working?\", \"Check for a low refrigerant level, a blown fuse, or a malfunctioning compressor. It may need to be inspected by a mechanic.\"),\n",
    "    (\"How often should I replace my wiper blades?\", \"Wiper blades should be replaced every six to twelve months, or when they no longer effectively clear the windshield.\"),\n",
    "    (\"What is an oil change and why is it important?\", \"An oil change involves draining old oil and replacing it with new oil, helping to keep the engine lubricated and preventing damage from friction.\"),\n",
    "    (\"What is the difference between front-wheel drive and rear-wheel drive?\", \"Front-wheel drive sends power to the front wheels, while rear-wheel drive sends power to the rear wheels. Front-wheel drive is more fuel efficient, while rear-wheel drive is often preferred for performance.\"),\n",
    "    (\"How can I tell if my car has a fuel leak?\", \"Signs include the smell of gasoline near your car, puddles of fuel under the car, or poor fuel efficiency.\"),\n",
    "    (\"What is the purpose of power steering fluid?\", \"Power steering fluid helps the steering system operate smoothly, making it easier to turn the steering wheel.\"),\n",
    "    (\"What is the purpose of a serpentine belt?\", \"The serpentine belt drives multiple peripheral devices like the alternator, air conditioning compressor, and power steering pump.\"),\n",
    "    (\"How do I check my car's oil level?\", \"To check the oil level, remove the dipstick, wipe it clean, reinsert it, and then remove it again to check the oil level against the markers on the dipstick.\"),\n",
    "    (\"What are the signs of a bad alternator?\", \"A bad alternator can cause dimming headlights, trouble starting the car, or a battery warning light on the dashboard.\"),\n",
    "    (\"Why is my car's steering wheel hard to turn?\", \"This could be caused by low power steering fluid, a malfunctioning power steering pump, or a broken steering belt.\"),\n",
    "    (\"How can I improve my car's braking performance?\", \"Ensure the brake pads and rotors are in good condition, and regularly flush and replace brake fluid.\"),\n",
    "    (\"How do I properly maintain my car's exhaust system?\", \"Ensure the exhaust system is free of leaks, get it checked for rust or damage, and have it inspected if you notice strange noises or smells.\"),\n",
    "    (\"What is the purpose of the radiator?\", \"The radiator helps cool the engine by dissipating heat from the coolant to keep the engine temperature within the ideal range.\"),\n",
    "    (\"How can I improve my car's handling?\", \"Regularly check tire pressure, replace worn suspension components, and consider upgrading to performance tires or shocks.\"),\n",
    "    (\"What should I do if my car's headlights are dim?\", \"Check for a faulty alternator, replace old bulbs, and inspect the battery connections. It could also be due to dirty lenses.\"),\n",
    "    (\"What is the function of the car's fuel filter?\", \"The fuel filter cleans the fuel before it enters the engine, preventing contaminants from damaging the fuel injectors or engine components.\"),\n",
    "    (\"What should I do if my car is making a grinding noise when braking?\", \"This could indicate worn-out brake pads or damaged rotors. It's important to have them inspected or replaced.\"),\n",
    "    (\"How do I check the transmission fluid?\", \"Transmission fluid can typically be checked using the dipstick under the hood, similar to checking oil, but check your manual for specific instructions.\"),\n",
    "    (\"Why does my car pull to one side while driving?\", \"It could be due to misaligned wheels, uneven tire pressure, or problems with the suspension or steering system.\"),\n",
    "    (\"What is the importance of having the right tire pressure?\", \"Proper tire pressure ensures better fuel efficiency, longer tire life, and improved safety by ensuring optimal tire contact with the road.\"),\n",
    "    (\"What is the purpose of the air filter?\", \"The air filter prevents dirt and debris from entering the engine, ensuring that the engine gets clean air for combustion.\"),\n",
    "    (\"Why is my car engine misfiring?\", \"Engine misfires can be caused by faulty spark plugs, bad ignition coils, or issues with the fuel injectors.\"),\n",
    "    (\"What should I do if my car's brake pedal feels soft?\", \"A soft brake pedal could indicate air in the brake lines, low brake fluid, or worn-out brake pads.\"),\n",
    "    (\"How can I check if my car's suspension is bad?\", \"Signs of bad suspension include a bumpy ride, excessive bouncing after hitting a bump, or difficulty steering.\"),\n",
    "    (\"What is the purpose of the car's thermostat?\", \"The thermostat helps regulate the engine's temperature by controlling the flow of coolant to the engine.\"),\n",
    "    (\"How do I know if my car's fuel injectors need cleaning?\", \"Signs of clogged fuel injectors include rough idling, poor acceleration, or engine misfires.\"),\n",
    "    (\"What should I do if my car's engine is overheating?\", \"If the engine overheats, pull over safely, turn off the AC, let the engine cool down, and check the coolant level.\"),\n",
    "    (\"How can I extend the life of my car's tires?\", \"Maintain proper tire pressure, rotate your tires regularly, and avoid sudden acceleration or hard braking to prevent uneven wear.\"),\n",
    "    (\"What is a car's timing chain and why is it important?\", \"The timing chain synchronizes the rotation of the crankshaft and camshaft, ensuring that the engine's valves open and close at the correct times.\"),\n",
    "    (\"Why does my car smell like burning rubber?\", \"A burning rubber smell could be caused by a slipping drive belt, overheating brakes, or the tires rubbing against something.\"),\n",
    "    (\"What are the signs of a clogged fuel filter?\", \"Signs include poor acceleration, engine sputtering, or difficulty starting the car.\"),\n",
    "    (\"How do I know if my car's alternator is failing?\", \"Symptoms of a failing alternator include dimming headlights, trouble starting the car, or a battery warning light on the dashboard.\"),\n",
    "    (\"What should I do if my car's exhaust system is leaking?\", \"If the exhaust system is leaking, you should get it repaired immediately, as it can be hazardous to your health and the environment.\"),\n",
    "    (\"Why is my car making a whining noise when I turn?\", \"A whining noise when turning could indicate low power steering fluid or a problem with the power steering pump.\"),\n",
    "    (\"How do I check my car's brake fluid?\", \"To check the brake fluid, locate the reservoir under the hood, check the fluid level, and ensure it's within the recommended range.\"),\n",
    "    (\"How do I know when to replace my car's spark plugs?\", \"Spark plugs should be replaced every thirty thousand to a hundred thousand miles, or sooner if you notice misfiring or poor acceleration.\"),\n",
    "    (\"What should I do if my car's engine won't start?\", \"Check the battery, fuel level, starter, and ignition system. It could also be a sign of a dead battery or faulty alternator.\"),\n",
    "    (\"What is the purpose of the O2 sensor?\", \"The O2 sensor measures the amount of oxygen in the exhaust gases, helping the engine control module adjust fuel mixture for better performance and efficiency.\"),\n",
    "    (\"Why does my car's engine sound louder than usual?\", \"A louder engine could indicate a problem with the exhaust system, such as a leak or a damaged muffler.\"),\n",
    "    (\"How do I properly maintain my car's timing belt?\", \"The timing belt should be replaced according to the manufacturer's recommendations, typically every seventy-five thousand to one hundred thousand miles.\"),\n",
    "    (\"Why is my car pulling to one side when braking?\", \"Uneven braking pressure or worn brake pads can cause the car to pull to one side. It might also indicate issues with the suspension or alignment.\"),\n",
    "    (\"What is the purpose of the power steering fluid?\", \"Power steering fluid helps provide the necessary pressure for the power steering system to assist with steering the vehicle.\"),\n",
    "    (\"How do I check the oil level in my car?\", \"To check the oil level, pull out the dipstick, wipe it clean, reinsert it fully, and then pull it out again to check the oil level.\"),\n",
    "    (\"What should I do if my car's air conditioning is not blowing cold air?\", \"Check the refrigerant level, inspect the AC compressor, or replace a faulty condenser to restore cold air flow.\"),\n",
    "    (\"Why is my car shaking when I drive at high speeds?\", \"Shaking at high speeds could indicate issues like unbalanced tires, misaligned wheels, or suspension problems.\"),\n",
    "    (\"How can I prevent my car from rusting?\", \"Regularly wash your car, apply rust protection coatings, and avoid driving on salty roads during winter to prevent rust buildup.\"),\n",
    "    (\"What is the purpose of the radiator in my car?\", \"The radiator helps to cool the engine by circulating coolant, maintaining the engine's optimal temperature.\"),\n",
    "    (\"How do I know if my car's water pump is failing?\", \"A failing water pump can cause engine overheating, coolant leaks, or a whining noise when the car is running.\"),\n",
    "    (\"Why is my car stalling?\", \"Stalling can be caused by issues with the fuel system, ignition system, or a malfunctioning sensor or throttle body.\"),\n",
    "    (\"How do I know if my car's timing belt is damaged?\", \"Signs of a damaged timing belt include engine misfires, strange noises, or the engine refusing to start.\"),\n",
    "    (\"What are the symptoms of a failing fuel pump?\", \"Symptoms include difficulty starting the car, sputtering engine, or losing power while driving.\"),\n",
    "    (\"How do I properly maintain my car's transmission?\", \"Regularly change the transmission fluid according to the manufacturer's recommendations, and avoid overloading the car.\"),\n",
    "    (\"What is the difference between disc brakes and drum brakes?\", \"Disc brakes use a rotor and caliper to stop the car, while drum brakes use a set of shoes that press against the inside of a drum to slow the vehicle.\"),\n",
    "    (\"What causes my car's battery to die?\", \"A battery can die due to age, a faulty alternator, leaving lights or accessories on, or extreme weather conditions.\"),\n",
    "    (\"How do I know if my car's clutch is going bad?\", \"Symptoms of a bad clutch include difficulty shifting gears, slippage, or a burning smell when driving.\"),\n",
    "    (\"What is a car's ECU and what does it do?\", \"The ECU (Engine Control Unit) is the computer that controls the engine's fuel injection, ignition timing, and other vital functions.\"),\n",
    "    (\"How do I know if my car's shocks are worn out?\", \"Signs of worn shocks include a bouncy ride, difficulty controlling the car, or a decrease in handling stability.\"),\n",
    "    (\"What should I do if my car's check engine light comes on?\", \"First, check for any obvious issues, like loose gas caps, then take the car to a mechanic to diagnose and repair any underlying problems.\"),\n",
    "    (\"How can I tell if my car's exhaust system is clogged?\", \"Signs of a clogged exhaust system include reduced engine performance, poor fuel efficiency, and loud engine noises.\"),\n",
    "    (\"How often should I get my car's alignment checked?\", \"Car alignment should be checked every one to two years, or more frequently if you notice uneven tire wear or your car pulling to one side.\"),\n",
    "    (\"What should I do if my car's transmission is slipping?\", \"If the transmission is slipping, check the fluid level, replace the fluid if necessary, or take the car to a mechanic for further diagnosis.\"),\n",
    "]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Model Initialization"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "from transformer import Transformer\n",
    "import transformer\n",
    "\n",
    "transformer_model = Transformer(embedding_dim=64, learning_rate=1e-3).to(device=transformer.device)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Model Training"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "transformer_model.train(dataset=qa_pairs, epochs=500)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Model Saving"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "import torch\n",
    "torch.save(transformer_model.state_dict(), \"./single_headed_transformer_v4_weights.pth\")"
   ]
  }
 ],
 "metadata": {
  "language_info": {
   "name": "python"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}