Che237/cyberforge-notebooks

ML Notebooks Execution Guide

This directory contains machine learning notebooks for the Cyber Forge AI platform. Follow this guide to run the notebooks in the correct order for optimal results.

📋 Prerequisites

Before running any notebooks, ensure you have:

1. Python Environment: Python 3.9+ installed
2. Dependencies: Install all required packages:

   cd ../
   pip install -r requirements.txt
   

3. Jupyter: Install Jupyter Notebook or JupyterLab:

   pip install jupyter jupyterlab
   

🎯 Execution Order

Run the notebooks in this specific order to ensure proper model training and dependencies:

1. Basic AI Agent Training 📚

File: ai_agent_training.py Purpose: Initial AI agent setup and basic training Runtime: ~10-15 minutes Description:

• Sets up the foundational AI agent
• Installs core dependencies programmatically
• Provides basic communication and cybersecurity skills
• RUN THIS FIRST - Required for other notebooks

cd ml-services/notebooks
python ai_agent_training.py

2. Advanced Cybersecurity ML Training 🛡️

File: advanced_cybersecurity_ml_training.ipynb Purpose: Comprehensive ML model training for threat detection Runtime: ~30-45 minutes Description:

• Data preparation and feature engineering
• Multiple ML model training (Random Forest, XGBoost, Neural Networks)
• Model evaluation and comparison
• Production model deployment preparation

jupyter notebook advanced_cybersecurity_ml_training.ipynb

3. Network Security Analysis 🌐

File: network_security_analysis.ipynb Purpose: Network-specific security analysis and monitoring Runtime: ~20-30 minutes Description:

• Network traffic analysis
• Intrusion detection model training
• Port scanning detection
• Network anomaly detection

jupyter notebook network_security_analysis.ipynb

4. Comprehensive AI Agent Training 🤖

File: ai_agent_comprehensive_training.ipynb Purpose: Advanced AI agent with full capabilities Runtime: ~45-60 minutes Description:

• Enhanced communication skills
• Web scraping and threat intelligence
• Real-time monitoring capabilities
• Natural language processing for security analysis
• RUN LAST - Integrates all previous models

jupyter notebook ai_agent_comprehensive_training.ipynb

📊 Expected Outputs

After running all notebooks, you should have:

1. Trained Models: Saved in ../models/ directory
2. Performance Metrics: Evaluation reports and visualizations
3. AI Agent: Fully trained agent ready for deployment
4. Configuration Files: Model configs for production use

🔧 Troubleshooting

Common Issues:

Memory Errors:

• Reduce batch size in deep learning models
• Close other applications to free RAM
• Consider using smaller datasets for testing

Package Installation Failures:

• Update pip: pip install --upgrade pip
• Use conda if pip fails: conda install <package>
• Check Python version compatibility

CUDA/GPU Issues:

• For TensorFlow GPU: Install CUDA 11.8+ and cuDNN
• For CPU-only: Models will run slower but still work
• Check GPU availability: tensorflow.test.is_gpu_available()

Data Download Issues:

• Ensure internet connection for Kaggle datasets
• Set up Kaggle API credentials if needed
• Some notebooks include fallback synthetic data generation

📝 Notes

• First Run: Initial execution takes longer due to package installation and data downloads
• Subsequent Runs: Much faster as dependencies are cached
• Customization: Modify hyperparameters in notebooks for different results
• Production: Use the saved models in the main application

🎯 Next Steps

After completing all notebooks:

1. Deploy Models: Copy trained models to production environment
2. Integration: Connect models with the desktop application
3. Monitoring: Set up model performance monitoring
4. Updates: Retrain models with new data periodically

🆘 Support

If you encounter issues:

1. Check the troubleshooting section above
2. Verify all prerequisites are met
3. Review notebook outputs for specific error messages
4. Create an issue in the repository with error details

---

Happy Training! 🚀