{"index": 0, "content": "Aur\u00e9lien G\u00e9ron\nHands-On \nMachine Learning \nwith Scikit-Learn \n& TensorFlow \nCONCEPTS, TOOLS, AND TECHNIQUES \nTO BUILD INTELLIGENT SYSTEMS\nDownload from finelybook www.finelybook.com\n\nDownload from finelybook www.finelybook.com\n\nAur\u00e9lien G\u00e9ron\nHands-On Machine Learning with\nScikit-Learn and TensorFlow\nConcepts, Tools, and Techniques to\nBuild Intelligent Systems\nBoston\nFarnham\nSebastopol\nTokyo\nBeijing\nBoston\nFarnham\nSebastopol\nTokyo\nBeijing\nDownload from finelybook www.finelybook.com", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 1, "content": "978-1-491-96229-9\n[LSI]\nHands-On Machine Learning with Scikit-Learn and TensorFlow\nby Aur\u00e9lien G\u00e9ron\nCopyright \u00a9 2017 Aur\u00e9lien G\u00e9ron. All rights reserved.\nPrinted in the United States of America.\nPublished by O\u2019Reilly Media, Inc., 1005 Gravenstein Highway North, Sebastopol, CA 95472.\nO\u2019Reilly books may be purchased for educational, business, or sales promotional use. Online editions are\nalso available for most titles (http://oreilly.com/safari). For more information, contact our corporate/insti\u2010\ntutional sales department: 800-998-9938 or corporate@oreilly.com.\nEditor: Nicole Tache\nProduction Editor: Nicholas Adams\nCopyeditor: Rachel Monaghan\nProofreader: Charles Roumeliotis\nIndexer: Wendy Catalano\nInterior Designer: David Futato\nCover Designer: Randy Comer\nIllustrator: Rebecca Demarest\nMarch 2017:\n First Edition\nRevision History for the First Edition\n2017-03-10: First Release\nSee http://oreilly.com/catalog/errata.csp?isbn=9781491962299 for release details.", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 2, "content": "See http://oreilly.com/catalog/errata.csp?isbn=9781491962299 for release details.\nThe O\u2019Reilly logo is a registered trademark of O\u2019Reilly Media, Inc. Hands-On Machine Learning with\nScikit-Learn and TensorFlow, the cover image, and related trade dress are trademarks of O\u2019Reilly Media,\nInc.\nWhile the publisher and the author have used good faith efforts to ensure that the information and\ninstructions contained in this work are accurate, the publisher and the author disclaim all responsibility\nfor errors or omissions, including without limitation responsibility for damages resulting from the use of\nor reliance on this work. Use of the information and instructions contained in this work is at your own\nrisk. If any code samples or other technology this work contains or describes is subject to open source\nlicenses or the intellectual property rights of others, it is your responsibility to ensure that your use\nthereof complies with such licenses and/or rights.", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 3, "content": "thereof complies with such licenses and/or rights.\nDownload from finelybook www.finelybook.com", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 4, "content": "Table of Contents\nPreface. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . xiii\nPart I. \nThe Fundamentals of Machine Learning\n1. The Machine Learning Landscape. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3\nWhat Is Machine Learning? 4\nWhy Use Machine Learning? 4\nTypes of Machine Learning Systems 7\nSupervised/Unsupervised Learning 8\nBatch and Online Learning 14", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 5, "content": "Instance-Based Versus Model-Based Learning 17\nMain Challenges of Machine Learning 22\nInsufficient Quantity of Training Data 22\nNonrepresentative Training Data 24\nPoor-Quality Data 25\nIrrelevant Features 25\nOverfitting the Training Data 26\nUnderfitting the Training Data 28", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 6, "content": "Stepping Back 28\nTesting and Validating 29\nExercises 31\n2. End-to-End Machine Learning Project. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 33\nWorking with Real Data 33\nLook at the Big Picture 35\nFrame the Problem 35\nSelect a Performance Measure 37\niii", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 7, "content": "iii\nDownload from finelybook www.finelybook.com", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 8, "content": "Check the Assumptions 40\nGet the Data 40\nCreate the Workspace 40\nDownload the Data 43\nTake a Quick Look at the Data Structure 45\nCreate a Test Set 49\nDiscover and Visualize the Data to Gain Insights 53\nVisualizing Geographical Data 53", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 9, "content": "Looking for Correlations 55\nExperimenting with Attribute Combinations 58\nPrepare the Data for Machine Learning Algorithms 59\nData Cleaning 60\nHandling Text and Categorical Attributes 62\nCustom Transformers 64\nFeature Scaling 65\nTransformation Pipelines 66", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 10, "content": "Select and Train a Model 68\nTraining and Evaluating on the Training Set 68\nBetter Evaluation Using Cross-Validation 69\nFine-Tune Your Model 71\nGrid Search 72\nRandomized Search 74\nEnsemble Methods 74\nAnalyze the Best Models and Their Errors 74", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 11, "content": "Evaluate Your System on the Test Set 75\nLaunch, Monitor, and Maintain Your System 76\nTry It Out! 77\nExercises 77\n3. Classification. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 79\nMNIST 79\nTraining a Binary Classifier 82\nPerformance Measures 82", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 12, "content": "Measuring Accuracy Using Cross-Validation 83\nConfusion Matrix 84\nPrecision and Recall 86\nPrecision/Recall Tradeoff 87\nThe ROC Curve 91\nMulticlass Classification 93\nError Analysis 96\nMultilabel Classification 100", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 13, "content": "Multioutput Classification 101\niv \n| \nTable of Contents\nDownload from finelybook www.finelybook.com", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 14, "content": "Exercises 102\n4. Training Models. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 105\nLinear Regression 106\nThe Normal Equation 108\nComputational Complexity 110\nGradient Descent 111\nBatch Gradient Descent 114\nStochastic Gradient Descent 117", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 15, "content": "Mini-batch Gradient Descent 119\nPolynomial Regression 121\nLearning Curves 123\nRegularized Linear Models 127\nRidge Regression 127\nLasso Regression 130\nElastic Net 132\nEarly Stopping 133", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 16, "content": "Logistic Regression 134\nEstimating Probabilities 134\nTraining and Cost Function 135\nDecision Boundaries 136\nSoftmax Regression 139\nExercises 142\n5. Support Vector Machines. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 145\nLinear SVM Classification 145", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 17, "content": "Soft Margin Classification 146\nNonlinear SVM Classification 149\nPolynomial Kernel 150\nAdding Similarity Features 151\nGaussian RBF Kernel 152\nComputational Complexity 153\nSVM Regression 154\nUnder the Hood 156", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 18, "content": "Decision Function and Predictions 156\nTraining Objective 157\nQuadratic Programming 159\nThe Dual Problem 160\nKernelized SVM 161\nOnline SVMs 164\nExercises 165\nTable of Contents \n| \nv\nDownload from finelybook www.finelybook.com", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 19, "content": "6. Decision Trees. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 167\nTraining and Visualizing a Decision Tree 167\nMaking Predictions 169\nEstimating Class Probabilities 171\nThe CART Training Algorithm 171\nComputational Complexity 172\nGini Impurity or Entropy? 172\nRegularization Hyperparameters 173", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 20, "content": "Regression 175\nInstability 177\nExercises 178\n7. Ensemble Learning and Random Forests. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 181\nVoting Classifiers 181\nBagging and Pasting 185\nBagging and Pasting in Scikit-Learn 186\nOut-of-Bag Evaluation 187", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 21, "content": "Random Patches and Random Subspaces 188\nRandom Forests 189\nExtra-Trees 190\nFeature Importance 190\nBoosting 191\nAdaBoost 192\nGradient Boosting 195\nStacking 200", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 22, "content": "Exercises 202\n8. Dimensionality Reduction. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 205\nThe Curse of Dimensionality 206\nMain Approaches for Dimensionality Reduction 207\nProjection 207\nManifold Learning 210\nPCA 211\nPreserving the Variance 211", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 23, "content": "Principal Components 212\nProjecting Down to d Dimensions 213\nUsing Scikit-Learn 214\nExplained Variance Ratio 214\nChoosing the Right Number of Dimensions 215\nPCA for Compression 216\nIncremental PCA 217\nRandomized PCA 218\nvi \n| \nTable of Contents\nDownload from finelybook www.finelybook.com", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 24, "content": "Kernel PCA 218\nSelecting a Kernel and Tuning Hyperparameters 219\nLLE 221\nOther Dimensionality Reduction Techniques 223\nExercises 224\nPart II. \nNeural Networks and Deep Learning\n9. Up and Running with TensorFlow. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 229\nInstallation 232\nCreating Your First Graph and Running It in a Session 232", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 25, "content": "Creating Your First Graph and Running It in a Session 232\nManaging Graphs 234\nLifecycle of a Node Value 235\nLinear Regression with TensorFlow 235\nImplementing Gradient Descent 237\nManually Computing the Gradients 237\nUsing autodiff 238\nUsing an Optimizer 239", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 26, "content": "Feeding Data to the Training Algorithm 239\nSaving and Restoring Models 241\nVisualizing the Graph and Training Curves Using TensorBoard 242\nName Scopes 245\nModularity 246\nSharing Variables 248\nExercises 251\n10. Introduction to Artificial Neural Networks. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 253", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 27, "content": "From Biological to Artificial Neurons 254\nBiological Neurons 255\nLogical Computations with Neurons 256\nThe Perceptron 257\nMulti-Layer Perceptron and Backpropagation 261\nTraining an MLP with TensorFlow\u2019s High-Level API 264\nTraining a DNN Using Plain TensorFlow 265\nConstruction Phase 265", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 28, "content": "Execution Phase 269\nUsing the Neural Network 270\nFine-Tuning Neural Network Hyperparameters 270\nNumber of Hidden Layers 270\nNumber of Neurons per Hidden Layer 272\nActivation Functions 272\nTable of Contents \n| \nvii\nDownload from finelybook www.finelybook.com", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 29, "content": "Exercises 273\n11. Training Deep Neural Nets. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 275\nVanishing/Exploding Gradients Problems 275\nXavier and He Initialization 277\nNonsaturating Activation Functions 279\nBatch Normalization 282\nGradient Clipping 286\nReusing Pretrained Layers 286", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 30, "content": "Reusing a TensorFlow Model 287\nReusing Models from Other Frameworks 288\nFreezing the Lower Layers 289\nCaching the Frozen Layers 290\nTweaking, Dropping, or Replacing the Upper Layers 290\nModel Zoos 291\nUnsupervised Pretraining 291\nPretraining on an Auxiliary Task 292", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 31, "content": "Faster Optimizers 293\nMomentum optimization 294\nNesterov Accelerated Gradient 295\nAdaGrad 296\nRMSProp 298\nAdam Optimization 298\nLearning Rate Scheduling 300\nAvoiding Overfitting Through Regularization 302", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 32, "content": "Early Stopping 303\n\u21131 and \u21132 Regularization 303\nDropout 304\nMax-Norm Regularization 307\nData Augmentation 309\nPractical Guidelines 310\nExercises 311\n12. Distributing TensorFlow Across Devices and Servers. . . . . . . . . . . . . . . . . . . . . . . . . . . 313", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 33, "content": "Multiple Devices on a Single Machine 314\nInstallation 314\nManaging the GPU RAM 317\nPlacing Operations on Devices 318\nParallel Execution 321\nControl Dependencies 323\nMultiple Devices Across Multiple Servers 323\nOpening a Session 325\nviii \n| \nTable of Contents\nDownload from finelybook www.finelybook.com", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 34, "content": "The Master and Worker Services 325\nPinning Operations Across Tasks 326\nSharding Variables Across Multiple Parameter Servers 327\nSharing State Across Sessions Using Resource Containers 328\nAsynchronous Communication Using TensorFlow Queues 329\nLoading Data Directly from the Graph 335\nParallelizing Neural Networks on a TensorFlow Cluster 342\nOne Neural Network per Device 342\nIn-Graph Versus Between-Graph Replication 343", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 35, "content": "In-Graph Versus Between-Graph Replication 343\nModel Parallelism 345\nData Parallelism 347\nExercises 352\n13. Convolutional Neural Networks. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 353\nThe Architecture of the Visual Cortex 354\nConvolutional Layer 355\nFilters 357", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 36, "content": "Stacking Multiple Feature Maps 358\nTensorFlow Implementation 360\nMemory Requirements 362\nPooling Layer 363\nCNN Architectures 365\nLeNet-5 366\nAlexNet 367\nGoogLeNet 368", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 37, "content": "ResNet 372\nExercises 376\n14. Recurrent Neural Networks. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 379\nRecurrent Neurons 380\nMemory Cells 382\nInput and Output Sequences 382\nBasic RNNs in TensorFlow 384\nStatic Unrolling Through Time 385", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 38, "content": "Dynamic Unrolling Through Time 387\nHandling Variable Length Input Sequences 387\nHandling Variable-Length Output Sequences 388\nTraining RNNs 389\nTraining a Sequence Classifier 389\nTraining to Predict Time Series 392\nCreative RNN 396\nDeep RNNs 396\nTable of Contents \n| \nix\nDownload from finelybook www.finelybook.com", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 39, "content": "Distributing a Deep RNN Across Multiple GPUs 397\nApplying Dropout 399\nThe Difficulty of Training over Many Time Steps 400\nLSTM Cell 401\nPeephole Connections 403\nGRU Cell 404\nNatural Language Processing 405\nWord Embeddings 405", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 40, "content": "An Encoder\u2013Decoder Network for Machine Translation 407\nExercises 410\n15. Autoencoders. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 411\nEfficient Data Representations 412\nPerforming PCA with an Undercomplete Linear Autoencoder 413\nStacked Autoencoders 415\nTensorFlow Implementation 416\nTying Weights 417", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 41, "content": "Training One Autoencoder at a Time 418\nVisualizing the Reconstructions 420\nVisualizing Features 421\nUnsupervised Pretraining Using Stacked Autoencoders 422\nDenoising Autoencoders 424\nTensorFlow Implementation 425\nSparse Autoencoders 426\nTensorFlow Implementation 427", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 42, "content": "Variational Autoencoders 428\nGenerating Digits 431\nOther Autoencoders 432\nExercises 433\n16. Reinforcement Learning. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 437\nLearning to Optimize Rewards 438\nPolicy Search 440\nIntroduction to OpenAI Gym 441", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 43, "content": "Neural Network Policies 444\nEvaluating Actions: The Credit Assignment Problem 447\nPolicy Gradients 448\nMarkov Decision Processes 453\nTemporal Difference Learning and Q-Learning 457\nExploration Policies 459\nApproximate Q-Learning 460\nLearning to Play Ms. Pac-Man Using Deep Q-Learning 460\nx \n| \nTable of Contents\nDownload from finelybook www.finelybook.com", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 44, "content": "Exercises 469\nThank You! 470\nA. Exercise Solutions. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 471\nB. Machine Learning Project Checklist. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 497\nC. SVM Dual Problem. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 503\nD. Autodiff. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 507\nE. Other Popular ANN Architectures. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 515", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 45, "content": "Index. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 525\nTable of Contents \n| \nxi\nDownload from finelybook www.finelybook.com", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 46, "content": "Download from finelybook www.finelybook.com", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 47, "content": "1 Available on Hinton\u2019s home page at http://www.cs.toronto.edu/~hinton/.\n2 Despite the fact that Yann Lecun\u2019s deep convolutional neural networks had worked well for image recognition\nsince the 1990s, although they were not as general purpose.\nPreface\nThe Machine Learning Tsunami\nIn 2006, Geoffrey Hinton et al. published a paper1 showing how to train a deep neural\nnetwork capable of recognizing handwritten digits with state-of-the-art precision\n(>98%). They branded this technique \u201cDeep Learning.\u201d Training a deep neural net\nwas widely considered impossible at the time,2 and most researchers had abandoned\nthe idea since the 1990s. This paper revived the interest of the scientific community\nand before long many new papers demonstrated that Deep Learning was not only\npossible, but capable of mind-blowing achievements that no other Machine Learning\n(ML) technique could hope to match (with the help of tremendous computing power", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 48, "content": "(ML) technique could hope to match (with the help of tremendous computing power\nand great amounts of data). This enthusiasm soon extended to many other areas of\nMachine Learning.\nFast-forward 10 years and Machine Learning has conquered the industry: it is now at\nthe heart of much of the magic in today\u2019s high-tech products, ranking your web\nsearch results, powering your smartphone\u2019s speech recognition, and recommending\nvideos, beating the world champion at the game of Go. Before you know it, it will be\ndriving your car.\nMachine Learning in Your Projects\nSo naturally you are excited about Machine Learning and you would love to join the\nparty!\nPerhaps you would like to give your homemade robot a brain of its own? Make it rec\u2010\nognize faces? Or learn to walk around?\nxiii\nDownload from finelybook www.finelybook.com", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 49, "content": "Or maybe your company has tons of data (user logs, financial data, production data,\nmachine sensor data, hotline stats, HR reports, etc.), and more than likely you could\nunearth some hidden gems if you just knew where to look; for example:\n\u2022 Segment customers and find the best marketing strategy for each group\n\u2022 Recommend products for each client based on what similar clients bought\n\u2022 Detect which transactions are likely to be fraudulent\n\u2022 Predict next year\u2019s revenue\n\u2022 And more\nWhatever the reason, you have decided to learn Machine Learning and implement it\nin your projects. Great idea!\nObjective and Approach\nThis book assumes that you know close to nothing about Machine Learning. Its goal\nis to give you the concepts, the intuitions, and the tools you need to actually imple\u2010\nment programs capable of learning from data.\nWe will cover a large number of techniques, from the simplest and most commonly\nused (such as linear regression) to some of the Deep Learning techniques that regu\u2010", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 50, "content": "used (such as linear regression) to some of the Deep Learning techniques that regu\u2010\nlarly win competitions.\nRather than implementing our own toy versions of each algorithm, we will be using\nactual production-ready Python frameworks:\n\u2022 Scikit-Learn is very easy to use, yet it implements many Machine Learning algo\u2010\nrithms efficiently, so it makes for a great entry point to learn Machine Learning.\n\u2022 TensorFlow is a more complex library for distributed numerical computation\nusing data flow graphs. It makes it possible to train and run very large neural net\u2010\nworks efficiently by distributing the computations across potentially thousands\nof multi-GPU servers. TensorFlow was created at Google and supports many of\ntheir large-scale Machine Learning applications. It was open-sourced in Novem\u2010\nber 2015.\nThe book favors a hands-on approach, growing an intuitive understanding of\nMachine Learning through concrete working examples and just a little bit of theory.", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 51, "content": "Machine Learning through concrete working examples and just a little bit of theory.\nWhile you can read this book without picking up your laptop, we highly recommend\nyou experiment with the code examples available online as Jupyter notebooks at\nhttps://github.com/ageron/handson-ml.\nxiv \n| \nPreface\nDownload from finelybook www.finelybook.com", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 52, "content": "Prerequisites\nThis book assumes that you have some Python programming experience and that you\nare familiar with Python\u2019s main scientific libraries, in particular NumPy, Pandas, and\nMatplotlib.\nAlso, if you care about what\u2019s under the hood you should have a reasonable under\u2010\nstanding of college-level math as well (calculus, linear algebra, probabilities, and sta\u2010\ntistics).\nIf you don\u2019t know Python yet, http://learnpython.org/ is a great place to start. The offi\u2010\ncial tutorial on python.org is also quite good.\nIf you have never used Jupyter, Chapter 2 will guide you through installation and the\nbasics: it is a great tool to have in your toolbox.\nIf you are not familiar with Python\u2019s scientific libraries, the provided Jupyter note\u2010\nbooks include a few tutorials. There is also a quick math tutorial for linear algebra.\nRoadmap\nThis book is organized in two parts. Part I, The Fundamentals of Machine Learning,\ncovers the following topics:", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 53, "content": "covers the following topics:\n\u2022 What is Machine Learning? What problems does it try to solve? What are the\nmain categories and fundamental concepts of Machine Learning systems?\n\u2022 The main steps in a typical Machine Learning project.\n\u2022 Learning by fitting a model to data.\n\u2022 Optimizing a cost function.\n\u2022 Handling, cleaning, and preparing data.\n\u2022 Selecting and engineering features.\n\u2022 Selecting a model and tuning hyperparameters using cross-validation.\n\u2022 The main challenges of Machine Learning, in particular underfitting and overfit\u2010\nting (the bias/variance tradeoff).\n\u2022 Reducing the dimensionality of the training data to fight the curse of dimension\u2010\nality.\n\u2022 The most common learning algorithms: Linear and Polynomial Regression,\nLogistic Regression, k-Nearest Neighbors, Support Vector Machines, Decision\nTrees, Random Forests, and Ensemble methods.\nPreface \n| \nxv\nDownload from finelybook www.finelybook.com", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 54, "content": "Part II, Neural Networks and Deep Learning, covers the following topics:\n\u2022 What are neural nets? What are they good for?\n\u2022 Building and training neural nets using TensorFlow.\n\u2022 The most important neural net architectures: feedforward neural nets, convolu\u2010\ntional nets, recurrent nets, long short-term memory (LSTM) nets, and autoen\u2010\ncoders.\n\u2022 Techniques for training deep neural nets.\n\u2022 Scaling neural networks for huge datasets.\n\u2022 Reinforcement learning.\nThe first part is based mostly on Scikit-Learn while the second part uses TensorFlow.\nDon\u2019t jump into deep waters too hastily: while Deep Learning is no\ndoubt one of the most exciting areas in Machine Learning, you\nshould master the fundamentals first. Moreover, most problems\ncan be solved quite well using simpler techniques such as Random\nForests and Ensemble methods (discussed in Part I). Deep Learn\u2010\ning is best suited for complex problems such as image recognition,\nspeech recognition, or natural language processing, provided you", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 55, "content": "speech recognition, or natural language processing, provided you\nhave enough data, computing power, and patience.\nOther Resources\nMany resources are available to learn about Machine Learning. Andrew Ng\u2019s ML\ncourse on Coursera and Geoffrey Hinton\u2019s course on neural networks and Deep\nLearning are amazing, although they both require a significant time investment\n(think months).\nThere are also many interesting websites about Machine Learning, including of\ncourse Scikit-Learn\u2019s exceptional User Guide. You may also enjoy Dataquest, which\nprovides very nice interactive tutorials, and ML blogs such as those listed on Quora.\nFinally, the Deep Learning website has a good list of resources to learn more.\nOf course there are also many other introductory books about Machine Learning, in\nparticular:\n\u2022 Joel Grus, Data Science from Scratch (O\u2019Reilly). This book presents the funda\u2010\nmentals of Machine Learning, and implements some of the main algorithms in", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 56, "content": "mentals of Machine Learning, and implements some of the main algorithms in\npure Python (from scratch, as the name suggests).\n\u2022 Stephen Marsland, Machine Learning: An Algorithmic Perspective (Chapman and\nHall). This book is a great introduction to Machine Learning, covering a wide\nxvi \n| \nPreface\nDownload from finelybook www.finelybook.com", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 57, "content": "range of topics in depth, with code examples in Python (also from scratch, but\nusing NumPy).\n\u2022 Sebastian Raschka, Python Machine Learning (Packt Publishing). Also a great\nintroduction to Machine Learning, this book leverages Python open source libra\u2010\nries (Pylearn 2 and Theano).\n\u2022 Yaser S. Abu-Mostafa, Malik Magdon-Ismail, and Hsuan-Tien Lin, Learning from\nData (AMLBook). A rather theoretical approach to ML, this book provides deep\ninsights, in particular on the bias/variance tradeoff (see Chapter 4).\n\u2022 Stuart Russell and Peter Norvig, Artificial Intelligence: A Modern Approach, 3rd\nEdition (Pearson). This is a great (and huge) book covering an incredible amount\nof topics, including Machine Learning. It helps put ML into perspective.\nFinally, a great way to learn is to join ML competition websites such as Kaggle.com\nthis will allow you to practice your skills on real-world problems, with help and\ninsights from some of the best ML professionals out there.\nConventions Used in This Book", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 58, "content": "insights from some of the best ML professionals out there.\nConventions Used in This Book\nThe following typographical conventions are used in this book:\nItalic\nIndicates new terms, URLs, email addresses, filenames, and file extensions.\nConstant width\nUsed for program listings, as well as within paragraphs to refer to program ele\u2010\nments such as variable or function names, databases, data types, environment\nvariables, statements and keywords.\nConstant width bold\nShows commands or other text that should be typed literally by the user.\nConstant width italic\nShows text that should be replaced with user-supplied values or by values deter\u2010\nmined by context.\nThis element signifies a tip or suggestion.\nPreface \n| \nxvii\nDownload from finelybook www.finelybook.com", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 59, "content": "This element signifies a general note.\nThis element indicates a warning or caution.\nUsing Code Examples\nSupplemental material (code examples, exercises, etc.) is available for download at\nhttps://github.com/ageron/handson-ml.\nThis book is here to help you get your job done. In general, if example code is offered\nwith this book, you may use it in your programs and documentation. You do not\nneed to contact us for permission unless you\u2019re reproducing a significant portion of\nthe code. For example, writing a program that uses several chunks of code from this\nbook does not require permission. Selling or distributing a CD-ROM of examples\nfrom O\u2019Reilly books does require permission. Answering a question by citing this\nbook and quoting example code does not require permission. Incorporating a signifi\u2010\ncant amount of example code from this book into your product\u2019s documentation does\nrequire permission.\nWe appreciate, but do not require, attribution. An attribution usually includes the", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 60, "content": "We appreciate, but do not require, attribution. An attribution usually includes the\ntitle, author, publisher, and ISBN. For example: \u201cHands-On Machine Learning with\nScikit-Learn and TensorFlow by Aur\u00e9lien G\u00e9ron (O\u2019Reilly). Copyright 2017 Aur\u00e9lien\nG\u00e9ron, 978-1-491-96229-9.\u201d\nIf you feel your use of code examples falls outside fair use or the permission given\nabove, feel free to contact us at permissions@oreilly.com.\nO\u2019Reilly Safari\nSafari (formerly Safari Books Online) is a membership-based\ntraining and reference platform for enterprise, government,\neducators, and individuals.\nMembers have access to thousands of books, training videos, Learning Paths, interac\u2010\ntive tutorials, and curated playlists from over 250 publishers, including O\u2019Reilly\nMedia, Harvard Business Review, Prentice Hall Professional, Addison-Wesley Profes\u2010\nsional, Microsoft Press, Sams, Que, Peachpit Press, Adobe, Focal Press, Cisco Press,\nxviii \n| \nPreface\nDownload from finelybook www.finelybook.com", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 61, "content": "John Wiley & Sons, Syngress, Morgan Kaufmann, IBM Redbooks, Packt, Adobe\nPress, FT Press, Apress, Manning, New Riders, McGraw-Hill, Jones & Bartlett, and\nCourse Technology, among others.\nFor more information, please visit http://oreilly.com/safari.\nHow to Contact Us\nPlease address comments and questions concerning this book to the publisher:\nO\u2019Reilly Media, Inc.\n1005 Gravenstein Highway North\nSebastopol, CA 95472\n800-998-9938 (in the United States or Canada)\n707-829-0515 (international or local)\n707-829-0104 (fax)\nWe have a web page for this book, where we list errata, examples, and any additional\ninformation. You can access this page at http://bit.ly/hands-on-machine-learning-\nwith-scikit-learn-and-tensorflow.\nTo comment or ask technical questions about this book, send email to bookques\u2010\ntions@oreilly.com.\nFor more information about our books, courses, conferences, and news, see our web\u2010\nsite at http://www.oreilly.com.\nFind us on Facebook: http://facebook.com/oreilly", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 62, "content": "site at http://www.oreilly.com.\nFind us on Facebook: http://facebook.com/oreilly\nFollow us on Twitter: http://twitter.com/oreillymedia\nWatch us on YouTube: http://www.youtube.com/oreillymedia\nAcknowledgments\nI would like to thank my Google colleagues, in particular the YouTube video classifi\u2010\ncation team, for teaching me so much about Machine Learning. I could never have\nstarted this project without them. Special thanks to my personal ML gurus: Cl\u00e9ment\nCourbet, Julien Dubois, Mathias Kende, Daniel Kitachewsky, James Pack, Alexander\nPak, Anosh Raj, Vitor Sessak, Wiktor Tomczak, Ingrid von Glehn, Rich Washington,\nand everyone at YouTube Paris.\nI am incredibly grateful to all the amazing people who took time out of their busy\nlives to review my book in so much detail. Thanks to Pete Warden for answering all\nmy TensorFlow questions, reviewing Part II, providing many interesting insights, and\nof course for being part of the core TensorFlow team. You should definitely check out\nPreface \n|", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 63, "content": "of course for being part of the core TensorFlow team. You should definitely check out\nPreface \n| \nxix\nDownload from finelybook www.finelybook.com", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 64, "content": "his blog! Many thanks to Lukas Biewald for his very thorough review of Part II: he left\nno stone unturned, tested all the code (and caught a few errors), made many great\nsuggestions, and his enthusiasm was contagious. You should check out his blog and\nhis cool robots! Thanks to Justin Francis, who also reviewed Part II very thoroughly,\ncatching errors and providing great insights, in particular in Chapter 16. Check out\nhis posts on TensorFlow!\nHuge thanks as well to David Andrzejewski, who reviewed Part I and provided\nincredibly useful feedback, identifying unclear sections and suggesting how to\nimprove them. Check out his website! Thanks to Gr\u00e9goire Mesnil, who reviewed\nPart II and contributed very interesting practical advice on training neural networks.\nThanks as well to Eddy Hung, Salim S\u00e9maoune, Karim Matrah, Ingrid von Glehn,\nIain Smears, and Vincent Guilbeau for reviewing Part I and making many useful sug\u2010", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 65, "content": "Iain Smears, and Vincent Guilbeau for reviewing Part I and making many useful sug\u2010\ngestions. And I also wish to thank my father-in-law, Michel Tessier, former mathe\u2010\nmatics teacher and now a great translator of Anton Chekhov, for helping me iron out\nsome of the mathematics and notations in this book and reviewing the linear algebra\nJupyter notebook.\nAnd of course, a gigantic \u201cthank you\u201d to my dear brother Sylvain, who reviewed every\nsingle chapter, tested every line of code, provided feedback on virtually every section,\nand encouraged me from the first line to the last. Love you, bro!\nMany thanks as well to O\u2019Reilly\u2019s fantastic staff, in particular Nicole Tache, who gave\nme insightful feedback, always cheerful, encouraging, and helpful. Thanks as well to\nMarie Beaugureau, Ben Lorica, Mike Loukides, and Laurel Ruma for believing in this\nproject and helping me define its scope. Thanks to Matt Hacker and all of the Atlas", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 66, "content": "project and helping me define its scope. Thanks to Matt Hacker and all of the Atlas\nteam for answering all my technical questions regarding formatting, asciidoc, and\nLaTeX, and thanks to Rachel Monaghan, Nick Adams, and all of the production team\nfor their final review and their hundreds of corrections.\nLast but not least, I am infinitely grateful to my beloved wife, Emmanuelle, and to our\nthree wonderful kids, Alexandre, R\u00e9mi, and Gabrielle, for encouraging me to work\nhard on this book, asking many questions (who said you can\u2019t teach neural networks\nto a seven-year-old?), and even bringing me cookies and coffee. What more can one\ndream of?\nxx \n| \nPreface\nDownload from finelybook www.finelybook.com", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 67, "content": "PART I\nThe Fundamentals of\nMachine Learning\nDownload from finelybook www.finelybook.com\n\nDownload from finelybook www.finelybook.com", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 68, "content": "CHAPTER 1\nThe Machine Learning Landscape\nWhen most people hear \u201cMachine Learning,\u201d they picture a robot: a dependable but\u2010\nler or a deadly Terminator depending on who you ask. But Machine Learning is not\njust a futuristic fantasy, it\u2019s already here. In fact, it has been around for decades in\nsome specialized applications, such as Optical Character Recognition (OCR). But the\nfirst ML application that really became mainstream, improving the lives of hundreds\nof millions of people, took over the world back in the 1990s: it was the spam filter.\nNot exactly a self-aware Skynet, but it does technically qualify as Machine Learning\n(it has actually learned so well that you seldom need to flag an email as spam any\u2010\nmore). It was followed by hundreds of ML applications that now quietly power hun\u2010\ndreds of products and features that you use regularly, from better recommendations\nto voice search.\nWhere does Machine Learning start and where does it end? What exactly does it", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 69, "content": "to voice search.\nWhere does Machine Learning start and where does it end? What exactly does it\nmean for a machine to learn something? If I download a copy of Wikipedia, has my\ncomputer really \u201clearned\u201d something? Is it suddenly smarter? In this chapter we will\nstart by clarifying what Machine Learning is and why you may want to use it.\nThen, before we set out to explore the Machine Learning continent, we will take a\nlook at the map and learn about the main regions and the most notable landmarks:\nsupervised versus unsupervised learning, online versus batch learning, instance-\nbased versus model-based learning. Then we will look at the workflow of a typical ML\nproject, discuss the main challenges you may face, and cover how to evaluate and\nfine-tune a Machine Learning system.\nThis chapter introduces a lot of fundamental concepts (and jargon) that every data\nscientist should know by heart. It will be a high-level overview (the only chapter", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 70, "content": "scientist should know by heart. It will be a high-level overview (the only chapter\nwithout much code), all rather simple, but you should make sure everything is\ncrystal-clear to you before continuing to the rest of the book. So grab a coffee and let\u2019s\nget started!\n3\nDownload from finelybook www.finelybook.com", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 71, "content": "If you already know all the Machine Learning basics, you may want\nto skip directly to Chapter 2. If you are not sure, try to answer all\nthe questions listed at the end of the chapter before moving on.\nWhat Is Machine Learning?\nMachine Learning is the science (and art) of programming computers so they can\nlearn from data.\nHere is a slightly more general definition:\n[Machine Learning is the] field of study that gives computers the ability to learn\nwithout being explicitly programmed.\n\u2014Arthur Samuel, 1959\nAnd a more engineering-oriented one:\nA computer program is said to learn from experience E with respect to some task T\nand some performance measure P, if its performance on T, as measured by P, improves\nwith experience E.\n\u2014Tom Mitchell, 1997\nFor example, your spam filter is a Machine Learning program that can learn to flag\nspam given examples of spam emails (e.g., flagged by users) and examples of regular", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 72, "content": "spam given examples of spam emails (e.g., flagged by users) and examples of regular\n(nonspam, also called \u201cham\u201d) emails. The examples that the system uses to learn are\ncalled the training set. Each training example is called a training instance (or sample).\nIn this case, the task T is to flag spam for new emails, the experience E is the training\ndata, and the performance measure P needs to be defined; for example, you can use\nthe ratio of correctly classified emails. This particular performance measure is called\naccuracy and it is often used in classification tasks.\nIf you just download a copy of Wikipedia, your computer has a lot more data, but it is\nnot suddenly better at any task. Thus, it is not Machine Learning.\nWhy Use Machine Learning?\nConsider how you would write a spam filter using traditional programming techni\u2010\nques (Figure 1-1):\n1. First you would look at what spam typically looks like. You might notice that", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 73, "content": "ques (Figure 1-1):\n1. First you would look at what spam typically looks like. You might notice that\nsome words or phrases (such as \u201c4U,\u201d \u201ccredit card,\u201d \u201cfree,\u201d and \u201camazing\u201d) tend to\ncome up a lot in the subject. Perhaps you would also notice a few other patterns\nin the sender\u2019s name, the email\u2019s body, and so on.\n4 \n| \nChapter 1: The Machine Learning Landscape\nDownload from finelybook www.finelybook.com", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 74, "content": "2. You would write a detection algorithm for each of the patterns that you noticed,\nand your program would flag emails as spam if a number of these patterns are\ndetected.\n3. You would test your program, and repeat steps 1 and 2 until it is good enough.\nFigure 1-1. The traditional approach\nSince the problem is not trivial, your program will likely become a long list of com\u2010\nplex rules\u2014pretty hard to maintain.\nIn contrast, a spam filter based on Machine Learning techniques automatically learns\nwhich words and phrases are good predictors of spam by detecting unusually fre\u2010\nquent patterns of words in the spam examples compared to the ham examples\n(Figure 1-2). The program is much shorter, easier to maintain, and most likely more\naccurate.\nFigure 1-2. Machine Learning approach\nWhy Use Machine Learning? \n| \n5\nDownload from finelybook www.finelybook.com", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 75, "content": "Moreover, if spammers notice that all their emails containing \u201c4U\u201d are blocked, they\nmight start writing \u201cFor U\u201d instead. A spam filter using traditional programming\ntechniques would need to be updated to flag \u201cFor U\u201d emails. If spammers keep work\u2010\ning around your spam filter, you will need to keep writing new rules forever.\nIn contrast, a spam filter based on Machine Learning techniques automatically noti\u2010\nces that \u201cFor U\u201d has become unusually frequent in spam flagged by users, and it starts\nflagging them without your intervention (Figure 1-3).\nFigure 1-3. Automatically adapting to change\nAnother area where Machine Learning shines is for problems that either are too com\u2010\nplex for traditional approaches or have no known algorithm. For example, consider \nspeech recognition: say you want to start simple and write a program capable of dis\u2010\ntinguishing the words \u201cone\u201d and \u201ctwo.\u201d You might notice that the word \u201ctwo\u201d starts", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 76, "content": "tinguishing the words \u201cone\u201d and \u201ctwo.\u201d You might notice that the word \u201ctwo\u201d starts\nwith a high-pitch sound (\u201cT\u201d), so you could hardcode an algorithm that measures\nhigh-pitch sound intensity and use that to distinguish ones and twos. Obviously this\ntechnique will not scale to thousands of words spoken by millions of very different\npeople in noisy environments and in dozens of languages. The best solution (at least\ntoday) is to write an algorithm that learns by itself, given many example recordings\nfor each word.\nFinally, Machine Learning can help humans learn (Figure 1-4): ML algorithms can be\ninspected to see what they have learned (although for some algorithms this can be\ntricky). For instance, once the spam filter has been trained on enough spam, it can\neasily be inspected to reveal the list of words and combinations of words that it\nbelieves are the best predictors of spam. Sometimes this will reveal unsuspected cor\u2010", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 77, "content": "believes are the best predictors of spam. Sometimes this will reveal unsuspected cor\u2010\nrelations or new trends, and thereby lead to a better understanding of the problem.\nApplying ML techniques to dig into large amounts of data can help discover patterns\nthat were not immediately apparent. This is called data mining.\n6 \n| \nChapter 1: The Machine Learning Landscape\nDownload from finelybook www.finelybook.com", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 78, "content": "Figure 1-4. Machine Learning can help humans learn\nTo summarize, Machine Learning is great for:\n\u2022 Problems for which existing solutions require a lot of hand-tuning or long lists of\nrules: one Machine Learning algorithm can often simplify code and perform bet\u2010\nter.\n\u2022 Complex problems for which there is no good solution at all using a traditional\napproach: the best Machine Learning techniques can find a solution.\n\u2022 Fluctuating environments: a Machine Learning system can adapt to new data.\n\u2022 Getting insights about complex problems and large amounts of data.\nTypes of Machine Learning Systems\nThere are so many different types of Machine Learning systems that it is useful to\nclassify them in broad categories based on:\n\u2022 Whether or not they are trained with human supervision (supervised, unsuper\u2010\nvised, semisupervised, and Reinforcement Learning)\n\u2022 Whether or not they can learn incrementally on the fly (online versus batch\nlearning)", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 79, "content": "\u2022 Whether or not they can learn incrementally on the fly (online versus batch\nlearning)\n\u2022 Whether they work by simply comparing new data points to known data points,\nor instead detect patterns in the training data and build a predictive model, much\nlike scientists do (instance-based versus model-based learning)\nThese criteria are not exclusive; you can combine them in any way you like. For\nexample, a state-of-the-art spam filter may learn on the fly using a deep neural net\u2010\nTypes of Machine Learning Systems \n| \n7\nDownload from finelybook www.finelybook.com", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 80, "content": "1 Fun fact: this odd-sounding name is a statistics term introduced by Francis Galton while he was studying the\nfact that the children of tall people tend to be shorter than their parents. Since children were shorter, he called\nthis regression to the mean. This name was then applied to the methods he used to analyze correlations\nbetween variables.\nwork model trained using examples of spam and ham; this makes it an online, model-\nbased, supervised learning system.\nLet\u2019s look at each of these criteria a bit more closely.\nSupervised/Unsupervised Learning\nMachine Learning systems can be classified according to the amount and type of\nsupervision they get during training. There are four major categories: supervised\nlearning, unsupervised learning, semisupervised learning, and Reinforcement Learn\u2010\ning.\nSupervised learning\nIn supervised learning, the training data you feed to the algorithm includes the desired\nsolutions, called labels (Figure 1-5).", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 81, "content": "solutions, called labels (Figure 1-5).\nFigure 1-5. A labeled training set for supervised learning (e.g., spam classification)\nA typical supervised learning task is classification. The spam filter is a good example\nof this: it is trained with many example emails along with their class (spam or ham),\nand it must learn how to classify new emails.\nAnother typical task is to predict a target numeric value, such as the price of a car,\ngiven a set of features (mileage, age, brand, etc.) called predictors. This sort of task is \ncalled regression (Figure 1-6).1 To train the system, you need to give it many examples\nof cars, including both their predictors and their labels (i.e., their prices).\n8 \n| \nChapter 1: The Machine Learning Landscape\nDownload from finelybook www.finelybook.com", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 82, "content": "2 Some neural network architectures can be unsupervised, such as autoencoders and restricted Boltzmann\nmachines. They can also be semisupervised, such as in deep belief networks and unsupervised pretraining.\nIn Machine Learning an attribute is a data type (e.g., \u201cMileage\u201d),\nwhile a feature has several meanings depending on the context, but\ngenerally means an attribute plus its value (e.g., \u201cMileage =\n15,000\u201d). Many people use the words attribute and feature inter\u2010\nchangeably, though.\nFigure 1-6. Regression\nNote that some regression algorithms can be used for classification as well, and vice\nversa. For example, Logistic Regression is commonly used for classification, as it can\noutput a value that corresponds to the probability of belonging to a given class (e.g.,\n20% chance of being spam).\nHere are some of the most important supervised learning algorithms (covered in this\nbook):\n\u2022 k-Nearest Neighbors\n\u2022 Linear Regression\n\u2022 Logistic Regression\n\u2022 Support Vector Machines (SVMs)", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 83, "content": "\u2022 k-Nearest Neighbors\n\u2022 Linear Regression\n\u2022 Logistic Regression\n\u2022 Support Vector Machines (SVMs)\n\u2022 Decision Trees and Random Forests\n\u2022 Neural networks2\nTypes of Machine Learning Systems \n| \n9\nDownload from finelybook www.finelybook.com", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 84, "content": "Unsupervised learning\nIn unsupervised learning, as you might guess, the training data is unlabeled\n(Figure 1-7). The system tries to learn without a teacher.\nFigure 1-7. An unlabeled training set for unsupervised learning\nHere are some of the most important unsupervised learning algorithms (we will\ncover dimensionality reduction in Chapter 8):\n\u2022 Clustering\n\u2014 k-Means\n\u2014 Hierarchical Cluster Analysis (HCA)\n\u2014 Expectation Maximization\n\u2022 Visualization and dimensionality reduction\n\u2014 Principal Component Analysis (PCA)\n\u2014 Kernel PCA\n\u2014 Locally-Linear Embedding (LLE)\n\u2014 t-distributed Stochastic Neighbor Embedding (t-SNE)\n\u2022 Association rule learning\n\u2014 Apriori\n\u2014 Eclat\nFor example, say you have a lot of data about your blog\u2019s visitors. You may want to\nrun a clustering algorithm to try to detect groups of similar visitors (Figure 1-8). At\nno point do you tell the algorithm which group a visitor belongs to: it finds those", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 85, "content": "no point do you tell the algorithm which group a visitor belongs to: it finds those\nconnections without your help. For example, it might notice that 40% of your visitors\nare males who love comic books and generally read your blog in the evening, while\n20% are young sci-fi lovers who visit during the weekends, and so on. If you use a\nhierarchical clustering algorithm, it may also subdivide each group into smaller\ngroups. This may help you target your posts for each group.\n10 \n| \nChapter 1: The Machine Learning Landscape\nDownload from finelybook www.finelybook.com", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 86, "content": "3 Notice how animals are rather well separated from vehicles, how horses are close to deer but far from birds,\nand so on. Figure reproduced with permission from Socher, Ganjoo, Manning, and Ng (2013), \u201cT-SNE visual\u2010\nization of the semantic word space.\u201d\nFigure 1-8. Clustering\nVisualization algorithms are also good examples of unsupervised learning algorithms:\nyou feed them a lot of complex and unlabeled data, and they output a 2D or 3D rep\u2010\nresentation of your data that can easily be plotted (Figure 1-9). These algorithms try\nto preserve as much structure as they can (e.g., trying to keep separate clusters in the\ninput space from overlapping in the visualization), so you can understand how the\ndata is organized and perhaps identify unsuspected patterns.\nFigure 1-9. Example of a t-SNE visualization highlighting semantic clusters3\nTypes of Machine Learning Systems \n| \n11\nDownload from finelybook www.finelybook.com", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 87, "content": "A related task is dimensionality reduction, in which the goal is to simplify the data\nwithout losing too much information. One way to do this is to merge several correla\u2010\nted features into one. For example, a car\u2019s mileage may be very correlated with its age,\nso the dimensionality reduction algorithm will merge them into one feature that rep\u2010\nresents the car\u2019s wear and tear. This is called feature extraction.\nIt is often a good idea to try to reduce the dimension of your train\u2010\ning data using a dimensionality reduction algorithm before you\nfeed it to another Machine Learning algorithm (such as a super\u2010\nvised learning algorithm). It will run much faster, the data will take\nup less disk and memory space, and in some cases it may also per\u2010\nform better.\nYet another important unsupervised task is anomaly detection\u2014for example, detect\u2010\ning unusual credit card transactions to prevent fraud, catching manufacturing defects,", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 88, "content": "ing unusual credit card transactions to prevent fraud, catching manufacturing defects,\nor automatically removing outliers from a dataset before feeding it to another learn\u2010\ning algorithm. The system is trained with normal instances, and when it sees a new\ninstance it can tell whether it looks like a normal one or whether it is likely an anom\u2010\naly (see Figure 1-10).\nFigure 1-10. Anomaly detection\nFinally, another common unsupervised task is association rule learning, in which the\ngoal is to dig into large amounts of data and discover interesting relations between\nattributes. For example, suppose you own a supermarket. Running an association rule\non your sales logs may reveal that people who purchase barbecue sauce and potato\nchips also tend to buy steak. Thus, you may want to place these items close to each \nother.\n12 \n| \nChapter 1: The Machine Learning Landscape\nDownload from finelybook www.finelybook.com", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 89, "content": "4 That\u2019s when the system works perfectly. In practice it often creates a few clusters per person, and sometimes\nmixes up two people who look alike, so you need to provide a few labels per person and manually clean up\nsome clusters.\nSemisupervised learning\nSome algorithms can deal with partially labeled training data, usually a lot of unla\u2010\nbeled data and a little bit of labeled data. This is called semisupervised learning\n(Figure 1-11).\nSome photo-hosting services, such as Google Photos, are good examples of this. Once\nyou upload all your family photos to the service, it automatically recognizes that the\nsame person A shows up in photos 1, 5, and 11, while another person B shows up in\nphotos 2, 5, and 7. This is the unsupervised part of the algorithm (clustering). Now all\nthe system needs is for you to tell it who these people are. Just one label per person,4\nand it is able to name everyone in every photo, which is useful for searching photos.\nFigure 1-11. Semisupervised learning", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 90, "content": "Figure 1-11. Semisupervised learning\nMost semisupervised learning algorithms are combinations of unsupervised and\nsupervised algorithms. For example, deep belief networks (DBNs) are based on unsu\u2010\npervised components called restricted Boltzmann machines (RBMs) stacked on top of\none another. RBMs are trained sequentially in an unsupervised manner, and then the\nwhole system is fine-tuned using supervised learning techniques.\nReinforcement Learning\nReinforcement Learning is a very different beast. The learning system, called an agent\nin this context, can observe the environment, select and perform actions, and get\nrewards in return (or penalties in the form of negative rewards, as in Figure 1-12). It\nmust then learn by itself what is the best strategy, called a policy, to get the most\nreward over time. A policy defines what action the agent should choose when it is in a\ngiven situation.\nTypes of Machine Learning Systems \n| \n13\nDownload from finelybook www.finelybook.com", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 91, "content": "Figure 1-12. Reinforcement Learning\nFor example, many robots implement Reinforcement Learning algorithms to learn\nhow to walk. DeepMind\u2019s AlphaGo program is also a good example of Reinforcement\nLearning: it made the headlines in March 2016 when it beat the world champion Lee\nSedol at the game of Go. It learned its winning policy by analyzing millions of games,\nand then playing many games against itself. Note that learning was turned off during\nthe games against the champion; AlphaGo was just applying the policy it had learned.\nBatch and Online Learning\nAnother criterion used to classify Machine Learning systems is whether or not the\nsystem can learn incrementally from a stream of incoming data.\nBatch learning\nIn batch learning, the system is incapable of learning incrementally: it must be trained\nusing all the available data. This will generally take a lot of time and computing\nresources, so it is typically done offline. First the system is trained, and then it is", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 92, "content": "resources, so it is typically done offline. First the system is trained, and then it is\nlaunched into production and runs without learning anymore; it just applies what it\nhas learned. This is called offline learning.\nIf you want a batch learning system to know about new data (such as a new type of\nspam), you need to train a new version of the system from scratch on the full dataset\n(not just the new data, but also the old data), then stop the old system and replace it\nwith the new one.\nFortunately, the whole process of training, evaluating, and launching a Machine\nLearning system can be automated fairly easily (as shown in Figure 1-3), so even a\n14 \n| \nChapter 1: The Machine Learning Landscape\nDownload from finelybook www.finelybook.com", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 93, "content": "batch learning system can adapt to change. Simply update the data and train a new\nversion of the system from scratch as often as needed.\nThis solution is simple and often works fine, but training using the full set of data can\ntake many hours, so you would typically train a new system only every 24 hours or\neven just weekly. If your system needs to adapt to rapidly changing data (e.g., to pre\u2010\ndict stock prices), then you need a more reactive solution.\nAlso, training on the full set of data requires a lot of computing resources (CPU,\nmemory space, disk space, disk I/O, network I/O, etc.). If you have a lot of data and\nyou automate your system to train from scratch every day, it will end up costing you a\nlot of money. If the amount of data is huge, it may even be impossible to use a batch\nlearning algorithm.\nFinally, if your system needs to be able to learn autonomously and it has limited\nresources (e.g., a smartphone application or a rover on Mars), then carrying around", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 94, "content": "resources (e.g., a smartphone application or a rover on Mars), then carrying around\nlarge amounts of training data and taking up a lot of resources to train for hours\nevery day is a showstopper.\nFortunately, a better option in all these cases is to use algorithms that are capable of\nlearning incrementally.\nOnline learning\nIn online learning, you train the system incrementally by feeding it data instances\nsequentially, either individually or by small groups called mini-batches. Each learning\nstep is fast and cheap, so the system can learn about new data on the fly, as it arrives\n(see Figure 1-13).\nFigure 1-13. Online learning\nOnline learning is great for systems that receive data as a continuous flow (e.g., stock\nprices) and need to adapt to change rapidly or autonomously. It is also a good option\nTypes of Machine Learning Systems \n| \n15\nDownload from finelybook www.finelybook.com", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 95, "content": "if you have limited computing resources: once an online learning system has learned\nabout new data instances, it does not need them anymore, so you can discard them\n(unless you want to be able to roll back to a previous state and \u201creplay\u201d the data). This\ncan save a huge amount of space.\nOnline learning algorithms can also be used to train systems on huge datasets that\ncannot fit in one machine\u2019s main memory (this is called out-of-core learning). The\nalgorithm loads part of the data, runs a training step on that data, and repeats the\nprocess until it has run on all of the data (see Figure 1-14).\nThis whole process is usually done offline (i.e., not on the live sys\u2010\ntem), so online learning can be a confusing name. Think of it as\nincremental learning.\nFigure 1-14. Using online learning to handle huge datasets\nOne important parameter of online learning systems is how fast they should adapt to\nchanging data: this is called the learning rate. If you set a high learning rate, then your", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 96, "content": "changing data: this is called the learning rate. If you set a high learning rate, then your\nsystem will rapidly adapt to new data, but it will also tend to quickly forget the old\ndata (you don\u2019t want a spam filter to flag only the latest kinds of spam it was shown).\nConversely, if you set a low learning rate, the system will have more inertia; that is, it\nwill learn more slowly, but it will also be less sensitive to noise in the new data or to\nsequences of nonrepresentative data points.\nA big challenge with online learning is that if bad data is fed to the system, the sys\u2010\ntem\u2019s performance will gradually decline. If we are talking about a live system, your\nclients will notice. For example, bad data could come from a malfunctioning sensor\non a robot, or from someone spamming a search engine to try to rank high in search\n16 \n| \nChapter 1: The Machine Learning Landscape\nDownload from finelybook www.finelybook.com", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 97, "content": "results. To reduce this risk, you need to monitor your system closely and promptly\nswitch learning off (and possibly revert to a previously working state) if you detect a\ndrop in performance. You may also want to monitor the input data and react to\nabnormal data (e.g., using an anomaly detection algorithm).\nInstance-Based Versus Model-Based Learning\nOne more way to categorize Machine Learning systems is by how they generalize.\nMost Machine Learning tasks are about making predictions. This means that given a\nnumber of training examples, the system needs to be able to generalize to examples it\nhas never seen before. Having a good performance measure on the training data is\ngood, but insufficient; the true goal is to perform well on new instances.\nThere are two main approaches to generalization: instance-based learning and\nmodel-based learning.\nInstance-based learning\nPossibly the most trivial form of learning is simply to learn by heart. If you were to", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 98, "content": "Possibly the most trivial form of learning is simply to learn by heart. If you were to\ncreate a spam filter this way, it would just flag all emails that are identical to emails\nthat have already been flagged by users\u2014not the worst solution, but certainly not the\nbest.\nInstead of just flagging emails that are identical to known spam emails, your spam\nfilter could be programmed to also flag emails that are very similar to known spam\nemails. This requires a measure of similarity between two emails. A (very basic) simi\u2010\nlarity measure between two emails could be to count the number of words they have\nin common. The system would flag an email as spam if it has many words in com\u2010\nmon with a known spam email.\nThis is called instance-based learning: the system learns the examples by heart, then\ngeneralizes to new cases using a similarity measure (Figure 1-15).\nFigure 1-15. Instance-based learning\nTypes of Machine Learning Systems \n| \n17\nDownload from finelybook www.finelybook.com", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 99, "content": "Model-based learning\nAnother way to generalize from a set of examples is to build a model of these exam\u2010\nples, then use that model to make predictions. This is called model-based learning\n(Figure 1-16).\nFigure 1-16. Model-based learning\nFor example, suppose you want to know if money makes people happy, so you down\u2010\nload the Better Life Index data from the OECD\u2019s website as well as stats about GDP\nper capita from the IMF\u2019s website. Then you join the tables and sort by GDP per cap\u2010\nita. Table 1-1 shows an excerpt of what you get.\nTable 1-1. Does money make people happier?\nCountry\nGDP per capita (USD) Life satisfaction\nHungary\n12,240\n4.9\nKorea\n27,195\n5.8\nFrance\n37,675\n6.5\nAustralia\n50,962\n7.3\nUnited States\n55,805\n7.2\nLet\u2019s plot the data for a few random countries (Figure 1-17).\n18 \n| \nChapter 1: The Machine Learning Landscape\nDownload from finelybook www.finelybook.com", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 100, "content": "5 By convention, the Greek letter \u03b8 (theta) is frequently used to represent model parameters.\nFigure 1-17. Do you see a trend here?\nThere does seem to be a trend here! Although the data is noisy (i.e., partly random), it\nlooks like life satisfaction goes up more or less linearly as the country\u2019s GDP per cap\u2010\nita increases. So you decide to model life satisfaction as a linear function of GDP per\ncapita. This step is called model selection: you selected a linear model of life satisfac\u2010\ntion with just one attribute, GDP per capita (Equation 1-1).\nEquation 1-1. A simple linear model\nlif e_satisf action = \u03b80 + \u03b81 \u00d7 GDP_per_capita\nThis model has two model parameters, \u03b80 and \u03b81.5 By tweaking these parameters, you\ncan make your model represent any linear function, as shown in Figure 1-18.\nFigure 1-18. A few possible linear models\nTypes of Machine Learning Systems \n| \n19\nDownload from finelybook www.finelybook.com", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 101, "content": "6 The code assumes that prepare_country_stats() is already defined: it merges the GDP and life satisfaction\ndata into a single Pandas dataframe.\n7 It\u2019s okay if you don\u2019t understand all the code yet; we will present Scikit-Learn in the following chapters.\nBefore you can use your model, you need to define the parameter values \u03b80 and \u03b81.\nHow can you know which values will make your model perform best? To answer this\nquestion, you need to specify a performance measure. You can either define a utility\nfunction (or fitness function) that measures how good your model is, or you can define\na cost function that measures how bad it is. For linear regression problems, people\ntypically use a cost function that measures the distance between the linear model\u2019s\npredictions and the training examples; the objective is to minimize this distance.\nThis is where the Linear Regression algorithm comes in: you feed it your training", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 102, "content": "This is where the Linear Regression algorithm comes in: you feed it your training\nexamples and it finds the parameters that make the linear model fit best to your data.\nThis is called training the model. In our case the algorithm finds that the optimal\nparameter values are \u03b80 = 4.85 and \u03b81 = 4.91 \u00d7 10\u20135.\nNow the model fits the training data as closely as possible (for a linear model), as you\ncan see in Figure 1-19.\nFigure 1-19. The linear model that fits the training data best\nYou are finally ready to run the model to make predictions. For example, say you\nwant to know how happy Cypriots are, and the OECD data does not have the answer.\nFortunately, you can use your model to make a good prediction: you look up Cyprus\u2019s\nGDP per capita, find $22,587, and then apply your model and find that life satisfac\u2010\ntion is likely to be somewhere around 4.85 + 22,587 \u00d7 4.91 \u00d7 10-5 = 5.96.\nTo whet your appetite, Example 1-1 shows the Python code that loads the data, pre\u2010", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 103, "content": "To whet your appetite, Example 1-1 shows the Python code that loads the data, pre\u2010\npares it,6 creates a scatterplot for visualization, and then trains a linear model and\nmakes a prediction.7\n20 \n| \nChapter 1: The Machine Learning Landscape\nDownload from finelybook www.finelybook.com", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 104, "content": "Example 1-1. Training and running a linear model using Scikit-Learn\nimport matplotlib\nimport matplotlib.pyplot as plt\nimport numpy as np\nimport pandas as pd\nimport sklearn\n# Load the data\noecd_bli = pd.read_csv(\"oecd_bli_2015.csv\", thousands=',')\ngdp_per_capita = pd.read_csv(\"gdp_per_capita.csv\",thousands=',',delimiter='\\t',\n encoding='latin1', na_values=\"n/a\")\n# Prepare the data\ncountry_stats = prepare_country_stats(oecd_bli, gdp_per_capita)\nX = np.c_[country_stats[\"GDP per capita\"]]\ny = np.c_[country_stats[\"Life satisfaction\"]]\n# Visualize the data\ncountry_stats.plot(kind='scatter', x=\"GDP per capita\", y='Life satisfaction')\nplt.show()\n# Select a linear model\nlin_reg_model = sklearn.linear_model.LinearRegression()\n# Train the model\nlin_reg_model.fit(X, y)\n# Make a prediction for Cyprus\nX_new = [[22587]] # Cyprus' GDP per capita\nprint(lin_reg_model.predict(X_new)) # outputs [[ 5.96242338]]\nIf you had used an instance-based learning algorithm instead, you", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 105, "content": "If you had used an instance-based learning algorithm instead, you\nwould have found that Slovenia has the closest GDP per capita to\nthat of Cyprus ($20,732), and since the OECD data tells us that\nSlovenians\u2019 life satisfaction is 5.7, you would have predicted a life\nsatisfaction of 5.7 for Cyprus. If you zoom out a bit and look at the\ntwo next closest countries, you will find Portugal and Spain with\nlife satisfactions of 5.1 and 6.5, respectively. Averaging these three\nvalues, you get 5.77, which is pretty close to your model-based pre\u2010\ndiction. This simple algorithm is called k-Nearest Neighbors regres\u2010\nsion (in this example, k = 3).\nReplacing the Linear Regression model with k-Nearest Neighbors\nregression in the previous code is as simple as replacing this line:\nclf = sklearn.linear_model.LinearRegression()\nwith this one:\nclf = sklearn.neighbors.KNeighborsRegressor(n_neighbors=3)\nTypes of Machine Learning Systems \n| \n21\nDownload from finelybook www.finelybook.com", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 106, "content": "If all went well, your model will make good predictions. If not, you may need to use\nmore attributes (employment rate, health, air pollution, etc.), get more or better qual\u2010\nity training data, or perhaps select a more powerful model (e.g., a Polynomial Regres\u2010\nsion model).\nIn summary:\n\u2022 You studied the data.\n\u2022 You selected a model.\n\u2022 You trained it on the training data (i.e., the learning algorithm searched for the\nmodel parameter values that minimize a cost function).\n\u2022 Finally, you applied the model to make predictions on new cases (this is called\ninference), hoping that this model will generalize well.\nThis is what a typical Machine Learning project looks like. In Chapter 2 you will\nexperience this first-hand by going through an end-to-end project.\nWe have covered a lot of ground so far: you now know what Machine Learning is\nreally about, why it is useful, what some of the most common categories of ML sys\u2010", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 107, "content": "really about, why it is useful, what some of the most common categories of ML sys\u2010\ntems are, and what a typical project workflow looks like. Now let\u2019s look at what can go\nwrong in learning and prevent you from making accurate predictions.\nMain Challenges of Machine Learning\nIn short, since your main task is to select a learning algorithm and train it on some\ndata, the two things that can go wrong are \u201cbad algorithm\u201d and \u201cbad data.\u201d Let\u2019s start\nwith examples of bad data.\nInsufficient Quantity of Training Data\nFor a toddler to learn what an apple is, all it takes is for you to point to an apple and\nsay \u201capple\u201d (possibly repeating this procedure a few times). Now the child is able to\nrecognize apples in all sorts of colors and shapes. Genius.\nMachine Learning is not quite there yet; it takes a lot of data for most Machine Learn\u2010\ning algorithms to work properly. Even for very simple problems you typically need", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 108, "content": "ing algorithms to work properly. Even for very simple problems you typically need\nthousands of examples, and for complex problems such as image or speech recogni\u2010\ntion you may need millions of examples (unless you can reuse parts of an existing\nmodel).\n22 \n| \nChapter 1: The Machine Learning Landscape\nDownload from finelybook www.finelybook.com", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 109, "content": "8 For example, knowing whether to write \u201cto,\u201d \u201ctwo,\u201d or \u201ctoo\u201d depending on the context.\n9 Figure reproduced with permission from Banko and Brill (2001), \u201cLearning Curves for Confusion Set Disam\u2010\nbiguation.\u201d\n10 \u201cThe Unreasonable Effectiveness of Data,\u201d Peter Norvig et al. (2009).\nThe Unreasonable Effectiveness of Data\nIn a famous paper published in 2001, Microsoft researchers Michele Banko and Eric\nBrill showed that very different Machine Learning algorithms, including fairly simple\nones, performed almost identically well on a complex problem of natural language\ndisambiguation8 once they were given enough data (as you can see in Figure 1-20).\nFigure 1-20. The importance of data versus algorithms9\nAs the authors put it: \u201cthese results suggest that we may want to reconsider the trade-\noff between spending time and money on algorithm development versus spending it\non corpus development.\u201d\nThe idea that data matters more than algorithms for complex problems was further", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 110, "content": "The idea that data matters more than algorithms for complex problems was further\npopularized by Peter Norvig et al. in a paper titled \u201cThe Unreasonable Effectiveness\nof Data\u201d published in 2009.10 It should be noted, however, that small- and medium-\nsized datasets are still very common, and it is not always easy or cheap to get extra\ntraining data, so don\u2019t abandon algorithms just yet.\nMain Challenges of Machine Learning \n| \n23\nDownload from finelybook www.finelybook.com", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 111, "content": "Nonrepresentative Training Data\nIn order to generalize well, it is crucial that your training data be representative of the\nnew cases you want to generalize to. This is true whether you use instance-based\nlearning or model-based learning.\nFor example, the set of countries we used earlier for training the linear model was not\nperfectly representative; a few countries were missing. Figure 1-21 shows what the\ndata looks like when you add the missing countries.\nFigure 1-21. A more representative training sample\nIf you train a linear model on this data, you get the solid line, while the old model is\nrepresented by the dotted line. As you can see, not only does adding a few missing\ncountries significantly alter the model, but it makes it clear that such a simple linear\nmodel is probably never going to work well. It seems that very rich countries are not\nhappier than moderately rich countries (in fact they seem unhappier), and conversely", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 112, "content": "happier than moderately rich countries (in fact they seem unhappier), and conversely\nsome poor countries seem happier than many rich countries.\nBy using a nonrepresentative training set, we trained a model that is unlikely to make\naccurate predictions, especially for very poor and very rich countries.\nIt is crucial to use a training set that is representative of the cases you want to general\u2010\nize to. This is often harder than it sounds: if the sample is too small, you will have\nsampling noise (i.e., nonrepresentative data as a result of chance), but even very large\nsamples can be nonrepresentative if the sampling method is flawed. This is called\nsampling bias.\nA Famous Example of Sampling Bias\nPerhaps the most famous example of sampling bias happened during the US presi\u2010\ndential election in 1936, which pitted Landon against Roosevelt: the Literary Digest\nconducted a very large poll, sending mail to about 10 million people. It got 2.4 million", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 113, "content": "conducted a very large poll, sending mail to about 10 million people. It got 2.4 million\nanswers, and predicted with high confidence that Landon would get 57% of the votes.\n24 \n| \nChapter 1: The Machine Learning Landscape\nDownload from finelybook www.finelybook.com", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 114, "content": "Instead, Roosevelt won with 62% of the votes. The flaw was in the Literary Digest\u2019s\nsampling method:\n\u2022 First, to obtain the addresses to send the polls to, the Literary Digest used tele\u2010\nphone directories, lists of magazine subscribers, club membership lists, and the\nlike. All of these lists tend to favor wealthier people, who are more likely to vote\nRepublican (hence Landon).\n\u2022 Second, less than 25% of the people who received the poll answered. Again, this\nintroduces a sampling bias, by ruling out people who don\u2019t care much about poli\u2010\ntics, people who don\u2019t like the Literary Digest, and other key groups. This is a spe\u2010\ncial type of sampling bias called nonresponse bias.\nHere is another example: say you want to build a system to recognize funk music vid\u2010\neos. One way to build your training set is to search \u201cfunk music\u201d on YouTube and use\nthe resulting videos. But this assumes that YouTube\u2019s search engine returns a set of", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 115, "content": "the resulting videos. But this assumes that YouTube\u2019s search engine returns a set of\nvideos that are representative of all the funk music videos on YouTube. In reality, the\nsearch results are likely to be biased toward popular artists (and if you live in Brazil\nyou will get a lot of \u201cfunk carioca\u201d videos, which sound nothing like James Brown).\nOn the other hand, how else can you get a large training set?\nPoor-Quality Data\nObviously, if your training data is full of errors, outliers, and noise (e.g., due to poor-\nquality measurements), it will make it harder for the system to detect the underlying\npatterns, so your system is less likely to perform well. It is often well worth the effort\nto spend time cleaning up your training data. The truth is, most data scientists spend\na significant part of their time doing just that. For example:\n\u2022 If some instances are clearly outliers, it may help to simply discard them or try to\nfix the errors manually.", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 116, "content": "fix the errors manually.\n\u2022 If some instances are missing a few features (e.g., 5% of your customers did not\nspecify their age), you must decide whether you want to ignore this attribute alto\u2010\ngether, ignore these instances, fill in the missing values (e.g., with the median\nage), or train one model with the feature and one model without it, and so on.\nIrrelevant Features\nAs the saying goes: garbage in, garbage out. Your system will only be capable of learn\u2010\ning if the training data contains enough relevant features and not too many irrelevant\nones. A critical part of the success of a Machine Learning project is coming up with a\ngood set of features to train on. This process, called feature engineering, involves:\nMain Challenges of Machine Learning \n| \n25\nDownload from finelybook www.finelybook.com", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 117, "content": "\u2022 Feature selection: selecting the most useful features to train on among existing\nfeatures.\n\u2022 Feature extraction: combining existing features to produce a more useful one (as\nwe saw earlier, dimensionality reduction algorithms can help).\n\u2022 Creating new features by gathering new data.\nNow that we have looked at many examples of bad data, let\u2019s look at a couple of exam\u2010\nples of bad algorithms.\nOverfitting the Training Data\nSay you are visiting a foreign country and the taxi driver rips you off. You might be\ntempted to say that all taxi drivers in that country are thieves. Overgeneralizing is\nsomething that we humans do all too often, and unfortunately machines can fall into\nthe same trap if we are not careful. In Machine Learning this is called overfitting: it\nmeans that the model performs well on the training data, but it does not generalize\nwell.\nFigure 1-22 shows an example of a high-degree polynomial life satisfaction model", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 118, "content": "well.\nFigure 1-22 shows an example of a high-degree polynomial life satisfaction model\nthat strongly overfits the training data. Even though it performs much better on the\ntraining data than the simple linear model, would you really trust its predictions?\nFigure 1-22. Overfitting the training data\nComplex models such as deep neural networks can detect subtle patterns in the data,\nbut if the training set is noisy, or if it is too small (which introduces sampling noise),\nthen the model is likely to detect patterns in the noise itself. Obviously these patterns\nwill not generalize to new instances. For example, say you feed your life satisfaction\nmodel many more attributes, including uninformative ones such as the country\u2019s\nname. In that case, a complex model may detect patterns like the fact that all coun\u2010\ntries in the training data with a w in their name have a life satisfaction greater than 7:\nNew Zealand (7.3), Norway (7.4), Sweden (7.2), and Switzerland (7.5). How confident\n26 \n|", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 119, "content": "New Zealand (7.3), Norway (7.4), Sweden (7.2), and Switzerland (7.5). How confident\n26 \n| \nChapter 1: The Machine Learning Landscape\nDownload from finelybook www.finelybook.com", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 120, "content": "are you that the W-satisfaction rule generalizes to Rwanda or Zimbabwe? Obviously\nthis pattern occurred in the training data by pure chance, but the model has no way\nto tell whether a pattern is real or simply the result of noise in the data.\nOverfitting happens when the model is too complex relative to the\namount and noisiness of the training data. The possible solutions\nare:\n\u2022 To simplify the model by selecting one with fewer parameters\n(e.g., a linear model rather than a high-degree polynomial\nmodel), by reducing the number of attributes in the training\ndata or by constraining the model\n\u2022 To gather more training data\n\u2022 To reduce the noise in the training data (e.g., fix data errors\nand remove outliers)\nConstraining a model to make it simpler and reduce the risk of overfitting is called\nregularization. For example, the linear model we defined earlier has two parameters,\n\u03b80 and \u03b81. This gives the learning algorithm two degrees of freedom to adapt the model", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 121, "content": "\u03b80 and \u03b81. This gives the learning algorithm two degrees of freedom to adapt the model\nto the training data: it can tweak both the height (\u03b80) and the slope (\u03b81) of the line. If\nwe forced \u03b81 = 0, the algorithm would have only one degree of freedom and would\nhave a much harder time fitting the data properly: all it could do is move the line up\nor down to get as close as possible to the training instances, so it would end up\naround the mean. A very simple model indeed! If we allow the algorithm to modify \u03b81\nbut we force it to keep it small, then the learning algorithm will effectively have some\u2010\nwhere in between one and two degrees of freedom. It will produce a simpler model\nthan with two degrees of freedom, but more complex than with just one. You want to\nfind the right balance between fitting the data perfectly and keeping the model simple\nenough to ensure that it will generalize well.\nFigure 1-23 shows three models: the dotted line represents the original model that", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 122, "content": "Figure 1-23 shows three models: the dotted line represents the original model that\nwas trained with a few countries missing, the dashed line is our second model trained\nwith all countries, and the solid line is a linear model trained with the same data as\nthe first model but with a regularization constraint. You can see that regularization\nforced the model to have a smaller slope, which fits a bit less the training data that the\nmodel was trained on, but actually allows it to generalize better to new examples.\nMain Challenges of Machine Learning \n| \n27\nDownload from finelybook www.finelybook.com", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 123, "content": "Figure 1-23. Regularization reduces the risk of overfitting\nThe amount of regularization to apply during learning can be controlled by a hyper\u2010\nparameter. A hyperparameter is a parameter of a learning algorithm (not of the\nmodel). As such, it is not affected by the learning algorithm itself; it must be set prior\nto training and remains constant during training. If you set the regularization hyper\u2010\nparameter to a very large value, you will get an almost flat model (a slope close to\nzero); the learning algorithm will almost certainly not overfit the training data, but it\nwill be less likely to find a good solution. Tuning hyperparameters is an important\npart of building a Machine Learning system (you will see a detailed example in the\nnext chapter).\nUnderfitting the Training Data\nAs you might guess, underfitting is the opposite of overfitting: it occurs when your\nmodel is too simple to learn the underlying structure of the data. For example, a lin\u2010", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 124, "content": "model is too simple to learn the underlying structure of the data. For example, a lin\u2010\near model of life satisfaction is prone to underfit; reality is just more complex than\nthe model, so its predictions are bound to be inaccurate, even on the training exam\u2010\nples.\nThe main options to fix this problem are:\n\u2022 Selecting a more powerful model, with more parameters\n\u2022 Feeding better features to the learning algorithm (feature engineering)\n\u2022 Reducing the constraints on the model (e.g., reducing the regularization hyper\u2010\nparameter)\nStepping Back\nBy now you already know a lot about Machine Learning. However, we went through\nso many concepts that you may be feeling a little lost, so let\u2019s step back and look at the\nbig picture:\n28 \n| \nChapter 1: The Machine Learning Landscape\nDownload from finelybook www.finelybook.com", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 125, "content": "\u2022 Machine Learning is about making machines get better at some task by learning\nfrom data, instead of having to explicitly code rules.\n\u2022 There are many different types of ML systems: supervised or not, batch or online,\ninstance-based or model-based, and so on.\n\u2022 In a ML project you gather data in a training set, and you feed the training set to\na learning algorithm. If the algorithm is model-based it tunes some parameters to\nfit the model to the training set (i.e., to make good predictions on the training set\nitself), and then hopefully it will be able to make good predictions on new cases\nas well. If the algorithm is instance-based, it just learns the examples by heart and\nuses a similarity measure to generalize to new instances.\n\u2022 The system will not perform well if your training set is too small, or if the data is\nnot representative, noisy, or polluted with irrelevant features (garbage in, garbage\nout). Lastly, your model needs to be neither too simple (in which case it will", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 126, "content": "out). Lastly, your model needs to be neither too simple (in which case it will\nunderfit) nor too complex (in which case it will overfit).\nThere\u2019s just one last important topic to cover: once you have trained a model, you\ndon\u2019t want to just \u201chope\u201d it generalizes to new cases. You want to evaluate it, and fine-\ntune it if necessary. Let\u2019s see how.\nTesting and Validating\nThe only way to know how well a model will generalize to new cases is to actually try\nit out on new cases. One way to do that is to put your model in production and moni\u2010\ntor how well it performs. This works well, but if your model is horribly bad, your\nusers will complain\u2014not the best idea.\nA better option is to split your data into two sets: the training set and the test set. As\nthese names imply, you train your model using the training set, and you test it using\nthe test set. The error rate on new cases is called the generalization error (or out-of-", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 127, "content": "the test set. The error rate on new cases is called the generalization error (or out-of-\nsample error), and by evaluating your model on the test set, you get an estimation of\nthis error. This value tells you how well your model will perform on instances it has\nnever seen before.\nIf the training error is low (i.e., your model makes few mistakes on the training set)\nbut the generalization error is high, it means that your model is overfitting the train\u2010\ning data.\nIt is common to use 80% of the data for training and hold out 20%\nfor testing.\nTesting and Validating \n| \n29\nDownload from finelybook www.finelybook.com", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 128, "content": "11 \u201cThe Lack of A Priori Distinctions Between Learning Algorithms,\u201d D. Wolperts (1996).\nSo evaluating a model is simple enough: just use a test set. Now suppose you are hesi\u2010\ntating between two models (say a linear model and a polynomial model): how can\nyou decide? One option is to train both and compare how well they generalize using\nthe test set.\nNow suppose that the linear model generalizes better, but you want to apply some \nregularization to avoid overfitting. The question is: how do you choose the value of\nthe regularization hyperparameter? One option is to train 100 different models using\n100 different values for this hyperparameter. Suppose you find the best hyperparame\u2010\nter value that produces a model with the lowest generalization error, say just 5% error.\nSo you launch this model into production, but unfortunately it does not perform as\nwell as expected and produces 15% errors. What just happened?", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 129, "content": "well as expected and produces 15% errors. What just happened?\nThe problem is that you measured the generalization error multiple times on the test\nset, and you adapted the model and hyperparameters to produce the best model for\nthat set. This means that the model is unlikely to perform as well on new data.\nA common solution to this problem is to have a second holdout set called the valida\u2010\ntion set. You train multiple models with various hyperparameters using the training\nset, you select the model and hyperparameters that perform best on the validation set,\nand when you\u2019re happy with your model you run a single final test against the test set\nto get an estimate of the generalization error.\nTo avoid \u201cwasting\u201d too much training data in validation sets, a common technique is\nto use cross-validation: the training set is split into complementary subsets, and each\nmodel is trained against a different combination of these subsets and validated", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 130, "content": "model is trained against a different combination of these subsets and validated\nagainst the remaining parts. Once the model type and hyperparameters have been\nselected, a final model is trained using these hyperparameters on the full training set,\nand the generalized error is measured on the test set.\nNo Free Lunch Theorem\nA model is a simplified version of the observations. The simplifications are meant to\ndiscard the superfluous details that are unlikely to generalize to new instances. How\u2010\never, to decide what data to discard and what data to keep, you must make assump\u2010\ntions. For example, a linear model makes the assumption that the data is\nfundamentally linear and that the distance between the instances and the straight line\nis just noise, which can safely be ignored.\nIn a famous 1996 paper,11 David Wolpert demonstrated that if you make absolutely\nno assumption about the data, then there is no reason to prefer one model over any", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 131, "content": "no assumption about the data, then there is no reason to prefer one model over any\nother. This is called the No Free Lunch (NFL) theorem. For some datasets the best\n30 \n| \nChapter 1: The Machine Learning Landscape\nDownload from finelybook www.finelybook.com", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 132, "content": "model is a linear model, while for other datasets it is a neural network. There is no\nmodel that is a priori guaranteed to work better (hence the name of the theorem). The\nonly way to know for sure which model is best is to evaluate them all. Since this is not\npossible, in practice you make some reasonable assumptions about the data and you\nevaluate only a few reasonable models. For example, for simple tasks you may evalu\u2010\nate linear models with various levels of regularization, and for a complex problem you\nmay evaluate various neural networks.\nExercises\nIn this chapter we have covered some of the most important concepts in Machine\nLearning. In the next chapters we will dive deeper and write more code, but before we\ndo, make sure you know how to answer the following questions:\n1. How would you define Machine Learning?\n2. Can you name four types of problems where it shines?\n3. What is a labeled training set?\n4. What are the two most common supervised tasks?", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 133, "content": "3. What is a labeled training set?\n4. What are the two most common supervised tasks?\n5. Can you name four common unsupervised tasks?\n6. What type of Machine Learning algorithm would you use to allow a robot to\nwalk in various unknown terrains?\n7. What type of algorithm would you use to segment your customers into multiple\ngroups?\n8. Would you frame the problem of spam detection as a supervised learning prob\u2010\nlem or an unsupervised learning problem?\n9. What is an online learning system?\n10. What is out-of-core learning?\n11. What type of learning algorithm relies on a similarity measure to make predic\u2010\ntions?\n12. What is the difference between a model parameter and a learning algorithm\u2019s\nhyperparameter?\n13. What do model-based learning algorithms search for? What is the most common\nstrategy they use to succeed? How do they make predictions?\n14. Can you name four of the main challenges in Machine Learning?", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 134, "content": "14. Can you name four of the main challenges in Machine Learning?\n15. If your model performs great on the training data but generalizes poorly to new\ninstances, what is happening? Can you name three possible solutions?\n16. What is a test set and why would you want to use it?\nExercises \n| \n31\nDownload from finelybook www.finelybook.com", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 135, "content": "17. What is the purpose of a validation set?\n18. What can go wrong if you tune hyperparameters using the test set?\n19. What is cross-validation and why would you prefer it to a validation set?\nSolutions to these exercises are available in Appendix A.\n32 \n| \nChapter 1: The Machine Learning Landscape\nDownload from finelybook www.finelybook.com", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 136, "content": "1 The example project is completely fictitious; the goal is just to illustrate the main steps of a Machine Learning\nproject, not to learn anything about the real estate business.\nCHAPTER 2\nEnd-to-End Machine Learning Project\nIn this chapter, you will go through an example project end to end, pretending to be a\nrecently hired data scientist in a real estate company.1 Here are the main steps you will\ngo through:\n1. Look at the big picture.\n2. Get the data.\n3. Discover and visualize the data to gain insights.\n4. Prepare the data for Machine Learning algorithms.\n5. Select a model and train it.\n6. Fine-tune your model.\n7. Present your solution.\n8. Launch, monitor, and maintain your system.\nWorking with Real Data\nWhen you are learning about Machine Learning it is best to actually experiment with\nreal-world data, not just artificial datasets. Fortunately, there are thousands of open\ndatasets to choose from, ranging across all sorts of domains. Here are a few places\nyou can look to get data:", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 137, "content": "you can look to get data:\n\u2022 Popular open data repositories:\n33\nDownload from finelybook www.finelybook.com", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 138, "content": "2 The original dataset appeared in R. Kelley Pace and Ronald Barry, \u201cSparse Spatial Autoregressions,\u201d Statistics\n& Probability Letters 33, no. 3 (1997): 291\u2013297.\n\u2014 UC Irvine Machine Learning Repository\n\u2014 Kaggle datasets\n\u2014 Amazon\u2019s AWS datasets\n\u2022 Meta portals (they list open data repositories):\n\u2014 http://dataportals.org/\n\u2014 http://opendatamonitor.eu/\n\u2014 http://quandl.com/\n\u2022 Other pages listing many popular open data repositories:\n\u2014 Wikipedia\u2019s list of Machine Learning datasets\n\u2014 Quora.com question\n\u2014 Datasets subreddit\nIn this chapter we chose the California Housing Prices dataset from the StatLib repos\u2010\nitory2 (see Figure 2-1). This dataset was based on data from the 1990 California cen\u2010\nsus. It is not exactly recent (you could still afford a nice house in the Bay Area at the\ntime), but it has many qualities for learning, so we will pretend it is recent data. We\nalso added a categorical attribute and removed a few features for teaching purposes.\nFigure 2-1. California housing prices\n34", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 139, "content": "Figure 2-1. California housing prices\n34 \n| \nChapter 2: End-to-End Machine Learning Project\nDownload from finelybook www.finelybook.com", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 140, "content": "3 A piece of information fed to a Machine Learning system is often called a signal in reference to Shannon\u2019s\ninformation theory: you want a high signal/noise ratio.\nLook at the Big Picture\nWelcome to Machine Learning Housing Corporation! The first task you are asked to\nperform is to build a model of housing prices in California using the California cen\u2010\nsus data. This data has metrics such as the population, median income, median hous\u2010\ning price, and so on for each block group in California. Block groups are the smallest\ngeographical unit for which the US Census Bureau publishes sample data (a block\ngroup typically has a population of 600 to 3,000 people). We will just call them \u201cdis\u2010\ntricts\u201d for short.\nYour model should learn from this data and be able to predict the median housing\nprice in any district, given all the other metrics.\nSince you are a well-organized data scientist, the first thing you do\nis to pull out your Machine Learning project checklist. You can", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 141, "content": "is to pull out your Machine Learning project checklist. You can\nstart with the one in Appendix B; it should work reasonably well\nfor most Machine Learning projects but make sure to adapt it to\nyour needs. In this chapter we will go through many checklist\nitems, but we will also skip a few, either because they are self-\nexplanatory or because they will be discussed in later chapters.\nFrame the Problem\nThe first question to ask your boss is what exactly is the business objective; building a\nmodel is probably not the end goal. How does the company expect to use and benefit\nfrom this model? This is important because it will determine how you frame the\nproblem, what algorithms you will select, what performance measure you will use to\nevaluate your model, and how much effort you should spend tweaking it.\nYour boss answers that your model\u2019s output (a prediction of a district\u2019s median hous\u2010\ning price) will be fed to another Machine Learning system (see Figure 2-2), along", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 142, "content": "ing price) will be fed to another Machine Learning system (see Figure 2-2), along\nwith many other signals.3 This downstream system will determine whether it is worth\ninvesting in a given area or not. Getting this right is critical, as it directly affects reve\u2010\nnue.\nLook at the Big Picture \n| \n35\nDownload from finelybook www.finelybook.com", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 143, "content": "Figure 2-2. A Machine Learning pipeline for real estate investments\nPipelines\nA sequence of data processing components is called a data pipeline. Pipelines are very\ncommon in Machine Learning systems, since there is a lot of data to manipulate and\nmany data transformations to apply.\nComponents typically run asynchronously. Each component pulls in a large amount\nof data, processes it, and spits out the result in another data store, and then some time\nlater the next component in the pipeline pulls this data and spits out its own output,\nand so on. Each component is fairly self-contained: the interface between components\nis simply the data store. This makes the system quite simple to grasp (with the help of\na data flow graph), and different teams can focus on different components. Moreover,\nif a component breaks down, the downstream components can often continue to run\nnormally (at least for a while) by just using the last output from the broken compo\u2010", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 144, "content": "normally (at least for a while) by just using the last output from the broken compo\u2010\nnent. This makes the architecture quite robust.\nOn the other hand, a broken component can go unnoticed for some time if proper\nmonitoring is not implemented. The data gets stale and the overall system\u2019s perfor\u2010\nmance drops.\nThe next question to ask is what the current solution looks like (if any). It will often\ngive you a reference performance, as well as insights on how to solve the problem.\nYour boss answers that the district housing prices are currently estimated manually\nby experts: a team gathers up-to-date information about a district (excluding median\nhousing prices), and they use complex rules to come up with an estimate. This is\ncostly and time-consuming, and their estimates are not great; their typical error rate\nis about 15%.\nOkay, with all this information you are now ready to start designing your system.\nFirst, you need to frame the problem: is it supervised, unsupervised, or Reinforce\u2010", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 145, "content": "First, you need to frame the problem: is it supervised, unsupervised, or Reinforce\u2010\nment Learning? Is it a classification task, a regression task, or something else? Should\n36 \n| \nChapter 2: End-to-End Machine Learning Project\nDownload from finelybook www.finelybook.com", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 146, "content": "4 The standard deviation, generally denoted \u03c3 (the Greek letter sigma), is the square root of the variance, which\nis the average of the squared deviation from the mean.\n5 When a feature has a bell-shaped normal distribution (also called a Gaussian distribution), which is very com\u2010\nmon, the \u201c68-95-99.7\u201d rule applies: about 68% of the values fall within 1\u03c3 of the mean, 95% within 2\u03c3, and\n99.7% within 3\u03c3.\nyou use batch learning or online learning techniques? Before you read on, pause and\ntry to answer these questions for yourself.\nHave you found the answers? Let\u2019s see: it is clearly a typical supervised learning task\nsince you are given labeled training examples (each instance comes with the expected\noutput, i.e., the district\u2019s median housing price). Moreover, it is also a typical regres\u2010\nsion task, since you are asked to predict a value. More specifically, this is a multivari\u2010\nate regression problem since the system will use multiple features to make a prediction", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 147, "content": "ate regression problem since the system will use multiple features to make a prediction\n(it will use the district\u2019s population, the median income, etc.). In the first chapter, you\npredicted life satisfaction based on just one feature, the GDP per capita, so it was a\nunivariate regression problem. Finally, there is no continuous flow of data coming in\nthe system, there is no particular need to adjust to changing data rapidly, and the data\nis small enough to fit in memory, so plain batch learning should do just fine.\nIf the data was huge, you could either split your batch learning\nwork across multiple servers (using the MapReduce technique, as\nwe will see later), or you could use an online learning technique\ninstead.\nSelect a Performance Measure\nYour next step is to select a performance measure. A typical performance measure for\nregression problems is the Root Mean Square Error (RMSE). It measures the standard", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 148, "content": "regression problems is the Root Mean Square Error (RMSE). It measures the standard\ndeviation4 of the errors the system makes in its predictions. For example, an RMSE\nequal to 50,000 means that about 68% of the system\u2019s predictions fall within $50,000\nof the actual value, and about 95% of the predictions fall within $100,000 of the actual\nvalue.5 Equation 2-1 shows the mathematical formula to compute the RMSE.\nEquation 2-1. Root Mean Square Error (RMSE)\nRMSE \ufffd, h =\n1\nm \u2211\ni = 1\nm\nh \ufffdi\n\u2212y i 2\nLook at the Big Picture \n| \n37\nDownload from finelybook www.finelybook.com", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 149, "content": "6 Recall that the transpose operator flips a column vector into a row vector (and vice versa).\nNotations\nThis equation introduces several very common Machine Learning notations that we\nwill use throughout this book:\n\u2022 m is the number of instances in the dataset you are measuring the RMSE on.\n\u2014 For example, if you are evaluating the RMSE on a validation set of 2,000 dis\u2010\ntricts, then m = 2,000.\n\u2022 x(i) is a vector of all the feature values (excluding the label) of the ith instance in\nthe dataset, and y(i) is its label (the desired output value for that instance).\n\u2014 For example, if the first district in the dataset is located at longitude \u2013118.29\u00b0,\nlatitude 33.91\u00b0, and it has 1,416 inhabitants with a median income of $38,372,\nand the median house value is $156,400 (ignoring the other features for now),\nthen:\n\ufffd1 =\n\u2212118 . 29\n33 . 91\n1, 416\n38, 372\nand:\ny 1 = 156, 400\n\u2022 X is a matrix containing all the feature values (excluding labels) of all instances in", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 150, "content": "\u2022 X is a matrix containing all the feature values (excluding labels) of all instances in\nthe dataset. There is one row per instance and the ith row is equal to the transpose\nof x(i), noted (x(i))T.6\n\u2014 For example, if the first district is as just described, then the matrix X looks\nlike this:\n\ufffd=\n\ufffd1 T\n\ufffd2 T\n\u22ee\n\ufffd1999 T\n\ufffd2000 T\n= \u2212118 . 29 33 . 91 1, 416 38, 372\n\u22ee\n\u22ee\n\u22ee\n\u22ee\n38 \n| \nChapter 2: End-to-End Machine Learning Project\nDownload from finelybook www.finelybook.com", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 151, "content": "\u2022 h is your system\u2019s prediction function, also called a hypothesis. When your system\nis given an instance\u2019s feature vector x(i), it outputs a predicted value \u0177(i) = h(x(i))\nfor that instance (\u0177 is pronounced \u201cy-hat\u201d).\n\u2014 For example, if your system predicts that the median housing price in the first\ndistrict is $158,400, then \u0177(1) = h(x(1)) = 158,400. The prediction error for this\ndistrict is \u0177(1) \u2013 y(1) = 2,000.\n\u2022 RMSE(X,h) is the cost function measured on the set of examples using your\nhypothesis h.\nWe use lowercase italic font for scalar values (such as m or y(i)) and function names\n(such as h), lowercase bold font for vectors (such as x(i)), and uppercase bold font for\nmatrices (such as X).\nEven though the RMSE is generally the preferred performance measure for regression\ntasks, in some contexts you may prefer to use another function. For example, suppose\nthat there are many outlier districts. In that case, you may consider using the Mean", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 152, "content": "that there are many outlier districts. In that case, you may consider using the Mean\nAbsolute Error (also called the Average Absolute Deviation; see Equation 2-2):\nEquation 2-2. Mean Absolute Error\nMAE \ufffd, h = 1\nm \u2211\ni = 1\nm\nh \ufffdi\n\u2212y i\nBoth the RMSE and the MAE are ways to measure the distance between two vectors:\nthe vector of predictions and the vector of target values. Various distance measures,\nor norms, are possible:\n\u2022 Computing the root of a sum of squares (RMSE) corresponds to the Euclidian\nnorm: it is the notion of distance you are familiar with. It is also called the \u21132\nnorm, noted \u2225 \u00b7 \u22252 (or just \u2225 \u00b7 \u2225).\n\u2022 Computing the sum of absolutes (MAE) corresponds to the \u21131 norm, noted \u2225 \u00b7 \u22251.\nIt is sometimes called the Manhattan norm because it measures the distance\nbetween two points in a city if you can only travel along orthogonal city blocks.\n\u2022 More generally, the \u2113k norm of a vector v containing n elements is defined as\n\u2225\ufffd\u2225k =\nv0\nk + v1\nk + \u22ef+ vn\nk\n1", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 153, "content": "\u2225\ufffd\u2225k =\nv0\nk + v1\nk + \u22ef+ vn\nk\n1\nk. \u21130 just gives the cardinality of the vector (i.e.,\nthe number of elements), and \u2113\u221e gives the maximum absolute value in the vector.\n\u2022 The higher the norm index, the more it focuses on large values and neglects small\nones. This is why the RMSE is more sensitive to outliers than the MAE. But when\nLook at the Big Picture \n| \n39\nDownload from finelybook www.finelybook.com", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 154, "content": "7 The latest version of Python 3 is recommended. Python 2.7+ should work fine too, but it is deprecated.\noutliers are exponentially rare (like in a bell-shaped curve), the RMSE performs\nvery well and is generally preferred.\nCheck the Assumptions\nLastly, it is good practice to list and verify the assumptions that were made so far (by\nyou or others); this can catch serious issues early on. For example, the district prices\nthat your system outputs are going to be fed into a downstream Machine Learning\nsystem, and we assume that these prices are going to be used as such. But what if the\ndownstream system actually converts the prices into categories (e.g., \u201ccheap,\u201d\n\u201cmedium,\u201d or \u201cexpensive\u201d) and then uses those categories instead of the prices them\u2010\nselves? In this case, getting the price perfectly right is not important at all; your sys\u2010\ntem just needs to get the category right. If that\u2019s so, then the problem should have", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 155, "content": "tem just needs to get the category right. If that\u2019s so, then the problem should have\nbeen framed as a classification task, not a regression task. You don\u2019t want to find this\nout after working on a regression system for months.\nFortunately, after talking with the team in charge of the downstream system, you are\nconfident that they do indeed need the actual prices, not just categories. Great! You\u2019re\nall set, the lights are green, and you can start coding now!\nGet the Data\nIt\u2019s time to get your hands dirty. Don\u2019t hesitate to pick up your laptop and walk\nthrough the following code examples in a Jupyter notebook. The full Jupyter note\u2010\nbook is available at https://github.com/ageron/handson-ml.\nCreate the Workspace\nFirst you will need to have Python installed. It is probably already installed on your\nsystem. If not, you can get it at https://www.python.org/.7\nNext you need to create a workspace directory for your Machine Learning code and", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 156, "content": "Next you need to create a workspace directory for your Machine Learning code and\ndatasets. Open a terminal and type the following commands (after the $ prompts):\n$ export ML_PATH=\"$HOME/ml\" # You can change the path if you prefer\n$ mkdir -p $ML_PATH\nYou will need a number of Python modules: Jupyter, NumPy, Pandas, Matplotlib, and\nScikit-Learn. If you already have Jupyter running with all these modules installed,\nyou can safely skip to \u201cDownload the Data\u201d on page 43. If you don\u2019t have them yet,\nthere are many ways to install them (and their dependencies). You can use your sys\u2010\ntem\u2019s packaging system (e.g., apt-get on Ubuntu, or MacPorts or HomeBrew on\n40 \n| \nChapter 2: End-to-End Machine Learning Project\nDownload from finelybook www.finelybook.com", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 157, "content": "8 We will show the installation steps using pip in a bash shell on a Linux or macOS system. You may need to\nadapt these commands to your own system. On Windows, we recommend installing Anaconda instead.\n9 You may need to have administrator rights to run this command; if so, try prefixing it with sudo.\nmacOS), install a Scientific Python distribution such as Anaconda and use its packag\u2010\ning system, or just use Python\u2019s own packaging system, pip, which is included by\ndefault with the Python binary installers (since Python 2.7.9).8 You can check to see if\npip is installed by typing the following command:\n$ pip3 --version\npip 9.0.1 from [...]/lib/python3.5/site-packages (python 3.5)\nYou should make sure you have a recent version of pip installed, at the very least >1.4\nto support binary module installation (a.k.a. wheels). To upgrade the pip module,\ntype:9\n$ pip3 install --upgrade pip\nCollecting pip\n[...]\nSuccessfully installed pip-9.0.1\nCreating an Isolated Environment", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 158, "content": "Collecting pip\n[...]\nSuccessfully installed pip-9.0.1\nCreating an Isolated Environment\nIf you would like to work in an isolated environment (which is strongly recom\u2010\nmended so you can work on different projects without having conflicting library ver\u2010\nsions), install virtualenv by running the following pip command:\n$ pip3 install --user --upgrade virtualenv\nCollecting virtualenv\n[...]\nSuccessfully installed virtualenv\nNow you can create an isolated Python environment by typing:\n$ cd $ML_PATH\n$ virtualenv env\nUsing base prefix '[...]'\nNew python executable in [...]/ml/env/bin/python3.5\nAlso creating executable in [...]/ml/env/bin/python\nInstalling setuptools, pip, wheel...done.\nNow every time you want to activate this environment, just open a terminal and type:\n$ cd $ML_PATH\n$ source env/bin/activate\nWhile the environment is active, any package you install using pip will be installed in\nthis isolated environment, and Python will only have access to these packages (if you", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 159, "content": "this isolated environment, and Python will only have access to these packages (if you\nalso want access to the system\u2019s site packages, you should create the environment\nGet the Data \n| \n41\nDownload from finelybook www.finelybook.com", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 160, "content": "10 Note that Jupyter can handle multiple versions of Python, and even many other languages such as R or\nOctave.\nusing virtualenv\u2019s --system-site-packages option). Check out virtualenv\u2019s docu\u2010\nmentation for more information.\nNow you can install all the required modules and their dependencies using this sim\u2010\nple pip command:\n$ pip3 install --upgrade jupyter matplotlib numpy pandas scipy scikit-learn\nCollecting jupyter\n Downloading jupyter-1.0.0-py2.py3-none-any.whl\nCollecting matplotlib\n [...]\nTo check your installation, try to import every module like this:\n$ python3 -c \"import jupyter, matplotlib, numpy, pandas, scipy, sklearn\"\nThere should be no output and no error. Now you can fire up Jupyter by typing:\n$ jupyter notebook\n[I 15:24 NotebookApp] Serving notebooks from local directory: [...]/ml\n[I 15:24 NotebookApp] 0 active kernels\n[I 15:24 NotebookApp] The Jupyter Notebook is running at: http://localhost:8888/", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 161, "content": "[I 15:24 NotebookApp] The Jupyter Notebook is running at: http://localhost:8888/\n[I 15:24 NotebookApp] Use Control-C to stop this server and shut down all\nkernels (twice to skip confirmation).\nA Jupyter server is now running in your terminal, listening to port 8888. You can visit\nthis server by opening your web browser to http://localhost:8888/ (this usually hap\u2010\npens automatically when the server starts). You should see your empty workspace\ndirectory (containing only the env directory if you followed the preceding virtualenv\ninstructions).\nNow create a new Python notebook by clicking on the New button and selecting the\nappropriate Python version10 (see Figure 2-3).\nThis does three things: first, it creates a new notebook file called Untitled.ipynb in\nyour workspace; second, it starts a Jupyter Python kernel to run this notebook; and\nthird, it opens this notebook in a new tab. You should start by renaming this note\u2010", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 162, "content": "third, it opens this notebook in a new tab. You should start by renaming this note\u2010\nbook to \u201cHousing\u201d (this will automatically rename the file to Housing.ipynb) by click\u2010\ning Untitled and typing the new name.\n42 \n| \nChapter 2: End-to-End Machine Learning Project\nDownload from finelybook www.finelybook.com", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 163, "content": "Figure 2-3. Your workspace in Jupyter\nA notebook contains a list of cells. Each cell can contain executable code or formatted\ntext. Right now the notebook contains only one empty code cell, labeled \u201cIn [1]:\u201d. Try\ntyping print(\"Hello world!\") in the cell, and click on the play button (see\nFigure 2-4) or press Shift-Enter. This sends the current cell to this notebook\u2019s Python\nkernel, which runs it and returns the output. The result is displayed below the cell,\nand since we reached the end of the notebook, a new cell is automatically created. Go\nthrough the User Interface Tour from Jupyter\u2019s Help menu to learn the basics.\nFigure 2-4. Hello world Python notebook\nDownload the Data\nIn typical environments your data would be available in a relational database (or\nsome other common datastore) and spread across multiple tables/documents/files. To\nGet the Data \n| \n43\nDownload from finelybook www.finelybook.com", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 164, "content": "11 You might also need to check legal constraints, such as private fields that should never be copied to unsafe\ndatastores.\n12 In a real project you would save this code in a Python file, but for now you can just write it in your Jupyter\nnotebook.\naccess it, you would first need to get your credentials and access authorizations,11 and\nfamiliarize yourself with the data schema. In this project, however, things are much\nsimpler: you will just download a single compressed file, housing.tgz, which contains a\ncomma-separated value (CSV) file called housing.csv with all the data.\nYou could use your web browser to download it, and run tar xzf housing.tgz to\ndecompress the file and extract the CSV file, but it is preferable to create a small func\u2010\ntion to do that. It is useful in particular if data changes regularly, as it allows you to\nwrite a small script that you can run whenever you need to fetch the latest data (or", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 165, "content": "write a small script that you can run whenever you need to fetch the latest data (or\nyou can set up a scheduled job to do that automatically at regular intervals). Auto\u2010\nmating the process of fetching the data is also useful if you need to install the dataset\non multiple machines.\nHere is the function to fetch the data:12\nimport os\nimport tarfile\nfrom six.moves import urllib\nDOWNLOAD_ROOT = \"https://raw.githubusercontent.com/ageron/handson-ml/master/\"\nHOUSING_PATH = \"datasets/housing\"\nHOUSING_URL = DOWNLOAD_ROOT + HOUSING_PATH + \"/housing.tgz\"\ndef fetch_housing_data(housing_url=HOUSING_URL, housing_path=HOUSING_PATH):\n if not os.path.isdir(housing_path):\n os.makedirs(housing_path)\n tgz_path = os.path.join(housing_path, \"housing.tgz\")\n urllib.request.urlretrieve(housing_url, tgz_path)\n housing_tgz = tarfile.open(tgz_path)\n housing_tgz.extractall(path=housing_path)\n housing_tgz.close()", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 166, "content": "housing_tgz.extractall(path=housing_path)\n housing_tgz.close()\nNow when you call fetch_housing_data(), it creates a datasets/housing directory in\nyour workspace, downloads the housing.tgz file, and extracts the housing.csv from it in\nthis directory.\nNow let\u2019s load the data using Pandas. Once again you should write a small function to\nload the data:\nimport pandas as pd\ndef load_housing_data(housing_path=HOUSING_PATH):\n csv_path = os.path.join(housing_path, \"housing.csv\")\n return pd.read_csv(csv_path)\n44 \n| \nChapter 2: End-to-End Machine Learning Project\nDownload from finelybook www.finelybook.com", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 167, "content": "This function returns a Pandas DataFrame object containing all the data.\nTake a Quick Look at the Data Structure\nLet\u2019s take a look at the top five rows using the DataFrame\u2019s head() method (see\nFigure 2-5).\nFigure 2-5. Top five rows in the dataset\nEach row represents one district. There are 10 attributes (you can see the first 6 in the\nscreenshot): longitude, latitude, housing_median_age, total_rooms, total_bed\nrooms, \npopulation, \nhouseholds, \nmedian_income, \nmedian_house_value, and\nocean_proximity.\nThe info() method is useful to get a quick description of the data, in particular the\ntotal number of rows, and each attribute\u2019s type and number of non-null values (see\nFigure 2-6).\nFigure 2-6. Housing info\nGet the Data \n| \n45\nDownload from finelybook www.finelybook.com", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 168, "content": "There are 20,640 instances in the dataset, which means that it is fairly small by\nMachine Learning standards, but it\u2019s perfect to get started. Notice that the total_bed\nrooms attribute has only 20,433 non-null values, meaning that 207 districts are miss\u2010\ning this feature. We will need to take care of this later.\nAll attributes are numerical, except the ocean_proximity field. Its type is object, so it\ncould hold any kind of Python object, but since you loaded this data from a CSV file\nyou know that it must be a text attribute. When you looked at the top five rows, you\nprobably noticed that the values in that column were repetitive, which means that it is\nprobably a categorical attribute. You can find out what categories exist and how many\ndistricts belong to each category by using the value_counts() method:\n>>> housing[\"ocean_proximity\"].value_counts()\n<1H OCEAN 9136\nINLAND 6551\nNEAR OCEAN 2658\nNEAR BAY 2290\nISLAND 5\nName: ocean_proximity, dtype: int64", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 169, "content": "NEAR OCEAN 2658\nNEAR BAY 2290\nISLAND 5\nName: ocean_proximity, dtype: int64\nLet\u2019s look at the other fields. The describe() method shows a summary of the\nnumerical attributes (Figure 2-7).\nFigure 2-7. Summary of each numerical attribute\nThe count, mean, min, and max rows are self-explanatory. Note that the null values are\nignored (so, for example, count of total_bedrooms is 20,433, not 20,640). The std\nrow shows the standard deviation (which measures how dispersed the values are).\nThe 25%, 50%, and 75% rows show the corresponding percentiles: a percentile indi\u2010\ncates the value below which a given percentage of observations in a group of observa\u2010\ntions falls. For example, 25% of the districts have a housing_median_age lower than\n46 \n| \nChapter 2: End-to-End Machine Learning Project\nDownload from finelybook www.finelybook.com", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 170, "content": "18, while 50% are lower than 29 and 75% are lower than 37. These are often called the\n25th percentile (or 1st quartile), the median, and the 75th percentile (or 3rd quartile).\nAnother quick way to get a feel of the type of data you are dealing with is to plot a \nhistogram for each numerical attribute. A histogram shows the number of instances\n(on the vertical axis) that have a given value range (on the horizontal axis). You can\neither plot this one attribute at a time, or you can call the hist() method on the\nwhole dataset, and it will plot a histogram for each numerical attribute (see\nFigure 2-8). For example, you can see that slightly over 800 districts have a\nmedian_house_value equal to about $500,000.\n%matplotlib inline # only in a Jupyter notebook\nimport matplotlib.pyplot as plt\nhousing.hist(bins=50, figsize=(20,15))\nplt.show()\nFigure 2-8. A histogram for each numerical attribute\nGet the Data \n| \n47\nDownload from finelybook www.finelybook.com", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 171, "content": "The hist() method relies on Matplotlib, which in turn relies on a\nuser-specified graphical backend to draw on your screen. So before\nyou can plot anything, you need to specify which backend Matplot\u2010\nlib should use. The simplest option is to use Jupyter\u2019s magic com\u2010\nmand %matplotlib inline. This tells Jupyter to set up Matplotlib\nso it uses Jupyter\u2019s own backend. Plots are then rendered within the\nnotebook itself. Note that calling show() is optional in a Jupyter\nnotebook, as Jupyter will automatically display plots when a cell is\nexecuted.\nNotice a few things in these histograms:\n1. First, the median income attribute does not look like it is expressed in US dollars\n(USD). After checking with the team that collected the data, you are told that the\ndata has been scaled and capped at 15 (actually 15.0001) for higher median\nincomes, and at 0.5 (actually 0.4999) for lower median incomes. Working with \npreprocessed attributes is common in Machine Learning, and it is not necessarily", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 172, "content": "preprocessed attributes is common in Machine Learning, and it is not necessarily\na problem, but you should try to understand how the data was computed.\n2. The housing median age and the median house value were also capped. The lat\u2010\nter may be a serious problem since it is your target attribute (your labels). Your\nMachine Learning algorithms may learn that prices never go beyond that limit.\nYou need to check with your client team (the team that will use your system\u2019s out\u2010\nput) to see if this is a problem or not. If they tell you that they need precise pre\u2010\ndictions even beyond $500,000, then you have mainly two options:\na. Collect proper labels for the districts whose labels were capped.\nb. Remove those districts from the training set (and also from the test set, since\nyour system should not be evaluated poorly if it predicts values beyond\n$500,000).\n3. These attributes have very different scales. We will discuss this later in this chap\u2010\nter when we explore feature scaling.", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 173, "content": "ter when we explore feature scaling.\n4. Finally, many histograms are tail heavy: they extend much farther to the right of\nthe median than to the left. This may make it a bit harder for some Machine\nLearning algorithms to detect patterns. We will try transforming these attributes\nlater on to have more bell-shaped distributions.\nHopefully you now have a better understanding of the kind of data you are dealing\nwith.\n48 \n| \nChapter 2: End-to-End Machine Learning Project\nDownload from finelybook www.finelybook.com", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 174, "content": "13 You will often see people set the random seed to 42. This number has no special property, other than to be\nThe Answer to the Ultimate Question of Life, the Universe, and Everything.\nWait! Before you look at the data any further, you need to create a\ntest set, put it aside, and never look at it.\nCreate a Test Set\nIt may sound strange to voluntarily set aside part of the data at this stage. After all,\nyou have only taken a quick glance at the data, and surely you should learn a whole\nlot more about it before you decide what algorithms to use, right? This is true, but\nyour brain is an amazing pattern detection system, which means that it is highly\nprone to overfitting: if you look at the test set, you may stumble upon some seemingly\ninteresting pattern in the test data that leads you to select a particular kind of\nMachine Learning model. When you estimate the generalization error using the test\nset, your estimate will be too optimistic and you will launch a system that will not", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 175, "content": "set, your estimate will be too optimistic and you will launch a system that will not\nperform as well as expected. This is called data snooping bias.\nCreating a test set is theoretically quite simple: just pick some instances randomly,\ntypically 20% of the dataset, and set them aside:\nimport numpy as np\ndef split_train_test(data, test_ratio):\n shuffled_indices = np.random.permutation(len(data))\n test_set_size = int(len(data) * test_ratio)\n test_indices = shuffled_indices[:test_set_size]\n train_indices = shuffled_indices[test_set_size:]\n return data.iloc[train_indices], data.iloc[test_indices]\nYou can then use this function like this:\n>>> train_set, test_set = split_train_test(housing, 0.2)\n>>> print(len(train_set), \"train +\", len(test_set), \"test\")\n16512 train + 4128 test\nWell, this works, but it is not perfect: if you run the program again, it will generate a\ndifferent test set! Over time, you (or your Machine Learning algorithms) will get to", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 176, "content": "different test set! Over time, you (or your Machine Learning algorithms) will get to\nsee the whole dataset, which is what you want to avoid.\nOne solution is to save the test set on the first run and then load it in subsequent\nruns. Another option is to set the random number generator\u2019s seed (e.g., np.ran\ndom.seed(42))13 before calling np.random.permutation(), so that it always generates\nthe same shuffled indices.\nGet the Data \n| \n49\nDownload from finelybook www.finelybook.com", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 177, "content": "14 The location information is actually quite coarse, and as a result many districts will have the exact same ID, so\nthey will end up in the same set (test or train). This introduces some unfortunate sampling bias.\nBut both these solutions will break next time you fetch an updated dataset. A com\u2010\nmon solution is to use each instance\u2019s identifier to decide whether or not it should go\nin the test set (assuming instances have a unique and immutable identifier). For\nexample, you could compute a hash of each instance\u2019s identifier, keep only the last\nbyte of the hash, and put the instance in the test set if this value is lower or equal to\n51 (~20% of 256). This ensures that the test set will remain consistent across multiple\nruns, even if you refresh the dataset. The new test set will contain 20% of the new\ninstances, but it will not contain any instance that was previously in the training set.\nHere is a possible implementation:\nimport hashlib", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 178, "content": "Here is a possible implementation:\nimport hashlib\ndef test_set_check(identifier, test_ratio, hash):\n return hash(np.int64(identifier)).digest()[-1] < 256 * test_ratio\ndef split_train_test_by_id(data, test_ratio, id_column, hash=hashlib.md5):\n ids = data[id_column]\n in_test_set = ids.apply(lambda id_: test_set_check(id_, test_ratio, hash))\n return data.loc[~in_test_set], data.loc[in_test_set]\nUnfortunately, the housing dataset does not have an identifier column. The simplest\nsolution is to use the row index as the ID:\nhousing_with_id = housing.reset_index() # adds an `index` column\ntrain_set, test_set = split_train_test_by_id(housing_with_id, 0.2, \"index\")\nIf you use the row index as a unique identifier, you need to make sure that new data\ngets appended to the end of the dataset, and no row ever gets deleted. If this is not\npossible, then you can try to use the most stable features to build a unique identifier.", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 179, "content": "possible, then you can try to use the most stable features to build a unique identifier.\nFor example, a district\u2019s latitude and longitude are guaranteed to be stable for a few\nmillion years, so you could combine them into an ID like so:14\nhousing_with_id[\"id\"] = housing[\"longitude\"] * 1000 + housing[\"latitude\"]\ntrain_set, test_set = split_train_test_by_id(housing_with_id, 0.2, \"id\")\nScikit-Learn provides a few functions to split datasets into multiple subsets in various\nways. The simplest function is train_test_split, which does pretty much the same\nthing as the function split_train_test defined earlier, with a couple of additional\nfeatures. First there is a random_state parameter that allows you to set the random\ngenerator seed as explained previously, and second you can pass it multiple datasets\nwith an identical number of rows, and it will split them on the same indices (this is\nvery useful, for example, if you have a separate DataFrame for labels):\n50 \n|", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 180, "content": "very useful, for example, if you have a separate DataFrame for labels):\n50 \n| \nChapter 2: End-to-End Machine Learning Project\nDownload from finelybook www.finelybook.com", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 181, "content": "from sklearn.model_selection import train_test_split\ntrain_set, test_set = train_test_split(housing, test_size=0.2, random_state=42)\nSo far we have considered purely random sampling methods. This is generally fine if\nyour dataset is large enough (especially relative to the number of attributes), but if it\nis not, you run the risk of introducing a significant sampling bias. When a survey\ncompany decides to call 1,000 people to ask them a few questions, they don\u2019t just pick\n1,000 people randomly in a phone booth. They try to ensure that these 1,000 people\nare representative of the whole population. For example, the US population is com\u2010\nposed of 51.3% female and 48.7% male, so a well-conducted survey in the US would\ntry to maintain this ratio in the sample: 513 female and 487 male. This is called strati\u2010\nfied sampling: the population is divided into homogeneous subgroups called strata,\nand the right number of instances is sampled from each stratum to guarantee that the", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 182, "content": "and the right number of instances is sampled from each stratum to guarantee that the\ntest set is representative of the overall population. If they used purely random sam\u2010\npling, there would be about 12% chance of sampling a skewed test set with either less\nthan 49% female or more than 54% female. Either way, the survey results would be\nsignificantly biased.\nSuppose you chatted with experts who told you that the median income is a very\nimportant attribute to predict median housing prices. You may want to ensure that\nthe test set is representative of the various categories of incomes in the whole dataset.\nSince the median income is a continuous numerical attribute, you first need to create\nan income category attribute. Let\u2019s look at the median income histogram more closely\n(see Figure 2-9):\nFigure 2-9. Histogram of income categories\nMost median income values are clustered around 2\u20135 (tens of thousands of dollars),", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 183, "content": "Most median income values are clustered around 2\u20135 (tens of thousands of dollars),\nbut some median incomes go far beyond 6. It is important to have a sufficient num\u2010\nGet the Data \n| \n51\nDownload from finelybook www.finelybook.com", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 184, "content": "ber of instances in your dataset for each stratum, or else the estimate of the stratum\u2019s\nimportance may be biased. This means that you should not have too many strata, and\neach stratum should be large enough. The following code creates an income category\nattribute by dividing the median income by 1.5 (to limit the number of income cate\u2010\ngories), and rounding up using ceil (to have discrete categories), and then merging\nall the categories greater than 5 into category 5:\nhousing[\"income_cat\"] = np.ceil(housing[\"median_income\"] / 1.5)\nhousing[\"income_cat\"].where(housing[\"income_cat\"] < 5, 5.0, inplace=True)\nNow you are ready to do stratified sampling based on the income category. For this\nyou can use Scikit-Learn\u2019s StratifiedShuffleSplit class:\nfrom sklearn.model_selection import StratifiedShuffleSplit\nsplit = StratifiedShuffleSplit(n_splits=1, test_size=0.2, random_state=42)\nfor train_index, test_index in split.split(housing, housing[\"income_cat\"]):", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 185, "content": "for train_index, test_index in split.split(housing, housing[\"income_cat\"]):\n strat_train_set = housing.loc[train_index]\n strat_test_set = housing.loc[test_index]\nLet\u2019s see if this worked as expected. You can start by looking at the income category\nproportions in the full housing dataset:\n>>> housing[\"income_cat\"].value_counts() / len(housing)\n3.0 0.350581\n2.0 0.318847\n4.0 0.176308\n5.0 0.114438\n1.0 0.039826\nName: income_cat, dtype: float64\nWith similar code you can measure the income category proportions in the test set.\nFigure 2-10 compares the income category proportions in the overall dataset, in the\ntest set generated with stratified sampling, and in a test set generated using purely\nrandom sampling. As you can see, the test set generated using stratified sampling has\nincome category proportions almost identical to those in the full dataset, whereas the\ntest set generated using purely random sampling is quite skewed.", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 186, "content": "test set generated using purely random sampling is quite skewed.\nFigure 2-10. Sampling bias comparison of stratified versus purely random sampling\n52 \n| \nChapter 2: End-to-End Machine Learning Project\nDownload from finelybook www.finelybook.com", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 187, "content": "Now you should remove the income_cat attribute so the data is back to its original\nstate:\nfor set in (strat_train_set, strat_test_set):\n set.drop([\"income_cat\"], axis=1, inplace=True)\nWe spent quite a bit of time on test set generation for a good reason: this is an often\nneglected but critical part of a Machine Learning project. Moreover, many of these\nideas will be useful later when we discuss cross-validation. Now it\u2019s time to move on\nto the next stage: exploring the data.\nDiscover and Visualize the Data to Gain Insights\nSo far you have only taken a quick glance at the data to get a general understanding of\nthe kind of data you are manipulating. Now the goal is to go a little bit more in depth.\nFirst, make sure you have put the test set aside and you are only exploring the train\u2010\ning set. Also, if the training set is very large, you may want to sample an exploration\nset, to make manipulations easy and fast. In our case, the set is quite small so you can", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 188, "content": "set, to make manipulations easy and fast. In our case, the set is quite small so you can\njust work directly on the full set. Let\u2019s create a copy so you can play with it without\nharming the training set:\nhousing = strat_train_set.copy()\nVisualizing Geographical Data\nSince there is geographical information (latitude and longitude), it is a good idea to\ncreate a scatterplot of all districts to visualize the data (Figure 2-11):\nhousing.plot(kind=\"scatter\", x=\"longitude\", y=\"latitude\")\nFigure 2-11. A geographical scatterplot of the data\nDiscover and Visualize the Data to Gain Insights \n| \n53\nDownload from finelybook www.finelybook.com", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 189, "content": "15 If you are reading this in grayscale, grab a red pen and scribble over most of the coastline from the Bay Area\ndown to San Diego (as you might expect). You can add a patch of yellow around Sacramento as well.\nThis looks like California all right, but other than that it is hard to see any particular\npattern. Setting the alpha option to 0.1 makes it much easier to visualize the places\nwhere there is a high density of data points (Figure 2-12):\nhousing.plot(kind=\"scatter\", x=\"longitude\", y=\"latitude\", alpha=0.1)\nFigure 2-12. A better visualization highlighting high-density areas\nNow that\u2019s much better: you can clearly see the high-density areas, namely the Bay\nArea and around Los Angeles and San Diego, plus a long line of fairly high density in\nthe Central Valley, in particular around Sacramento and Fresno.\nMore generally, our brains are very good at spotting patterns on pictures, but you\nmay need to play around with visualization parameters to make the patterns stand\nout.", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 190, "content": "may need to play around with visualization parameters to make the patterns stand\nout.\nNow let\u2019s look at the housing prices (Figure 2-13). The radius of each circle represents\nthe district\u2019s population (option s), and the color represents the price (option c). We\nwill use a predefined color map (option cmap) called jet, which ranges from blue\n(low values) to red (high prices):15\nhousing.plot(kind=\"scatter\", x=\"longitude\", y=\"latitude\", alpha=0.4,\n s=housing[\"population\"]/100, label=\"population\",\n c=\"median_house_value\", cmap=plt.get_cmap(\"jet\"), colorbar=True,\n)\nplt.legend()\n54 \n| \nChapter 2: End-to-End Machine Learning Project\nDownload from finelybook www.finelybook.com", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 191, "content": "Figure 2-13. California housing prices\nThis image tells you that the housing prices are very much related to the location\n(e.g., close to the ocean) and to the population density, as you probably knew already.\nIt will probably be useful to use a clustering algorithm to detect the main clusters, and\nadd new features that measure the proximity to the cluster centers. The ocean prox\u2010\nimity attribute may be useful as well, although in Northern California the housing\nprices in coastal districts are not too high, so it is not a simple rule.\nLooking for Correlations\nSince the dataset is not too large, you can easily compute the standard correlation\ncoefficient (also called Pearson\u2019s r) between every pair of attributes using the corr()\nmethod:\ncorr_matrix = housing.corr()\nNow let\u2019s look at how much each attribute correlates with the median house value:\n>>> corr_matrix[\"median_house_value\"].sort_values(ascending=False)\nmedian_house_value 1.000000\nmedian_income 0.687170", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 192, "content": "median_house_value 1.000000\nmedian_income 0.687170\ntotal_rooms 0.135231\nhousing_median_age 0.114220\nhouseholds 0.064702\nDiscover and Visualize the Data to Gain Insights \n| \n55\nDownload from finelybook www.finelybook.com", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 193, "content": "total_bedrooms 0.047865\npopulation -0.026699\nlongitude -0.047279\nlatitude -0.142826\nName: median_house_value, dtype: float64\nThe correlation coefficient ranges from \u20131 to 1. When it is close to 1, it means that\nthere is a strong positive correlation; for example, the median house value tends to go\nup when the median income goes up. When the coefficient is close to \u20131, it means\nthat there is a strong negative correlation; you can see a small negative correlation\nbetween the latitude and the median house value (i.e., prices have a slight tendency to\ngo down when you go north). Finally, coefficients close to zero mean that there is no\nlinear correlation. Figure 2-14 shows various plots along with the correlation coeffi\u2010\ncient between their horizontal and vertical axes.\nFigure 2-14. Standard correlation coefficient of various datasets (source: Wikipedia;\npublic domain image)\nThe correlation coefficient only measures linear correlations (\u201cif x", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 194, "content": "public domain image)\nThe correlation coefficient only measures linear correlations (\u201cif x\ngoes up, then y generally goes up/down\u201d). It may completely miss\nout on nonlinear relationships (e.g., \u201cif x is close to zero then y gen\u2010\nerally goes up\u201d). Note how all the plots of the bottom row have a\ncorrelation coefficient equal to zero despite the fact that their axes\nare clearly not independent: these are examples of nonlinear rela\u2010\ntionships. Also, the second row shows examples where the correla\u2010\ntion coefficient is equal to 1 or \u20131; notice that this has nothing to\ndo with the slope. For example, your height in inches has a correla\u2010\ntion coefficient of 1 with your height in feet or in nanometers.\nAnother way to check for correlation between attributes is to use Pandas\u2019\nscatter_matrix function, which plots every numerical attribute against every other\nnumerical attribute. Since there are now 11 numerical attributes, you would get 112 =\n56 \n| \nChapter 2: End-to-End Machine Learning Project", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 195, "content": "56 \n| \nChapter 2: End-to-End Machine Learning Project\nDownload from finelybook www.finelybook.com", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 196, "content": "121 plots, which would not fit on a page, so let\u2019s just focus on a few promising\nattributes that seem most correlated with the median housing value (Figure 2-15):\nfrom pandas.tools.plotting import scatter_matrix\nattributes = [\"median_house_value\", \"median_income\", \"total_rooms\",\n \"housing_median_age\"]\nscatter_matrix(housing[attributes], figsize=(12, 8))\nFigure 2-15. Scatter matrix\nThe main diagonal (top left to bottom right) would be full of straight lines if Pandas\nplotted each variable against itself, which would not be very useful. So instead Pandas\ndisplays a histogram of each attribute (other options are available; see Pandas\u2019 docu\u2010\nmentation for more details).\nThe most promising attribute to predict the median house value is the median\nincome, so let\u2019s zoom in on their correlation scatterplot (Figure 2-16):\nhousing.plot(kind=\"scatter\", x=\"median_income\", y=\"median_house_value\",\n alpha=0.1)\nDiscover and Visualize the Data to Gain Insights \n| \n57", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 197, "content": "alpha=0.1)\nDiscover and Visualize the Data to Gain Insights \n| \n57\nDownload from finelybook www.finelybook.com", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 198, "content": "Figure 2-16. Median income versus median house value\nThis plot reveals a few things. First, the correlation is indeed very strong; you can\nclearly see the upward trend and the points are not too dispersed. Second, the price\ncap that we noticed earlier is clearly visible as a horizontal line at $500,000. But this\nplot reveals other less obvious straight lines: a horizontal line around $450,000,\nanother around $350,000, perhaps one around $280,000, and a few more below that.\nYou may want to try removing the corresponding districts to prevent your algorithms\nfrom learning to reproduce these data quirks.\nExperimenting with Attribute Combinations\nHopefully the previous sections gave you an idea of a few ways you can explore the\ndata and gain insights. You identified a few data quirks that you may want to clean up\nbefore feeding the data to a Machine Learning algorithm, and you found interesting\ncorrelations between attributes, in particular with the target attribute. You also", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 199, "content": "correlations between attributes, in particular with the target attribute. You also\nnoticed that some attributes have a tail-heavy distribution, so you may want to trans\u2010\nform them (e.g., by computing their logarithm). Of course, your mileage will vary\nconsiderably with each project, but the general ideas are similar.\nOne last thing you may want to do before actually preparing the data for Machine\nLearning algorithms is to try out various attribute combinations. For example, the\ntotal number of rooms in a district is not very useful if you don\u2019t know how many\nhouseholds there are. What you really want is the number of rooms per household.\nSimilarly, the total number of bedrooms by itself is not very useful: you probably\nwant to compare it to the number of rooms. And the population per household also\n58 \n| \nChapter 2: End-to-End Machine Learning Project\nDownload from finelybook www.finelybook.com", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 200, "content": "seems like an interesting attribute combination to look at. Let\u2019s create these new\nattributes:\nhousing[\"rooms_per_household\"] = housing[\"total_rooms\"]/housing[\"households\"]\nhousing[\"bedrooms_per_room\"] = housing[\"total_bedrooms\"]/housing[\"total_rooms\"]\nhousing[\"population_per_household\"]=housing[\"population\"]/housing[\"households\"]\nAnd now let\u2019s look at the correlation matrix again:\n>>> corr_matrix = housing.corr()\n>>> corr_matrix[\"median_house_value\"].sort_values(ascending=False)\nmedian_house_value 1.000000\nmedian_income 0.687170\nrooms_per_household 0.199343\ntotal_rooms 0.135231\nhousing_median_age 0.114220\nhouseholds 0.064702\ntotal_bedrooms 0.047865\npopulation_per_household -0.021984\npopulation -0.026699\nlongitude -0.047279\nlatitude -0.142826\nbedrooms_per_room -0.260070\nName: median_house_value, dtype: float64", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 201, "content": "bedrooms_per_room -0.260070\nName: median_house_value, dtype: float64\nHey, not bad! The new bedrooms_per_room attribute is much more correlated with\nthe median house value than the total number of rooms or bedrooms. Apparently\nhouses with a lower bedroom/room ratio tend to be more expensive. The number of\nrooms per household is also more informative than the total number of rooms in a\ndistrict\u2014obviously the larger the houses, the more expensive they are.\nThis round of exploration does not have to be absolutely thorough; the point is to\nstart off on the right foot and quickly gain insights that will help you get a first rea\u2010\nsonably good prototype. But this is an iterative process: once you get a prototype up\nand running, you can analyze its output to gain more insights and come back to this\nexploration step.\nPrepare the Data for Machine Learning Algorithms\nIt\u2019s time to prepare the data for your Machine Learning algorithms. Instead of just", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 202, "content": "It\u2019s time to prepare the data for your Machine Learning algorithms. Instead of just\ndoing this manually, you should write functions to do that, for several good reasons:\n\u2022 This will allow you to reproduce these transformations easily on any dataset (e.g.,\nthe next time you get a fresh dataset).\n\u2022 You will gradually build a library of transformation functions that you can reuse\nin future projects.\n\u2022 You can use these functions in your live system to transform the new data before\nfeeding it to your algorithms.\nPrepare the Data for Machine Learning Algorithms \n| \n59\nDownload from finelybook www.finelybook.com", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 203, "content": "\u2022 This will make it possible for you to easily try various transformations and see\nwhich combination of transformations works best.\nBut first let\u2019s revert to a clean training set (by copying strat_train_set once again),\nand let\u2019s separate the predictors and the labels since we don\u2019t necessarily want to apply\nthe same transformations to the predictors and the target values (note that drop() \ncreates a copy of the data and does not affect strat_train_set):\nhousing = strat_train_set.drop(\"median_house_value\", axis=1)\nhousing_labels = strat_train_set[\"median_house_value\"].copy()\nData Cleaning\nMost Machine Learning algorithms cannot work with missing features, so let\u2019s create\na few functions to take care of them. You noticed earlier that the total_bedrooms\nattribute has some missing values, so let\u2019s fix this. You have three options:\n\u2022 Get rid of the corresponding districts.\n\u2022 Get rid of the whole attribute.\n\u2022 Set the values to some value (zero, the mean, the median, etc.).", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 204, "content": "\u2022 Set the values to some value (zero, the mean, the median, etc.).\nYou can accomplish these easily using DataFrame\u2019s dropna(), drop(), and fillna()\nmethods:\nhousing.dropna(subset=[\"total_bedrooms\"]) # option 1\nhousing.drop(\"total_bedrooms\", axis=1) # option 2\nmedian = housing[\"total_bedrooms\"].median()\nhousing[\"total_bedrooms\"].fillna(median) # option 3\nIf you choose option 3, you should compute the median value on the training set, and\nuse it to fill the missing values in the training set, but also don\u2019t forget to save the\nmedian value that you have computed. You will need it later to replace missing values\nin the test set when you want to evaluate your system, and also once the system goes\nlive to replace missing values in new data.\nScikit-Learn provides a handy class to take care of missing values: Imputer. Here is\nhow to use it. First, you need to create an Imputer instance, specifying that you want", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 205, "content": "how to use it. First, you need to create an Imputer instance, specifying that you want\nto replace each attribute\u2019s missing values with the median of that attribute:\nfrom sklearn.preprocessing import Imputer\nimputer = Imputer(strategy=\"median\")\nSince the median can only be computed on numerical attributes, we need to create a\ncopy of the data without the text attribute ocean_proximity:\nhousing_num = housing.drop(\"ocean_proximity\", axis=1)\n60 \n| \nChapter 2: End-to-End Machine Learning Project\nDownload from finelybook www.finelybook.com", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 206, "content": "16 For more details on the design principles, see \u201cAPI design for machine learning software: experiences from\nthe scikit-learn project,\u201d L. Buitinck, G. Louppe, M. Blondel, F. Pedregosa, A. M\u00fcller, et al. (2013).\nNow you can fit the imputer instance to the training data using the fit() method:\nimputer.fit(housing_num)\nThe imputer has simply computed the median of each attribute and stored the result\nin its statistics_ instance variable. Only the total_bedrooms attribute had missing\nvalues, but we cannot be sure that there won\u2019t be any missing values in new data after\nthe system goes live, so it is safer to apply the imputer to all the numerical attributes:\n>>> imputer.statistics_\narray([ -118.51 , 34.26 , 29. , 2119. , 433. , 1164. , 408. , 3.5414])\n>>> housing_num.median().values\narray([ -118.51 , 34.26 , 29. , 2119. , 433. , 1164. , 408. , 3.5414])\nNow you can use this \u201ctrained\u201d imputer to transform the training set by replacing\nmissing values by the learned medians:", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 207, "content": "missing values by the learned medians:\nX = imputer.transform(housing_num)\nThe result is a plain Numpy array containing the transformed features. If you want to\nput it back into a Pandas DataFrame, it\u2019s simple:\nhousing_tr = pd.DataFrame(X, columns=housing_num.columns)\nScikit-Learn Design\nScikit-Learn\u2019s API is remarkably well designed. The main design principles are:16\n\u2022 Consistency. All objects share a consistent and simple interface:\n\u2014 Estimators. Any object that can estimate some parameters based on a dataset\nis called an estimator (e.g., an imputer is an estimator). The estimation itself is\nperformed by the fit() method, and it takes only a dataset as a parameter (or\ntwo for supervised learning algorithms; the second dataset contains the\nlabels). Any other parameter needed to guide the estimation process is con\u2010\nsidered a hyperparameter (such as an imputer\u2019s strategy), and it must be set\nas an instance variable (generally via a constructor parameter).", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 208, "content": "as an instance variable (generally via a constructor parameter).\n\u2014 Transformers. Some estimators (such as an imputer) can also transform a\ndataset; these are called transformers. Once again, the API is quite simple: the\ntransformation is performed by the transform() method with the dataset to\ntransform as a parameter. It returns the transformed dataset. This transforma\u2010\ntion generally relies on the learned parameters, as is the case for an imputer.\nAll transformers also have a convenience method called fit_transform() \nPrepare the Data for Machine Learning Algorithms \n| \n61\nDownload from finelybook www.finelybook.com", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 209, "content": "17 Some predictors also provide methods to measure the confidence of their predictions.\nthat is equivalent to calling fit() and then transform() (but sometimes\nfit_transform() is optimized and runs much faster).\n\u2014 Predictors. Finally, some estimators are capable of making predictions given a\ndataset; they are called predictors. For example, the LinearRegression model \nin the previous chapter was a predictor: it predicted life satisfaction given a\ncountry\u2019s GDP per capita. A predictor has a predict() method that takes a\ndataset of new instances and returns a dataset of corresponding predictions. It\nalso has a score() method that measures the quality of the predictions given\na test set (and the corresponding labels in the case of supervised learning\nalgorithms).17\n\u2022 Inspection. All the estimator\u2019s hyperparameters are accessible directly via public\ninstance variables (e.g., imputer.strategy), and all the estimator\u2019s learned", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 210, "content": "instance variables (e.g., imputer.strategy), and all the estimator\u2019s learned\nparameters are also accessible via public instance variables with an underscore\nsuffix (e.g., imputer.statistics_).\n\u2022 Nonproliferation of classes. Datasets are represented as NumPy arrays or SciPy\nsparse matrices, instead of homemade classes. Hyperparameters are just regular\nPython strings or numbers.\n\u2022 Composition. Existing building blocks are reused as much as possible. For\nexample, it is easy to create a Pipeline estimator from an arbitrary sequence of\ntransformers followed by a final estimator, as we will see.\n\u2022 Sensible defaults. Scikit-Learn provides reasonable default values for most\nparameters, making it easy to create a baseline working system quickly.\nHandling Text and Categorical Attributes\nEarlier we left out the categorical attribute ocean_proximity because it is a text\nattribute so we cannot compute its median. Most Machine Learning algorithms pre\u2010", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 211, "content": "attribute so we cannot compute its median. Most Machine Learning algorithms pre\u2010\nfer to work with numbers anyway, so let\u2019s convert these text labels to numbers.\nScikit-Learn provides a transformer for this task called LabelEncoder:\n>>> from sklearn.preprocessing import LabelEncoder\n>>> encoder = LabelEncoder()\n>>> housing_cat = housing[\"ocean_proximity\"]\n>>> housing_cat_encoded = encoder.fit_transform(housing_cat)\n>>> housing_cat_encoded\narray([1, 1, 4, ..., 1, 0, 3])\n62 \n| \nChapter 2: End-to-End Machine Learning Project\nDownload from finelybook www.finelybook.com", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 212, "content": "18 NumPy\u2019s reshape() function allows one dimension to be \u20131, which means \u201cunspecified\u201d: the value is inferred\nfrom the length of the array and the remaining dimensions.\n19 See SciPy\u2019s documentation for more details.\nThis is better: now we can use this numerical data in any ML algorithm. You can look\nat the mapping that this encoder has learned using the classes_ attribute (\u201c<1H\nOCEAN\u201d is mapped to 0, \u201cINLAND\u201d is mapped to 1, etc.):\n>>> print(encoder.classes_)\n['<1H OCEAN' 'INLAND' 'ISLAND' 'NEAR BAY' 'NEAR OCEAN']\nOne issue with this representation is that ML algorithms will assume that two nearby\nvalues are more similar than two distant values. Obviously this is not the case (for\nexample, categories 0 and 4 are more similar than categories 0 and 1). To fix this\nissue, a common solution is to create one binary attribute per category: one attribute\nequal to 1 when the category is \u201c<1H OCEAN\u201d (and 0 otherwise), another attribute", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 213, "content": "equal to 1 when the category is \u201c<1H OCEAN\u201d (and 0 otherwise), another attribute\nequal to 1 when the category is \u201cINLAND\u201d (and 0 otherwise), and so on. This is\ncalled one-hot encoding, because only one attribute will be equal to 1 (hot), while the\nothers will be 0 (cold).\nScikit-Learn provides a OneHotEncoder encoder to convert integer categorical values\ninto one-hot vectors. Let\u2019s encode the categories as one-hot vectors. Note that\nfit_transform() expects a 2D array, but housing_cat_encoded is a 1D array, so we\nneed to reshape it:18\n>>> from sklearn.preprocessing import OneHotEncoder\n>>> encoder = OneHotEncoder()\n>>> housing_cat_1hot = encoder.fit_transform(housing_cat_encoded.reshape(-1,1))\n>>> housing_cat_1hot\n<16513x5 sparse matrix of type ''\n \nwith 16513 stored elements in Compressed Sparse Row format>\nNotice that the output is a SciPy sparse matrix, instead of a NumPy array. This is very", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 214, "content": "Notice that the output is a SciPy sparse matrix, instead of a NumPy array. This is very\nuseful when you have categorical attributes with thousands of categories. After one-\nhot encoding we get a matrix with thousands of columns, and the matrix is full of\nzeros except for one 1 per row. Using up tons of memory mostly to store zeros would\nbe very wasteful, so instead a sparse matrix only stores the location of the nonzero\nelements. You can use it mostly like a normal 2D array,19 but if you really want to con\u2010\nvert it to a (dense) NumPy array, just call the toarray() method:\n>>> housing_cat_1hot.toarray()\narray([[ 0., 1., 0., 0., 0.],\n [ 0., 1., 0., 0., 0.],\n [ 0., 0., 0., 0., 1.],\n ...,\n [ 0., 1., 0., 0., 0.],\nPrepare the Data for Machine Learning Algorithms \n| \n63\nDownload from finelybook www.finelybook.com", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 215, "content": "[ 1., 0., 0., 0., 0.],\n [ 0., 0., 0., 1., 0.]])\nWe can apply both transformations (from text categories to integer categories, then\nfrom integer categories to one-hot vectors) in one shot using the LabelBinarizer \nclass:\n>>> from sklearn.preprocessing import LabelBinarizer\n>>> encoder = LabelBinarizer()\n>>> housing_cat_1hot = encoder.fit_transform(housing_cat)\n>>> housing_cat_1hot\narray([[0, 1, 0, 0, 0],\n [0, 1, 0, 0, 0],\n [0, 0, 0, 0, 1],\n ...,\n [0, 1, 0, 0, 0],\n [1, 0, 0, 0, 0],\n [0, 0, 0, 1, 0]])\nNote that this returns a dense NumPy array by default. You can get a sparse matrix\ninstead by passing sparse_output=True to the LabelBinarizer constructor.\nCustom Transformers\nAlthough Scikit-Learn provides many useful transformers, you will need to write\nyour own for tasks such as custom cleanup operations or combining specific\nattributes. You will want your transformer to work seamlessly with Scikit-Learn func\u2010", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 216, "content": "attributes. You will want your transformer to work seamlessly with Scikit-Learn func\u2010\ntionalities (such as pipelines), and since Scikit-Learn relies on duck typing (not inher\u2010\nitance), all you need is to create a class and implement three methods: fit()\n(returning self), transform(), and fit_transform(). You can get the last one for\nfree by simply adding TransformerMixin as a base class. Also, if you add BaseEstima\ntor as a base class (and avoid *args and **kargs in your constructor) you will get\ntwo extra methods (get_params() and set_params()) that will be useful for auto\u2010\nmatic hyperparameter tuning. For example, here is a small transformer class that adds\nthe combined attributes we discussed earlier:\nfrom sklearn.base import BaseEstimator, TransformerMixin\nrooms_ix, bedrooms_ix, population_ix, household_ix = 3, 4, 5, 6\nclass CombinedAttributesAdder(BaseEstimator, TransformerMixin):\n def __init__(self, add_bedrooms_per_room = True): # no *args or **kargs", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 217, "content": "def __init__(self, add_bedrooms_per_room = True): # no *args or **kargs\n self.add_bedrooms_per_room = add_bedrooms_per_room\n def fit(self, X, y=None):\n return self # nothing else to do\n def transform(self, X, y=None):\n rooms_per_household = X[:, rooms_ix] / X[:, household_ix]\n population_per_household = X[:, population_ix] / X[:, household_ix]\n if self.add_bedrooms_per_room:\n bedrooms_per_room = X[:, bedrooms_ix] / X[:, rooms_ix]\n64 \n| \nChapter 2: End-to-End Machine Learning Project\nDownload from finelybook www.finelybook.com", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 218, "content": "return np.c_[X, rooms_per_household, population_per_household,\n bedrooms_per_room]\n else:\n return np.c_[X, rooms_per_household, population_per_household]\nattr_adder = CombinedAttributesAdder(add_bedrooms_per_room=False)\nhousing_extra_attribs = attr_adder.transform(housing.values)\nIn this example the transformer has one hyperparameter, add_bedrooms_per_room,\nset to True by default (it is often helpful to provide sensible defaults). This hyperpara\u2010\nmeter will allow you to easily find out whether adding this attribute helps the\nMachine Learning algorithms or not. More generally, you can add a hyperparameter\nto gate any data preparation step that you are not 100% sure about. The more you\nautomate these data preparation steps, the more combinations you can automatically\ntry out, making it much more likely that you will find a great combination (and sav\u2010\ning you a lot of time).\nFeature Scaling", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 219, "content": "ing you a lot of time).\nFeature Scaling\nOne of the most important transformations you need to apply to your data is feature\nscaling. With few exceptions, Machine Learning algorithms don\u2019t perform well when\nthe input numerical attributes have very different scales. This is the case for the hous\u2010\ning data: the total number of rooms ranges from about 6 to 39,320, while the median\nincomes only range from 0 to 15. Note that scaling the target values is generally not\nrequired.\nThere are two common ways to get all attributes to have the same scale: min-max\nscaling and standardization.\nMin-max scaling (many people call this normalization) is quite simple: values are\nshifted and rescaled so that they end up ranging from 0 to 1. We do this by subtract\u2010\ning the min value and dividing by the max minus the min. Scikit-Learn provides a\ntransformer called MinMaxScaler for this. It has a feature_range hyperparameter\nthat lets you change the range if you don\u2019t want 0\u20131 for some reason.", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 220, "content": "that lets you change the range if you don\u2019t want 0\u20131 for some reason.\nStandardization is quite different: first it subtracts the mean value (so standardized\nvalues always have a zero mean), and then it divides by the variance so that the result\u2010\ning distribution has unit variance. Unlike min-max scaling, standardization does not\nbound values to a specific range, which may be a problem for some algorithms (e.g.,\nneural networks often expect an input value ranging from 0 to 1). However, standard\u2010\nization is much less affected by outliers. For example, suppose a district had a median\nincome equal to 100 (by mistake). Min-max scaling would then crush all the other\nvalues from 0\u201315 down to 0\u20130.15, whereas standardization would not be much affec\u2010\nted. Scikit-Learn provides a transformer called StandardScaler for standardization.\nPrepare the Data for Machine Learning Algorithms \n| \n65\nDownload from finelybook www.finelybook.com", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 221, "content": "As with all the transformations, it is important to fit the scalers to\nthe training data only, not to the full dataset (including the test set).\nOnly then can you use them to transform the training set and the\ntest set (and new data).\nTransformation Pipelines\nAs you can see, there are many data transformation steps that need to be executed in\nthe right order. Fortunately, Scikit-Learn provides the Pipeline class to help with\nsuch sequences of transformations. Here is a small pipeline for the numerical\nattributes:\nfrom sklearn.pipeline import Pipeline\nfrom sklearn.preprocessing import StandardScaler\nnum_pipeline = Pipeline([\n ('imputer', Imputer(strategy=\"median\")),\n ('attribs_adder', CombinedAttributesAdder()),\n ('std_scaler', StandardScaler()),\n ])\nhousing_num_tr = num_pipeline.fit_transform(housing_num)\nThe Pipeline constructor takes a list of name/estimator pairs defining a sequence of", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 222, "content": "The Pipeline constructor takes a list of name/estimator pairs defining a sequence of\nsteps. All but the last estimator must be transformers (i.e., they must have a\nfit_transform() method). The names can be anything you like.\nWhen you call the pipeline\u2019s fit() method, it calls fit_transform() sequentially on\nall transformers, passing the output of each call as the parameter to the next call, until\nit reaches the final estimator, for which it just calls the fit() method.\nThe pipeline exposes the same methods as the final estimator. In this example, the last\nestimator is a StandardScaler, which is a transformer, so the pipeline has a trans\nform() method that applies all the transforms to the data in sequence (it also has a\nfit_transform method that we could have used instead of calling fit() and then\ntransform()).\nYou now have a pipeline for numerical values, and you also need to apply the LabelBi\nnarizer on the categorical values: how can you join these transformations into a sin\u2010", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 223, "content": "narizer on the categorical values: how can you join these transformations into a sin\u2010\ngle pipeline? Scikit-Learn provides a FeatureUnion class for this. You give it a list of\ntransformers (which can be entire transformer pipelines), and when its transform()\nmethod is called it runs each transformer\u2019s transform() method in parallel, waits for\ntheir output, and then concatenates them and returns the result (and of course calling\nits fit() method calls all each transformer\u2019s fit() method). A full pipeline handling\nboth numerical and categorical attributes may look like this:\n66 \n| \nChapter 2: End-to-End Machine Learning Project\nDownload from finelybook www.finelybook.com", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 224, "content": "20 But check out Pull Request #3886, which may introduce a ColumnTransformer class making attribute-specific\ntransformations easy. You could also run pip3 install sklearn-pandas to get a DataFrameMapper class with\na similar objective.\nfrom sklearn.pipeline import FeatureUnion\nnum_attribs = list(housing_num)\ncat_attribs = [\"ocean_proximity\"]\nnum_pipeline = Pipeline([\n ('selector', DataFrameSelector(num_attribs)),\n ('imputer', Imputer(strategy=\"median\")),\n ('attribs_adder', CombinedAttributesAdder()),\n ('std_scaler', StandardScaler()),\n ])\ncat_pipeline = Pipeline([\n ('selector', DataFrameSelector(cat_attribs)),\n ('label_binarizer', LabelBinarizer()),\n ])\nfull_pipeline = FeatureUnion(transformer_list=[\n (\"num_pipeline\", num_pipeline),\n (\"cat_pipeline\", cat_pipeline),\n ])\nAnd you can run the whole pipeline simply:\n>>> housing_prepared = full_pipeline.fit_transform(housing)\n>>> housing_prepared", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 225, "content": ">>> housing_prepared = full_pipeline.fit_transform(housing)\n>>> housing_prepared\narray([[ 0.73225807, -0.67331551, 0.58426443, ..., 0. ,\n 0. , 0. ],\n [-0.99102923, 1.63234656, -0.92655887, ..., 0. ,\n 0. , 0. ],\n [...]\n>>> housing_prepared.shape\n(16513, 17)\nEach subpipeline starts with a selector transformer: it simply transforms the data by\nselecting the desired attributes (numerical or categorical), dropping the rest, and con\u2010\nverting the resulting DataFrame to a NumPy array. There is nothing in Scikit-Learn\nto handle Pandas DataFrames,20 so we need to write a simple custom transformer for\nthis task:\nfrom sklearn.base import BaseEstimator, TransformerMixin\nclass DataFrameSelector(BaseEstimator, TransformerMixin):\n def __init__(self, attribute_names):\n self.attribute_names = attribute_names\n def fit(self, X, y=None):\n return self\nPrepare the Data for Machine Learning Algorithms \n| \n67", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 226, "content": "return self\nPrepare the Data for Machine Learning Algorithms \n| \n67\nDownload from finelybook www.finelybook.com", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 227, "content": "def transform(self, X):\n return X[self.attribute_names].values\nSelect and Train a Model\nAt last! You framed the problem, you got the data and explored it, you sampled a\ntraining set and a test set, and you wrote transformation pipelines to clean up and\nprepare your data for Machine Learning algorithms automatically. You are now ready\nto select and train a Machine Learning model.\nTraining and Evaluating on the Training Set\nThe good news is that thanks to all these previous steps, things are now going to be\nmuch simpler than you might think. Let\u2019s first train a Linear Regression model, like\nwe did in the previous chapter:\nfrom sklearn.linear_model import LinearRegression\nlin_reg = LinearRegression()\nlin_reg.fit(housing_prepared, housing_labels)\nDone! You now have a working Linear Regression model. Let\u2019s try it out on a few\ninstances from the training set:\n>>> some_data = housing.iloc[:5]\n>>> some_labels = housing_labels.iloc[:5]", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 228, "content": ">>> some_data = housing.iloc[:5]\n>>> some_labels = housing_labels.iloc[:5]\n>>> some_data_prepared = full_pipeline.transform(some_data)\n>>> print(\"Predictions:\\t\", lin_reg.predict(some_data_prepared))\nPredictions: \n [ 303104. 44800. 308928. 294208. 368704.]\n>>> print(\"Labels:\\t\\t\", list(some_labels))\nLabels: \n [359400.0, 69700.0, 302100.0, 301300.0, 351900.0]\nIt works, although the predictions are not exactly accurate (e.g., the second prediction\nis off by more than 50%!). Let\u2019s measure this regression model\u2019s RMSE on the whole\ntraining set using Scikit-Learn\u2019s mean_squared_error function:\n>>> from sklearn.metrics import mean_squared_error\n>>> housing_predictions = lin_reg.predict(housing_prepared)\n>>> lin_mse = mean_squared_error(housing_labels, housing_predictions)\n>>> lin_rmse = np.sqrt(lin_mse)\n>>> lin_rmse\n68628.413493824875\nOkay, this is better than nothing but clearly not a great score: most districts\u2019", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 229, "content": "68628.413493824875\nOkay, this is better than nothing but clearly not a great score: most districts\u2019\nmedian_housing_values range between $120,000 and $265,000, so a typical predic\u2010\ntion error of $68,628 is not very satisfying. This is an example of a model underfitting\nthe training data. When this happens it can mean that the features do not provide\nenough information to make good predictions, or that the model is not powerful\nenough. As we saw in the previous chapter, the main ways to fix underfitting are to\n68 \n| \nChapter 2: End-to-End Machine Learning Project\nDownload from finelybook www.finelybook.com", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 230, "content": "select a more powerful model, to feed the training algorithm with better features, or\nto reduce the constraints on the model. This model is not regularized, so this rules\nout the last option. You could try to add more features (e.g., the log of the popula\u2010\ntion), but first let\u2019s try a more complex model to see how it does.\nLet\u2019s train a DecisionTreeRegressor. This is a powerful model, capable of finding\ncomplex nonlinear relationships in the data (Decision Trees are presented in more\ndetail in Chapter 6). The code should look familiar by now:\nfrom sklearn.tree import DecisionTreeRegressor\ntree_reg = DecisionTreeRegressor()\ntree_reg.fit(housing_prepared, housing_labels)\nNow that the model is trained, let\u2019s evaluate it on the training set:\n>>> housing_predictions = tree_reg.predict(housing_prepared)\n>>> tree_mse = mean_squared_error(housing_labels, housing_predictions)\n>>> tree_rmse = np.sqrt(tree_mse)\n>>> tree_rmse\n0.0", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 231, "content": ">>> tree_rmse = np.sqrt(tree_mse)\n>>> tree_rmse\n0.0\nWait, what!? No error at all? Could this model really be absolutely perfect? Of course,\nit is much more likely that the model has badly overfit the data. How can you be sure?\nAs we saw earlier, you don\u2019t want to touch the test set until you are ready to launch a\nmodel you are confident about, so you need to use part of the training set for train\u2010\ning, and part for model validation.\nBetter Evaluation Using Cross-Validation\nOne way to evaluate the Decision Tree model would be to use the train_test_split\nfunction to split the training set into a smaller training set and a validation set, then\ntrain your models against the smaller training set and evaluate them against the vali\u2010\ndation set. It\u2019s a bit of work, but nothing too difficult and it would work fairly well.\nA great alternative is to use Scikit-Learn\u2019s cross-validation feature. The following code", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 232, "content": "A great alternative is to use Scikit-Learn\u2019s cross-validation feature. The following code\nperforms K-fold cross-validation: it randomly splits the training set into 10 distinct\nsubsets called folds, then it trains and evaluates the Decision Tree model 10 times,\npicking a different fold for evaluation every time and training on the other 9 folds.\nThe result is an array containing the 10 evaluation scores:\nfrom sklearn.model_selection import cross_val_score\nscores = cross_val_score(tree_reg, housing_prepared, housing_labels,\n scoring=\"neg_mean_squared_error\", cv=10)\nrmse_scores = np.sqrt(-scores)\nSelect and Train a Model \n| \n69\nDownload from finelybook www.finelybook.com", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 233, "content": "Scikit-Learn cross-validation features expect a utility function\n(greater is better) rather than a cost function (lower is better), so\nthe scoring function is actually the opposite of the MSE (i.e., a neg\u2010\native value), which is why the preceding code computes -scores\nbefore calculating the square root.\nLet\u2019s look at the results:\n>>> def display_scores(scores):\n... print(\"Scores:\", scores)\n... print(\"Mean:\", scores.mean())\n... print(\"Standard deviation:\", scores.std())\n...\n>>> display_scores(tree_rmse_scores)\nScores: [ 74678.4916885 64766.2398337 69632.86942005 69166.67693232\n 71486.76507766 73321.65695983 71860.04741226 71086.32691692\n 76934.2726093 69060.93319262]\nMean: 71199.4280043\nStandard deviation: 3202.70522793\nNow the Decision Tree doesn\u2019t look as good as it did earlier. In fact, it seems to per\u2010\nform worse than the Linear Regression model! Notice that cross-validation allows", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 234, "content": "form worse than the Linear Regression model! Notice that cross-validation allows\nyou to get not only an estimate of the performance of your model, but also a measure\nof how precise this estimate is (i.e., its standard deviation). The Decision Tree has a\nscore of approximately 71,200, generally \u00b13,200. You would not have this information\nif you just used one validation set. But cross-validation comes at the cost of training\nthe model several times, so it is not always possible.\nLet\u2019s compute the same scores for the Linear Regression model just to be sure:\n>>> lin_scores = cross_val_score(lin_reg, housing_prepared, housing_labels,\n... scoring=\"neg_mean_squared_error\", cv=10)\n...\n>>> lin_rmse_scores = np.sqrt(-lin_scores)\n>>> display_scores(lin_rmse_scores)\nScores: [ 70423.5893262 65804.84913139 66620.84314068 72510.11362141\n 66414.74423281 71958.89083606 67624.90198297 67825.36117664\n 72512.36533141 68028.11688067]", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 235, "content": "72512.36533141 68028.11688067]\nMean: 68972.377566\nStandard deviation: 2493.98819069\nThat\u2019s right: the Decision Tree model is overfitting so badly that it performs worse\nthan the Linear Regression model.\nLet\u2019s try one last model now: the RandomForestRegressor. As we will see in Chap\u2010\nter 7, Random Forests work by training many Decision Trees on random subsets of\nthe features, then averaging out their predictions. Building a model on top of many\nother models is called Ensemble Learning, and it is often a great way to push ML algo\u2010\nrithms even further. We will skip most of the code since it is essentially the same as\nfor the other models:\n70 \n| \nChapter 2: End-to-End Machine Learning Project\nDownload from finelybook www.finelybook.com", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 236, "content": ">>> from sklearn.ensemble import RandomForestRegressor\n>>> forest_reg = RandomForestRegressor()\n>>> forest_reg.fit(housing_prepared, housing_labels)\n>>> [...]\n>>> forest_rmse\n22542.396440343684\n>>> display_scores(forest_rmse_scores)\nScores: [ 53789.2879722 50256.19806622 52521.55342602 53237.44937943\n 52428.82176158 55854.61222549 52158.02291609 50093.66125649\n 53240.80406125 52761.50852822]\nMean: 52634.1919593\nStandard deviation: 1576.20472269\nWow, this is much better: Random Forests look very promising. However, note that\nthe score on the training set is still much lower than on the validation sets, meaning\nthat the model is still overfitting the training set. Possible solutions for overfitting are\nto simplify the model, constrain it (i.e., regularize it), or get a lot more training data.\nHowever, before you dive much deeper in Random Forests, you should try out many\nother models from various categories of Machine Learning algorithms (several Sup\u2010", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 237, "content": "other models from various categories of Machine Learning algorithms (several Sup\u2010\nport Vector Machines with different kernels, possibly a neural network, etc.), without\nspending too much time tweaking the hyperparameters. The goal is to shortlist a few\n(two to five) promising models.\nYou should save every model you experiment with, so you can\ncome back easily to any model you want. Make sure you save both\nthe hyperparameters and the trained parameters, as well as the\ncross-validation scores and perhaps the actual predictions as well.\nThis will allow you to easily compare scores across model types,\nand compare the types of errors they make. You can easily save\nScikit-Learn models by using Python\u2019s pickle module, or using\nsklearn.externals.joblib, which is more efficient at serializing \nlarge NumPy arrays:\nfrom sklearn.externals import joblib\njoblib.dump(my_model, \"my_model.pkl\")\n# and later...\nmy_model_loaded = joblib.load(\"my_model.pkl\")\nFine-Tune Your Model", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 238, "content": "# and later...\nmy_model_loaded = joblib.load(\"my_model.pkl\")\nFine-Tune Your Model\nLet\u2019s assume that you now have a shortlist of promising models. You now need to\nfine-tune them. Let\u2019s look at a few ways you can do that.\nFine-Tune Your Model \n| \n71\nDownload from finelybook www.finelybook.com", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 239, "content": "Grid Search\nOne way to do that would be to fiddle with the hyperparameters manually, until you\nfind a great combination of hyperparameter values. This would be very tedious work,\nand you may not have time to explore many combinations.\nInstead you should get Scikit-Learn\u2019s GridSearchCV to search for you. All you need to\ndo is tell it which hyperparameters you want it to experiment with, and what values to\ntry out, and it will evaluate all the possible combinations of hyperparameter values,\nusing cross-validation. For example, the following code searches for the best combi\u2010\nnation of hyperparameter values for the RandomForestRegressor:\nfrom sklearn.model_selection import GridSearchCV\nparam_grid = [\n {'n_estimators': [3, 10, 30], 'max_features': [2, 4, 6, 8]},\n {'bootstrap': [False], 'n_estimators': [3, 10], 'max_features': [2, 3, 4]},\n ]\nforest_reg = RandomForestRegressor()\ngrid_search = GridSearchCV(forest_reg, param_grid, cv=5,", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 240, "content": "]\nforest_reg = RandomForestRegressor()\ngrid_search = GridSearchCV(forest_reg, param_grid, cv=5,\n scoring='neg_mean_squared_error')\ngrid_search.fit(housing_prepared, housing_labels)\nWhen you have no idea what value a hyperparameter should have,\na simple approach is to try out consecutive powers of 10 (or a\nsmaller number if you want a more fine-grained search, as shown\nin this example with the n_estimators hyperparameter).\nThis param_grid tells Scikit-Learn to first evaluate all 3 \u00d7 4 = 12 combinations of\nn_estimators and max_features hyperparameter values specified in the first dict\n(don\u2019t worry about what these hyperparameters mean for now; they will be explained\nin Chapter 7), then try all 2 \u00d7 3 = 6 combinations of hyperparameter values in the\nsecond dict, but this time with the bootstrap hyperparameter set to False instead of\nTrue (which is the default value for this hyperparameter).", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 241, "content": "True (which is the default value for this hyperparameter).\nAll in all, the grid search will explore 12 + 6 = 18 combinations of RandomForestRe\ngressor hyperparameter values, and it will train each model five times (since we are\nusing five-fold cross validation). In other words, all in all, there will be 18 \u00d7 5 = 90\nrounds of training! It may take quite a long time, but when it is done you can get the\nbest combination of parameters like this:\n>>> grid_search.best_params_\n{'max_features': 6, 'n_estimators': 30}\n72 \n| \nChapter 2: End-to-End Machine Learning Project\nDownload from finelybook www.finelybook.com", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 242, "content": "Since 30 is the maximum value of n_estimators that was evalu\u2010\nated, you should probably evaluate higher values as well, since the\nscore may continue to improve.\nYou can also get the best estimator directly:\n>>> grid_search.best_estimator_\nRandomForestRegressor(bootstrap=True, criterion='mse', max_depth=None,\n max_features=6, max_leaf_nodes=None, min_samples_leaf=1,\n min_samples_split=2, min_weight_fraction_leaf=0.0,\n n_estimators=30, n_jobs=1, oob_score=False, random_state=None,\n verbose=0, warm_start=False)\nIf GridSearchCV is initialized with refit=True (which is the\ndefault), then once it finds the best estimator using cross-\nvalidation, it retrains it on the whole training set. This is usually a\ngood idea since feeding it more data will likely improve its perfor\u2010\nmance.\nAnd of course the evaluation scores are also available:\n>>> cvres = grid_search.cv_results_\n... for mean_score, params in zip(cvres[\"mean_test_score\"], cvres[\"params\"]):", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 243, "content": "... for mean_score, params in zip(cvres[\"mean_test_score\"], cvres[\"params\"]):\n... print(np.sqrt(-mean_score), params)\n...\n64912.0351358 {'max_features': 2, 'n_estimators': 3}\n55535.2786524 {'max_features': 2, 'n_estimators': 10}\n52940.2696165 {'max_features': 2, 'n_estimators': 30}\n60384.0908354 {'max_features': 4, 'n_estimators': 3}\n52709.9199934 {'max_features': 4, 'n_estimators': 10}\n50503.5985321 {'max_features': 4, 'n_estimators': 30}\n59058.1153485 {'max_features': 6, 'n_estimators': 3}\n52172.0292957 {'max_features': 6, 'n_estimators': 10}\n49958.9555932 {'max_features': 6, 'n_estimators': 30}\n59122.260006 {'max_features': 8, 'n_estimators': 3}\n52441.5896087 {'max_features': 8, 'n_estimators': 10}\n50041.4899416 {'max_features': 8, 'n_estimators': 30}\n62371.1221202 {'bootstrap': False, 'max_features': 2, 'n_estimators': 3}\n54572.2557534 {'bootstrap': False, 'max_features': 2, 'n_estimators': 10}\n59634.0533132 {'bootstrap': False, 'max_features': 3, 'n_estimators': 3}", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 244, "content": "59634.0533132 {'bootstrap': False, 'max_features': 3, 'n_estimators': 3}\n52456.0883904 {'bootstrap': False, 'max_features': 3, 'n_estimators': 10}\n58825.665239 {'bootstrap': False, 'max_features': 4, 'n_estimators': 3}\n52012.9945396 {'bootstrap': False, 'max_features': 4, 'n_estimators': 10}\nIn this example, we obtain the best solution by setting the max_features hyperpara\u2010\nmeter to 6, and the n_estimators hyperparameter to 30. The RMSE score for this\ncombination is 49,959, which is slightly better than the score you got earlier using the\nFine-Tune Your Model \n| \n73\nDownload from finelybook www.finelybook.com", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 245, "content": "default hyperparameter values (which was 52,634). Congratulations, you have suc\u2010\ncessfully fine-tuned your best model!\nDon\u2019t forget that you can treat some of the data preparation steps as\nhyperparameters. For example, the grid search will automatically\nfind out whether or not to add a feature you were not sure about\n(e.g., using the add_bedrooms_per_room hyperparameter of your\nCombinedAttributesAdder transformer). It may similarly be used\nto automatically find the best way to handle outliers, missing fea\u2010\ntures, feature selection, and more.\nRandomized Search\nThe grid search approach is fine when you are exploring relatively few combinations,\nlike in the previous example, but when the hyperparameter search space is large, it is\noften preferable to use RandomizedSearchCV instead. This class can be used in much\nthe same way as the GridSearchCV class, but instead of trying out all possible combi\u2010\nnations, it evaluates a given number of random combinations by selecting a random", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 246, "content": "nations, it evaluates a given number of random combinations by selecting a random\nvalue for each hyperparameter at every iteration. This approach has two main bene\u2010\nfits:\n\u2022 If you let the randomized search run for, say, 1,000 iterations, this approach will\nexplore 1,000 different values for each hyperparameter (instead of just a few val\u2010\nues per hyperparameter with the grid search approach).\n\u2022 You have more control over the computing budget you want to allocate to hyper\u2010\nparameter search, simply by setting the number of iterations.\nEnsemble Methods\nAnother way to fine-tune your system is to try to combine the models that perform\nbest. The group (or \u201censemble\u201d) will often perform better than the best individual\nmodel (just like Random Forests perform better than the individual Decision Trees\nthey rely on), especially if the individual models make very different types of errors.\nWe will cover this topic in more detail in Chapter 7.\nAnalyze the Best Models and Their Errors", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 247, "content": "We will cover this topic in more detail in Chapter 7.\nAnalyze the Best Models and Their Errors\nYou will often gain good insights on the problem by inspecting the best models. For\nexample, the RandomForestRegressor can indicate the relative importance of each\nattribute for making accurate predictions:\n>>> feature_importances = grid_search.best_estimator_.feature_importances_\n>>> feature_importances\narray([ 7.14156423e-02, 6.76139189e-02, 4.44260894e-02,\n74 \n| \nChapter 2: End-to-End Machine Learning Project\nDownload from finelybook www.finelybook.com", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 248, "content": "1.66308583e-02, 1.66076861e-02, 1.82402545e-02,\n 1.63458761e-02, 3.26497987e-01, 6.04365775e-02,\n 1.13055290e-01, 7.79324766e-02, 1.12166442e-02,\n 1.53344918e-01, 8.41308969e-05, 2.68483884e-03,\n 3.46681181e-03])\nLet\u2019s display these importance scores next to their corresponding attribute names:\n>>> extra_attribs = [\"rooms_per_hhold\", \"pop_per_hhold\", \"bedrooms_per_room\"]\n>>> cat_one_hot_attribs = list(encoder.classes_)\n>>> attributes = num_attribs + extra_attribs + cat_one_hot_attribs\n>>> sorted(zip(feature_importances, attributes), reverse=True)\n[(0.32649798665134971, 'median_income'),\n (0.15334491760305854, 'INLAND'),\n (0.11305529021187399, 'pop_per_hhold'),\n (0.07793247662544775, 'bedrooms_per_room'),\n (0.071415642259275158, 'longitude'),\n (0.067613918945568688, 'latitude'),\n (0.060436577499703222, 'rooms_per_hhold'),\n (0.04442608939578685, 'housing_median_age'),\n (0.018240254462909437, 'population'),", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 249, "content": "(0.04442608939578685, 'housing_median_age'),\n (0.018240254462909437, 'population'),\n (0.01663085833886218, 'total_rooms'),\n (0.016607686091288865, 'total_bedrooms'),\n (0.016345876147580776, 'households'),\n (0.011216644219017424, '<1H OCEAN'),\n (0.0034668118081117387, 'NEAR OCEAN'),\n (0.0026848388432755429, 'NEAR BAY'),\n (8.4130896890070617e-05, 'ISLAND')]\nWith this information, you may want to try dropping some of the less useful features\n(e.g., apparently only one ocean_proximity category is really useful, so you could try\ndropping the others).\nYou should also look at the specific errors that your system makes, then try to under\u2010\nstand why it makes them and what could fix the problem (adding extra features or, on\nthe contrary, getting rid of uninformative ones, cleaning up outliers, etc.).\nEvaluate Your System on the Test Set\nAfter tweaking your models for a while, you eventually have a system that performs", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 250, "content": "After tweaking your models for a while, you eventually have a system that performs\nsufficiently well. Now is the time to evaluate the final model on the test set. There is\nnothing special about this process; just get the predictors and the labels from your\ntest set, run your full_pipeline to transform the data (call transform(), not\nfit_transform()!), and evaluate the final model on the test set:\nfinal_model = grid_search.best_estimator_\nX_test = strat_test_set.drop(\"median_house_value\", axis=1)\ny_test = strat_test_set[\"median_house_value\"].copy()\nX_test_prepared = full_pipeline.transform(X_test)\nFine-Tune Your Model \n| \n75\nDownload from finelybook www.finelybook.com", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 251, "content": "final_predictions = final_model.predict(X_test_prepared)\nfinal_mse = mean_squared_error(y_test, final_predictions)\nfinal_rmse = np.sqrt(final_mse) # => evaluates to 48,209.6\nThe performance will usually be slightly worse than what you measured using cross-\nvalidation if you did a lot of hyperparameter tuning (because your system ends up\nfine-tuned to perform well on the validation data, and will likely not perform as well\non unknown datasets). It is not the case in this example, but when this happens you\nmust resist the temptation to tweak the hyperparameters to make the numbers look\ngood on the test set; the improvements would be unlikely to generalize to new data.\nNow comes the project prelaunch phase: you need to present your solution (high\u2010\nlighting what you have learned, what worked and what did not, what assumptions\nwere made, and what your system\u2019s limitations are), document everything, and create", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 252, "content": "were made, and what your system\u2019s limitations are), document everything, and create\nnice presentations with clear visualizations and easy-to-remember statements (e.g.,\n\u201cthe median income is the number one predictor of housing prices\u201d).\nLaunch, Monitor, and Maintain Your System\nPerfect, you got approval to launch! You need to get your solution ready for produc\u2010\ntion, in particular by plugging the production input data sources into your system\nand writing tests.\nYou also need to write monitoring code to check your system\u2019s live performance at\nregular intervals and trigger alerts when it drops. This is important to catch not only\nsudden breakage, but also performance degradation. This is quite common because\nmodels tend to \u201crot\u201d as data evolves over time, unless the models are regularly trained\non fresh data.\nEvaluating your system\u2019s performance will require sampling the system\u2019s predictions\nand evaluating them. This will generally require a human analysis. These analysts", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 253, "content": "and evaluating them. This will generally require a human analysis. These analysts\nmay be field experts, or workers on a crowdsourcing platform (such as Amazon\nMechanical Turk or CrowdFlower). Either way, you need to plug the human evalua\u2010\ntion pipeline into your system.\nYou should also make sure you evaluate the system\u2019s input data quality. Sometimes\nperformance will degrade slightly because of a poor quality signal (e.g., a malfunc\u2010\ntioning sensor sending random values, or another team\u2019s output becoming stale), but\nit may take a while before your system\u2019s performance degrades enough to trigger an\nalert. If you monitor your system\u2019s inputs, you may catch this earlier. Monitoring the\ninputs is particularly important for online learning systems.\nFinally, you will generally want to train your models on a regular basis using fresh\ndata. You should automate this process as much as possible. If you don\u2019t, you are very\n76 \n| \nChapter 2: End-to-End Machine Learning Project", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 254, "content": "76 \n| \nChapter 2: End-to-End Machine Learning Project\nDownload from finelybook www.finelybook.com", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 255, "content": "likely to refresh your model only every six months (at best), and your system\u2019s perfor\u2010\nmance may fluctuate severely over time. If your system is an online learning system,\nyou should make sure you save snapshots of its state at regular intervals so you can\neasily roll back to a previously working state.\nTry It Out!\nHopefully this chapter gave you a good idea of what a Machine Learning project\nlooks like, and showed you some of the tools you can use to train a great system. As\nyou can see, much of the work is in the data preparation step, building monitoring\ntools, setting up human evaluation pipelines, and automating regular model training.\nThe Machine Learning algorithms are also important, of course, but it is probably\npreferable to be comfortable with the overall process and know three or four algo\u2010\nrithms well rather than to spend all your time exploring advanced algorithms and not\nenough time on the overall process.", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 256, "content": "enough time on the overall process.\nSo, if you have not already done so, now is a good time to pick up a laptop, select a\ndataset that you are interested in, and try to go through the whole process from A to\nZ. A good place to start is on a competition website such as http://kaggle.com/: you\nwill have a dataset to play with, a clear goal, and people to share the experience with.\nExercises\nUsing this chapter\u2019s housing dataset:\n1. Try a Support Vector Machine regressor (sklearn.svm.SVR), with various hyper\u2010\nparameters such as kernel=\"linear\" (with various values for the C hyperpara\u2010\nmeter) or kernel=\"rbf\" (with various values for the C and gamma\nhyperparameters). Don\u2019t worry about what these hyperparameters mean for now.\nHow does the best SVR predictor perform?\n2. Try replacing GridSearchCV with RandomizedSearchCV.\n3. Try adding a transformer in the preparation pipeline to select only the most\nimportant attributes.", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 257, "content": "important attributes.\n4. Try creating a single pipeline that does the full data preparation plus the final\nprediction.\n5. Automatically explore some preparation options using GridSearchCV.\nSolutions to these exercises are available in the online Jupyter notebooks at https://\ngithub.com/ageron/handson-ml.\nTry It Out! \n| \n77\nDownload from finelybook www.finelybook.com", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 258, "content": "Download from finelybook www.finelybook.com", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 259, "content": "1 By default Scikit-Learn caches downloaded datasets in a directory called $HOME/scikit_learn_data.\nCHAPTER 3\nClassification\nIn Chapter 1 we mentioned that the most common supervised learning tasks are\nregression (predicting values) and classification (predicting classes). In Chapter 2 we\nexplored a regression task, predicting housing values, using various algorithms such\nas Linear Regression, Decision Trees, and Random Forests (which will be explained\nin further detail in later chapters). Now we will turn our attention to classification\nsystems.\nMNIST\nIn this chapter, we will be using the MNIST dataset, which is a set of 70,000 small\nimages of digits handwritten by high school students and employees of the US Cen\u2010\nsus Bureau. Each image is labeled with the digit it represents. This set has been stud\u2010\nied so much that it is often called the \u201cHello World\u201d of Machine Learning: whenever\npeople come up with a new classification algorithm, they are curious to see how it", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 260, "content": "people come up with a new classification algorithm, they are curious to see how it\nwill perform on MNIST. Whenever someone learns Machine Learning, sooner or\nlater they tackle MNIST.\nScikit-Learn provides many helper functions to download popular datasets. MNIST is\none of them. The following code fetches the MNIST dataset:1\n>>> from sklearn.datasets import fetch_mldata\n>>> mnist = fetch_mldata('MNIST original')\n>>> mnist\n{'COL_NAMES': ['label', 'data'],\n 'DESCR': 'mldata.org dataset: mnist-original',\n 'data': array([[0, 0, 0, ..., 0, 0, 0],\n [0, 0, 0, ..., 0, 0, 0],\n79\nDownload from finelybook www.finelybook.com", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 261, "content": "[0, 0, 0, ..., 0, 0, 0],\n ...,\n [0, 0, 0, ..., 0, 0, 0],\n [0, 0, 0, ..., 0, 0, 0],\n [0, 0, 0, ..., 0, 0, 0]], dtype=uint8),\n 'target': array([ 0., 0., 0., ..., 9., 9., 9.])}\nDatasets loaded by Scikit-Learn generally have a similar dictionary structure includ\u2010\ning:\n\u2022 A DESCR key describing the dataset\n\u2022 A data key containing an array with one row per instance and one column per\nfeature\n\u2022 A target key containing an array with the labels\nLet\u2019s look at these arrays:\n>>> X, y = mnist[\"data\"], mnist[\"target\"]\n>>> X.shape\n(70000, 784)\n>>> y.shape\n(70000,)\nThere are 70,000 images, and each image has 784 features. This is because each image\nis 28\u00d728 pixels, and each feature simply represents one pixel\u2019s intensity, from 0\n(white) to 255 (black). Let\u2019s take a peek at one digit from the dataset. All you need to\ndo is grab an instance\u2019s feature vector, reshape it to a 28\u00d728 array, and display it using\nMatplotlib\u2019s imshow() function:\n%matplotlib inline", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 262, "content": "Matplotlib\u2019s imshow() function:\n%matplotlib inline\nimport matplotlib\nimport matplotlib.pyplot as plt\nsome_digit = X[36000]\nsome_digit_image = some_digit.reshape(28, 28)\nplt.imshow(some_digit_image, cmap = matplotlib.cm.binary,\n interpolation=\"nearest\")\nplt.axis(\"off\")\nplt.show()\nThis looks like a 5, and indeed that\u2019s what the label tells us:\n80 \n| \nChapter 3: Classification\nDownload from finelybook www.finelybook.com", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 263, "content": "2 Shuffling may be a bad idea in some contexts\u2014for example, if you are working on time series data (such as\nstock market prices or weather conditions). We will explore this in the next chapters.\n>>> y[36000]\n5.0\nFigure 3-1 shows a few more images from the MNIST dataset to give you a feel for\nthe complexity of the classification task.\nFigure 3-1. A few digits from the MNIST dataset\nBut wait! You should always create a test set and set it aside before inspecting the data\nclosely. The MNIST dataset is actually already split into a training set (the first 60,000\nimages) and a test set (the last 10,000 images):\nX_train, X_test, y_train, y_test = X[:60000], X[60000:], y[:60000], y[60000:]\nLet\u2019s also shuffle the training set; this will guarantee that all cross-validation folds will\nbe similar (you don\u2019t want one fold to be missing some digits). Moreover, some learn\u2010\ning algorithms are sensitive to the order of the training instances, and they perform", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 264, "content": "ing algorithms are sensitive to the order of the training instances, and they perform\npoorly if they get many similar instances in a row. Shuffling the dataset ensures that\nthis won\u2019t happen:2\nMNIST \n| \n81\nDownload from finelybook www.finelybook.com", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 265, "content": "import numpy as np\nshuffle_index = np.random.permutation(60000)\nX_train, y_train = X_train[shuffle_index], y_train[shuffle_index]\nTraining a Binary Classifier\nLet\u2019s simplify the problem for now and only try to identify one digit\u2014for example,\nthe number 5. This \u201c5-detector\u201d will be an example of a binary classifier, capable of\ndistinguishing between just two classes, 5 and not-5. Let\u2019s create the target vectors for\nthis classification task:\ny_train_5 = (y_train == 5) # True for all 5s, False for all other digits.\ny_test_5 = (y_test == 5)\nOkay, now let\u2019s pick a classifier and train it. A good place to start is with a Stochastic\nGradient Descent (SGD) classifier, using Scikit-Learn\u2019s SGDClassifier class. This clas\u2010\nsifier has the advantage of being capable of handling very large datasets efficiently.\nThis is in part because SGD deals with training instances independently, one at a time\n(which also makes SGD well suited for online learning), as we will see later. Let\u2019s create", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 266, "content": "(which also makes SGD well suited for online learning), as we will see later. Let\u2019s create\nan SGDClassifier and train it on the whole training set:\nfrom sklearn.linear_model import SGDClassifier\nsgd_clf = SGDClassifier(random_state=42)\nsgd_clf.fit(X_train, y_train_5)\nThe SGDClassifier relies on randomness during training (hence\nthe name \u201cstochastic\u201d). If you want reproducible results, you\nshould set the random_state parameter.\nNow you can use it to detect images of the number 5:\n>>> sgd_clf.predict([some_digit])\narray([ True], dtype=bool)\nThe classifier guesses that this image represents a 5 (True). Looks like it guessed right\nin this particular case! Now, let\u2019s evaluate this model\u2019s performance.\nPerformance Measures\nEvaluating a classifier is often significantly trickier than evaluating a regressor, so we\nwill spend a large part of this chapter on this topic. There are many performance\nmeasures available, so grab another coffee and get ready to learn many new concepts\nand acronyms!", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 267, "content": "measures available, so grab another coffee and get ready to learn many new concepts\nand acronyms!\n82 \n| \nChapter 3: Classification\nDownload from finelybook www.finelybook.com", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 268, "content": "Measuring Accuracy Using Cross-Validation\nA good way to evaluate a model is to use cross-validation, just as you did in Chap\u2010\nter 2.\nImplementing Cross-Validation\nOccasionally you will need more control over the cross-validation process than what\ncross_val_score() and similar functions provide. In these cases, you can implement\ncross-validation yourself; it is actually fairly straightforward. The following code does\nroughly the same thing as the preceding cross_val_score() code, and prints the \nsame result:\nfrom sklearn.model_selection import StratifiedKFold\nfrom sklearn.base import clone\nskfolds = StratifiedKFold(n_splits=3, random_state=42)\nfor train_index, test_index in skfolds.split(X_train, y_train_5):\n clone_clf = clone(sgd_clf)\n X_train_folds = X_train[train_index]\n y_train_folds = (y_train_5[train_index])\n X_test_fold = X_train[test_index]\n y_test_fold = (y_train_5[test_index])\n clone_clf.fit(X_train_folds, y_train_folds)", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 269, "content": "y_test_fold = (y_train_5[test_index])\n clone_clf.fit(X_train_folds, y_train_folds)\n y_pred = clone_clf.predict(X_test_fold)\n n_correct = sum(y_pred == y_test_fold)\n print(n_correct / len(y_pred)) # prints 0.9502, 0.96565 and 0.96495\nThe StratifiedKFold class performs stratified sampling (as explained in Chapter 2)\nto produce folds that contain a representative ratio of each class. At each iteration the\ncode creates a clone of the classifier, trains that clone on the training folds, and makes\npredictions on the test fold. Then it counts the number of correct predictions and\noutputs the ratio of correct predictions.\nLet\u2019s use the cross_val_score() function to evaluate your SGDClassifier model\nusing K-fold cross-validation, with three folds. Remember that K-fold cross-\nvalidation means splitting the training set into K-folds (in this case, three), then mak\u2010\ning predictions and evaluating them on each fold using a model trained on the\nremaining folds (see Chapter 2):", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 270, "content": "remaining folds (see Chapter 2):\n>>> from sklearn.model_selection import cross_val_score\n>>> cross_val_score(sgd_clf, X_train, y_train_5, cv=3, scoring=\"accuracy\")\narray([ 0.9502 , 0.96565, 0.96495])\nPerformance Measures \n| \n83\nDownload from finelybook www.finelybook.com", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 271, "content": "Wow! Above 95% accuracy (ratio of correct predictions) on all cross-validation folds? \nThis looks amazing, doesn\u2019t it? Well, before you get too excited, let\u2019s look at a very\ndumb classifier that just classifies every single image in the \u201cnot-5\u201d class:\nfrom sklearn.base import BaseEstimator\nclass Never5Classifier(BaseEstimator):\n def fit(self, X, y=None):\n pass\n def predict(self, X):\n return np.zeros((len(X), 1), dtype=bool)\nCan you guess this model\u2019s accuracy? Let\u2019s find out:\n>>> never_5_clf = Never5Classifier()\n>>> cross_val_score(never_5_clf, X_train, y_train_5, cv=3, scoring=\"accuracy\")\narray([ 0.909 , 0.90715, 0.9128 ])\nThat\u2019s right, it has over 90% accuracy! This is simply because only about 10% of the\nimages are 5s, so if you always guess that an image is not a 5, you will be right about\n90% of the time. Beats Nostradamus.\nThis demonstrates why accuracy is generally not the preferred performance measure", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 272, "content": "This demonstrates why accuracy is generally not the preferred performance measure\nfor classifiers, especially when you are dealing with skewed datasets (i.e., when some\nclasses are much more frequent than others).\nConfusion Matrix\nA much better way to evaluate the performance of a classifier is to look at the confu\u2010\nsion matrix. The general idea is to count the number of times instances of class A are\nclassified as class B. For example, to know the number of times the classifier confused\nimages of 5s with 3s, you would look in the 5th row and 3rd column of the confusion\nmatrix.\nTo compute the confusion matrix, you first need to have a set of predictions, so they\ncan be compared to the actual targets. You could make predictions on the test set, but\nlet\u2019s keep it untouched for now (remember that you want to use the test set only at the\nvery end of your project, once you have a classifier that you are ready to launch).\nInstead, you can use the cross_val_predict() function:", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 273, "content": "Instead, you can use the cross_val_predict() function:\nfrom sklearn.model_selection import cross_val_predict\ny_train_pred = cross_val_predict(sgd_clf, X_train, y_train_5, cv=3)\nJust like the cross_val_score() function, cross_val_predict() performs K-fold\ncross-validation, but instead of returning the evaluation scores, it returns the predic\u2010\ntions made on each test fold. This means that you get a clean prediction for each\ninstance in the training set (\u201cclean\u201d meaning that the prediction is made by a model\nthat never saw the data during training).\n84 \n| \nChapter 3: Classification\nDownload from finelybook www.finelybook.com", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 274, "content": "Now you are ready to get the confusion matrix using the confusion_matrix() func\u2010\ntion. Just pass it the target classes (y_train_5) and the predicted classes\n(y_train_pred):\n>>> from sklearn.metrics import confusion_matrix\n>>> confusion_matrix(y_train_5, y_train_pred)\narray([[53272, 1307],\n [ 1077, 4344]])\nEach row in a confusion matrix represents an actual class, while each column repre\u2010\nsents a predicted class. The first row of this matrix considers non-5 images (the nega\u2010\ntive class): 53,272 of them were correctly classified as non-5s (they are called true\nnegatives), while the remaining 1,307 were wrongly classified as 5s (false positives).\nThe second row considers the images of 5s (the positive class): 1,077 were wrongly\nclassified as non-5s (false negatives), while the remaining 4,344 were correctly classi\u2010\nfied as 5s (true positives). A perfect classifier would have only true positives and true", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 275, "content": "fied as 5s (true positives). A perfect classifier would have only true positives and true\nnegatives, so its confusion matrix would have nonzero values only on its main diago\u2010\nnal (top left to bottom right):\n>>> confusion_matrix(y_train_5, y_train_perfect_predictions)\narray([[54579, 0],\n [ 0, 5421]])\nThe confusion matrix gives you a lot of information, but sometimes you may prefer a\nmore concise metric. An interesting one to look at is the accuracy of the positive pre\u2010\ndictions; this is called the precision of the classifier (Equation 3-1).\nEquation 3-1. Precision\nprecision =\nTP\nTP + FP\nTP is the number of true positives, and FP is the number of false positives.\nA trivial way to have perfect precision is to make one single positive prediction and\nensure it is correct (precision = 1/1 = 100%). This would not be very useful since the\nclassifier would ignore all but one positive instance. So precision is typically used", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 276, "content": "classifier would ignore all but one positive instance. So precision is typically used\nalong with another metric named recall, also called sensitivity or true positive rate\n(TPR): this is the ratio of positive instances that are correctly detected by the classifier\n(Equation 3-2).\nEquation 3-2. Recall\nrecall =\nTP\nTP + FN\nFN is of course the number of false negatives.\nPerformance Measures \n| \n85\nDownload from finelybook www.finelybook.com", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 277, "content": "If you are confused about the confusion matrix, Figure 3-2 may help.\nFigure 3-2. An illustrated confusion matrix\nPrecision and Recall\nScikit-Learn provides several functions to compute classifier metrics, including preci\u2010\nsion and recall:\n>>> from sklearn.metrics import precision_score, recall_score\n>>> precision_score(y_train_5, y_pred) # == 4344 / (4344 + 1307)\n0.76871350203503808\n>>> recall_score(y_train_5, y_train_pred) # == 4344 / (4344 + 1077)\n0.79136690647482011\nNow your 5-detector does not look as shiny as it did when you looked at its accuracy.\nWhen it claims an image represents a 5, it is correct only 77% of the time. Moreover,\nit only detects 79% of the 5s.\nIt is often convenient to combine precision and recall into a single metric called the F1\nscore, in particular if you need a simple way to compare two classifiers. The F1 score is \nthe harmonic mean of precision and recall (Equation 3-3). Whereas the regular mean", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 278, "content": "the harmonic mean of precision and recall (Equation 3-3). Whereas the regular mean\ntreats all values equally, the harmonic mean gives much more weight to low values.\nAs a result, the classifier will only get a high F1 score if both recall and precision are\nhigh.\nEquation 3-3. F1 score\nF1 =\n2\n1\nprecision +\n1\nrecall\n= 2 \u00d7 precision \u00d7 recall\nprecision + recall =\nTP\nTP + FN + FP\n2\n86 \n| \nChapter 3: Classification\nDownload from finelybook www.finelybook.com", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 279, "content": "To compute the F1 score, simply call the f1_score() function:\n>>> from sklearn.metrics import f1_score\n>>> f1_score(y_train_5, y_pred)\n0.78468208092485547\nThe F1 score favors classifiers that have similar precision and recall. This is not always\nwhat you want: in some contexts you mostly care about precision, and in other con\u2010\ntexts you really care about recall. For example, if you trained a classifier to detect vid\u2010\neos that are safe for kids, you would probably prefer a classifier that rejects many\ngood videos (low recall) but keeps only safe ones (high precision), rather than a clas\u2010\nsifier that has a much higher recall but lets a few really bad videos show up in your\nproduct (in such cases, you may even want to add a human pipeline to check the clas\u2010\nsifier\u2019s video selection). On the other hand, suppose you train a classifier to detect\nshoplifters on surveillance images: it is probably fine if your classifier has only 30%", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 280, "content": "shoplifters on surveillance images: it is probably fine if your classifier has only 30%\nprecision as long as it has 99% recall (sure, the security guards will get a few false\nalerts, but almost all shoplifters will get caught).\nUnfortunately, you can\u2019t have it both ways: increasing precision reduces recall, and\nvice versa. This is called the precision/recall tradeoff.\nPrecision/Recall Tradeoff\nTo understand this tradeoff, let\u2019s look at how the SGDClassifier makes its classifica\u2010\ntion decisions. For each instance, it computes a score based on a decision function, \nand if that score is greater than a threshold, it assigns the instance to the positive\nclass, or else it assigns it to the negative class. Figure 3-3 shows a few digits positioned\nfrom the lowest score on the left to the highest score on the right. Suppose the deci\u2010\nsion threshold is positioned at the central arrow (between the two 5s): you will find 4", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 281, "content": "sion threshold is positioned at the central arrow (between the two 5s): you will find 4\ntrue positives (actual 5s) on the right of that threshold, and one false positive (actually\na 6). Therefore, with that threshold, the precision is 80% (4 out of 5). But out of 6\nactual 5s, the classifier only detects 4, so the recall is 67% (4 out of 6). Now if you\nraise the threshold (move it to the arrow on the right), the false positive (the 6)\nbecomes a true negative, thereby increasing precision (up to 100% in this case), but\none true positive becomes a false negative, decreasing recall down to 50%. Conversely,\nlowering the threshold increases recall and reduces precision.\nPerformance Measures \n| \n87\nDownload from finelybook www.finelybook.com", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 282, "content": "Figure 3-3. Decision threshold and precision/recall tradeoff\nScikit-Learn does not let you set the threshold directly, but it does give you access to\nthe decision scores that it uses to make predictions. Instead of calling the classifier\u2019s\npredict() method, you can call its decision_function() method, which returns a\nscore for each instance, and then make predictions based on those scores using any\nthreshold you want:\n>>> y_scores = sgd_clf.decision_function([some_digit])\n>>> y_scores\narray([ 161855.74572176])\n>>> threshold = 0\n>>> y_some_digit_pred = (y_scores > threshold)\narray([ True], dtype=bool)\nThe SGDClassifier uses a threshold equal to 0, so the previous code returns the same\nresult as the predict() method (i.e., True). Let\u2019s raise the threshold:\n>>> threshold = 200000\n>>> y_some_digit_pred = (y_scores > threshold)\n>>> y_some_digit_pred\narray([False], dtype=bool)\nThis confirms that raising the threshold decreases recall. The image actually repre\u2010", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 283, "content": "This confirms that raising the threshold decreases recall. The image actually repre\u2010\nsents a 5, and the classifier detects it when the threshold is 0, but it misses it when the\nthreshold is increased to 200,000.\nSo how can you decide which threshold to use? For this you will first need to get the\nscores of all instances in the training set using the cross_val_predict() function\nagain, but this time specifying that you want it to return decision scores instead of\npredictions:\ny_scores = cross_val_predict(sgd_clf, X_train, y_train_5, cv=3,\n method=\"decision_function\")\nNow with these scores you can compute precision and recall for all possible thresh\u2010\nolds using the precision_recall_curve() function:\n88 \n| \nChapter 3: Classification\nDownload from finelybook www.finelybook.com", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 284, "content": "from sklearn.metrics import precision_recall_curve\nprecisions, recalls, thresholds = precision_recall_curve(y_train_5, y_scores)\nFinally, you can plot precision and recall as functions of the threshold value using\nMatplotlib (Figure 3-4):\ndef plot_precision_recall_vs_threshold(precisions, recalls, thresholds):\n plt.plot(thresholds, precisions[:-1], \"b--\", label=\"Precision\")\n plt.plot(thresholds, recalls[:-1], \"g-\", label=\"Recall\")\n plt.xlabel(\"Threshold\")\n plt.legend(loc=\"upper left\")\n plt.ylim([0, 1])\nplot_precision_recall_vs_threshold(precisions, recalls, thresholds)\nplt.show()\nFigure 3-4. Precision and recall versus the decision threshold\nYou may wonder why the precision curve is bumpier than the recall\ncurve in Figure 3-4. The reason is that precision may sometimes go\ndown when you raise the threshold (although in general it will go\nup). To understand why, look back at Figure 3-3 and notice what\nhappens when you start from the central threshold and move it just", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 285, "content": "happens when you start from the central threshold and move it just\none digit to the right: precision goes from 4/5 (80%) down to 3/4\n(75%). On the other hand, recall can only go down when the thres\u2010\nhold is increased, which explains why its curve looks smooth.\nNow you can simply select the threshold value that gives you the best precision/recall\ntradeoff for your task. Another way to select a good precision/recall tradeoff is to plot\nprecision directly against recall, as shown in Figure 3-5.\nPerformance Measures \n| \n89\nDownload from finelybook www.finelybook.com", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 286, "content": "Figure 3-5. Precision versus recall\nYou can see that precision really starts to fall sharply around 80% recall. You will\nprobably want to select a precision/recall tradeoff just before that drop\u2014for example,\nat around 60% recall. But of course the choice depends on your project.\nSo let\u2019s suppose you decide to aim for 90% precision. You look up the first plot\n(zooming in a bit) and find that you need to use a threshold of about 70,000. To make\npredictions (on the training set for now), instead of calling the classifier\u2019s predict()\nmethod, you can just run this code:\ny_train_pred_90 = (y_scores > 70000)\nLet\u2019s check these predictions\u2019 precision and recall:\n>>> precision_score(y_train_5, y_train_pred_90)\n0.8998702983138781\n>>> recall_score(y_train_5, y_train_pred_90)\n0.63991883416343853\nGreat, you have a 90% precision classifier (or close enough)! As you can see, it is\nfairly easy to create a classifier with virtually any precision you want: just set a high", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 287, "content": "fairly easy to create a classifier with virtually any precision you want: just set a high\nenough threshold, and you\u2019re done. Hmm, not so fast. A high-precision classifier is\nnot very useful if its recall is too low!\nIf someone says \u201clet\u2019s reach 99% precision,\u201d you should ask, \u201cat\nwhat recall?\u201d\n90 \n| \nChapter 3: Classification\nDownload from finelybook www.finelybook.com", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 288, "content": "The ROC Curve\nThe receiver operating characteristic (ROC) curve is another common tool used with\nbinary classifiers. It is very similar to the precision/recall curve, but instead of plot\u2010\nting precision versus recall, the ROC curve plots the true positive rate (another name\nfor recall) against the false positive rate. The FPR is the ratio of negative instances that\nare incorrectly classified as positive. It is equal to one minus the true negative rate, \nwhich is the ratio of negative instances that are correctly classified as negative. The\nTNR is also called specificity. Hence the ROC curve plots sensitivity (recall) versus\n1 \u2013 specificity.\nTo plot the ROC curve, you first need to compute the TPR and FPR for various thres\u2010\nhold values, using the roc_curve() function:\nfrom sklearn.metrics import roc_curve\nfpr, tpr, thresholds = roc_curve(y_train_5, y_scores)\nThen you can plot the FPR against the TPR using Matplotlib. This code produces the\nplot in Figure 3-6:", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 289, "content": "plot in Figure 3-6:\ndef plot_roc_curve(fpr, tpr, label=None):\n plt.plot(fpr, tpr, linewidth=2, label=label)\n plt.plot([0, 1], [0, 1], 'k--')\n plt.axis([0, 1, 0, 1])\n plt.xlabel('False Positive Rate')\n plt.ylabel('True Positive Rate')\nplot_roc_curve(fpr, tpr)\nplt.show()\nFigure 3-6. ROC curve\nPerformance Measures \n| \n91\nDownload from finelybook www.finelybook.com", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 290, "content": "Once again there is a tradeoff: the higher the recall (TPR), the more false positives\n(FPR) the classifier produces. The dotted line represents the ROC curve of a purely\nrandom classifier; a good classifier stays as far away from that line as possible (toward\nthe top-left corner).\nOne way to compare classifiers is to measure the area under the curve (AUC). A per\u2010\nfect classifier will have a ROC AUC equal to 1, whereas a purely random classifier will\nhave a ROC AUC equal to 0.5. Scikit-Learn provides a function to compute the ROC\nAUC:\n>>> from sklearn.metrics import roc_auc_score\n>>> roc_auc_score(y_train_5, y_scores)\n0.97061072797174941\nSince the ROC curve is so similar to the precision/recall (or PR)\ncurve, you may wonder how to decide which one to use. As a rule\nof thumb, you should prefer the PR curve whenever the positive\nclass is rare or when you care more about the false positives than\nthe false negatives, and the ROC curve otherwise. For example,", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 291, "content": "the false negatives, and the ROC curve otherwise. For example,\nlooking at the previous ROC curve (and the ROC AUC score), you\nmay think that the classifier is really good. But this is mostly\nbecause there are few positives (5s) compared to the negatives\n(non-5s). In contrast, the PR curve makes it clear that the classifier\nhas room for improvement (the curve could be closer to the top-\nright corner).\nLet\u2019s train a RandomForestClassifier and compare its ROC curve and ROC AUC\nscore to the SGDClassifier. First, you need to get scores for each instance in the\ntraining set. But due to the way it works (see Chapter 7), the RandomForestClassi\nfier class does not have a decision_function() method. Instead it has a pre\ndict_proba() method. Scikit-Learn classifiers generally have one or the other. The\npredict_proba() method returns an array containing a row per instance and a col\u2010\numn per class, each containing the probability that the given instance belongs to the", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 292, "content": "umn per class, each containing the probability that the given instance belongs to the\ngiven class (e.g., 70% chance that the image represents a 5):\nfrom sklearn.ensemble import RandomForestClassifier\nforest_clf = RandomForestClassifier(random_state=42)\ny_probas_forest = cross_val_predict(forest_clf, X_train, y_train_5, cv=3,\n method=\"predict_proba\")\nBut to plot a ROC curve, you need scores, not probabilities. A simple solution is to\nuse the positive class\u2019s probability as the score:\ny_scores_forest = y_probas_forest[:, 1] # score = proba of positive class\nfpr_forest, tpr_forest, thresholds_forest = roc_curve(y_train_5,y_scores_forest)\n92 \n| \nChapter 3: Classification\nDownload from finelybook www.finelybook.com", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 293, "content": "Now you are ready to plot the ROC curve. It is useful to plot the first ROC curve as\nwell to see how they compare (Figure 3-7):\nplt.plot(fpr, tpr, \"b:\", label=\"SGD\")\nplot_roc_curve(fpr_forest, tpr_forest, \"Random Forest\")\nplt.legend(loc=\"bottom right\")\nplt.show()\nFigure 3-7. Comparing ROC curves\nAs you can see in Figure 3-7, the RandomForestClassifier\u2019s ROC curve looks much\nbetter than the SGDClassifier\u2019s: it comes much closer to the top-left corner. As a\nresult, its ROC AUC score is also significantly better:\n>>> roc_auc_score(y_train_5, y_scores_forest)\n0.99312433660038291\nTry measuring the precision and recall scores: you should find 98.5% precision and\n82.8% recall. Not too bad!\nHopefully you now know how to train binary classifiers, choose the appropriate met\u2010\nric for your task, evaluate your classifiers using cross-validation, select the precision/\nrecall tradeoff that fits your needs, and compare various models using ROC curves", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 294, "content": "recall tradeoff that fits your needs, and compare various models using ROC curves\nand ROC AUC scores. Now let\u2019s try to detect more than just the 5s.\nMulticlass Classification\nWhereas binary classifiers distinguish between two classes, multiclass classifiers (also\ncalled multinomial classifiers) can distinguish between more than two classes.\nMulticlass Classification \n| \n93\nDownload from finelybook www.finelybook.com", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 295, "content": "Some algorithms (such as Random Forest classifiers or naive Bayes classifiers) are\ncapable of handling multiple classes directly. Others (such as Support Vector Machine\nclassifiers or Linear classifiers) are strictly binary classifiers. However, there are vari\u2010\nous strategies that you can use to perform multiclass classification using multiple\nbinary classifiers.\nFor example, one way to create a system that can classify the digit images into 10\nclasses (from 0 to 9) is to train 10 binary classifiers, one for each digit (a 0-detector, a\n1-detector, a 2-detector, and so on). Then when you want to classify an image, you get\nthe decision score from each classifier for that image and you select the class whose\nclassifier outputs the highest score. This is called the one-versus-all (OvA) strategy \n(also called one-versus-the-rest).\nAnother strategy is to train a binary classifier for every pair of digits: one to distin\u2010", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 296, "content": "Another strategy is to train a binary classifier for every pair of digits: one to distin\u2010\nguish 0s and 1s, another to distinguish 0s and 2s, another for 1s and 2s, and so on.\nThis is called the one-versus-one (OvO) strategy. If there are N classes, you need to\ntrain N \u00d7 (N \u2013 1) / 2 classifiers. For the MNIST problem, this means training 45\nbinary classifiers! When you want to classify an image, you have to run the image\nthrough all 45 classifiers and see which class wins the most duels. The main advan\u2010\ntage of OvO is that each classifier only needs to be trained on the part of the training\nset for the two classes that it must distinguish.\nSome algorithms (such as Support Vector Machine classifiers) scale poorly with the\nsize of the training set, so for these algorithms OvO is preferred since it is faster to\ntrain many classifiers on small training sets than training few classifiers on large\ntraining sets. For most binary classification algorithms, however, OvA is preferred.", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 297, "content": "training sets. For most binary classification algorithms, however, OvA is preferred.\nScikit-Learn detects when you try to use a binary classification algorithm for a multi\u2010\nclass classification task, and it automatically runs OvA (except for SVM classifiers for\nwhich it uses OvO). Let\u2019s try this with the SGDClassifier:\n>>> sgd_clf.fit(X_train, y_train) # y_train, not y_train_5\n>>> sgd_clf.predict([some_digit])\narray([ 5.])\nThat was easy! This code trains the SGDClassifier on the training set using the origi\u2010\nnal target classes from 0 to 9 (y_train), instead of the 5-versus-all target classes\n(y_train_5). Then it makes a prediction (a correct one in this case). Under the hood,\nScikit-Learn actually trained 10 binary classifiers, got their decision scores for the\nimage, and selected the class with the highest score.\nTo see that this is indeed the case, you can call the decision_function() method.\nInstead of returning just one score per instance, it now returns 10 scores, one per\nclass:", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 298, "content": "Instead of returning just one score per instance, it now returns 10 scores, one per\nclass:\n>>> some_digit_scores = sgd_clf.decision_function([some_digit])\n>>> some_digit_scores\n94 \n| \nChapter 3: Classification\nDownload from finelybook www.finelybook.com", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 299, "content": "array([[-311402.62954431, -363517.28355739, -446449.5306454 ,\n -183226.61023518, -414337.15339485, 161855.74572176,\n -452576.39616343, -471957.14962573, -518542.33997148,\n -536774.63961222]])\nThe highest score is indeed the one corresponding to class 5:\n>>> np.argmax(some_digit_scores)\n5\n>>> sgd_clf.classes_\narray([ 0., 1., 2., 3., 4., 5., 6., 7., 8., 9.])\n>>> sgd_clf.classes[5]\n5.0\nWhen a classifier is trained, it stores the list of target classes in its\nclasses_ attribute, ordered by value. In this case, the index of each\nclass in the classes_ array conveniently matches the class itself\n(e.g., the class at index 5 happens to be class 5), but in general you\nwon\u2019t be so lucky.\nIf you want to force ScikitLearn to use one-versus-one or one-versus-all, you can use\nthe OneVsOneClassifier or OneVsRestClassifier classes. Simply create an instance\nand pass a binary classifier to its constructor. For example, this code creates a multi\u2010", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 300, "content": "and pass a binary classifier to its constructor. For example, this code creates a multi\u2010\nclass classifier using the OvO strategy, based on a SGDClassifier:\n>>> from sklearn.multiclass import OneVsOneClassifier\n>>> ovo_clf = OneVsOneClassifier(SGDClassifier(random_state=42))\n>>> ovo_clf.fit(X_train, y_train)\n>>> ovo_clf.predict([some_digit])\narray([ 5.])\n>>> len(ovo_clf.estimators_)\n45\nTraining a RandomForestClassifier is just as easy:\n>>> forest_clf.fit(X_train, y_train)\n>>> forest_clf.predict([some_digit])\narray([ 5.])\nThis time Scikit-Learn did not have to run OvA or OvO because Random Forest\nclassifiers can directly classify instances into multiple classes. You can call\npredict_proba() to get the list of probabilities that the classifier assigned to each\ninstance for each class:\n>>> forest_clf.predict_proba([some_digit])\narray([[ 0.1, 0. , 0. , 0.1, 0. , 0.8, 0. , 0. , 0. , 0. ]])\nYou can see that the classifier is fairly confident about its prediction: the 0.8 at the 5th", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 301, "content": "You can see that the classifier is fairly confident about its prediction: the 0.8 at the 5th\nindex in the array means that the model estimates an 80% probability that the image\nMulticlass Classification \n| \n95\nDownload from finelybook www.finelybook.com", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 302, "content": "represents a 5. It also thinks that the image could instead be a 0 or a 3 (10% chance\neach).\nNow of course you want to evaluate these classifiers. As usual, you want to use cross-\nvalidation. Let\u2019s evaluate the SGDClassifier\u2019s accuracy using the cross_val_score()\nfunction:\n>>> cross_val_score(sgd_clf, X_train, y_train, cv=3, scoring=\"accuracy\")\narray([ 0.84063187, 0.84899245, 0.86652998])\nIt gets over 84% on all test folds. If you used a random classifier, you would get 10%\naccuracy, so this is not such a bad score, but you can still do much better. For exam\u2010\nple, simply scaling the inputs (as discussed in Chapter 2) increases accuracy above\n90%:\n>>> from sklearn.preprocessing import StandardScaler\n>>> scaler = StandardScaler()\n>>> X_train_scaled = scaler.fit_transform(X_train.astype(np.float64))\n>>> cross_val_score(sgd_clf, X_train_scaled, y_train, cv=3, scoring=\"accuracy\")\narray([ 0.91011798, 0.90874544, 0.906636 ])\nError Analysis", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 303, "content": "array([ 0.91011798, 0.90874544, 0.906636 ])\nError Analysis\nOf course, if this were a real project, you would follow the steps in your Machine\nLearning project checklist (see Appendix B): exploring data preparation options, try\u2010\ning out multiple models, shortlisting the best ones and fine-tuning their hyperpara\u2010\nmeters using GridSearchCV, and automating as much as possible, as you did in the\nprevious chapter. Here, we will assume that you have found a promising model and\nyou want to find ways to improve it. One way to do this is to analyze the types of\nerrors it makes.\nFirst, you can look at the confusion matrix. You need to make predictions using the\ncross_val_predict() function, then call the confusion_matrix() function, just like\nyou did earlier:\n>>> y_train_pred = cross_val_predict(sgd_clf, X_train_scaled, y_train, cv=3)\n>>> conf_mx = confusion_matrix(y_train, y_train_pred)\n>>> conf_mx\narray([[5725, 3, 24, 9, 10, 49, 50, 10, 39, 4],", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 304, "content": ">>> conf_mx\narray([[5725, 3, 24, 9, 10, 49, 50, 10, 39, 4],\n [ 2, 6493, 43, 25, 7, 40, 5, 10, 109, 8],\n [ 51, 41, 5321, 104, 89, 26, 87, 60, 166, 13],\n [ 47, 46, 141, 5342, 1, 231, 40, 50, 141, 92],\n [ 19, 29, 41, 10, 5366, 9, 56, 37, 86, 189],\n [ 73, 45, 36, 193, 64, 4582, 111, 30, 193, 94],\n [ 29, 34, 44, 2, 42, 85, 5627, 10, 45, 0],\n [ 25, 24, 74, 32, 54, 12, 6, 5787, 15, 236],\n [ 52, 161, 73, 156, 10, 163, 61, 25, 5027, 123],\n [ 43, 35, 26, 92, 178, 28, 2, 223, 82, 5240]])\n96 \n| \nChapter 3: Classification\nDownload from finelybook www.finelybook.com", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 305, "content": "That\u2019s a lot of numbers. It\u2019s often more convenient to look at an image representation\nof the confusion matrix, using Matplotlib\u2019s matshow() function:\nplt.matshow(conf_mx, cmap=plt.cm.gray)\nplt.show()\nThis confusion matrix looks fairly good, since most images are on the main diagonal,\nwhich means that they were classified correctly. The 5s look slightly darker than the\nother digits, which could mean that there are fewer images of 5s in the dataset or that\nthe classifier does not perform as well on 5s as on other digits. In fact, you can verify\nthat both are the case.\nLet\u2019s focus the plot on the errors. First, you need to divide each value in the confusion\nmatrix by the number of images in the corresponding class, so you can compare error\nrates instead of absolute number of errors (which would make abundant classes look\nunfairly bad):\nrow_sums = conf_mx.sum(axis=1, keepdims=True)\nnorm_conf_mx = conf_mx / row_sums", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 306, "content": "unfairly bad):\nrow_sums = conf_mx.sum(axis=1, keepdims=True)\nnorm_conf_mx = conf_mx / row_sums\nNow let\u2019s fill the diagonal with zeros to keep only the errors, and let\u2019s plot the result:\nnp.fill_diagonal(norm_conf_mx, 0)\nplt.matshow(norm_conf_mx, cmap=plt.cm.gray)\nplt.show()\nError Analysis \n| \n97\nDownload from finelybook www.finelybook.com", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 307, "content": "Now you can clearly see the kinds of errors the classifier makes. Remember that rows\nrepresent actual classes, while columns represent predicted classes. The columns for\nclasses 8 and 9 are quite bright, which tells you that many images get misclassified as\n8s or 9s. Similarly, the rows for classes 8 and 9 are also quite bright, telling you that 8s\nand 9s are often confused with other digits. Conversely, some rows are pretty dark,\nsuch as row 1: this means that most 1s are classified correctly (a few are confused\nwith 8s, but that\u2019s about it). Notice that the errors are not perfectly symmetrical; for\nexample, there are more 5s misclassified as 8s than the reverse.\nAnalyzing the confusion matrix can often give you insights on ways to improve your\nclassifier. Looking at this plot, it seems that your efforts should be spent on improving\nclassification of 8s and 9s, as well as fixing the specific 3/5 confusion. For example,", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 308, "content": "classification of 8s and 9s, as well as fixing the specific 3/5 confusion. For example,\nyou could try to gather more training data for these digits. Or you could engineer\nnew features that would help the classifier\u2014for example, writing an algorithm to\ncount the number of closed loops (e.g., 8 has two, 6 has one, 5 has none). Or you\ncould preprocess the images (e.g., using Scikit-Image, Pillow, or OpenCV) to make\nsome patterns stand out more, such as closed loops.\nAnalyzing individual errors can also be a good way to gain insights on what your\nclassifier is doing and why it is failing, but it is more difficult and time-consuming.\nFor example, let\u2019s plot examples of 3s and 5s:\ncl_a, cl_b = 3, 5\nX_aa = X_train[(y_train == cl_a) & (y_train_pred == cl_a)]\nX_ab = X_train[(y_train == cl_a) & (y_train_pred == cl_b)]\nX_ba = X_train[(y_train == cl_b) & (y_train_pred == cl_a)]\nX_bb = X_train[(y_train == cl_b) & (y_train_pred == cl_b)]\nplt.figure(figsize=(8,8))", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 309, "content": "X_bb = X_train[(y_train == cl_b) & (y_train_pred == cl_b)]\nplt.figure(figsize=(8,8))\nplt.subplot(221); plot_digits(X_aa[:25], images_per_row=5)\nplt.subplot(222); plot_digits(X_ab[:25], images_per_row=5)\n98 \n| \nChapter 3: Classification\nDownload from finelybook www.finelybook.com", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 310, "content": "3 But remember that our brain is a fantastic pattern recognition system, and our visual system does a lot of\ncomplex preprocessing before any information reaches our consciousness, so the fact that it feels simple does\nnot mean that it is.\nplt.subplot(223); plot_digits(X_ba[:25], images_per_row=5)\nplt.subplot(224); plot_digits(X_bb[:25], images_per_row=5)\nplt.show()\nThe two 5\u00d75 blocks on the left show digits classified as 3s, and the two 5\u00d75 blocks on\nthe right show images classified as 5s. Some of the digits that the classifier gets wrong\n(i.e., in the bottom-left and top-right blocks) are so badly written that even a human\nwould have trouble classifying them (e.g., the 5 on the 8th row and 1st column truly\nlooks like a 3). However, most misclassified images seem like obvious errors to us,\nand it\u2019s hard to understand why the classifier made the mistakes it did.3 The reason is\nthat we used a simple SGDClassifier, which is a linear model. All it does is assign a", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 311, "content": "that we used a simple SGDClassifier, which is a linear model. All it does is assign a\nweight per class to each pixel, and when it sees a new image it just sums up the weigh\u2010\nted pixel intensities to get a score for each class. So since 3s and 5s differ only by a few\npixels, this model will easily confuse them.\nThe main difference between 3s and 5s is the position of the small line that joins the\ntop line to the bottom arc. If you draw a 3 with the junction slightly shifted to the left,\nthe classifier might classify it as a 5, and vice versa. In other words, this classifier is\nquite sensitive to image shifting and rotation. So one way to reduce the 3/5 confusion\nwould be to preprocess the images to ensure that they are well centered and not too\nrotated. This will probably help reduce other errors as well.\nError Analysis \n| \n99\nDownload from finelybook www.finelybook.com", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 312, "content": "Multilabel Classification\nUntil now each instance has always been assigned to just one class. In some cases you\nmay want your classifier to output multiple classes for each instance. For example,\nconsider a face-recognition classifier: what should it do if it recognizes several people\non the same picture? Of course it should attach one label per person it recognizes. Say\nthe classifier has been trained to recognize three faces, Alice, Bob, and Charlie; then\nwhen it is shown a picture of Alice and Charlie, it should output [1, 0, 1] (meaning\n\u201cAlice yes, Bob no, Charlie yes\u201d). Such a classification system that outputs multiple\nbinary labels is called a multilabel classification system.\nWe won\u2019t go into face recognition just yet, but let\u2019s look at a simpler example, just for\nillustration purposes:\nfrom sklearn.neighbors import KNeighborsClassifier\ny_train_large = (y_train >= 7)\ny_train_odd = (y_train % 2 == 1)\ny_multilabel = np.c_[y_train_large, y_train_odd]", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 313, "content": "y_train_odd = (y_train % 2 == 1)\ny_multilabel = np.c_[y_train_large, y_train_odd]\nknn_clf = KNeighborsClassifier()\nknn_clf.fit(X_train, y_multilabel)\nThis code creates a y_multilabel array containing two target labels for each digit\nimage: the first indicates whether or not the digit is large (7, 8, or 9) and the second\nindicates whether or not it is odd. The next lines create a KNeighborsClassifier \ninstance (which supports multilabel classification, but not all classifiers do) and we\ntrain it using the multiple targets array. Now you can make a prediction, and notice\nthat it outputs two labels:\n>>> knn_clf.predict([some_digit])\narray([[False, True]], dtype=bool)\nAnd it gets it right! The digit 5 is indeed not large (False) and odd (True).\nThere are many ways to evaluate a multilabel classifier, and selecting the right metric\nreally depends on your project. For example, one approach is to measure the F1 score", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 314, "content": "really depends on your project. For example, one approach is to measure the F1 score\nfor each individual label (or any other binary classifier metric discussed earlier), then\nsimply compute the average score. This code computes the average F1 score across all\nlabels:\n>>> y_train_knn_pred = cross_val_predict(knn_clf, X_train, y_train, cv=3)\n>>> f1_score(y_train, y_train_knn_pred, average=\"macro\")\n0.96845540180280221\nThis assumes that all labels are equally important, which may not be the case. In par\u2010\nticular, if you have many more pictures of Alice than of Bob or Charlie, you may want\nto give more weight to the classifier\u2019s score on pictures of Alice. One simple option is\n100 \n| \nChapter 3: Classification\nDownload from finelybook www.finelybook.com", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 315, "content": "4 Scikit-Learn offers a few other averaging options and multilabel classifier metrics; see the documentation for\nmore details.\nto give each label a weight equal to its support (i.e., the number of instances with that\ntarget label). To do this, simply set average=\"weighted\" in the preceding code.4\nMultioutput Classification\nThe last type of classification task we are going to discuss here is called multioutput-\nmulticlass classification (or simply multioutput classification). It is simply a generaliza\u2010\ntion of multilabel classification where each label can be multiclass (i.e., it can have\nmore than two possible values).\nTo illustrate this, let\u2019s build a system that removes noise from images. It will take as\ninput a noisy digit image, and it will (hopefully) output a clean digit image, repre\u2010\nsented as an array of pixel intensities, just like the MNIST images. Notice that the\nclassifier\u2019s output is multilabel (one label per pixel) and each label can have multiple", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 316, "content": "classifier\u2019s output is multilabel (one label per pixel) and each label can have multiple\nvalues (pixel intensity ranges from 0 to 255). It is thus an example of a multioutput\nclassification system.\nThe line between classification and regression is sometimes blurry,\nsuch as in this example. Arguably, predicting pixel intensity is more\nakin to regression than to classification. Moreover, multioutput\nsystems are not limited to classification tasks; you could even have\na system that outputs multiple labels per instance, including both\nclass labels and value labels.\nLet\u2019s start by creating the training and test sets by taking the MNIST images and\nadding noise to their pixel intensities using NumPy\u2019s randint() function. The target\nimages will be the original images:\nnoise = rnd.randint(0, 100, (len(X_train), 784))\nnoise = rnd.randint(0, 100, (len(X_test), 784))\nX_train_mod = X_train + noise\nX_test_mod = X_test + noise\ny_train_mod = X_train\ny_test_mod = X_test", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 317, "content": "X_train_mod = X_train + noise\nX_test_mod = X_test + noise\ny_train_mod = X_train\ny_test_mod = X_test\nLet\u2019s take a peek at an image from the test set (yes, we\u2019re snooping on the test data, so\nyou should be frowning right now):\nMultioutput Classification \n| \n101\nDownload from finelybook www.finelybook.com", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 318, "content": "5 You can use the shift() function from the scipy.ndimage.interpolation module. For example,\nshift(image, [2, 1], cval=0) shifts the image 2 pixels down and 1 pixel to the right.\nOn the left is the noisy input image, and on the right is the clean target image. Now\nlet\u2019s train the classifier and make it clean this image:\nknn_clf.fit(X_train_mod, y_train_mod)\nclean_digit = knn_clf.predict([X_test_mod[some_index]])\nplot_digit(clean_digit)\nLooks close enough to the target! This concludes our tour of classification. Hopefully\nyou should now know how to select good metrics for classification tasks, pick the\nappropriate precision/recall tradeoff, compare classifiers, and more generally build\ngood classification systems for a variety of tasks.\nExercises\n1. Try to build a classifier for the MNIST dataset that achieves over 97% accuracy\non the test set. Hint: the KNeighborsClassifier works quite well for this task;\nyou just need to find good hyperparameter values (try a grid search on the", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 319, "content": "you just need to find good hyperparameter values (try a grid search on the\nweights and n_neighbors hyperparameters).\n2. Write a function that can shift an MNIST image in any direction (left, right, up,\nor down) by one pixel.5 Then, for each image in the training set, create four shif\u2010\nted copies (one per direction) and add them to the training set. Finally, train your\nbest model on this expanded training set and measure its accuracy on the test set.\nYou should observe that your model performs even better now! This technique of\n102 \n| \nChapter 3: Classification\nDownload from finelybook www.finelybook.com", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 320, "content": "artificially growing the training set is called data augmentation or training set\nexpansion.\n3. Tackle the Titanic dataset. A great place to start is on Kaggle.\n4. Build a spam classifier (a more challenging exercise):\n\u2022 Download examples of spam and ham from Apache SpamAssassin\u2019s public\ndatasets.\n\u2022 Unzip the datasets and familiarize yourself with the data format.\n\u2022 Split the datasets into a training set and a test set.\n\u2022 Write a data preparation pipeline to convert each email into a feature vector.\nYour preparation pipeline should transform an email into a (sparse) vector\nindicating the presence or absence of each possible word. For example, if all\nemails only ever contain four words, \u201cHello,\u201d \u201chow,\u201d \u201care,\u201d \u201cyou,\u201d then the email\n\u201cHello you Hello Hello you\u201d would be converted into a vector [1, 0, 0, 1]\n(meaning [\u201cHello\u201d is present, \u201chow\u201d is absent, \u201care\u201d is absent, \u201cyou\u201d is\npresent]), or [3, 0, 0, 2] if you prefer to count the number of occurrences of\neach word.", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 321, "content": "present]), or [3, 0, 0, 2] if you prefer to count the number of occurrences of\neach word.\n\u2022 You may want to add hyperparameters to your preparation pipeline to control\nwhether or not to strip off email headers, convert each email to lowercase,\nremove punctuation, replace all URLs with \u201cURL,\u201d replace all numbers with\n\u201cNUMBER,\u201d or even perform stemming (i.e., trim off word endings; there are\nPython libraries available to do this).\n\u2022 Then try out several classifiers and see if you can build a great spam classifier,\nwith both high recall and high precision.\nSolutions to these exercises are available in the online Jupyter notebooks at https://\ngithub.com/ageron/handson-ml.\nExercises \n| \n103\nDownload from finelybook www.finelybook.com", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 322, "content": "Download from finelybook www.finelybook.com", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 323, "content": "CHAPTER 4\nTraining Models\nSo far we have treated Machine Learning models and their training algorithms mostly\nlike black boxes. If you went through some of the exercises in the previous chapters,\nyou may have been surprised by how much you can get done without knowing any\u2010\nthing about what\u2019s under the hood: you optimized a regression system, you improved\na digit image classifier, and you even built a spam classifier from scratch\u2014all this\nwithout knowing how they actually work. Indeed, in many situations you don\u2019t really\nneed to know the implementation details.\nHowever, having a good understanding of how things work can help you quickly\nhome in on the appropriate model, the right training algorithm to use, and a good set\nof hyperparameters for your task. Understanding what\u2019s under the hood will also help\nyou debug issues and perform error analysis more efficiently. Lastly, most of the top\u2010\nics discussed in this chapter will be essential in understanding, building, and training", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 324, "content": "ics discussed in this chapter will be essential in understanding, building, and training\nneural networks (discussed in Part II of this book).\nIn this chapter, we will start by looking at the Linear Regression model, one of the\nsimplest models there is. We will discuss two very different ways to train it:\n\u2022 Using a direct \u201cclosed-form\u201d equation that directly computes the model parame\u2010\nters that best fit the model to the training set (i.e., the model parameters that\nminimize the cost function over the training set).\n\u2022 Using an iterative optimization approach, called Gradient Descent (GD), that\ngradually tweaks the model parameters to minimize the cost function over the\ntraining set, eventually converging to the same set of parameters as the first\nmethod. We will look at a few variants of Gradient Descent that we will use again\nand again when we study neural networks in Part II: Batch GD, Mini-batch GD,\nand Stochastic GD.\n105\nDownload from finelybook www.finelybook.com", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 325, "content": "Next we will look at Polynomial Regression, a more complex model that can fit non\u2010\nlinear datasets. Since this model has more parameters than Linear Regression, it is\nmore prone to overfitting the training data, so we will look at how to detect whether\nor not this is the case, using learning curves, and then we will look at several regulari\u2010\nzation techniques that can reduce the risk of overfitting the training set.\nFinally, we will look at two more models that are commonly used for classification\ntasks: Logistic Regression and Softmax Regression.\nThere will be quite a few math equations in this chapter, using basic\nnotions of linear algebra and calculus. To understand these equa\u2010\ntions, you will need to know what vectors and matrices are, how to\ntranspose them, what the dot product is, what matrix inverse is,\nand what partial derivatives are. If you are unfamiliar with these\nconcepts, please go through the linear algebra and calculus intro\u2010", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 326, "content": "concepts, please go through the linear algebra and calculus intro\u2010\nductory tutorials available as Jupyter notebooks in the online sup\u2010\nplemental material. For those who are truly allergic to\nmathematics, you should still go through this chapter and simply\nskip the equations; hopefully, the text will be sufficient to help you\nunderstand most of the concepts.\nLinear Regression\nIn Chapter 1, we looked at a simple regression model of life satisfaction: life_satisfac\u2010\ntion = \u03b80 + \u03b81 \u00d7 GDP_per_capita.\nThis model is just a linear function of the input feature GDP_per_capita. \u03b80 and \u03b81 are\nthe model\u2019s parameters.\nMore generally, a linear model makes a prediction by simply computing a weighted\nsum of the input features, plus a constant called the bias term (also called the intercept\nterm), as shown in Equation 4-1.\nEquation 4-1. Linear Regression model prediction\ny = \u03b80 + \u03b81x1 + \u03b82x2 + \u22ef+ \u03b8nxn\n\u2022 \u0177 is the predicted value.\n\u2022 n is the number of features.\n\u2022 xi is the ith feature value.", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 327, "content": "\u2022 \u0177 is the predicted value.\n\u2022 n is the number of features.\n\u2022 xi is the ith feature value.\n\u2022 \u03b8j is the jth model parameter (including the bias term \u03b80 and the feature weights\n\u03b81, \u03b82, \u22ef, \u03b8n).\n106 \n| \nChapter 4: Training Models\nDownload from finelybook www.finelybook.com", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 328, "content": "1 It is often the case that a learning algorithm will try to optimize a different function than the performance\nmeasure used to evaluate the final model. This is generally because that function is easier to compute, because\nit has useful differentiation properties that the performance measure lacks, or because we want to constrain\nthe model during training, as we will see when we discuss regularization.\nThis can be written much more concisely using a vectorized form, as shown in Equa\u2010\ntion 4-2.\nEquation 4-2. Linear Regression model prediction (vectorized form)\ny = h\u03b8 \ufffd= \u03b8T \u00b7 \ufffd\n\u2022 \u03b8 is the model\u2019s parameter vector, containing the bias term \u03b80 and the feature\nweights \u03b81 to \u03b8n.\n\u2022 \u03b8T is the transpose of \u03b8 (a row vector instead of a column vector).\n\u2022 x is the instance\u2019s feature vector, containing x0 to xn, with x0 always equal to 1.\n\u2022 \u03b8T \u00b7 x is the dot product of \u03b8T and x.\n\u2022 h\u03b8 is the hypothesis function, using the model parameters \u03b8.", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 329, "content": "\u2022 h\u03b8 is the hypothesis function, using the model parameters \u03b8.\nOkay, that\u2019s the Linear Regression model, so now how do we train it? Well, recall that\ntraining a model means setting its parameters so that the model best fits the training\nset. For this purpose, we first need a measure of how well (or poorly) the model fits\nthe training data. In Chapter 2 we saw that the most common performance measure\nof a regression model is the Root Mean Square Error (RMSE) (Equation 2-1). There\u2010\nfore, to train a Linear Regression model, you need to find the value of \u03b8 that minimi\u2010\nzes the RMSE. In practice, it is simpler to minimize the Mean Square Error (MSE)\nthan the RMSE, and it leads to the same result (because the value that minimizes a\nfunction also minimizes its square root).1\nThe MSE of a Linear Regression hypothesis h\u03b8 on a training set X is calculated using\nEquation 4-3.\nEquation 4-3. MSE cost function for a Linear Regression model\nMSE \ufffd, h\u03b8 = 1\nm \u2211\ni = 1\nm\n\u03b8T \u00b7 \ufffdi \u2212y i 2", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 330, "content": "MSE \ufffd, h\u03b8 = 1\nm \u2211\ni = 1\nm\n\u03b8T \u00b7 \ufffdi \u2212y i 2\nMost of these notations were presented in Chapter 2 (see \u201cNotations\u201d on page 38).\nThe only difference is that we write h\u03b8 instead of just h in order to make it clear that\nthe model is parametrized by the vector \u03b8. To simplify notations, we will just write\nMSE(\u03b8) instead of MSE(X, h\u03b8).\nLinear Regression \n| \n107\nDownload from finelybook www.finelybook.com", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 331, "content": "2 The demonstration that this returns the value of \u03b8 that minimizes the cost function is outside the scope of this\nbook.\nThe Normal Equation\nTo find the value of \u03b8 that minimizes the cost function, there is a closed-form solution\n\u2014in other words, a mathematical equation that gives the result directly. This is called\nthe Normal Equation (Equation 4-4).2\nEquation 4-4. Normal Equation\n\u03b8 = \ufffdT \u00b7 \ufffd\n\u22121 \u00b7 \ufffdT \u00b7 \ufffd\n\u2022 \u03b8 is the value of \u03b8 that minimizes the cost function.\n\u2022 y is the vector of target values containing y(1) to y(m).\nLet\u2019s generate some linear-looking data to test this equation on (Figure 4-1):\nimport numpy as np\nX = 2 * np.random.rand(100, 1)\ny = 4 + 3 * X + np.random.randn(100, 1)\nFigure 4-1. Randomly generated linear dataset\n108 \n| \nChapter 4: Training Models\nDownload from finelybook www.finelybook.com", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 332, "content": "Now let\u2019s compute \u03b8 using the Normal Equation. We will use the inv() function from\nNumPy\u2019s Linear Algebra module (np.linalg) to compute the inverse of a matrix, and\nthe dot() method for matrix multiplication:\nX_b = np.c_[np.ones((100, 1)), X] # add x0 = 1 to each instance\ntheta_best = np.linalg.inv(X_b.T.dot(X_b)).dot(X_b.T).dot(y)\nThe actual function that we used to generate the data is y = 4 + 3x0 + Gaussian noise.\nLet\u2019s see what the equation found:\n>>> theta_best\narray([[ 4.21509616],\n [ 2.77011339]])\nWe would have hoped for \u03b80 = 4 and \u03b81 = 3 instead of \u03b80 = 3.865 and \u03b81 = 3.139. Close\nenough, but the noise made it impossible to recover the exact parameters of the origi\u2010\nnal function.\nNow you can make predictions using \u03b8:\n>>> X_new = np.array([[0], [2]])\n>>> X_new_b = np.c_[np.ones((2, 1)), X_new] # add x0 = 1 to each instance\n>>> y_predict = X_new_b.dot(theta_best)\n>>> y_predict\narray([[ 4.21509616],\n [ 9.75532293]])\nLet\u2019s plot this model\u2019s predictions (Figure 4-2):", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 333, "content": "array([[ 4.21509616],\n [ 9.75532293]])\nLet\u2019s plot this model\u2019s predictions (Figure 4-2):\nplt.plot(X_new, y_predict, \"r-\")\nplt.plot(X, y, \"b.\")\nplt.axis([0, 2, 0, 15])\nplt.show()\nFigure 4-2. Linear Regression model predictions\nLinear Regression \n| \n109\nDownload from finelybook www.finelybook.com", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 334, "content": "3 Note that Scikit-Learn separates the bias term (intercept_) from the feature weights (coef_).\nThe equivalent code using Scikit-Learn looks like this:3\n>>> from sklearn.linear_model import LinearRegression\n>>> lin_reg = LinearRegression()\n>>> lin_reg.fit(X, y)\n>>> lin_reg.intercept_, lin_reg.coef_\n(array([ 4.21509616]), array([[ 2.77011339]]))\n>>> lin_reg.predict(X_new)\narray([[ 4.21509616],\n [ 9.75532293]])\nComputational Complexity\nThe Normal Equation computes the inverse of XT \u00b7 X, which is an n \u00d7 n matrix\n(where n is the number of features). The computational complexity of inverting such a\nmatrix is typically about O(n2.4) to O(n3) (depending on the implementation). In\nother words, if you double the number of features, you multiply the computation\ntime by roughly 22.4 = 5.3 to 23 = 8.\nThe Normal Equation gets very slow when the number of features\ngrows large (e.g., 100,000).\nOn the positive side, this equation is linear with regards to the number of instances in", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 335, "content": "On the positive side, this equation is linear with regards to the number of instances in\nthe training set (it is O(m)), so it handles large training sets efficiently, provided they\ncan fit in memory.\nAlso, once you have trained your Linear Regression model (using the Normal Equa\u2010\ntion or any other algorithm), predictions are very fast: the computational complexity\nis linear with regards to both the number of instances you want to make predictions\non and the number of features. In other words, making predictions on twice as many\ninstances (or twice as many features) will just take roughly twice as much time.\nNow we will look at very different ways to train a Linear Regression model, better\nsuited for cases where there are a large number of features, or too many training\ninstances to fit in memory.\n110 \n| \nChapter 4: Training Models\nDownload from finelybook www.finelybook.com", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 336, "content": "Gradient Descent\nGradient Descent is a very generic optimization algorithm capable of finding optimal\nsolutions to a wide range of problems. The general idea of Gradient Descent is to\ntweak parameters iteratively in order to minimize a cost function.\nSuppose you are lost in the mountains in a dense fog; you can only feel the slope of\nthe ground below your feet. A good strategy to get to the bottom of the valley quickly\nis to go downhill in the direction of the steepest slope. This is exactly what Gradient\nDescent does: it measures the local gradient of the error function with regards to the \nparameter vector \u03b8, and it goes in the direction of descending gradient. Once the gra\u2010\ndient is zero, you have reached a minimum!\nConcretely, you start by filling \u03b8 with random values (this is called random initializa\u2010\ntion), and then you improve it gradually, taking one baby step at a time, each step\nattempting to decrease the cost function (e.g., the MSE), until the algorithm converges", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 337, "content": "attempting to decrease the cost function (e.g., the MSE), until the algorithm converges\nto a minimum (see Figure 4-3).\nFigure 4-3. Gradient Descent\nAn important parameter in Gradient Descent is the size of the steps, determined by \nthe learning rate hyperparameter. If the learning rate is too small, then the algorithm\nwill have to go through many iterations to converge, which will take a long time (see\nFigure 4-4).\nGradient Descent \n| \n111\nDownload from finelybook www.finelybook.com", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 338, "content": "Figure 4-4. Learning rate too small\nOn the other hand, if the learning rate is too high, you might jump across the valley\nand end up on the other side, possibly even higher up than you were before. This\nmight make the algorithm diverge, with larger and larger values, failing to find a good\nsolution (see Figure 4-5).\nFigure 4-5. Learning rate too large\nFinally, not all cost functions look like nice regular bowls. There may be holes, ridges,\nplateaus, and all sorts of irregular terrains, making convergence to the minimum very\ndifficult. Figure 4-6 shows the two main challenges with Gradient Descent: if the ran\u2010\ndom initialization starts the algorithm on the left, then it will converge to a local mini\u2010\nmum, which is not as good as the global minimum. If it starts on the right, then it will\ntake a very long time to cross the plateau, and if you stop too early you will never\nreach the global minimum.\n112 \n| \nChapter 4: Training Models\nDownload from finelybook www.finelybook.com", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 339, "content": "4 Technically speaking, its derivative is Lipschitz continuous.\n5 Since feature 1 is smaller, it takes a larger change in \u03b81 to affect the cost function, which is why the bowl is\nelongated along the \u03b81 axis.\nFigure 4-6. Gradient Descent pitfalls\nFortunately, the MSE cost function for a Linear Regression model happens to be a\nconvex function, which means that if you pick any two points on the curve, the line\nsegment joining them never crosses the curve. This implies that there are no local\nminima, just one global minimum. It is also a continuous function with a slope that\nnever changes abruptly.4 These two facts have a great consequence: Gradient Descent\nis guaranteed to approach arbitrarily close the global minimum (if you wait long\nenough and if the learning rate is not too high).\nIn fact, the cost function has the shape of a bowl, but it can be an elongated bowl if\nthe features have very different scales. Figure 4-7 shows Gradient Descent on a train\u2010", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 340, "content": "the features have very different scales. Figure 4-7 shows Gradient Descent on a train\u2010\ning set where features 1 and 2 have the same scale (on the left), and on a training set\nwhere feature 1 has much smaller values than feature 2 (on the right).5\nFigure 4-7. Gradient Descent with and without feature scaling\nGradient Descent \n| \n113\nDownload from finelybook www.finelybook.com", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 341, "content": "As you can see, on the left the Gradient Descent algorithm goes straight toward the\nminimum, thereby reaching it quickly, whereas on the right it first goes in a direction\nalmost orthogonal to the direction of the global minimum, and it ends with a long\nmarch down an almost flat valley. It will eventually reach the minimum, but it will\ntake a long time.\nWhen using Gradient Descent, you should ensure that all features\nhave a similar scale (e.g., using Scikit-Learn\u2019s StandardScaler\nclass), or else it will take much longer to converge.\nThis diagram also illustrates the fact that training a model means searching for a\ncombination of model parameters that minimizes a cost function (over the training\nset). It is a search in the model\u2019s parameter space: the more parameters a model has,\nthe more dimensions this space has, and the harder the search is: searching for a nee\u2010\ndle in a 300-dimensional haystack is much trickier than in three dimensions. Fortu\u2010", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 342, "content": "dle in a 300-dimensional haystack is much trickier than in three dimensions. Fortu\u2010\nnately, since the cost function is convex in the case of Linear Regression, the needle is\nsimply at the bottom of the bowl.\nBatch Gradient Descent\nTo implement Gradient Descent, you need to compute the gradient of the cost func\u2010\ntion with regards to each model parameter \u03b8j. In other words, you need to calculate\nhow much the cost function will change if you change \u03b8j just a little bit. This is called \na partial derivative. It is like asking \u201cwhat is the slope of the mountain under my feet\nif I face east?\u201d and then asking the same question facing north (and so on for all other\ndimensions, if you can imagine a universe with more than three dimensions). Equa\u2010\ntion 4-5 computes the partial derivative of the cost function with regards to parame\u2010\nter \u03b8j, noted \u2202\n\u2202\u03b8jMSE \u03b8 .\nEquation 4-5. Partial derivatives of the cost function\n\u2202\n\u2202\u03b8j\nMSE \u03b8 = 2\nm \u2211\ni = 1\nm\n\u03b8T \u00b7 \ufffdi \u2212y i xj\ni", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 343, "content": "\u2202\n\u2202\u03b8j\nMSE \u03b8 = 2\nm \u2211\ni = 1\nm\n\u03b8T \u00b7 \ufffdi \u2212y i xj\ni\nInstead of computing these gradients individually, you can use Equation 4-6 to com\u2010\npute them all in one go. The gradient vector, noted \u2207\u03b8MSE(\u03b8), contains all the partial\nderivatives of the cost function (one for each model parameter).\n114 \n| \nChapter 4: Training Models\nDownload from finelybook www.finelybook.com", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 344, "content": "6 Eta (\u03b7) is the 7th letter of the Greek alphabet.\nEquation 4-6. Gradient vector of the cost function\n\u2207\u03b8 MSE \u03b8 =\n\u2202\n\u2202\u03b80\nMSE \u03b8\n\u2202\n\u2202\u03b81\nMSE \u03b8\n\u22ee\n\u2202\n\u2202\u03b8n\nMSE \u03b8\n= 2\nm\ufffdT \u00b7 \ufffd\u00b7 \u03b8 \u2212\ufffd\nNotice that this formula involves calculations over the full training\nset X, at each Gradient Descent step! This is why the algorithm is\ncalled Batch Gradient Descent: it uses the whole batch of training\ndata at every step. As a result it is terribly slow on very large train\u2010\ning sets (but we will see much faster Gradient Descent algorithms\nshortly). However, Gradient Descent scales well with the number of\nfeatures; training a Linear Regression model when there are hun\u2010\ndreds of thousands of features is much faster using Gradient\nDescent than using the Normal Equation.\nOnce you have the gradient vector, which points uphill, just go in the opposite direc\u2010\ntion to go downhill. This means subtracting \u2207\u03b8MSE(\u03b8) from \u03b8. This is where the \nlearning rate \u03b7 comes into play:6 multiply the gradient vector by \u03b7 to determine the", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 345, "content": "learning rate \u03b7 comes into play:6 multiply the gradient vector by \u03b7 to determine the\nsize of the downhill step (Equation 4-7).\nEquation 4-7. Gradient Descent step\n\u03b8 next step = \u03b8 \u2212\u03b7\u2207\u03b8 MSE \u03b8\nLet\u2019s look at a quick implementation of this algorithm:\neta = 0.1 # learning rate\nn_iterations = 1000\nm = 100\ntheta = np.random.randn(2,1) # random initialization\nfor iteration in range(n_iterations):\n gradients = 2/m * X_b.T.dot(X_b.dot(theta) - y)\n theta = theta - eta * gradients\nGradient Descent \n| \n115\nDownload from finelybook www.finelybook.com", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 346, "content": "That wasn\u2019t too hard! Let\u2019s look at the resulting theta:\n>>> theta\narray([[ 4.21509616],\n [ 2.77011339]])\nHey, that\u2019s exactly what the Normal Equation found! Gradient Descent worked per\u2010\nfectly. But what if you had used a different learning rate eta? Figure 4-8 shows the\nfirst 10 steps of Gradient Descent using three different learning rates (the dashed line\nrepresents the starting point).\nFigure 4-8. Gradient Descent with various learning rates\nOn the left, the learning rate is too low: the algorithm will eventually reach the solu\u2010\ntion, but it will take a long time. In the middle, the learning rate looks pretty good: in\njust a few iterations, it has already converged to the solution. On the right, the learn\u2010\ning rate is too high: the algorithm diverges, jumping all over the place and actually\ngetting further and further away from the solution at every step.\nTo find a good learning rate, you can use grid search (see Chapter 2). However, you", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 347, "content": "To find a good learning rate, you can use grid search (see Chapter 2). However, you\nmay want to limit the number of iterations so that grid search can eliminate models\nthat take too long to converge.\nYou may wonder how to set the number of iterations. If it is too low, you will still be\nfar away from the optimal solution when the algorithm stops, but if it is too high, you\nwill waste time while the model parameters do not change anymore. A simple solu\u2010\ntion is to set a very large number of iterations but to interrupt the algorithm when the\ngradient vector becomes tiny\u2014that is, when its norm becomes smaller than a tiny\nnumber \u03f5 (called the tolerance)\u2014because this happens when Gradient Descent has\n(almost) reached the minimum.\n116 \n| \nChapter 4: Training Models\nDownload from finelybook www.finelybook.com", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 348, "content": "7 Out-of-core algorithms are discussed in Chapter 1.\nConvergence Rate\nWhen the cost function is convex and its slope does not change abruptly (as is the\ncase for the MSE cost function), it can be shown that Batch Gradient Descent with a\nfixed learning rate has a convergence rate of O\n1\niterations . In other words, if you divide\nthe tolerance \u03f5 by 10 (to have a more precise solution), then the algorithm will have\nto run about 10 times more iterations.\nStochastic Gradient Descent\nThe main problem with Batch Gradient Descent is the fact that it uses the whole\ntraining set to compute the gradients at every step, which makes it very slow when\nthe training set is large. At the opposite extreme, Stochastic Gradient Descent just\npicks a random instance in the training set at every step and computes the gradients\nbased only on that single instance. Obviously this makes the algorithm much faster\nsince it has very little data to manipulate at every iteration. It also makes it possible to", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 349, "content": "since it has very little data to manipulate at every iteration. It also makes it possible to\ntrain on huge training sets, since only one instance needs to be in memory at each\niteration (SGD can be implemented as an out-of-core algorithm.7)\nOn the other hand, due to its stochastic (i.e., random) nature, this algorithm is much\nless regular than Batch Gradient Descent: instead of gently decreasing until it reaches\nthe minimum, the cost function will bounce up and down, decreasing only on aver\u2010\nage. Over time it will end up very close to the minimum, but once it gets there it will\ncontinue to bounce around, never settling down (see Figure 4-9). So once the algo\u2010\nrithm stops, the final parameter values are good, but not optimal.\nFigure 4-9. Stochastic Gradient Descent\nGradient Descent \n| \n117\nDownload from finelybook www.finelybook.com", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 350, "content": "When the cost function is very irregular (as in Figure 4-6), this can actually help the\nalgorithm jump out of local minima, so Stochastic Gradient Descent has a better\nchance of finding the global minimum than Batch Gradient Descent does.\nTherefore randomness is good to escape from local optima, but bad because it means\nthat the algorithm can never settle at the minimum. One solution to this dilemma is\nto gradually reduce the learning rate. The steps start out large (which helps make\nquick progress and escape local minima), then get smaller and smaller, allowing the\nalgorithm to settle at the global minimum. This process is called simulated annealing,\nbecause it resembles the process of annealing in metallurgy where molten metal is\nslowly cooled down. The function that determines the learning rate at each iteration\nis called the learning schedule. If the learning rate is reduced too quickly, you may get\nstuck in a local minimum, or even end up frozen halfway to the minimum. If the", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 351, "content": "stuck in a local minimum, or even end up frozen halfway to the minimum. If the\nlearning rate is reduced too slowly, you may jump around the minimum for a long\ntime and end up with a suboptimal solution if you halt training too early.\nThis code implements Stochastic Gradient Descent using a simple learning schedule:\nn_epochs = 50\nt0, t1 = 5, 50 # learning schedule hyperparameters\ndef learning_schedule(t):\n return t0 / (t + t1)\ntheta = np.random.randn(2,1) # random initialization\nfor epoch in range(n_epochs):\n for i in range(m):\n random_index = np.random.randint(m)\n xi = X_b[random_index:random_index+1]\n yi = y[random_index:random_index+1]\n gradients = 2 * xi.T.dot(xi.dot(theta) - yi)\n eta = learning_schedule(epoch * m + i)\n theta = theta - eta * gradients\nBy convention we iterate by rounds of m iterations; each round is called an epoch. \nWhile the Batch Gradient Descent code iterated 1,000 times through the whole train\u2010", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 352, "content": "While the Batch Gradient Descent code iterated 1,000 times through the whole train\u2010\ning set, this code goes through the training set only 50 times and reaches a fairly good\nsolution:\n>>> theta\narray([[ 4.21076011],\n [ 2.74856079]])\nFigure 4-10 shows the first 10 steps of training (notice how irregular the steps are).\n118 \n| \nChapter 4: Training Models\nDownload from finelybook www.finelybook.com", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 353, "content": "Figure 4-10. Stochastic Gradient Descent first 10 steps\nNote that since instances are picked randomly, some instances may be picked several\ntimes per epoch while others may not be picked at all. If you want to be sure that the\nalgorithm goes through every instance at each epoch, another approach is to shuffle\nthe training set, then go through it instance by instance, then shuffle it again, and so\non. However, this generally converges more slowly.\nTo perform Linear Regression using SGD with Scikit-Learn, you can use the SGDRe\ngressor class, which defaults to optimizing the squared error cost function. The fol\u2010\nlowing code runs 50 epochs, starting with a learning rate of 0.1 (eta0=0.1), using the\ndefault learning schedule (different from the preceding one), and it does not use any \nregularization (penalty=None; more details on this shortly):\nfrom sklearn.linear_model import SGDRegressor\nsgd_reg = SGDRegressor(n_iter=50, penalty=None, eta0=0.1)\nsgd_reg.fit(X, y.ravel())", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 354, "content": "sgd_reg = SGDRegressor(n_iter=50, penalty=None, eta0=0.1)\nsgd_reg.fit(X, y.ravel())\nOnce again, you find a solution very close to the one returned by the Normal Equa\u2010\ntion:\n>>> sgd_reg.intercept_, sgd_reg.coef_\n(array([ 4.18380366]), array([ 2.74205299]))\nMini-batch Gradient Descent\nThe last Gradient Descent algorithm we will look at is called Mini-batch Gradient\nDescent. It is quite simple to understand once you know Batch and Stochastic Gradi\u2010\nent Descent: at each step, instead of computing the gradients based on the full train\u2010\ning set (as in Batch GD) or based on just one instance (as in Stochastic GD), Mini-\nGradient Descent \n| \n119\nDownload from finelybook www.finelybook.com", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 355, "content": "8 While the Normal Equation can only perform Linear Regression, the Gradient Descent algorithms can be\nused to train many other models, as we will see.\nbatch GD computes the gradients on small random sets of instances called mini-\nbatches. The main advantage of Mini-batch GD over Stochastic GD is that you can\nget a performance boost from hardware optimization of matrix operations, especially\nwhen using GPUs.\nThe algorithm\u2019s progress in parameter space is less erratic than with SGD, especially\nwith fairly large mini-batches. As a result, Mini-batch GD will end up walking\naround a bit closer to the minimum than SGD. But, on the other hand, it may be\nharder for it to escape from local minima (in the case of problems that suffer from\nlocal minima, unlike Linear Regression as we saw earlier). Figure 4-11 shows the\npaths taken by the three Gradient Descent algorithms in parameter space during\ntraining. They all end up near the minimum, but Batch GD\u2019s path actually stops at the", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 356, "content": "training. They all end up near the minimum, but Batch GD\u2019s path actually stops at the\nminimum, while both Stochastic GD and Mini-batch GD continue to walk around.\nHowever, don\u2019t forget that Batch GD takes a lot of time to take each step, and Stochas\u2010\ntic GD and Mini-batch GD would also reach the minimum if you used a good learn\u2010\ning schedule.\nFigure 4-11. Gradient Descent paths in parameter space\nLet\u2019s compare the algorithms we\u2019ve discussed so far for Linear Regression8 (recall that\nm is the number of training instances and n is the number of features); see Table 4-1.\nTable 4-1. Comparison of algorithms for Linear Regression\nAlgorithm\nLarge m\nOut-of-core support\nLarge n\nHyperparams\nScaling required\nScikit-Learn\nNormal Equation\nFast\nNo\nSlow\n0\nNo\nLinearRegression\nBatch GD\nSlow\nNo\nFast\n2\nYes\nn/a\n120 \n| \nChapter 4: Training Models\nDownload from finelybook www.finelybook.com", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 357, "content": "9 A quadratic equation is of the form y = ax2 + bx + c.\nAlgorithm\nLarge m Out-of-core support Large n Hyperparams\nScaling required Scikit-Learn\nStochastic GD\nFast\nYes\nFast\n\u22652\nYes\nSGDRegressor\nMini-batch GD\nFast\nYes\nFast\n\u22652\nYes\nn/a\nThere is almost no difference after training: all these algorithms\nend up with very similar models and make predictions in exactly \nthe same way.\nPolynomial Regression\nWhat if your data is actually more complex than a simple straight line? Surprisingly,\nyou can actually use a linear model to fit nonlinear data. A simple way to do this is to\nadd powers of each feature as new features, then train a linear model on this extended\nset of features. This technique is called Polynomial Regression.\nLet\u2019s look at an example. First, let\u2019s generate some nonlinear data, based on a simple\nquadratic equation9 (plus some noise; see Figure 4-12):\nm = 100\nX = 6 * np.random.rand(m, 1) - 3\ny = 0.5 * X**2 + X + 2 + np.random.randn(m, 1)", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 358, "content": "m = 100\nX = 6 * np.random.rand(m, 1) - 3\ny = 0.5 * X**2 + X + 2 + np.random.randn(m, 1)\nFigure 4-12. Generated nonlinear and noisy dataset\nPolynomial Regression \n| \n121\nDownload from finelybook www.finelybook.com", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 359, "content": "Clearly, a straight line will never fit this data properly. So let\u2019s use Scikit-Learn\u2019s Poly\nnomialFeatures class to transform our training data, adding the square (2nd-degree\npolynomial) of each feature in the training set as new features (in this case there is\njust one feature):\n>>> from sklearn.preprocessing import PolynomialFeatures\n>>> poly_features = PolynomialFeatures(degree=2, include_bias=False)\n>>> X_poly = poly_features.fit_transform(X)\n>>> X[0]\narray([-0.75275929])\n>>> X_poly[0]\narray([-0.75275929, 0.56664654])\nX_poly now contains the original feature of X plus the square of this feature. Now you\ncan fit a LinearRegression model to this extended training data (Figure 4-13):\n>>> lin_reg = LinearRegression()\n>>> lin_reg.fit(X_poly, y)\n>>> lin_reg.intercept_, lin_reg.coef_\n(array([ 1.78134581]), array([[ 0.93366893, 0.56456263]]))\nFigure 4-13. Polynomial Regression model predictions\nNot bad: the model estimates y = 0 . 56x1\n2 + 0 . 93x1 + 1 . 78 when in fact the original", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 360, "content": "Not bad: the model estimates y = 0 . 56x1\n2 + 0 . 93x1 + 1 . 78 when in fact the original\nfunction was y = 0 . 5x1\n2 + 1 . 0x1 + 2 . 0 + Gaussian noise.\nNote that when there are multiple features, Polynomial Regression is capable of find\u2010\ning relationships between features (which is something a plain Linear Regression\nmodel cannot do). This is made possible by the fact that PolynomialFeatures also\nadds all combinations of features up to the given degree. For example, if there were\n122 \n| \nChapter 4: Training Models\nDownload from finelybook www.finelybook.com", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 361, "content": "two features a and b, PolynomialFeatures with degree=3 would not only add the\nfeatures a2, a3, b2, and b3, but also the combinations ab, a2b, and ab2.\nPolynomialFeatures(degree=d) transforms an array containing n\nfeatures into an array containing n + d !\nd! n! features, where n! is the\nfactorial of n, equal to 1 \u00d7 2 \u00d7 3 \u00d7 \u22ef \u00d7 n. Beware of the combinato\u2010\nrial explosion of the number of features!\nLearning Curves\nIf you perform high-degree Polynomial Regression, you will likely fit the training\ndata much better than with plain Linear Regression. For example, Figure 4-14 applies\na 300-degree polynomial model to the preceding training data, and compares the\nresult with a pure linear model and a quadratic model (2nd-degree polynomial).\nNotice how the 300-degree polynomial model wiggles around to get as close as possi\u2010\nble to the training instances.\nFigure 4-14. High-degree Polynomial Regression\nOf course, this high-degree Polynomial Regression model is severely overfitting the", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 362, "content": "Of course, this high-degree Polynomial Regression model is severely overfitting the\ntraining data, while the linear model is underfitting it. The model that will generalize\nbest in this case is the quadratic model. It makes sense since the data was generated\nusing a quadratic model, but in general you won\u2019t know what function generated the\ndata, so how can you decide how complex your model should be? How can you tell\nthat your model is overfitting or underfitting the data?\nLearning Curves \n| \n123\nDownload from finelybook www.finelybook.com", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 363, "content": "In Chapter 2 you used cross-validation to get an estimate of a model\u2019s generalization\nperformance. If a model performs well on the training data but generalizes poorly\naccording to the cross-validation metrics, then your model is overfitting. If it per\u2010\nforms poorly on both, then it is underfitting. This is one way to tell when a model is\ntoo simple or too complex.\nAnother way is to look at the learning curves: these are plots of the model\u2019s perfor\u2010\nmance on the training set and the validation set as a function of the training set size.\nTo generate the plots, simply train the model several times on different sized subsets\nof the training set. The following code defines a function that plots the learning\ncurves of a model given some training data:\nfrom sklearn.metrics import mean_squared_error\nfrom sklearn.model_selection import train_test_split\ndef plot_learning_curves(model, X, y):\n X_train, X_val, y_train, y_val = train_test_split(X, y, test_size=0.2)", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 364, "content": "X_train, X_val, y_train, y_val = train_test_split(X, y, test_size=0.2)\n train_errors, val_errors = [], []\n for m in range(1, len(X_train)):\n model.fit(X_train[:m], y_train[:m])\n y_train_predict = model.predict(X_train[:m])\n y_val_predict = model.predict(X_val)\n train_errors.append(mean_squared_error(y_train_predict, y_train[:m]))\n val_errors.append(mean_squared_error(y_val_predict, y_val))\n plt.plot(np.sqrt(train_errors), \"r-+\", linewidth=2, label=\"train\")\n plt.plot(np.sqrt(val_errors), \"b-\", linewidth=3, label=\"val\")\nLet\u2019s look at the learning curves of the plain Linear Regression model (a straight line;\nFigure 4-15):\nlin_reg = LinearRegression()\nplot_learning_curves(lin_reg, X, y)\nFigure 4-15. Learning curves\n124 \n| \nChapter 4: Training Models\nDownload from finelybook www.finelybook.com", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 365, "content": "This deserves a bit of explanation. First, let\u2019s look at the performance on the training\ndata: when there are just one or two instances in the training set, the model can fit\nthem perfectly, which is why the curve starts at zero. But as new instances are added\nto the training set, it becomes impossible for the model to fit the training data per\u2010\nfectly, both because the data is noisy and because it is not linear at all. So the error on\nthe training data goes up until it reaches a plateau, at which point adding new instan\u2010\nces to the training set doesn\u2019t make the average error much better or worse. Now let\u2019s\nlook at the performance of the model on the validation data. When the model is\ntrained on very few training instances, it is incapable of generalizing properly, which\nis why the validation error is initially quite big. Then as the model is shown more\ntraining examples, it learns and thus the validation error slowly goes down. However,", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 366, "content": "training examples, it learns and thus the validation error slowly goes down. However,\nonce again a straight line cannot do a good job modeling the data, so the error ends\nup at a plateau, very close to the other curve.\nThese learning curves are typical of an underfitting model. Both curves have reached\na plateau; they are close and fairly high.\nIf your model is underfitting the training data, adding more train\u2010\ning examples will not help. You need to use a more complex model\nor come up with better features.\nNow let\u2019s look at the learning curves of a 10th-degree polynomial model on the same\ndata (Figure 4-16):\nfrom sklearn.pipeline import Pipeline\npolynomial_regression = Pipeline((\n (\"poly_features\", PolynomialFeatures(degree=10, include_bias=False)),\n (\"sgd_reg\", LinearRegression()),\n ))\nplot_learning_curves(polynomial_regression, X, y)\nThese learning curves look a bit like the previous ones, but there are two very impor\u2010\ntant differences:", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 367, "content": "tant differences:\n\u2022 The error on the training data is much lower than with the Linear Regression\nmodel.\n\u2022 There is a gap between the curves. This means that the model performs signifi\u2010\ncantly better on the training data than on the validation data, which is the hall\u2010\nmark of an overfitting model. However, if you used a much larger training set,\nthe two curves would continue to get closer.\nLearning Curves \n| \n125\nDownload from finelybook www.finelybook.com", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 368, "content": "10 This notion of bias is not to be confused with the bias term of linear models.\nFigure 4-16. Learning curves for the polynomial model\nOne way to improve an overfitting model is to feed it more training\ndata until the validation error reaches the training error.\nThe Bias/Variance Tradeoff\nAn important theoretical result of statistics and Machine Learning is the fact that a\nmodel\u2019s generalization error can be expressed as the sum of three very different\nerrors:\nBias\nThis part of the generalization error is due to wrong assumptions, such as assum\u2010\ning that the data is linear when it is actually quadratic. A high-bias model is most\nlikely to underfit the training data.10\nVariance\nThis part is due to the model\u2019s excessive sensitivity to small variations in the\ntraining data. A model with many degrees of freedom (such as a high-degree pol\u2010\nynomial model) is likely to have high variance, and thus to overfit the training\ndata.\n126 \n| \nChapter 4: Training Models", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 369, "content": "data.\n126 \n| \nChapter 4: Training Models\nDownload from finelybook www.finelybook.com", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 370, "content": "Irreducible error\nThis part is due to the noisiness of the data itself. The only way to reduce this\npart of the error is to clean up the data (e.g., fix the data sources, such as broken\nsensors, or detect and remove outliers).\nIncreasing a model\u2019s complexity will typically increase its variance and reduce its bias.\nConversely, reducing a model\u2019s complexity increases its bias and reduces its variance. \nThis is why it is called a tradeoff.\nRegularized Linear Models\nAs we saw in Chapters 1 and 2, a good way to reduce overfitting is to regularize the\nmodel (i.e., to constrain it): the fewer degrees of freedom it has, the harder it will be\nfor it to overfit the data. For example, a simple way to regularize a polynomial model\nis to reduce the number of polynomial degrees.\nFor a linear model, regularization is typically achieved by constraining the weights of\nthe model. We will now look at Ridge Regression, Lasso Regression, and Elastic Net,", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 371, "content": "the model. We will now look at Ridge Regression, Lasso Regression, and Elastic Net,\nwhich implement three different ways to constrain the weights.\nRidge Regression\nRidge Regression (also called Tikhonov regularization) is a regularized version of Lin\u2010\near Regression: a regularization term equal to \u03b1\u2211i = 1\nn\n\u03b8i\n2 is added to the cost function. \nThis forces the learning algorithm to not only fit the data but also keep the model\nweights as small as possible. Note that the regularization term should only be added\nto the cost function during training. Once the model is trained, you want to evaluate\nthe model\u2019s performance using the unregularized performance measure.\nIt is quite common for the cost function used during training to be\ndifferent from the performance measure used for testing. Apart\nfrom regularization, another reason why they might be different is\nthat a good training cost function should have optimization-\nfriendly derivatives, while the performance measure used for test\u2010", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 372, "content": "friendly derivatives, while the performance measure used for test\u2010\ning should be as close as possible to the final objective. A good\nexample of this is a classifier trained using a cost function such as\nthe log loss (discussed in a moment) but evaluated using precision/\nrecall.\nThe hyperparameter \u03b1 controls how much you want to regularize the model. If \u03b1 = 0\nthen Ridge Regression is just Linear Regression. If \u03b1 is very large, then all weights end\nRegularized Linear Models \n| \n127\nDownload from finelybook www.finelybook.com", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 373, "content": "11 It is common to use the notation J(\u03b8) for cost functions that don\u2019t have a short name; we will often use this\nnotation throughout the rest of this book. The context will make it clear which cost function is being dis\u2010\ncussed.\n12 Norms are discussed in Chapter 2.\n13 A square matrix full of 0s except for 1s on the main diagonal (top-left to bottom-right).\nup very close to zero and the result is a flat line going through the data\u2019s mean. Equa\u2010\ntion 4-8 presents the Ridge Regression cost function.11\nEquation 4-8. Ridge Regression cost function\nJ \u03b8 = MSE \u03b8 + \u03b11\n2 \u2211\ni = 1\nn\n\u03b8i\n2\nNote that the bias term \u03b80 is not regularized (the sum starts at i = 1, not 0). If we\ndefine w as the vector of feature weights (\u03b81 to \u03b8n), then the regularization term is\nsimply equal to \u00bd(\u2225 w \u22252)2, where \u2225 \u00b7 \u22252 represents the \u21132 norm of the weight vector.12\nFor Gradient Descent, just add \u03b1w to the MSE gradient vector (Equation 4-6).\nIt is important to scale the data (e.g., using a StandardScaler)", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 374, "content": "It is important to scale the data (e.g., using a StandardScaler) \nbefore performing Ridge Regression, as it is sensitive to the scale of\nthe input features. This is true of most regularized models.\nFigure 4-17 shows several Ridge models trained on some linear data using different \u03b1\nvalue. On the left, plain Ridge models are used, leading to linear predictions. On the\nright, the data is first expanded using PolynomialFeatures(degree=10), then it is\nscaled using a StandardScaler, and finally the Ridge models are applied to the result\u2010\ning features: this is Polynomial Regression with Ridge regularization. Note how\nincreasing \u03b1 leads to flatter (i.e., less extreme, more reasonable) predictions; this\nreduces the model\u2019s variance but increases its bias.\nAs with Linear Regression, we can perform Ridge Regression either by computing a \nclosed-form equation or by performing Gradient Descent. The pros and cons are the", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 375, "content": "closed-form equation or by performing Gradient Descent. The pros and cons are the\nsame. Equation 4-9 shows the closed-form solution (where A is the n \u00d7 n identity\nmatrix13 except with a 0 in the top-left cell, corresponding to the bias term).\n128 \n| \nChapter 4: Training Models\nDownload from finelybook www.finelybook.com", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 376, "content": "14 Alternatively you can use the Ridge class with the \"sag\" solver. Stochastic Average GD is a variant of SGD.\nFor more details, see the presentation \u201cMinimizing Finite Sums with the Stochastic Average Gradient Algo\u2010\nrithm\u201d by Mark Schmidt et al. from the University of British Columbia.\nFigure 4-17. Ridge Regression\nEquation 4-9. Ridge Regression closed-form solution\n\u03b8 = \ufffdT \u00b7 \ufffd+ \u03b1\ufffd\n\u22121 \u00b7 \ufffdT \u00b7 \ufffd\nHere is how to perform Ridge Regression with Scikit-Learn using a closed-form solu\u2010\ntion (a variant of Equation 4-9 using a matrix factorization technique by Andr\u00e9-Louis\nCholesky):\n>>> from sklearn.linear_model import Ridge\n>>> ridge_reg = Ridge(alpha=1, solver=\"cholesky\")\n>>> ridge_reg.fit(X, y)\n>>> ridge_reg.predict([[1.5]])\narray([[ 1.55071465]])\nAnd using Stochastic Gradient Descent:14\n>>> sgd_reg = SGDRegressor(penalty=\"l2\")\n>>> sgd_reg.fit(X, y.ravel())\n>>> sgd_reg.predict([[1.5]])\narray([[ 1.13500145]])\nThe penalty hyperparameter sets the type of regularization term to use. Specifying", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 377, "content": "The penalty hyperparameter sets the type of regularization term to use. Specifying\n\"l2\" indicates that you want SGD to add a regularization term to the cost function \nequal to half the square of the \u21132 norm of the weight vector: this is simply Ridge\nRegression.\nRegularized Linear Models \n| \n129\nDownload from finelybook www.finelybook.com", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 378, "content": "Lasso Regression\nLeast Absolute Shrinkage and Selection Operator Regression (simply called Lasso\nRegression) is another regularized version of Linear Regression: just like Ridge\nRegression, it adds a regularization term to the cost function, but it uses the \u21131 norm\nof the weight vector instead of half the square of the \u21132 norm (see Equation 4-10).\nEquation 4-10. Lasso Regression cost function\nJ \u03b8 = MSE \u03b8 + \u03b1 \u2211\ni = 1\nn\n\u03b8i\nFigure 4-18 shows the same thing as Figure 4-17 but replaces Ridge models with\nLasso models and uses smaller \u03b1 values.\nFigure 4-18. Lasso Regression\nAn important characteristic of Lasso Regression is that it tends to completely elimi\u2010\nnate the weights of the least important features (i.e., set them to zero). For example,\nthe dashed line in the right plot on Figure 4-18 (with \u03b1 = 10-7) looks quadratic, almost\nlinear: all the weights for the high-degree polynomial features are equal to zero. In", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 379, "content": "linear: all the weights for the high-degree polynomial features are equal to zero. In\nother words, Lasso Regression automatically performs feature selection and outputs a\nsparse model (i.e., with few nonzero feature weights).\nYou can get a sense of why this is the case by looking at Figure 4-19: on the top-left\nplot, the background contours (ellipses) represent an unregularized MSE cost func\u2010\ntion (\u03b1 = 0), and the white circles show the Batch Gradient Descent path with that\ncost function. The foreground contours (diamonds) represent the \u21131 penalty, and the\ntriangles show the BGD path for this penalty only (\u03b1 \u2192 \u221e). Notice how the path first\n130 \n| \nChapter 4: Training Models\nDownload from finelybook www.finelybook.com", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 380, "content": "15 You can think of a subgradient vector at a nondifferentiable point as an intermediate vector between the gra\u2010\ndient vectors around that point.\nreaches \u03b81 = 0, then rolls down a gutter until it reaches \u03b82 = 0. On the top-right plot,\nthe contours represent the same cost function plus an \u21131 penalty with \u03b1 = 0.5. The\nglobal minimum is on the \u03b82 = 0 axis. BGD first reaches \u03b82 = 0, then rolls down the\ngutter until it reaches the global minimum. The two bottom plots show the same\nthing but uses an \u21132 penalty instead. The regularized minimum is closer to \u03b8 = 0 than\nthe unregularized minimum, but the weights do not get fully eliminated.\nFigure 4-19. Lasso versus Ridge regularization\nOn the Lasso cost function, the BGD path tends to bounce across\nthe gutter toward the end. This is because the slope changes\nabruptly at \u03b82 = 0. You need to gradually reduce the learning rate in\norder to actually converge to the global minimum.", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 381, "content": "order to actually converge to the global minimum.\nThe Lasso cost function is not differentiable at \u03b8i = 0 (for i = 1, 2, \u22ef, n), but Gradient\nDescent still works fine if you use a subgradient vector g15 instead when any \u03b8i = 0.\nEquation 4-11 shows a subgradient vector equation you can use for Gradient Descent\nwith the Lasso cost function.\nRegularized Linear Models \n| \n131\nDownload from finelybook www.finelybook.com", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 382, "content": "Equation 4-11. Lasso Regression subgradient vector\ng \u03b8, J = \u2207\u03b8 MSE \u03b8 + \u03b1\nsign \u03b81\nsign \u03b82\n\u22ee\nsign \u03b8n\nwhere sign \u03b8i =\n\u22121 if \u03b8i < 0\n0\nif \u03b8i = 0\n+1 if \u03b8i > 0\nHere is a small Scikit-Learn example using the Lasso class. Note that you could\ninstead use an SGDRegressor(penalty=\"l1\").\n>>> from sklearn.linear_model import Lasso\n>>> lasso_reg = Lasso(alpha=0.1)\n>>> lasso_reg.fit(X, y)\n>>> lasso_reg.predict([[1.5]])\narray([ 1.53788174])\nElastic Net\nElastic Net is a middle ground between Ridge Regression and Lasso Regression. The\nregularization term is a simple mix of both Ridge and Lasso\u2019s regularization terms,\nand you can control the mix ratio r. When r = 0, Elastic Net is equivalent to Ridge\nRegression, and when r = 1, it is equivalent to Lasso Regression (see Equation 4-12).\nEquation 4-12. Elastic Net cost function\nJ \u03b8 = MSE \u03b8 + r\u03b1 \u2211\ni = 1\nn\n\u03b8i + 1 \u2212r\n2\n\u03b1 \u2211\ni = 1\nn\n\u03b8i\n2\nSo when should you use Linear Regression, Ridge, Lasso, or Elastic Net? It is almost", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 383, "content": "i = 1\nn\n\u03b8i\n2\nSo when should you use Linear Regression, Ridge, Lasso, or Elastic Net? It is almost\nalways preferable to have at least a little bit of regularization, so generally you should\navoid plain Linear Regression. Ridge is a good default, but if you suspect that only a\nfew features are actually useful, you should prefer Lasso or Elastic Net since they tend\nto reduce the useless features\u2019 weights down to zero as we have discussed. In general, \nElastic Net is preferred over Lasso since Lasso may behave erratically when the num\u2010\nber of features is greater than the number of training instances or when several fea\u2010\ntures are strongly correlated.\nHere is a short example using Scikit-Learn\u2019s ElasticNet (l1_ratio corresponds to\nthe mix ratio r):\n>>> from sklearn.linear_model import ElasticNet\n>>> elastic_net = ElasticNet(alpha=0.1, l1_ratio=0.5)\n>>> elastic_net.fit(X, y)\n>>> elastic_net.predict([[1.5]])\narray([ 1.54333232])\n132 \n| \nChapter 4: Training Models", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 384, "content": ">>> elastic_net.predict([[1.5]])\narray([ 1.54333232])\n132 \n| \nChapter 4: Training Models\nDownload from finelybook www.finelybook.com", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 385, "content": "Early Stopping\nA very different way to regularize iterative learning algorithms such as Gradient\nDescent is to stop training as soon as the validation error reaches a minimum. This is\ncalled early stopping. Figure 4-20 shows a complex model (in this case a high-degree\nPolynomial Regression model) being trained using Batch Gradient Descent. As the\nepochs go by, the algorithm learns and its prediction error (RMSE) on the training set\nnaturally goes down, and so does its prediction error on the validation set. However,\nafter a while the validation error stops decreasing and actually starts to go back up.\nThis indicates that the model has started to overfit the training data. With early stop\u2010\nping you just stop training as soon as the validation error reaches the minimum. It is\nsuch a simple and efficient regularization technique that Geoffrey Hinton called it a\n\u201cbeautiful free lunch.\u201d\nFigure 4-20. Early stopping regularization", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 386, "content": "\u201cbeautiful free lunch.\u201d\nFigure 4-20. Early stopping regularization\nWith Stochastic and Mini-batch Gradient Descent, the curves are\nnot so smooth, and it may be hard to know whether you have\nreached the minimum or not. One solution is to stop only after the\nvalidation error has been above the minimum for some time (when\nyou are confident that the model will not do any better), then roll\nback the model parameters to the point where the validation error\nwas at a minimum.\nHere is a basic implementation of early stopping:\nfrom sklearn.base import clone\nRegularized Linear Models \n| \n133\nDownload from finelybook www.finelybook.com", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 387, "content": "sgd_reg = SGDRegressor(n_iter=1, warm_start=True, penalty=None,\n learning_rate=\"constant\", eta0=0.0005)\nminimum_val_error = float(\"inf\")\nbest_epoch = None\nbest_model = None\nfor epoch in range(1000):\n sgd_reg.fit(X_train_poly_scaled, y_train) # continues where it left off\n y_val_predict = sgd_reg.predict(X_val_poly_scaled)\n val_error = mean_squared_error(y_val_predict, y_val)\n if val_error < minimum_val_error:\n minimum_val_error = val_error\n best_epoch = epoch\n best_model = clone(sgd_reg)\nNote that with warm_start=True, when the fit() method is called, it just continues\ntraining where it left off instead of restarting from scratch.\nLogistic Regression\nAs we discussed in Chapter 1, some regression algorithms can be used for classifica\u2010\ntion as well (and vice versa). Logistic Regression (also called Logit Regression) is com\u2010\nmonly used to estimate the probability that an instance belongs to a particular class", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 388, "content": "monly used to estimate the probability that an instance belongs to a particular class\n(e.g., what is the probability that this email is spam?). If the estimated probability is\ngreater than 50%, then the model predicts that the instance belongs to that class\n(called the positive class, labeled \u201c1\u201d), or else it predicts that it does not (i.e., it\nbelongs to the negative class, labeled \u201c0\u201d). This makes it a binary classifier.\nEstimating Probabilities\nSo how does it work? Just like a Linear Regression model, a Logistic Regression\nmodel computes a weighted sum of the input features (plus a bias term), but instead\nof outputting the result directly like the Linear Regression model does, it outputs the\nlogistic of this result (see Equation 4-13).\nEquation 4-13. Logistic Regression model estimated probability (vectorized form)\np = h\u03b8 \ufffd= \u03c3 \u03b8T \u00b7 \ufffd\nThe logistic\u2014also called the logit, noted \u03c3(\u00b7)\u2014is a sigmoid function (i.e., S-shaped)", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 389, "content": "The logistic\u2014also called the logit, noted \u03c3(\u00b7)\u2014is a sigmoid function (i.e., S-shaped)\nthat outputs a number between 0 and 1. It is defined as shown in Equation 4-14 and\nFigure 4-21.\n134 \n| \nChapter 4: Training Models\nDownload from finelybook www.finelybook.com", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 390, "content": "Equation 4-14. Logistic function\n\u03c3 t =\n1\n1 + exp\n\u2212t\nFigure 4-21. Logistic function\nOnce the Logistic Regression model has estimated the probability p = h\u03b8(x) that an\ninstance x belongs to the positive class, it can make its prediction \u0177 easily (see Equa\u2010\ntion 4-15).\nEquation 4-15. Logistic Regression model prediction\ny = 0 if p < 0 . 5,\n1 if p \u22650 . 5 .\nNotice that \u03c3(t) < 0.5 when t < 0, and \u03c3(t) \u2265 0.5 when t \u2265 0, so a Logistic Regression\nmodel predicts 1 if \u03b8T \u00b7 x is positive, and 0 if it is negative.\nTraining and Cost Function\nGood, now you know how a Logistic Regression model estimates probabilities and\nmakes predictions. But how is it trained? The objective of training is to set the param\u2010\neter vector \u03b8 so that the model estimates high probabilities for positive instances (y =\n1) and low probabilities for negative instances (y = 0). This idea is captured by the\ncost function shown in Equation 4-16 for a single training instance x.", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 391, "content": "cost function shown in Equation 4-16 for a single training instance x.\nEquation 4-16. Cost function of a single training instance\nc \u03b8 =\n\u2212log p\nif y = 1,\n\u2212log 1 \u2212p if y = 0 .\nThis cost function makes sense because \u2013 log(t) grows very large when t approaches\n0, so the cost will be large if the model estimates a probability close to 0 for a positive\nLogistic Regression \n| \n135\nDownload from finelybook www.finelybook.com", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 392, "content": "instance, and it will also be very large if the model estimates a probability close to 1\nfor a negative instance. On the other hand, \u2013 log(t) is close to 0 when t is close to 1, so\nthe cost will be close to 0 if the estimated probability is close to 0 for a negative\ninstance or close to 1 for a positive instance, which is precisely what we want.\nThe cost function over the whole training set is simply the average cost over all train\u2010\ning instances. It can be written in a single expression (as you can verify easily), called \nthe log loss, shown in Equation 4-17.\nEquation 4-17. Logistic Regression cost function (log loss)\nJ \u03b8 = \u22121\nm \u2211\ni = 1\nm\ny i log p i\n+ 1 \u2212y i log 1 \u2212p i\nThe bad news is that there is no known closed-form equation to compute the value of\n\u03b8 that minimizes this cost function (there is no equivalent of the Normal Equation).\nBut the good news is that this cost function is convex, so Gradient Descent (or any", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 393, "content": "But the good news is that this cost function is convex, so Gradient Descent (or any\nother optimization algorithm) is guaranteed to find the global minimum (if the learn\u2010\ning rate is not too large and you wait long enough). The partial derivatives of the cost\nfunction with regards to the jth model parameter \u03b8j is given by Equation 4-18.\nEquation 4-18. Logistic cost function partial derivatives\n\u2202\n\u2202\u03b8j\nJ \u03b8 = 1\nm \u2211\ni = 1\nm\n\u03c3 \u03b8T \u00b7 \ufffdi\n\u2212y i xj\ni\nThis equation looks very much like Equation 4-5: for each instance it computes the\nprediction error and multiplies it by the jth feature value, and then it computes the\naverage over all training instances. Once you have the gradient vector containing all\nthe partial derivatives you can use it in the Batch Gradient Descent algorithm. That\u2019s\nit: you now know how to train a Logistic Regression model. For Stochastic GD you\nwould of course just take one instance at a time, and for Mini-batch GD you would\nuse a mini-batch at a time.\nDecision Boundaries", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 394, "content": "use a mini-batch at a time.\nDecision Boundaries\nLet\u2019s use the iris dataset to illustrate Logistic Regression. This is a famous dataset that\ncontains the sepal and petal length and width of 150 iris flowers of three different\nspecies: Iris-Setosa, Iris-Versicolor, and Iris-Virginica (see Figure 4-22).\n136 \n| \nChapter 4: Training Models\nDownload from finelybook www.finelybook.com", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 395, "content": "16 Photos reproduced from the corresponding Wikipedia pages. Iris-Virginica photo by Frank Mayfield (Crea\u2010\ntive Commons BY-SA 2.0), Iris-Versicolor photo by D. Gordon E. Robertson (Creative Commons BY-SA 3.0),\nand Iris-Setosa photo is public domain.\nFigure 4-22. Flowers of three iris plant species16\nLet\u2019s try to build a classifier to detect the Iris-Virginica type based only on the petal\nwidth feature. First let\u2019s load the data:\n>>> from sklearn import datasets\n>>> iris = datasets.load_iris()\n>>> list(iris.keys())\n['data', 'target_names', 'feature_names', 'target', 'DESCR']\n>>> X = iris[\"data\"][:, 3:] # petal width\n>>> y = (iris[\"target\"] == 2).astype(np.int) # 1 if Iris-Virginica, else 0\nNow let\u2019s train a Logistic Regression model:\nfrom sklearn.linear_model import LogisticRegression\nlog_reg = LogisticRegression()\nlog_reg.fit(X, y)\nLet\u2019s look at the model\u2019s estimated probabilities for flowers with petal widths varying\nfrom 0 to 3 cm (Figure 4-23):", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 396, "content": "from 0 to 3 cm (Figure 4-23):\nX_new = np.linspace(0, 3, 1000).reshape(-1, 1)\ny_proba = log_reg.predict_proba(X_new)\nplt.plot(X_new, y_proba[:, 1], \"g-\", label=\"Iris-Virginica\")\nplt.plot(X_new, y_proba[:, 0], \"b--\", label=\"Not Iris-Virginica\")\n# + more Matplotlib code to make the image look pretty\nLogistic Regression \n| \n137\nDownload from finelybook www.finelybook.com", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 397, "content": "17 It is the the set of points x such that \u03b80 + \u03b81x1 + \u03b82x2 = 0, which defines a straight line.\nFigure 4-23. Estimated probabilities and decision boundary\nThe petal width of Iris-Virginica flowers (represented by triangles) ranges from 1.4\ncm to 2.5 cm, while the other iris flowers (represented by squares) generally have a\nsmaller petal width, ranging from 0.1 cm to 1.8 cm. Notice that there is a bit of over\u2010\nlap. Above about 2 cm the classifier is highly confident that the flower is an Iris-\nVirginica (it outputs a high probability to that class), while below 1 cm it is highly\nconfident that it is not an Iris-Virginica (high probability for the \u201cNot Iris-Virginica\u201d\nclass). In between these extremes, the classifier is unsure. However, if you ask it to\npredict the class (using the predict() method rather than the predict_proba()\nmethod), it will return whichever class is the most likely. Therefore, there is a decision", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 398, "content": "method), it will return whichever class is the most likely. Therefore, there is a decision\nboundary at around 1.6 cm where both probabilities are equal to 50%: if the petal\nwidth is higher than 1.6 cm, the classifier will predict that the flower is an Iris-\nVirginica, or else it will predict that it is not (even if it is not very confident):\n>>> log_reg.predict([[1.7], [1.5]])\narray([1, 0])\nFigure 4-24 shows the same dataset but this time displaying two features: petal width\nand length. Once trained, the Logistic Regression classifier can estimate the probabil\u2010\nity that a new flower is an Iris-Virginica based on these two features. The dashed line\nrepresents the points where the model estimates a 50% probability: this is the model\u2019s\ndecision boundary. Note that it is a linear boundary.17 Each parallel line represents the\npoints where the model outputs a specific probability, from 15% (bottom left) to 90%\n(top right). All the flowers beyond the top-right line have an over 90% chance of", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 399, "content": "(top right). All the flowers beyond the top-right line have an over 90% chance of\nbeing Iris-Virginica according to the model.\n138 \n| \nChapter 4: Training Models\nDownload from finelybook www.finelybook.com", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 400, "content": "Figure 4-24. Linear decision boundary\nJust like the other linear models, Logistic Regression models can be regularized using \n\u21131 or \u21132 penalties. Scitkit-Learn actually adds an \u21132 penalty by default.\nThe hyperparameter controlling the regularization strength of a\nScikit-Learn LogisticRegression model is not alpha (as in other\nlinear models), but its inverse: C. The higher the value of C, the less\nthe model is regularized.\nSoftmax Regression\nThe Logistic Regression model can be generalized to support multiple classes directly,\nwithout having to train and combine multiple binary classifiers (as discussed in\nChapter 3). This is called Softmax Regression, or Multinomial Logistic Regression.\nThe idea is quite simple: when given an instance x, the Softmax Regression model\nfirst computes a score sk(x) for each class k, then estimates the probability of each\nclass by applying the softmax function (also called the normalized exponential) to the", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 401, "content": "class by applying the softmax function (also called the normalized exponential) to the\nscores. The equation to compute sk(x) should look familiar, as it is just like the equa\u2010\ntion for Linear Regression prediction (see Equation 4-19).\nEquation 4-19. Softmax score for class k\nsk \ufffd= \u03b8k\nT \u00b7 \ufffd\nNote that each class has its own dedicated parameter vector \u03b8k. All these vectors are\ntypically stored as rows in a parameter matrix \u0398.\nOnce you have computed the score of every class for the instance x, you can estimate\nthe probability pk that the instance belongs to class k by running the scores through\nLogistic Regression \n| \n139\nDownload from finelybook www.finelybook.com", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 402, "content": "the softmax function (Equation 4-20): it computes the exponential of every score,\nthen normalizes them (dividing by the sum of all the exponentials).\nEquation 4-20. Softmax function\npk = \u03c3 \ufffd\ufffd\nk =\nexp sk \ufffd\n\u2211j = 1\nK\nexp sj \ufffd\n\u2022 K is the number of classes.\n\u2022 s(x) is a vector containing the scores of each class for the instance x.\n\u2022 \u03c3(s(x))k is the estimated probability that the instance x belongs to class k given\nthe scores of each class for that instance.\nJust like the Logistic Regression classifier, the Softmax Regression classifier predicts\nthe class with the highest estimated probability (which is simply the class with the\nhighest score), as shown in Equation 4-21.\nEquation 4-21. Softmax Regression classifier prediction\ny = argmax\nk\n\u03c3 \ufffd\ufffd\nk = argmax\nk\nsk \ufffd= argmax\nk\n\u03b8k\nT \u00b7 \ufffd\n\u2022 The argmax operator returns the value of a variable that maximizes a function. In\nthis equation, it returns the value of k that maximizes the estimated probability\n\u03c3(s(x))k.", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 403, "content": "this equation, it returns the value of k that maximizes the estimated probability\n\u03c3(s(x))k.\nThe Softmax Regression classifier predicts only one class at a time\n(i.e., it is multiclass, not multioutput) so it should be used only with\nmutually exclusive classes such as different types of plants. You\ncannot use it to recognize multiple people in one picture.\nNow that you know how the model estimates probabilities and makes predictions,\nlet\u2019s take a look at training. The objective is to have a model that estimates a high\nprobability for the target class (and consequently a low probability for the other\nclasses). Minimizing the cost function shown in Equation 4-22, called the cross\nentropy, should lead to this objective because it penalizes the model when it estimates\na low probability for a target class. Cross entropy is frequently used to measure how\nwell a set of estimated class probabilities match the target classes (we will use it again\nseveral times in the following chapters).\n140", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 404, "content": "several times in the following chapters).\n140 \n| \nChapter 4: Training Models\nDownload from finelybook www.finelybook.com", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 405, "content": "Equation 4-22. Cross entropy cost function\nJ \u0398 = \u22121\nm \u2211\ni = 1\nm\n\u2211\nk = 1\nK\nyk\ni log pk\ni\n\u2022 yk\ni is equal to 1 if the target class for the ith instance is k; otherwise, it is equal to\n0.\nNotice that when there are just two classes (K = 2), this cost function is equivalent to\nthe Logistic Regression\u2019s cost function (log loss; see Equation 4-17).\nCross Entropy\nCross entropy originated from information theory. Suppose you want to efficiently\ntransmit information about the weather every day. If there are eight options (sunny,\nrainy, etc.), you could encode each option using 3 bits since 23 = 8. However, if you\nthink it will be sunny almost every day, it would be much more efficient to code\n\u201csunny\u201d on just one bit (0) and the other seven options on 4 bits (starting with a 1).\nCross entropy measures the average number of bits you actually send per option. If\nyour assumption about the weather is perfect, cross entropy will just be equal to the", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 406, "content": "your assumption about the weather is perfect, cross entropy will just be equal to the\nentropy of the weather itself (i.e., its intrinsic unpredictability). But if your assump\u2010\ntions are wrong (e.g., if it rains often), cross entropy will be greater by an amount \ncalled the Kullback\u2013Leibler divergence.\nThe cross entropy between two probability distributions p and q is defined as\nH p, q = \u2212\u2211x p x log q x (at least when the distributions are discrete).\nThe gradient vector of this cost function with regards to \u03b8k is given by Equation 4-23:\nEquation 4-23. Cross entropy gradient vector for class k\n\u2207\u03b8k J \u0398 = 1\nm \u2211\ni = 1\nm\npk\ni \u2212yk\ni \ufffdi\nNow you can compute the gradient vector for every class, then use Gradient Descent\n(or any other optimization algorithm) to find the parameter matrix \u0398 that minimizes\nthe cost function.\nLet\u2019s use Softmax Regression to classify the iris flowers into all three classes. Scikit-\nLearn\u2019s LogisticRegression uses one-versus-all by default when you train it on more", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 407, "content": "Learn\u2019s LogisticRegression uses one-versus-all by default when you train it on more\nthan two classes, but you can set the multi_class hyperparameter to \"multinomial\"\nto switch it to Softmax Regression instead. You must also specify a solver that sup\u2010\nports Softmax Regression, such as the \"lbfgs\" solver (see Scikit-Learn\u2019s documenta\u2010\nLogistic Regression \n| \n141\nDownload from finelybook www.finelybook.com", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 408, "content": "tion for more details). It also applies \u21132 regularization by default, which you can\ncontrol using the hyperparameter C.\nX = iris[\"data\"][:, (2, 3)] # petal length, petal width\ny = iris[\"target\"]\nsoftmax_reg = LogisticRegression(multi_class=\"multinomial\",solver=\"lbfgs\", C=10)\nsoftmax_reg.fit(X, y)\nSo the next time you find an iris with 5 cm long and 2 cm wide petals, you can ask\nyour model to tell you what type of iris it is, and it will answer Iris-Virginica (class 2)\nwith 94.2% probability (or Iris-Versicolor with 5.8% probability):\n>>> softmax_reg.predict([[5, 2]])\narray([2])\n>>> softmax_reg.predict_proba([[5, 2]])\narray([[ 6.33134078e-07, 5.75276067e-02, 9.42471760e-01]])\nFigure 4-25 shows the resulting decision boundaries, represented by the background\ncolors. Notice that the decision boundaries between any two classes are linear. The\nfigure also shows the probabilities for the Iris-Versicolor class, represented by the", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 409, "content": "figure also shows the probabilities for the Iris-Versicolor class, represented by the\ncurved lines (e.g., the line labeled with 0.450 represents the 45% probability bound\u2010\nary). Notice that the model can predict a class that has an estimated probability below\n50%. For example, at the point where all decision boundaries meet, all classes have an\nequal estimated probability of 33%.\nFigure 4-25. Softmax Regression decision boundaries\nExercises\n1. What Linear Regression training algorithm can you use if you have a training set\nwith millions of features?\n2. Suppose the features in your training set have very different scales. What algo\u2010\nrithms might suffer from this, and how? What can you do about it?\n142 \n| \nChapter 4: Training Models\nDownload from finelybook www.finelybook.com", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 410, "content": "3. Can Gradient Descent get stuck in a local minimum when training a Logistic\nRegression model?\n4. Do all Gradient Descent algorithms lead to the same model provided you let\nthem run long enough?\n5. Suppose you use Batch Gradient Descent and you plot the validation error at\nevery epoch. If you notice that the validation error consistently goes up, what is\nlikely going on? How can you fix this?\n6. Is it a good idea to stop Mini-batch Gradient Descent immediately when the vali\u2010\ndation error goes up?\n7. Which Gradient Descent algorithm (among those we discussed) will reach the\nvicinity of the optimal solution the fastest? Which will actually converge? How\ncan you make the others converge as well?\n8. Suppose you are using Polynomial Regression. You plot the learning curves and\nyou notice that there is a large gap between the training error and the validation\nerror. What is happening? What are three ways to solve this?", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 411, "content": "error. What is happening? What are three ways to solve this?\n9. Suppose you are using Ridge Regression and you notice that the training error\nand the validation error are almost equal and fairly high. Would you say that the\nmodel suffers from high bias or high variance? Should you increase the regulari\u2010\nzation hyperparameter \u03b1 or reduce it?\n10. Why would you want to use:\n\u2022 Ridge Regression instead of Linear Regression?\n\u2022 Lasso instead of Ridge Regression?\n\u2022 Elastic Net instead of Lasso?\n11. Suppose you want to classify pictures as outdoor/indoor and daytime/nighttime.\nShould you implement two Logistic Regression classifiers or one Softmax Regres\u2010\nsion classifier?\n12. Implement Batch Gradient Descent with early stopping for Softmax Regression \n(without using Scikit-Learn).\nSolutions to these exercises are available in Appendix A.\nExercises \n| \n143\nDownload from finelybook www.finelybook.com", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 412, "content": "Download from finelybook www.finelybook.com", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 413, "content": "CHAPTER 5\nSupport Vector Machines\nA Support Vector Machine (SVM) is a very powerful and versatile Machine Learning\nmodel, capable of performing linear or nonlinear classification, regression, and even\noutlier detection. It is one of the most popular models in Machine Learning, and any\u2010\none interested in Machine Learning should have it in their toolbox. SVMs are partic\u2010\nularly well suited for classification of complex but small- or medium-sized datasets.\nThis chapter will explain the core concepts of SVMs, how to use them, and how they\nwork.\nLinear SVM Classification\nThe fundamental idea behind SVMs is best explained with some pictures. Figure 5-1\nshows part of the iris dataset that was introduced at the end of Chapter 4. The two\nclasses can clearly be separated easily with a straight line (they are linearly separable).\nThe left plot shows the decision boundaries of three possible linear classifiers. The\nmodel whose decision boundary is represented by the dashed line is so bad that it", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 414, "content": "model whose decision boundary is represented by the dashed line is so bad that it\ndoes not even separate the classes properly. The other two models work perfectly on\nthis training set, but their decision boundaries come so close to the instances that\nthese models will probably not perform as well on new instances. In contrast, the\nsolid line in the plot on the right represents the decision boundary of an SVM classi\u2010\nfier; this line not only separates the two classes but also stays as far away from the\nclosest training instances as possible. You can think of an SVM classifier as fitting the\nwidest possible street (represented by the parallel dashed lines) between the classes.\nThis is called large margin classification.\n145\nDownload from finelybook www.finelybook.com", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 415, "content": "Figure 5-1. Large margin classification\nNotice that adding more training instances \u201coff the street\u201d will not affect the decision\nboundary at all: it is fully determined (or \u201csupported\u201d) by the instances located on the\nedge of the street. These instances are called the support vectors (they are circled in\nFigure 5-1).\nSVMs are sensitive to the feature scales, as you can see in\nFigure 5-2: on the left plot, the vertical scale is much larger than the\nhorizontal scale, so the widest possible street is close to horizontal.\nAfter feature scaling (e.g., using Scikit-Learn\u2019s StandardScaler), \nthe decision boundary looks much better (on the right plot).\nFigure 5-2. Sensitivity to feature scales\nSoft Margin Classification\nIf we strictly impose that all instances be off the street and on the right side, this is\ncalled hard margin classification. There are two main issues with hard margin classifi\u2010\ncation. First, it only works if the data is linearly separable, and second it is quite sensi\u2010", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 416, "content": "cation. First, it only works if the data is linearly separable, and second it is quite sensi\u2010\ntive to outliers. Figure 5-3 shows the iris dataset with just one additional outlier: on\nthe left, it is impossible to find a hard margin, and on the right the decision boundary\nends up very different from the one we saw in Figure 5-1 without the outlier, and it\nwill probably not generalize as well.\n146 \n| \nChapter 5: Support Vector Machines\nDownload from finelybook www.finelybook.com", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 417, "content": "Figure 5-3. Hard margin sensitivity to outliers\nTo avoid these issues it is preferable to use a more flexible model. The objective is to\nfind a good balance between keeping the street as large as possible and limiting the\nmargin violations (i.e., instances that end up in the middle of the street or even on the\nwrong side). This is called soft margin classification.\nIn Scikit-Learn\u2019s SVM classes, you can control this balance using the C hyperparame\u2010\nter: a smaller C value leads to a wider street but more margin violations. Figure 5-4\nshows the decision boundaries and margins of two soft margin SVM classifiers on a\nnonlinearly separable dataset. On the left, using a high C value the classifier makes\nfewer margin violations but ends up with a smaller margin. On the right, using a low\nC value the margin is much larger, but many instances end up on the street. However,\nit seems likely that the second classifier will generalize better: in fact even on this", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 418, "content": "it seems likely that the second classifier will generalize better: in fact even on this\ntraining set it makes fewer prediction errors, since most of the margin violations are\nactually on the correct side of the decision boundary.\nFigure 5-4. Fewer margin violations versus large margin\nIf your SVM model is overfitting, you can try regularizing it by\nreducing C.\nThe following Scikit-Learn code loads the iris dataset, scales the features, and then\ntrains a linear SVM model (using the LinearSVC class with C = 0.1 and the hinge loss\nLinear SVM Classification \n| \n147\nDownload from finelybook www.finelybook.com", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 419, "content": "function, described shortly) to detect Iris-Virginica flowers. The resulting model is\nrepresented on the right of Figure 5-4.\nimport numpy as np\nfrom sklearn import datasets\nfrom sklearn.pipeline import Pipeline\nfrom sklearn.preprocessing import StandardScaler\nfrom sklearn.svm import LinearSVC\niris = datasets.load_iris()\nX = iris[\"data\"][:, (2, 3)] # petal length, petal width\ny = (iris[\"target\"] == 2).astype(np.float64) # Iris-Virginica\nsvm_clf = Pipeline((\n (\"scaler\", StandardScaler()),\n (\"linear_svc\", LinearSVC(C=1, loss=\"hinge\")),\n ))\nsvm_clf.fit(X_scaled, y)\nThen, as usual, you can use the model to make predictions:\n>>> svm_clf.predict([[5.5, 1.7]])\narray([ 1.])\nUnlike Logistic Regression classifiers, SVM classifiers do not out\u2010\nput probabilities for each class.\nAlternatively, you could use the SVC class, using SVC(kernel=\"linear\", C=1), but it\nis much slower, especially with large training sets, so it is not recommended. Another", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 420, "content": "is much slower, especially with large training sets, so it is not recommended. Another\noption is to use the SGDClassifier class, with SGDClassifier(loss=\"hinge\",\nalpha=1/(m*C)). This applies regular Stochastic Gradient Descent (see Chapter 4) to\ntrain a linear SVM classifier. It does not converge as fast as the LinearSVC class, but it\ncan be useful to handle huge datasets that do not fit in memory (out-of-core train\u2010\ning), or to handle online classification tasks.\nThe LinearSVC class regularizes the bias term, so you should center\nthe training set first by subtracting its mean. This is automatic if\nyou scale the data using the StandardScaler. Moreover, make sure\nyou set the loss hyperparameter to \"hinge\", as it is not the default\nvalue. Finally, for better performance you should set the dual\nhyperparameter to False, unless there are more features than\ntraining instances (we will discuss duality later in the chapter).\n148 \n| \nChapter 5: Support Vector Machines", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 421, "content": "148 \n| \nChapter 5: Support Vector Machines\nDownload from finelybook www.finelybook.com", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 422, "content": "Nonlinear SVM Classification\nAlthough linear SVM classifiers are efficient and work surprisingly well in many\ncases, many datasets are not even close to being linearly separable. One approach to\nhandling nonlinear datasets is to add more features, such as polynomial features (as\nyou did in Chapter 4); in some cases this can result in a linearly separable dataset.\nConsider the left plot in Figure 5-5: it represents a simple dataset with just one feature\nx1. This dataset is not linearly separable, as you can see. But if you add a second fea\u2010\nture x2 = (x1)2, the resulting 2D dataset is perfectly linearly separable.\nFigure 5-5. Adding features to make a dataset linearly separable\nTo implement this idea using Scikit-Learn, you can create a Pipeline containing a\nPolynomialFeatures transformer (discussed in \u201cPolynomial Regression\u201d on page\n121), followed by a StandardScaler and a LinearSVC. Let\u2019s test this on the moons\ndataset (see Figure 5-6):\nfrom sklearn.datasets import make_moons", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 423, "content": "dataset (see Figure 5-6):\nfrom sklearn.datasets import make_moons\nfrom sklearn.pipeline import Pipeline\nfrom sklearn.preprocessing import PolynomialFeatures\npolynomial_svm_clf = Pipeline((\n (\"poly_features\", PolynomialFeatures(degree=3)),\n (\"scaler\", StandardScaler()),\n (\"svm_clf\", LinearSVC(C=10, loss=\"hinge\"))\n ))\npolynomial_svm_clf.fit(X, y)\nNonlinear SVM Classification \n| \n149\nDownload from finelybook www.finelybook.com", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 424, "content": "Figure 5-6. Linear SVM classifier using polynomial features\nPolynomial Kernel\nAdding polynomial features is simple to implement and can work great with all sorts\nof Machine Learning algorithms (not just SVMs), but at a low polynomial degree it\ncannot deal with very complex datasets, and with a high polynomial degree it creates\na huge number of features, making the model too slow.\nFortunately, when using SVMs you can apply an almost miraculous mathematical\ntechnique called the kernel trick (it is explained in a moment). It makes it possible to\nget the same result as if you added many polynomial features, even with very high-\ndegree polynomials, without actually having to add them. So there is no combinato\u2010\nrial explosion of the number of features since you don\u2019t actually add any features. This\ntrick is implemented by the SVC class. Let\u2019s test it on the moons dataset:\nfrom sklearn.svm import SVC\npoly_kernel_svm_clf = Pipeline((\n (\"scaler\", StandardScaler()),", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 425, "content": "from sklearn.svm import SVC\npoly_kernel_svm_clf = Pipeline((\n (\"scaler\", StandardScaler()),\n (\"svm_clf\", SVC(kernel=\"poly\", degree=3, coef0=1, C=5))\n ))\npoly_kernel_svm_clf.fit(X, y)\nThis code trains an SVM classifier using a 3rd-degree polynomial kernel. It is repre\u2010\nsented on the left of Figure 5-7. On the right is another SVM classifier using a 10th-\ndegree polynomial kernel. Obviously, if your model is overfitting, you might want to\n150 \n| \nChapter 5: Support Vector Machines\nDownload from finelybook www.finelybook.com", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 426, "content": "reduce the polynomial degree. Conversely, if it is underfitting, you can try increasing\nit. The hyperparameter coef0 controls how much the model is influenced by high-\ndegree polynomials versus low-degree polynomials.\nFigure 5-7. SVM classifiers with a polynomial kernel\nA common approach to find the right hyperparameter values is to\nuse grid search (see Chapter 2). It is often faster to first do a very\ncoarse grid search, then a finer grid search around the best values\nfound. Having a good sense of what each hyperparameter actually\ndoes can also help you search in the right part of the hyperparame\u2010\nter space.\nAdding Similarity Features\nAnother technique to tackle nonlinear problems is to add features computed using a\nsimilarity function that measures how much each instance resembles a particular\nlandmark. For example, let\u2019s take the one-dimensional dataset discussed earlier and\nadd two landmarks to it at x1 = \u20132 and x1 = 1 (see the left plot in Figure 5-8). Next,", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 427, "content": "add two landmarks to it at x1 = \u20132 and x1 = 1 (see the left plot in Figure 5-8). Next,\nlet\u2019s define the similarity function to be the Gaussian Radial Basis Function (RBF)\nwith \u03b3 = 0.3 (see Equation 5-1).\nEquation 5-1. Gaussian RBF\n\u03d5\u03b3 \ufffd, \u2113= exp \u2212\u03b3\u2225\ufffd\u2212\u2113\u22252\nIt is a bell-shaped function varying from 0 (very far away from the landmark) to 1 (at\nthe landmark). Now we are ready to compute the new features. For example, let\u2019s look\nat the instance x1 = \u20131: it is located at a distance of 1 from the first landmark, and 2\nfrom the second landmark. Therefore its new features are x2 = exp (\u20130.3 \u00d7 12) \u2248 0.74\nand x3 = exp (\u20130.3 \u00d7 22) \u2248 0.30. The plot on the right of Figure 5-8 shows the trans\u2010\nformed dataset (dropping the original features). As you can see, it is now linearly\nseparable.\nNonlinear SVM Classification \n| \n151\nDownload from finelybook www.finelybook.com", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 428, "content": "Figure 5-8. Similarity features using the Gaussian RBF\nYou may wonder how to select the landmarks. The simplest approach is to create a\nlandmark at the location of each and every instance in the dataset. This creates many\ndimensions and thus increases the chances that the transformed training set will be\nlinearly separable. The downside is that a training set with m instances and n features\ngets transformed into a training set with m instances and m features (assuming you\ndrop the original features). If your training set is very large, you end up with an\nequally large number of features.\nGaussian RBF Kernel\nJust like the polynomial features method, the similarity features method can be useful\nwith any Machine Learning algorithm, but it may be computationally expensive to\ncompute all the additional features, especially on large training sets. However, once\nagain the kernel trick does its SVM magic: it makes it possible to obtain a similar", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 429, "content": "again the kernel trick does its SVM magic: it makes it possible to obtain a similar\nresult as if you had added many similarity features, without actually having to add\nthem. Let\u2019s try the Gaussian RBF kernel using the SVC class:\nrbf_kernel_svm_clf = Pipeline((\n (\"scaler\", StandardScaler()),\n (\"svm_clf\", SVC(kernel=\"rbf\", gamma=5, C=0.001))\n ))\nrbf_kernel_svm_clf.fit(X, y)\nThis model is represented on the bottom left of Figure 5-9. The other plots show\nmodels trained with different values of hyperparameters gamma (\u03b3) and C. Increasing\ngamma makes the bell-shape curve narrower (see the left plot of Figure 5-8), and as a\nresult each instance\u2019s range of influence is smaller: the decision boundary ends up\nbeing more irregular, wiggling around individual instances. Conversely, a small gamma \nvalue makes the bell-shaped curve wider, so instances have a larger range of influ\u2010\nence, and the decision boundary ends up smoother. So \u03b3 acts like a regularization", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 430, "content": "ence, and the decision boundary ends up smoother. So \u03b3 acts like a regularization\nhyperparameter: if your model is overfitting, you should reduce it, and if it is under\u2010\nfitting, you should increase it (similar to the C hyperparameter).\n152 \n| \nChapter 5: Support Vector Machines\nDownload from finelybook www.finelybook.com", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 431, "content": "1 \u201cA Dual Coordinate Descent Method for Large-scale Linear SVM,\u201d Lin et al. (2008).\nFigure 5-9. SVM classifiers using an RBF kernel\nOther kernels exist but are used much more rarely. For example, some kernels are\nspecialized for specific data structures. String kernels are sometimes used when classi\u2010\nfying text documents or DNA sequences (e.g., using the string subsequence kernel or\nkernels based on the Levenshtein distance).\nWith so many kernels to choose from, how can you decide which\none to use? As a rule of thumb, you should always try the linear\nkernel first (remember that LinearSVC is much faster than SVC(ker\nnel=\"linear\")), especially if the training set is very large or if it\nhas plenty of features. If the training set is not too large, you should\ntry the Gaussian RBF kernel as well; it works well in most cases.\nThen if you have spare time and computing power, you can also\nexperiment with a few other kernels using cross-validation and grid", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 432, "content": "experiment with a few other kernels using cross-validation and grid\nsearch, especially if there are kernels specialized for your training\nset\u2019s data structure.\nComputational Complexity\nThe LinearSVC class is based on the liblinear library, which implements an optimized\nalgorithm for linear SVMs.1 It does not support the kernel trick, but it scales almost\nNonlinear SVM Classification \n| \n153\nDownload from finelybook www.finelybook.com", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 433, "content": "2 \u201cSequential Minimal Optimization (SMO),\u201d J. Platt (1998).\nlinearly with the number of training instances and the number of features: its training\ntime complexity is roughly O(m \u00d7 n).\nThe algorithm takes longer if you require a very high precision. This is controlled by\nthe tolerance hyperparameter \u03f5 (called tol in Scikit-Learn). In most classification\ntasks, the default tolerance is fine.\nThe SVC class is based on the libsvm library, which implements an algorithm that sup\u2010\nports the kernel trick.2 The training time complexity is usually between O(m2 \u00d7 n)\nand O(m3 \u00d7 n). Unfortunately, this means that it gets dreadfully slow when the num\u2010\nber of training instances gets large (e.g., hundreds of thousands of instances). This\nalgorithm is perfect for complex but small or medium training sets. However, it scales\nwell with the number of features, especially with sparse features (i.e., when each\ninstance has few nonzero features). In this case, the algorithm scales roughly with the", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 434, "content": "instance has few nonzero features). In this case, the algorithm scales roughly with the\naverage number of nonzero features per instance. Table 5-1 compares Scikit-Learn\u2019s\nSVM classification classes.\nTable 5-1. Comparison of Scikit-Learn classes for SVM classification\nClass\nTime complexity\nOut-of-core support\nScaling required\nKernel trick\nLinearSVC\nO(m \u00d7 n)\nNo\nYes\nNo\nSGDClassifier O(m \u00d7 n)\nYes\nYes\nNo\nSVC\nO(m\u00b2 \u00d7 n) to O(m\u00b3 \u00d7 n) No\nYes\nYes\nSVM Regression\nAs we mentioned earlier, the SVM algorithm is quite versatile: not only does it sup\u2010\nport linear and nonlinear classification, but it also supports linear and nonlinear\nregression. The trick is to reverse the objective: instead of trying to fit the largest pos\u2010\nsible street between two classes while limiting margin violations, SVM Regression\ntries to fit as many instances as possible on the street while limiting margin violations\n(i.e., instances off the street). The width of the street is controlled by a hyperparame\u2010", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 435, "content": "(i.e., instances off the street). The width of the street is controlled by a hyperparame\u2010\nter \u03f5. Figure 5-10 shows two linear SVM Regression models trained on some random\nlinear data, one with a large margin (\u03f5 = 1.5) and the other with a small margin (\u03f5 =\n0.5).\n154 \n| \nChapter 5: Support Vector Machines\nDownload from finelybook www.finelybook.com", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 436, "content": "Figure 5-10. SVM Regression\nAdding more training instances within the margin does not affect the model\u2019s predic\u2010\ntions; thus, the model is said to be \u03f5-insensitive.\nYou can use Scikit-Learn\u2019s LinearSVR class to perform linear SVM Regression. The\nfollowing code produces the model represented on the left of Figure 5-10 (the train\u2010\ning data should be scaled and centered first):\nfrom sklearn.svm import LinearSVR\nsvm_reg = LinearSVR(epsilon=1.5)\nsvm_reg.fit(X, y)\nTo tackle nonlinear regression tasks, you can use a kernelized SVM model. For exam\u2010\nple, Figure 5-11 shows SVM Regression on a random quadratic training set, using a\n2nd-degree polynomial kernel. There is little regularization on the left plot (i.e., a large\nC value), and much more regularization on the right plot (i.e., a small C value).\nFigure 5-11. SVM regression using a 2nd-degree polynomial kernel\nSVM Regression \n| \n155\nDownload from finelybook www.finelybook.com", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 437, "content": "The following code produces the model represented on the left of Figure 5-11 using\nScikit-Learn\u2019s SVR class (which supports the kernel trick). The SVR class is the regres\u2010\nsion equivalent of the SVC class, and the LinearSVR class is the regression equivalent\nof the LinearSVC class. The LinearSVR class scales linearly with the size of the train\u2010\ning set (just like the LinearSVC class), while the SVR class gets much too slow when\nthe training set grows large (just like the SVC class).\nfrom sklearn.svm import SVR\nsvm_poly_reg = SVR(kernel=\"poly\", degree=2, C=100, epsilon=0.1)\nsvm_poly_reg.fit(X, y)\nSVMs can also be used for outlier detection; see Scikit-Learn\u2019s doc\u2010\numentation for more details.\nUnder the Hood\nThis section explains how SVMs make predictions and how their training algorithms\nwork, starting with linear SVM classifiers. You can safely skip it and go straight to the\nexercises at the end of this chapter if you are just getting started with Machine Learn\u2010", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 438, "content": "exercises at the end of this chapter if you are just getting started with Machine Learn\u2010\ning, and come back later when you want to get a deeper understanding of SVMs.\nFirst, a word about notations: in Chapter 4 we used the convention of putting all the \nmodel parameters in one vector \u03b8, including the bias term \u03b80 and the input feature\nweights \u03b81 to \u03b8n, and adding a bias input x0 = 1 to all instances. In this chapter, we will\nuse a different convention, which is more convenient (and more common) when you\nare dealing with SVMs: the bias term will be called b and the feature weights vector\nwill be called w. No bias feature will be added to the input feature vectors.\nDecision Function and Predictions\nThe linear SVM classifier model predicts the class of a new instance x by simply com\u2010\nputing the decision function wT \u00b7 x + b = w1 x1 + \u22ef + wn xn + b: if the result is posi\u2010\ntive, the predicted class \u0177 is the positive class (1), or else it is the negative class (0); see\nEquation 5-2.", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 439, "content": "Equation 5-2.\nEquation 5-2. Linear SVM classifier prediction\ny = 0 if \ufffdT \u00b7 \ufffd+ b < 0,\n1 if \ufffdT \u00b7 \ufffd+ b \u22650\n156 \n| \nChapter 5: Support Vector Machines\nDownload from finelybook www.finelybook.com", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 440, "content": "3 More generally, when there are n features, the decision function is an n-dimensional hyperplane, and the deci\u2010\nsion boundary is an (n \u2013 1)-dimensional hyperplane.\nFigure 5-12 shows the decision function that corresponds to the model on the right of\nFigure 5-4: it is a two-dimensional plane since this dataset has two features (petal\nwidth and petal length). The decision boundary is the set of points where the decision\nfunction is equal to 0: it is the intersection of two planes, which is a straight line (rep\u2010\nresented by the thick solid line).3\nFigure 5-12. Decision function for the iris dataset\nThe dashed lines represent the points where the decision function is equal to 1 or \u20131:\nthey are parallel and at equal distance to the decision boundary, forming a margin\naround it. Training a linear SVM classifier means finding the value of w and b that\nmake this margin as wide as possible while avoiding margin violations (hard margin)\nor limiting them (soft margin).\nTraining Objective", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 441, "content": "or limiting them (soft margin).\nTraining Objective\nConsider the slope of the decision function: it is equal to the norm of the weight vec\u2010\ntor, \u2225 w \u2225. If we divide this slope by 2, the points where the decision function is equal\nto \u00b11 are going to be twice as far away from the decision boundary. In other words,\ndividing the slope by 2 will multiply the margin by 2. Perhaps this is easier to visual\u2010\nize in 2D in Figure 5-13. The smaller the weight vector w, the larger the margin.\nUnder the Hood \n| \n157\nDownload from finelybook www.finelybook.com", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 442, "content": "4 Zeta (\u03b6) is the 8th letter of the Greek alphabet.\nFigure 5-13. A smaller weight vector results in a larger margin\nSo we want to minimize \u2225 w \u2225 to get a large margin. However, if we also want to avoid\nany margin violation (hard margin), then we need the decision function to be greater\nthan 1 for all positive training instances, and lower than \u20131 for negative training\ninstances. If we define t(i) = \u20131 for negative instances (if y(i) = 0) and t(i) = 1 for positive\ninstances (if y(i) = 1), then we can express this constraint as t(i)(wT \u00b7 x(i) + b) \u2265 1 for all\ninstances.\nWe can therefore express the hard margin linear SVM classifier objective as the con\u2010\nstrained optimization problem in Equation 5-3.\nEquation 5-3. Hard margin linear SVM classifier objective\nminimize\n\ufffd, b\n1\n2\ufffdT \u00b7 \ufffd\nsubject to\nt i \ufffdT \u00b7 \ufffdi + b \u22651\nfor i = 1, 2, \u22ef, m\nWe are minimizing 1\n2wT \u00b7 w, which is equal to 1\n2\u2225 w \u22252, rather than\nminimizing \u2225 w \u2225. This is because it will give the same result (since", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 443, "content": "2\u2225 w \u22252, rather than\nminimizing \u2225 w \u2225. This is because it will give the same result (since\nthe values of w and b that minimize a value also minimize half of\nits square), but 1\n2\u2225 w \u22252 has a nice and simple derivative (it is just\nw) while \u2225 w \u2225 is not differentiable at w = 0. Optimization algo\u2010\nrithms work much better on differentiable functions.\nTo get the soft margin objective, we need to introduce a slack variable \u03b6(i) \u2265 0 for each\ninstance:4 \u03b6(i) measures how much the ith instance is allowed to violate the margin. We\nnow have two conflicting objectives: making the slack variables as small as possible to\nreduce the margin violations, and making 1\n2wT \u00b7 w as small as possible to increase the\nmargin. This is where the C hyperparameter comes in: it allows us to define the trade\u2010\n158 \n| \nChapter 5: Support Vector Machines\nDownload from finelybook www.finelybook.com", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 444, "content": "5 To learn more about Quadratic Programming, you can start by reading Stephen Boyd and Lieven Vanden\u2010\nberghe, Convex Optimization (Cambridge, UK: Cambridge University Press, 2004) or watch Richard Brown\u2019s\nseries of video lectures.\noff between these two objectives. This gives us the constrained optimization problem\nin Equation 5-4.\nEquation 5-4. Soft margin linear SVM classifier objective\nminimize\n\ufffd, b, \u03b6\n1\n2\ufffdT \u00b7 \ufffd+ C \u2211\ni = 1\nm\n\u03b6 i\nsubject to\nt i \ufffdT \u00b7 \ufffdi + b \u22651 \u2212\u03b6 i\nand\n\u03b6 i \u22650\nfor i = 1, 2, \u22ef, m\nQuadratic Programming\nThe hard margin and soft margin problems are both convex quadratic optimization\nproblems with linear constraints. Such problems are known as Quadratic Program\u2010\nming (QP) problems. Many off-the-shelf solvers are available to solve QP problems\nusing a variety of techniques that are outside the scope of this book.5 The general\nproblem formulation is given by Equation 5-5.\nEquation 5-5. Quadratic Programming problem\nMinimize\n\ufffd\n1\n2\ufffdT \u00b7 \ufffd\u00b7 \ufffd\n+\n\ufffdT \u00b7 \ufffd\nsubject to\n\ufffd\u00b7 \ufffd\u2264\ufffd\nwhere\n\ufffd", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 445, "content": "Minimize\n\ufffd\n1\n2\ufffdT \u00b7 \ufffd\u00b7 \ufffd\n+\n\ufffdT \u00b7 \ufffd\nsubject to\n\ufffd\u00b7 \ufffd\u2264\ufffd\nwhere\n\ufffd\nis an np\u2010dimensional vector (np = number of parameters),\n\ufffd\nis an np \u00d7 np matrix,\n\ufffd\nis an np\u2010dimensional vector,\n\ufffd\nis an nc \u00d7 np matrix (nc = number of constraints),\n\ufffd\nis an nc\u2010dimensional vector.\nNote that the expression A \u00b7 p \u2264 b actually defines nc constraints: pT \u00b7 a(i) \u2264 b(i) for i =\n1, 2, \u22ef, nc, where a(i) is the vector containing the elements of the ith row of A and b(i) is\nthe ith element of b.\nYou can easily verify that if you set the QP parameters in the following way, you get\nthe hard margin linear SVM classifier objective:\n\u2022 np = n + 1, where n is the number of features (the +1 is for the bias term).\nUnder the Hood \n| \n159\nDownload from finelybook www.finelybook.com", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 446, "content": "6 The objective function is convex, and the inequality constraints are continuously differentiable and convex\nfunctions.\n\u2022 nc = m, where m is the number of training instances.\n\u2022 H is the np \u00d7 np identity matrix, except with a zero in the top-left cell (to ignore\nthe bias term).\n\u2022 f = 0, an np-dimensional vector full of 0s.\n\u2022 b = 1, an nc-dimensional vector full of 1s.\n\u2022 a(i) = \u2013t(i) \ufffd\u02d9(i), where \ufffd\u02d9(i) is equal to x(i) with an extra bias feature \ufffd\u02d90 = 1.\nSo one way to train a hard margin linear SVM classifier is just to use an off-the-shelf\nQP solver by passing it the preceding parameters. The resulting vector p will contain\nthe bias term b = p0 and the feature weights wi = pi for i = 1, 2, \u22ef, m. Similarly, you\ncan use a QP solver to solve the soft margin problem (see the exercises at the end of\nthe chapter).\nHowever, to use the kernel trick we are going to look at a different constrained opti\u2010\nmization problem.\nThe Dual Problem", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 447, "content": "mization problem.\nThe Dual Problem\nGiven a constrained optimization problem, known as the primal problem, it is possi\u2010\nble to express a different but closely related problem, called its dual problem. The sol\u2010\nution to the dual problem typically gives a lower bound to the solution of the primal\nproblem, but under some conditions it can even have the same solutions as the pri\u2010\nmal problem. Luckily, the SVM problem happens to meet these conditions,6 so you\ncan choose to solve the primal problem or the dual problem; both will have the same\nsolution. Equation 5-6 shows the dual form of the linear SVM objective (if you are\ninterested in knowing how to derive the dual problem from the primal problem, see\nAppendix C).\nEquation 5-6. Dual form of the linear SVM objective\nminimize\n\u03b1\n1\n2 \u2211\ni = 1\nm\n\u2211\nj = 1\nm\n\u03b1 i \u03b1 j t i t j \ufffdi T \u00b7 \ufffdj\n\u2212\n\u2211\ni = 1\nm\n\u03b1 i\nsubject to\n\u03b1 i \u22650\nfor i = 1, 2, \u22ef, m\n160 \n| \nChapter 5: Support Vector Machines\nDownload from finelybook www.finelybook.com", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 448, "content": "Once you find the vector \u03b1 that minimizes this equation (using a QP solver), you can\ncompute \ufffd and b that minimize the primal problem by using Equation 5-7.\nEquation 5-7. From the dual solution to the primal solution\n\ufffd= \u2211\ni = 1\nm\n\u03b1 i t i \ufffdi\nb = 1\nns \u2211\ni = 1\n\u03b1 i > 0\nm\n1 \u2212t i \ufffdT \u00b7 \ufffdi\nThe dual problem is faster to solve than the primal when the number of training\ninstances is smaller than the number of features. More importantly, it makes the ker\u2010\nnel trick possible, while the primal does not. So what is this kernel trick anyway?\nKernelized SVM\nSuppose you want to apply a 2nd-degree polynomial transformation to a two-\ndimensional training set (such as the moons training set), then train a linear SVM\nclassifier on the transformed training set. Equation 5-8 shows the 2nd-degree polyno\u2010\nmial mapping function \u03d5 that you want to apply.\nEquation 5-8. Second-degree polynomial mapping\n\u03d5 \ufffd= \u03d5\nx1\nx2\n=\nx1\n2\n2 x1x2\nx2\n2", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 449, "content": "Equation 5-8. Second-degree polynomial mapping\n\u03d5 \ufffd= \u03d5\nx1\nx2\n=\nx1\n2\n2 x1x2\nx2\n2\nNotice that the transformed vector is three-dimensional instead of two-dimensional.\nNow let\u2019s look at what happens to a couple of two-dimensional vectors, a and b, if we\napply this 2nd-degree polynomial mapping and then compute the dot product of the\ntransformed vectors (See Equation 5-9).\nUnder the Hood \n| \n161\nDownload from finelybook www.finelybook.com", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 450, "content": "Equation 5-9. Kernel trick for a 2nd-degree polynomial mapping\n\u03d5 \ufffdT \u00b7 \u03d5 \ufffd\n=\na1\n2\n2 a1a2\na2\n2\nT\n\u00b7\nb1\n2\n2 b1b2\nb2\n2\n= a1\n2b1\n2 + 2a1b1a2b2 + a2\n2b2\n2\n= a1b1 + a2b2\n2 =\na1\na2\nT\n\u00b7\nb1\nb2\n2\n= \ufffdT \u00b7 \ufffd\n2\nHow about that? The dot product of the transformed vectors is equal to the square of\nthe dot product of the original vectors: \u03d5(a)T \u00b7 \u03d5(b) = (aT \u00b7 b)2.\nNow here is the key insight: if you apply the transformation \u03d5 to all training instan\u2010\nces, then the dual problem (see Equation 5-6) will contain the dot product \u03d5(x(i))T \u00b7\n\u03d5(x(j)). But if \u03d5 is the 2nd-degree polynomial transformation defined in Equation 5-8,\nthen you can replace this dot product of transformed vectors simply by \ufffdi T \u00b7 \ufffdj\n2\n.\nSo you don\u2019t actually need to transform the training instances at all: just replace the\ndot product by its square in Equation 5-6. The result will be strictly the same as if you\nwent through the trouble of actually transforming the training set then fitting a linear", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 451, "content": "went through the trouble of actually transforming the training set then fitting a linear\nSVM algorithm, but this trick makes the whole process much more computationally\nefficient. This is the essence of the kernel trick.\nThe function K(a, b) = (aT \u00b7 b)2 is called a 2nd-degree polynomial kernel. In Machine\nLearning, a kernel is a function capable of computing the dot product \u03d5(a)T \u00b7 \u03d5(b)\nbased only on the original vectors a and b, without having to compute (or even to\nknow about) the transformation \u03d5. Equation 5-10 lists some of the most commonly\nused kernels.\nEquation 5-10. Common kernels\nLinear:\nK \ufffd, \ufffd= \ufffdT \u00b7 \ufffd\nPolynomial:\nK \ufffd, \ufffd= \u03b3\ufffdT \u00b7 \ufffd+ r\nd\nGaussian RBF:\nK \ufffd, \ufffd= exp \u2212\u03b3\u2225\ufffd\u2212\ufffd\u22252\nSigmoid:\nK \ufffd, \ufffd= tanh \u03b3\ufffdT \u00b7 \ufffd+ r\n162 \n| \nChapter 5: Support Vector Machines\nDownload from finelybook www.finelybook.com", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 452, "content": "Mercer\u2019s Theorem\nAccording to Mercer\u2019s theorem, if a function K(a, b) respects a few mathematical con\u2010\nditions called Mercer\u2019s conditions (K must be continuous, symmetric in its arguments\nso K(a, b) = K(b, a), etc.), then there exists a function \u03d5 that maps a and b into\nanother space (possibly with much higher dimensions) such that K(a, b) = \u03d5(a)T \u00b7\n\u03d5(b). So you can use K as a kernel since you know \u03d5 exists, even if you don\u2019t know\nwhat \u03d5 is. In the case of the Gaussian RBF kernel, it can be shown that \u03d5 actually\nmaps each training instance to an infinite-dimensional space, so it\u2019s a good thing you\ndon\u2019t need to actually perform the mapping!\nNote that some frequently used kernels (such as the Sigmoid kernel) don\u2019t respect all\nof Mercer\u2019s conditions, yet they generally work well in practice.\nThere is still one loose end we must tie. Equation 5-7 shows how to go from the dual\nsolution to the primal solution in the case of a linear SVM classifier, but if you apply", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 453, "content": "solution to the primal solution in the case of a linear SVM classifier, but if you apply\nthe kernel trick you end up with equations that include \u03d5(x(i)). In fact, \ufffd must have\nthe same number of dimensions as \u03d5(x(i)), which may be huge or even infinite, so you\ncan\u2019t compute it. But how can you make predictions without knowing \ufffd? Well, the\ngood news is that you can plug in the formula for \ufffd from Equation 5-7 into the deci\u2010\nsion function for a new instance x(n), and you get an equation with only dot products\nbetween input vectors. This makes it possible to use the kernel trick, once again\n(Equation 5-11).\nEquation 5-11. Making predictions with a kernelized SVM\nh\ufffd, b \u03d5 \ufffdn\n= \ufffdT \u00b7 \u03d5 \ufffdn\n+ b = \u2211\ni = 1\nm\n\u03b1 i t i \u03d5 \ufffdi\nT\n\u00b7 \u03d5 \ufffdn\n+ b\n= \u2211\ni = 1\nm\n\u03b1 i t i \u03d5 \ufffdi T \u00b7 \u03d5 \ufffdn\n+ b\n=\n\u2211\ni = 1\n\u03b1 i > 0\nm\n\u03b1 i t i K \ufffdi , \ufffdn\n+ b\nNote that since \u03b1(i) \u2260 0 only for support vectors, making predictions involves comput\u2010\ning the dot product of the new input vector x(n) with only the support vectors, not all", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 454, "content": "ing the dot product of the new input vector x(n) with only the support vectors, not all\nthe training instances. Of course, you also need to compute the bias term b, using the\nsame trick (Equation 5-12).\nUnder the Hood \n| \n163\nDownload from finelybook www.finelybook.com", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 455, "content": "Equation 5-12. Computing the bias term using the kernel trick\nb = 1\nns \u2211\ni = 1\n\u03b1 i > 0\nm\n1 \u2212t i \ufffdT \u00b7 \u03d5 \ufffdi\n= 1\nns \u2211\ni = 1\n\u03b1 i > 0\nm\n1 \u2212t i \u2211\nj = 1\nm\n\u03b1 j t j \u03d5 \ufffdj\nT\n\u00b7 \u03d5 \ufffdi\n= 1\nns \u2211\ni = 1\n\u03b1 i > 0\nm\n1 \u2212t i\n\u2211\nj = 1\n\u03b1 j > 0\nm\n\u03b1 j t j K \ufffdi , \ufffdj\nIf you are starting to get a headache, it\u2019s perfectly normal: it\u2019s an unfortunate side\neffects of the kernel trick.\nOnline SVMs\nBefore concluding this chapter, let\u2019s take a quick look at online SVM classifiers (recall\nthat online learning means learning incrementally, typically as new instances arrive).\nFor linear SVM classifiers, one method is to use Gradient Descent (e.g., using\nSGDClassifier) to minimize the cost function in Equation 5-13, which is derived\nfrom the primal problem. Unfortunately it converges much more slowly than the\nmethods based on QP.\nEquation 5-13. Linear SVM classifier cost function\nJ \ufffd, b = 1\n2\ufffdT \u00b7 \ufffd\n+\nC \u2211\ni = 1\nm\nmax 0, 1 \u2212t i \ufffdT \u00b7 \ufffdi + b\nThe first sum in the cost function will push the model to have a small weight vector", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 456, "content": "The first sum in the cost function will push the model to have a small weight vector\nw, leading to a larger margin. The second sum computes the total of all margin viola\u2010\ntions. An instance\u2019s margin violation is equal to 0 if it is located off the street and on\nthe correct side, or else it is proportional to the distance to the correct side of the\nstreet. Minimizing this term ensures that the model makes the margin violations as\nsmall and as few as possible\nHinge Loss\nThe function max(0, 1 \u2013 t) is called the hinge loss function (represented below). It is\nequal to 0 when t \u2265 1. Its derivative (slope) is equal to \u20131 if t < 1 and 0 if t > 1. It is not\ndifferentiable at t = 1, but just like for Lasso Regression (see \u201cLasso Regression\u201d on\npage 130) you can still use Gradient Descent using any subderivative at t = 0 (i.e., any\nvalue between \u20131 and 0).\n164 \n| \nChapter 5: Support Vector Machines\nDownload from finelybook www.finelybook.com", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 457, "content": "7 \u201cIncremental and Decremental Support Vector Machine Learning,\u201d G. Cauwenberghs, T. Poggio (2001).\n8 \u201cFast Kernel Classifiers with Online and Active Learning,\u201c A. Bordes, S. Ertekin, J. Weston, L. Bottou (2005).\nIt is also possible to implement online kernelized SVMs\u2014for example, using \u201cIncre\u2010\nmental and Decremental SVM Learning\u201d7 or \u201cFast Kernel Classifiers with Online and\nActive Learning.\u201d8 However, these are implemented in Matlab and C++. For large-\nscale nonlinear problems, you may want to consider using neural networks instead \n(see Part II).\nExercises\n1. What is the fundamental idea behind Support Vector Machines?\n2. What is a support vector?\n3. Why is it important to scale the inputs when using SVMs?\n4. Can an SVM classifier output a confidence score when it classifies an instance?\nWhat about a probability?\n5. Should you use the primal or the dual form of the SVM problem to train a model\non a training set with millions of instances and hundreds of features?", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 458, "content": "on a training set with millions of instances and hundreds of features?\n6. Say you trained an SVM classifier with an RBF kernel. It seems to underfit the\ntraining set: should you increase or decrease \u03b3 (gamma)? What about C?\n7. How should you set the QP parameters (H, f, A, and b) to solve the soft margin\nlinear SVM classifier problem using an off-the-shelf QP solver?\n8. Train a LinearSVC on a linearly separable dataset. Then train an SVC and a\nSGDClassifier on the same dataset. See if you can get them to produce roughly\nthe same model.\n9. Train an SVM classifier on the MNIST dataset. Since SVM classifiers are binary\nclassifiers, you will need to use one-versus-all to classify all 10 digits. You may\nExercises \n| \n165\nDownload from finelybook www.finelybook.com", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 459, "content": "want to tune the hyperparameters using small validation sets to speed up the pro\u2010\ncess. What accuracy can you reach?\n10. Train an SVM regressor on the California housing dataset.\nSolutions to these exercises are available in Appendix A.\n166 \n| \nChapter 5: Support Vector Machines\nDownload from finelybook www.finelybook.com", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 460, "content": "CHAPTER 6\nDecision Trees\nLike SVMs, Decision Trees are versatile Machine Learning algorithms that can per\u2010\nform both classification and regression tasks, and even multioutput tasks. They are\nvery powerful algorithms, capable of fitting complex datasets. For example, in Chap\u2010\nter 2 you trained a DecisionTreeRegressor model on the California housing dataset,\nfitting it perfectly (actually overfitting it).\nDecision Trees are also the fundamental components of Random Forests (see Chap\u2010\nter 7), which are among the most powerful Machine Learning algorithms available\ntoday.\nIn this chapter we will start by discussing how to train, visualize, and make predic\u2010\ntions with Decision Trees. Then we will go through the CART training algorithm\nused by Scikit-Learn, and we will discuss how to regularize trees and use them for\nregression tasks. Finally, we will discuss some of the limitations of Decision Trees.\nTraining and Visualizing a Decision Tree", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 461, "content": "Training and Visualizing a Decision Tree\nTo understand Decision Trees, let\u2019s just build one and take a look at how it makes pre\u2010\ndictions. The following code trains a DecisionTreeClassifier on the iris dataset\n(see Chapter 4):\nfrom sklearn.datasets import load_iris\nfrom sklearn.tree import DecisionTreeClassifier\niris = load_iris()\nX = iris.data[:, 2:] # petal length and width\ny = iris.target\ntree_clf = DecisionTreeClassifier(max_depth=2)\ntree_clf.fit(X, y)\n167\nDownload from finelybook www.finelybook.com", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 462, "content": "1 Graphviz is an open source graph visualization software package, available at http://www.graphviz.org/.\nYou can visualize the trained Decision Tree by first using the export_graphviz() \nmethod to output a graph definition file called iris_tree.dot:\nfrom sklearn.tree import export_graphviz\nexport_graphviz(\n tree_clf,\n out_file=image_path(\"iris_tree.dot\"),\n feature_names=iris.feature_names[2:],\n class_names=iris.target_names,\n rounded=True,\n filled=True\n )\nThen you can convert this .dot file to a variety of formats such as PDF or PNG using\nthe dot command-line tool from the graphviz package.1 This command line converts\nthe .dot file to a .png image file:\n$ dot -Tpng iris_tree.dot -o iris_tree.png\nYour first decision tree looks like Figure 6-1.\nFigure 6-1. Iris Decision Tree\n168 \n| \nChapter 6: Decision Trees\nDownload from finelybook www.finelybook.com", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 463, "content": "Making Predictions\nLet\u2019s see how the tree represented in Figure 6-1 makes predictions. Suppose you find\nan iris flower and you want to classify it. You start at the root node (depth 0, at the\ntop): this node asks whether the flower\u2019s petal length is smaller than 2.45 cm. If it is,\nthen you move down to the root\u2019s left child node (depth 1, left). In this case, it is a leaf\nnode (i.e., it does not have any children nodes), so it does not ask any questions: you\ncan simply look at the predicted class for that node and the Decision Tree predicts\nthat your flower is an Iris-Setosa (class=setosa).\nNow suppose you find another flower, but this time the petal length is greater than\n2.45 cm. You must move down to the root\u2019s right child node (depth 1, right), which is\nnot a leaf node, so it asks another question: is the petal width smaller than 1.75 cm? If\nit is, then your flower is most likely an Iris-Versicolor (depth 2, left). If not, it is likely", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 464, "content": "it is, then your flower is most likely an Iris-Versicolor (depth 2, left). If not, it is likely\nan Iris-Virginica (depth 2, right). It\u2019s really that simple.\nOne of the many qualities of Decision Trees is that they require\nvery little data preparation. In particular, they don\u2019t require feature\nscaling or centering at all.\nA node\u2019s samples attribute counts how many training instances it applies to. For\nexample, 100 training instances have a petal length greater than 2.45 cm (depth 1,\nright), among which 54 have a petal width smaller than 1.75 cm (depth 2, left). A\nnode\u2019s value attribute tells you how many training instances of each class this node\napplies to: for example, the bottom-right node applies to 0 Iris-Setosa, 1 Iris-\nVersicolor, and 45 Iris-Virginica. Finally, a node\u2019s gini attribute measures its impur\u2010\nity: a node is \u201cpure\u201d (gini=0) if all training instances it applies to belong to the same\nclass. For example, since the depth-1 left node applies only to Iris-Setosa training", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 465, "content": "class. For example, since the depth-1 left node applies only to Iris-Setosa training\ninstances, it is pure and its gini score is 0. Equation 6-1 shows how the training algo\u2010\nrithm computes the gini score Gi of the ith node. For example, the depth-2 left node\nhas a gini score equal to 1 \u2013 (0/54)2 \u2013 (49/54)2 \u2013 (5/54)2 \u2248 0.168. Another impurity\nmeasure is discussed shortly.\nEquation 6-1. Gini impurity\nGi = 1 \u2212\u2211\nk = 1\nn\npi, k\n2\n\u2022 pi,k is the ratio of class k instances among the training instances in the ith node.\nMaking Predictions \n| \n169\nDownload from finelybook www.finelybook.com", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 466, "content": "Scikit-Learn uses the CART algorithm, which produces only binary\ntrees: nonleaf nodes always have two children (i.e., questions only\nhave yes/no answers). However, other algorithms such as ID3 can\nproduce Decision Trees with nodes that have more than two chil\u2010\ndren.\nFigure 6-2 shows this Decision Tree\u2019s decision boundaries. The thick vertical line rep\u2010\nresents the decision boundary of the root node (depth 0): petal length = 2.45 cm.\nSince the left area is pure (only Iris-Setosa), it cannot be split any further. However,\nthe right area is impure, so the depth-1 right node splits it at petal width = 1.75 cm\n(represented by the dashed line). Since max_depth was set to 2, the Decision Tree\nstops right there. However, if you set max_depth to 3, then the two depth-2 nodes\nwould each add another decision boundary (represented by the dotted lines).\nFigure 6-2. Decision Tree decision boundaries\nModel Interpretation: White Box Versus Black Box", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 467, "content": "Figure 6-2. Decision Tree decision boundaries\nModel Interpretation: White Box Versus Black Box\nAs you can see Decision Trees are fairly intuitive and their decisions are easy to inter\u2010\npret. Such models are often called white box models. In contrast, as we will see, Ran\u2010\ndom Forests or neural networks are generally considered black box models. They\nmake great predictions, and you can easily check the calculations that they performed\nto make these predictions; nevertheless, it is usually hard to explain in simple terms\nwhy the predictions were made. For example, if a neural network says that a particu\u2010\nlar person appears on a picture, it is hard to know what actually contributed to this\nprediction: did the model recognize that person\u2019s eyes? Her mouth? Her nose? Her\nshoes? Or even the couch that she was sitting on? Conversely, Decision Trees provide\nnice and simple classification rules that can even be applied manually if need be (e.g.,\nfor flower classification).\n170 \n|", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 468, "content": "for flower classification).\n170 \n| \nChapter 6: Decision Trees\nDownload from finelybook www.finelybook.com", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 469, "content": "Estimating Class Probabilities\nA Decision Tree can also estimate the probability that an instance belongs to a partic\u2010\nular class k: first it traverses the tree to find the leaf node for this instance, and then it\nreturns the ratio of training instances of class k in this node. For example, suppose\nyou have found a flower whose petals are 5 cm long and 1.5 cm wide. The corre\u2010\nsponding leaf node is the depth-2 left node, so the Decision Tree should output the\nfollowing probabilities: 0% for Iris-Setosa (0/54), 90.7% for Iris-Versicolor (49/54),\nand 9.3% for Iris-Virginica (5/54). And of course if you ask it to predict the class, it\nshould output Iris-Versicolor (class 1) since it has the highest probability. Let\u2019s check\nthis:\n>>> tree_clf.predict_proba([[5, 1.5]])\narray([[ 0. , 0.90740741, 0.09259259]])\n>>> tree_clf.predict([[5, 1.5]])\narray([1])\nPerfect! Notice that the estimated probabilities would be identical anywhere else in", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 470, "content": "array([1])\nPerfect! Notice that the estimated probabilities would be identical anywhere else in\nthe bottom-right rectangle of Figure 6-2\u2014for example, if the petals were 6 cm long\nand 1.5 cm wide (even though it seems obvious that it would most likely be an Iris-\nVirginica in this case).\nThe CART Training Algorithm\nScikit-Learn uses the Classification And Regression Tree (CART) algorithm to train\nDecision Trees (also called \u201cgrowing\u201d trees). The idea is really quite simple: the algo\u2010\nrithm first splits the training set in two subsets using a single feature k and a thres\u2010\nhold tk (e.g., \u201cpetal length \u2264 2.45 cm\u201d). How does it choose k and tk? It searches for the\npair (k, tk) that produces the purest subsets (weighted by their size). The cost function\nthat the algorithm tries to minimize is given by Equation 6-2.\nEquation 6-2. CART cost function for classification\nJ k, tk =\nmleft\nm Gleft +\nmright\nm\nGright\nwhere\nGleft/right measures the impurity of the left/right subset,", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 471, "content": "mleft\nm Gleft +\nmright\nm\nGright\nwhere\nGleft/right measures the impurity of the left/right subset,\nmleft/right is the number of instances in the left/right subset.\nOnce it has successfully split the training set in two, it splits the subsets using the\nsame logic, then the sub-subsets and so on, recursively. It stops recursing once it rea\u2010\nches the maximum depth (defined by the max_depth hyperparameter), or if it cannot\nfind a split that will reduce impurity. A few other hyperparameters (described in a\nEstimating Class Probabilities \n| \n171\nDownload from finelybook www.finelybook.com", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 472, "content": "2 P is the set of problems that can be solved in polynomial time. NP is the set of problems whose solutions can\nbe verified in polynomial time. An NP-Hard problem is a problem to which any NP problem can be reduced\nin polynomial time. An NP-Complete problem is both NP and NP-Hard. A major open mathematical ques\u2010\ntion is whether or not P = NP. If P \u2260 NP (which seems likely), then no polynomial algorithm will ever be\nfound for any NP-Complete problem (except perhaps on a quantum computer).\n3 log2 is the binary logarithm. It is equal to log2(m) = log(m) / log(2).\nmoment) control additional stopping conditions (min_samples_split, min_sam\nples_leaf, min_weight_fraction_leaf, and max_leaf_nodes).\nAs you can see, the CART algorithm is a greedy algorithm: it greed\u2010\nily searches for an optimum split at the top level, then repeats the\nprocess at each level. It does not check whether or not the split will\nlead to the lowest possible impurity several levels down. A greedy", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 473, "content": "lead to the lowest possible impurity several levels down. A greedy\nalgorithm often produces a reasonably good solution, but it is not\nguaranteed to be the optimal solution.\nUnfortunately, finding the optimal tree is known to be an NP-Complete problem:2 it\nrequires O(exp(m)) time, making the problem intractable even for fairly small train\u2010\ning sets. This is why we must settle for a \u201creasonably good\u201d solution.\nComputational Complexity\nMaking predictions requires traversing the Decision Tree from the root to a leaf.\nDecision Trees are generally approximately balanced, so traversing the Decision Tree\nrequires going through roughly O(log2(m)) nodes.3 Since each node only requires\nchecking the value of one feature, the overall prediction complexity is just O(log2(m)),\nindependent of the number of features. So predictions are very fast, even when deal\u2010\ning with large training sets.\nHowever, the training algorithm compares all features (or less if max_features is set)", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 474, "content": "However, the training algorithm compares all features (or less if max_features is set)\non all samples at each node. This results in a training complexity of O(n \u00d7 m log(m)).\nFor small training sets (less than a few thousand instances), Scikit-Learn can speed up\ntraining by presorting the data (set presort=True), but this slows down training con\u2010\nsiderably for larger training sets.\nGini Impurity or Entropy?\nBy default, the Gini impurity measure is used, but you can select the entropy impurity\nmeasure instead by setting the criterion hyperparameter to \"entropy\". The concept\nof entropy originated in thermodynamics as a measure of molecular disorder:\nentropy approaches zero when molecules are still and well ordered. It later spread to a\nwide variety of domains, including Shannon\u2019s information theory, where it measures\n172 \n| \nChapter 6: Decision Trees\nDownload from finelybook www.finelybook.com", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 475, "content": "4 A reduction of entropy is often called an information gain.\n5 See Sebastian Raschka\u2019s interesting analysis for more details.\nthe average information content of a message:4 entropy is zero when all messages are\nidentical. In Machine Learning, it is frequently used as an impurity measure: a set\u2019s\nentropy is zero when it contains instances of only one class. Equation 6-3 shows the\ndefinition of the entropy of the ith node. For example, the depth-2 left node in\nFigure 6-1 has an entropy equal to \u221249\n54 log\n49\n54 \u22125\n54 log\n5\n54 \u2248 0.31.\nEquation 6-3. Entropy\nHi = \u2212\n\u2211\nk = 1\npi, k \u22600\nn\npi, k log pi, k\nSo should you use Gini impurity or entropy? The truth is, most of the time it does not\nmake a big difference: they lead to similar trees. Gini impurity is slightly faster to\ncompute, so it is a good default. However, when they differ, Gini impurity tends to\nisolate the most frequent class in its own branch of the tree, while entropy tends to\nproduce slightly more balanced trees.5", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 476, "content": "produce slightly more balanced trees.5\nRegularization Hyperparameters\nDecision Trees make very few assumptions about the training data (as opposed to lin\u2010\near models, which obviously assume that the data is linear, for example). If left\nunconstrained, the tree structure will adapt itself to the training data, fitting it very\nclosely, and most likely overfitting it. Such a model is often called a nonparametric\nmodel, not because it does not have any parameters (it often has a lot) but because the\nnumber of parameters is not determined prior to training, so the model structure is\nfree to stick closely to the data. In contrast, a parametric model such as a linear model\nhas a predetermined number of parameters, so its degree of freedom is limited,\nreducing the risk of overfitting (but increasing the risk of underfitting).\nTo avoid overfitting the training data, you need to restrict the Decision Tree\u2019s freedom", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 477, "content": "To avoid overfitting the training data, you need to restrict the Decision Tree\u2019s freedom\nduring training. As you know by now, this is called regularization. The regularization\nhyperparameters depend on the algorithm used, but generally you can at least restrict\nthe maximum depth of the Decision Tree. In Scikit-Learn, this is controlled by the\nmax_depth hyperparameter (the default value is None, which means unlimited).\nReducing max_depth will regularize the model and thus reduce the risk of overfitting.\nThe DecisionTreeClassifier class has a few other parameters that similarly restrict\nthe shape of the Decision Tree: min_samples_split (the minimum number of sam\u2010\nRegularization Hyperparameters \n| \n173\nDownload from finelybook www.finelybook.com", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 478, "content": "ples a node must have before it can be split), min_samples_leaf (the minimum num\u2010\nber of samples a leaf node must have), min_weight_fraction_leaf (same as\nmin_samples_leaf but expressed as a fraction of the total number of weighted\ninstances), max_leaf_nodes (maximum number of leaf nodes), and max_features\n(maximum number of features that are evaluated for splitting at each node). Increas\u2010\ning min_* hyperparameters or reducing max_* hyperparameters will regularize the\nmodel.\nOther algorithms work by first training the Decision Tree without\nrestrictions, then pruning (deleting) unnecessary nodes. A node\nwhose children are all leaf nodes is considered unnecessary if the\npurity improvement it provides is not statistically significant. Stan\u2010\ndard statistical tests, such as the \u03c72 test, are used to estimate the\nprobability that the improvement is purely the result of chance\n(which is called the null hypothesis). If this probability, called the p-", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 479, "content": "(which is called the null hypothesis). If this probability, called the p-\nvalue, is higher than a given threshold (typically 5%, controlled by\na hyperparameter), then the node is considered unnecessary and its\nchildren are deleted. The pruning continues until all unnecessary\nnodes have been pruned.\nFigure 6-3 shows two Decision Trees trained on the moons dataset (introduced in\nChapter 5). On the left, the Decision Tree is trained with the default hyperparameters\n(i.e., no restrictions), and on the right the Decision Tree is trained with min_sam\nples_leaf=4. It is quite obvious that the model on the left is overfitting, and the\nmodel on the right will probably generalize better.\nFigure 6-3. Regularization using min_samples_leaf\n174 \n| \nChapter 6: Decision Trees\nDownload from finelybook www.finelybook.com", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 480, "content": "Regression\nDecision Trees are also capable of performing regression tasks. Let\u2019s build a regres\u2010\nsion tree using Scikit-Learn\u2019s DecisionTreeRegressor class, training it on a noisy\nquadratic dataset with max_depth=2:\nfrom sklearn.tree import DecisionTreeRegressor\ntree_reg = DecisionTreeRegressor(max_depth=2)\ntree_reg.fit(X, y)\nThe resulting tree is represented on Figure 6-4.\nFigure 6-4. A Decision Tree for regression\nThis tree looks very similar to the classification tree you built earlier. The main differ\u2010\nence is that instead of predicting a class in each node, it predicts a value. For example,\nsuppose you want to make a prediction for a new instance with x1 = 0.6. You traverse\nthe tree starting at the root, and you eventually reach the leaf node that predicts\nvalue=0.1106. This prediction is simply the average target value of the 110 training\ninstances associated to this leaf node. This prediction results in a Mean Squared Error\n(MSE) equal to 0.0151 over these 110 instances.", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 481, "content": "(MSE) equal to 0.0151 over these 110 instances.\nThis model\u2019s predictions are represented on the left of Figure 6-5. If you set\nmax_depth=3, you get the predictions represented on the right. Notice how the pre\u2010\ndicted value for each region is always the average target value of the instances in that\nregion. The algorithm splits each region in a way that makes most training instances\nas close as possible to that predicted value.\nRegression \n| \n175\nDownload from finelybook www.finelybook.com", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 482, "content": "Figure 6-5. Predictions of two Decision Tree regression models\nThe CART algorithm works mostly the same way as earlier, except that instead of try\u2010\ning to split the training set in a way that minimizes impurity, it now tries to split the\ntraining set in a way that minimizes the MSE. Equation 6-4 shows the cost function\nthat the algorithm tries to minimize.\nEquation 6-4. CART cost function for regression\nJ k, tk =\nmleft\nm MSEleft +\nmright\nm\nMSEright\nwhere\nMSEnode =\n\u2211\ni \u2208node\nynode \u2212y i 2\nynode =\n1\nmnode\n\u2211\ni \u2208node\ny i\nJust like for classification tasks, Decision Trees are prone to overfitting when dealing\nwith regression tasks. Without any regularization (i.e., using the default hyperpara\u2010\nmeters), you get the predictions on the left of Figure 6-6. It is obviously overfitting\nthe training set very badly. Just setting min_samples_leaf=10 results in a much more\nreasonable model, represented on the right of Figure 6-6.\nFigure 6-6. Regularizing a Decision Tree regressor\n176 \n|", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 483, "content": "Figure 6-6. Regularizing a Decision Tree regressor\n176 \n| \nChapter 6: Decision Trees\nDownload from finelybook www.finelybook.com", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 484, "content": "6 It randomly selects the set of features to evaluate at each node.\nInstability\nHopefully by now you are convinced that Decision Trees have a lot going for them:\nthey are simple to understand and interpret, easy to use, versatile, and powerful.\nHowever they do have a few limitations. First, as you may have noticed, Decision\nTrees love orthogonal decision boundaries (all splits are perpendicular to an axis),\nwhich makes them sensitive to training set rotation. For example, Figure 6-7 shows a\nsimple linearly separable dataset: on the left, a Decision Tree can split it easily, while\non the right, after the dataset is rotated by 45\u00b0, the decision boundary looks unneces\u2010\nsarily convoluted. Although both Decision Trees fit the training set perfectly, it is very\nlikely that the model on the right will not generalize well. One way to limit this prob\u2010\nlem is to use PCA (see Chapter 8), which often results in a better orientation of the\ntraining data.", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 485, "content": "training data.\nFigure 6-7. Sensitivity to training set rotation\nMore generally, the main issue with Decision Trees is that they are very sensitive to\nsmall variations in the training data. For example, if you just remove the widest Iris-\nVersicolor from the iris training set (the one with petals 4.8 cm long and 1.8 cm wide)\nand train a new Decision Tree, you may get the model represented in Figure 6-8. As\nyou can see, it looks very different from the previous Decision Tree (Figure 6-2).\nActually, since the training algorithm used by Scikit-Learn is stochastic6 you may\nget very different models even on the same training data (unless you set the\nrandom_state hyperparameter).\nInstability \n| \n177\nDownload from finelybook www.finelybook.com", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 486, "content": "Figure 6-8. Sensitivity to training set details\nRandom Forests can limit this instability by averaging predictions over many trees, as\nwe will see in the next chapter.\nExercises\n1. What is the approximate depth of a Decision Tree trained (without restrictions)\non a training set with 1 million instances?\n2. Is a node\u2019s Gini impurity generally lower or greater than its parent\u2019s? Is it gener\u2010\nally lower/greater, or always lower/greater?\n3. If a Decision Tree is overfitting the training set, is it a good idea to try decreasing\nmax_depth?\n4. If a Decision Tree is underfitting the training set, is it a good idea to try scaling\nthe input features?\n5. If it takes one hour to train a Decision Tree on a training set containing 1 million\ninstances, roughly how much time will it take to train another Decision Tree on a\ntraining set containing 10 million instances?\n6. If your training set contains 100,000 instances, will setting presort=True speed\nup training?", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 487, "content": "6. If your training set contains 100,000 instances, will setting presort=True speed\nup training?\n7. Train and fine-tune a Decision Tree for the moons dataset.\na. Generate a moons dataset using make_moons(n_samples=10000, noise=0.4).\nb. Split it into a training set and a test set using train_test_split().\n178 \n| \nChapter 6: Decision Trees\nDownload from finelybook www.finelybook.com", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 488, "content": "c. Use grid search with cross-validation (with the help of the GridSearchCV\nclass) to find good hyperparameter values for a DecisionTreeClassifier. \nHint: try various values for max_leaf_nodes.\nd. Train it on the full training set using these hyperparameters, and measure\nyour model\u2019s performance on the test set. You should get roughly 85% to 87%\naccuracy.\n8. Grow a forest.\na. Continuing the previous exercise, generate 1,000 subsets of the training set,\neach containing 100 instances selected randomly. Hint: you can use Scikit-\nLearn\u2019s ShuffleSplit class for this.\nb. Train one Decision Tree on each subset, using the best hyperparameter values\nfound above. Evaluate these 1,000 Decision Trees on the test set. Since they\nwere trained on smaller sets, these Decision Trees will likely perform worse\nthan the first Decision Tree, achieving only about 80% accuracy.\nc. Now comes the magic. For each test set instance, generate the predictions of", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 489, "content": "c. Now comes the magic. For each test set instance, generate the predictions of\nthe 1,000 Decision Trees, and keep only the most frequent prediction (you can\nuse SciPy\u2019s mode() function for this). This gives you majority-vote predictions\nover the test set.\nd. Evaluate these predictions on the test set: you should obtain a slightly higher\naccuracy than your first model (about 0.5 to 1.5% higher). Congratulations,\nyou have trained a Random Forest classifier!\nSolutions to these exercises are available in Appendix A.\nExercises \n| \n179\nDownload from finelybook www.finelybook.com", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 490, "content": "Download from finelybook www.finelybook.com", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 491, "content": "CHAPTER 7\nEnsemble Learning and Random Forests\nSuppose you ask a complex question to thousands of random people, then aggregate\ntheir answers. In many cases you will find that this aggregated answer is better than\nan expert\u2019s answer. This is called the wisdom of the crowd. Similarly, if you aggregate\nthe predictions of a group of predictors (such as classifiers or regressors), you will\noften get better predictions than with the best individual predictor. A group of pre\u2010\ndictors is called an ensemble; thus, this technique is called Ensemble Learning, and an\nEnsemble Learning algorithm is called an Ensemble method.\nFor example, you can train a group of Decision Tree classifiers, each on a different\nrandom subset of the training set. To make predictions, you just obtain the predic\u2010\ntions of all individual trees, then predict the class that gets the most votes (see the last\nexercise in Chapter 6). Such an ensemble of Decision Trees is called a Random Forest,", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 492, "content": "exercise in Chapter 6). Such an ensemble of Decision Trees is called a Random Forest, \nand despite its simplicity, this is one of the most powerful Machine Learning algo\u2010\nrithms available today.\nMoreover, as we discussed in Chapter 2, you will often use Ensemble methods near\nthe end of a project, once you have already built a few good predictors, to combine\nthem into an even better predictor. In fact, the winning solutions in Machine Learn\u2010\ning competitions often involve several Ensemble methods (most famously in the Net\u2010\nflix Prize competition).\nIn this chapter we will discuss the most popular Ensemble methods, including bag\u2010\nging, boosting, stacking, and a few others. We will also explore Random Forests.\nVoting Classifiers\nSuppose you have trained a few classifiers, each one achieving about 80% accuracy.\nYou may have a Logistic Regression classifier, an SVM classifier, a Random Forest\nclassifier, a K-Nearest Neighbors classifier, and perhaps a few more (see Figure 7-1).\n181", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 493, "content": "classifier, a K-Nearest Neighbors classifier, and perhaps a few more (see Figure 7-1).\n181\nDownload from finelybook www.finelybook.com", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 494, "content": "Figure 7-1. Training diverse classifiers\nA very simple way to create an even better classifier is to aggregate the predictions of\neach classifier and predict the class that gets the most votes. This majority-vote classi\u2010\nfier is called a hard voting classifier (see Figure 7-2).\nFigure 7-2. Hard voting classifier predictions\nSomewhat surprisingly, this voting classifier often achieves a higher accuracy than the\nbest classifier in the ensemble. In fact, even if each classifier is a weak learner (mean\u2010\ning it does only slightly better than random guessing), the ensemble can still be a\nstrong learner (achieving high accuracy), provided there are a sufficient number of\nweak learners and they are sufficiently diverse.\n182 \n| \nChapter 7: Ensemble Learning and Random Forests\nDownload from finelybook www.finelybook.com", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 495, "content": "How is this possible? The following analogy can help shed some light on this mystery.\nSuppose you have a slightly biased coin that has a 51% chance of coming up heads,\nand 49% chance of coming up tails. If you toss it 1,000 times, you will generally get\nmore or less 510 heads and 490 tails, and hence a majority of heads. If you do the\nmath, you will find that the probability of obtaining a majority of heads after 1,000\ntosses is close to 75%. The more you toss the coin, the higher the probability (e.g.,\nwith 10,000 tosses, the probability climbs over 97%). This is due to the law of large\nnumbers: as you keep tossing the coin, the ratio of heads gets closer and closer to the\nprobability of heads (51%). Figure 7-3 shows 10 series of biased coin tosses. You can\nsee that as the number of tosses increases, the ratio of heads approaches 51%. Eventu\u2010\nally all 10 series end up so close to 51% that they are consistently above 50%.\nFigure 7-3. The law of large numbers", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 496, "content": "Figure 7-3. The law of large numbers\nSimilarly, suppose you build an ensemble containing 1,000 classifiers that are individ\u2010\nually correct only 51% of the time (barely better than random guessing). If you pre\u2010\ndict the majority voted class, you can hope for up to 75% accuracy! However, this is\nonly true if all classifiers are perfectly independent, making uncorrelated errors,\nwhich is clearly not the case since they are trained on the same data. They are likely to\nmake the same types of errors, so there will be many majority votes for the wrong\nclass, reducing the ensemble\u2019s accuracy.\nEnsemble methods work best when the predictors are as independ\u2010\nent from one another as possible. One way to get diverse classifiers\nis to train them using very different algorithms. This increases the\nchance that they will make very different types of errors, improving\nthe ensemble\u2019s accuracy.\nVoting Classifiers \n| \n183\nDownload from finelybook www.finelybook.com", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 497, "content": "The following code creates and trains a voting classifier in Scikit-Learn, composed of\nthree diverse classifiers (the training set is the moons dataset, introduced in Chap\u2010\nter 5):\nfrom sklearn.ensemble import RandomForestClassifier\nfrom sklearn.ensemble import VotingClassifier\nfrom sklearn.linear_model import LogisticRegression\nfrom sklearn.svm import SVC\nlog_clf = LogisticRegression()\nrnd_clf = RandomForestClassifier()\nsvm_clf = SVC()\nvoting_clf = VotingClassifier(\n estimators=[('lr', log_clf), ('rf', rnd_clf), ('svc', svm_clf)],\n voting='hard'\n )\nvoting_clf.fit(X_train, y_train)\nLet\u2019s look at each classifier\u2019s accuracy on the test set:\n>>> from sklearn.metrics import accuracy_score\n>>> for clf in (log_clf, rnd_clf, svm_clf, voting_clf):\n>>> clf.fit(X_train, y_train)\n>>> y_pred = clf.predict(X_test)\n>>> print(clf.__class__.__name__, accuracy_score(y_test, y_pred))\nLogisticRegression 0.864\nRandomForestClassifier 0.872\nSVC 0.888\nVotingClassifier 0.896", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 498, "content": "LogisticRegression 0.864\nRandomForestClassifier 0.872\nSVC 0.888\nVotingClassifier 0.896\nThere you have it! The voting classifier slightly outperforms all the individual classifi\u2010\ners.\nIf all classifiers are able to estimate class probabilities (i.e., they have a pre\ndict_proba() method), then you can tell Scikit-Learn to predict the class with the\nhighest class probability, averaged over all the individual classifiers. This is called soft\nvoting. It often achieves higher performance than hard voting because it gives more\nweight to highly confident votes. All you need to do is replace voting=\"hard\" with\nvoting=\"soft\" and ensure that all classifiers can estimate class probabilities. This is\nnot the case of the SVC class by default, so you need to set its probability hyperpara\u2010\nmeter to True (this will make the SVC class use cross-validation to estimate class prob\u2010\nabilities, slowing down training, and it will add a predict_proba() method). If you", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 499, "content": "abilities, slowing down training, and it will add a predict_proba() method). If you\nmodify the preceding code to use soft voting, you will find that the voting classifier\nachieves over 91% accuracy!\n184 \n| \nChapter 7: Ensemble Learning and Random Forests\nDownload from finelybook www.finelybook.com", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 500, "content": "1 \u201cBagging Predictors,\u201d L. Breiman (1996).\n2 In statistics, resampling with replacement is called bootstrapping.\n3 \u201cPasting small votes for classification in large databases and on-line,\u201d L. Breiman (1999).\n4 Bias and variance were introduced in Chapter 4.\nBagging and Pasting\nOne way to get a diverse set of classifiers is to use very different training algorithms,\nas just discussed. Another approach is to use the same training algorithm for every\npredictor, but to train them on different random subsets of the training set. When\nsampling is performed with replacement, this method is called bagging1 (short for\nbootstrap aggregating2). When sampling is performed without replacement, it is called\npasting.3\nIn other words, both bagging and pasting allow training instances to be sampled sev\u2010\neral times across multiple predictors, but only bagging allows training instances to be\nsampled several times for the same predictor. This sampling and training process is\nrepresented in Figure 7-4.", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 501, "content": "represented in Figure 7-4.\nFigure 7-4. Pasting/bagging training set sampling and training\nOnce all predictors are trained, the ensemble can make a prediction for a new\ninstance by simply aggregating the predictions of all predictors. The aggregation\nfunction is typically the statistical mode (i.e., the most frequent prediction, just like a\nhard voting classifier) for classification, or the average for regression. Each individual\npredictor has a higher bias than if it were trained on the original training set, but\naggregation reduces both bias and variance.4 Generally, the net result is that the\nBagging and Pasting \n| \n185\nDownload from finelybook www.finelybook.com", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 502, "content": "5 max_samples can alternatively be set to a float between 0.0 and 1.0, in which case the max number of instances\nto sample is equal to the size of the training set times max_samples.\nensemble has a similar bias but a lower variance than a single predictor trained on the\noriginal training set.\nAs you can see in Figure 7-4, predictors can all be trained in parallel, via different\nCPU cores or even different servers. Similarly, predictions can be made in parallel.\nThis is one of the reasons why bagging and pasting are such popular methods: they\nscale very well.\nBagging and Pasting in Scikit-Learn\nScikit-Learn offers a simple API for both bagging and pasting with the BaggingClas\nsifier class (or BaggingRegressor for regression). The following code trains an\nensemble of 500 Decision Tree classifiers,5 each trained on 100 training instances ran\u2010\ndomly sampled from the training set with replacement (this is an example of bagging,", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 503, "content": "domly sampled from the training set with replacement (this is an example of bagging,\nbut if you want to use pasting instead, just set bootstrap=False). The n_jobs param\u2010\neter tells Scikit-Learn the number of CPU cores to use for training and predictions\n(\u20131 tells Scikit-Learn to use all available cores):\nfrom sklearn.ensemble import BaggingClassifier\nfrom sklearn.tree import DecisionTreeClassifier\nbag_clf = BaggingClassifier(\n DecisionTreeClassifier(), n_estimators=500,\n max_samples=100, bootstrap=True, n_jobs=-1\n )\nbag_clf.fit(X_train, y_train)\ny_pred = bag_clf.predict(X_test)\nThe BaggingClassifier automatically performs soft voting\ninstead of hard voting if the base classifier can estimate class proba\u2010\nbilities (i.e., if it has a predict_proba() method), which is the case\nwith Decision Trees classifiers.\nFigure 7-5 compares the decision boundary of a single Decision Tree with the deci\u2010\nsion boundary of a bagging ensemble of 500 trees (from the preceding code), both", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 504, "content": "sion boundary of a bagging ensemble of 500 trees (from the preceding code), both\ntrained on the moons dataset. As you can see, the ensemble\u2019s predictions will likely\ngeneralize much better than the single Decision Tree\u2019s predictions: the ensemble has a\ncomparable bias but a smaller variance (it makes roughly the same number of errors\non the training set, but the decision boundary is less irregular).\n186 \n| \nChapter 7: Ensemble Learning and Random Forests\nDownload from finelybook www.finelybook.com", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 505, "content": "6 As m grows, this ratio approaches 1 \u2013 exp(\u20131) \u2248 63.212%.\nFigure 7-5. A single Decision Tree versus a bagging ensemble of 500 trees\nBootstrapping introduces a bit more diversity in the subsets that each predictor is\ntrained on, so bagging ends up with a slightly higher bias than pasting, but this also\nmeans that predictors end up being less correlated so the ensemble\u2019s variance is\nreduced. Overall, bagging often results in better models, which explains why it is gen\u2010\nerally preferred. However, if you have spare time and CPU power you can use cross-\nvalidation to evaluate both bagging and pasting and select the one that works best.\nOut-of-Bag Evaluation\nWith bagging, some instances may be sampled several times for any given predictor,\nwhile others may not be sampled at all. By default a BaggingClassifier samples m\ntraining instances with replacement (bootstrap=True), where m is the size of the\ntraining set. This means that only about 63% of the training instances are sampled on", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 506, "content": "training set. This means that only about 63% of the training instances are sampled on\naverage for each predictor.6 The remaining 37% of the training instances that are not\nsampled are called out-of-bag (oob) instances. Note that they are not the same 37%\nfor all predictors.\nSince a predictor never sees the oob instances during training, it can be evaluated on\nthese instances, without the need for a separate validation set or cross-validation. You\ncan evaluate the ensemble itself by averaging out the oob evaluations of each predic\u2010\ntor.\nIn Scikit-Learn, you can set oob_score=True when creating a BaggingClassifier to\nrequest an automatic oob evaluation after training. The following code demonstrates\nthis. The resulting evaluation score is available through the oob_score_ variable:\n>>> bag_clf = BaggingClassifier(\n>>> DecisionTreeClassifier(), n_estimators=500,\n>>> bootstrap=True, n_jobs=-1, oob_score=True)\nBagging and Pasting \n| \n187", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 507, "content": ">>> bootstrap=True, n_jobs=-1, oob_score=True)\nBagging and Pasting \n| \n187\nDownload from finelybook www.finelybook.com", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 508, "content": "7 \u201cEnsembles on Random Patches,\u201d G. Louppe and P. Geurts (2012).\n8 \u201cThe random subspace method for constructing decision forests,\u201d Tin Kam Ho (1998).\n>>> bag_clf.fit(X_train, y_train)\n>>> bag_clf.oob_score_\n0.93066666666666664\nAccording to this oob evaluation, this BaggingClassifier is likely to achieve about\n93.1% accuracy on the test set. Let\u2019s verify this:\n>>> from sklearn.metrics import accuracy_score\n>>> y_pred = bag_clf.predict(X_test)\n>>> accuracy_score(y_test, y_pred)\n0.93600000000000005\nWe get 93.6% accuracy on the test set\u2014close enough!\nThe oob decision function for each training instance is also available through the\noob_decision_function_ variable. In this case (since the base estimator has a pre\ndict_proba() method) the decision function returns the class probabilities for each\ntraining instance. For example, the oob evaluation estimates that the second training\ninstance has a 60.6% probability of belonging to the positive class (and 39.4% of", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 509, "content": "instance has a 60.6% probability of belonging to the positive class (and 39.4% of\nbelonging to the positive class):\n>>> bag_clf.oob_decision_function_\narray([[ 0. , 1. ],\n [ 0.60588235, 0.39411765],\n [ 1. , 0. ],\n ...\n [ 1. , 0. ],\n [ 0. , 1. ],\n [ 0.48958333, 0.51041667]])\nRandom Patches and Random Subspaces\nThe BaggingClassifier class supports sampling the features as well. This is con\u2010\ntrolled by two hyperparameters: max_features and bootstrap_features. They work\nthe same way as max_samples and bootstrap, but for feature sampling instead of\ninstance sampling. Thus, each predictor will be trained on a random subset of the\ninput features.\nThis is particularly useful when you are dealing with high-dimensional inputs (such\nas images). Sampling both training instances and features is called the Random\nPatches method.7 Keeping all training instances (i.e., bootstrap=False and max_sam", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 510, "content": "Patches method.7 Keeping all training instances (i.e., bootstrap=False and max_sam\nples=1.0) but sampling features (i.e., bootstrap_features=True and/or max_fea\ntures smaller than 1.0) is called the Random Subspaces method.8\n188 \n| \nChapter 7: Ensemble Learning and Random Forests\nDownload from finelybook www.finelybook.com", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 511, "content": "9 \u201cRandom Decision Forests,\u201d T. Ho (1995).\n10 The BaggingClassifier class remains useful if you want a bag of something other than Decision Trees.\n11 There are a few notable exceptions: splitter is absent (forced to \"random\"), presort is absent (forced to\nFalse), max_samples is absent (forced to 1.0), and base_estimator is absent (forced to DecisionTreeClassi\nfier with the provided hyperparameters).\nSampling features results in even more predictor diversity, trading a bit more bias for\na lower variance.\nRandom Forests\nAs we have discussed, a Random Forest9 is an ensemble of Decision Trees, generally\ntrained via the bagging method (or sometimes pasting), typically with max_samples\nset to the size of the training set. Instead of building a BaggingClassifier and pass\u2010\ning it a DecisionTreeClassifier, you can instead use the RandomForestClassifier\nclass, which is more convenient and optimized for Decision Trees10 (similarly, there is", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 512, "content": "class, which is more convenient and optimized for Decision Trees10 (similarly, there is\na RandomForestRegressor class for regression tasks). The following code trains a\nRandom Forest classifier with 500 trees (each limited to maximum 16 nodes), using\nall available CPU cores:\nfrom sklearn.ensemble import RandomForestClassifier\nrnd_clf = RandomForestClassifier(n_estimators=500, max_leaf_nodes=16, n_jobs=-1)\nrnd_clf.fit(X_train, y_train)\ny_pred_rf = rnd_clf.predict(X_test)\nWith a few exceptions, a RandomForestClassifier has all the hyperparameters of a\nDecisionTreeClassifier (to control how trees are grown), plus all the hyperpara\u2010\nmeters of a BaggingClassifier to control the ensemble itself.11\nThe Random Forest algorithm introduces extra randomness when growing trees;\ninstead of searching for the very best feature when splitting a node (see Chapter 6), it\nsearches for the best feature among a random subset of features. This results in a", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 513, "content": "searches for the best feature among a random subset of features. This results in a\ngreater tree diversity, which (once again) trades a higher bias for a lower variance,\ngenerally yielding an overall better model. The following BaggingClassifier is\nroughly equivalent to the previous RandomForestClassifier:\nbag_clf = BaggingClassifier(\n DecisionTreeClassifier(splitter=\"random\", max_leaf_nodes=16),\n n_estimators=500, max_samples=1.0, bootstrap=True, n_jobs=-1\n )\nRandom Forests \n| \n189\nDownload from finelybook www.finelybook.com", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 514, "content": "12 \u201cExtremely randomized trees,\u201d P. Geurts, D. Ernst, L. Wehenkel (2005).\nExtra-Trees\nWhen you are growing a tree in a Random Forest, at each node only a random subset\nof the features is considered for splitting (as discussed earlier). It is possible to make\ntrees even more random by also using random thresholds for each feature rather than\nsearching for the best possible thresholds (like regular Decision Trees do).\nA forest of such extremely random trees is simply called an Extremely Randomized\nTrees ensemble12 (or Extra-Trees for short). Once again, this trades more bias for a\nlower variance. It also makes Extra-Trees much faster to train than regular Random\nForests since finding the best possible threshold for each feature at every node is one\nof the most time-consuming tasks of growing a tree.\nYou can create an Extra-Trees classifier using Scikit-Learn\u2019s ExtraTreesClassifier\nclass. Its API is identical to the RandomForestClassifier class. Similarly, the Extra", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 515, "content": "class. Its API is identical to the RandomForestClassifier class. Similarly, the Extra\nTreesRegressor class has the same API as the RandomForestRegressor class.\nIt is hard to tell in advance whether a RandomForestClassifier\nwill perform better or worse than an ExtraTreesClassifier. Gen\u2010\nerally, the only way to know is to try both and compare them using\ncross-validation (and tuning the hyperparameters using grid\nsearch).\nFeature Importance\nLastly, if you look at a single Decision Tree, important features are likely to appear\ncloser to the root of the tree, while unimportant features will often appear closer to\nthe leaves (or not at all). It is therefore possible to get an estimate of a feature\u2019s impor\u2010\ntance by computing the average depth at which it appears across all trees in the forest.\nScikit-Learn computes this automatically for every feature after training. You can\naccess the result using the feature_importances_ variable. For example, the follow\u2010", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 516, "content": "access the result using the feature_importances_ variable. For example, the follow\u2010\ning code trains a RandomForestClassifier on the iris dataset (introduced in Chap\u2010\nter 4) and outputs each feature\u2019s importance. It seems that the most important\nfeatures are the petal length (44%) and width (42%), while sepal length and width are\nrather unimportant in comparison (11% and 2%, respectively):\n>>> from sklearn.datasets import load_iris\n>>> iris = load_iris()\n>>> rnd_clf = RandomForestClassifier(n_estimators=500, n_jobs=-1)\n>>> rnd_clf.fit(iris[\"data\"], iris[\"target\"])\n>>> for name, score in zip(iris[\"feature_names\"], rnd_clf.feature_importances_):\n>>> print(name, score)\nsepal length (cm) 0.112492250999\n190 \n| \nChapter 7: Ensemble Learning and Random Forests\nDownload from finelybook www.finelybook.com", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 517, "content": "13 \u201cA Decision-Theoretic Generalization of On-Line Learning and an Application to Boosting,\u201d Yoav Freund,\nRobert E. Schapire (1997).\nsepal width (cm) 0.0231192882825\npetal length (cm) 0.441030464364\npetal width (cm) 0.423357996355\nSimilarly, if you train a Random Forest classifier on the MNIST dataset (introduced\nin Chapter 3) and plot each pixel\u2019s importance, you get the image represented in\nFigure 7-6.\nFigure 7-6. MNIST pixel importance (according to a Random Forest classifier)\nRandom Forests are very handy to get a quick understanding of what features\nactually matter, in particular if you need to perform feature selection.\nBoosting\nBoosting (originally called hypothesis boosting) refers to any Ensemble method that\ncan combine several weak learners into a strong learner. The general idea of most\nboosting methods is to train predictors sequentially, each trying to correct its prede\u2010\ncessor. There are many boosting methods available, but by far the most popular are", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 518, "content": "cessor. There are many boosting methods available, but by far the most popular are\nAdaBoost13 (short for Adaptive Boosting) and Gradient Boosting. Let\u2019s start with Ada\u2010\nBoost.\nBoosting \n| \n191\nDownload from finelybook www.finelybook.com", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 519, "content": "14 This is just for illustrative purposes. SVMs are generally not good base predictors for AdaBoost, because they\nare slow and tend to be unstable with AdaBoost.\nAdaBoost\nOne way for a new predictor to correct its predecessor is to pay a bit more attention\nto the training instances that the predecessor underfitted. This results in new predic\u2010\ntors focusing more and more on the hard cases. This is the technique used by Ada\u2010\nBoost.\nFor example, to build an AdaBoost classifier, a first base classifier (such as a Decision\nTree) is trained and used to make predictions on the training set. The relative weight\nof misclassified training instances is then increased. A second classifier is trained\nusing the updated weights and again it makes predictions on the training set, weights\nare updated, and so on (see Figure 7-7).\nFigure 7-7. AdaBoost sequential training with instance weight updates\nFigure 7-8 shows the decision boundaries of five consecutive predictors on the", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 520, "content": "Figure 7-8 shows the decision boundaries of five consecutive predictors on the\nmoons dataset (in this example, each predictor is a highly regularized SVM classifier\nwith an RBF kernel14). The first classifier gets many instances wrong, so their weights\nget boosted. The second classifier therefore does a better job on these instances, and\nso on. The plot on the right represents the same sequence of predictors except that\nthe learning rate is halved (i.e., the misclassified instance weights are boosted half as\nmuch at every iteration). As you can see, this sequential learning technique has some\nsimilarities with Gradient Descent, except that instead of tweaking a single predictor\u2019s\n192 \n| \nChapter 7: Ensemble Learning and Random Forests\nDownload from finelybook www.finelybook.com", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 521, "content": "parameters to minimize a cost function, AdaBoost adds predictors to the ensemble,\ngradually making it better.\nFigure 7-8. Decision boundaries of consecutive predictors\nOnce all predictors are trained, the ensemble makes predictions very much like bag\u2010\nging or pasting, except that predictors have different weights depending on their\noverall accuracy on the weighted training set.\nThere is one important drawback to this sequential learning techni\u2010\nque: it cannot be parallelized (or only partially), since each predic\u2010\ntor can only be trained after the previous predictor has been\ntrained and evaluated. As a result, it does not scale as well as bag\u2010\nging or pasting.\nLet\u2019s take a closer look at the AdaBoost algorithm. Each instance weight w(i) is initially\nset to 1\nm. A first predictor is trained and its weighted error rate r1 is computed on the\ntraining set; see Equation 7-1.\nEquation 7-1. Weighted error rate of the jth predictor\nrj =\n\u2211\ni = 1\ny j\ni \u2260y i\nm\nw i\n\u2211\ni = 1\nm\nw i\nwhere y j", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 522, "content": "rj =\n\u2211\ni = 1\ny j\ni \u2260y i\nm\nw i\n\u2211\ni = 1\nm\nw i\nwhere y j\ni is the jth predictor\u2019s prediction for the ith instance.\nBoosting \n| \n193\nDownload from finelybook www.finelybook.com", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 523, "content": "15 The original AdaBoost algorithm does not use a learning rate hyperparameter.\nThe predictor\u2019s weight \u03b1j is then computed using Equation 7-2, where \u03b7 is the learn\u2010\ning rate hyperparameter (defaults to 1).15 The more accurate the predictor is, the\nhigher its weight will be. If it is just guessing randomly, then its weight will be close to\nzero. However, if it is most often wrong (i.e., less accurate than random guessing),\nthen its weight will be negative.\nEquation 7-2. Predictor weight\n\u03b1j = \u03b7 log\n1 \u2212rj\nrj\nNext the instance weights are updated using Equation 7-3: the misclassified instances\nare boosted.\nEquation 7-3. Weight update rule\nfor i = 1, 2, \u22ef, m\nw i\nw i\nif yj\ni = y i\nw i exp \u03b1j if yj\ni \u2260y i\nThen all the instance weights are normalized (i.e., divided by \u2211i = 1\nm\nw i ).\nFinally, a new predictor is trained using the updated weights, and the whole process is\nrepeated (the new predictor\u2019s weight is computed, the instance weights are updated,", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 524, "content": "repeated (the new predictor\u2019s weight is computed, the instance weights are updated,\nthen another predictor is trained, and so on). The algorithm stops when the desired\nnumber of predictors is reached, or when a perfect predictor is found.\nTo make predictions, AdaBoost simply computes the predictions of all the predictors\nand weighs them using the predictor weights \u03b1j. The predicted class is the one that\nreceives the majority of weighted votes (see Equation 7-4).\nEquation 7-4. AdaBoost predictions\ny \ufffd= argmax\nk\n\u2211\nj = 1\ny j \ufffd= k\nN\n\u03b1j\nwhere N is the number of predictors.\n194 \n| \nChapter 7: Ensemble Learning and Random Forests\nDownload from finelybook www.finelybook.com", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 525, "content": "16 For more details, see \u201cMulti-Class AdaBoost,\u201d J. Zhu et al. (2006).\n17 First introduced in \u201cArcing the Edge,\u201d L. Breiman (1997).\nScikit-Learn actually uses a multiclass version of AdaBoost called SAMME16 (which\nstands for Stagewise Additive Modeling using a Multiclass Exponential loss function).\nWhen there are just two classes, SAMME is equivalent to AdaBoost. Moreover, if the\npredictors can estimate class probabilities (i.e., if they have a predict_proba()\nmethod), Scikit-Learn can use a variant of SAMME called SAMME.R (the R stands\nfor \u201cReal\u201d), which relies on class probabilities rather than predictions and generally\nperforms better.\nThe following code trains an AdaBoost classifier based on 200 Decision Stumps using\nScikit-Learn\u2019s AdaBoostClassifier class (as you might expect, there is also an Ada\nBoostRegressor class). A Decision Stump is a Decision Tree with max_depth=1\u2014in\nother words, a tree composed of a single decision node plus two leaf nodes. This is", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 526, "content": "other words, a tree composed of a single decision node plus two leaf nodes. This is\nthe default base estimator for the AdaBoostClassifier class:\nfrom sklearn.ensemble import AdaBoostClassifier\nada_clf = AdaBoostClassifier(\n DecisionTreeClassifier(max_depth=1), n_estimators=200,\n algorithm=\"SAMME.R\", learning_rate=0.5\n )\nada_clf.fit(X_train, y_train)\nIf your AdaBoost ensemble is overfitting the training set, you can\ntry reducing the number of estimators or more strongly regulariz\u2010\ning the base estimator.\nGradient Boosting\nAnother very popular Boosting algorithm is Gradient Boosting.17 Just like AdaBoost,\nGradient Boosting works by sequentially adding predictors to an ensemble, each one\ncorrecting its predecessor. However, instead of tweaking the instance weights at every\niteration like AdaBoost does, this method tries to fit the new predictor to the residual\nerrors made by the previous predictor.", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 527, "content": "errors made by the previous predictor.\nLet\u2019s go through a simple regression example using Decision Trees as the base predic\u2010\ntors (of course Gradient Boosting also works great with regression tasks). This is\ncalled Gradient Tree Boosting, or Gradient Boosted Regression Trees (GBRT). First, let\u2019s\nfit a DecisionTreeRegressor to the training set (for example, a noisy quadratic train\u2010\ning set):\nBoosting \n| \n195\nDownload from finelybook www.finelybook.com", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 528, "content": "from sklearn.tree import DecisionTreeRegressor\ntree_reg1 = DecisionTreeRegressor(max_depth=2)\ntree_reg1.fit(X, y)\nNow train a second DecisionTreeRegressor on the residual errors made by the first\npredictor:\ny2 = y - tree_reg1.predict(X)\ntree_reg2 = DecisionTreeRegressor(max_depth=2)\ntree_reg2.fit(X, y2)\nThen we train a third regressor on the residual errors made by the second predictor:\ny3 = y2 - tree_reg2.predict(X)\ntree_reg3 = DecisionTreeRegressor(max_depth=2)\ntree_reg3.fit(X, y3)\nNow we have an ensemble containing three trees. It can make predictions on a new\ninstance simply by adding up the predictions of all the trees:\ny_pred = sum(tree.predict(X_new) for tree in (tree_reg1, tree_reg2, tree_reg3))\nFigure 7-9 represents the predictions of these three trees in the left column, and the\nensemble\u2019s predictions in the right column. In the first row, the ensemble has just one\ntree, so its predictions are exactly the same as the first tree\u2019s predictions. In the second", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 529, "content": "tree, so its predictions are exactly the same as the first tree\u2019s predictions. In the second\nrow, a new tree is trained on the residual errors of the first tree. On the right you can\nsee that the ensemble\u2019s predictions are equal to the sum of the predictions of the first\ntwo trees. Similarly, in the third row another tree is trained on the residual errors of\nthe second tree. You can see that the ensemble\u2019s predictions gradually get better as\ntrees are added to the ensemble.\nA simpler way to train GBRT ensembles is to use Scikit-Learn\u2019s GradientBoostingRe\ngressor class. Much like the RandomForestRegressor class, it has hyperparameters to\ncontrol the growth of Decision Trees (e.g., max_depth, min_samples_leaf, and so on),\nas well as hyperparameters to control the ensemble training, such as the number of\ntrees (n_estimators). The following code creates the same ensemble as the previous\none:\nfrom sklearn.ensemble import GradientBoostingRegressor", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 530, "content": "one:\nfrom sklearn.ensemble import GradientBoostingRegressor\ngbrt = GradientBoostingRegressor(max_depth=2, n_estimators=3, learning_rate=1.0)\ngbrt.fit(X, y)\n196 \n| \nChapter 7: Ensemble Learning and Random Forests\nDownload from finelybook www.finelybook.com", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 531, "content": "Figure 7-9. Gradient Boosting\nThe learning_rate hyperparameter scales the contribution of each tree. If you set it\nto a low value, such as 0.1, you will need more trees in the ensemble to fit the train\u2010\ning set, but the predictions will usually generalize better. This is a regularization tech\u2010\nnique called shrinkage. Figure 7-10 shows two GBRT ensembles trained with a low\nlearning rate: the one on the left does not have enough trees to fit the training set,\nwhile the one on the right has too many trees and overfits the training set.\nBoosting \n| \n197\nDownload from finelybook www.finelybook.com", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 532, "content": "Figure 7-10. GBRT ensembles with not enough predictors (left) and too many (right)\nIn order to find the optimal number of trees, you can use early stopping (see Chap\u2010\nter 4). A simple way to implement this is to use the staged_predict() method: it\nreturns an iterator over the predictions made by the ensemble at each stage of train\u2010\ning (with one tree, two trees, etc.). The following code trains a GBRT ensemble with\n120 trees, then measures the validation error at each stage of training to find the opti\u2010\nmal number of trees, and finally trains another GBRT ensemble using the optimal\nnumber of trees:\nimport numpy as np\nfrom sklearn.model_selection import train_test_split\nfrom sklearn.metrics import mean_squared_error\nX_train, X_val, y_train, y_val = train_test_split(X, y)\ngbrt = GradientBoostingRegressor(max_depth=2, n_estimators=120)\ngbrt.fit(X_train, y_train)\nerrors = [mean_squared_error(y_val, y_pred)\n for y_pred in gbrt.staged_predict(X_val)]", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 533, "content": "errors = [mean_squared_error(y_val, y_pred)\n for y_pred in gbrt.staged_predict(X_val)]\nbst_n_estimators = np.argmin(errors)\ngbrt_best = GradientBoostingRegressor(max_depth=2,n_estimators=bst_n_estimators)\ngbrt_best.fit(X_train, y_train)\nThe validation errors are represented on the left of Figure 7-11, and the best model\u2019s\npredictions are represented on the right.\n198 \n| \nChapter 7: Ensemble Learning and Random Forests\nDownload from finelybook www.finelybook.com", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 534, "content": "Figure 7-11. Tuning the number of trees using early stopping\nIt is also possible to implement early stopping by actually stopping training early\n(instead of training a large number of trees first and then looking back to find the\noptimal number). You can do so by setting warm_start=True, which makes Scikit-\nLearn keep existing trees when the fit() method is called, allowing incremental\ntraining. The following code stops training when the validation error does not\nimprove for five iterations in a row:\ngbrt = GradientBoostingRegressor(max_depth=2, warm_start=True)\nmin_val_error = float(\"inf\")\nerror_going_up = 0\nfor n_estimators in range(1, 120):\n gbrt.n_estimators = n_estimators\n gbrt.fit(X_train, y_train)\n y_pred = gbrt.predict(X_val)\n val_error = mean_squared_error(y_val, y_pred)\n if val_error < min_val_error:\n min_val_error = val_error\n error_going_up = 0\n else:\n error_going_up += 1\n if error_going_up == 5:", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 535, "content": "error_going_up = 0\n else:\n error_going_up += 1\n if error_going_up == 5:\n break # early stopping\nThe GradientBoostingRegressor class also supports a subsample hyperparameter,\nwhich specifies the fraction of training instances to be used for training each tree. For\nexample, if subsample=0.25, then each tree is trained on 25% of the training instan\u2010\nces, selected randomly. As you can probably guess by now, this trades a higher bias\nfor a lower variance. It also speeds up training considerably. This technique is called\nStochastic Gradient Boosting.\nBoosting \n| \n199\nDownload from finelybook www.finelybook.com", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 536, "content": "18 \u201cStacked Generalization,\u201d D. Wolpert (1992).\n19 Alternatively, it is possible to use out-of-fold predictions. In some contexts this is called stacking, while using a\nhold-out set is called blending. However, for many people these terms are synonymous.\nIt is possible to use Gradient Boosting with other cost functions.\nThis is controlled by the loss hyperparameter (see Scikit-Learn\u2019s\ndocumentation for more details).\nStacking\nThe last Ensemble method we will discuss in this chapter is called stacking (short for\nstacked generalization).18 It is based on a simple idea: instead of using trivial functions\n(such as hard voting) to aggregate the predictions of all predictors in an ensemble,\nwhy don\u2019t we train a model to perform this aggregation? Figure 7-12 shows such an\nensemble performing a regression task on a new instance. Each of the bottom three\npredictors predicts a different value (3.1, 2.7, and 2.9), and then the final predictor", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 537, "content": "predictors predicts a different value (3.1, 2.7, and 2.9), and then the final predictor \n(called a blender, or a meta learner) takes these predictions as inputs and makes the\nfinal prediction (3.0).\nFigure 7-12. Aggregating predictions using a blending predictor\nTo train the blender, a common approach is to use a hold-out set.19 Let\u2019s see how it\nworks. First, the training set is split in two subsets. The first subset is used to train the\npredictors in the first layer (see Figure 7-13).\n200 \n| \nChapter 7: Ensemble Learning and Random Forests\nDownload from finelybook www.finelybook.com", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 538, "content": "Figure 7-13. Training the first layer\nNext, the first layer predictors are used to make predictions on the second (held-out)\nset (see Figure 7-14). This ensures that the predictions are \u201cclean,\u201d since the predictors\nnever saw these instances during training. Now for each instance in the hold-out set\nthere are three predicted values. We can create a new training set using these predic\u2010\nted values as input features (which makes this new training set three-dimensional),\nand keeping the target values. The blender is trained on this new training set, so it\nlearns to predict the target value given the first layer\u2019s predictions.\nFigure 7-14. Training the blender\nStacking \n| \n201\nDownload from finelybook www.finelybook.com", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 539, "content": "It is actually possible to train several different blenders this way (e.g., one using Lin\u2010\near Regression, another using Random Forest Regression, and so on): we get a whole\nlayer of blenders. The trick is to split the training set into three subsets: the first one is\nused to train the first layer, the second one is used to create the training set used to\ntrain the second layer (using predictions made by the predictors of the first layer),\nand the third one is used to create the training set to train the third layer (using pre\u2010\ndictions made by the predictors of the second layer). Once this is done, we can make\na prediction for a new instance by going through each layer sequentially, as shown in\nFigure 7-15.\nFigure 7-15. Predictions in a multilayer stacking ensemble\nUnfortunately, Scikit-Learn does not support stacking directly, but it is not too hard\nto roll out your own implementation (see the following exercises). Alternatively, you", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 540, "content": "to roll out your own implementation (see the following exercises). Alternatively, you\ncan use an open source implementation such as brew (available at https://github.com/\nviisar/brew).\nExercises\n1. If you have trained five different models on the exact same training data, and\nthey all achieve 95% precision, is there any chance that you can combine these\nmodels to get better results? If so, how? If not, why?\n2. What is the difference between hard and soft voting classifiers?\n202 \n| \nChapter 7: Ensemble Learning and Random Forests\nDownload from finelybook www.finelybook.com", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 541, "content": "3. Is it possible to speed up training of a bagging ensemble by distributing it across\nmultiple servers? What about pasting ensembles, boosting ensembles, random\nforests, or stacking ensembles?\n4. What is the benefit of out-of-bag evaluation?\n5. What makes Extra-Trees more random than regular Random Forests? How can\nthis extra randomness help? Are Extra-Trees slower or faster than regular Ran\u2010\ndom Forests?\n6. If your AdaBoost ensemble underfits the training data, what hyperparameters\nshould you tweak and how?\n7. If your Gradient Boosting ensemble overfits the training set, should you increase\nor decrease the learning rate?\n8. Load the MNIST data (introduced in Chapter 3), and split it into a training set, a\nvalidation set, and a test set (e.g., use the first 40,000 instances for training, the\nnext 10,000 for validation, and the last 10,000 for testing). Then train various\nclassifiers, such as a Random Forest classifier, an Extra-Trees classifier, and an", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 542, "content": "classifiers, such as a Random Forest classifier, an Extra-Trees classifier, and an\nSVM. Next, try to combine them into an ensemble that outperforms them all on\nthe validation set, using a soft or hard voting classifier. Once you have found one,\ntry it on the test set. How much better does it perform compared to the individ\u2010\nual classifiers?\n9. Run the individual classifiers from the previous exercise to make predictions on\nthe validation set, and create a new training set with the resulting predictions:\neach training instance is a vector containing the set of predictions from all your\nclassifiers for an image, and the target is the image\u2019s class. Congratulations, you\nhave just trained a blender, and together with the classifiers they form a stacking\nensemble! Now let\u2019s evaluate the ensemble on the test set. For each image in the\ntest set, make predictions with all your classifiers, then feed the predictions to the", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 543, "content": "test set, make predictions with all your classifiers, then feed the predictions to the\nblender to get the ensemble\u2019s predictions. How does it compare to the voting clas\u2010\nsifier you trained earlier?\nSolutions to these exercises are available in Appendix A.\nExercises \n| \n203\nDownload from finelybook www.finelybook.com", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 544, "content": "Download from finelybook www.finelybook.com", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 545, "content": "CHAPTER 8\nDimensionality Reduction\nMany Machine Learning problems involve thousands or even millions of features for\neach training instance. Not only does this make training extremely slow, it can also\nmake it much harder to find a good solution, as we will see. This problem is often\nreferred to as the curse of dimensionality.\nFortunately, in real-world problems, it is often possible to reduce the number of fea\u2010\ntures considerably, turning an intractable problem into a tractable one. For example,\nconsider the MNIST images (introduced in Chapter 3): the pixels on the image bor\u2010\nders are almost always white, so you could completely drop these pixels from the\ntraining set without losing much information. Figure 7-6 confirms that these pixels\nare utterly unimportant for the classification task. Moreover, two neighboring pixels\nare often highly correlated: if you merge them into a single pixel (e.g., by taking the\nmean of the two pixel intensities), you will not lose much information.", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 546, "content": "mean of the two pixel intensities), you will not lose much information.\nReducing dimensionality does lose some information (just like\ncompressing an image to JPEG can degrade its quality), so even\nthough it will speed up training, it may also make your system per\u2010\nform slightly worse. It also makes your pipelines a bit more com\u2010\nplex and thus harder to maintain. So you should first try to train\nyour system with the original data before considering using dimen\u2010\nsionality reduction if training is too slow. In some cases, however,\nreducing the dimensionality of the training data may filter out\nsome noise and unnecessary details and thus result in higher per\u2010\nformance (but in general it won\u2019t; it will just speed up training).\nApart from speeding up training, dimensionality reduction is also extremely useful\nfor data visualization (or DataViz). Reducing the number of dimensions down to two\n205\nDownload from finelybook www.finelybook.com", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 547, "content": "1 Well, four dimensions if you count time, and a few more if you are a string theorist.\n2 Watch a rotating tesseract projected into 3D space at http://goo.gl/OM7ktJ. Image by Wikipedia user Nerd\u2010\nBoy1392 (Creative Commons BY-SA 3.0). Reproduced from https://en.wikipedia.org/wiki/Tesseract.\n3 Fun fact: anyone you know is probably an extremist in at least one dimension (e.g., how much sugar they put\nin their coffee), if you consider enough dimensions.\n(or three) makes it possible to plot a high-dimensional training set on a graph and\noften gain some important insights by visually detecting patterns, such as clusters.\nIn this chapter we will discuss the curse of dimensionality and get a sense of what\ngoes on in high-dimensional space. Then, we will present the two main approaches to\ndimensionality reduction (projection and Manifold Learning), and we will go\nthrough three of the most popular dimensionality reduction techniques: PCA, Kernel\nPCA, and LLE.\nThe Curse of Dimensionality", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 548, "content": "PCA, and LLE.\nThe Curse of Dimensionality\nWe are so used to living in three dimensions1 that our intuition fails us when we try\nto imagine a high-dimensional space. Even a basic 4D hypercube is incredibly hard to\npicture in our mind (see Figure 8-1), let alone a 200-dimensional ellipsoid bent in a\n1,000-dimensional space.\nFigure 8-1. Point, segment, square, cube, and tesseract (0D to 4D hypercubes)2\nIt turns out that many things behave very differently in high-dimensional space. For\nexample, if you pick a random point in a unit square (a 1 \u00d7 1 square), it will have only\nabout a 0.4% chance of being located less than 0.001 from a border (in other words, it\nis very unlikely that a random point will be \u201cextreme\u201d along any dimension). But in a\n10,000-dimensional unit hypercube (a 1 \u00d7 1 \u00d7 \u22ef \u00d7 1 cube, with ten thousand 1s), this\nprobability is greater than 99.999999%. Most points in a high-dimensional hypercube\nare very close to the border.3\n206 \n| \nChapter 8: Dimensionality Reduction", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 549, "content": "are very close to the border.3\n206 \n| \nChapter 8: Dimensionality Reduction\nDownload from finelybook www.finelybook.com", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 550, "content": "Here is a more troublesome difference: if you pick two points randomly in a unit\nsquare, the distance between these two points will be, on average, roughly 0.52. If you\npick two random points in a unit 3D cube, the average distance will be roughly 0.66.\nBut what about two points picked randomly in a 1,000,000-dimensional hypercube?\nWell, the average distance, believe it or not, will be about 408.25 (roughly\n1, 000, 000/6)! This is quite counterintuitive: how can two points be so far apart\nwhen they both lie within the same unit hypercube? This fact implies that high-\ndimensional datasets are at risk of being very sparse: most training instances are\nlikely to be far away from each other. Of course, this also means that a new instance\nwill likely be far away from any training instance, making predictions much less relia\u2010\nble than in lower dimensions, since they will be based on much larger extrapolations.", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 551, "content": "ble than in lower dimensions, since they will be based on much larger extrapolations.\nIn short, the more dimensions the training set has, the greater the risk of overfitting\nit.\nIn theory, one solution to the curse of dimensionality could be to increase the size of\nthe training set to reach a sufficient density of training instances. Unfortunately, in\npractice, the number of training instances required to reach a given density grows\nexponentially with the number of dimensions. With just 100 features (much less than\nin the MNIST problem), you would need more training instances than atoms in the\nobservable universe in order for training instances to be within 0.1 of each other on\naverage, assuming they were spread out uniformly across all dimensions.\nMain Approaches for Dimensionality Reduction\nBefore we dive into specific dimensionality reduction algorithms, let\u2019s take a look at\nthe two main approaches to reducing dimensionality: projection and Manifold\nLearning.\nProjection", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 552, "content": "the two main approaches to reducing dimensionality: projection and Manifold\nLearning.\nProjection\nIn most real-world problems, training instances are not spread out uniformly across\nall dimensions. Many features are almost constant, while others are highly correlated\n(as discussed earlier for MNIST). As a result, all training instances actually lie within\n(or close to) a much lower-dimensional subspace of the high-dimensional space. This\nsounds very abstract, so let\u2019s look at an example. In Figure 8-2 you can see a 3D data\u2010\nset represented by the circles.\nMain Approaches for Dimensionality Reduction \n| \n207\nDownload from finelybook www.finelybook.com", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 553, "content": "Figure 8-2. A 3D dataset lying close to a 2D subspace\nNotice that all training instances lie close to a plane: this is a lower-dimensional (2D)\nsubspace of the high-dimensional (3D) space. Now if we project every training\ninstance perpendicularly onto this subspace (as represented by the short lines con\u2010\nnecting the instances to the plane), we get the new 2D dataset shown in Figure 8-3.\nTa-da! We have just reduced the dataset\u2019s dimensionality from 3D to 2D. Note that\nthe axes correspond to new features z1 and z2 (the coordinates of the projections on\nthe plane).\nFigure 8-3. The new 2D dataset after projection\n208 \n| \nChapter 8: Dimensionality Reduction\nDownload from finelybook www.finelybook.com", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 554, "content": "However, projection is not always the best approach to dimensionality reduction. In\nmany cases the subspace may twist and turn, such as in the famous Swiss roll toy data\u2010\nset represented in Figure 8-4.\nFigure 8-4. Swiss roll dataset\nSimply projecting onto a plane (e.g., by dropping x3) would squash different layers of\nthe Swiss roll together, as shown on the left of Figure 8-5. However, what you really\nwant is to unroll the Swiss roll to obtain the 2D dataset on the right of Figure 8-5.\nFigure 8-5. Squashing by projecting onto a plane (left) versus unrolling the Swiss roll\n(right)\nMain Approaches for Dimensionality Reduction \n| \n209\nDownload from finelybook www.finelybook.com", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 555, "content": "Manifold Learning\nThe Swiss roll is an example of a 2D manifold. Put simply, a 2D manifold is a 2D\nshape that can be bent and twisted in a higher-dimensional space. More generally, a\nd-dimensional manifold is a part of an n-dimensional space (where d < n) that locally\nresembles a d-dimensional hyperplane. In the case of the Swiss roll, d = 2 and n = 3: it\nlocally resembles a 2D plane, but it is rolled in the third dimension.\nMany dimensionality reduction algorithms work by modeling the manifold on which\nthe training instances lie; this is called Manifold Learning. It relies on the manifold\nassumption, also called the manifold hypothesis, which holds that most real-world\nhigh-dimensional datasets lie close to a much lower-dimensional manifold. This\nassumption is very often empirically observed.\nOnce again, think about the MNIST dataset: all handwritten digit images have some\nsimilarities. They are made of connected lines, the borders are white, they are more", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 556, "content": "similarities. They are made of connected lines, the borders are white, they are more\nor less centered, and so on. If you randomly generated images, only a ridiculously\ntiny fraction of them would look like handwritten digits. In other words, the degrees\nof freedom available to you if you try to create a digit image are dramatically lower\nthan the degrees of freedom you would have if you were allowed to generate any\nimage you wanted. These constraints tend to squeeze the dataset into a lower-\ndimensional manifold.\nThe manifold assumption is often accompanied by another implicit assumption: that\nthe task at hand (e.g., classification or regression) will be simpler if expressed in the\nlower-dimensional space of the manifold. For example, in the top row of Figure 8-6\nthe Swiss roll is split into two classes: in the 3D space (on the left), the decision\nboundary would be fairly complex, but in the 2D unrolled manifold space (on the\nright), the decision boundary is a simple straight line.", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 557, "content": "right), the decision boundary is a simple straight line.\nHowever, this assumption does not always hold. For example, in the bottom row of\nFigure 8-6, the decision boundary is located at x1 = 5. This decision boundary looks\nvery simple in the original 3D space (a vertical plane), but it looks more complex in\nthe unrolled manifold (a collection of four independent line segments).\nIn short, if you reduce the dimensionality of your training set before training a\nmodel, it will definitely speed up training, but it may not always lead to a better or\nsimpler solution; it all depends on the dataset.\nHopefully you now have a good sense of what the curse of dimensionality is and how\ndimensionality reduction algorithms can fight it, especially when the manifold\nassumption holds. The rest of this chapter will go through some of the most popular\nalgorithms.\n210 \n| \nChapter 8: Dimensionality Reduction\nDownload from finelybook www.finelybook.com", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 558, "content": "Figure 8-6. The decision boundary may not always be simpler with lower dimensions\nPCA\nPrincipal Component Analysis (PCA) is by far the most popular dimensionality reduc\u2010\ntion algorithm. First it identifies the hyperplane that lies closest to the data, and then\nit projects the data onto it.\nPreserving the Variance\nBefore you can project the training set onto a lower-dimensional hyperplane, you\nfirst need to choose the right hyperplane. For example, a simple 2D dataset is repre\u2010\nsented on the left of Figure 8-7, along with three different axes (i.e., one-dimensional\nhyperplanes). On the right is the result of the projection of the dataset onto each of\nthese axes. As you can see, the projection onto the solid line preserves the maximum\nvariance, while the projection onto the dotted line preserves very little variance, and\nthe projection onto the dashed line preserves an intermediate amount of variance.\nPCA \n| \n211\nDownload from finelybook www.finelybook.com", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 559, "content": "4 \u201cOn Lines and Planes of Closest Fit to Systems of Points in Space,\u201d K. Pearson (1901).\nFigure 8-7. Selecting the subspace onto which to project\nIt seems reasonable to select the axis that preserves the maximum amount of var\u2010\niance, as it will most likely lose less information than the other projections. Another\nway to justify this choice is that it is the axis that minimizes the mean squared dis\u2010\ntance between the original dataset and its projection onto that axis. This is the rather\nsimple idea behind PCA.4\nPrincipal Components\nPCA identifies the axis that accounts for the largest amount of variance in the train\u2010\ning set. In Figure 8-7, it is the solid line. It also finds a second axis, orthogonal to the\nfirst one, that accounts for the largest amount of remaining variance. In this 2D\nexample there is no choice: it is the dotted line. If it were a higher-dimensional data\u2010\nset, PCA would also find a third axis, orthogonal to both previous axes, and a fourth,", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 560, "content": "set, PCA would also find a third axis, orthogonal to both previous axes, and a fourth,\na fifth, and so on\u2014as many axes as the number of dimensions in the dataset.\nThe unit vector that defines the ith axis is called the ith principal component (PC). In\nFigure 8-7, the 1st PC is c1 and the 2nd PC is c2. In Figure 8-2 the first two PCs are\nrepresented by the orthogonal arrows in the plane, and the third PC would be\northogonal to the plane (pointing up or down).\n212 \n| \nChapter 8: Dimensionality Reduction\nDownload from finelybook www.finelybook.com", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 561, "content": "The direction of the principal components is not stable: if you per\u2010\nturb the training set slightly and run PCA again, some of the new\nPCs may point in the opposite direction of the original PCs. How\u2010\never, they will generally still lie on the same axes. In some cases, a\npair of PCs may even rotate or swap, but the plane they define will\ngenerally remain the same.\nSo how can you find the principal components of a training set? Luckily, there is a\nstandard matrix factorization technique called Singular Value Decomposition (SVD)\nthat can decompose the training set matrix X into the dot product of three matrices U\n\u00b7 \u03a3 \u00b7 VT, where VT contains all the principal components that we are looking for, as\nshown in Equation 8-1.\nEquation 8-1. Principal components matrix\n\ufffdT =\n\u2223\n\u2223\n\u2223\n\ufffd1 \ufffd2 \u22ef\ufffd\ufffd\n\u2223\n\u2223\n\u2223\nThe following Python code uses NumPy\u2019s svd() function to obtain all the principal\ncomponents of the training set, then extracts the first two PCs:\nX_centered = X - X.mean(axis=0)", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 562, "content": "components of the training set, then extracts the first two PCs:\nX_centered = X - X.mean(axis=0)\nU, s, V = np.linalg.svd(X_centered)\nc1 = V.T[:, 0]\nc2 = V.T[:, 1]\nPCA assumes that the dataset is centered around the origin. As we\nwill see, Scikit-Learn\u2019s PCA classes take care of centering the data\nfor you. However, if you implement PCA yourself (as in the pre\u2010\nceding example), or if you use other libraries, don\u2019t forget to center\nthe data first.\nProjecting Down to d Dimensions\nOnce you have identified all the principal components, you can reduce the dimen\u2010\nsionality of the dataset down to d dimensions by projecting it onto the hyperplane\ndefined by the first d principal components. Selecting this hyperplane ensures that the\nprojection will preserve as much variance as possible. For example, in Figure 8-2 the\n3D dataset is projected down to the 2D plane defined by the first two principal com\u2010\nponents, preserving a large part of the dataset\u2019s variance. As a result, the 2D projec\u2010", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 563, "content": "ponents, preserving a large part of the dataset\u2019s variance. As a result, the 2D projec\u2010\ntion looks very much like the original 3D dataset.\nTo project the training set onto the hyperplane, you can simply compute the dot\nproduct of the training set matrix X by the matrix Wd, defined as the matrix contain\u2010\nPCA \n| \n213\nDownload from finelybook www.finelybook.com", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 564, "content": "ing the first d principal components (i.e., the matrix composed of the first d columns\nof VT), as shown in Equation 8-2.\nEquation 8-2. Projecting the training set down to d dimensions\n\ufffdd\u2010proj = \ufffd\u00b7 \ufffdd\nThe following Python code projects the training set onto the plane defined by the first\ntwo principal components:\nW2 = V.T[:, :2]\nX2D = X_centered.dot(W2)\nThere you have it! You now know how to reduce the dimensionality of any dataset\ndown to any number of dimensions, while preserving as much variance as possible.\nUsing Scikit-Learn\nScikit-Learn\u2019s PCA class implements PCA using SVD decomposition just like we did\nbefore. The following code applies PCA to reduce the dimensionality of the dataset\ndown to two dimensions (note that it automatically takes care of centering the data):\nfrom sklearn.decomposition import PCA\npca = PCA(n_components = 2)\nX2D = pca.fit_transform(X)\nAfter fitting the PCA transformer to the dataset, you can access the principal compo\u2010", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 565, "content": "After fitting the PCA transformer to the dataset, you can access the principal compo\u2010\nnents using the components_ variable (note that it contains the PCs as horizontal vec\u2010\ntors, so, for example, the first principal component is equal to pca.components_.T[:,\n0]).\nExplained Variance Ratio\nAnother very useful piece of information is the explained variance ratio of each prin\u2010\ncipal component, available via the explained_variance_ratio_ variable. It indicates\nthe proportion of the dataset\u2019s variance that lies along the axis of each principal com\u2010\nponent. For example, let\u2019s look at the explained variance ratios of the first two compo\u2010\nnents of the 3D dataset represented in Figure 8-2:\n>>> print(pca.explained_variance_ratio_)\narray([ 0.84248607, 0.14631839])\nThis tells you that 84.2% of the dataset\u2019s variance lies along the first axis, and 14.6%\nlies along the second axis. This leaves less than 1.2% for the third axis, so it is reason\u2010", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 566, "content": "lies along the second axis. This leaves less than 1.2% for the third axis, so it is reason\u2010\nable to assume that it probably carries little information.\n214 \n| \nChapter 8: Dimensionality Reduction\nDownload from finelybook www.finelybook.com", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 567, "content": "Choosing the Right Number of Dimensions\nInstead of arbitrarily choosing the number of dimensions to reduce down to, it is\ngenerally preferable to choose the number of dimensions that add up to a sufficiently\nlarge portion of the variance (e.g., 95%). Unless, of course, you are reducing dimen\u2010\nsionality for data visualization\u2014in that case you will generally want to reduce the\ndimensionality down to 2 or 3.\nThe following code computes PCA without reducing dimensionality, then computes\nthe minimum number of dimensions required to preserve 95% of the training set\u2019s\nvariance:\npca = PCA()\npca.fit(X)\ncumsum = np.cumsum(pca.explained_variance_ratio_)\nd = np.argmax(cumsum >= 0.95) + 1\nYou could then set n_components=d and run PCA again. However, there is a much\nbetter option: instead of specifying the number of principal components you want to\npreserve, you can set n_components to be a float between 0.0 and 1.0, indicating the\nratio of variance you wish to preserve:", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 568, "content": "ratio of variance you wish to preserve:\npca = PCA(n_components=0.95)\nX_reduced = pca.fit_transform(X)\nYet another option is to plot the explained variance as a function of the number of\ndimensions (simply plot cumsum; see Figure 8-8). There will usually be an elbow in the\ncurve, where the explained variance stops growing fast. You can think of this as the\nintrinsic dimensionality of the dataset. In this case, you can see that reducing the\ndimensionality down to about 100 dimensions wouldn\u2019t lose too much explained var\u2010\niance.\nFigure 8-8. Explained variance as a function of the number of dimensions\nPCA \n| \n215\nDownload from finelybook www.finelybook.com", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 569, "content": "PCA for Compression\nObviously after dimensionality reduction, the training set takes up much less space.\nFor example, try applying PCA to the MNIST dataset while preserving 95% of its var\u2010\niance. You should find that each instance will have just over 150 features, instead of\nthe original 784 features. So while most of the variance is preserved, the dataset is\nnow less than 20% of its original size! This is a reasonable compression ratio, and you\ncan see how this can speed up a classification algorithm (such as an SVM classifier)\ntremendously.\nIt is also possible to decompress the reduced dataset back to 784 dimensions by\napplying the inverse transformation of the PCA projection. Of course this won\u2019t give\nyou back the original data, since the projection lost a bit of information (within the\n5% variance that was dropped), but it will likely be quite close to the original data.\nThe mean squared distance between the original data and the reconstructed data", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 570, "content": "The mean squared distance between the original data and the reconstructed data\n(compressed and then decompressed) is called the reconstruction error. For example,\nthe following code compresses the MNIST dataset down to 154 dimensions, then uses\nthe inverse_transform() method to decompress it back to 784 dimensions.\nFigure 8-9 shows a few digits from the original training set (on the left), and the cor\u2010\nresponding digits after compression and decompression. You can see that there is a\nslight image quality loss, but the digits are still mostly intact.\npca = PCA(n_components = 154)\nX_mnist_reduced = pca.fit_transform(X_mnist)\nX_mnist_recovered = pca.inverse_transform(X_mnist_reduced)\nFigure 8-9. MNIST compression preserving 95% of the variance\n216 \n| \nChapter 8: Dimensionality Reduction\nDownload from finelybook www.finelybook.com", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 571, "content": "5 Scikit-Learn uses the algorithm described in \u201cIncremental Learning for Robust Visual Tracking,\u201d D. Ross et al.\n(2007).\nThe equation of the inverse transformation is shown in Equation 8-3.\nEquation 8-3. PCA inverse transformation, back to the original number of\ndimensions\n\ufffdrecovered = \ufffdd\u2010proj \u00b7 \ufffdd\nT\nIncremental PCA\nOne problem with the preceding implementation of PCA is that it requires the whole\ntraining set to fit in memory in order for the SVD algorithm to run. Fortunately,\nIncremental PCA (IPCA) algorithms have been developed: you can split the training\nset into mini-batches and feed an IPCA algorithm one mini-batch at a time. This is\nuseful for large training sets, and also to apply PCA online (i.e., on the fly, as new\ninstances arrive).\nThe following code splits the MNIST dataset into 100 mini-batches (using NumPy\u2019s\narray_split() function) and feeds them to Scikit-Learn\u2019s IncrementalPCA class5 to \nreduce the dimensionality of the MNIST dataset down to 154 dimensions (just like", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 572, "content": "reduce the dimensionality of the MNIST dataset down to 154 dimensions (just like\nbefore). Note that you must call the partial_fit() method with each mini-batch\nrather than the fit() method with the whole training set:\nfrom sklearn.decomposition import IncrementalPCA\nn_batches = 100\ninc_pca = IncrementalPCA(n_components=154)\nfor X_batch in np.array_split(X_mnist, n_batches):\n inc_pca.partial_fit(X_batch)\nX_mnist_reduced = inc_pca.transform(X_mnist)\nAlternatively, you can use NumPy\u2019s memmap class, which allows you to manipulate a\nlarge array stored in a binary file on disk as if it were entirely in memory; the class\nloads only the data it needs in memory, when it needs it. Since the IncrementalPCA\nclass uses only a small part of the array at any given time, the memory usage remains\nunder control. This makes it possible to call the usual fit() method, as you can see\nin the following code:\nX_mm = np.memmap(filename, dtype=\"float32\", mode=\"readonly\", shape=(m, n))", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 573, "content": "in the following code:\nX_mm = np.memmap(filename, dtype=\"float32\", mode=\"readonly\", shape=(m, n))\nbatch_size = m // n_batches\ninc_pca = IncrementalPCA(n_components=154, batch_size=batch_size)\ninc_pca.fit(X_mm)\nPCA \n| \n217\nDownload from finelybook www.finelybook.com", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 574, "content": "6 \u201cKernel Principal Component Analysis,\u201d B. Sch\u00f6lkopf, A. Smola, K. M\u00fcller (1999).\nRandomized PCA\nScikit-Learn offers yet another option to perform PCA, called Randomized PCA. This\nis a stochastic algorithm that quickly finds an approximation of the first d principal\ncomponents. Its computational complexity is O(m \u00d7 d2) + O(d3), instead of O(m \u00d7 n2)\n+ O(n3), so it is dramatically faster than the previous algorithms when d is much \nsmaller than n.\nrnd_pca = PCA(n_components=154, svd_solver=\"randomized\")\nX_reduced = rnd_pca.fit_transform(X_mnist)\nKernel PCA\nIn Chapter 5 we discussed the kernel trick, a mathematical technique that implicitly\nmaps instances into a very high-dimensional space (called the feature space), enabling\nnonlinear classification and regression with Support Vector Machines. Recall that a\nlinear decision boundary in the high-dimensional feature space corresponds to a\ncomplex nonlinear decision boundary in the original space.", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 575, "content": "complex nonlinear decision boundary in the original space.\nIt turns out that the same trick can be applied to PCA, making it possible to perform\ncomplex nonlinear projections for dimensionality reduction. This is called Kernel\nPCA (kPCA).6 It is often good at preserving clusters of instances after projection, or\nsometimes even unrolling datasets that lie close to a twisted manifold.\nFor example, the following code uses Scikit-Learn\u2019s KernelPCA class to perform kPCA\nwith an RBF kernel (see Chapter 5 for more details about the RBF kernel and the\nother kernels):\nfrom sklearn.decomposition import KernelPCA\nrbf_pca = KernelPCA(n_components = 2, kernel=\"rbf\", gamma=0.04)\nX_reduced = rbf_pca.fit_transform(X)\nFigure 8-10 shows the Swiss roll, reduced to two dimensions using a linear kernel\n(equivalent to simply using the PCA class), an RBF kernel, and a sigmoid kernel\n(Logistic).\n218 \n| \nChapter 8: Dimensionality Reduction\nDownload from finelybook www.finelybook.com", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 576, "content": "Figure 8-10. Swiss roll reduced to 2D using kPCA with various kernels\nSelecting a Kernel and Tuning Hyperparameters\nAs kPCA is an unsupervised learning algorithm, there is no obvious performance\nmeasure to help you select the best kernel and hyperparameter values. However,\ndimensionality reduction is often a preparation step for a supervised learning task\n(e.g., classification), so you can simply use grid search to select the kernel and hyper\u2010\nparameters that lead to the best performance on that task. For example, the following\ncode creates a two-step pipeline, first reducing dimensionality to two dimensions\nusing kPCA, then applying Logistic Regression for classification. Then it uses Grid\nSearchCV to find the best kernel and gamma value for kPCA in order to get the best\nclassification accuracy at the end of the pipeline:\nfrom sklearn.model_selection import GridSearchCV\nfrom sklearn.linear_model import LogisticRegression\nfrom sklearn.pipeline import Pipeline\nclf = Pipeline([", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 577, "content": "from sklearn.pipeline import Pipeline\nclf = Pipeline([\n (\"kpca\", KernelPCA(n_components=2)),\n (\"log_reg\", LogisticRegression())\n ])\nparam_grid = [{\n \"kpca__gamma\": np.linspace(0.03, 0.05, 10),\n \"kpca__kernel\": [\"rbf\", \"sigmoid\"]\n }]\ngrid_search = GridSearchCV(clf, param_grid, cv=3)\ngrid_search.fit(X, y)\nThe best kernel and hyperparameters are then available through the best_params_\nvariable:\n>>> print(grid_search.best_params_)\n{'kpca__gamma': 0.043333333333333335, 'kpca__kernel': 'rbf'}\nKernel PCA \n| \n219\nDownload from finelybook www.finelybook.com", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 578, "content": "Another approach, this time entirely unsupervised, is to select the kernel and hyper\u2010\nparameters that yield the lowest reconstruction error. However, reconstruction is not\nas easy as with linear PCA. Here\u2019s why. Figure 8-11 shows the original Swiss roll 3D\ndataset (top left), and the resulting 2D dataset after kPCA is applied using an RBF\nkernel (top right). Thanks to the kernel trick, this is mathematically equivalent to\nmapping the training set to an infinite-dimensional feature space (bottom right)\nusing the feature map \u03c6, then projecting the transformed training set down to 2D\nusing linear PCA. Notice that if we could invert the linear PCA step for a given\ninstance in the reduced space, the reconstructed point would lie in feature space, not\nin the original space (e.g., like the one represented by an x in the diagram). Since the\nfeature space is infinite-dimensional, we cannot compute the reconstructed point,", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 579, "content": "feature space is infinite-dimensional, we cannot compute the reconstructed point,\nand therefore we cannot compute the true reconstruction error. Fortunately, it is pos\u2010\nsible to find a point in the original space that would map close to the reconstructed\npoint. This is called the reconstruction pre-image. Once you have this pre-image, you\ncan measure its squared distance to the original instance. You can then select the ker\u2010\nnel and hyperparameters that minimize this reconstruction pre-image error.\nFigure 8-11. Kernel PCA and the reconstruction pre-image error\n220 \n| \nChapter 8: Dimensionality Reduction\nDownload from finelybook www.finelybook.com", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 580, "content": "7 Scikit-Learn uses the algorithm based on Kernel Ridge Regression described in Gokhan H. Bak\u0131r, Jason\nWeston, and Bernhard Scholkopf, \u201cLearning to Find Pre-images\u201d (Tubingen, Germany: Max Planck Institute\nfor Biological Cybernetics, 2004).\n8 \u201cNonlinear Dimensionality Reduction by Locally Linear Embedding,\u201d S. Roweis, L. Saul (2000).\nYou may be wondering how to perform this reconstruction. One solution is to train a\nsupervised regression model, with the projected instances as the training set and the\noriginal instances as the targets. Scikit-Learn will do this automatically if you set\nfit_inverse_transform=True, as shown in the following code:7\nrbf_pca = KernelPCA(n_components = 2, kernel=\"rbf\", gamma=0.0433,\n fit_inverse_transform=True)\nX_reduced = rbf_pca.fit_transform(X)\nX_preimage = rbf_pca.inverse_transform(X_reduced)\nBy default, fit_inverse_transform=False and KernelPCA has no\ninverse_transform() method. This method only gets created", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 581, "content": "inverse_transform() method. This method only gets created\nwhen you set fit_inverse_transform=True.\nYou can then compute the reconstruction pre-image error:\n>>> from sklearn.metrics import mean_squared_error\n>>> mean_squared_error(X, X_preimage)\n32.786308795766132\nNow you can use grid search with cross-validation to find the kernel and hyperpara\u2010\nmeters that minimize this pre-image reconstruction error.\nLLE\nLocally Linear Embedding (LLE)8 is another very powerful nonlinear dimensionality\nreduction (NLDR) technique. It is a Manifold Learning technique that does not rely\non projections like the previous algorithms. In a nutshell, LLE works by first measur\u2010\ning how each training instance linearly relates to its closest neighbors (c.n.), and then\nlooking for a low-dimensional representation of the training set where these local\nrelationships are best preserved (more details shortly). This makes it particularly", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 582, "content": "relationships are best preserved (more details shortly). This makes it particularly\ngood at unrolling twisted manifolds, especially when there is not too much noise.\nFor example, the following code uses Scikit-Learn\u2019s LocallyLinearEmbedding class to\nunroll the Swiss roll. The resulting 2D dataset is shown in Figure 8-12. As you can\nsee, the Swiss roll is completely unrolled and the distances between instances are\nlocally well preserved. However, distances are not preserved on a larger scale: the left\npart of the unrolled Swiss roll is squeezed, while the right part is stretched. Neverthe\u2010\nless, LLE did a pretty good job at modeling the manifold.\nLLE \n| \n221\nDownload from finelybook www.finelybook.com", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 583, "content": "from sklearn.manifold import LocallyLinearEmbedding\nlle = LocallyLinearEmbedding(n_components=2, n_neighbors=10)\nX_reduced = lle.fit_transform(X)\nFigure 8-12. Unrolled Swiss roll using LLE\nHere\u2019s how LLE works: first, for each training instance x(i), the algorithm identifies its\nk closest neighbors (in the preceding code k = 10), then tries to reconstruct x(i) as a\nlinear function of these neighbors. More specifically, it finds the weights wi,j such that\nthe squared distance between x(i) and \u2211j = 1\nm\nwi, j\ufffdj is as small as possible, assuming\nwi,j = 0 if x(j) is not one of the k closest neighbors of x(i). Thus the first step of LLE is\nthe constrained optimization problem described in Equation 8-4, where W is the\nweight matrix containing all the weights wi,j. The second constraint simply normalizes\nthe weights for each training instance x(i).\n222 \n| \nChapter 8: Dimensionality Reduction\nDownload from finelybook www.finelybook.com", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 584, "content": "Equation 8-4. LLE step 1: linearly modeling local relationships\n\ufffd= argmin\n\ufffd\n\u2211\ni = 1\nm\n\u2225\ufffdi \u2212\u2211\nj = 1\nm\nwi, j\ufffdj \u2225\n2\nsubject to\nwi, j = 0\nif \ufffdj is not one of the k c.n. of \ufffdi\n\u2211\nj = 1\nm\nwi, j = 1 for i = 1, 2, \u22ef, m\nAfter this step, the weight matrix \ufffd (containing the weights wi, j) encodes the local\nlinear relationships between the training instances. Now the second step is to map the\ntraining instances into a d-dimensional space (where d < n) while preserving these\nlocal relationships as much as possible. If z(i) is the image of x(i) in this d-dimensional\nspace, then we want the squared distance between z(i) and \u2211j = 1\nm\nwi, j\ufffdj to be as small\nas possible. This idea leads to the unconstrained optimization problem described in\nEquation 8-5. It looks very similar to the first step, but instead of keeping the instan\u2010\nces fixed and finding the optimal weights, we are doing the reverse: keeping the\nweights fixed and finding the optimal position of the instances\u2019 images in the low-", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 585, "content": "weights fixed and finding the optimal position of the instances\u2019 images in the low-\ndimensional space. Note that Z is the matrix containing all z(i).\nEquation 8-5. LLE step 2: reducing dimensionality while preserving relationships\n\ufffd= argmin\n\ufffd\n\u2211\ni = 1\nm\n\u2225\ufffdi \u2212\u2211\nj = 1\nm\nwi, j\ufffdj \u2225\n2\nScikit-Learn\u2019s LLE implementation has the following computational complexity:\nO(m log(m)n log(k)) for finding the k nearest neighbors, O(mnk3) for optimizing the\nweights, and O(dm2) for constructing the low-dimensional representations. Unfortu\u2010\nnately, the m2 in the last term makes this algorithm scale poorly to very large datasets.\nOther Dimensionality Reduction Techniques\nThere are many other dimensionality reduction techniques, several of which are\navailable in Scikit-Learn. Here are some of the most popular:\n\u2022 Multidimensional Scaling (MDS) reduces dimensionality while trying to preserve\nthe distances between the instances (see Figure 8-13).\nOther Dimensionality Reduction Techniques \n| \n223", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 586, "content": "Other Dimensionality Reduction Techniques \n| \n223\nDownload from finelybook www.finelybook.com", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 587, "content": "9 The geodesic distance between two nodes in a graph is the number of nodes on the shortest path between\nthese nodes.\n\u2022 Isomap creates a graph by connecting each instance to its nearest neighbors, then\nreduces dimensionality while trying to preserve the geodesic distances9 between\nthe instances.\n\u2022 t-Distributed Stochastic Neighbor Embedding (t-SNE) reduces dimensionality\nwhile trying to keep similar instances close and dissimilar instances apart. It is\nmostly used for visualization, in particular to visualize clusters of instances in\nhigh-dimensional space (e.g., to visualize the MNIST images in 2D).\n\u2022 Linear Discriminant Analysis (LDA) is actually a classification algorithm, but dur\u2010\ning training it learns the most discriminative axes between the classes, and these\naxes can then be used to define a hyperplane onto which to project the data. The\nbenefit is that the projection will keep classes as far apart as possible, so LDA is a", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 588, "content": "benefit is that the projection will keep classes as far apart as possible, so LDA is a\ngood technique to reduce dimensionality before running another classification\nalgorithm such as an SVM classifier.\nFigure 8-13. Reducing the Swiss roll to 2D using various techniques\nExercises\n1. What are the main motivations for reducing a dataset\u2019s dimensionality? What are\nthe main drawbacks?\n2. What is the curse of dimensionality?\n3. Once a dataset\u2019s dimensionality has been reduced, is it possible to reverse the\noperation? If so, how? If not, why?\n4. Can PCA be used to reduce the dimensionality of a highly nonlinear dataset?\n5. Suppose you perform PCA on a 1,000-dimensional dataset, setting the explained\nvariance ratio to 95%. How many dimensions will the resulting dataset have?\n224 \n| \nChapter 8: Dimensionality Reduction\nDownload from finelybook www.finelybook.com", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 589, "content": "6. In what cases would you use vanilla PCA, Incremental PCA, Randomized PCA,\nor Kernel PCA?\n7. How can you evaluate the performance of a dimensionality reduction algorithm\non your dataset?\n8. Does it make any sense to chain two different dimensionality reduction algo\u2010\nrithms?\n9. Load the MNIST dataset (introduced in Chapter 3) and split it into a training set\nand a test set (take the first 60,000 instances for training, and the remaining\n10,000 for testing). Train a Random Forest classifier on the dataset and time how\nlong it takes, then evaluate the resulting model on the test set. Next, use PCA to\nreduce the dataset\u2019s dimensionality, with an explained variance ratio of 95%.\nTrain a new Random Forest classifier on the reduced dataset and see how long it\ntakes. Was training much faster? Next evaluate the classifier on the test set: how\ndoes it compare to the previous classifier?\n10. Use t-SNE to reduce the MNIST dataset down to two dimensions and plot the", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 590, "content": "10. Use t-SNE to reduce the MNIST dataset down to two dimensions and plot the\nresult using Matplotlib. You can use a scatterplot using 10 different colors to rep\u2010\nresent each image\u2019s target class. Alternatively, you can write colored digits at the\nlocation of each instance, or even plot scaled-down versions of the digit images\nthemselves (if you plot all digits, the visualization will be too cluttered, so you\nshould either draw a random sample or plot an instance only if no other instance\nhas already been plotted at a close distance). You should get a nice visualization\nwith well-separated clusters of digits. Try using other dimensionality reduction\nalgorithms such as PCA, LLE, or MDS and compare the resulting visualizations.\nSolutions to these exercises are available in Appendix A.\nExercises \n| \n225\nDownload from finelybook www.finelybook.com", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 591, "content": "Download from finelybook www.finelybook.com\n\nPART II\nNeural Networks and Deep Learning\nDownload from finelybook www.finelybook.com\n\nDownload from finelybook www.finelybook.com", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 592, "content": "CHAPTER 9\nUp and Running with TensorFlow\nTensorFlow is a powerful open source software library for numerical computation,\nparticularly well suited and fine-tuned for large-scale Machine Learning. Its basic\nprinciple is simple: you first define in Python a graph of computations to perform\n(for example, the one in Figure 9-1), and then TensorFlow takes that graph and runs\nit efficiently using optimized C++ code.\nFigure 9-1. A simple computation graph\nMost importantly, it is possible to break up the graph into several chunks and run\nthem in parallel across multiple CPUs or GPUs (as shown in Figure 9-2). TensorFlow\nalso supports distributed computing, so you can train colossal neural networks on\nhumongous training sets in a reasonable amount of time by splitting the computa\u2010\ntions across hundreds of servers (see Chapter 12). TensorFlow can train a network\nwith millions of parameters on a training set composed of billions of instances with", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 593, "content": "with millions of parameters on a training set composed of billions of instances with\nmillions of features each. This should come as no surprise, since TensorFlow was\n229\nDownload from finelybook www.finelybook.com", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 594, "content": "1 TensorFlow is not limited to neural networks or even Machine Learning; you could run quantum physics sim\u2010\nulations if you wanted.\ndeveloped by the Google Brain team and it powers many of Google\u2019s large-scale serv\u2010\nices, such as Google Cloud Speech, Google Photos, and Google Search.\nFigure 9-2. Parallel computation on multiple CPUs/GPUs/servers\nWhen TensorFlow was open-sourced in November 2015, there were already many\npopular open source libraries for Deep Learning (Table 9-1 lists a few), and to be fair\nmost of TensorFlow\u2019s features already existed in one library or another. Nevertheless,\nTensorFlow\u2019s clean design, scalability, flexibility,1 and great documentation (not to\nmention Google\u2019s name) quickly boosted it to the top of the list. In short, TensorFlow\nwas designed to be flexible, scalable, and production-ready, and existing frameworks\narguably hit only two out of the three of these. Here are some of TensorFlow\u2019s high\u2010\nlights:", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 595, "content": "arguably hit only two out of the three of these. Here are some of TensorFlow\u2019s high\u2010\nlights:\n\u2022 It runs not only on Windows, Linux, and macOS, but also on mobile devices,\nincluding both iOS and Android.\n230 \n| \nChapter 9: Up and Running with TensorFlow\nDownload from finelybook www.finelybook.com", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 596, "content": "2 Not to be confused with the TFLearn library, which is an independent project.\n\u2022 It provides a very simple Python API called TF.Learn2 (tensorflow.con\ntrib.learn), compatible with Scikit-Learn. As you will see, you can use it to\ntrain various types of neural networks in just a few lines of code. It was previ\u2010\nously an independent project called Scikit Flow (or skflow).\n\u2022 It also provides another simple API called TF-slim (tensorflow.contrib.slim)\nto simplify building, training, and evaluating neural networks.\n\u2022 Several other high-level APIs have been built independently on top of Tensor\u2010\nFlow, such as Keras or Pretty Tensor.\n\u2022 Its main Python API offers much more flexibility (at the cost of higher complex\u2010\nity) to create all sorts of computations, including any neural network architecture\nyou can think of.\n\u2022 It includes highly efficient C++ implementations of many ML operations, partic\u2010\nularly those needed to build neural networks. There is also a C++ API to define", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 597, "content": "ularly those needed to build neural networks. There is also a C++ API to define\nyour own high-performance operations.\n\u2022 It provides several advanced optimization nodes to search for the parameters that\nminimize a cost function. These are very easy to use since TensorFlow automati\u2010\ncally takes care of computing the gradients of the functions you define. This is \ncalled automatic differentiating (or autodiff).\n\u2022 It also comes with a great visualization tool called TensorBoard that allows you to\nbrowse through the computation graph, view learning curves, and more.\n\u2022 Google also launched a cloud service to run TensorFlow graphs.\n\u2022 Last but not least, it has a dedicated team of passionate and helpful developers,\nand a growing community contributing to improving it. It is one of the most\npopular open source projects on GitHub, and more and more great projects are\nbeing built on top of it (for examples, check out the resources page on https://", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 598, "content": "being built on top of it (for examples, check out the resources page on https://\nwww.tensorflow.org/, or https://github.com/jtoy/awesome-tensorflow). To ask\ntechnical questions, you should use http://stackoverflow.com/ and tag your ques\u2010\ntion with \"tensorflow\". You can file bugs and feature requests through GitHub.\nFor general discussions, join the Google group.\nIn this chapter, we will go through the basics of TensorFlow, from installation to cre\u2010\nating, running, saving, and visualizing simple computational graphs. Mastering these\nbasics is important before you build your first neural network (which we will do in\nthe next chapter).\nUp and Running with TensorFlow \n| \n231\nDownload from finelybook www.finelybook.com", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 599, "content": "Table 9-1. Open source Deep Learning libraries (not an exhaustive list)\nLibrary\nAPI\nPlatforms\nStarted by\nYear\nCaffe\nPython, C++, Matlab Linux, macOS, Windows\nY. Jia, UC Berkeley (BVLC)\n2013\nDeeplearning4j Java, Scala, Clojure\nLinux, macOS, Windows, Android\nA. Gibson, J.Patterson\n2014\nH2O\nPython, R\nLinux, macOS, Windows\nH2O.ai\n2014\nMXNet\nPython, C++, others\nLinux, macOS, Windows, iOS, Android\nDMLC\n2015\nTensorFlow\nPython, C++\nLinux, macOS, Windows, iOS, Android\nGoogle\n2015\nTheano\nPython\nLinux, macOS, iOS\nUniversity of Montreal\n2010\nTorch\nC++, Lua\nLinux, macOS, iOS, Android\nR. Collobert, K. Kavukcuoglu, C.\nFarabet\n2002\nInstallation\nLet\u2019s get started! Assuming you installed Jupyter and Scikit-Learn by following the\ninstallation instructions in Chapter 2, you can simply use pip to install TensorFlow. If\nyou created an isolated environment using virtualenv, you first need to activate it:\n$ cd $ML_PATH # Your ML working directory (e.g., $HOME/ml)\n$ source env/bin/activate", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 600, "content": "$ cd $ML_PATH # Your ML working directory (e.g., $HOME/ml)\n$ source env/bin/activate\nNext, install TensorFlow:\n$ pip3 install --upgrade tensorflow\nFor GPU support, you need to install tensorflow-gpu instead of\ntensorflow. See Chapter 12 for more details.\nTo test your installation, type the following command. It should output the version of\nTensorFlow you installed.\n$ python3 -c 'import tensorflow; print(tensorflow.__version__)'\n1.0.0\nCreating Your First Graph and Running It in a Session\nThe following code creates the graph represented in Figure 9-1:\nimport tensorflow as tf\nx = tf.Variable(3, name=\"x\")\ny = tf.Variable(4, name=\"y\")\nf = x*x*y + y + 2\n232 \n| \nChapter 9: Up and Running with TensorFlow\nDownload from finelybook www.finelybook.com", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 601, "content": "3 In distributed TensorFlow, variable values are stored on the servers instead of the session, as we will see in\nChapter 12.\nThat\u2019s all there is to it! The most important thing to understand is that this code does\nnot actually perform any computation, even though it looks like it does (especially the\nlast line). It just creates a computation graph. In fact, even the variables are not ini\u2010\ntialized yet. To evaluate this graph, you need to open a TensorFlow session and use it\nto initialize the variables and evaluate f. A TensorFlow session takes care of placing\nthe operations onto devices such as CPUs and GPUs and running them, and it holds\nall the variable values.3 The following code creates a session, initializes the variables,\nand evaluates, and f then closes the session (which frees up resources):\n>>> sess = tf.Session()\n>>> sess.run(x.initializer)\n>>> sess.run(y.initializer)\n>>> result = sess.run(f)\n>>> print(result)\n42\n>>> sess.close()", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 602, "content": ">>> sess.run(y.initializer)\n>>> result = sess.run(f)\n>>> print(result)\n42\n>>> sess.close()\nHaving to repeat sess.run() all the time is a bit cumbersome, but fortunately there is\na better way:\nwith tf.Session() as sess:\n x.initializer.run()\n y.initializer.run()\n result = f.eval()\nInside the with block, the session is set as the default session. Calling x.initial\nizer.run() is equivalent to calling tf.get_default_session().run(x.initial\nizer), \nand \nsimilarly \nf.eval() \nis \nequivalent \nto \ncalling\ntf.get_default_session().run(f). This makes the code easier to read. Moreover,\nthe session is automatically closed at the end of the block.\nInstead of manually running the initializer for every single variable, you can use the\nglobal_variables_initializer() function. Note that it does not actually perform\nthe initialization immediately, but rather creates a node in the graph that will initialize\nall variables when it is run:", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 603, "content": "all variables when it is run:\ninit = tf.global_variables_initializer() # prepare an init node\nwith tf.Session() as sess:\n init.run() # actually initialize all the variables\n result = f.eval()\nInside Jupyter or within a Python shell you may prefer to create an InteractiveSes\nsion. The only difference from a regular Session is that when an InteractiveSes\nsion is created it automatically sets itself as the default session, so you don\u2019t need a\nCreating Your First Graph and Running It in a Session \n| \n233\nDownload from finelybook www.finelybook.com", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 604, "content": "with block (but you do need to close the session manually when you are done with\nit):\n>>> sess = tf.InteractiveSession()\n>>> init.run()\n>>> result = f.eval()\n>>> print(result)\n42\n>>> sess.close()\nA TensorFlow program is typically split into two parts: the first part builds a compu\u2010\ntation graph (this is called the construction phase), and the second part runs it (this is\nthe execution phase). The construction phase typically builds a computation graph\nrepresenting the ML model and the computations required to train it. The execution\nphase generally runs a loop that evaluates a training step repeatedly (for example, one\nstep per mini-batch), gradually improving the model parameters. We will go through\nan example shortly.\nManaging Graphs\nAny node you create is automatically added to the default graph:\n>>> x1 = tf.Variable(1)\n>>> x1.graph is tf.get_default_graph()\nTrue\nIn most cases this is fine, but sometimes you may want to manage multiple independ\u2010", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 605, "content": "True\nIn most cases this is fine, but sometimes you may want to manage multiple independ\u2010\nent graphs. You can do this by creating a new Graph and temporarily making it the\ndefault graph inside a with block, like so:\n>>> graph = tf.Graph()\n>>> with graph.as_default():\n... x2 = tf.Variable(2)\n...\n>>> x2.graph is graph\nTrue\n>>> x2.graph is tf.get_default_graph()\nFalse\nIn Jupyter (or in a Python shell), it is common to run the same\ncommands more than once while you are experimenting. As a\nresult, you may end up with a default graph containing many\nduplicate nodes. One solution is to restart the Jupyter kernel (or\nthe Python shell), but a more convenient solution is to just reset the\ndefault graph by running tf.reset_default_graph().\n234 \n| \nChapter 9: Up and Running with TensorFlow\nDownload from finelybook www.finelybook.com", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 606, "content": "Lifecycle of a Node Value\nWhen you evaluate a node, TensorFlow automatically determines the set of nodes\nthat it depends on and it evaluates these nodes first. For example, consider the follow\u2010\ning code:\nw = tf.constant(3)\nx = w + 2\ny = x + 5\nz = x * 3\nwith tf.Session() as sess:\n print(y.eval()) # 10\n print(z.eval()) # 15\nFirst, this code defines a very simple graph. Then it starts a session and runs the\ngraph to evaluate y: TensorFlow automatically detects that y depends on w, which\ndepends on x, so it first evaluates w, then x, then y, and returns the value of y. Finally,\nthe code runs the graph to evaluate z. Once again, TensorFlow detects that it must\nfirst evaluate w and x. It is important to note that it will not reuse the result of the\nprevious evaluation of w and x. In short, the preceding code evaluates w and x twice.\nAll node values are dropped between graph runs, except variable values, which are", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 607, "content": "All node values are dropped between graph runs, except variable values, which are\nmaintained by the session across graph runs (queues and readers also maintain some\nstate, as we will see in Chapter 12). A variable starts its life when its initializer is run,\nand it ends when the session is closed.\nIf you want to evaluate y and z efficiently, without evaluating w and x twice as in the\nprevious code, you must ask TensorFlow to evaluate both y and z in just one graph\nrun, as shown in the following code:\nwith tf.Session() as sess:\n y_val, z_val = sess.run([y, z])\n print(y_val) # 10\n print(z_val) # 15\nIn single-process TensorFlow, multiple sessions do not share any\nstate, even if they reuse the same graph (each session would have its\nown copy of every variable). In distributed TensorFlow (see Chap\u2010\nter 12), variable state is stored on the servers, not in the sessions, so\nmultiple sessions can share the same variables.\nLinear Regression with TensorFlow", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 608, "content": "multiple sessions can share the same variables.\nLinear Regression with TensorFlow\nTensorFlow operations (also called ops for short) can take any number of inputs and\nproduce any number of outputs. For example, the addition and multiplication ops\neach take two inputs and produce one output. Constants and variables take no input\nLifecycle of a Node Value \n| \n235\nDownload from finelybook www.finelybook.com", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 609, "content": "4 Note that housing.target is a 1D array, but we need to reshape it to a column vector to compute theta.\nRecall that NumPy\u2019s reshape() function accepts \u20131 (meaning \u201cunspecified\u201d) for one of the dimensions: that\ndimension will be computed based on the array\u2019s length and the remaining dimensions.\n(they are called source ops). The inputs and outputs are multidimensional arrays,\ncalled tensors (hence the name \u201ctensor flow\u201d). Just like NumPy arrays, tensors have a\ntype and a shape. In fact, in the Python API tensors are simply represented by NumPy\nndarrays. They typically contain floats, but you can also use them to carry strings\n(arbitrary byte arrays).\nIn the examples so far, the tensors just contained a single scalar value, but you can of\ncourse perform computations on arrays of any shape. For example, the following code\nmanipulates 2D arrays to perform Linear Regression on the California housing data\u2010", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 610, "content": "manipulates 2D arrays to perform Linear Regression on the California housing data\u2010\nset (introduced in Chapter 2). It starts by fetching the dataset; then it adds an extra\nbias input feature (x0 = 1) to all training instances (it does so using NumPy so it runs\nimmediately); then it creates two TensorFlow constant nodes, X and y, to hold this\ndata and the targets,4 and it uses some of the matrix operations provided by Tensor\u2010\nFlow to define theta. These matrix functions\u2014transpose(), matmul(), and\nmatrix_inverse()\u2014are self-explanatory, but as usual they do not perform any com\u2010\nputations immediately; instead, they create nodes in the graph that will perform them\nwhen the graph is run. You may recognize that the definition of theta corresponds to\nthe Normal Equation (\u03b8 = XT \u00b7 X)\u20131 \u00b7 XT \u00b7 y; see Chapter 4). Finally, the code creates a\nsession and uses it to evaluate theta.\nimport numpy as np\nfrom sklearn.datasets import fetch_california_housing\nhousing = fetch_california_housing()", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 611, "content": "from sklearn.datasets import fetch_california_housing\nhousing = fetch_california_housing()\nm, n = housing.data.shape\nhousing_data_plus_bias = np.c_[np.ones((m, 1)), housing.data]\nX = tf.constant(housing_data_plus_bias, dtype=tf.float32, name=\"X\")\ny = tf.constant(housing.target.reshape(-1, 1), dtype=tf.float32, name=\"y\")\nXT = tf.transpose(X)\ntheta = tf.matmul(tf.matmul(tf.matrix_inverse(tf.matmul(XT, X)), XT), y)\nwith tf.Session() as sess:\n theta_value = theta.eval()\nThe main benefit of this code versus computing the Normal Equation directly using\nNumPy is that TensorFlow will automatically run this on your GPU card if you have\none (provided you installed TensorFlow with GPU support, of course; see Chapter 12\nfor more details).\n236 \n| \nChapter 9: Up and Running with TensorFlow\nDownload from finelybook www.finelybook.com", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 612, "content": "Implementing Gradient Descent\nLet\u2019s try using Batch Gradient Descent (introduced in Chapter 4) instead of the Nor\u2010\nmal Equation. First we will do this by manually computing the gradients, then we will\nuse TensorFlow\u2019s autodiff feature to let TensorFlow compute the gradients automati\u2010\ncally, and finally we will use a couple of TensorFlow\u2019s out-of-the-box optimizers.\nWhen using Gradient Descent, remember that it is important to\nfirst normalize the input feature vectors, or else training may be\nmuch slower. You can do this using TensorFlow, NumPy, Scikit-\nLearn\u2019s StandardScaler, or any other solution you prefer. The fol\u2010\nlowing code assumes that this normalization has already been\ndone.\nManually Computing the Gradients\nThe following code should be fairly self-explanatory, except for a few new elements:\n\u2022 The random_uniform() function creates a node in the graph that will generate a\ntensor containing random values, given its shape and value range, much like\nNumPy\u2019s rand() function.", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 613, "content": "NumPy\u2019s rand() function.\n\u2022 The assign() function creates a node that will assign a new value to a variable.\nIn this case, it implements the Batch Gradient Descent step \u03b8(next step) = \u03b8 \u2013\n\u03b7\u2207\u03b8MSE(\u03b8).\n\u2022 The main loop executes the training step over and over again (n_epochs times),\nand every 100 iterations it prints out the current Mean Squared Error (mse). You\nshould see the MSE go down at every iteration.\nn_epochs = 1000\nlearning_rate = 0.01\nX = tf.constant(scaled_housing_data_plus_bias, dtype=tf.float32, name=\"X\")\ny = tf.constant(housing.target.reshape(-1, 1), dtype=tf.float32, name=\"y\")\ntheta = tf.Variable(tf.random_uniform([n + 1, 1], -1.0, 1.0), name=\"theta\")\ny_pred = tf.matmul(X, theta, name=\"predictions\")\nerror = y_pred - y\nmse = tf.reduce_mean(tf.square(error), name=\"mse\")\ngradients = 2/m * tf.matmul(tf.transpose(X), error)\ntraining_op = tf.assign(theta, theta - learning_rate * gradients)\ninit = tf.global_variables_initializer()\nwith tf.Session() as sess:\n sess.run(init)", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 614, "content": "init = tf.global_variables_initializer()\nwith tf.Session() as sess:\n sess.run(init)\n for epoch in range(n_epochs):\nImplementing Gradient Descent \n| \n237\nDownload from finelybook www.finelybook.com", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 615, "content": "if epoch % 100 == 0:\n print(\"Epoch\", epoch, \"MSE =\", mse.eval())\n sess.run(training_op)\n best_theta = theta.eval()\nUsing autodiff\nThe preceding code works fine, but it requires mathematically deriving the gradients\nfrom the cost function (MSE). In the case of Linear Regression, it is reasonably easy,\nbut if you had to do this with deep neural networks you would get quite a headache:\nit would be tedious and error-prone. You could use symbolic differentiation to auto\u2010\nmatically find the equations for the partial derivatives for you, but the resulting code\nwould not necessarily be very efficient.\nTo understand why, consider the function f(x)= exp(exp(exp(x))). If you know calcu\u2010\nlus, you can figure out its derivative f\u2032(x) = exp(x) \u00d7 exp(exp(x)) \u00d7 exp(exp(exp(x))).\nIf you code f(x) and f\u2032(x) separately and exactly as they appear, your code will not be\nas efficient as it could be. A more efficient solution would be to write a function that", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 616, "content": "as efficient as it could be. A more efficient solution would be to write a function that\nfirst computes exp(x), then exp(exp(x)), then exp(exp(exp(x))), and returns all three.\nThis gives you f(x) directly (the third term), and if you need the derivative you can\njust multiply all three terms and you are done. With the na\u00efve approach you would\nhave had to call the exp function nine times to compute both f(x) and f\u2032(x). With this\napproach you just need to call it three times.\nIt gets worse when your function is defined by some arbitrary code. Can you find the\nequation (or the code) to compute the partial derivatives of the following function?\nHint: don\u2019t even try.\ndef my_func(a, b):\n z = 0\n for i in range(100):\n z = a * np.cos(z + i) + z * np.sin(b - i)\n return z\nFortunately, TensorFlow\u2019s autodiff feature comes to the rescue: it can automatically\nand efficiently compute the gradients for you. Simply replace the gradients = ...", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 617, "content": "and efficiently compute the gradients for you. Simply replace the gradients = ...\nline in the Gradient Descent code in the previous section with the following line, and\nthe code will continue to work just fine:\ngradients = tf.gradients(mse, [theta])[0]\nThe gradients() function takes an op (in this case mse) and a list of variables (in this\ncase just theta), and it creates a list of ops (one per variable) to compute the gradi\u2010\nents of the op with regards to each variable. So the gradients node will compute the\ngradient vector of the MSE with regards to theta.\n238 \n| \nChapter 9: Up and Running with TensorFlow\nDownload from finelybook www.finelybook.com", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 618, "content": "There are four main approaches to computing gradients automatically. They are sum\u2010\nmarized in Table 9-2. TensorFlow uses reverse-mode autodiff, which is perfect (effi\u2010\ncient and accurate) when there are many inputs and few outputs, as is often the case\nin neural networks. It computes all the partial derivatives of the outputs with regards\nto all the inputs in just noutputs + 1 graph traversals.\nTable 9-2. Main solutions to compute gradients automatically\nTechnique\nNb of graph traversals to\ncompute all gradients\nAccuracy\nSupports\narbitrary code\nComment\nNumerical differentiation ninputs + 1\nLow\nYes\nTrivial to implement\nSymbolic differentiation\nN/A\nHigh\nNo\nBuilds a very different graph\nForward-mode autodiff\nninputs\nHigh\nYes\nUses dual numbers\nReverse-mode autodiff\nnoutputs + 1\nHigh\nYes\nImplemented by TensorFlow\nIf you are interested in how this magic works, check out Appendix D.\nUsing an Optimizer\nSo TensorFlow computes the gradients for you. But it gets even easier: it also provides", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 619, "content": "So TensorFlow computes the gradients for you. But it gets even easier: it also provides\na number of optimizers out of the box, including a Gradient Descent optimizer. You\ncan simply replace the preceding gradients = ... and training_op = ... lines\nwith the following code, and once again everything will just work fine:\noptimizer = tf.train.GradientDescentOptimizer(learning_rate=learning_rate)\ntraining_op = optimizer.minimize(mse)\nIf you want to use a different type of optimizer, you just need to change one line. For\nexample, you can use a momentum optimizer (which often converges much faster\nthan Gradient Descent; see Chapter 11) by defining the optimizer like this:\noptimizer = tf.train.MomentumOptimizer(learning_rate=learning_rate,\n momentum=0.9)\nFeeding Data to the Training Algorithm\nLet\u2019s try to modify the previous code to implement Mini-batch Gradient Descent. For", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 620, "content": "Let\u2019s try to modify the previous code to implement Mini-batch Gradient Descent. For\nthis, we need a way to replace X and y at every iteration with the next mini-batch. The\nsimplest way to do this is to use placeholder nodes. These nodes are special because\nthey don\u2019t actually perform any computation, they just output the data you tell them\nto output at runtime. They are typically used to pass the training data to TensorFlow\nduring training. If you don\u2019t specify a value at runtime for a placeholder, you get an\nexception.\nTo create a placeholder node, you must call the placeholder() function and specify\nthe output tensor\u2019s data type. Optionally, you can also specify its shape, if you want to\nFeeding Data to the Training Algorithm \n| \n239\nDownload from finelybook www.finelybook.com", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 621, "content": "enforce it. If you specify None for a dimension, it means \u201cany size.\u201d For example, the\nfollowing code creates a placeholder node A, and also a node B = A + 5. When we\nevaluate B, we pass a feed_dict to the eval() method that specifies the value of A.\nNote that A must have rank 2 (i.e., it must be two-dimensional) and there must be\nthree columns (or else an exception is raised), but it can have any number of rows.\n>>> A = tf.placeholder(tf.float32, shape=(None, 3))\n>>> B = A + 5\n>>> with tf.Session() as sess:\n... B_val_1 = B.eval(feed_dict={A: [[1, 2, 3]]})\n... B_val_2 = B.eval(feed_dict={A: [[4, 5, 6], [7, 8, 9]]})\n...\n>>> print(B_val_1)\n[[ 6. 7. 8.]]\n>>> print(B_val_2)\n[[ 9. 10. 11.]\n [ 12. 13. 14.]]\nYou can actually feed the output of any operations, not just place\u2010\nholders. In this case TensorFlow does not try to evaluate these\noperations; it uses the values you feed it.\nTo implement Mini-batch Gradient Descent, we only need to tweak the existing code", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 622, "content": "To implement Mini-batch Gradient Descent, we only need to tweak the existing code\nslightly. First change the definition of X and y in the construction phase to make them \nplaceholder nodes:\nX = tf.placeholder(tf.float32, shape=(None, n + 1), name=\"X\")\ny = tf.placeholder(tf.float32, shape=(None, 1), name=\"y\")\nThen define the batch size and compute the total number of batches:\nbatch_size = 100\nn_batches = int(np.ceil(m / batch_size))\nFinally, in the execution phase, fetch the mini-batches one by one, then provide the\nvalue of X and y via the feed_dict parameter when evaluating a node that depends\non either of them.\ndef fetch_batch(epoch, batch_index, batch_size):\n [...] # load the data from disk\n return X_batch, y_batch\nwith tf.Session() as sess:\n sess.run(init)\n for epoch in range(n_epochs):\n for batch_index in range(n_batches):\n X_batch, y_batch = fetch_batch(epoch, batch_index, batch_size)", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 623, "content": "X_batch, y_batch = fetch_batch(epoch, batch_index, batch_size)\n sess.run(training_op, feed_dict={X: X_batch, y: y_batch})\n240 \n| \nChapter 9: Up and Running with TensorFlow\nDownload from finelybook www.finelybook.com", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 624, "content": "best_theta = theta.eval()\nWe don\u2019t need to pass the value of X and y when evaluating theta\nsince it does not depend on either of them.\nSaving and Restoring Models\nOnce you have trained your model, you should save its parameters to disk so you can\ncome back to it whenever you want, use it in another program, compare it to other\nmodels, and so on. Moreover, you probably want to save checkpoints at regular inter\u2010\nvals during training so that if your computer crashes during training you can con\u2010\ntinue from the last checkpoint rather than start over from scratch.\nTensorFlow makes saving and restoring a model very easy. Just create a Saver node at\nthe end of the construction phase (after all variable nodes are created); then, in the\nexecution phase, just call its save() method whenever you want to save the model,\npassing it the session and path of the checkpoint file:\n[...]\ntheta = tf.Variable(tf.random_uniform([n + 1, 1], -1.0, 1.0), name=\"theta\")\n[...]", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 625, "content": "[...]\ntheta = tf.Variable(tf.random_uniform([n + 1, 1], -1.0, 1.0), name=\"theta\")\n[...]\ninit = tf.global_variables_initializer()\nsaver = tf.train.Saver()\nwith tf.Session() as sess:\n sess.run(init)\n for epoch in range(n_epochs):\n if epoch % 100 == 0: # checkpoint every 100 epochs\n save_path = saver.save(sess, \"/tmp/my_model.ckpt\")\n sess.run(training_op)\n best_theta = theta.eval()\n save_path = saver.save(sess, \"/tmp/my_model_final.ckpt\")\nRestoring a model is just as easy: you create a Saver at the end of the construction\nphase just like before, but then at the beginning of the execution phase, instead of ini\u2010\ntializing the variables using the init node, you call the restore() method of the\nSaver object:\nwith tf.Session() as sess:\n saver.restore(sess, \"/tmp/my_model_final.ckpt\")\n [...]\nSaving and Restoring Models \n| \n241\nDownload from finelybook www.finelybook.com", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 626, "content": "By default a Saver saves and restores all variables under their own name, but if you\nneed more control, you can specify which variables to save or restore, and what\nnames to use. For example, the following Saver will save or restore only the theta\nvariable under the name weights:\nsaver = tf.train.Saver({\"weights\": theta})\nVisualizing the Graph and Training Curves Using\nTensorBoard\nSo now we have a computation graph that trains a Linear Regression model using\nMini-batch Gradient Descent, and we are saving checkpoints at regular intervals.\nSounds sophisticated, doesn\u2019t it? However, we are still relying on the print() func\u2010\ntion to visualize progress during training. There is a better way: enter TensorBoard. If\nyou feed it some training stats, it will display nice interactive visualizations of these\nstats in your web browser (e.g., learning curves). You can also provide it the graph\u2019s\ndefinition and it will give you a great interface to browse through it. This is very use\u2010", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 627, "content": "definition and it will give you a great interface to browse through it. This is very use\u2010\nful to identify errors in the graph, to find bottlenecks, and so on.\nThe first step is to tweak your program a bit so it writes the graph definition and\nsome training stats\u2014for example, the training error (MSE)\u2014to a log directory that\nTensorBoard will read from. You need to use a different log directory every time you\nrun your program, or else TensorBoard will merge stats from different runs, which\nwill mess up the visualizations. The simplest solution for this is to include a time\u2010\nstamp in the log directory name. Add the following code at the beginning of the pro\u2010\ngram:\nfrom datetime import datetime\nnow = datetime.utcnow().strftime(\"%Y%m%d%H%M%S\")\nroot_logdir = \"tf_logs\"\nlogdir = \"{}/run-{}/\".format(root_logdir, now)\nNext, add the following code at the very end of the construction phase:\nmse_summary = tf.summary.scalar('MSE', mse)", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 628, "content": "mse_summary = tf.summary.scalar('MSE', mse)\nfile_writer = tf.summary.FileWriter(logdir, tf.get_default_graph())\nThe first line creates a node in the graph that will evaluate the MSE value and write it\nto a TensorBoard-compatible binary log string called a summary. The second line cre\u2010\nates a FileWriter that you will use to write summaries to logfiles in the log directory.\nThe first parameter indicates the path of the log directory (in this case something like\ntf_logs/run-20160906091959/, relative to the current directory). The second\n(optional) parameter is the graph you want to visualize. Upon creation, the File\nWriter creates the log directory if it does not already exist (and its parent directories\nif needed), and writes the graph definition in a binary logfile called an events file.\n242 \n| \nChapter 9: Up and Running with TensorFlow\nDownload from finelybook www.finelybook.com", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 629, "content": "Next you need to update the execution phase to evaluate the mse_summary node regu\u2010\nlarly during training (e.g., every 10 mini-batches). This will output a summary that\nyou can then write to the events file using the file_writer. Here is the updated code:\n [...]\n for batch_index in range(n_batches):\n X_batch, y_batch = fetch_batch(epoch, batch_index, batch_size)\n if batch_index % 10 == 0:\n summary_str = mse_summary.eval(feed_dict={X: X_batch, y: y_batch})\n step = epoch * n_batches + batch_index\n file_writer.add_summary(summary_str, step)\n sess.run(training_op, feed_dict={X: X_batch, y: y_batch})\n [...]\nAvoid logging training stats at every single training step, as this\nwould significantly slow down training.\nFinally, you want to close the FileWriter at the end of the program:\nfile_writer.close()\nNow run this program: it will create the log directory and write an events file in this", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 630, "content": "Now run this program: it will create the log directory and write an events file in this\ndirectory, containing both the graph definition and the MSE values. Open up a shell\nand go to your working directory, then type ls -l tf_logs/run* to list the contents\nof the log directory:\n$ cd $ML_PATH # Your ML working directory (e.g., $HOME/ml)\n$ ls -l tf_logs/run*\ntotal 40\n-rw-r--r-- 1 ageron staff 18620 Sep 6 11:10 events.out.tfevents.1472553182.mymac\nIf you run the program a second time, you should see a second directory in the\ntf_logs/ directory:\n$ ls -l tf_logs/\ntotal 0\ndrwxr-xr-x 3 ageron staff 102 Sep 6 10:07 run-20160906091959\ndrwxr-xr-x 3 ageron staff 102 Sep 6 10:22 run-20160906092202\nGreat! Now it\u2019s time to fire up the TensorBoard server. You need to activate your vir\u2010\ntualenv environment if you created one, then start the server by running the tensor\nboard command, pointing it to the root log directory. This starts the TensorBoard", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 631, "content": "board command, pointing it to the root log directory. This starts the TensorBoard\nweb server, listening on port 6006 (which is \u201cgoog\u201d written upside down):\n$ source env/bin/activate\n$ tensorboard --logdir tf_logs/\nStarting TensorBoard on port 6006\n(You can navigate to http://0.0.0.0:6006)\nVisualizing the Graph and Training Curves Using TensorBoard \n| \n243\nDownload from finelybook www.finelybook.com", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 632, "content": "Next open a browser and go to http://0.0.0.0:6006/ (or http://localhost:6006/). Wel\u2010\ncome to TensorBoard! In the Events tab you should see MSE on the right. If you click\non it, you will see a plot of the MSE during training, for both runs (Figure 9-3). You\ncan check or uncheck the runs you want to see, zoom in or out, hover over the curve\nto get details, and so on.\nFigure 9-3. Visualizing training stats using TensorBoard\nNow click on the Graphs tab. You should see the graph shown in Figure 9-4.\nTo reduce clutter, the nodes that have many edges (i.e., connections to other nodes)\nare separated out to an auxiliary area on the right (you can move a node back and\nforth between the main graph and the auxiliary area by right-clicking on it). Some\nparts of the graph are also collapsed by default. For example, try hovering over the\ngradients node, then click on the \u2295 icon to expand this subgraph. Next, in this sub\u2010\ngraph, try expanding the mse_grad subgraph.\n244 \n|", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 633, "content": "graph, try expanding the mse_grad subgraph.\n244 \n| \nChapter 9: Up and Running with TensorFlow\nDownload from finelybook www.finelybook.com", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 634, "content": "Figure 9-4. Visualizing the graph using TensorBoard\nIf you want to take a peek at the graph directly within Jupyter, you\ncan use the show_graph() function available in the notebook for\nthis chapter. It was originally written by A. Mordvintsev in his great\ndeepdream tutorial notebook. Another option is to install E. Jang\u2019s\nTensorFlow debugger tool which includes a Jupyter extension for\ngraph visualization (and more).\nName Scopes\nWhen dealing with more complex models such as neural networks, the graph can\neasily become cluttered with thousands of nodes. To avoid this, you can create name\nscopes to group related nodes. For example, let\u2019s modify the previous code to define\nthe error and mse ops within a name scope called \"loss\":\nwith tf.name_scope(\"loss\") as scope:\n error = y_pred - y\n mse = tf.reduce_mean(tf.square(error), name=\"mse\")\nThe name of each op defined within the scope is now prefixed with \"loss/\":\n>>> print(error.op.name)\nloss/sub\n>>> print(mse.op.name)\nloss/mse", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 635, "content": ">>> print(error.op.name)\nloss/sub\n>>> print(mse.op.name)\nloss/mse\nIn TensorBoard, the mse and error nodes now appear inside the loss namespace,\nwhich appears collapsed by default (Figure 9-5).\nName Scopes \n| \n245\nDownload from finelybook www.finelybook.com", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 636, "content": "Figure 9-5. A collapsed namescope in TensorBoard\nModularity\nSuppose you want to create a graph that adds the output of two rectified linear units\n(ReLU). A ReLU computes a linear function of the inputs, and outputs the result if it\nis positive, and 0 otherwise, as shown in Equation 9-1.\nEquation 9-1. Rectified linear unit\nh\ufffd, b \ufffd= max \ufffd\u00b7 \ufffd+ b, 0\nThe following code does the job, but it\u2019s quite repetitive:\nn_features = 3\nX = tf.placeholder(tf.float32, shape=(None, n_features), name=\"X\")\nw1 = tf.Variable(tf.random_normal((n_features, 1)), name=\"weights1\")\nw2 = tf.Variable(tf.random_normal((n_features, 1)), name=\"weights2\")\nb1 = tf.Variable(0.0, name=\"bias1\")\nb2 = tf.Variable(0.0, name=\"bias2\")\nz1 = tf.add(tf.matmul(X, w1), b1, name=\"z1\")\nz2 = tf.add(tf.matmul(X, w2), b2, name=\"z2\")\nrelu1 = tf.maximum(z1, 0., name=\"relu1\")\nrelu2 = tf.maximum(z1, 0., name=\"relu2\")\noutput = tf.add(relu1, relu2, name=\"output\")", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 637, "content": "relu2 = tf.maximum(z1, 0., name=\"relu2\")\noutput = tf.add(relu1, relu2, name=\"output\")\nSuch repetitive code is hard to maintain and error-prone (in fact, this code contains a\ncut-and-paste error; did you spot it?). It would become even worse if you wanted to\n246 \n| \nChapter 9: Up and Running with TensorFlow\nDownload from finelybook www.finelybook.com", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 638, "content": "add a few more ReLUs. Fortunately, TensorFlow lets you stay DRY (Don\u2019t Repeat\nYourself): simply create a function to build a ReLU. The following code creates five\nReLUs and outputs their sum (note that add_n() creates an operation that will com\u2010\npute the sum of a list of tensors):\ndef relu(X):\n w_shape = (int(X.get_shape()[1]), 1)\n w = tf.Variable(tf.random_normal(w_shape), name=\"weights\")\n b = tf.Variable(0.0, name=\"bias\")\n z = tf.add(tf.matmul(X, w), b, name=\"z\")\n return tf.maximum(z, 0., name=\"relu\")\nn_features = 3\nX = tf.placeholder(tf.float32, shape=(None, n_features), name=\"X\")\nrelus = [relu(X) for i in range(5)]\noutput = tf.add_n(relus, name=\"output\")\nNote that when you create a node, TensorFlow checks whether its name already\nexists, and if it does it appends an underscore followed by an index to make the name\nunique. So the first ReLU contains nodes named \"weights\", \"bias\", \"z\", and \"relu\"", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 639, "content": "unique. So the first ReLU contains nodes named \"weights\", \"bias\", \"z\", and \"relu\"\n(plus many more nodes with their default name, such as \"MatMul\"); the second ReLU\ncontains nodes named \"weights_1\", \"bias_1\", and so on; the third ReLU contains\nnodes named \"weights_2\", \"bias_2\", and so on. TensorBoard identifies such series\nand collapses them together to reduce clutter (as you can see in Figure 9-6).\nFigure 9-6. Collapsed node series\nModularity \n| \n247\nDownload from finelybook www.finelybook.com", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 640, "content": "Using name scopes, you can make the graph much clearer. Simply move all the con\u2010\ntent of the relu() function inside a name scope. Figure 9-7 shows the resulting\ngraph. Notice that TensorFlow also gives the name scopes unique names by append\u2010\ning _1, _2, and so on.\ndef relu(X):\n with tf.name_scope(\"relu\"):\n [...]\nFigure 9-7. A clearer graph using name-scoped units\nSharing Variables\nIf you want to share a variable between various components of your graph, one sim\u2010\nple option is to create it first, then pass it as a parameter to the functions that need it.\nFor example, suppose you want to control the ReLU threshold (currently hardcoded\nto 0) using a shared threshold variable for all ReLUs. You could just create that vari\u2010\nable first, and then pass it to the relu() function:\ndef relu(X, threshold):\n with tf.name_scope(\"relu\"):\n [...]\n return tf.maximum(z, threshold, name=\"max\")\nthreshold = tf.Variable(0.0, name=\"threshold\")", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 641, "content": "return tf.maximum(z, threshold, name=\"max\")\nthreshold = tf.Variable(0.0, name=\"threshold\")\nX = tf.placeholder(tf.float32, shape=(None, n_features), name=\"X\")\nrelus = [relu(X, threshold) for i in range(5)]\noutput = tf.add_n(relus, name=\"output\")\nThis works fine: now you can control the threshold for all ReLUs using the threshold\nvariable. However, if there are many shared parameters such as this one, it will be\npainful to have to pass them around as parameters all the time. Many people create a\nPython dictionary containing all the variables in their model, and pass it around to\nevery function. Others create a class for each module (e.g., a ReLU class using class\nvariables to handle the shared parameter). Yet another option is to set the shared vari\u2010\nable as an attribute of the relu() function upon the first call, like so:\n248 \n| \nChapter 9: Up and Running with TensorFlow\nDownload from finelybook www.finelybook.com", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 642, "content": "5 Creating a ReLU class is arguably the cleanest option, but it is rather heavyweight.\ndef relu(X):\n with tf.name_scope(\"relu\"):\n if not hasattr(relu, \"threshold\"):\n relu.threshold = tf.Variable(0.0, name=\"threshold\")\n [...]\n return tf.maximum(z, relu.threshold, name=\"max\")\nTensorFlow offers another option, which may lead to slightly cleaner and more mod\u2010\nular code than the previous solutions.5 This solution is a bit tricky to understand at\nfirst, but since it is used a lot in TensorFlow it is worth going into a bit of detail. The\nidea is to use the get_variable() function to create the shared variable if it does not\nexist yet, or reuse it if it already exists. The desired behavior (creating or reusing) is\ncontrolled by an attribute of the current variable_scope(). For example, the follow\u2010\ning code will create a variable named \"relu/threshold\" (as a scalar, since shape=(),\nand using 0.0 as the initial value):\nwith tf.variable_scope(\"relu\"):", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 643, "content": "and using 0.0 as the initial value):\nwith tf.variable_scope(\"relu\"):\n threshold = tf.get_variable(\"threshold\", shape=(),\n initializer=tf.constant_initializer(0.0))\nNote that if the variable has already been created by an earlier call to get_vari\nable(), this code will raise an exception. This behavior prevents reusing variables by\nmistake. If you want to reuse a variable, you need to explicitly say so by setting the\nvariable scope\u2019s reuse attribute to True (in which case you don\u2019t have to specify the\nshape or the initializer):\nwith tf.variable_scope(\"relu\", reuse=True):\n threshold = tf.get_variable(\"threshold\")\nThis code will fetch the existing \"relu/threshold\" variable, or raise an exception if it\ndoes not exist or if it was not created using get_variable(). Alternatively, you can\nset the reuse attribute to True inside the block by calling the scope\u2019s reuse_vari\nables() method:\nwith tf.variable_scope(\"relu\") as scope:\n scope.reuse_variables()", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 644, "content": "ables() method:\nwith tf.variable_scope(\"relu\") as scope:\n scope.reuse_variables()\n threshold = tf.get_variable(\"threshold\")\nOnce reuse is set to True, it cannot be set back to False within the\nblock. Moreover, if you define other variable scopes inside this one,\nthey will automatically inherit reuse=True. Lastly, only variables\ncreated by get_variable() can be reused this way.\nSharing Variables \n| \n249\nDownload from finelybook www.finelybook.com", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 645, "content": "Now you have all the pieces you need to make the relu() function access the thres\nhold variable without having to pass it as a parameter:\ndef relu(X):\n with tf.variable_scope(\"relu\", reuse=True):\n threshold = tf.get_variable(\"threshold\") # reuse existing variable\n [...]\n return tf.maximum(z, threshold, name=\"max\")\nX = tf.placeholder(tf.float32, shape=(None, n_features), name=\"X\")\nwith tf.variable_scope(\"relu\"): # create the variable\n threshold = tf.get_variable(\"threshold\", shape=(),\n initializer=tf.constant_initializer(0.0))\nrelus = [relu(X) for relu_index in range(5)]\noutput = tf.add_n(relus, name=\"output\")\nThis code first defines the relu() function, then creates the relu/threshold variable\n(as a scalar that will later be initialized to 0.0) and builds five ReLUs by calling the\nrelu() function. The relu() function reuses the relu/threshold variable, and cre\u2010\nates the other ReLU nodes.", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 646, "content": "ates the other ReLU nodes.\nVariables created using get_variable() are always named using\nthe name of their variable_scope as a prefix (e.g., \"relu/thres\nhold\"), but for all other nodes (including variables created with\ntf.Variable()) the variable scope acts like a new name scope. In\nparticular, if a name scope with an identical name was already cre\u2010\nated, then a suffix is added to make the name unique. For example,\nall nodes created in the preceding code (except the threshold vari\u2010\nable) have a name prefixed with \"relu_1/\" to \"relu_5/\", as shown\nin Figure 9-8.\nFigure 9-8. Five ReLUs sharing the threshold variable\nIt is somewhat unfortunate that the threshold variable must be defined outside the\nrelu() function, where all the rest of the ReLU code resides. To fix this, the following\ncode creates the threshold variable within the relu() function upon the first call,\nthen reuses it in subsequent calls. Now the relu() function does not have to worry", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 647, "content": "then reuses it in subsequent calls. Now the relu() function does not have to worry\nabout name scopes or variable sharing: it just calls get_variable(), which will create\n250 \n| \nChapter 9: Up and Running with TensorFlow\nDownload from finelybook www.finelybook.com", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 648, "content": "or reuse the threshold variable (it does not need to know which is the case). The rest\nof the code calls relu() five times, making sure to set reuse=False on the first call,\nand reuse=True for the other calls.\ndef relu(X):\n threshold = tf.get_variable(\"threshold\", shape=(),\n initializer=tf.constant_initializer(0.0))\n [...]\n return tf.maximum(z, threshold, name=\"max\")\nX = tf.placeholder(tf.float32, shape=(None, n_features), name=\"X\")\nrelus = []\nfor relu_index in range(5):\n with tf.variable_scope(\"relu\", reuse=(relu_index >= 1)) as scope:\n relus.append(relu(X))\noutput = tf.add_n(relus, name=\"output\")\nThe resulting graph is slightly different than before, since the shared variable lives\nwithin the first ReLU (see Figure 9-9).\nFigure 9-9. Five ReLUs sharing the threshold variable\nThis concludes this introduction to TensorFlow. We will discuss more advanced top\u2010", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 649, "content": "This concludes this introduction to TensorFlow. We will discuss more advanced top\u2010\nics as we go through the following chapters, in particular many operations related to\ndeep neural networks, convolutional neural networks, and recurrent neural networks\nas well as how to scale up with TensorFlow using multithreading, queues, multiple\nGPUs, and multiple servers.\nExercises\n1. What are the main benefits of creating a computation graph rather than directly\nexecuting the computations? What are the main drawbacks?\n2. Is the statement a_val = a.eval(session=sess) equivalent to a_val =\nsess.run(a)?\n3. Is the statement a_val, b_val = a.eval(session=sess), b.eval(ses\nsion=sess) equivalent to a_val, b_val = sess.run([a, b])?\n4. Can you run two graphs in the same session?\nExercises \n| \n251\nDownload from finelybook www.finelybook.com", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 650, "content": "5. If you create a graph g containing a variable w, then start two threads and open a\nsession in each thread, both using the same graph g, will each session have its\nown copy of the variable w or will it be shared?\n6. When is a variable initialized? When is it destroyed?\n7. What is the difference between a placeholder and a variable?\n8. What happens when you run the graph to evaluate an operation that depends on\na placeholder but you don\u2019t feed its value? What happens if the operation does\nnot depend on the placeholder?\n9. When you run a graph, can you feed the output value of any operation, or just\nthe value of placeholders?\n10. How can you set a variable to any value you want (during the execution phase)?\n11. How many times does reverse-mode autodiff need to traverse the graph in order\nto compute the gradients of the cost function with regards to 10 variables? What\nabout forward-mode autodiff? And symbolic differentiation?", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 651, "content": "about forward-mode autodiff? And symbolic differentiation?\n12. Implement Logistic Regression with Mini-batch Gradient Descent using Tensor\u2010\nFlow. Train it and evaluate it on the moons dataset (introduced in Chapter 5). Try\nadding all the bells and whistles:\n\u2022 Define the graph within a logistic_regression() function that can be reused\neasily.\n\u2022 Save checkpoints using a Saver at regular intervals during training, and save\nthe final model at the end of training.\n\u2022 Restore the last checkpoint upon startup if training was interrupted.\n\u2022 Define the graph using nice scopes so the graph looks good in TensorBoard.\n\u2022 Add summaries to visualize the learning curves in TensorBoard.\n\u2022 Try tweaking some hyperparameters such as the learning rate or the mini-\nbatch size and look at the shape of the learning curve.\nSolutions to these exercises are available in Appendix A.\n252 \n| \nChapter 9: Up and Running with TensorFlow\nDownload from finelybook www.finelybook.com", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 652, "content": "1 You can get the best of both worlds by being open to biological inspirations without being afraid to create\nbiologically unrealistic models, as long as they work well.\nCHAPTER 10\nIntroduction to Artificial Neural Networks\nBirds inspired us to fly, burdock plants inspired velcro, and nature has inspired many\nother inventions. It seems only logical, then, to look at the brain\u2019s architecture for\ninspiration on how to build an intelligent machine. This is the key idea that inspired\nartificial neural networks (ANNs). However, although planes were inspired by birds,\nthey don\u2019t have to flap their wings. Similarly, ANNs have gradually become quite dif\u2010\nferent from their biological cousins. Some researchers even argue that we should drop\nthe biological analogy altogether (e.g., by saying \u201cunits\u201d rather than \u201cneurons\u201d), lest\nwe restrict our creativity to biologically plausible systems.1\nANNs are at the very core of Deep Learning. They are versatile, powerful, and scala\u2010", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 653, "content": "ANNs are at the very core of Deep Learning. They are versatile, powerful, and scala\u2010\nble, making them ideal to tackle large and highly complex Machine Learning tasks,\nsuch as classifying billions of images (e.g., Google Images), powering speech recogni\u2010\ntion services (e.g., Apple\u2019s Siri), recommending the best videos to watch to hundreds\nof millions of users every day (e.g., YouTube), or learning to beat the world champion\nat the game of Go by examining millions of past games and then playing against itself\n(DeepMind\u2019s AlphaGo).\nIn this chapter, we will introduce artificial neural networks, starting with a quick tour\nof the very first ANN architectures. Then we will present Multi-Layer Perceptrons\n(MLPs) and implement one using TensorFlow to tackle the MNIST digit classification\nproblem (introduced in Chapter 3).\n253\nDownload from finelybook www.finelybook.com", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 654, "content": "2 \u201cA Logical Calculus of Ideas Immanent in Nervous Activity,\u201d W. McCulloch and W. Pitts (1943).\nFrom Biological to Artificial Neurons\nSurprisingly, ANNs have been around for quite a while: they were first introduced\nback in 1943 by the neurophysiologist Warren McCulloch and the mathematician\nWalter Pitts. In their landmark paper,2 \u201cA Logical Calculus of Ideas Immanent in\nNervous Activity,\u201d McCulloch and Pitts presented a simplified computational model\nof how biological neurons might work together in animal brains to perform complex\ncomputations using propositional logic. This was the first artificial neural network\narchitecture. Since then many other architectures have been invented, as we will see.\nThe early successes of ANNs until the 1960s led to the widespread belief that we\nwould soon be conversing with truly intelligent machines. When it became clear that\nthis promise would go unfulfilled (at least for quite a while), funding flew elsewhere", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 655, "content": "this promise would go unfulfilled (at least for quite a while), funding flew elsewhere\nand ANNs entered a long dark era. In the early 1980s there was a revival of interest in\nANNs as new network architectures were invented and better training techniques\nwere developed. But by the 1990s, powerful alternative Machine Learning techniques\nsuch as Support Vector Machines (see Chapter 5) were favored by most researchers,\nas they seemed to offer better results and stronger theoretical foundations. Finally, we\nare now witnessing yet another wave of interest in ANNs. Will this wave die out like\nthe previous ones did? There are a few good reasons to believe that this one is differ\u2010\nent and will have a much more profound impact on our lives:\n\u2022 There is now a huge quantity of data available to train neural networks, and\nANNs frequently outperform other ML techniques on very large and complex\nproblems.\n\u2022 The tremendous increase in computing power since the 1990s now makes it pos\u2010", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 656, "content": "problems.\n\u2022 The tremendous increase in computing power since the 1990s now makes it pos\u2010\nsible to train large neural networks in a reasonable amount of time. This is in\npart due to Moore\u2019s Law, but also thanks to the gaming industry, which has pro\u2010\nduced powerful GPU cards by the millions.\n\u2022 The training algorithms have been improved. To be fair they are only slightly dif\u2010\nferent from the ones used in the 1990s, but these relatively small tweaks have a\nhuge positive impact.\n\u2022 Some theoretical limitations of ANNs have turned out to be benign in practice.\nFor example, many people thought that ANN training algorithms were doomed\nbecause they were likely to get stuck in local optima, but it turns out that this is\nrather rare in practice (or when it is the case, they are usually fairly close to the\nglobal optimum).\n\u2022 ANNs seem to have entered a virtuous circle of funding and progress. Amazing\nproducts based on ANNs regularly make the headline news, which pulls more\n254 \n|", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 657, "content": "products based on ANNs regularly make the headline news, which pulls more\n254 \n| \nChapter 10: Introduction to Artificial Neural Networks\nDownload from finelybook www.finelybook.com", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 658, "content": "3 Image by Bruce Blaus (Creative Commons 3.0). Reproduced from https://en.wikipedia.org/wiki/Neuron.\nand more attention and funding toward them, resulting in more and more pro\u2010\ngress, and even more amazing products.\nBiological Neurons\nBefore we discuss artificial neurons, let\u2019s take a quick look at a biological neuron (rep\u2010\nresented in Figure 10-1). It is an unusual-looking cell mostly found in animal cerebral\ncortexes (e.g., your brain), composed of a cell body containing the nucleus and most\nof the cell\u2019s complex components, and many branching extensions called dendrites,\nplus one very long extension called the axon. The axon\u2019s length may be just a few\ntimes longer than the cell body, or up to tens of thousands of times longer. Near its\nextremity the axon splits off into many branches called telodendria, and at the tip of\nthese branches are minuscule structures called synaptic terminals (or simply synap\u2010", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 659, "content": "these branches are minuscule structures called synaptic terminals (or simply synap\u2010\nses), which are connected to the dendrites (or directly to the cell body) of other neu\u2010\nrons. Biological neurons receive short electrical impulses called signals from other\nneurons via these synapses. When a neuron receives a sufficient number of signals\nfrom other neurons within a few milliseconds, it fires its own signals.\nFigure 10-1. Biological neuron3\nThus, individual biological neurons seem to behave in a rather simple way, but they\nare organized in a vast network of billions of neurons, each neuron typically connec\u2010\nted to thousands of other neurons. Highly complex computations can be performed\nby a vast network of fairly simple neurons, much like a complex anthill can emerge\nfrom the combined efforts of simple ants. The architecture of biological neural net\u2010\nFrom Biological to Artificial Neurons \n| \n255\nDownload from finelybook www.finelybook.com", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 660, "content": "4 In the context of Machine Learning, the phrase \u201cneural networks\u201d generally refers to ANNs, not BNNs.\n5 Drawing of a cortical lamination by S. Ramon y Cajal (public domain). Reproduced from https://en.wikipe\ndia.org/wiki/Cerebral_cortex.\nworks (BNN)4 is still the subject of active research, but some parts of the brain have\nbeen mapped, and it seems that neurons are often organized in consecutive layers, as \nshown in Figure 10-2.\nFigure 10-2. Multiple layers in a biological neural network (human cortex)5\nLogical Computations with Neurons\nWarren McCulloch and Walter Pitts proposed a very simple model of the biological\nneuron, which later became known as an artificial neuron: it has one or more binary\n(on/off) inputs and one binary output. The artificial neuron simply activates its out\u2010\nput when more than a certain number of its inputs are active. McCulloch and Pitts\nshowed that even with such a simplified model it is possible to build a network of", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 661, "content": "showed that even with such a simplified model it is possible to build a network of\nartificial neurons that computes any logical proposition you want. For example, let\u2019s\nbuild a few ANNs that perform various logical computations (see Figure 10-3),\nassuming that a neuron is activated when at least two of its inputs are active.\nFigure 10-3. ANNs performing simple logical computations\n256 \n| \nChapter 10: Introduction to Artificial Neural Networks\nDownload from finelybook www.finelybook.com", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 662, "content": "\u2022 The first network on the left is simply the identity function: if neuron A is activa\u2010\nted, then neuron C gets activated as well (since it receives two input signals from\nneuron A), but if neuron A is off, then neuron C is off as well.\n\u2022 The second network performs a logical AND: neuron C is activated only when\nboth neurons A and B are activated (a single input signal is not enough to acti\u2010\nvate neuron C).\n\u2022 The third network performs a logical OR: neuron C gets activated if either neu\u2010\nron A or neuron B is activated (or both).\n\u2022 Finally, if we suppose that an input connection can inhibit the neuron\u2019s activity\n(which is the case with biological neurons), then the fourth network computes a\nslightly more complex logical proposition: neuron C is activated only if neuron A\nis active and if neuron B is off. If neuron A is active all the time, then you get a\nlogical NOT: neuron C is active when neuron B is off, and vice versa.", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 663, "content": "logical NOT: neuron C is active when neuron B is off, and vice versa.\nYou can easily imagine how these networks can be combined to compute complex\nlogical expressions (see the exercises at the end of the chapter).\nThe Perceptron\nThe Perceptron is one of the simplest ANN architectures, invented in 1957 by Frank\nRosenblatt. It is based on a slightly different artificial neuron (see Figure 10-4) called \na linear threshold unit (LTU): the inputs and output are now numbers (instead of\nbinary on/off values) and each input connection is associated with a weight. The LTU\ncomputes a weighted sum of its inputs (z = w1 x1 + w2 x2 + \u22ef + wn xn = wT \u00b7 x), then\napplies a step function to that sum and outputs the result: hw(x) = step (z) = step (wT \u00b7\nx).\nFigure 10-4. Linear threshold unit\nThe most common step function used in Perceptrons is the Heaviside step function\n(see Equation 10-1). Sometimes the sign function is used instead.\nFrom Biological to Artificial Neurons \n| \n257", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 664, "content": "From Biological to Artificial Neurons \n| \n257\nDownload from finelybook www.finelybook.com", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 665, "content": "6 The name Perceptron is sometimes used to mean a tiny network with a single LTU.\nEquation 10-1. Common step functions used in Perceptrons\nheaviside z = 0 if z < 0\n1 if z \u22650\nsgn z =\n\u22121 if z < 0\n0\nif z = 0\n+1 if z > 0\nA single LTU can be used for simple linear binary classification. It computes a linear\ncombination of the inputs and if the result exceeds a threshold, it outputs the positive\nclass or else outputs the negative class (just like a Logistic Regression classifier or a\nlinear SVM). For example, you could use a single LTU to classify iris flowers based on\nthe petal length and width (also adding an extra bias feature x0 = 1, just like we did in\nprevious chapters). Training an LTU means finding the right values for w0, w1, and w2\n(the training algorithm is discussed shortly).\nA Perceptron is simply composed of a single layer of LTUs,6 with each neuron con\u2010\nnected to all the inputs. These connections are often represented using special pass\u2010", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 666, "content": "nected to all the inputs. These connections are often represented using special pass\u2010\nthrough neurons called input neurons: they just output whatever input they are fed.\nMoreover, an extra bias feature is generally added (x0 = 1). This bias feature is typi\u2010\ncally represented using a special type of neuron called a bias neuron, which just out\u2010\nputs 1 all the time.\nA Perceptron with two inputs and three outputs is represented in Figure 10-5. This\nPerceptron can classify instances simultaneously into three different binary classes,\nwhich makes it a multioutput classifier.\nFigure 10-5. Perceptron diagram\nSo how is a Perceptron trained? The Perceptron training algorithm proposed by\nFrank Rosenblatt was largely inspired by Hebb\u2019s rule. In his book The Organization of\nBehavior, published in 1949, Donald Hebb suggested that when a biological neuron\n258 \n| \nChapter 10: Introduction to Artificial Neural Networks\nDownload from finelybook www.finelybook.com", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 667, "content": "7 Note that this solution is generally not unique: in general when the data are linearly separable, there is an\ninfinity of hyperplanes that can separate them.\noften triggers another neuron, the connection between these two neurons grows\nstronger. This idea was later summarized by Siegrid L\u00f6wel in this catchy phrase:\n\u201cCells that fire together, wire together.\u201d This rule later became known as Hebb\u2019s rule \n(or Hebbian learning); that is, the connection weight between two neurons is\nincreased whenever they have the same output. Perceptrons are trained using a var\u2010\niant of this rule that takes into account the error made by the network; it does not\nreinforce connections that lead to the wrong output. More specifically, the Perceptron\nis fed one training instance at a time, and for each instance it makes its predictions.\nFor every output neuron that produced a wrong prediction, it reinforces the connec\u2010\ntion weights from the inputs that would have contributed to the correct prediction.", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 668, "content": "tion weights from the inputs that would have contributed to the correct prediction.\nThe rule is shown in Equation 10-2.\nEquation 10-2. Perceptron learning rule (weight update)\nwi, j\nnext step = wi, j + \u03b7 y j \u2212yj xi\n\u2022 wi, j is the connection weight between the ith input neuron and the jth output neu\u2010\nron.\n\u2022 xi is the ith input value of the current training instance.\n\u2022 yj is the output of the jth output neuron for the current training instance.\n\u2022 yj is the target output of the jth output neuron for the current training instance.\n\u2022 \u03b7 is the learning rate.\nThe decision boundary of each output neuron is linear, so Perceptrons are incapable\nof learning complex patterns (just like Logistic Regression classifiers). However, if the\ntraining instances are linearly separable, Rosenblatt demonstrated that this algorithm\nwould converge to a solution.7 This is called the Perceptron convergence theorem.\nScikit-Learn provides a Perceptron class that implements a single LTU network. It", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 669, "content": "Scikit-Learn provides a Perceptron class that implements a single LTU network. It\ncan be used pretty much as you would expect\u2014for example, on the iris dataset (intro\u2010\nduced in Chapter 4):\nimport numpy as np\nfrom sklearn.datasets import load_iris\nfrom sklearn.linear_model import Perceptron\niris = load_iris()\nX = iris.data[:, (2, 3)] # petal length, petal width\ny = (iris.target == 0).astype(np.int) # Iris Setosa?\nFrom Biological to Artificial Neurons \n| \n259\nDownload from finelybook www.finelybook.com", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 670, "content": "per_clf = Perceptron(random_state=42)\nper_clf.fit(X, y)\ny_pred = per_clf.predict([[2, 0.5]])\nYou may have recognized that the Perceptron learning algorithm strongly resembles \nStochastic Gradient Descent. In fact, Scikit-Learn\u2019s Perceptron class is equivalent to\nusing an SGDClassifier with the following hyperparameters: loss=\"perceptron\",\nlearning_rate=\"constant\", eta0=1 (the learning rate), and penalty=None (no regu\u2010\nlarization).\nNote that contrary to Logistic Regression classifiers, Perceptrons do not output a class\nprobability; rather, they just make predictions based on a hard threshold. This is one\nof the good reasons to prefer Logistic Regression over Perceptrons.\nIn their 1969 monograph titled Perceptrons, Marvin Minsky and Seymour Papert\nhighlighted a number of serious weaknesses of Perceptrons, in particular the fact that\nthey are incapable of solving some trivial problems (e.g., the Exclusive OR (XOR)", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 671, "content": "they are incapable of solving some trivial problems (e.g., the Exclusive OR (XOR)\nclassification problem; see the left side of Figure 10-6). Of course this is true of any\nother linear classification model as well (such as Logistic Regression classifiers), but\nresearchers had expected much more from Perceptrons, and their disappointment\nwas great: as a result, many researchers dropped connectionism altogether (i.e., the\nstudy of neural networks) in favor of higher-level problems such as logic, problem\nsolving, and search.\nHowever, it turns out that some of the limitations of Perceptrons can be eliminated by\nstacking multiple Perceptrons. The resulting ANN is called a Multi-Layer Perceptron\n(MLP). In particular, an MLP can solve the XOR problem, as you can verify by com\u2010\nputing the output of the MLP represented on the right of Figure 10-6, for each com\u2010\nbination of inputs: with inputs (0, 0) or (1, 1) the network outputs 0, and with inputs\n(0, 1) or (1, 0) it outputs 1.", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 672, "content": "(0, 1) or (1, 0) it outputs 1.\nFigure 10-6. XOR classification problem and an MLP that solves it\n260 \n| \nChapter 10: Introduction to Artificial Neural Networks\nDownload from finelybook www.finelybook.com", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 673, "content": "8 \u201cLearning Internal Representations by Error Propagation,\u201d D. Rumelhart, G. Hinton, R. Williams (1986).\n9 This algorithm was actually invented several times by various researchers in different fields, starting with\nP. Werbos in 1974.\nMulti-Layer Perceptron and Backpropagation\nAn MLP is composed of one (passthrough) input layer, one or more layers of LTUs,\ncalled hidden layers, and one final layer of LTUs called the output layer (see\nFigure 10-7). Every layer except the output layer includes a bias neuron and is fully\nconnected to the next layer. When an ANN has two or more hidden layers, it is called \na deep neural network (DNN).\nFigure 10-7. Multi-Layer Perceptron\nFor many years researchers struggled to find a way to train MLPs, without success.\nBut in 1986, D. E. Rumelhart et al. published a groundbreaking article8 introducing\nthe backpropagation training algorithm.9 Today we would describe it as Gradient", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 674, "content": "the backpropagation training algorithm.9 Today we would describe it as Gradient\nDescent using reverse-mode autodiff (Gradient Descent was introduced in Chapter 4,\nand autodiff was discussed in Chapter 9).\nFor each training instance, the algorithm feeds it to the network and computes the\noutput of every neuron in each consecutive layer (this is the forward pass, just like\nwhen making predictions). Then it measures the network\u2019s output error (i.e., the dif\u2010\nference between the desired output and the actual output of the network), and it\ncomputes how much each neuron in the last hidden layer contributed to each output\nneuron\u2019s error. It then proceeds to measure how much of these error contributions\ncame from each neuron in the previous hidden layer\u2014and so on until the algorithm\nreaches the input layer. This reverse pass efficiently measures the error gradient\nacross all the connection weights in the network by propagating the error gradient", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 675, "content": "across all the connection weights in the network by propagating the error gradient\nbackward in the network (hence the name of the algorithm). If you check out the\nFrom Biological to Artificial Neurons \n| \n261\nDownload from finelybook www.finelybook.com", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 676, "content": "reverse-mode autodiff algorithm in Appendix D, you will find that the forward and\nreverse passes of backpropagation simply perform reverse-mode autodiff. The last\nstep of the backpropagation algorithm is a Gradient Descent step on all the connec\u2010\ntion weights in the network, using the error gradients measured earlier.\nLet\u2019s make this even shorter: for each training instance the backpropagation algo\u2010\nrithm first makes a prediction (forward pass), measures the error, then goes through\neach layer in reverse to measure the error contribution from each connection (reverse\npass), and finally slightly tweaks the connection weights to reduce the error (Gradient\nDescent step).\nIn order for this algorithm to work properly, the authors made a key change to the\nMLP\u2019s architecture: they replaced the step function with the logistic function, \u03c3(z) =\n1 / (1 + exp(\u2013z)). This was essential because the step function contains only flat seg\u2010", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 677, "content": "1 / (1 + exp(\u2013z)). This was essential because the step function contains only flat seg\u2010\nments, so there is no gradient to work with (Gradient Descent cannot move on a flat\nsurface), while the logistic function has a well-defined nonzero derivative every\u2010\nwhere, allowing Gradient Descent to make some progress at every step. The backpro\u2010\npagation algorithm may be used with other activation functions, instead of the logistic\nfunction. Two other popular activation functions are:\nThe hyperbolic tangent function tanh (z) = 2\u03c3(2z) \u2013 1\nJust like the logistic function it is S-shaped, continuous, and differentiable, but its\noutput value ranges from \u20131 to 1 (instead of 0 to 1 in the case of the logistic func\u2010\ntion), which tends to make each layer\u2019s output more or less normalized (i.e., cen\u2010\ntered around 0) at the beginning of training. This often helps speed up\nconvergence.\nThe ReLU function (introduced in Chapter 9)", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 678, "content": "convergence.\nThe ReLU function (introduced in Chapter 9)\nReLU (z) = max (0, z). It is continuous but unfortunately not differentiable at z =\n0 (the slope changes abruptly, which can make Gradient Descent bounce\naround). However, in practice it works very well and has the advantage of being\nfast to compute. Most importantly, the fact that it does not have a maximum out\u2010\nput value also helps reduce some issues during Gradient Descent (we will come\nback to this in Chapter 11).\nThese popular activation functions and their derivatives are represented in\nFigure 10-8.\n262 \n| \nChapter 10: Introduction to Artificial Neural Networks\nDownload from finelybook www.finelybook.com", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 679, "content": "Figure 10-8. Activation functions and their derivatives\nAn MLP is often used for classification, with each output corresponding to a different\nbinary class (e.g., spam/ham, urgent/not-urgent, and so on). When the classes are\nexclusive (e.g., classes 0 through 9 for digit image classification), the output layer is\ntypically modified by replacing the individual activation functions by a shared soft\u2010\nmax function (see Figure 10-9). The softmax function was introduced in Chapter 3.\nThe output of each neuron corresponds to the estimated probability of the corre\u2010\nsponding class. Note that the signal flows only in one direction (from the inputs to\nthe outputs), so this architecture is an example of a feedforward neural network\n(FNN).\nFigure 10-9. A modern MLP (including ReLU and softmax) for classification\nFrom Biological to Artificial Neurons \n| \n263\nDownload from finelybook www.finelybook.com", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 680, "content": "Biological neurons seem to implement a roughly sigmoid (S-\nshaped) activation function, so researchers stuck to sigmoid func\u2010\ntions for a very long time. But it turns out that the ReLU activation\nfunction generally works better in ANNs. This is one of the cases\nwhere the biological analogy was misleading.\nTraining an MLP with TensorFlow\u2019s High-Level API\nThe simplest way to train an MLP with TensorFlow is to use the high-level API\nTF.Learn, which is quite similar to Scikit-Learn\u2019s API. The DNNClassifier class\nmakes it trivial to train a deep neural network with any number of hidden layers, and\na softmax output layer to output estimated class probabilities. For example, the fol\u2010\nlowing code trains a DNN for classification with two hidden layers (one with 300\nneurons, and the other with 100 neurons) and a softmax output layer with 10\nneurons:\nimport tensorflow as tf\nfeature_columns = tf.contrib.learn.infer_real_valued_columns_from_input(X_train)", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 681, "content": "feature_columns = tf.contrib.learn.infer_real_valued_columns_from_input(X_train)\ndnn_clf = tf.contrib.learn.DNNClassifier(hidden_units=[300, 100], n_classes=10,\n feature_columns=feature_columns)\ndnn_clf.fit(x=X_train, y=y_train, batch_size=50, steps=40000)\nIf you run this code on the MNIST dataset (after scaling it, e.g., by using Scikit-\nLearn\u2019s StandardScaler), you may actually get a model that achieves over 98.1%\naccuracy on the test set! That\u2019s better than the best model we trained in Chapter 3:\n>>> from sklearn.metrics import accuracy_score\n>>> y_pred = list(dnn_clf.predict(X_test))\n>>> accuracy_score(y_test, y_pred)\n0.98180000000000001\nThe TF.Learn library also provides some convenience functions to evaluate models:\n>>> dnn_clf.evaluate(X_test, y_test)\n{'accuracy': 0.98180002, 'global_step': 40000, 'loss': 0.073678359}\nUnder the hood, the DNNClassifier class creates all the neuron layers, based on the", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 682, "content": "Under the hood, the DNNClassifier class creates all the neuron layers, based on the\nReLU activation function (we can change this by setting the activation_fn hyper\u2010\nparameter). The output layer relies on the softmax function, and the cost function is\ncross entropy (introduced in Chapter 4).\nThe TF.Learn API is still quite new, so some of the names and func\u2010\ntions used in these examples may evolve a bit by the time you read\nthis book. However, the general ideas should not change.\n264 \n| \nChapter 10: Introduction to Artificial Neural Networks\nDownload from finelybook www.finelybook.com", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 683, "content": "Training a DNN Using Plain TensorFlow\nIf you want more control over the architecture of the network, you may prefer to use\nTensorFlow\u2019s lower-level Python API (introduced in Chapter 9). In this section we\nwill build the same model as before using this API, and we will implement Mini-\nbatch Gradient Descent to train it on the MNIST dataset. The first step is the con\u2010\nstruction phase, building the TensorFlow graph. The second step is the execution\nphase, where you actually run the graph to train the model.\nConstruction Phase\nLet\u2019s start. First we need to import the tensorflow library. Then we must specify the\nnumber of inputs and outputs, and set the number of hidden neurons in each layer:\nimport tensorflow as tf\nn_inputs = 28*28 # MNIST\nn_hidden1 = 300\nn_hidden2 = 100\nn_outputs = 10\nNext, just like you did in Chapter 9, you can use placeholder nodes to represent the\ntraining data and targets. The shape of X is only partially defined. We know that it will", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 684, "content": "training data and targets. The shape of X is only partially defined. We know that it will\nbe a 2D tensor (i.e., a matrix), with instances along the first dimension and features\nalong the second dimension, and we know that the number of features is going to be\n28 x 28 (one feature per pixel), but we don\u2019t know yet how many instances each train\u2010\ning batch will contain. So the shape of X is (None, n_inputs). Similarly, we know\nthat y will be a 1D tensor with one entry per instance, but again we don\u2019t know the\nsize of the training batch at this point, so the shape is (None).\nX = tf.placeholder(tf.float32, shape=(None, n_inputs), name=\"X\")\ny = tf.placeholder(tf.int64, shape=(None), name=\"y\")\nNow let\u2019s create the actual neural network. The placeholder X will act as the input\nlayer; during the execution phase, it will be replaced with one training batch at a time\n(note that all the instances in a training batch will be processed simultaneously by the", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 685, "content": "(note that all the instances in a training batch will be processed simultaneously by the\nneural network). Now you need to create the two hidden layers and the output layer.\nThe two hidden layers are almost identical: they differ only by the inputs they are\nconnected to and by the number of neurons they contain. The output layer is also\nvery similar, but it uses a softmax activation function instead of a ReLU activation\nfunction. So let\u2019s create a neuron_layer() function that we will use to create one layer\nat a time. It will need parameters to specify the inputs, the number of neurons, the\nactivation function, and the name of the layer:\ndef neuron_layer(X, n_neurons, name, activation=None):\n with tf.name_scope(name):\n n_inputs = int(X.get_shape()[1])\nTraining a DNN Using Plain TensorFlow \n| \n265\nDownload from finelybook www.finelybook.com", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 686, "content": "10 Using a truncated normal distribution rather than a regular normal distribution ensures that there won\u2019t be\nany large weights, which could slow down training.\n11 For example, if you set all the weights to 0, then all neurons will output 0, and the error gradient will be the\nsame for all neurons in a given hidden layer. The Gradient Descent step will then update all the weights in\nexactly the same way in each layer, so they will all remain equal. In other words, despite having hundreds of\nneurons per layer, your model will act as if there were only one neuron per layer. It is not going to fly.\n stddev = 2 / np.sqrt(n_inputs)\n init = tf.truncated_normal((n_inputs, n_neurons), stddev=stddev)\n W = tf.Variable(init, name=\"weights\")\n b = tf.Variable(tf.zeros([n_neurons]), name=\"biases\")\n z = tf.matmul(X, W) + b\n if activation==\"relu\":\n return tf.nn.relu(z)\n else:\n return z\nLet\u2019s go through this code line by line:", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 687, "content": "else:\n return z\nLet\u2019s go through this code line by line:\n1. First we create a name scope using the name of the layer: it will contain all the\ncomputation nodes for this neuron layer. This is optional, but the graph will look\nmuch nicer in TensorBoard if its nodes are well organized.\n2. Next, we get the number of inputs by looking up the input matrix\u2019s shape and\ngetting the size of the second dimension (the first dimension is for instances).\n3. The next three lines create a W variable that will hold the weights matrix. It will be\na 2D tensor containing all the connection weights between each input and each\nneuron; hence, its shape will be (n_inputs, n_neurons). It will be initialized\nrandomly, using a truncated10 normal (Gaussian) distribution with a standard\ndeviation of 2/ ninputs. Using this specific standard deviation helps the algorithm\nconverge much faster (we will discuss this further in Chapter 11; it is one of those", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 688, "content": "converge much faster (we will discuss this further in Chapter 11; it is one of those\nsmall tweaks to neural networks that have had a tremendous impact on their effi\u2010\nciency). It is important to initialize connection weights randomly for all hidden\nlayers to avoid any symmetries that the Gradient Descent algorithm would be\nunable to break.11\n4. The next line creates a b variable for biases, initialized to 0 (no symmetry issue in\nthis case), with one bias parameter per neuron.\n5. Then we create a subgraph to compute z = X \u00b7 W + b. This vectorized implemen\u2010\ntation will efficiently compute the weighted sums of the inputs plus the bias term\nfor each and every neuron in the layer, for all the instances in the batch in just\none shot.\n6. Finally, if the activation parameter is set to \"relu\", the code returns relu(z) \n(i.e., max (0, z)), or else it just returns z.\n266 \n| \nChapter 10: Introduction to Artificial Neural Networks\nDownload from finelybook www.finelybook.com", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 689, "content": "Okay, so now you have a nice function to create a neuron layer. Let\u2019s use it to create\nthe deep neural network! The first hidden layer takes X as its input. The second takes\nthe output of the first hidden layer as its input. And finally, the output layer takes the\noutput of the second hidden layer as its input.\nwith tf.name_scope(\"dnn\"):\n hidden1 = neuron_layer(X, n_hidden1, \"hidden1\", activation=\"relu\")\n hidden2 = neuron_layer(hidden1, n_hidden2, \"hidden2\", activation=\"relu\")\n logits = neuron_layer(hidden2, n_outputs, \"outputs\")\nNotice that once again we used a name scope for clarity. Also note that logits is the\noutput of the neural network before going through the softmax activation function:\nfor optimization reasons, we will handle the softmax computation later.\nAs you might expect, TensorFlow comes with many handy functions to create\nstandard neural network layers, so there\u2019s often no need to define your own", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 690, "content": "standard neural network layers, so there\u2019s often no need to define your own\nneuron_layer() function like we just did. For example, TensorFlow\u2019s fully_connec\nted() function creates a fully connected layer, where all the inputs are connected to\nall the neurons in the layer. It takes care of creating the weights and biases variables,\nwith the proper initialization strategy, and it uses the ReLU activation function by\ndefault (we can change this using the activation_fn argument). As we will see in\nChapter 11, it also supports regularization and normalization parameters. Let\u2019s tweak\nthe preceding code to use the fully_connected() function instead of our neu\nron_layer() function. Simply import the function and replace the dnn construction\nsection with the following code:\nfrom tensorflow.contrib.layers import fully_connected\nwith tf.name_scope(\"dnn\"):\n hidden1 = fully_connected(X, n_hidden1, scope=\"hidden1\")\n hidden2 = fully_connected(hidden1, n_hidden2, scope=\"hidden2\")", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 691, "content": "hidden2 = fully_connected(hidden1, n_hidden2, scope=\"hidden2\")\n logits = fully_connected(hidden2, n_outputs, scope=\"outputs\",\n activation_fn=None)\nThe tensorflow.contrib package contains many useful functions,\nbut it is a place for experimental code that has not yet graduated to\nbe part of the main TensorFlow API. So the fully_connected()\nfunction (and any other contrib code) may change or move in the\nfuture.\nNow that we have the neural network model ready to go, we need to define the cost\nfunction that we will use to train it. Just as we did for Softmax Regression in Chap\u2010\nter 4, we will use cross entropy. As we discussed earlier, cross entropy will penalize\nmodels that estimate a low probability for the target class. TensorFlow provides\nseveral functions to compute cross entropy. We will use sparse_soft\nmax_cross_entropy_with_logits(): it computes the cross entropy based on the\nTraining a DNN Using Plain TensorFlow \n| \n267", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 692, "content": "Training a DNN Using Plain TensorFlow \n| \n267\nDownload from finelybook www.finelybook.com", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 693, "content": "\u201clogits\u201d (i.e., the output of the network before going through the softmax activation\nfunction), and it expects labels in the form of integers ranging from 0 to the number\nof classes minus 1 (in our case, from 0 to 9). This will give us a 1D tensor containing\nthe cross entropy for each instance. We can then use TensorFlow\u2019s reduce_mean()\nfunction to compute the mean cross entropy over all instances.\nwith tf.name_scope(\"loss\"):\n xentropy = tf.nn.sparse_softmax_cross_entropy_with_logits(\n labels=y, logits=logits)\n loss = tf.reduce_mean(xentropy, name=\"loss\")\nThe sparse_softmax_cross_entropy_with_logits() function is\nequivalent to applying the softmax activation function and then\ncomputing the cross entropy, but it is more efficient, and it prop\u2010\nerly takes care of corner cases like logits equal to 0. This is why we\ndid not apply the softmax activation function earlier. There is also\nanother function called softmax_cross_entropy_with_logits(),", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 694, "content": "another function called softmax_cross_entropy_with_logits(),\nwhich takes labels in the form of one-hot vectors (instead of ints\nfrom 0 to the number of classes minus 1).\nWe have the neural network model, we have the cost function, and now we need to\ndefine a GradientDescentOptimizer that will tweak the model parameters to mini\u2010\nmize the cost function. Nothing new; it\u2019s just like we did in Chapter 9:\nlearning_rate = 0.01\nwith tf.name_scope(\"train\"):\n optimizer = tf.train.GradientDescentOptimizer(learning_rate)\n training_op = optimizer.minimize(loss)\nThe last important step in the construction phase is to specify how to evaluate the\nmodel. We will simply use accuracy as our performance measure. First, for each\ninstance, determine if the neural network\u2019s prediction is correct by checking whether\nor not the highest logit corresponds to the target class. For this you can use the\nin_top_k() function. This returns a 1D tensor full of boolean values, so we need to", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 695, "content": "in_top_k() function. This returns a 1D tensor full of boolean values, so we need to\ncast these booleans to floats and then compute the average. This will give us the net\u2010\nwork\u2019s overall accuracy.\nwith tf.name_scope(\"eval\"):\n correct = tf.nn.in_top_k(logits, y, 1)\n accuracy = tf.reduce_mean(tf.cast(correct, tf.float32))\nAnd, as usual, we need to create a node to initialize all variables, and we will also cre\u2010\nate a Saver to save our trained model parameters to disk:\ninit = tf.global_variables_initializer()\nsaver = tf.train.Saver()\n268 \n| \nChapter 10: Introduction to Artificial Neural Networks\nDownload from finelybook www.finelybook.com", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 696, "content": "Phew! This concludes the construction phase. This was fewer than 40 lines of code,\nbut it was pretty intense: we created placeholders for the inputs and the targets, we\ncreated a function to build a neuron layer, we used it to create the DNN, we defined\nthe cost function, we created an optimizer, and finally we defined the performance\nmeasure. Now on to the execution phase.\nExecution Phase\nThis part is much shorter and simpler. First, let\u2019s load MNIST. We could use Scikit-\nLearn for that as we did in previous chapters, but TensorFlow offers its own helper\nthat fetches the data, scales it (between 0 and 1), shuffles it, and provides a simple\nfunction to load one mini-batches a time. So let\u2019s use it instead:\nfrom tensorflow.examples.tutorials.mnist import input_data\nmnist = input_data.read_data_sets(\"/tmp/data/\")\nNow we define the number of epochs that we want to run, as well as the size of the\nmini-batches:\nn_epochs = 400\nbatch_size = 50\nAnd now we can train the model:", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 697, "content": "mini-batches:\nn_epochs = 400\nbatch_size = 50\nAnd now we can train the model:\nwith tf.Session() as sess:\n init.run()\n for epoch in range(n_epochs):\n for iteration in range(mnist.train.num_examples // batch_size):\n X_batch, y_batch = mnist.train.next_batch(batch_size)\n sess.run(training_op, feed_dict={X: X_batch, y: y_batch})\n acc_train = accuracy.eval(feed_dict={X: X_batch, y: y_batch})\n acc_test = accuracy.eval(feed_dict={X: mnist.test.images,\n y: mnist.test.labels})\n print(epoch, \"Train accuracy:\", acc_train, \"Test accuracy:\", acc_test)\n save_path = saver.save(sess, \"./my_model_final.ckpt\")\nThis code opens a TensorFlow session, and it runs the init node that initializes all\nthe variables. Then it runs the main training loop: at each epoch, the code iterates\nthrough a number of mini-batches that corresponds to the training set size. Each", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 698, "content": "through a number of mini-batches that corresponds to the training set size. Each\nmini-batch is fetched via the next_batch() method, and then the code simply runs\nthe training operation, feeding it the current mini-batch input data and targets. Next,\nat the end of each epoch, the code evaluates the model on the last mini-batch and on\nthe full training set, and it prints out the result. Finally, the model parameters are\nsaved to disk.\nTraining a DNN Using Plain TensorFlow \n| \n269\nDownload from finelybook www.finelybook.com", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 699, "content": "Using the Neural Network\nNow that the neural network is trained, you can use it to make predictions. To do\nthat, you can reuse the same construction phase, but change the execution phase like\nthis:\nwith tf.Session() as sess:\n saver.restore(sess, \"./my_model_final.ckpt\")\n X_new_scaled = [...] # some new images (scaled from 0 to 1)\n Z = logits.eval(feed_dict={X: X_new_scaled})\n y_pred = np.argmax(Z, axis=1)\nFirst the code loads the model parameters from disk. Then it loads some new images\nthat you want to classify. Remember to apply the same feature scaling as for the train\u2010\ning data (in this case, scale it from 0 to 1). Then the code evaluates the logits node.\nIf you wanted to know all the estimated class probabilities, you would need to apply\nthe softmax() function to the logits, but if you just want to predict a class, you can\nsimply pick the class that has the highest logit value (using the argmax() function\ndoes the trick).\nFine-Tuning Neural Network Hyperparameters", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 700, "content": "does the trick).\nFine-Tuning Neural Network Hyperparameters\nThe flexibility of neural networks is also one of their main drawbacks: there are many\nhyperparameters to tweak. Not only can you use any imaginable network topology\n(how neurons are interconnected), but even in a simple MLP you can change the\nnumber of layers, the number of neurons per layer, the type of activation function to\nuse in each layer, the weight initialization logic, and much more. How do you know\nwhat combination of hyperparameters is the best for your task?\nOf course, you can use grid search with cross-validation to find the right hyperpara\u2010\nmeters, like you did in previous chapters, but since there are many hyperparameters\nto tune, and since training a neural network on a large dataset takes a lot of time, you\nwill only be able to explore a tiny part of the hyperparameter space in a reasonable\namount of time. It is much better to use randomized search, as we discussed in Chap\u2010", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 701, "content": "amount of time. It is much better to use randomized search, as we discussed in Chap\u2010\nter 2. Another option is to use a tool such as Oscar, which implements more complex\nalgorithms to help you find a good set of hyperparameters quickly.\nIt helps to have an idea of what values are reasonable for each hyperparameter, so you\ncan restrict the search space. Let\u2019s start with the number of hidden layers.\nNumber of Hidden Layers\nFor many problems, you can just begin with a single hidden layer and you will get\nreasonable results. It has actually been shown that an MLP with just one hidden layer\ncan model even the most complex functions provided it has enough neurons. For a\nlong time, these facts convinced researchers that there was no need to investigate any\n270 \n| \nChapter 10: Introduction to Artificial Neural Networks\nDownload from finelybook www.finelybook.com", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 702, "content": "deeper neural networks. But they overlooked the fact that deep networks have a much\nhigher parameter efficiency than shallow ones: they can model complex functions\nusing exponentially fewer neurons than shallow nets, making them much faster to\ntrain.\nTo understand why, suppose you are asked to draw a forest using some drawing soft\u2010\nware, but you are forbidden to use copy/paste. You would have to draw each tree\nindividually, branch per branch, leaf per leaf. If you could instead draw one leaf,\ncopy/paste it to draw a branch, then copy/paste that branch to create a tree, and\nfinally copy/paste this tree to make a forest, you would be finished in no time. Real-\nworld data is often structured in such a hierarchical way and DNNs automatically\ntake advantage of this fact: lower hidden layers model low-level structures (e.g., line\nsegments of various shapes and orientations), intermediate hidden layers combine", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 703, "content": "segments of various shapes and orientations), intermediate hidden layers combine\nthese low-level structures to model intermediate-level structures (e.g., squares, cir\u2010\ncles), and the highest hidden layers and the output layer combine these intermediate\nstructures to model high-level structures (e.g., faces).\nNot only does this hierarchical architecture help DNNs converge faster to a good sol\u2010\nution, it also improves their ability to generalize to new datasets. For example, if you\nhave already trained a model to recognize faces in pictures, and you now want to\ntrain a new neural network to recognize hairstyles, then you can kickstart training by\nreusing the lower layers of the first network. Instead of randomly initializing the\nweights and biases of the first few layers of the new neural network, you can initialize\nthem to the value of the weights and biases of the lower layers of the first network.\nThis way the network will not have to learn from scratch all the low-level structures", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 704, "content": "This way the network will not have to learn from scratch all the low-level structures\nthat occur in most pictures; it will only have to learn the higher-level structures (e.g.,\nhairstyles).\nIn summary, for many problems you can start with just one or two hidden layers and\nit will work just fine (e.g., you can easily reach above 97% accuracy on the MNIST\ndataset using just one hidden layer with a few hundred neurons, and above 98% accu\u2010\nracy using two hidden layers with the same total amount of neurons, in roughly the\nsame amount of training time). For more complex problems, you can gradually ramp\nup the number of hidden layers, until you start overfitting the training set. Very com\u2010\nplex tasks, such as large image classification or speech recognition, typically require\nnetworks with dozens of layers (or even hundreds, but not fully connected ones, as\nwe will see in Chapter 13), and they need a huge amount of training data. However,", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 705, "content": "we will see in Chapter 13), and they need a huge amount of training data. However,\nyou will rarely have to train such networks from scratch: it is much more common to\nreuse parts of a pretrained state-of-the-art network that performs a similar task.\nTraining will be a lot faster and require much less data (we will discuss this in Chap\u2010\nter 11).\nFine-Tuning Neural Network Hyperparameters \n| \n271\nDownload from finelybook www.finelybook.com", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 706, "content": "12 By Vincent Vanhoucke in his Deep Learning class on Udacity.com.\n13 A few extra ANN architectures are presented in Appendix E.\nNumber of Neurons per Hidden Layer\nObviously the number of neurons in the input and output layers is determined by the\ntype of input and output your task requires. For example, the MNIST task requires 28\nx 28 = 784 input neurons and 10 output neurons. As for the hidden layers, a common\npractice is to size them to form a funnel, with fewer and fewer neurons at each layer\u2014\nthe rationale being that many low-level features can coalesce into far fewer high-level\nfeatures. For example, a typical neural network for MNIST may have two hidden lay\u2010\ners, the first with 300 neurons and the second with 100. However, this practice is not\nas common now, and you may simply use the same size for all hidden layers\u2014for\nexample, all hidden layers with 150 neurons: that\u2019s just one hyperparameter to tune", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 707, "content": "example, all hidden layers with 150 neurons: that\u2019s just one hyperparameter to tune\ninstead of one per layer. Just like for the number of layers, you can try increasing the\nnumber of neurons gradually until the network starts overfitting. In general you will\nget more bang for the buck by increasing the number of layers than the number of\nneurons per layer. Unfortunately, as you can see, finding the perfect amount of neu\u2010\nrons is still somewhat of a black art.\nA simpler approach is to pick a model with more layers and neurons than you\nactually need, then use early stopping to prevent it from overfitting (and other regu\u2010\nlarization techniques, especially dropout, as we will see in Chapter 11). This has been\ndubbed the \u201cstretch pants\u201d approach:12 instead of wasting time looking for pants that\nperfectly match your size, just use large stretch pants that will shrink down to the\nright size.\nActivation Functions", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 708, "content": "right size.\nActivation Functions\nIn most cases you can use the ReLU activation function in the hidden layers (or one\nof its variants, as we will see in Chapter 11). It is a bit faster to compute than other\nactivation functions, and Gradient Descent does not get stuck as much on plateaus,\nthanks to the fact that it does not saturate for large input values (as opposed to the\nlogistic function or the hyperbolic tangent function, which saturate at 1).\nFor the output layer, the softmax activation function is generally a good choice for\nclassification tasks (when the classes are mutually exclusive). For regression tasks,\nyou can simply use no activation function at all.\nThis concludes this introduction to artificial neural networks. In the following chap\u2010\nters, we will discuss techniques to train very deep nets, and distribute training across\nmultiple servers and GPUs. Then we will explore a few other popular neural network", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 709, "content": "multiple servers and GPUs. Then we will explore a few other popular neural network\narchitectures: convolutional neural networks, recurrent neural networks, and autoen\u2010\ncoders.13\n272 \n| \nChapter 10: Introduction to Artificial Neural Networks\nDownload from finelybook www.finelybook.com", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 710, "content": "Exercises\n1. Draw an ANN using the original artificial neurons (like the ones in Figure 10-3)\nthat computes A \u2295 B (where \u2295 represents the XOR operation). Hint: A \u2295 B = (A\n\u2227 \u00ac B) \u2228 (\u00ac A \u2227 B).\n2. Why is it generally preferable to use a Logistic Regression classifier rather than a\nclassical Perceptron (i.e., a single layer of linear threshold units trained using the\nPerceptron training algorithm)? How can you tweak a Perceptron to make it\nequivalent to a Logistic Regression classifier?\n3. Why was the logistic activation function a key ingredient in training the first\nMLPs?\n4. Name three popular activation functions. Can you draw them?\n5. Suppose you have an MLP composed of one input layer with 10 passthrough\nneurons, followed by one hidden layer with 50 artificial neurons, and finally one\noutput layer with 3 artificial neurons. All artificial neurons use the ReLU activa\u2010\ntion function.\n\u2022 What is the shape of the input matrix X?", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 711, "content": "tion function.\n\u2022 What is the shape of the input matrix X?\n\u2022 What about the shape of the hidden layer\u2019s weight vector Wh, and the shape of\nits bias vector bh?\n\u2022 What is the shape of the output layer\u2019s weight vector Wo, and its bias vector bo?\n\u2022 What is the shape of the network\u2019s output matrix Y?\n\u2022 Write the equation that computes the network\u2019s output matrix Y as a function\nof X, Wh, bh, Wo and bo.\n6. How many neurons do you need in the output layer if you want to classify email\ninto spam or ham? What activation function should you use in the output layer?\nIf instead you want to tackle MNIST, how many neurons do you need in the out\u2010\nput layer, using what activation function? Answer the same questions for getting\nyour network to predict housing prices as in Chapter 2.\n7. What is backpropagation and how does it work? What is the difference between\nbackpropagation and reverse-mode autodiff?\n8. Can you list all the hyperparameters you can tweak in an MLP? If the MLP over\u2010", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 712, "content": "8. Can you list all the hyperparameters you can tweak in an MLP? If the MLP over\u2010\nfits the training data, how could you tweak these hyperparameters to try to solve\nthe problem?\n9. Train a deep MLP on the MNIST dataset and see if you can get over 98% preci\u2010\nsion. Just like in the last exercise of Chapter 9, try adding all the bells and whistles\nExercises \n| \n273\nDownload from finelybook www.finelybook.com", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 713, "content": "(i.e., save checkpoints, restore the last checkpoint in case of an interruption, add\nsummaries, plot learning curves using TensorBoard, and so on).\nSolutions to these exercises are available in Appendix A.\n274 \n| \nChapter 10: Introduction to Artificial Neural Networks\nDownload from finelybook www.finelybook.com", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 714, "content": "CHAPTER 11\nTraining Deep Neural Nets\nIn Chapter 10 we introduced artificial neural networks and trained our first deep\nneural network. But it was a very shallow DNN, with only two hidden layers. What if\nyou need to tackle a very complex problem, such as detecting hundreds of types of\nobjects in high-resolution images? You may need to train a much deeper DNN, per\u2010\nhaps with (say) 10 layers, each containing hundreds of neurons, connected by hun\u2010\ndreds of thousands of connections. This would not be a walk in the park:\n\u2022 First, you would be faced with the tricky vanishing gradients problem (or the\nrelated exploding gradients problem) that affects deep neural networks and makes\nlower layers very hard to train.\n\u2022 Second, with such a large network, training would be extremely slow.\n\u2022 Third, a model with millions of parameters would severely risk overfitting the\ntraining set.\nIn this chapter, we will go through each of these problems in turn and present techni\u2010", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 715, "content": "In this chapter, we will go through each of these problems in turn and present techni\u2010\nques to solve them. We will start by explaining the vanishing gradients problem and\nexploring some of the most popular solutions to this problem. Next we will look at\nvarious optimizers that can speed up training large models tremendously compared\nto plain Gradient Descent. Finally, we will go through a few popular regularization\ntechniques for large neural networks.\nWith these tools, you will be able to train very deep nets: welcome to Deep Learning!\nVanishing/Exploding Gradients Problems\nAs we discussed in Chapter 10, the backpropagation algorithm works by going from\nthe output layer to the input layer, propagating the error gradient on the way. Once\nthe algorithm has computed the gradient of the cost function with regards to each\n275\nDownload from finelybook www.finelybook.com", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 716, "content": "1 \u201cUnderstanding the Difficulty of Training Deep Feedforward Neural Networks,\u201d X. Glorot, Y Bengio (2010).\nparameter in the network, it uses these gradients to update each parameter with a\nGradient Descent step.\nUnfortunately, gradients often get smaller and smaller as the algorithm progresses\ndown to the lower layers. As a result, the Gradient Descent update leaves the lower\nlayer connection weights virtually unchanged, and training never converges to a good\nsolution. This is called the vanishing gradients problem. In some cases, the opposite\ncan happen: the gradients can grow bigger and bigger, so many layers get insanely\nlarge weight updates and the algorithm diverges. This is the exploding gradients prob\u2010\nlem, which is mostly encountered in recurrent neural networks (see Chapter 14).\nMore generally, deep neural networks suffer from unstable gradients; different layers\nmay learn at widely different speeds.", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 717, "content": "may learn at widely different speeds.\nAlthough this unfortunate behavior has been empirically observed for quite a while\n(it was one of the reasons why deep neural networks were mostly abandoned for a\nlong time), it is only around 2010 that significant progress was made in understand\u2010\ning it. A paper titled \u201cUnderstanding the Difficulty of Training Deep Feedforward\nNeural Networks\u201d by Xavier Glorot and Yoshua Bengio1 found a few suspects, includ\u2010\ning the combination of the popular logistic sigmoid activation function and the\nweight initialization technique that was most popular at the time, namely random ini\u2010\ntialization using a normal distribution with a mean of 0 and a standard deviation of 1.\nIn short, they showed that with this activation function and this initialization scheme,\nthe variance of the outputs of each layer is much greater than the variance of its\ninputs. Going forward in the network, the variance keeps increasing after each layer", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 718, "content": "inputs. Going forward in the network, the variance keeps increasing after each layer\nuntil the activation function saturates at the top layers. This is actually made worse by\nthe fact that the logistic function has a mean of 0.5, not 0 (the hyperbolic tangent\nfunction has a mean of 0 and behaves slightly better than the logistic function in deep\nnetworks).\nLooking at the logistic activation function (see Figure 11-1), you can see that when\ninputs become large (negative or positive), the function saturates at 0 or 1, with a\nderivative extremely close to 0. Thus when backpropagation kicks in, it has virtually\nno gradient to propagate back through the network, and what little gradient exists\nkeeps getting diluted as backpropagation progresses down through the top layers, so\nthere is really nothing left for the lower layers.\n276 \n| \nChapter 11: Training Deep Neural Nets\nDownload from finelybook www.finelybook.com", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 719, "content": "2 Here\u2019s an analogy: if you set a microphone amplifier\u2019s knob too close to zero, people won\u2019t hear your voice, but\nif you set it too close to the max, your voice will be saturated and people won\u2019t understand what you are say\u2010\ning. Now imagine a chain of such amplifiers: they all need to be set properly in order for your voice to come\nout loud and clear at the end of the chain. Your voice has to come out of each amplifier at the same amplitude\nas it came in.\nFigure 11-1. Logistic activation function saturation\nXavier and He Initialization\nIn their paper, Glorot and Bengio propose a way to significantly alleviate this prob\u2010\nlem. We need the signal to flow properly in both directions: in the forward direction\nwhen making predictions, and in the reverse direction when backpropagating gradi\u2010\nents. We don\u2019t want the signal to die out, nor do we want it to explode and saturate.\nFor the signal to flow properly, the authors argue that we need the variance of the", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 720, "content": "For the signal to flow properly, the authors argue that we need the variance of the\noutputs of each layer to be equal to the variance of its inputs,2 and we also need the\ngradients to have equal variance before and after flowing through a layer in the\nreverse direction (please check out the paper if you are interested in the mathematical\ndetails). It is actually not possible to guarantee both unless the layer has an equal\nnumber of input and output connections, but they proposed a good compromise that\nhas proven to work very well in practice: the connection weights must be initialized\nrandomly as described in Equation 11-1, where ninputs and noutputs are the number of\ninput and output connections for the layer whose weights are being initialized (also\ncalled fan-in and fan-out). This initialization strategy is often called Xavier initializa\u2010\ntion (after the author\u2019s first name), or sometimes Glorot initialization.\nVanishing/Exploding Gradients Problems \n| \n277", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 721, "content": "Vanishing/Exploding Gradients Problems \n| \n277\nDownload from finelybook www.finelybook.com", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 722, "content": "3 This simplified strategy was actually already proposed much earlier\u2014for example, in the 1998 book Neural\nNetworks: Tricks of the Trade by Genevieve Orr and Klaus-Robert M\u00fcller (Springer).\n4 Such as \u201cDelving Deep into Rectifiers: Surpassing Human-Level Performance on ImageNet Classification,\u201d K.\nHe et al. (2015).\nEquation 11-1. Xavier initialization (when using the logistic activation function)\nNormal distribution with mean 0 and standard deviation \u03c3 =\n2\nninputs + noutputs\nOr a uniform distribution between \u2010r and +r, with r =\n6\nninputs + noutputs\nWhen the number of input connections is roughly equal to the number of output\nconnections, you get simpler equations (e.g., \u03c3 = 1/ ninputs or r =\n3/ ninputs). We\nused this simplified strategy in Chapter 10.3\nUsing the Xavier initialization strategy can speed up training considerably, and it is\none of the tricks that led to the current success of Deep Learning. Some recent papers4", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 723, "content": "one of the tricks that led to the current success of Deep Learning. Some recent papers4\nhave provided similar strategies for different activation functions, as shown in\nTable 11-1. The initialization strategy for the ReLU activation function (and its var\u2010\niants, including the ELU activation described shortly) is sometimes called He initiali\u2010\nzation (after the last name of its author).\nTable 11-1. Initialization parameters for each type of activation function\nActivation function\nUniform distribution [\u2013r, r] Normal distribution\nLogistic\nr =\n6\nninputs + noutputs\n\u03c3 =\n2\nninputs + noutputs\nHyperbolic tangent\nr = 4\n6\nninputs + noutputs\n\u03c3 = 4\n2\nninputs + noutputs\nReLU (and its variants)\nr =\n2\n6\nninputs + noutputs\n\u03c3 =\n2\n2\nninputs + noutputs\nBy default, the fully_connected() function (introduced in Chapter 10) uses Xavier\ninitialization (with a uniform distribution). You can change this to He initialization\nby using the variance_scaling_initializer() function like this:", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 724, "content": "by using the variance_scaling_initializer() function like this:\nhe_init = tf.contrib.layers.variance_scaling_initializer()\nhidden1 = fully_connected(X, n_hidden1, weights_initializer=he_init, scope=\"h1\")\n278 \n| \nChapter 11: Training Deep Neural Nets\nDownload from finelybook www.finelybook.com", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 725, "content": "5 \u201cEmpirical Evaluation of Rectified Activations in Convolution Network,\u201d B. Xu et al. (2015).\nHe initialization considers only the fan-in, not the average between\nfan-in and fan-out like in Xavier initialization. This is also the\ndefault for the variance_scaling_initializer() function, but\nyou can change this by setting the argument mode=\"FAN_AVG\".\nNonsaturating Activation Functions\nOne of the insights in the 2010 paper by Glorot and Bengio was that the vanishing/\nexploding gradients problems were in part due to a poor choice of activation func\u2010\ntion. Until then most people had assumed that if Mother Nature had chosen to use\nroughly sigmoid activation functions in biological neurons, they must be an excellent\nchoice. But it turns out that other activation functions behave much better in deep\nneural networks, in particular the ReLU activation function, mostly because it does\nnot saturate for positive values (and also because it is quite fast to compute).", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 726, "content": "not saturate for positive values (and also because it is quite fast to compute).\nUnfortunately, the ReLU activation function is not perfect. It suffers from a problem\nknown as the dying ReLUs: during training, some neurons effectively die, meaning\nthey stop outputting anything other than 0. In some cases, you may find that half of\nyour network\u2019s neurons are dead, especially if you used a large learning rate. During\ntraining, if a neuron\u2019s weights get updated such that the weighted sum of the neuron\u2019s\ninputs is negative, it will start outputting 0. When this happen, the neuron is unlikely\nto come back to life since the gradient of the ReLU function is 0 when its input is\nnegative.\nTo solve this problem, you may want to use a variant of the ReLU function, such as\nthe leaky ReLU. This function is defined as LeakyReLU\u03b1(z) = max(\u03b1z, z) (see\nFigure 11-2). The hyperparameter \u03b1 defines how much the function \u201cleaks\u201d: it is the", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 727, "content": "Figure 11-2). The hyperparameter \u03b1 defines how much the function \u201cleaks\u201d: it is the\nslope of the function for z < 0, and is typically set to 0.01. This small slope ensures\nthat leaky ReLUs never die; they can go into a long coma, but they have a chance to\neventually wake up. A recent paper5 compared several variants of the ReLU activation\nfunction and one of its conclusions was that the leaky variants always outperformed\nthe strict ReLU activation function. In fact, setting \u03b1 = 0.2 (huge leak) seemed to\nresult in better performance than \u03b1 = 0.01 (small leak). They also evaluated the\nrandomized leaky ReLU (RReLU), where \u03b1 is picked randomly in a given range during\ntraining, and it is fixed to an average value during testing. It also performed fairly well\nand seemed to act as a regularizer (reducing the risk of overfitting the training set).\nFinally, they also evaluated the parametric leaky ReLU (PReLU), where \u03b1 is authorized", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 728, "content": "Finally, they also evaluated the parametric leaky ReLU (PReLU), where \u03b1 is authorized\nto be learned during training (instead of being a hyperparameter, it becomes a\nparameter that can be modified by backpropagation like any other parameter). This\nVanishing/Exploding Gradients Problems \n| \n279\nDownload from finelybook www.finelybook.com", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 729, "content": "6 \u201cFast and Accurate Deep Network Learning by Exponential Linear Units (ELUs),\u201d D. Clevert, T. Unterthiner,\nS. Hochreiter (2015).\nwas reported to strongly outperform ReLU on large image datasets, but on smaller\ndatasets it runs the risk of overfitting the training set.\nFigure 11-2. Leaky ReLU\nLast but not least, a 2015 paper by Djork-Arn\u00e9 Clevert et al.6 proposed a new activa\u2010\ntion function called the exponential linear unit (ELU) that outperformed all the ReLU\nvariants in their experiments: training time was reduced and the neural network per\u2010\nformed better on the test set. It is represented in Figure 11-3, and Equation 11-2\nshows its definition.\nEquation 11-2. ELU activation function\nELU\u03b1 z = \u03b1 exp z \u22121 if z < 0\nz\nif z \u22650\nFigure 11-3. ELU activation function\n280 \n| \nChapter 11: Training Deep Neural Nets\nDownload from finelybook www.finelybook.com", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 730, "content": "It looks a lot like the ReLU function, with a few major differences:\n\u2022 First it takes on negative values when z < 0, which allows the unit to have an\naverage output closer to 0. This helps alleviate the vanishing gradients problem,\nas discussed earlier. The hyperparameter \u03b1 defines the value that the ELU func\u2010\ntion approaches when z is a large negative number. It is usually set to 1, but you\ncan tweak it like any other hyperparameter if you want.\n\u2022 Second, it has a nonzero gradient for z < 0, which avoids the dying units issue.\n\u2022 Third, the function is smooth everywhere, including around z = 0, which helps\nspeed up Gradient Descent, since it does not bounce as much left and right of z =\n0.\nThe main drawback of the ELU activation function is that it is slower to compute\nthan the ReLU and its variants (due to the use of the exponential function), but dur\u2010\ning training this is compensated by the faster convergence rate. However, at test time", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 731, "content": "ing training this is compensated by the faster convergence rate. However, at test time\nan ELU network will be slower than a ReLU network.\nSo which activation function should you use for the hidden layers\nof your deep neural networks? Although your mileage will vary, in\ngeneral ELU > leaky ReLU (and its variants) > ReLU > tanh > logis\u2010\ntic. If you care a lot about runtime performance, then you may pre\u2010\nfer leaky ReLUs over ELUs. If you don\u2019t want to tweak yet another\nhyperparameter, you may just use the default \u03b1 values suggested\nearlier (0.01 for the leaky ReLU, and 1 for ELU). If you have spare\ntime and computing power, you can use cross-validation to evalu\u2010\nate other activation functions, in particular RReLU if your network\nis overfitting, or PReLU if you have a huge training set.\nTensorFlow offers an elu() function that you can use to build your neural network.\nSimply set the activation_fn argument when calling the fully_connected() func\u2010\ntion, like this:", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 732, "content": "Simply set the activation_fn argument when calling the fully_connected() func\u2010\ntion, like this:\nhidden1 = fully_connected(X, n_hidden1, activation_fn=tf.nn.elu)\nTensorFlow does not have a predefined function for leaky ReLUs, but it is easy\nenough to define:\ndef leaky_relu(z, name=None):\n return tf.maximum(0.01 * z, z, name=name)\nhidden1 = fully_connected(X, n_hidden1, activation_fn=leaky_relu)\nVanishing/Exploding Gradients Problems \n| \n281\nDownload from finelybook www.finelybook.com", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 733, "content": "7 \u201cBatch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift,\u201d S. Ioffe\nand C. Szegedy (2015).\nBatch Normalization\nAlthough using He initialization along with ELU (or any variant of ReLU) can signifi\u2010\ncantly reduce the vanishing/exploding gradients problems at the beginning of train\u2010\ning, it doesn\u2019t guarantee that they won\u2019t come back during training.\nIn a 2015 paper,7 Sergey Ioffe and Christian Szegedy proposed a technique called\nBatch Normalization (BN) to address the vanishing/exploding gradients problems,\nand more generally the problem that the distribution of each layer\u2019s inputs changes\nduring training, as the parameters of the previous layers change (which they call the\nInternal Covariate Shift problem).\nThe technique consists of adding an operation in the model just before the activation\nfunction of each layer, simply zero-centering and normalizing the inputs, then scaling", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 734, "content": "function of each layer, simply zero-centering and normalizing the inputs, then scaling\nand shifting the result using two new parameters per layer (one for scaling, the other\nfor shifting). In other words, this operation lets the model learn the optimal scale and\nmean of the inputs for each layer.\nIn order to zero-center and normalize the inputs, the algorithm needs to estimate the\ninputs\u2019 mean and standard deviation. It does so by evaluating the mean and standard\ndeviation of the inputs over the current mini-batch (hence the name \u201cBatch Normal\u2010\nization\u201d). The whole operation is summarized in Equation 11-3.\nEquation 11-3. Batch Normalization algorithm\n1 .\n\u03bcB = 1\nmB \u2211\ni = 1\nmB\n\ufffdi\n2 .\n\u03c3B\n2 = 1\nmB \u2211\ni = 1\nmB\n\ufffdi \u2212\u03bcB\n2\n3 .\n\ufffdi =\n\ufffdi \u2212\u03bcB\n\u03c3B\n2 + \ufffd\n4 .\n\ufffdi = \u03b3\ufffdi + \u03b2\n\u2022 \u03bcB is the empirical mean, evaluated over the whole mini-batch B.\n\u2022 \u03c3B is the empirical standard deviation, also evaluated over the whole mini-batch.\n\u2022 mB is the number of instances in the mini-batch.\n282 \n|", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 735, "content": "\u2022 mB is the number of instances in the mini-batch.\n282 \n| \nChapter 11: Training Deep Neural Nets\nDownload from finelybook www.finelybook.com", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 736, "content": "\u2022 \ufffd(i) is the zero-centered and normalized input.\n\u2022 \u03b3 is the scaling parameter for the layer.\n\u2022 \u03b2 is the shifting parameter (offset) for the layer.\n\u2022 \u03f5 is a tiny number to avoid division by zero (typically 10\u20133). This is called a\nsmoothing term.\n\u2022 z(i) is the output of the BN operation: it is a scaled and shifted version of the\ninputs.\nAt test time, there is no mini-batch to compute the empirical mean and standard\ndeviation, so instead you simply use the whole training set\u2019s mean and standard devi\u2010\nation. These are typically efficiently computed during training using a moving aver\u2010\nage. So, in total, four parameters are learned for each batch-normalized layer: \u03b3\n(scale), \u03b2 (offset), \u03bc (mean), and \u03c3 (standard deviation).\nThe authors demonstrated that this technique considerably improved all the deep\nneural networks they experimented with. The vanishing gradients problem was\nstrongly reduced, to the point that they could use saturating activation functions such", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 737, "content": "strongly reduced, to the point that they could use saturating activation functions such\nas the tanh and even the logistic activation function. The networks were also much\nless sensitive to the weight initialization. They were able to use much larger learning\nrates, significantly speeding up the learning process. Specifically, they note that\n\u201cApplied to a state-of-the-art image classification model, Batch Normalization ach\u2010\nieves the same accuracy with 14 times fewer training steps, and beats the original\nmodel by a significant margin. [\u2026] Using an ensemble of batch-normalized net\u2010\nworks, we improve upon the best published result on ImageNet classification: reach\u2010\ning 4.9% top-5 validation error (and 4.8% test error), exceeding the accuracy of\nhuman raters.\u201d Finally, like a gift that keeps on giving, Batch Normalization also acts\nlike a regularizer, reducing the need for other regularization techniques (such as\ndropout, described later in the chapter).", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 738, "content": "dropout, described later in the chapter).\nBatch Normalization does, however, add some complexity to the model (although it\nremoves the need for normalizing the input data since the first hidden layer will take\ncare of that, provided it is batch-normalized). Moreover, there is a runtime penalty:\nthe neural network makes slower predictions due to the extra computations required\nat each layer. So if you need predictions to be lightning-fast, you may want to check\nhow well plain ELU + He initialization perform before playing with Batch Normaliza\u2010\ntion.\nYou may find that training is rather slow at first while Gradient\nDescent is searching for the optimal scales and offsets for each\nlayer, but it accelerates once it has found reasonably good values.\nVanishing/Exploding Gradients Problems \n| \n283\nDownload from finelybook www.finelybook.com", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 739, "content": "Implementing Batch Normalization with TensorFlow\nTensorFlow provides a batch_normalization() function that simply centers and\nnormalizes the inputs, but you must compute the mean and standard deviation your\u2010\nself (based on the mini-batch data during training or on the full dataset during test\u2010\ning, as just discussed) and pass them as parameters to this function, and you must\nalso handle the creation of the scaling and offset parameters (and pass them to this\nfunction). It is doable, but not the most convenient approach. Instead, you should use\nthe batch_norm() function, which handles all this for you. You can either call it\ndirectly or tell the fully_connected() function to use it, such as in the following\ncode:\nimport tensorflow as tf\nfrom tensorflow.contrib.layers import batch_norm\nn_inputs = 28 * 28\nn_hidden1 = 300\nn_hidden2 = 100\nn_outputs = 10\nX = tf.placeholder(tf.float32, shape=(None, n_inputs), name=\"X\")\nis_training = tf.placeholder(tf.bool, shape=(), name='is_training')", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 740, "content": "is_training = tf.placeholder(tf.bool, shape=(), name='is_training')\nbn_params = {\n 'is_training': is_training,\n 'decay': 0.99,\n 'updates_collections': None\n}\nhidden1 = fully_connected(X, n_hidden1, scope=\"hidden1\",\n normalizer_fn=batch_norm, normalizer_params=bn_params)\nhidden2 = fully_connected(hidden1, n_hidden2, scope=\"hidden2\",\n normalizer_fn=batch_norm, normalizer_params=bn_params)\nlogits = fully_connected(hidden2, n_outputs, activation_fn=None,scope=\"outputs\",\n normalizer_fn=batch_norm, normalizer_params=bn_params)\nLet\u2019s walk through this code. The first lines are fairly self-explanatory, until we define\nthe is_training placeholder, which will either be True or False. This will be used to\ntell the batch_norm() function whether it should use the current mini-batch\u2019s mean\nand standard deviation (during training) or the running averages that it keeps track\nof (during testing).", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 741, "content": "of (during testing).\nNext we define bn_params, which is a dictionary that defines the parameters that will\nbe passed to the batch_norm() function, including is_training of course. The algo\u2010\nrithm uses exponential decay to compute the running averages, which is why it\nrequires the decay parameters. Given a new value v, the running average v is updated\nthrough the equation v\nv \u00d7 decay + v \u00d7 1 \u2212decay . A good decay value is typically\nclose to 1\u2014for example, 0.9, 0.99, or 0.999 (you want more 9s for larger datasets and\n284 \n| \nChapter 11: Training Deep Neural Nets\nDownload from finelybook www.finelybook.com", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 742, "content": "smaller mini-batches). Finally, updates_collections should be set to None if you\nwant the batch_norm() function to update the running averages right before it per\u2010\nforms batch normalization during training (i.e., when is_training=True). If you\ndon\u2019t set this parameter, by default TensorFlow will just add the operations that\nupdate the running averages to a collection of operations that you must run yourself.\nLastly, we create the layers by calling the fully_connected() function, just like we\ndid in Chapter 10, but this time we tell it to use the batch_norm() function (with the\nparameters nb_params) to normalize the inputs right before calling the activation\nfunction.\nNote that by default batch_norm() only centers, normalizes, and shifts the inputs; it\ndoes not scale them (i.e., \u03b3 is fixed to 1). This makes sense for layers with no activa\u2010\ntion function or with the ReLU activation function, since the next layer\u2019s weights can", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 743, "content": "tion function or with the ReLU activation function, since the next layer\u2019s weights can\ntake care of scaling, but for any other activation function, you should add \"scale\":\nTrue to bn_params.\nYou may have noticed that defining the preceding three layers was fairly repetitive\nsince several parameters were identical. To avoid repeating the same parameters over\nand over again, you can create an argument scope using the arg_scope() function:\nthe first parameter is a list of functions, and the other parameters will be passed to\nthese functions automatically. The last three lines of the preceding code can be modi\u2010\nfied like so:\n[...]\nwith tf.contrib.framework.arg_scope(\n [fully_connected],\n normalizer_fn=batch_norm,\n normalizer_params=bn_params):\n hidden1 = fully_connected(X, n_hidden1, scope=\"hidden1\")\n hidden2 = fully_connected(hidden1, n_hidden2, scope=\"hidden2\")\n logits = fully_connected(hidden2, n_outputs, scope=\"outputs\",", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 744, "content": "logits = fully_connected(hidden2, n_outputs, scope=\"outputs\",\n activation_fn=None)\nIt may not look much better than before in this small example, but if you have 10 lay\u2010\ners and want to set the activation function, the initializers, the normalizers, the regu\u2010\nlarizers, and so on, it will make your code much more readable.\nThe rest of the construction phase is the same as in Chapter 10: define the cost func\u2010\ntion, create an optimizer, tell it to minimize the cost function, define the evaluation\noperations, create a Saver, and so on.\nThe execution phase is also pretty much the same, with one exception. Whenever you\nrun an operation that depends on the batch_norm layer, you need to set the is_train\ning placeholder to True or False:\nVanishing/Exploding Gradients Problems \n| \n285\nDownload from finelybook www.finelybook.com", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 745, "content": "8 \u201cOn the difficulty of training recurrent neural networks,\u201d R. Pascanu et al. (2013).\nwith tf.Session() as sess:\n sess.run(init)\n for epoch in range(n_epochs):\n [...]\n for X_batch, y_batch in zip(X_batches, y_batches):\n sess.run(training_op,\n feed_dict={is_training: True, X: X_batch, y: y_batch})\n accuracy_score = accuracy.eval(\n feed_dict={is_training: False, X: X_test_scaled, y: y_test}))\n print(accuracy_score)\nThat\u2019s all! In this tiny example with just two layers, it\u2019s unlikely that Batch Normaliza\u2010\ntion will have a very positive impact, but for deeper networks it can make a tremen\u2010\ndous difference.\nGradient Clipping\nA popular technique to lessen the exploding gradients problem is to simply clip the\ngradients during backpropagation so that they never exceed some threshold (this is\nmostly useful for recurrent neural networks; see Chapter 14). This is called Gradient", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 746, "content": "mostly useful for recurrent neural networks; see Chapter 14). This is called Gradient\nClipping.8 In general people now prefer Batch Normalization, but it\u2019s still useful to\nknow about Gradient Clipping and how to implement it.\nIn TensorFlow, the optimizer\u2019s minimize() function takes care of both computing the\ngradients and applying them, so you must instead call the optimizer\u2019s compute_gradi\nents() method first, then create an operation to clip the gradients using the\nclip_by_value() function, and finally create an operation to apply the clipped gradi\u2010\nents using the optimizer\u2019s apply_gradients() method:\nthreshold = 1.0\noptimizer = tf.train.GradientDescentOptimizer(learning_rate)\ngrads_and_vars = optimizer.compute_gradients(loss)\ncapped_gvs = [(tf.clip_by_value(grad, -threshold, threshold), var)\n for grad, var in grads_and_vars]\ntraining_op = optimizer.apply_gradients(capped_gvs)\nYou would then run this training_op at every training step, as usual. It will compute", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 747, "content": "You would then run this training_op at every training step, as usual. It will compute\nthe gradients, clip them between \u20131.0 and 1.0, and apply them. The threshold is a\nhyperparameter you can tune.\nReusing Pretrained Layers\nIt is generally not a good idea to train a very large DNN from scratch: instead, you\nshould always try to find an existing neural network that accomplishes a similar task\n286 \n| \nChapter 11: Training Deep Neural Nets\nDownload from finelybook www.finelybook.com", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 748, "content": "to the one you are trying to tackle, then just reuse the lower layers of this network:\nthis is called transfer learning. It will not only speed up training considerably, but will\nalso require much less training data.\nFor example, suppose that you have access to a DNN that was trained to classify pic\u2010\ntures into 100 different categories, including animals, plants, vehicles, and everyday\nobjects. You now want to train a DNN to classify specific types of vehicles. These\ntasks are very similar, so you should try to reuse parts of the first network (see\nFigure 11-4).\nFigure 11-4. Reusing pretrained layers\nIf the input pictures of your new task don\u2019t have the same size as\nthe ones used in the original task, you will have to add a prepro\u2010\ncessing step to resize them to the size expected by the original\nmodel. More generally, transfer learning will work only well if the\ninputs have similar low-level features.\nReusing a TensorFlow Model", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 749, "content": "inputs have similar low-level features.\nReusing a TensorFlow Model\nIf the original model was trained using TensorFlow, you can simply restore it and\ntrain it on the new task:\n[...] # construct the original model\nwith tf.Session() as sess:\n saver.restore(sess, \"./my_original_model.ckpt\")\n [...] # Train it on your new task\nReusing Pretrained Layers \n| \n287\nDownload from finelybook www.finelybook.com", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 750, "content": "However, in general you will want to reuse only part of the original model (as we will\ndiscuss in a moment). A simple solution is to configure the Saver to restore only a\nsubset of the variables from the original model. For example, the following code\nrestores only hidden layers 1, 2, and 3:\n[...] # build new model with the same definition as before for hidden layers 1-3\ninit = tf.global_variables_initializer()\nreuse_vars = tf.get_collection(tf.GraphKeys.TRAINABLE_VARIABLES,\n scope=\"hidden[123]\")\nreuse_vars_dict = dict([(var.name, var.name) for var in reuse_vars])\noriginal_saver = tf.Saver(reuse_vars_dict) # saver to restore the original model\nnew_saver = tf.Saver() # saver to save the new model\nwith tf.Session() as sess:\n sess.run(init)\n original_saver.restore(\"./my_original_model.ckpt\") # restore layers 1 to 3\n [...] # train the new model\n new_saver.save(\"./my_new_model.ckpt\") # save the whole model", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 751, "content": "[...] # train the new model\n new_saver.save(\"./my_new_model.ckpt\") # save the whole model\nFirst we build the new model, making sure to copy the original model\u2019s hidden layers\n1 to 3. We also create a node to initialize all variables. Then we get the list of all vari\u2010\nables that were just created with \"trainable=True\" (which is the default), and we\nkeep only the ones whose scope matches the regular expression \"hidden[123]\" (i.e.,\nwe get all trainable variables in hidden layers 1 to 3). Next we create a dictionary\nmapping the name of each variable in the original model to its name in the new\nmodel (generally you want to keep the exact same names). Then we create a Saver\nthat will restore only these variables, and we create another Saver to save the entire\nnew model, not just layers 1 to 3. We then start a session and initialize all variables in\nthe model, then restore the variable values from the original model\u2019s layers 1 to 3.", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 752, "content": "the model, then restore the variable values from the original model\u2019s layers 1 to 3.\nFinally, we train the model on the new task and save it.\nThe more similar the tasks are, the more layers you want to reuse\n(starting with the lower layers). For very similar tasks, you can try\nkeeping all the hidden layers and just replace the output layer.\nReusing Models from Other Frameworks\nIf the model was trained using another framework, you will need to load the weights\nmanually (e.g., using Theano code if it was trained with Theano), then assign them to\nthe appropriate variables. This can be quite tedious. For example, the following code\nshows how you would copy the weight and biases from the first hidden layer of a \nmodel trained using another framework:\n288 \n| \nChapter 11: Training Deep Neural Nets\nDownload from finelybook www.finelybook.com", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 753, "content": "original_w = [...] # Load the weights from the other framework\noriginal_b = [...] # Load the biases from the other framework\nX = tf.placeholder(tf.float32, shape=(None, n_inputs), name=\"X\")\nhidden1 = fully_connected(X, n_hidden1, scope=\"hidden1\")\n[...] # # Build the rest of the model\n# Get a handle on the variables created by fully_connected()\nwith tf.variable_scope(\"\", default_name=\"\", reuse=True): # root scope\n hidden1_weights = tf.get_variable(\"hidden1/weights\")\n hidden1_biases = tf.get_variable(\"hidden1/biases\")\n# Create nodes to assign arbitrary values to the weights and biases\noriginal_weights = tf.placeholder(tf.float32, shape=(n_inputs, n_hidden1))\noriginal_biases = tf.placeholder(tf.float32, shape=(n_hidden1))\nassign_hidden1_weights = tf.assign(hidden1_weights, original_weights)\nassign_hidden1_biases = tf.assign(hidden1_biases, original_biases)\ninit = tf.global_variables_initializer()\nwith tf.Session() as sess:\n sess.run(init)", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 754, "content": "init = tf.global_variables_initializer()\nwith tf.Session() as sess:\n sess.run(init)\n sess.run(assign_hidden1_weights, feed_dict={original_weights: original_w})\n sess.run(assign_hidden1_biases, feed_dict={original_biases: original_b})\n [...] # Train the model on your new task\nFreezing the Lower Layers\nIt is likely that the lower layers of the first DNN have learned to detect low-level fea\u2010\ntures in pictures that will be useful across both image classification tasks, so you can\njust reuse these layers as they are. It is generally a good idea to \u201cfreeze\u201d their weights\nwhen training the new DNN: if the lower-layer weights are fixed, then the higher-\nlayer weights will be easier to train (because they won\u2019t have to learn a moving target).\nTo freeze the lower layers during training, the simplest solution is to give the opti\u2010\nmizer the list of variables to train, excluding the variables from the lower layers:\ntrain_vars = tf.get_collection(tf.GraphKeys.TRAINABLE_VARIABLES,", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 755, "content": "train_vars = tf.get_collection(tf.GraphKeys.TRAINABLE_VARIABLES,\n scope=\"hidden[34]|outputs\")\ntraining_op = optimizer.minimize(loss, var_list=train_vars)\nThe first line gets the list of all trainable variables in hidden layers 3 and 4 and in the\noutput layer. This leaves out the variables in the hidden layers 1 and 2. Next we pro\u2010\nvide this restricted list of trainable variables to the optimizer\u2019s minimize() function.\nTa-da! Layers 1 and 2 are now frozen: they will not budge during training (these are\noften called frozen layers).\nReusing Pretrained Layers \n| \n289\nDownload from finelybook www.finelybook.com", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 756, "content": "Caching the Frozen Layers\nSince the frozen layers won\u2019t change, it is possible to cache the output of the topmost\nfrozen layer for each training instance. Since training goes through the whole dataset\nmany times, this will give you a huge speed boost as you will only need to go through\nthe frozen layers once per training instance (instead of once per epoch). For example,\nyou could first run the whole training set through the lower layers (assuming you\nhave enough RAM):\nhidden2_outputs = sess.run(hidden2, feed_dict={X: X_train})\nThen during training, instead of building batches of training instances, you would\nbuild batches of outputs from hidden layer 2 and feed them to the training operation:\nimport numpy as np\nn_epochs = 100\nn_batches = 500\nfor epoch in range(n_epochs):\n shuffled_idx = rnd.permutation(len(hidden2_outputs))\n hidden2_batches = np.array_split(hidden2_outputs[shuffled_idx], n_batches)\n y_batches = np.array_split(y_train[shuffled_idx], n_batches)", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 757, "content": "y_batches = np.array_split(y_train[shuffled_idx], n_batches)\n for hidden2_batch, y_batch in zip(hidden2_batches, y_batches):\n sess.run(training_op, feed_dict={hidden2: hidden2_batch, y: y_batch})\nThe last line runs the training operation defined earlier (which freezes layers 1 and 2),\nand feeds it a batch of outputs from the second hidden layer (as well as the targets for\nthat batch). Since we give TensorFlow the output of hidden layer 2, it does not try to\nevaluate it (or any node it depends on).\nTweaking, Dropping, or Replacing the Upper Layers\nThe output layer of the original model should usually be replaced since it is most\nlikely not useful at all for the new task, and it may not even have the right number of\noutputs for the new task.\nSimilarly, the upper hidden layers of the original model are less likely to be as useful\nas the lower layers, since the high-level features that are most useful for the new task", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 758, "content": "as the lower layers, since the high-level features that are most useful for the new task\nmay differ significantly from the ones that were most useful for the original task. You\nwant to find the right number of layers to reuse.\nTry freezing all the copied layers first, then train your model and see how it performs.\nThen try unfreezing one or two of the top hidden layers to let backpropagation tweak\nthem and see if performance improves. The more training data you have, the more\nlayers you can unfreeze.\nIf you still cannot get good performance, and you have little training data, try drop\u2010\nping the top hidden layer(s) and freeze all remaining hidden layers again. You can\n290 \n| \nChapter 11: Training Deep Neural Nets\nDownload from finelybook www.finelybook.com", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 759, "content": "iterate until you find the right number of layers to reuse. If you have plenty of train\u2010\ning data, you may try replacing the top hidden layers instead of dropping them, and\neven add more hidden layers.\nModel Zoos\nWhere can you find a neural network trained for a task similar to the one you want to\ntackle? The first place to look is obviously in your own catalog of models. This is one\ngood reason to save all your models and organize them so you can retrieve them later\neasily. Another option is to search in a model zoo. Many people train Machine Learn\u2010\ning models for various tasks and kindly release their pretrained models to the public.\nTensorFlow has its own model zoo available at https://github.com/tensorflow/models.\nIn particular, it contains most of the state-of-the-art image classification nets such as\nVGG, Inception, and ResNet (see Chapter 13, and check out the models/slim direc\u2010\ntory), including the code, the pretrained models, and tools to download popular\nimage datasets.", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 760, "content": "tory), including the code, the pretrained models, and tools to download popular\nimage datasets.\nAnother popular model zoo is Caffe\u2019s Model Zoo. It also contains many computer\nvision models (e.g., LeNet, AlexNet, ZFNet, GoogLeNet, VGGNet, inception) trained\non various datasets (e.g., ImageNet, Places Database, CIFAR10, etc.). Saumitro Das\u2010\ngupta wrote a converter, which is available at https://github.com/ethereon/caffe-\ntensorflow.\nUnsupervised Pretraining\nSuppose you want to tackle a complex task for which you don\u2019t have much labeled\ntraining data, but unfortunately you cannot find a model trained on a similar task.\nDon\u2019t lose all hope! First, you should of course try to gather more labeled training\ndata, but if this is too hard or too expensive, you may still be able to perform unsuper\u2010\nvised pretraining (see Figure 11-5). That is, if you have plenty of unlabeled training\ndata, you can try to train the layers one by one, starting with the lowest layer and then", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 761, "content": "data, you can try to train the layers one by one, starting with the lowest layer and then\ngoing up, using an unsupervised feature detector algorithm such as Restricted Boltz\u2010\nmann Machines (RBMs; see Appendix E) or autoencoders (see Chapter 15). Each\nlayer is trained on the output of the previously trained layers (all layers except the one\nbeing trained are frozen). Once all layers have been trained this way, you can fine-\ntune the network using supervised learning (i.e., with backpropagation).\nThis is a rather long and tedious process, but it often works well; in fact, it is this\ntechnique that Geoffrey Hinton and his team used in 2006 and which led to the\nrevival of neural networks and the success of Deep Learning. Until 2010, unsuper\u2010\nvised pretraining (typically using RBMs) was the norm for deep nets, and it was only\nafter the vanishing gradients problem was alleviated that it became much more com\u2010\nmon to train DNNs purely using backpropagation. However, unsupervised pretrain\u2010", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 762, "content": "mon to train DNNs purely using backpropagation. However, unsupervised pretrain\u2010\ning (today typically using autoencoders rather than RBMs) is still a good option when\nReusing Pretrained Layers \n| \n291\nDownload from finelybook www.finelybook.com", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 763, "content": "9 Another option is to come up with a supervised task for which you can easily gather a lot of labeled training\ndata, then use transfer learning, as explained earlier. For example, if you want to train a model to identify your\nfriends in pictures, you could download millions of faces on the internet and train a classifier to detect\nwhether two faces are identical or not, then use this classifier to compare a new picture with each picture of\nyour friends.\nyou have a complex task to solve, no similar model you can reuse, and little labeled\ntraining data but plenty of unlabeled training data.9\nFigure 11-5. Unsupervised pretraining\nPretraining on an Auxiliary Task\nOne last option is to train a first neural network on an auxiliary task for which you\ncan easily obtain or generate labeled training data, then reuse the lower layers of that\nnetwork for your actual task. The first neural network\u2019s lower layers will learn feature", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 764, "content": "network for your actual task. The first neural network\u2019s lower layers will learn feature\ndetectors that will likely be reusable by the second neural network.\nFor example, if you want to build a system to recognize faces, you may only have a\nfew pictures of each individual\u2014clearly not enough to train a good classifier. Gather\u2010\ning hundreds of pictures of each person would not be practical. However, you could\ngather a lot of pictures of random people on the internet and train a first neural net\u2010\nwork to detect whether or not two different pictures feature the same person. Such a\n292 \n| \nChapter 11: Training Deep Neural Nets\nDownload from finelybook www.finelybook.com", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 765, "content": "10 At least for now: research is moving fast, especially in the field of optimization. Be sure to take a look at the\nlatest and greatest optimizers every time a new version of TensorFlow is released.\nnetwork would learn good feature detectors for faces, so reusing its lower layers\nwould allow you to train a good face classifier using little training data.\nIt is often rather cheap to gather unlabeled training examples, but quite expensive to\nlabel them. In this situation, a common technique is to label all your training exam\u2010\nples as \u201cgood,\u201d then generate many new training instances by corrupting the good\nones, and label these corrupted instances as \u201cbad.\u201d Then you can train a first neural\nnetwork to classify instances as good or bad. For example, you could download mil\u2010\nlions of sentences, label them as \u201cgood,\u201d then randomly change a word in each sen\u2010\ntence and label the resulting sentences as \u201cbad.\u201d If a neural network can tell that \u201cThe", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 766, "content": "tence and label the resulting sentences as \u201cbad.\u201d If a neural network can tell that \u201cThe\ndog sleeps\u201d is a good sentence but \u201cThe dog they\u201d is bad, it probably knows quite a lot\nabout language. Reusing its lower layers will likely help in many language processing\ntasks.\nAnother approach is to train a first network to output a score for each training\ninstance, and use a cost function that ensures that a good instance\u2019s score is greater\nthan a bad instance\u2019s score by at least some margin. This is called max margin learn\u2010\ning.\nFaster Optimizers\nTraining a very large deep neural network can be painfully slow. So far we have seen\nfour ways to speed up training (and reach a better solution): applying a good initiali\u2010\nzation strategy for the connection weights, using a good activation function, using\nBatch Normalization, and reusing parts of a pretrained network. Another huge speed\nboost comes from using a faster optimizer than the regular Gradient Descent opti\u2010", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 767, "content": "boost comes from using a faster optimizer than the regular Gradient Descent opti\u2010\nmizer. In this section we will present the most popular ones: Momentum optimiza\u2010\ntion, Nesterov Accelerated Gradient, AdaGrad, RMSProp, and finally Adam\noptimization.\nSpoiler alert: the conclusion of this section is that you should almost always use \nAdam optimization,10 so if you don\u2019t care about how it works, simply replace your\nGradientDescentOptimizer with an AdamOptimizer and skip to the next section!\nWith just this small change, training will typically be several times faster. However,\nAdam optimization does have three hyperparameters that you can tune (plus the\nlearning rate); the default values usually work fine, but if you ever need to tweak them\nit may be helpful to know what they do. Adam optimization combines several ideas\nfrom other optimization algorithms, so it is useful to look at these algorithms first.\nFaster Optimizers \n| \n293\nDownload from finelybook www.finelybook.com", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 768, "content": "11 \u201cSome methods of speeding up the convergence of iteration methods,\u201d B. Polyak (1964).\nMomentum optimization\nImagine a bowling ball rolling down a gentle slope on a smooth surface: it will start\nout slowly, but it will quickly pick up momentum until it eventually reaches terminal\nvelocity (if there is some friction or air resistance). This is the very simple idea behind\nMomentum optimization, proposed by Boris Polyak in 1964.11 In contrast, regular\nGradient Descent will simply take small regular steps down the slope, so it will take\nmuch more time to reach the bottom.\nRecall that Gradient Descent simply updates the weights \u03b8 by directly subtracting the\ngradient of the cost function J(\u03b8) with regards to the weights (\u2207\u03b8J(\u03b8)) multiplied by\nthe learning rate \u03b7. The equation is: \u03b8 \u2190 \u03b8 \u2013 \u03b7\u2207\u03b8J(\u03b8). It does not care about what the\nearlier gradients were. If the local gradient is tiny, it goes very slowly.\nMomentum optimization cares a great deal about what previous gradients were: at", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 769, "content": "Momentum optimization cares a great deal about what previous gradients were: at\neach iteration, it adds the local gradient to the momentum vector m (multiplied by the\nlearning rate \u03b7), and it updates the weights by simply subtracting this momentum\nvector (see Equation 11-4). In other words, the gradient is used as an acceleration, not\nas a speed. To simulate some sort of friction mechanism and prevent the momentum\nfrom growing too large, the algorithm introduces a new hyperparameter \u03b2, simply\ncalled the momentum, which must be set between 0 (high friction) and 1 (no friction).\nA typical momentum value is 0.9.\nEquation 11-4. Momentum algorithm\n1 .\n\ufffd\n\u03b2\ufffd+ \u03b7\u2207\u03b8J \u03b8\n2 .\n\u03b8\n\u03b8 \u2212\ufffd\nYou can easily verify that if the gradient remains constant, the terminal velocity (i.e.,\nthe maximum size of the weight updates) is equal to that gradient multiplied by the\nlearning rate \u03b7 multiplied by \n1\n1 \u2212\u03b2. For example, if \u03b2 = 0.9, then the terminal velocity", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 770, "content": "learning rate \u03b7 multiplied by \n1\n1 \u2212\u03b2. For example, if \u03b2 = 0.9, then the terminal velocity\nis equal to 10 times the gradient times the learning rate, so Momentum optimization\nends up going 10 times faster than Gradient Descent! This allows Momentum opti\u2010\nmization to escape from plateaus much faster than Gradient Descent. In particular,\nwe saw in Chapter 4 that when the inputs have very different scales the cost function\nwill look like an elongated bowl (see Figure 4-7). Gradient Descent goes down the\nsteep slope quite fast, but then it takes a very long time to go down the valley. In con\u2010\ntrast, Momentum optimization will roll down the bottom of the valley faster and\nfaster until it reaches the bottom (the optimum). In deep neural networks that don\u2019t\nuse Batch Normalization, the upper layers will often end up having inputs with very\n294 \n| \nChapter 11: Training Deep Neural Nets\nDownload from finelybook www.finelybook.com", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 771, "content": "12 \u201cA Method for Unconstrained Convex Minimization Problem with the Rate of Convergence O(1/k2),\u201d Yurii\nNesterov (1983).\ndifferent scales, so using Momentum optimization helps a lot. It can also help roll\npast local optima.\nDue to the momentum, the optimizer may overshoot a bit, then\ncome back, overshoot again, and oscillate like this many times\nbefore stabilizing at the minimum. This is one of the reasons why it\nis good to have a bit of friction in the system: it gets rid of these\noscillations and thus speeds up convergence.\nImplementing Momentum optimization in TensorFlow is a no-brainer: just replace\nthe GradientDescentOptimizer with the MomentumOptimizer, then lie back and\nprofit!\noptimizer = tf.train.MomentumOptimizer(learning_rate=learning_rate,\n momentum=0.9)\nThe one drawback of Momentum optimization is that it adds yet another hyperpara\u2010\nmeter to tune. However, the momentum value of 0.9 usually works well in practice", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 772, "content": "meter to tune. However, the momentum value of 0.9 usually works well in practice\nand almost always goes faster than Gradient Descent.\nNesterov Accelerated Gradient\nOne small variant to Momentum optimization, proposed by Yurii Nesterov in 1983,12\nis almost always faster than vanilla Momentum optimization. The idea of Nesterov\nMomentum optimization, or Nesterov Accelerated Gradient (NAG), is to measure the\ngradient of the cost function not at the local position but slightly ahead in the direc\u2010\ntion of the momentum (see Equation 11-5). The only difference from vanilla\nMomentum optimization is that the gradient is measured at \u03b8 + \u03b2m rather than at \u03b8.\nEquation 11-5. Nesterov Accelerated Gradient algorithm\n1 .\n\ufffd\n\u03b2\ufffd+ \u03b7\u2207\u03b8J \u03b8 + \u03b2\ufffd\n2 .\n\u03b8\n\u03b8 \u2212\ufffd\nThis small tweak works because in general the momentum vector will be pointing in\nthe right direction (i.e., toward the optimum), so it will be slightly more accurate to\nuse the gradient measured a bit farther in that direction rather than using the gradi\u2010", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 773, "content": "use the gradient measured a bit farther in that direction rather than using the gradi\u2010\nent at the original position, as you can see in Figure 11-6 (where \u22071 represents the\ngradient of the cost function measured at the starting point \u03b8, and \u22072 represents the\ngradient at the point located at \u03b8 + \u03b2m). As you can see, the Nesterov update ends up\nFaster Optimizers \n| \n295\nDownload from finelybook www.finelybook.com", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 774, "content": "slightly closer to the optimum. After a while, these small improvements add up and\nNAG ends up being significantly faster than regular Momentum optimization. More\u2010\nover, note that when the momentum pushes the weights across a valley, \u22071 continues\nto push further across the valley, while \u22072 pushes back toward the bottom of the val\u2010\nley. This helps reduce oscillations and thus converges faster.\nFigure 11-6. Regular versus Nesterov Momentum optimization\nNAG will almost always speed up training compared to regular Momentum optimi\u2010\nzation. To use it, simply set use_nesterov=True when creating the MomentumOptim\nizer:\noptimizer = tf.train.MomentumOptimizer(learning_rate=learning_rate,\n momentum=0.9, use_nesterov=True)\nAdaGrad\nConsider the elongated bowl problem again: Gradient Descent starts by quickly going\ndown the steepest slope, then slowly goes down the bottom of the valley. It would be", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 775, "content": "down the steepest slope, then slowly goes down the bottom of the valley. It would be\nnice if the algorithm could detect this early on and correct its direction to point a bit\nmore toward the global optimum.\n296 \n| \nChapter 11: Training Deep Neural Nets\nDownload from finelybook www.finelybook.com", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 776, "content": "13 \u201cAdaptive Subgradient Methods for Online Learning and Stochastic Optimization,\u201d J. Duchi et al. (2011).\nThe AdaGrad algorithm13 achieves this by scaling down the gradient vector along the\nsteepest dimensions (see Equation 11-6):\nEquation 11-6. AdaGrad algorithm\n1 .\n\ufffd\n\ufffd+ \u2207\u03b8J \u03b8 \u2297\u2207\u03b8J \u03b8\n2 .\n\u03b8\n\u03b8 \u2212\u03b7 \u2207\u03b8J \u03b8 \u2298\n\ufffd+ \ufffd\nThe first step accumulates the square of the gradients into the vector s (the \u2297 symbol\nrepresents the element-wise multiplication). This vectorized form is equivalent to\ncomputing si \u2190 si + (\u2202 / \u2202 \u03b8i J(\u03b8))2 for each element si of the vector s; in other words,\neach si accumulates the squares of the partial derivative of the cost function with\nregards to parameter \u03b8i. If the cost function is steep along the ith dimension, then si\nwill get larger and larger at each iteration.\nThe second step is almost identical to Gradient Descent, but with one big difference:\nthe gradient vector is scaled down by a factor of \ufffd+ \ufffd (the \u2298 symbol represents the", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 777, "content": "the gradient vector is scaled down by a factor of \ufffd+ \ufffd (the \u2298 symbol represents the\nelement-wise division, and \u03f5 is a smoothing term to avoid division by zero, typically\nset \nto \n10\u201310). \nThis \nvectorized \nform \nis \nequivalent \nto \ncomputing\n\u03b8i\n\u03b8i \u2212\u03b7 \u2202/\u2202\u03b8i J \u03b8 / si + \ufffd for all parameters \u03b8i (simultaneously).\nIn short, this algorithm decays the learning rate, but it does so faster for steep dimen\u2010\nsions than for dimensions with gentler slopes. This is called an adaptive learning rate. \nIt helps point the resulting updates more directly toward the global optimum (see\nFigure 11-7). One additional benefit is that it requires much less tuning of the learn\u2010\ning rate hyperparameter \u03b7.\nFigure 11-7. AdaGrad versus Gradient Descent\nFaster Optimizers \n| \n297\nDownload from finelybook www.finelybook.com", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 778, "content": "14 This algorithm was created by Tijmen Tieleman and Geoffrey Hinton in 2012, and presented by Geoffrey\nHinton in his Coursera class on neural networks (slides: http://goo.gl/RsQeis; video: https://goo.gl/XUbIyJ).\nAmusingly, since the authors have not written a paper to describe it, researchers often cite \u201cslide 29 in lecture\n6\u201d in their papers.\n15 \u201cAdam: A Method for Stochastic Optimization,\u201d D. Kingma, J. Ba (2015).\nAdaGrad often performs well for simple quadratic problems, but unfortunately it\noften stops too early when training neural networks. The learning rate gets scaled\ndown so much that the algorithm ends up stopping entirely before reaching the\nglobal optimum. So even though TensorFlow has an AdagradOptimizer, you should\nnot use it to train deep neural networks (it may be efficient for simpler tasks such as\nLinear Regression, though).\nRMSProp\nAlthough AdaGrad slows down a bit too fast and ends up never converging to the", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 779, "content": "RMSProp\nAlthough AdaGrad slows down a bit too fast and ends up never converging to the\nglobal optimum, the RMSProp algorithm14 fixes this by accumulating only the gradi\u2010\nents from the most recent iterations (as opposed to all the gradients since the begin\u2010\nning of training). It does so by using exponential decay in the first step (see Equation\n11-7).\nEquation 11-7. RMSProp algorithm\n1 .\n\ufffd\n\u03b2\ufffd+ 1 \u2212\u03b2 \u2207\u03b8J \u03b8 \u2297\u2207\u03b8J \u03b8\n2 .\n\u03b8\n\u03b8 \u2212\u03b7 \u2207\u03b8J \u03b8 \u2298\n\ufffd+ \ufffd\nThe decay rate \u03b2 is typically set to 0.9. Yes, it is once again a new hyperparameter, but\nthis default value often works well, so you may not need to tune it at all.\nAs you might expect, TensorFlow has an RMSPropOptimizer class:\noptimizer = tf.train.RMSPropOptimizer(learning_rate=learning_rate,\n momentum=0.9, decay=0.9, epsilon=1e-10)\nExcept on very simple problems, this optimizer almost always performs much better\nthan AdaGrad. It also generally performs better than Momentum optimization and", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 780, "content": "than AdaGrad. It also generally performs better than Momentum optimization and\nNesterov Accelerated Gradients. In fact, it was the preferred optimization algorithm\nof many researchers until Adam optimization came around.\nAdam Optimization\nAdam,15 which stands for adaptive moment estimation, combines the ideas of Momen\u2010\ntum optimization and RMSProp: just like Momentum optimization it keeps track of\nan exponentially decaying average of past gradients, and just like RMSProp it keeps\n298 \n| \nChapter 11: Training Deep Neural Nets\nDownload from finelybook www.finelybook.com", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 781, "content": "16 These are estimations of the mean and (uncentered) variance of the gradients. The mean is often called the\nfirst moment, while the variance is often called the second moment, hence the name of the algorithm.\ntrack of an exponentially decaying average of past squared gradients (see Equation\n11-8).16\nEquation 11-8. Adam algorithm\n1 .\n\ufffd\n\u03b21\ufffd+ 1 \u2212\u03b21 \u2207\u03b8J \u03b8\n2 .\n\ufffd\n\u03b22\ufffd+ 1 \u2212\u03b22 \u2207\u03b8J \u03b8 \u2297\u2207\u03b8J \u03b8\n3 .\n\ufffd\n\ufffd\n1 \u2212\u03b21\nT\n4 .\n\ufffd\n\ufffd\n1 \u2212\u03b22\nT\n5 .\n\u03b8\n\u03b8 \u2212\u03b7 \ufffd\u2298\n\ufffd+ \ufffd\n\u2022 T represents the iteration number (starting at 1).\nIf you just look at steps 1, 2, and 5, you will notice Adam\u2019s close similarity to both\nMomentum optimization and RMSProp. The only difference is that step 1 computes\nan exponentially decaying average rather than an exponentially decaying sum, but\nthese are actually equivalent except for a constant factor (the decaying average is just\n1 \u2013 \u03b21 times the decaying sum). Steps 3 and 4 are somewhat of a technical detail: since", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 782, "content": "1 \u2013 \u03b21 times the decaying sum). Steps 3 and 4 are somewhat of a technical detail: since\nm and s are initialized at 0, they will be biased toward 0 at the beginning of training,\nso these two steps will help boost m and s at the beginning of training.\nThe momentum decay hyperparameter \u03b21 is typically initialized to 0.9, while the scal\u2010\ning decay hyperparameter \u03b22 is often initialized to 0.999. As earlier, the smoothing\nterm \u03f5 is usually initialized to a tiny number such as 10\u20138. These are the default values\nfor TensorFlow\u2019s AdamOptimizer class, so you can simply use:\noptimizer = tf.train.AdamOptimizer(learning_rate=learning_rate)\nIn fact, since Adam is an adaptive learning rate algorithm (like AdaGrad and\nRMSProp), it requires less tuning of the learning rate hyperparameter \u03b7. You can\noften use the default value \u03b7 = 0.001, making Adam even easier to use than Gradient\nDescent.\nFaster Optimizers \n| \n299\nDownload from finelybook www.finelybook.com", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 783, "content": "17 \u201cPrimal-Dual Subgradient Methods for Convex Problems,\u201d Yurii Nesterov (2005).\n18 \u201cAd Click Prediction: a View from the Trenches,\u201d H. McMahan et al. (2013).\nAll the optimization techniques discussed so far only rely on the\nfirst-order partial derivatives (Jacobians). The optimization litera\u2010\nture contains amazing algorithms based on the second-order partial\nderivatives (the Hessians). Unfortunately, these algorithms are very\nhard to apply to deep neural networks because there are n2 Hessi\u2010\nans per output (where n is the number of parameters), as opposed\nto just n Jacobians per output. Since DNNs typically have tens of\nthousands of parameters, the second-order optimization algo\u2010\nrithms often don\u2019t even fit in memory, and even when they do,\ncomputing the Hessians is just too slow.\nTraining Sparse Models\nAll the optimization algorithms just presented produce dense models, meaning that\nmost parameters will be nonzero. If you need a blazingly fast model at runtime, or if", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 784, "content": "most parameters will be nonzero. If you need a blazingly fast model at runtime, or if\nyou need it to take up less memory, you may prefer to end up with a sparse model\ninstead.\nOne trivial way to achieve this is to train the model as usual, then get rid of the tiny\nweights (set them to 0).\nAnother option is to apply strong \u21131 regularization during training, as it pushes the\noptimizer to zero out as many weights as it can (as discussed in Chapter 4 about Lasso\nRegression).\nHowever, in some cases these techniques may remain insufficient. One last option is\nto apply Dual Averaging, often called Follow The Regularized Leader (FTRL), a techni\u2010\nque proposed by Yurii Nesterov.17 When used with \u21131 regularization, this technique\noften leads to very sparse models. TensorFlow implements a variant of FTRL called\nFTRL-Proximal18 in the FTRLOptimizer class.\nLearning Rate Scheduling\nFinding a good learning rate can be tricky. If you set it way too high, training may", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 785, "content": "Finding a good learning rate can be tricky. If you set it way too high, training may\nactually diverge (as we discussed in Chapter 4). If you set it too low, training will\neventually converge to the optimum, but it will take a very long time. If you set it\nslightly too high, it will make progress very quickly at first, but it will end up dancing\naround the optimum, never settling down (unless you use an adaptive learning rate\noptimization algorithm such as AdaGrad, RMSProp, or Adam, but even then it may\ntake time to settle). If you have a limited computing budget, you may have to inter\u2010\n300 \n| \nChapter 11: Training Deep Neural Nets\nDownload from finelybook www.finelybook.com", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 786, "content": "rupt training before it has converged properly, yielding a suboptimal solution (see\nFigure 11-8).\nFigure 11-8. Learning curves for various learning rates \u03b7\nYou may be able to find a fairly good learning rate by training your network several\ntimes during just a few epochs using various learning rates and comparing the learn\u2010\ning curves. The ideal learning rate will learn quickly and converge to good solution.\nHowever, you can do better than a constant learning rate: if you start with a high\nlearning rate and then reduce it once it stops making fast progress, you can reach a\ngood solution faster than with the optimal constant learning rate. There are many dif\u2010\nferent strategies to reduce the learning rate during training. These strategies are called\nlearning schedules (we briefly introduced this concept in Chapter 4), the most com\u2010\nmon of which are:\nPredetermined piecewise constant learning rate\nFor example, set the learning rate to \u03b70 = 0.1 at first, then to \u03b71 = 0.001 after 50", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 787, "content": "For example, set the learning rate to \u03b70 = 0.1 at first, then to \u03b71 = 0.001 after 50\nepochs. Although this solution can work very well, it often requires fiddling\naround to figure out the right learning rates and when to use them.\nPerformance scheduling\nMeasure the validation error every N steps (just like for early stopping) and\nreduce the learning rate by a factor of \u03bb when the error stops dropping.\nExponential scheduling\nSet the learning rate to a function of the iteration number t: \u03b7(t) = \u03b70 10\u2013t/r. This\nworks great, but it requires tuning \u03b70 and r. The learning rate will drop by a fac\u2010\ntor of 10 every r steps.\nPower scheduling\nSet the learning rate to \u03b7(t) = \u03b70 (1 + t/r)\u2013c. The hyperparameter c is typically set\nto 1. This is similar to exponential scheduling, but the learning rate drops much\nmore slowly.\nFaster Optimizers \n| \n301\nDownload from finelybook www.finelybook.com", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 788, "content": "19 \u201cAn Empirical Study of Learning Rates in Deep Neural Networks for Speech Recognition,\u201d A. Senior et al.\n(2013).\nA 2013 paper19 by Andrew Senior et al. compared the performance of some of the\nmost popular learning schedules when training deep neural networks for speech rec\u2010\nognition using Momentum optimization. The authors concluded that, in this setting,\nboth performance scheduling and exponential scheduling performed well, but they\nfavored exponential scheduling because it is simpler to implement, is easy to tune,\nand converged slightly faster to the optimal solution.\nImplementing a learning schedule with TensorFlow is fairly straightforward:\ninitial_learning_rate = 0.1\ndecay_steps = 10000\ndecay_rate = 1/10\nglobal_step = tf.Variable(0, trainable=False)\nlearning_rate = tf.train.exponential_decay(initial_learning_rate, global_step,\n decay_steps, decay_rate)\noptimizer = tf.train.MomentumOptimizer(learning_rate, momentum=0.9)", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 789, "content": "optimizer = tf.train.MomentumOptimizer(learning_rate, momentum=0.9)\ntraining_op = optimizer.minimize(loss, global_step=global_step)\nAfter setting the hyperparameter values, we create a nontrainable variable\nglobal_step (initialized to 0) to keep track of the current training iteration number.\nThen we define an exponentially decaying learning rate (with \u03b70 = 0.1 and r = 10,000)\nusing TensorFlow\u2019s exponential_decay() function. Next, we create an optimizer (in\nthis example, a MomentumOptimizer) using this decaying learning rate. Finally, we cre\u2010\nate the training operation by calling the optimizer\u2019s minimize() method; since we\npass it the global_step variable, it will kindly take care of incrementing it. That\u2019s it!\nSince AdaGrad, RMSProp, and Adam optimization automatically reduce the learning\nrate during training, it is not necessary to add an extra learning schedule. For other\noptimization algorithms, using exponential decay or performance scheduling can", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 790, "content": "optimization algorithms, using exponential decay or performance scheduling can\nconsiderably speed up convergence.\nAvoiding Overfitting Through Regularization\nWith four parameters I can fit an elephant and with five I can make him wiggle his\ntrunk.\n\u2014John von Neumann, cited by Enrico Fermi in Nature 427\nDeep neural networks typically have tens of thousands of parameters, sometimes\neven millions. With so many parameters, the network has an incredible amount of\nfreedom and can fit a huge variety of complex datasets. But this great flexibility also\nmeans that it is prone to overfitting the training set.\n302 \n| \nChapter 11: Training Deep Neural Nets\nDownload from finelybook www.finelybook.com", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 791, "content": "With millions of parameters you can fit the whole zoo. In this section we will present\nsome of the most popular regularization techniques for neural networks, and how to\nimplement them with TensorFlow: early stopping, \u21131 and \u21132 regularization, dropout,\nmax-norm regularization, and data augmentation.\nEarly Stopping\nTo avoid overfitting the training set, a great solution is early stopping (introduced in\nChapter 4): just interrupt training when its performance on the validation set starts\ndropping.\nOne way to implement this with TensorFlow is to evaluate the model on a validation\nset at regular intervals (e.g., every 50 steps), and save a \u201cwinner\u201d snapshot if it outper\u2010\nforms previous \u201cwinner\u201d snapshots. Count the number of steps since the last \u201cwin\u2010\nner\u201d snapshot was saved, and interrupt training when this number reaches some limit\n(e.g., 2,000 steps). Then restore the last \u201cwinner\u201d snapshot.\nAlthough early stopping works very well in practice, you can usually get much higher", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 792, "content": "Although early stopping works very well in practice, you can usually get much higher\nperformance out of your network by combining it with other regularization techni\u2010\nques.\n\u21131 and \u21132 Regularization\nJust like you did in Chapter 4 for simple linear models, you can use \u21131 and \u21132 regulari\u2010\nzation to constrain a neural network\u2019s connection weights (but typically not its bia\u2010\nses).\nOne way to do this using TensorFlow is to simply add the appropriate regularization\nterms to your cost function. For example, assuming you have just one hidden layer\nwith weights weights1 and one output layer with weights weights2, then you can\napply \u21131 regularization like this:\n[...] # construct the neural network\nbase_loss = tf.reduce_mean(xentropy, name=\"avg_xentropy\")\nreg_losses = tf.reduce_sum(tf.abs(weights1)) + tf.reduce_sum(tf.abs(weights2))\nloss = tf.add(base_loss, scale * reg_losses, name=\"loss\")\nHowever, if there are many layers, this approach is not very convenient. Fortunately,", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 793, "content": "However, if there are many layers, this approach is not very convenient. Fortunately,\nTensorFlow provides a better option. Many functions that create variables (such as\nget_variable() or fully_connected()) accept a *_regularizer argument for each\ncreated variable (e.g., weights_regularizer). You can pass any function that takes\nweights as an argument and returns the corresponding regularization loss. The\nl1_regularizer(), l2_regularizer(), and l1_l2_regularizer() functions return \nsuch functions. The following code puts all this together:\nwith arg_scope(\n [fully_connected],\nAvoiding Overfitting Through Regularization \n| \n303\nDownload from finelybook www.finelybook.com", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 794, "content": "20 \u201cImproving neural networks by preventing co-adaptation of feature detectors,\u201d G. Hinton et al. (2012).\n21 \u201cDropout: A Simple Way to Prevent Neural Networks from Overfitting,\u201d N. Srivastava et al. (2014).\n weights_regularizer=tf.contrib.layers.l1_regularizer(scale=0.01)):\n hidden1 = fully_connected(X, n_hidden1, scope=\"hidden1\")\n hidden2 = fully_connected(hidden1, n_hidden2, scope=\"hidden2\")\n logits = fully_connected(hidden2, n_outputs, activation_fn=None,scope=\"out\")\nThis code creates a neural network with two hidden layers and one output layer, and\nit also creates nodes in the graph to compute the \u21131 regularization loss corresponding\nto each layer\u2019s weights. TensorFlow automatically adds these nodes to a special collec\u2010\ntion containing all the regularization losses. You just need to add these regularization\nlosses to your overall loss, like this:\nreg_losses = tf.get_collection(tf.GraphKeys.REGULARIZATION_LOSSES)", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 795, "content": "reg_losses = tf.get_collection(tf.GraphKeys.REGULARIZATION_LOSSES)\nloss = tf.add_n([base_loss] + reg_losses, name=\"loss\")\nDon\u2019t forget to add the regularization losses to your overall loss, or\nelse they will simply be ignored.\nDropout\nThe most popular regularization technique for deep neural networks is arguably\ndropout. It was proposed20 by G. E. Hinton in 2012 and further detailed in a paper21 by\nNitish Srivastava et al., and it has proven to be highly successful: even the state-of-\nthe-art neural networks got a 1\u20132% accuracy boost simply by adding dropout. This\nmay not sound like a lot, but when a model already has 95% accuracy, getting a 2%\naccuracy boost means dropping the error rate by almost 40% (going from 5% error to\nroughly 3%).\nIt is a fairly simple algorithm: at every training step, every neuron (including the\ninput neurons but excluding the output neurons) has a probability p of being tem\u2010", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 796, "content": "input neurons but excluding the output neurons) has a probability p of being tem\u2010\nporarily \u201cdropped out,\u201d meaning it will be entirely ignored during this training step,\nbut it may be active during the next step (see Figure 11-9). The hyperparameter p is\ncalled the dropout rate, and it is typically set to 50%. After training, neurons don\u2019t get\ndropped anymore. And that\u2019s all (except for a technical detail we will discuss momen\u2010\ntarily).\n304 \n| \nChapter 11: Training Deep Neural Nets\nDownload from finelybook www.finelybook.com", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 797, "content": "Figure 11-9. Dropout regularization\nIt is quite surprising at first that this rather brutal technique works at all. Would a\ncompany perform better if its employees were told to toss a coin every morning to\ndecide whether or not to go to work? Well, who knows; perhaps it would! The com\u2010\npany would obviously be forced to adapt its organization; it could not rely on any sin\u2010\ngle person to fill in the coffee machine or perform any other critical tasks, so this\nexpertise would have to be spread across several people. Employees would have to\nlearn to cooperate with many of their coworkers, not just a handful of them. The\ncompany would become much more resilient. If one person quit, it wouldn\u2019t make\nmuch of a difference. It\u2019s unclear whether this idea would actually work for compa\u2010\nnies, but it certainly does for neural networks. Neurons trained with dropout cannot\nco-adapt with their neighboring neurons; they have to be as useful as possible on", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 798, "content": "co-adapt with their neighboring neurons; they have to be as useful as possible on\ntheir own. They also cannot rely excessively on just a few input neurons; they must\npay attention to each of their input neurons. They end up being less sensitive to slight\nchanges in the inputs. In the end you get a more robust network that generalizes bet\u2010\nter.\nAnother way to understand the power of dropout is to realize that a unique neural\nnetwork is generated at each training step. Since each neuron can be either present or\nabsent, there is a total of 2N possible networks (where N is the total number of drop\u2010\npable neurons). This is such a huge number that it is virtually impossible for the same\nneural network to be sampled twice. Once you have run a 10,000 training steps, you\nhave essentially trained 10,000 different neural networks (each with just one training\ninstance). These neural networks are obviously not independent since they share", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 799, "content": "instance). These neural networks are obviously not independent since they share\nmany of their weights, but they are nevertheless all different. The resulting neural\nnetwork can be seen as an averaging ensemble of all these smaller neural networks.\nThere is one small but important technical detail. Suppose p = 50, in which case dur\u2010\ning testing a neuron will be connected to twice as many input neurons as it was (on\naverage) during training. To compensate for this fact, we need to multiply each neu\u2010\nAvoiding Overfitting Through Regularization \n| \n305\nDownload from finelybook www.finelybook.com", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 800, "content": "ron\u2019s input connection weights by 0.5 after training. If we don\u2019t, each neuron will get a\ntotal input signal roughly twice as large as what the network was trained on, and it is\nunlikely to perform well. More generally, we need to multiply each input connection\nweight by the keep probability (1 \u2013 p) after training. Alternatively, we can divide each\nneuron\u2019s output by the keep probability during training (these alternatives are not\nperfectly equivalent, but they work equally well).\nTo implement dropout using TensorFlow, you can simply apply the dropout() func\u2010\ntion to the input layer and to the output of every hidden layer. During training, this\nfunction randomly drops some items (setting them to 0) and divides the remaining\nitems by the keep probability. After training, this function does nothing at all. The\nfollowing code applies dropout regularization to our three-layer neural network:\nfrom tensorflow.contrib.layers import dropout\n[...]", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 801, "content": "from tensorflow.contrib.layers import dropout\n[...]\nis_training = tf.placeholder(tf.bool, shape=(), name='is_training')\nkeep_prob = 0.5\nX_drop = dropout(X, keep_prob, is_training=is_training)\nhidden1 = fully_connected(X_drop, n_hidden1, scope=\"hidden1\")\nhidden1_drop = dropout(hidden1, keep_prob, is_training=is_training)\nhidden2 = fully_connected(hidden1_drop, n_hidden2, scope=\"hidden2\")\nhidden2_drop = dropout(hidden2, keep_prob, is_training=is_training)\nlogits = fully_connected(hidden2_drop, n_outputs, activation_fn=None,\n scope=\"outputs\")\nYou want to use the dropout() function in tensorflow.con\ntrib.layers, not the one in tensorflow.nn. The first one turns off\n(no-op) when not training, which is what you want, while the sec\u2010\nond one does not.\nOf course, just like you did earlier for Batch Normalization, you need to set is_train\ning to True when training, and to False when testing.", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 802, "content": "ing to True when training, and to False when testing.\nIf you observe that the model is overfitting, you can increase the dropout rate (i.e.,\nreduce the keep_prob hyperparameter). Conversely, you should try decreasing the\ndropout rate (i.e., increasing keep_prob) if the model underfits the training set. It can\nalso help to increase the dropout rate for large layers, and reduce it for small ones.\nDropout does tend to significantly slow down convergence, but it usually results in a\nmuch better model when tuned properly. So, it is generally well worth the extra time\nand effort.\n306 \n| \nChapter 11: Training Deep Neural Nets\nDownload from finelybook www.finelybook.com", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 803, "content": "Dropconnect is a variant of dropout where individual connections\nare dropped randomly rather than whole neurons. In general drop\u2010\nout performs better.\nMax-Norm Regularization\nAnother regularization technique that is quite popular for neural networks is called\nmax-norm regularization: for each neuron, it constrains the weights w of the incom\u2010\ning connections such that \u2225 w \u22252 \u2264 r, where r is the max-norm hyperparameter and\n\u2225 \u00b7 \u22252 is the \u21132 norm.\nWe typically implement this constraint by computing \u2225w\u22252 after each training step\nand clipping w if needed (\ufffd\n\ufffd\nr\n\u2225\ufffd\u22252).\nReducing r increases the amount of regularization and helps reduce overfitting. Max-\nnorm regularization can also help alleviate the vanishing/exploding gradients prob\u2010\nlems (if you are not using Batch Normalization).\nTensorFlow does not provide an off-the-shelf max-norm regularizer, but it is not too\nhard to implement. The following code creates a node clip_weights that will clip the", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 804, "content": "hard to implement. The following code creates a node clip_weights that will clip the\nweights variable along the second axis so that each row vector has a maximum norm\nof 1.0:\nthreshold = 1.0\nclipped_weights = tf.clip_by_norm(weights, clip_norm=threshold, axes=1)\nclip_weights = tf.assign(weights, clipped_weights)\nYou would then apply this operation after each training step, like so:\nwith tf.Session() as sess:\n [...]\n for epoch in range(n_epochs):\n [...]\n for X_batch, y_batch in zip(X_batches, y_batches):\n sess.run(training_op, feed_dict={X: X_batch, y: y_batch})\n clip_weights.eval()\nYou may wonder how to get access to the weights variable of each layer. For this you\ncan simply use a variable scope like this:\nhidden1 = fully_connected(X, n_hidden1, scope=\"hidden1\")\nwith tf.variable_scope(\"hidden1\", reuse=True):\n weights1 = tf.get_variable(\"weights\")\nAlternatively, you can use the root variable scope:", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 805, "content": "weights1 = tf.get_variable(\"weights\")\nAlternatively, you can use the root variable scope:\nhidden1 = fully_connected(X, n_hidden1, scope=\"hidden1\")\nhidden2 = fully_connected(hidden1, n_hidden2, scope=\"hidden2\")\nAvoiding Overfitting Through Regularization \n| \n307\nDownload from finelybook www.finelybook.com", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 806, "content": "[...]\nwith tf.variable_scope(\"\", default_name=\"\", reuse=True): # root scope\n weights1 = tf.get_variable(\"hidden1/weights\")\n weights2 = tf.get_variable(\"hidden2/weights\")\nIf you don\u2019t know what the name of a variable is, you can either use TensorBoard to\nfind out or simply use the global_variables() function and print out all the variable\nnames:\nfor variable in tf.global_variables():\n print(variable.name)\nAlthough the preceding solution should work fine, it is a bit messy. A cleaner solution\nis to create a max_norm_regularizer() function and use it just like the earlier l1_reg\nularizer() function:\ndef max_norm_regularizer(threshold, axes=1, name=\"max_norm\",\n collection=\"max_norm\"):\n def max_norm(weights):\n clipped = tf.clip_by_norm(weights, clip_norm=threshold, axes=axes)\n clip_weights = tf.assign(weights, clipped, name=name)\n tf.add_to_collection(collection, clip_weights)", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 807, "content": "tf.add_to_collection(collection, clip_weights)\n return None # there is no regularization loss term\n return max_norm\nThis function returns a parametrized max_norm() function that you can use like any\nother regularizer:\nmax_norm_reg = max_norm_regularizer(threshold=1.0)\nhidden1 = fully_connected(X, n_hidden1, scope=\"hidden1\",\n weights_regularizer=max_norm_reg)\nNote that max-norm regularization does not require adding a regularization loss term\nto your overall loss function, so the max_norm() function returns None. But you still\nneed to be able to run the clip_weights operation after each training step, so you\nneed to be able to get a handle on it. This is why the max_norm() function adds the\nclip_weights node to a collection of max-norm clipping operations. You need to\nfetch these clipping operations and run them after each training step:\nclip_all_weights = tf.get_collection(\"max_norm\")\nwith tf.Session() as sess:\n [...]", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 808, "content": "clip_all_weights = tf.get_collection(\"max_norm\")\nwith tf.Session() as sess:\n [...]\n for epoch in range(n_epochs):\n [...]\n for X_batch, y_batch in zip(X_batches, y_batches):\n sess.run(training_op, feed_dict={X: X_batch, y: y_batch})\n sess.run(clip_all_weights)\nMuch cleaner code, isn\u2019t it?\n308 \n| \nChapter 11: Training Deep Neural Nets\nDownload from finelybook www.finelybook.com", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 809, "content": "Data Augmentation\nOne last regularization technique, data augmentation, consists of generating new\ntraining instances from existing ones, artificially boosting the size of the training set.\nThis will reduce overfitting, making this a regularization technique. The trick is to\ngenerate realistic training instances; ideally, a human should not be able to tell which\ninstances were generated and which ones were not. Moreover, simply adding white\nnoise will not help; the modifications you apply should be learnable (white noise is\nnot).\nFor example, if your model is meant to classify pictures of mushrooms, you can\nslightly shift, rotate, and resize every picture in the training set by various amounts\nand add the resulting pictures to the training set (see Figure 11-10). This forces the\nmodel to be more tolerant to the position, orientation, and size of the mushrooms in\nthe picture. If you want the model to be more tolerant to lighting conditions, you can", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 810, "content": "the picture. If you want the model to be more tolerant to lighting conditions, you can\nsimilarly generate many images with various contrasts. Assuming the mushrooms are\nsymmetrical, you can also flip the pictures horizontally. By combining these transfor\u2010\nmations you can greatly increase the size of your training set.\nFigure 11-10. Generating new training instances from existing ones\nIt is often preferable to generate training instances on the fly during training rather\nthan wasting storage space and network bandwidth. TensorFlow offers several image\nmanipulation operations such as transposing (shifting), rotating, resizing, flipping,\nand cropping, as well as adjusting the brightness, contrast, saturation, and hue (see\nAvoiding Overfitting Through Regularization \n| \n309\nDownload from finelybook www.finelybook.com", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 811, "content": "the API documentation for more details). This makes it easy to implement data aug\u2010\nmentation for image datasets.\nAnother powerful technique to train very deep neural networks is\nto add skip connections (a skip connection is when you add the\ninput of a layer to the output of a higher layer). We will explore this\nidea in Chapter 13 when we talk about deep residual networks.\nPractical Guidelines\nIn this chapter, we have covered a wide range of techniques and you may be wonder\u2010\ning which ones you should use. The configuration in Table 11-2 will work fine in\nmost cases.\nTable 11-2. Default DNN configuration\nInitialization\nHe initialization\nActivation function\nELU\nNormalization\nBatch Normalization\nRegularization\nDropout\nOptimizer\nAdam\nLearning rate schedule\nNone\nOf course, you should try to reuse parts of a pretrained neural network if you can\nfind one that solves a similar problem.\nThis default configuration may need to be tweaked:", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 812, "content": "find one that solves a similar problem.\nThis default configuration may need to be tweaked:\n\u2022 If you can\u2019t find a good learning rate (convergence was too slow, so you increased\nthe training rate, and now convergence is fast but the network\u2019s accuracy is sub\u2010\noptimal), then you can try adding a learning schedule such as exponential decay.\n\u2022 If your training set is a bit too small, you can implement data augmentation.\n\u2022 If you need a sparse model, you can add some \u21131 regularization to the mix (and\noptionally zero out the tiny weights after training). If you need an even sparser\nmodel, you can try using FTRL instead of Adam optimization, along with \u21131 reg\u2010\nularization.\n\u2022 If you need a lightning-fast model at runtime, you may want to drop Batch Nor\u2010\nmalization, and possibly replace the ELU activation function with the leaky\nReLU. Having a sparse model will also help.\nWith these guidelines, you are now ready to train very deep nets\u2014well, if you are", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 813, "content": "With these guidelines, you are now ready to train very deep nets\u2014well, if you are\nvery patient, that is! If you use a single machine, you may have to wait for days or\n310 \n| \nChapter 11: Training Deep Neural Nets\nDownload from finelybook www.finelybook.com", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 814, "content": "even months for training to complete. In the next chapter we will discuss how to use\ndistributed TensorFlow to train and run models across many servers and GPUs.\nExercises\n1. Is it okay to initialize all the weights to the same value as long as that value is\nselected randomly using He initialization?\n2. Is it okay to initialize the bias terms to 0?\n3. Name three advantages of the ELU activation function over ReLU.\n4. In which cases would you want to use each of the following activation functions:\nELU, leaky ReLU (and its variants), ReLU, tanh, logistic, and softmax?\n5. What may happen if you set the momentum hyperparameter too close to 1 (e.g.,\n0.99999) when using a MomentumOptimizer?\n6. Name three ways you can produce a sparse model.\n7. Does dropout slow down training? Does it slow down inference (i.e., making\npredictions on new instances)?\n8. Deep Learning.\na. Build a DNN with five hidden layers of 100 neurons each, He initialization,\nand the ELU activation function.", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 815, "content": "and the ELU activation function.\nb. Using Adam optimization and early stopping, try training it on MNIST but\nonly on digits 0 to 4, as we will use transfer learning for digits 5 to 9 in the\nnext exercise. You will need a softmax output layer with five neurons, and as\nalways make sure to save checkpoints at regular intervals and save the final\nmodel so you can reuse it later.\nc. Tune the hyperparameters using cross-validation and see what precision you\ncan achieve.\nd. Now try adding Batch Normalization and compare the learning curves: is it\nconverging faster than before? Does it produce a better model?\ne. Is the model overfitting the training set? Try adding dropout to every layer\nand try again. Does it help?\n9. Transfer learning.\na. Create a new DNN that reuses all the pretrained hidden layers of the previous\nmodel, freezes them, and replaces the softmax output layer with a fresh new\none.\nb. Train this new DNN on digits 5 to 9, using only 100 images per digit, and time", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 816, "content": "one.\nb. Train this new DNN on digits 5 to 9, using only 100 images per digit, and time\nhow long it takes. Despite this small number of examples, can you achieve\nhigh precision?\nExercises \n| \n311\nDownload from finelybook www.finelybook.com", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 817, "content": "c. Try caching the frozen layers, and train the model again: how much faster is it\nnow?\nd. Try again reusing just four hidden layers instead of five. Can you achieve a\nhigher precision?\ne. Now unfreeze the top two hidden layers and continue training: can you get\nthe model to perform even better?\n10. Pretraining on an auxiliary task.\na. In this exercise you will build a DNN that compares two MNIST digit images\nand predicts whether they represent the same digit or not. Then you will reuse\nthe lower layers of this network to train an MNIST classifier using very little\ntraining data. Start by building two DNNs (let\u2019s call them DNN A and B), both\nsimilar to the one you built earlier but without the output layer: each DNN\nshould have five hidden layers of 100 neurons each, He initialization, and ELU\nactivation. Next, add a single output layer on top of both DNNs. You should\nuse TensorFlow\u2019s concat() function with axis=1 to concatenate the outputs", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 818, "content": "use TensorFlow\u2019s concat() function with axis=1 to concatenate the outputs\nof both DNNs along the horizontal axis, then feed the result to the output\nlayer. This output layer should contain a single neuron using the logistic acti\u2010\nvation function.\nb. Split the MNIST training set in two sets: split #1 should containing 55,000\nimages, and split #2 should contain contain 5,000 images. Create a function\nthat generates a training batch where each instance is a pair of MNIST images\npicked from split #1. Half of the training instances should be pairs of images\nthat belong to the same class, while the other half should be images from dif\u2010\nferent classes. For each pair, the training label should be 0 if the images are\nfrom the same class, or 1 if they are from different classes.\nc. Train the DNN on this training set. For each image pair, you can simultane\u2010\nously feed the first image to DNN A and the second image to DNN B. The", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 819, "content": "ously feed the first image to DNN A and the second image to DNN B. The\nwhole network will gradually learn to tell whether two images belong to the\nsame class or not.\nd. Now create a new DNN by reusing and freezing the hidden layers of DNN A\nand adding a softmax output layer on with 10 neurons. Train this network on\nsplit #2 and see if you can achieve high performance despite having only 500\nimages per class.\nSolutions to these exercises are available in Appendix A.\n312 \n| \nChapter 11: Training Deep Neural Nets\nDownload from finelybook www.finelybook.com", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 820, "content": "CHAPTER 12\nDistributing TensorFlow Across\nDevices and Servers\nIn Chapter 11 we discussed several techniques that can considerably speed up train\u2010\ning: better weight initialization, Batch Normalization, sophisticated optimizers, and\nso on. However, even with all of these techniques, training a large neural network on\na single machine with a single CPU can take days or even weeks.\nIn this chapter we will see how to use TensorFlow to distribute computations across\nmultiple devices (CPUs and GPUs) and run them in parallel (see Figure 12-1). First\nwe will distribute computations across multiple devices on just one machine, then on\nmultiple devices across multiple machines.\nFigure 12-1. Executing a TensorFlow graph across multiple devices in parallel\n313\nDownload from finelybook www.finelybook.com", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 821, "content": "TensorFlow\u2019s support of distributed computing is one of its main highlights com\u2010\npared to other neural network frameworks. It gives you full control over how to split\n(or replicate) your computation graph across devices and servers, and it lets you par\u2010\nallelize and synchronize operations in flexible ways so you can choose between all\nsorts of parallelization approaches.\nWe will look at some of the most popular approaches to parallelizing the execution\nand training of a neural network. Instead of waiting for weeks for a training algo\u2010\nrithm to complete, you may end up waiting for just a few hours. Not only does this\nsave an enormous amount of time, it also means that you can experiment with vari\u2010\nous models much more easily, and frequently retrain your models on fresh data.\nOther great use cases of parallelization include exploring a much larger hyperparame\u2010\nter space when fine-tuning your model, and running large ensembles of neural net\u2010\nworks efficiently.", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 822, "content": "works efficiently.\nBut we must learn to walk before we can run. Let\u2019s start by parallelizing simple graphs\nacross several GPUs on a single machine.\nMultiple Devices on a Single Machine\nYou can often get a major performance boost simply by adding GPU cards to a single\nmachine. In fact, in many cases this will suffice; you won\u2019t need to use multiple\nmachines at all. For example, you can typically train a neural network just as fast\nusing 8 GPUs on a single machine rather than 16 GPUs across multiple machines\n(due to the extra delay imposed by network communications in a multimachine\nsetup).\nIn this section we will look at how to set up your environment so that TensorFlow can\nuse multiple GPU cards on one machine. Then we will look at how you can distribute\noperations across available devices and execute them in parallel.\nInstallation\nIn order to run TensorFlow on multiple GPU cards, you first need to make sure your", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 823, "content": "Installation\nIn order to run TensorFlow on multiple GPU cards, you first need to make sure your\nGPU cards have NVidia Compute Capability (greater or equal to 3.0). This includes\nNvidia\u2019s Titan, Titan X, K20, and K40 cards (if you own another card, you can check\nits compatibility at https://developer.nvidia.com/cuda-gpus).\n314 \n| \nChapter 12: Distributing TensorFlow Across Devices and Servers\nDownload from finelybook www.finelybook.com", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 824, "content": "If you don\u2019t own any GPU cards, you can use a hosting service with\nGPU capability such as Amazon AWS. Detailed instructions to set\nup TensorFlow 0.9 with Python 3.5 on an Amazon AWS GPU\ninstance are available in \u017diga Avsec\u2019s helpful blog post. It should\nnot be too hard to update it to the latest version of TensorFlow.\nGoogle also released a cloud service called Cloud Machine Learning\nto run TensorFlow graphs. In May 2016, they announced that their\nplatform now includes servers equipped with tensor processing units\n(TPUs), processors specialized for Machine Learning that are much\nfaster than GPUs for many ML tasks. Of course, another option is\nsimply to buy your own GPU card. Tim Dettmers wrote a great\nblog post to help you choose, and he updates it fairly regularly.\nYou must then download and install the appropriate version of the CUDA and\ncuDNN libraries (CUDA 8.0 and cuDNN 5.1 if you are using the binary installation", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 825, "content": "cuDNN libraries (CUDA 8.0 and cuDNN 5.1 if you are using the binary installation\nof TensorFlow 1.0.0), and set a few environment variables so TensorFlow knows\nwhere to find CUDA and cuDNN. The detailed installation instructions are likely to\nchange fairly quickly, so it is best that you follow the instructions on TensorFlow\u2019s\nwebsite.\nNvidia\u2019s Compute Unified Device Architecture library (CUDA) allows developers to\nuse CUDA-enabled GPUs for all sorts of computations (not just graphics accelera\u2010\ntion). Nvidia\u2019s CUDA Deep Neural Network library (cuDNN) is a GPU-accelerated\nlibrary of primitives for DNNs. It provides optimized implementations of common\nDNN computations such as activation layers, normalization, forward and backward\nconvolutions, and pooling (see Chapter 13). It is part of Nvidia\u2019s Deep Learning SDK\n(note that it requires creating an Nvidia developer account in order to download it).\nTensorFlow uses CUDA and cuDNN to control the GPU cards and accelerate com\u2010", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 826, "content": "TensorFlow uses CUDA and cuDNN to control the GPU cards and accelerate com\u2010\nputations (see Figure 12-2).\nFigure 12-2. TensorFlow uses CUDA and cuDNN to control GPUs and boost DNNs\nYou can use the nvidia-smi command to check that CUDA is properly installed. It\nlists the available GPU cards, as well as processes running on each card:\nMultiple Devices on a Single Machine \n| \n315\nDownload from finelybook www.finelybook.com", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 827, "content": "$ nvidia-smi\nWed Sep 16 09:50:03 2016\n+------------------------------------------------------+\n| NVIDIA-SMI 352.63 Driver Version: 352.63 |\n|-------------------------------+----------------------+----------------------+\n| GPU Name Persistence-M| Bus-Id Disp.A | Volatile Uncorr. ECC |\n| Fan Temp Perf Pwr:Usage/Cap| Memory-Usage | GPU-Util Compute M. |\n|===============================+======================+======================|\n| 0 GRID K520 Off | 0000:00:03.0 Off | N/A |\n| N/A 27C P8 17W / 125W | 11MiB / 4095MiB | 0% Default |\n+-------------------------------+----------------------+----------------------+\n+-----------------------------------------------------------------------------+\n| Processes: GPU Memory |\n| GPU PID Type Process name Usage |", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 828, "content": "| GPU PID Type Process name Usage |\n|=============================================================================|\n| No running processes found |\n+-----------------------------------------------------------------------------+\nFinally, you must install TensorFlow with GPU support. If you created an isolated\nenvironment using virtualenv, you first need to activate it:\n$ cd $ML_PATH # Your ML working directory (e.g., $HOME/ml)\n$ source env/bin/activate\nThen install the appropriate GPU-enabled version of TensorFlow:\n$ pip3 install --upgrade tensorflow-gpu\nNow you can open up a Python shell and check that TensorFlow detects and uses\nCUDA and cuDNN properly by importing TensorFlow and creating a session:\n>>> import tensorflow as tf\nI [...]/dso_loader.cc:108] successfully opened CUDA library libcublas.so locally", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 829, "content": "I [...]/dso_loader.cc:108] successfully opened CUDA library libcublas.so locally\nI [...]/dso_loader.cc:108] successfully opened CUDA library libcudnn.so locally\nI [...]/dso_loader.cc:108] successfully opened CUDA library libcufft.so locally\nI [...]/dso_loader.cc:108] successfully opened CUDA library libcuda.so.1 locally\nI [...]/dso_loader.cc:108] successfully opened CUDA library libcurand.so locally\n>>> sess = tf.Session()\n[...]\nI [...]/gpu_init.cc:102] Found device 0 with properties:\nname: GRID K520\nmajor: 3 minor: 0 memoryClockRate (GHz) 0.797\npciBusID 0000:00:03.0\nTotal memory: 4.00GiB\nFree memory: 3.95GiB\nI [...]/gpu_init.cc:126] DMA: 0\nI [...]/gpu_init.cc:136] 0: Y\nI [...]/gpu_device.cc:839] Creating TensorFlow device\n(/gpu:0) -> (device: 0, name: GRID K520, pci bus id: 0000:00:03.0)\nLooks good! TensorFlow detected the CUDA and cuDNN libraries, and it used the\nCUDA library to detect the GPU card (in this case an Nvidia Grid K520 card).\n316 \n|", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 830, "content": "CUDA library to detect the GPU card (in this case an Nvidia Grid K520 card).\n316 \n| \nChapter 12: Distributing TensorFlow Across Devices and Servers\nDownload from finelybook www.finelybook.com", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 831, "content": "Managing the GPU RAM\nBy default TensorFlow automatically grabs all the RAM in all available GPUs the first\ntime you run a graph, so you will not be able to start a second TensorFlow program\nwhile the first one is still running. If you try, you will get the following error:\nE [...]/cuda_driver.cc:965] failed to allocate 3.66G (3928915968 bytes) from\ndevice: CUDA_ERROR_OUT_OF_MEMORY\nOne solution is to run each process on different GPU cards. To do this, the simplest\noption is to set the CUDA_VISIBLE_DEVICES environment variable so that each process\nonly sees the appropriate GPU cards. For example, you could start two programs like\nthis:\n$ CUDA_VISIBLE_DEVICES=0,1 python3 program_1.py\n# and in another terminal:\n$ CUDA_VISIBLE_DEVICES=3,2 python3 program_2.py\nProgram #1 will only see GPU cards 0 and 1 (numbered 0 and 1, respectively), and\nprogram #2 will only see GPU cards 2 and 3 (numbered 1 and 0, respectively). Every\u2010\nthing will work fine (see Figure 12-3).", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 832, "content": "thing will work fine (see Figure 12-3).\nFigure 12-3. Each program gets two GPUs for itself\nAnother option is to tell TensorFlow to grab only a fraction of the memory. For\nexample, to make TensorFlow grab only 40% of each GPU\u2019s memory, you must create\na ConfigProto object, set its gpu_options.per_process_gpu_memory_fraction \noption to 0.4, and create the session using this configuration:\nconfig = tf.ConfigProto()\nconfig.gpu_options.per_process_gpu_memory_fraction = 0.4\nsession = tf.Session(config=config)\nNow two programs like this one can run in parallel using the same GPU cards (but\nnot three, since 3 \u00d7 0.4 > 1). See Figure 12-4.\nMultiple Devices on a Single Machine \n| \n317\nDownload from finelybook www.finelybook.com", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 833, "content": "1 \u201cTensorFlow: Large-Scale Machine Learning on Heterogeneous Distributed Systems,\u201d Google Research\n(2015).\nFigure 12-4. Each program gets all four GPUs, but with only 40% of the RAM each\nIf you run the nvidia-smi command while both programs are running, you should\nsee that each process holds roughly 40% of the total RAM of each card:\n$ nvidia-smi\n[...]\n+-----------------------------------------------------------------------------+\n| Processes: GPU Memory |\n| GPU PID Type Process name Usage |\n|=============================================================================|\n| 0 5231 C python 1677MiB |\n| 0 5262 C python 1677MiB |\n| 1 5231 C python 1677MiB |\n| 1 5262 C python 1677MiB |", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 834, "content": "| 1 5262 C python 1677MiB |\n[...]\nYet another option is to tell TensorFlow to grab memory only when it needs it. To do\nthis you must set config.gpu_options.allow_growth to True. However, TensorFlow\nnever releases memory once it has grabbed it (to avoid memory fragmentation) so\nyou may still run out of memory after a while. It may be harder to guarantee a deter\u2010\nministic behavior using this option, so in general you probably want to stick with one\nof the previous options.\nOkay, now you have a working GPU-enabled TensorFlow installation. Let\u2019s see how\nto use it!\nPlacing Operations on Devices\nThe TensorFlow whitepaper1 presents a friendly dynamic placer algorithm that auto\u2010\nmagically distributes operations across all available devices, taking into account\nthings like the measured computation time in previous runs of the graph, estimations\nof the size of the input and output tensors to each operation, the amount of RAM", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 835, "content": "of the size of the input and output tensors to each operation, the amount of RAM\navailable in each device, communication delay when transferring data in and out of\n318 \n| \nChapter 12: Distributing TensorFlow Across Devices and Servers\nDownload from finelybook www.finelybook.com", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 836, "content": "devices, hints and constraints from the user, and more. Unfortunately, this sophistica\u2010\nted algorithm is internal to Google; it was not released in the open source version of\nTensorFlow. The reason it was left out seems to be that in practice a small set of place\u2010\nment rules specified by the user actually results in more efficient placement than what\nthe dynamic placer is capable of. However, the TensorFlow team is working on\nimproving the dynamic placer, and perhaps it will eventually be good enough to be\nreleased.\nUntil then TensorFlow relies on the simple placer, which (as its name suggests) is very\nbasic.\nSimple placement\nWhenever you run a graph, if TensorFlow needs to evaluate a node that is not placed\non a device yet, it uses the simple placer to place it, along with all other nodes that are\nnot placed yet. The simple placer respects the following rules:\n\u2022 If a node was already placed on a device in a previous run of the graph, it is left\non that device.", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 837, "content": "on that device.\n\u2022 Else, if the user pinned a node to a device (described next), the placer places it on\nthat device.\n\u2022 Else, it defaults to GPU #0, or the CPU if there is no GPU.\nAs you can see, placing operations on the appropriate device is mostly up to you. If\nyou don\u2019t do anything, the whole graph will be placed on the default device. To pin\nnodes onto a device, you must create a device block using the device() function. For\nexample, the following code pins the variable a and the constant b on the CPU, but\nthe multiplication node c is not pinned on any device, so it will be placed on the \ndefault device:\nwith tf.device(\"/cpu:0\"):\n a = tf.Variable(3.0)\n b = tf.constant(4.0)\nc = a * b\nThe \"/cpu:0\" device aggregates all CPUs on a multi-CPU system.\nThere is currently no way to pin nodes on specific CPUs or to use\njust a subset of all CPUs.\nMultiple Devices on a Single Machine \n| \n319\nDownload from finelybook www.finelybook.com", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 838, "content": "Logging placements\nLet\u2019s check that the simple placer respects the placement constraints we have just\ndefined. For this you can set the log_device_placement option to True; this tells the\nplacer to log a message whenever it places a node. For example:\n>>> config = tf.ConfigProto()\n>>> config.log_device_placement = True\n>>> sess = tf.Session(config=config)\nI [...] Creating TensorFlow device (/gpu:0) -> (device: 0, name: GRID K520,\npci bus id: 0000:00:03.0)\n[...]\n>>> x.initializer.run(session=sess)\nI [...] a: /job:localhost/replica:0/task:0/cpu:0\nI [...] a/read: /job:localhost/replica:0/task:0/cpu:0\nI [...] mul: /job:localhost/replica:0/task:0/gpu:0\nI [...] a/Assign: /job:localhost/replica:0/task:0/cpu:0\nI [...] b: /job:localhost/replica:0/task:0/cpu:0\nI [...] a/initial_value: /job:localhost/replica:0/task:0/cpu:0\n>>> sess.run(c)\n12\nThe lines starting with \"I\" for Info are the log messages. When we create a session,", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 839, "content": "12\nThe lines starting with \"I\" for Info are the log messages. When we create a session,\nTensorFlow logs a message to tell us that it has found a GPU card (in this case the\nGrid K520 card). Then the first time we run the graph (in this case when initializing\nthe variable a), the simple placer is run and places each node on the device it was\nassigned to. As expected, the log messages show that all nodes are placed on \"/cpu:0\"\nexcept the multiplication node, which ends up on the default device \"/gpu:0\" (you\ncan safely ignore the prefix /job:localhost/replica:0/task:0 for now; we will talk\nabout it in a moment). Notice that the second time we run the graph (to compute c),\nthe placer is not used since all the nodes TensorFlow needs to compute c are already\nplaced.\nDynamic placement function\nWhen you create a device block, you can specify a function instead of a device name.\nTensorFlow will call this function for each operation it needs to place in the device", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 840, "content": "TensorFlow will call this function for each operation it needs to place in the device\nblock, and the function must return the name of the device to pin the operation on.\nFor example, the following code pins all the variable nodes to \"/cpu:0\" (in this case\njust the variable a) and all other nodes to \"/gpu:0\":\ndef variables_on_cpu(op):\n if op.type == \"Variable\":\n return \"/cpu:0\"\n else:\n return \"/gpu:0\"\nwith tf.device(variables_on_cpu):\n a = tf.Variable(3.0)\n320 \n| \nChapter 12: Distributing TensorFlow Across Devices and Servers\nDownload from finelybook www.finelybook.com", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 841, "content": "b = tf.constant(4.0)\n c = a * b\nYou can easily implement more complex algorithms, such as pinning variables across\nGPUs in a round-robin fashion.\nOperations and kernels\nFor a TensorFlow operation to run on a device, it needs to have an implementation\nfor that device; this is called a kernel. Many operations have kernels for both CPUs\nand GPUs, but not all of them. For example, TensorFlow does not have a GPU kernel\nfor integer variables, so the following code will fail when TensorFlow tries to place the\nvariable i on GPU #0:\n>>> with tf.device(\"/gpu:0\"):\n... i = tf.Variable(3)\n[...]\n>>> sess.run(i.initializer)\nTraceback (most recent call last):\n[...]\ntensorflow.python.framework.errors.InvalidArgumentError: Cannot assign a device\nto node 'Variable': Could not satisfy explicit device specification\nNote that TensorFlow infers that the variable must be of type int32 since the initiali\u2010\nzation value is an integer. If you change the initialization value to 3.0 instead of 3, or", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 842, "content": "zation value is an integer. If you change the initialization value to 3.0 instead of 3, or\nif you explicitly set dtype=tf.float32 when creating the variable, everything will\nwork fine.\nSoft placement\nBy default, if you try to pin an operation on a device for which the operation has no\nkernel, you get the exception shown earlier when TensorFlow tries to place the opera\u2010\ntion on the device. If you prefer TensorFlow to fall back to the CPU instead, you can\nset the allow_soft_placement configuration option to True:\nwith tf.device(\"/gpu:0\"):\n i = tf.Variable(3)\nconfig = tf.ConfigProto()\nconfig.allow_soft_placement = True\nsess = tf.Session(config=config)\nsess.run(i.initializer) # the placer runs and falls back to /cpu:0\nSo far we have discussed how to place nodes on different devices. Now let\u2019s see how\nTensorFlow will run these nodes in parallel.\nParallel Execution\nWhen TensorFlow runs a graph, it starts by finding out the list of nodes that need to", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 843, "content": "When TensorFlow runs a graph, it starts by finding out the list of nodes that need to\nbe evaluated, and it counts how many dependencies each of them has. TensorFlow\nMultiple Devices on a Single Machine \n| \n321\nDownload from finelybook www.finelybook.com", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 844, "content": "then starts evaluating the nodes with zero dependencies (i.e., source nodes). If these\nnodes are placed on separate devices, they obviously get evaluated in parallel. If they\nare placed on the same device, they get evaluated in different threads, so they may run\nin parallel too (in separate GPU threads or CPU cores).\nTensorFlow manages a thread pool on each device to parallelize operations (see\nFigure 12-5). These are called the inter-op thread pools. Some operations have multi\u2010\nthreaded kernels: they can use other thread pools (one per device) called the intra-op\nthread pools.\nFigure 12-5. Parallelized execution of a TensorFlow graph\nFor example, in Figure 12-5, operations A, B, and C are source ops, so they can\nimmediately be evaluated. Operations A and B are placed on GPU #0, so they are sent\nto this device\u2019s inter-op thread pool, and immediately evaluated in parallel. Operation\nA happens to have a multithreaded kernel; its computations are split in three parts,", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 845, "content": "A happens to have a multithreaded kernel; its computations are split in three parts,\nwhich are executed in parallel by the intra-op thread pool. Operation C goes to GPU\n#1\u2019s inter-op thread pool.\nAs soon as operation C finishes, the dependency counters of operations D and E will\nbe decremented and will both reach 0, so both operations will be sent to the inter-op\nthread pool to be executed.\nYou can control the number of threads per inter-op pool by setting\nthe inter_op_parallelism_threads option. Note that the first ses\u2010\nsion you start creates the inter-op thread pools. All other sessions\nwill just reuse them unless you set the use_per_session_threads\noption to True. You can control the number of threads per intra-op\npool by setting the intra_op_parallelism_threads option.\n322 \n| \nChapter 12: Distributing TensorFlow Across Devices and Servers\nDownload from finelybook www.finelybook.com", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 846, "content": "Control Dependencies\nIn some cases, it may be wise to postpone the evaluation of an operation even though\nall the operations it depends on have been executed. For example, if it uses a lot of\nmemory but its value is needed only much further in the graph, it would be best to\nevaluate it at the last moment to avoid needlessly occupying RAM that other opera\u2010\ntions may need. Another example is a set of operations that depend on data located\noutside of the device. If they all run at the same time, they may saturate the device\u2019s\ncommunication bandwidth, and they will end up all waiting on I/O. Other operations\nthat need to communicate data will also be blocked. It would be preferable to execute\nthese communication-heavy operations sequentially, allowing the device to perform\nother operations in parallel.\nTo postpone evaluation of some nodes, a simple solution is to add control dependen\u2010\ncies. For example, the following code tells TensorFlow to evaluate x and y only after a", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 847, "content": "cies. For example, the following code tells TensorFlow to evaluate x and y only after a\nand b have been evaluated:\na = tf.constant(1.0)\nb = a + 2.0\nwith tf.control_dependencies([a, b]):\n x = tf.constant(3.0)\n y = tf.constant(4.0)\nz = x + y\nObviously, since z depends on x and y, evaluating z also implies waiting for a and b to\nbe evaluated, even though it is not explicitly in the control_dependencies() block.\nAlso, since b depends on a, we could simplify the preceding code by just creating a\ncontrol dependency on [b] instead of [a, b], but in some cases \u201cexplicit is better\nthan implicit.\u201d\nGreat! Now you know:\n\u2022 How to place operations on multiple devices in any way you please\n\u2022 How these operations get executed in parallel\n\u2022 How to create control dependencies to optimize parallel execution\nIt\u2019s time to distribute computations across multiple servers!\nMultiple Devices Across Multiple Servers\nTo run a graph across multiple servers, you first need to define a cluster. A cluster is", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 848, "content": "To run a graph across multiple servers, you first need to define a cluster. A cluster is\ncomposed of one or more TensorFlow servers, called tasks, typically spread across\nseveral machines (see Figure 12-6). Each task belongs to a job. A job is just a named\ngroup of tasks that typically have a common role, such as keeping track of the model\nMultiple Devices Across Multiple Servers \n| \n323\nDownload from finelybook www.finelybook.com", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 849, "content": "parameters (such a job is usually named \"ps\" for parameter server), or performing\ncomputations (such a job is usually named \"worker\").\nFigure 12-6. TensorFlow cluster\nThe following cluster specification defines two jobs, \"ps\" and \"worker\", containing\none task and two tasks, respectively. In this example, machine A hosts two Tensor\u2010\nFlow servers (i.e., tasks), listening on different ports: one is part of the \"ps\" job, and\nthe other is part of the \"worker\" job. Machine B just hosts one TensorFlow server,\npart of the \"worker\" job.\ncluster_spec = tf.train.ClusterSpec({\n \"ps\": [\n \"machine-a.example.com:2221\", # /job:ps/task:0\n ],\n \"worker\": [\n \"machine-a.example.com:2222\", # /job:worker/task:0\n \"machine-b.example.com:2222\", # /job:worker/task:1\n ]})\nTo start a TensorFlow server, you must create a Server object, passing it the cluster\nspecification (so it can communicate with other servers) and its own job name and", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 850, "content": "specification (so it can communicate with other servers) and its own job name and\ntask number. For example, to start the first worker task, you would run the following\ncode on machine A:\nserver = tf.train.Server(cluster_spec, job_name=\"worker\", task_index=0)\nIt is usually simpler to just run one task per machine, but the previous example dem\u2010\nonstrates that TensorFlow allows you to run multiple tasks on the same machine if\n324 \n| \nChapter 12: Distributing TensorFlow Across Devices and Servers\nDownload from finelybook www.finelybook.com", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 851, "content": "2 You can even start multiple tasks in the same process. It may be useful for tests, but it is not recommended in\nproduction.\n3 It is the next version of Google\u2019s internal Stubby service, which Google has used successfully for over a decade.\nSee http://grpc.io/ for more details.\nyou want.2 If you have several servers on one machine, you will need to ensure that\nthey don\u2019t all try to grab all the RAM of every GPU, as explained earlier. For example,\nin Figure 12-6 the \"ps\" task does not see the GPU devices, since presumably its pro\u2010\ncess was launched with CUDA_VISIBLE_DEVICES=\"\". Note that the CPU is shared by\nall tasks located on the same machine.\nIf you want the process to do nothing other than run the TensorFlow server, you can\nblock the main thread by telling it to wait for the server to finish using the join() \nmethod (otherwise the server will be killed as soon as your main thread exits). Since\nthere is currently no way to stop the server, this will actually block forever:", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 852, "content": "there is currently no way to stop the server, this will actually block forever:\nserver.join() # blocks until the server stops (i.e., never)\nOpening a Session\nOnce all the tasks are up and running (doing nothing yet), you can open a session on\nany of the servers, from a client located in any process on any machine (even from a\nprocess running one of the tasks), and use that session like a regular local session. For\nexample:\na = tf.constant(1.0)\nb = a + 2\nc = a * 3\nwith tf.Session(\"grpc://machine-b.example.com:2222\") as sess:\n print(c.eval()) # 9.0\nThis client code first creates a simple graph, then opens a session on the TensorFlow\nserver located on machine B (which we will call the master), and instructs it to evalu\u2010\nate c. The master starts by placing the operations on the appropriate devices. In this\nexample, since we did not pin any operation on any device, the master simply places\nthem all on its own default device\u2014in this case, machine B\u2019s GPU device. Then it just", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 853, "content": "them all on its own default device\u2014in this case, machine B\u2019s GPU device. Then it just\nevaluates c as instructed by the client, and it returns the result.\nThe Master and Worker Services\nThe client uses the gRPC protocol (Google Remote Procedure Call) to communicate\nwith the server. This is an efficient open source framework to call remote functions\nand get their outputs across a variety of platforms and languages.3 It is based on\nHTTP2, which opens a connection and leaves it open during the whole session,\nallowing efficient bidirectional communication once the connection is established.\nMultiple Devices Across Multiple Servers \n| \n325\nDownload from finelybook www.finelybook.com", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 854, "content": "Data is transmitted in the form of protocol buffers, another open source Google tech\u2010\nnology. This is a lightweight binary data interchange format.\nAll servers in a TensorFlow cluster may communicate with any\nother server in the cluster, so make sure to open the appropriate\nports on your firewall.\nEvery TensorFlow server provides two services: the master service and the worker ser\u2010\nvice. The master service allows clients to open sessions and use them to run graphs. It\ncoordinates the computations across tasks, relying on the worker service to actually\nexecute computations on other tasks and get their results.\nThis architecture gives you a lot of flexibility. One client can connect to multiple\nservers by opening multiple sessions in different threads. One server can handle mul\u2010\ntiple sessions simultaneously from one or more clients. You can run one client per\ntask (typically within the same process), or just one client to control all tasks. All\noptions are open.", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 855, "content": "options are open.\nPinning Operations Across Tasks\nYou can use device blocks to pin operations on any device managed by any task, by\nspecifying the job name, task index, device type, and device index. For example, the\nfollowing code pins a to the CPU of the first task in the \"ps\" job (that\u2019s the CPU on\nmachine A), and it pins b to the second GPU managed by the first task of the\n\"worker\" job (that\u2019s GPU #1 on machine A). Finally, c is not pinned to any device, so\nthe master places it on its own default device (machine B\u2019s GPU #0 device).\nwith tf.device(\"/job:ps/task:0/cpu:0\")\n a = tf.constant(1.0)\nwith tf.device(\"/job:worker/task:0/gpu:1\")\n b = a + 2\nc = a + b\nAs earlier, if you omit the device type and index, TensorFlow will default to the task\u2019s\ndefault device; for example, pinning an operation to \"/job:ps/task:0\" will place it\non the default device of the first task of the \"ps\" job (machine A\u2019s CPU). If you also", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 856, "content": "on the default device of the first task of the \"ps\" job (machine A\u2019s CPU). If you also\nomit the task index (e.g., \"/job:ps\"), TensorFlow defaults to \"/task:0\". If you omit\nthe job name and the task index, TensorFlow defaults to the session\u2019s master task.\n326 \n| \nChapter 12: Distributing TensorFlow Across Devices and Servers\nDownload from finelybook www.finelybook.com", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 857, "content": "Sharding Variables Across Multiple Parameter Servers\nAs we will see shortly, a common pattern when training a neural network on a dis\u2010\ntributed setup is to store the model parameters on a set of parameter servers (i.e., the\ntasks in the \"ps\" job) while other tasks focus on computations (i.e., the tasks in the\n\"worker\" job). For large models with millions of parameters, it is useful to shard\nthese parameters across multiple parameter servers, to reduce the risk of saturating a\nsingle parameter server\u2019s network card. If you were to manually pin every variable to\na different parameter server, it would be quite tedious. Fortunately, TensorFlow pro\u2010\nvides the replica_device_setter() function, which distributes variables across all\nthe \"ps\" tasks in a round-robin fashion. For example, the following code pins five\nvariables to two parameter servers:\nwith tf.device(tf.train.replica_device_setter(ps_tasks=2):\n v1 = tf.Variable(1.0) # pinned to /job:ps/task:0", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 858, "content": "v1 = tf.Variable(1.0) # pinned to /job:ps/task:0\n v2 = tf.Variable(2.0) # pinned to /job:ps/task:1\n v3 = tf.Variable(3.0) # pinned to /job:ps/task:0\n v4 = tf.Variable(4.0) # pinned to /job:ps/task:1\n v5 = tf.Variable(5.0) # pinned to /job:ps/task:0\nInstead of passing the number of ps_tasks, you can pass the cluster spec clus\nter=cluster_spec and TensorFlow will simply count the number of tasks in the \"ps\"\njob.\nIf you create other operations in the block, beyond just variables, TensorFlow auto\u2010\nmatically pins them to \"/job:worker\", which will default to the first device managed\nby the first task in the \"worker\" job. You can pin them to another device by setting\nthe worker_device parameter, but a better approach is to use embedded device\nblocks. An inner device block can override the job, task, or device defined in an outer\nblock. For example:\nwith tf.device(tf.train.replica_device_setter(ps_tasks=2)):", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 859, "content": "block. For example:\nwith tf.device(tf.train.replica_device_setter(ps_tasks=2)):\n v1 = tf.Variable(1.0) # pinned to /job:ps/task:0 (+ defaults to /cpu:0)\n v2 = tf.Variable(2.0) # pinned to /job:ps/task:1 (+ defaults to /cpu:0)\n v3 = tf.Variable(3.0) # pinned to /job:ps/task:0 (+ defaults to /cpu:0)\n [...]\n s = v1 + v2 # pinned to /job:worker (+ defaults to task:0/gpu:0)\n with tf.device(\"/gpu:1\"):\n p1 = 2 * s # pinned to /job:worker/gpu:1 (+ defaults to /task:0)\n with tf.device(\"/task:1\"):\n p2 = 3 * s # pinned to /job:worker/task:1/gpu:1\nThis example assumes that the parameter servers are CPU-only,\nwhich is typically the case since they only need to store and com\u2010\nmunicate parameters, not perform intensive computations.\nMultiple Devices Across Multiple Servers \n| \n327\nDownload from finelybook www.finelybook.com", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 860, "content": "Sharing State Across Sessions Using Resource Containers\nWhen you are using a plain local session (not the distributed kind), each variable\u2019s\nstate is managed by the session itself; as soon as it ends, all variable values are lost.\nMoreover, multiple local sessions cannot share any state, even if they both run the\nsame graph; each session has its own copy of every variable (as we discussed in Chap\u2010\nter 9). In contrast, when you are using distributed sessions, variable state is managed\nby resource containers located on the cluster itself, not by the sessions. So if you create\na variable named x using one client session, it will automatically be available to any\nother session on the same cluster (even if both sessions are connected to a different\nserver). For example, consider the following client code:\n# simple_client.py\nimport tensorflow as tf\nimport sys\nx = tf.Variable(0.0, name=\"x\")\nincrement_x = tf.assign(x, x + 1)\nwith tf.Session(sys.argv[1]) as sess:", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 861, "content": "increment_x = tf.assign(x, x + 1)\nwith tf.Session(sys.argv[1]) as sess:\n if sys.argv[2:]==[\"init\"]:\n sess.run(x.initializer)\n sess.run(increment_x)\n print(x.eval())\nLet\u2019s suppose you have a TensorFlow cluster up and running on machines A and B,\nport 2222. You could launch the client, have it open a session with the server on\nmachine A, and tell it to initialize the variable, increment it, and print its value by\nlaunching the following command:\n$ python3 simple_client.py grpc://machine-a.example.com:2222 init\n1.0\nNow if you launch the client with the following command, it will connect to the\nserver on machine B and magically reuse the same variable x (this time we don\u2019t ask\nthe server to initialize the variable):\n$ python3 simple_client.py grpc://machine-b.example.com:2222\n2.0\nThis feature cuts both ways: it\u2019s great if you want to share variables across multiple\nsessions, but if you want to run completely independent computations on the same", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 862, "content": "sessions, but if you want to run completely independent computations on the same\ncluster you will have to be careful not to use the same variable names by accident.\nOne way to ensure that you won\u2019t have name clashes is to wrap all of your construc\u2010\ntion phase inside a variable scope with a unique name for each computation, for\nexample:\nwith tf.variable_scope(\"my_problem_1\"):\n [...] # Construction phase of problem 1\n328 \n| \nChapter 12: Distributing TensorFlow Across Devices and Servers\nDownload from finelybook www.finelybook.com", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 863, "content": "A better option is to use a container block:\nwith tf.container(\"my_problem_1\"):\n [...] # Construction phase of problem 1\nThis will use a container dedicated to problem #1, instead of the default one (whose\nname is an empty string \"\"). One advantage is that variable names remain nice and\nshort. Another advantage is that you can easily reset a named container. For example,\nthe following command will connect to the server on machine A and ask it to reset\nthe container named \"my_problem_1\", which will free all the resources this container\nused (and also close all sessions open on the server). Any variable managed by this\ncontainer must be initialized before you can use it again:\ntf.Session.reset(\"grpc://machine-a.example.com:2222\", [\"my_problem_1\"])\nResource containers make it easy to share variables across sessions in flexible ways.\nFor example, Figure 12-7 shows four clients running different graphs on the same", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 864, "content": "For example, Figure 12-7 shows four clients running different graphs on the same\ncluster, but sharing some variables. Clients A and B share the same variable x man\u2010\naged by the default container, while clients C and D share another variable named x\nmanaged by the container named \"my_problem_1\". Note that client C even uses vari\u2010\nables from both containers.\nFigure 12-7. Resource containers\nResource containers also take care of preserving the state of other stateful operations,\nnamely queues and readers. Let\u2019s take a look at queues first.\nAsynchronous Communication Using TensorFlow Queues\nQueues are another great way to exchange data between multiple sessions; for exam\u2010\nple, one common use case is to have a client create a graph that loads the training data\nand pushes it into a queue, while another client creates a graph that pulls the data\nfrom the queue and trains a model (see Figure 12-8). This can speed up training con\u2010\nMultiple Devices Across Multiple Servers \n| \n329", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 865, "content": "Multiple Devices Across Multiple Servers \n| \n329\nDownload from finelybook www.finelybook.com", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 866, "content": "siderably because the training operations don\u2019t have to wait for the next mini-batch at\nevery step.\nFigure 12-8. Using queues to load the training data asynchronously\nTensorFlow provides various kinds of queues. The simplest kind is the first-in first-\nout (FIFO) queue. For example, the following code creates a FIFO queue that can\nstore up to 10 tensors containing two float values each:\nq = tf.FIFOQueue(capacity=10, dtypes=[tf.float32], shapes=[[2]],\n name=\"q\", shared_name=\"shared_q\")\nTo share variables across sessions, all you had to do was to specify\nthe same name and container on both ends. With queues Tensor\u2010\nFlow does not use the name attribute but instead uses shared_name,\nso it is important to specify it (even if it is the same as the name).\nAnd, of course, use the same container.\nEnqueuing data\nTo push data to a queue, you must create an enqueue operation. For example, the fol\u2010\nlowing code pushes three training instances to the queue:\n# training_data_loader.py", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 867, "content": "lowing code pushes three training instances to the queue:\n# training_data_loader.py\nimport tensorflow as tf\nq = [...]\ntraining_instance = tf.placeholder(tf.float32, shape=(2))\nenqueue = q.enqueue([training_instance])\nwith tf.Session(\"grpc://machine-a.example.com:2222\") as sess:\n sess.run(enqueue, feed_dict={training_instance: [1., 2.]})\n sess.run(enqueue, feed_dict={training_instance: [3., 4.]})\n sess.run(enqueue, feed_dict={training_instance: [5., 6.]})\n330 \n| \nChapter 12: Distributing TensorFlow Across Devices and Servers\nDownload from finelybook www.finelybook.com", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 868, "content": "Instead of enqueuing instances one by one, you can enqueue several at a time using\nan enqueue_many operation:\n[...]\ntraining_instances = tf.placeholder(tf.float32, shape=(None, 2))\nenqueue_many = q.enqueue([training_instances])\nwith tf.Session(\"grpc://machine-a.example.com:2222\") as sess:\n sess.run(enqueue_many,\n feed_dict={training_instances: [[1., 2.], [3., 4.], [5., 6.]]})\nBoth examples enqueue the same three tensors to the queue.\nDequeuing data\nTo pull the instances out of the queue, on the other end, you need to use a dequeue\noperation:\n# trainer.py\nimport tensorflow as tf\nq = [...]\ndequeue = q.dequeue()\nwith tf.Session(\"grpc://machine-a.example.com:2222\") as sess:\n print(sess.run(dequeue)) # [1., 2.]\n print(sess.run(dequeue)) # [3., 4.]\n print(sess.run(dequeue)) # [5., 6.]\nIn general you will want to pull a whole mini-batch at once, instead of pulling just\none instance at a time. To do so, you must use a dequeue_many operation, specifying", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 869, "content": "one instance at a time. To do so, you must use a dequeue_many operation, specifying\nthe mini-batch size:\n[...]\nbatch_size = 2\ndequeue_mini_batch= q.dequeue_many(batch_size)\nwith tf.Session(\"grpc://machine-a.example.com:2222\") as sess:\n print(sess.run(dequeue_mini_batch)) # [[1., 2.], [4., 5.]]\n print(sess.run(dequeue_mini_batch)) # blocked waiting for another instance\nWhen a queue is full, the enqueue operation will block until items are pulled out by a\ndequeue operation. Similarly, when a queue is empty (or you are using\ndequeue_many() and there are fewer items than the mini-batch size), the dequeue\noperation will block until enough items are pushed into the queue using an enqueue\noperation.\nMultiple Devices Across Multiple Servers \n| \n331\nDownload from finelybook www.finelybook.com", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 870, "content": "Queues of tuples\nEach item in a queue can be a tuple of tensors (of various types and shapes) instead of\njust a single tensor. For example, the following queue stores pairs of tensors, one of\ntype int32 and shape (), and the other of type float32 and shape [3,2]:\nq = tf.FIFOQueue(capacity=10, dtypes=[tf.int32, tf.float32], shapes=[[],[3,2]],\n name=\"q\", shared_name=\"shared_q\")\nThe enqueue operation must be given pairs of tensors (note that each pair represents\nonly one item in the queue):\na = tf.placeholder(tf.int32, shape=())\nb = tf.placeholder(tf.float32, shape=(3, 2))\nenqueue = q.enqueue((a, b))\nwith tf.Session([...]) as sess:\n sess.run(enqueue, feed_dict={a: 10, b:[[1., 2.], [3., 4.], [5., 6.]]})\n sess.run(enqueue, feed_dict={a: 11, b:[[2., 4.], [6., 8.], [0., 2.]]})\n sess.run(enqueue, feed_dict={a: 12, b:[[3., 6.], [9., 2.], [5., 8.]]})\nOn the other end, the dequeue() function now creates a pair of dequeue operations:\ndequeue_a, dequeue_b = q.dequeue()", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 871, "content": "dequeue_a, dequeue_b = q.dequeue()\nIn general, you should run these operations together:\nwith tf.Session([...]) as sess:\n a_val, b_val = sess.run([dequeue_a, dequeue_b])\n print(a_val) # 10\n print(b_val) # [[1., 2.], [3., 4.], [5., 6.]]\nIf you run dequeue_a on its own, it will dequeue a pair and return\nonly the first element; the second element will be lost (and simi\u2010\nlarly, if you run dequeue_b on its own, the first element will be\nlost).\nThe dequeue_many() function also returns a pair of operations:\nbatch_size = 2\ndequeue_as, dequeue_bs = q.dequeue_many(batch_size)\nYou can use it as you would expect:\nwith tf.Session([...]) as sess:\n a, b = sess.run([dequeue_a, dequeue_b])\n print(a) # [10, 11]\n print(b) # [[[1., 2.], [3., 4.], [5., 6.]], [[2., 4.], [6., 8.], [0., 2.]]]\n a, b = sess.run([dequeue_a, dequeue_b]) # blocked waiting for another pair\n332 \n| \nChapter 12: Distributing TensorFlow Across Devices and Servers\nDownload from finelybook www.finelybook.com", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 872, "content": "Closing a queue\nIt is possible to close a queue to signal to the other sessions that no more data will be\nenqueued:\nclose_q = q.close()\nwith tf.Session([...]) as sess:\n [...]\n sess.run(close_q)\nSubsequent executions of enqueue or enqueue_many operations will raise an excep\u2010\ntion. By default, any pending enqueue request will be honored, unless you call\nq.close(cancel_pending_enqueues=True).\nSubsequent executions of dequeue or dequeue_many operations will continue to suc\u2010\nceed as long as there are items in the queue, but they will fail when there are not\nenough items left in the queue. If you are using a dequeue_many operation and there\nare a few instances left in the queue, but fewer than the mini-batch size, they will be\nlost. You may prefer to use a dequeue_up_to operation instead; it behaves exactly like\ndequeue_many except when a queue is closed and there are fewer than batch_size\ninstances left in the queue, in which case it just returns them.\nRandomShuffleQueue", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 873, "content": "instances left in the queue, in which case it just returns them.\nRandomShuffleQueue\nTensorFlow also supports a couple more types of queues, including RandomShuffle\nQueue, which can be used just like a FIFOQueue except that items are dequeued in a\nrandom order. This can be useful to shuffle training instances at each epoch during\ntraining. First, let\u2019s create the queue:\nq = tf.RandomShuffleQueue(capacity=50, min_after_dequeue=10,\n dtypes=[tf.float32], shapes=[()],\n name=\"q\", shared_name=\"shared_q\")\nThe min_after_dequeue specifies the minimum number of items that must remain in\nthe queue after a dequeue operation. This ensures that there will be enough instances\nin the queue to have enough randomness (once the queue is closed, the\nmin_after_dequeue limit is ignored). Now suppose that you enqueued 22 items in\nthis queue (floats 1. to 22.). Here is how you could dequeue them:\ndequeue = q.dequeue_many(5)\nwith tf.Session([...]) as sess:", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 874, "content": "dequeue = q.dequeue_many(5)\nwith tf.Session([...]) as sess:\n print(sess.run(dequeue)) # [ 20. 15. 11. 12. 4.] (17 items left)\n print(sess.run(dequeue)) # [ 5. 13. 6. 0. 17.] (12 items left)\n print(sess.run(dequeue)) # 12 - 5 < 10: blocked waiting for 3 more instances\nMultiple Devices Across Multiple Servers \n| \n333\nDownload from finelybook www.finelybook.com", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 875, "content": "PaddingFifoQueue\nA PaddingFIFOQueue can also be used just like a FIFOQueue except that it accepts ten\u2010\nsors of variable sizes along any dimension (but with a fixed rank). When you are\ndequeuing them with a dequeue_many or dequeue_up_to operation, each tensor is\npadded with zeros along every variable dimension to make it the same size as the\nlargest tensor in the mini-batch. For example, you could enqueue 2D tensors (matri\u2010\nces) of arbitrary sizes:\nq = tf.PaddingFIFOQueue(capacity=50, dtypes=[tf.float32], shapes=[(None, None)]\n name=\"q\", shared_name=\"shared_q\")\nv = tf.placeholder(tf.float32, shape=(None, None))\nenqueue = q.enqueue([v])\nwith tf.Session([...]) as sess:\n sess.run(enqueue, feed_dict={v: [[1., 2.], [3., 4.], [5., 6.]]}) # 3x2\n sess.run(enqueue, feed_dict={v: [[1.]]}) # 1x1\n sess.run(enqueue, feed_dict={v: [[7., 8., 9., 5.], [6., 7., 8., 9.]]}) # 2x4", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 876, "content": "sess.run(enqueue, feed_dict={v: [[7., 8., 9., 5.], [6., 7., 8., 9.]]}) # 2x4\nIf we just dequeue one item at a time, we get the exact same tensors that were\nenqueued. But if we dequeue several items at a time (using dequeue_many() or\ndequeue_up_to()), the queue automatically pads the tensors appropriately. For exam\u2010\nple, if we dequeue all three items at once, all tensors will be padded with zeros to\nbecome 3 \u00d7 4 tensors, since the maximum size for the first dimension is 3 (first item)\nand the maximum size for the second dimension is 4 (third item):\n>>> q = [...]\n>>> dequeue = q.dequeue_many(3)\n>>> with tf.Session([...]) as sess:\n... print(sess.run(dequeue))\n[[[ 1. 2. 0. 0.]\n [ 3. 4. 0. 0.]\n [ 5. 6. 0. 0.]]\n [[ 1. 0. 0. 0.]\n [ 0. 0. 0. 0.]\n [ 0. 0. 0. 0.]]\n [[ 7. 8. 9. 5.]\n [ 6. 7. 8. 9.]\n [ 0. 0. 0. 0.]]]\nThis type of queue can be useful when you are dealing with variable length inputs,\nsuch as sequences of words (see Chapter 14).", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 877, "content": "such as sequences of words (see Chapter 14).\nOkay, now let\u2019s pause for a second: so far you have learned to distribute computations\nacross multiple devices and servers, share variables across sessions, and communicate\nasynchronously using queues. Before you start training neural networks, though,\nthere\u2019s one last topic we need to discuss: how to efficiently load training data.\n334 \n| \nChapter 12: Distributing TensorFlow Across Devices and Servers\nDownload from finelybook www.finelybook.com", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 878, "content": "Loading Data Directly from the Graph\nSo far we have assumed that the clients would load the training data and feed it to the\ncluster using placeholders. This is simple and works quite well for simple setups, but\nit is rather inefficient since it transfers the training data several times:\n1. From the filesystem to the client\n2. From the client to the master task\n3. Possibly from the master task to other tasks where the data is needed\nIt gets worse if you have several clients training various neural networks using the\nsame training data (for example, for hyperparameter tuning): if every client loads the\ndata simultaneously, you may end up even saturating your file server or the network\u2019s\nbandwidth.\nPreload the data into a variable\nFor datasets that can fit in memory, a better option is to load the training data once\nand assign it to a variable, then just use that variable in your graph. This is called\npreloading the training set. This way the data will be transferred only once from the", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 879, "content": "preloading the training set. This way the data will be transferred only once from the\nclient to the cluster (but it may still need to be moved around from task to task\ndepending on which operations need it). The following code shows how to load the\nfull training set into a variable:\ntraining_set_init = tf.placeholder(tf.float32, shape=(None, n_features))\ntraining_set = tf.Variable(training_set_init, trainable=False, collections=[],\n name=\"training_set\")\nwith tf.Session([...]) as sess:\n data = [...] # load the training data from the datastore\n sess.run(training_set.initializer, feed_dict={training_set_init: data})\nYou must set trainable=False so the optimizers don\u2019t try to tweak this variable. You\nshould also set collections=[] to ensure that this variable won\u2019t get added to the\nGraphKeys.GLOBAL_VARIABLES collection, which is used for saving and restoring\ncheckpoints.\nThis example assumes that all of your training set (including the", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 880, "content": "checkpoints.\nThis example assumes that all of your training set (including the\nlabels) consists only of float32 values. If that\u2019s not the case, you\nwill need one variable per type.\nReading the training data directly from the graph\nIf the training set does not fit in memory, a good solution is to use reader operations: \nthese are operations capable of reading data directly from the filesystem. This way the\nMultiple Devices Across Multiple Servers \n| \n335\nDownload from finelybook www.finelybook.com", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 881, "content": "training data never needs to flow through the clients at all. TensorFlow provides read\u2010\ners for various file formats:\n\u2022 CSV\n\u2022 Fixed-length binary records\n\u2022 TensorFlow\u2019s own TFRecords format, based on protocol buffers\nLet\u2019s look at a simple example reading from a CSV file (for other formats, please\ncheck out the API documentation). Suppose you have file named my_test.csv that\ncontains training instances, and you want to create operations to read it. Suppose it\nhas the following content, with two float features x1 and x2 and one integer target\nrepresenting a binary class:\nx1, x2, target\n1. , 2. , 0\n4. , 5 , 1\n7. , , 0\nFirst, let\u2019s create a TextLineReader to read this file. A TextLineReader opens a file\n(once we tell it which one to open) and reads lines one by one. It is a stateful opera\u2010\ntion, like variables and queues: it preserves its state across multiple runs of the graph,\nkeeping track of which file it is currently reading and what its current position is in\nthis file.", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 882, "content": "keeping track of which file it is currently reading and what its current position is in\nthis file.\nreader = tf.TextLineReader(skip_header_lines=1)\nNext, we create a queue that the reader will pull from to know which file to read next.\nWe also create an enqueue operation and a placeholder to push any filename we want\nto the queue, and we create an operation to close the queue once we have no more \nfiles to read:\nfilename_queue = tf.FIFOQueue(capacity=10, dtypes=[tf.string], shapes=[()])\nfilename = tf.placeholder(tf.string)\nenqueue_filename = filename_queue.enqueue([filename])\nclose_filename_queue = filename_queue.close()\nNow we are ready to create a read operation that will read one record (i.e., a line) at a\ntime and return a key/value pair. The key is the record\u2019s unique identifier\u2014a string\ncomposed of the filename, a colon (:), and the line number\u2014and the value is simply\na string containing the content of the line:\nkey, value = reader.read(filename_queue)", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 883, "content": "a string containing the content of the line:\nkey, value = reader.read(filename_queue)\nWe have all we need to read the file line by line! But we are not quite done yet\u2014we\nneed to parse this string to get the features and target:\nx1, x2, target = tf.decode_csv(value, record_defaults=[[-1.], [-1.], [-1]])\nfeatures = tf.stack([x1, x2])\n336 \n| \nChapter 12: Distributing TensorFlow Across Devices and Servers\nDownload from finelybook www.finelybook.com", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 884, "content": "The first line uses TensorFlow\u2019s CSV parser to extract the values from the current\nline. The default values are used when a field is missing (in this example the third\ntraining instance\u2019s x2 feature), and they are also used to determine the type of each\nfield (in this case two floats and one integer).\nFinally, we can push this training instance and its target to a RandomShuffleQueue\nthat we will share with the training graph (so it can pull mini-batches from it), and we\ncreate an operation to close that queue when we are done pushing instances to it:\ninstance_queue = tf.RandomShuffleQueue(\n capacity=10, min_after_dequeue=2,\n dtypes=[tf.float32, tf.int32], shapes=[[2],[]],\n name=\"instance_q\", shared_name=\"shared_instance_q\")\nenqueue_instance = instance_queue.enqueue([features, target])\nclose_instance_queue = instance_queue.close()\nWow! That was a lot of work just to read a file. Plus we only created the graph, so now\nwe need to run it:\nwith tf.Session([...]) as sess:", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 885, "content": "we need to run it:\nwith tf.Session([...]) as sess:\n sess.run(enqueue_filename, feed_dict={filename: \"my_test.csv\"})\n sess.run(close_filename_queue)\n try:\n while True:\n sess.run(enqueue_instance)\n except tf.errors.OutOfRangeError as ex:\n pass # no more records in the current file and no more files to read\n sess.run(close_instance_queue)\nFirst we open the session, and then we enqueue the filename \"my_test.csv\" and\nimmediately close that queue since we will not enqueue any more filenames. Then we\nrun an infinite loop to enqueue instances one by one. The enqueue_instance\ndepends on the reader reading the next line, so at every iteration a new record is read\nuntil it reaches the end of the file. At that point it tries to read the filename queue to\nknow which file to read next, and since the queue is closed it throws an OutOfRan\ngeError exception (if we did not close the queue, it would just remain blocked until", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 886, "content": "geError exception (if we did not close the queue, it would just remain blocked until\nwe pushed another filename or closed the queue). Lastly, we close the instance queue\nso that the training operations pulling from it won\u2019t get blocked forever. Figure 12-9\nsummarizes what we have learned; it represents a typical graph for reading training\ninstances from a set of CSV files.\nMultiple Devices Across Multiple Servers \n| \n337\nDownload from finelybook www.finelybook.com", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 887, "content": "Figure 12-9. A graph dedicated to reading training instances from CSV files\nIn the training graph, you need to create the shared instance queue and simply\ndequeue mini-batches from it:\ninstance_queue = tf.RandomShuffleQueue([...], shared_name=\"shared_instance_q\")\nmini_batch_instances, mini_batch_targets = instance_queue.dequeue_up_to(2)\n[...] # use the mini_batch instances and targets to build the training graph\ntraining_op = [...]\nwith tf.Session([...]) as sess:\n try:\n for step in range(max_steps):\n sess.run(training_op)\n except tf.errors.OutOfRangeError as ex:\n pass # no more training instances\nIn this example, the first mini-batch will contain the first two instances of the CSV\nfile, and the second mini-batch will contain the last instance.\nTensorFlow queues don\u2019t handle sparse tensors well, so if your\ntraining instances are sparse you should parse the records after the\ninstance queue.", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 888, "content": "training instances are sparse you should parse the records after the\ninstance queue.\nThis architecture will only use one thread to read records and push them to the\ninstance queue. You can get a much higher throughput by having multiple threads\nread simultaneously from multiple files using multiple readers. Let\u2019s see how.\nMultithreaded readers using a Coordinator and a QueueRunner\nTo have multiple threads read instances simultaneously, you could create Python\nthreads (using the threading module) and manage them yourself. However, Tensor\u2010\nFlow provides some tools to make this simpler: the Coordinator class and the QueueR\nunner class.\n338 \n| \nChapter 12: Distributing TensorFlow Across Devices and Servers\nDownload from finelybook www.finelybook.com", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 889, "content": "A Coordinator is a very simple object whose sole purpose is to coordinate stopping\nmultiple threads. First you create a Coordinator:\ncoord = tf.train.Coordinator()\nThen you give it to all threads that need to stop jointly, and their main loop looks like\nthis:\nwhile not coord.should_stop():\n [...] # do something\nAny thread can request that every thread stop by calling the Coordinator\u2019s\nrequest_stop() method:\ncoord.request_stop()\nEvery thread will stop as soon as it finishes its current iteration. You can wait for all of\nthe threads to finish by calling the Coordinator\u2019s join() method, passing it the list of\nthreads:\ncoord.join(list_of_threads)\nA QueueRunner runs multiple threads that each run an enqueue operation repeatedly,\nfilling up a queue as fast as possible. As soon as the queue is closed, the next thread\nthat tries to push an item to the queue will get an OutOfRangeError; this thread\ncatches the error and immediately tells other threads to stop using a Coordinator.", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 890, "content": "catches the error and immediately tells other threads to stop using a Coordinator.\nThe following code shows how you can use a QueueRunner to have five threads read\u2010\ning instances simultaneously and pushing them to an instance queue:\n[...] # same construction phase as earlier\nqueue_runner = tf.train.QueueRunner(instance_queue, [enqueue_instance] * 5)\nwith tf.Session() as sess:\n sess.run(enqueue_filename, feed_dict={filename: \"my_test.csv\"})\n sess.run(close_filename_queue)\n coord = tf.train.Coordinator()\n enqueue_threads = queue_runner.create_threads(sess, coord=coord, start=True)\nThe first line creates the QueueRunner and tells it to run five threads, all running the\nsame enqueue_instance operation repeatedly. Then we start a session and we\nenqueue the name of the files to read (in this case just \"my_test.csv\"). Next we cre\u2010\nate a Coordinator that the QueueRunner will use to stop gracefully, as just explained.", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 891, "content": "ate a Coordinator that the QueueRunner will use to stop gracefully, as just explained.\nFinally, we tell the QueueRunner to create the threads and start them. The threads will\nread all training instances and push them to the instance queue, and then they will all\nstop gracefully.\nThis will be a bit more efficient than earlier, but we can do better. Currently all\nthreads are reading from the same file. We can make them read simultaneously from\nseparate files instead (assuming the training data is sharded across multiple CSV files)\nby creating multiple readers (see Figure 12-10).\nMultiple Devices Across Multiple Servers \n| \n339\nDownload from finelybook www.finelybook.com", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 892, "content": "Figure 12-10. Reading simultaneously from multiple files\nFor this we need to write a small function to create a reader and the nodes that will\nread and push one instance to the instance queue:\ndef read_and_push_instance(filename_queue, instance_queue):\n reader = tf.TextLineReader(skip_header_lines=1)\n key, value = reader.read(filename_queue)\n x1, x2, target = tf.decode_csv(value, record_defaults=[[-1.], [-1.], [-1]])\n features = tf.stack([x1, x2])\n enqueue_instance = instance_queue.enqueue([features, target])\n return enqueue_instance\nNext we define the queues:\nfilename_queue = tf.FIFOQueue(capacity=10, dtypes=[tf.string], shapes=[()])\nfilename = tf.placeholder(tf.string)\nenqueue_filename = filename_queue.enqueue([filename])\nclose_filename_queue = filename_queue.close()\ninstance_queue = tf.RandomShuffleQueue([...])\nAnd finally we create the QueueRunner, but this time we give it a list of different", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 893, "content": "And finally we create the QueueRunner, but this time we give it a list of different\nenqueue operations. Each operation will use a different reader, so the threads will\nsimultaneously read from different files:\nread_and_enqueue_ops = [\n read_and_push_instance(filename_queue, instance_queue)\n for i in range(5)]\nqueue_runner = tf.train.QueueRunner(instance_queue, read_and_enqueue_ops)\nThe execution phase is then the same as before: first push the names of the files to\nread, then create a Coordinator and create and start the QueueRunner threads. This\ntime all threads will read from different files simultaneously until all files are read\nentirely, and then the QueueRunner will close the instance queue so that other ops\npulling from it don\u2019t get blocked.\n340 \n| \nChapter 12: Distributing TensorFlow Across Devices and Servers\nDownload from finelybook www.finelybook.com", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 894, "content": "Other convenience functions\nTensorFlow also offers a few convenience functions to simplify some common tasks\nwhen reading training instances. We will go over just a few (see the API documenta\u2010\ntion for the full list).\nThe string_input_producer() takes a 1D tensor containing a list of filenames, cre\u2010\nates a thread that pushes one filename at a time to the filename queue, and then\ncloses the queue. If you specify a number of epochs, it will cycle through the file\u2010\nnames once per epoch before closing the queue. By default, it shuffles the filenames at\neach epoch. It creates a QueueRunner to manage its thread, and adds it to the Graph\nKeys.QUEUE_RUNNERS collection. To start every QueueRunner in that collection, you\ncan call the tf.train.start_queue_runners() function. Note that if you forget to\nstart the QueueRunner, the filename queue will be open and empty, and your readers\nwill be blocked forever.\nThere are a few other producer functions that similarly create a queue and a corre\u2010", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 895, "content": "There are a few other producer functions that similarly create a queue and a corre\u2010\nsponding QueueRunner for running an enqueue operation (e.g., input_producer(),\nrange_input_producer(), and slice_input_producer()).\nThe shuffle_batch() function takes a list of tensors (e.g., [features, target]) and\ncreates:\n\u2022 A RandomShuffleQueue\n\u2022 A QueueRunner to enqueue the tensors to the queue (added to the Graph\nKeys.QUEUE_RUNNERS collection)\n\u2022 A dequeue_many operation to extract a mini-batch from the queue\nThis makes it easy to manage in a single process a multithreaded input pipeline feed\u2010\ning a queue and a training pipeline reading mini-batches from that queue. Also check\nout the batch(), batch_join(), and shuffle_batch_join() functions that provide\nsimilar functionality.\nOkay! You now have all the tools you need to start training and running neural net\u2010\nworks efficiently across multiple devices and servers on a TensorFlow cluster. Let\u2019s\nreview what you have learned:", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 896, "content": "review what you have learned:\n\u2022 Using multiple GPU devices\n\u2022 Setting up and starting a TensorFlow cluster\n\u2022 Distributing computations across multiple devices and servers\n\u2022 Sharing variables (and other stateful ops such as queues and readers) across ses\u2010\nsions using containers\n\u2022 Coordinating multiple graphs working asynchronously using queues\nMultiple Devices Across Multiple Servers \n| \n341\nDownload from finelybook www.finelybook.com", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 897, "content": "4 Not 100% linear if you wait for all devices to finish, since the total time will be the time taken by the slowest\ndevice.\n\u2022 Reading inputs efficiently using readers, queue runners, and coordinators\nNow let\u2019s use all of this to parallelize neural networks!\nParallelizing Neural Networks on a TensorFlow Cluster\nIn this section, first we will look at how to parallelize several neural networks by sim\u2010\nply placing each one on a different device. Then we will look at the much trickier\nproblem of training a single neural network across multiple devices and servers.\nOne Neural Network per Device\nThe most trivial way to train and run neural networks on a TensorFlow cluster is to\ntake the exact same code you would use for a single device on a single machine, and\nspecify the master server\u2019s address when creating the session. That\u2019s it\u2014you\u2019re done!\nYour code will be running on the server\u2019s default device. You can change the device", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 898, "content": "Your code will be running on the server\u2019s default device. You can change the device\nthat will run your graph simply by putting your code\u2019s construction phase within a\ndevice block.\nBy running several client sessions in parallel (in different threads or different pro\u2010\ncesses), connecting them to different servers, and configuring them to use different\ndevices, you can quite easily train or run many neural networks in parallel, across all\ndevices and all machines in your cluster (see Figure 12-11). The speedup is almost\nlinear.4 Training 100 neural networks across 50 servers with 2 GPUs each will not take\nmuch longer than training just 1 neural network on 1 GPU.\nFigure 12-11. Training one neural network per device\n342 \n| \nChapter 12: Distributing TensorFlow Across Devices and Servers\nDownload from finelybook www.finelybook.com", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 899, "content": "This solution is perfect for hyperparameter tuning: each device in the cluster will\ntrain a different model with its own set of hyperparameters. The more computing\npower you have, the larger the hyperparameter space you can explore.\nIt also works perfectly if you host a web service that receives a large number of queries\nper second (QPS) and you need your neural network to make a prediction for each\nquery. Simply replicate the neural network across all devices on the cluster and dis\u2010\npatch queries across all devices. By adding more servers you can handle an unlimited\nnumber of QPS (however, this will not reduce the time it takes to process a single\nrequest since it will still have to wait for a neural network to make a prediction).\nAnother option is to serve your neural networks using TensorFlow\nServing. It is an open source system, released by Google in Febru\u2010\nary 2016, designed to serve a high volume of queries to Machine", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 900, "content": "ary 2016, designed to serve a high volume of queries to Machine\nLearning models (typically built with TensorFlow). It handles\nmodel versioning, so you can easily deploy a new version of your\nnetwork to production, or experiment with various algorithms\nwithout interrupting your service, and it can sustain a heavy load\nby adding more servers. For more details, check out https://tensor\nflow.github.io/serving/.\nIn-Graph Versus Between-Graph Replication\nYou can also parallelize the training of a large ensemble of neural networks by simply\nplacing every neural network on a different device (ensembles were introduced in\nChapter 7). However, once you want to run the ensemble, you will need to aggregate\nthe individual predictions made by each neural network to produce the ensemble\u2019s\nprediction, and this requires a bit of coordination.\nThere are two major approaches to handling a neural network ensemble (or any other\ngraph that contains large chunks of independent computations):", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 901, "content": "graph that contains large chunks of independent computations):\n\u2022 You can create one big graph, containing every neural network, each pinned to a\ndifferent device, plus the computations needed to aggregate the individual pre\u2010\ndictions from all the neural networks (see Figure 12-12). Then you just create\none session to any server in the cluster and let it take care of everything (includ\u2010\ning waiting for all individual predictions to be available before aggregating them).\nThis approach is called in-graph replication.\nParallelizing Neural Networks on a TensorFlow Cluster \n| \n343\nDownload from finelybook www.finelybook.com", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 902, "content": "Figure 12-12. In-graph replication\n\u2022 Alternatively, you can create one separate graph for each neural network and\nhandle synchronization between these graphs yourself. This approach is called\nbetween-graph replication. One typical implementation is to coordinate the exe\u2010\ncution of these graphs using queues (see Figure 12-13). A set of clients handles\none neural network each, reading from its dedicated input queue, and writing to\nits dedicated prediction queue. Another client is in charge of reading the inputs\nand pushing them to all the input queues (copying all inputs to every queue).\nFinally, one last client is in charge of reading one prediction from each prediction\nqueue and aggregating them to produce the ensemble\u2019s prediction.\nFigure 12-13. Between-graph replication\n344 \n| \nChapter 12: Distributing TensorFlow Across Devices and Servers\nDownload from finelybook www.finelybook.com", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 903, "content": "These solutions have their pros and cons. In-graph replication is somewhat simpler to\nimplement since you don\u2019t have to manage multiple clients and multiple queues.\nHowever, between-graph replication is a bit easier to organize into well-bounded and\neasy-to-test modules. Moreover, it gives you more flexibility. For example, you could\nadd a dequeue timeout in the aggregator client so that the ensemble would not fail\neven if one of the neural network clients crashes or if one neural network takes too\nlong to produce its prediction. TensorFlow lets you specify a timeout when calling the\nrun() function by passing a RunOptions with timeout_in_ms:\nwith tf.Session([...]) as sess:\n [...]\n run_options = tf.RunOptions()\n run_options.timeout_in_ms = 1000 # 1s timeout\n try:\n pred = sess.run(dequeue_prediction, options=run_options)\n except tf.errors.DeadlineExceededError as ex:\n [...] # the dequeue operation timed out after 1s", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 904, "content": "[...] # the dequeue operation timed out after 1s\nAnother way you can specify a timeout is to set the session\u2019s operation_time\nout_in_ms configuration option, but in this case the run() function times out if any\noperation takes longer than the timeout delay:\nconfig = tf.ConfigProto()\nconfig.operation_timeout_in_ms = 1000 # 1s timeout for every operation\nwith tf.Session([...], config=config) as sess:\n [...]\n try:\n pred = sess.run(dequeue_prediction)\n except tf.errors.DeadlineExceededError as ex:\n [...] # the dequeue operation timed out after 1s\nModel Parallelism\nSo far we have run each neural network on a single device. What if we want to run a\nsingle neural network across multiple devices? This requires chopping your model\ninto separate chunks and running each chunk on a different device. This is called\nmodel parallelism. Unfortunately, model parallelism turns out to be pretty tricky, and", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 905, "content": "model parallelism. Unfortunately, model parallelism turns out to be pretty tricky, and\nit really depends on the architecture of your neural network. For fully connected net\u2010\nworks, there is generally not much to be gained from this approach (see\nFigure 12-14). Intuitively, it may seem that an easy way to split the model is to place\neach layer on a different device, but this does not work since each layer needs to wait\nfor the output of the previous layer before it can do anything. So perhaps you can\nslice it vertically\u2014for example, with the left half of each layer on one device, and the\nright part on another device? This is slightly better, since both halves of each layer can\nindeed work in parallel, but the problem is that each half of the next layer requires the\noutput of both halves, so there will be a lot of cross-device communication (repre\u2010\nParallelizing Neural Networks on a TensorFlow Cluster \n| \n345\nDownload from finelybook www.finelybook.com", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 906, "content": "sented by the dashed arrows). This is likely to completely cancel out the benefit of the\nparallel computation, since cross-device communication is slow (especially if it is\nacross separate machines).\nFigure 12-14. Splitting a fully connected neural network\nHowever, as we will see in Chapter 13, some neural network architectures, such as\nconvolutional neural networks, contain layers that are only partially connected to the\nlower layers, so it is much easier to distribute chunks across devices in an efficient\nway.\nFigure 12-15. Splitting a partially connected neural network\nMoreover, as we will see in Chapter 14, some deep recurrent neural networks are\ncomposed of several layers of memory cells (see the left side of Figure 12-16). A cell\u2019s\noutput at time t is fed back to its input at time t + 1 (as you can see more clearly on\nthe right side of Figure 12-16). If you split such a network horizontally, placing each", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 907, "content": "the right side of Figure 12-16). If you split such a network horizontally, placing each\nlayer on a different device, then at the first step only one device will be active, at the\nsecond step two will be active, and by the time the signal propagates to the output\nlayer all devices will be active simultaneously. There is still a lot of cross-device com\u2010\nmunication going on, but since each cell may be fairly complex, the benefit of run\u2010\nning multiple cells in parallel often outweighs the communication penalty.\n346 \n| \nChapter 12: Distributing TensorFlow Across Devices and Servers\nDownload from finelybook www.finelybook.com", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 908, "content": "Figure 12-16. Splitting a deep recurrent neural network\nIn short, model parallelism can speed up running or training some types of neural\nnetworks, but not all, and it requires special care and tuning, such as making sure\nthat devices that need to communicate the most run on the same machine.\nData Parallelism\nAnother way to parallelize the training of a neural network is to replicate it on each\ndevice, run a training step simultaneously on all replicas using a different mini-batch\nfor each, and then aggregate the gradients to update the model parameters. This is\ncalled data parallelism (see Figure 12-17).\nFigure 12-17. Data parallelism\nThere are two variants of this approach: synchronous updates and asynchronous\nupdates.\nParallelizing Neural Networks on a TensorFlow Cluster \n| \n347\nDownload from finelybook www.finelybook.com", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 909, "content": "5 This name is slightly confusing since it sounds like some replicas are special, doing nothing. In reality, all rep\u2010\nlicas are equivalent: they all work hard to be among the fastest at each training step, and the losers vary at\nevery step (unless some devices are really slower than others).\nSynchronous updates\nWith synchronous updates, the aggregator waits for all gradients to be available before\ncomputing the average and applying the result (i.e., using the aggregated gradients to\nupdate the model parameters). Once a replica has finished computing its gradients, it\nmust wait for the parameters to be updated before it can proceed to the next mini-\nbatch. The downside is that some devices may be slower than others, so all other\ndevices will have to wait for them at every step. Moreover, the parameters will be\ncopied to every device almost at the same time (immediately after the gradients are\napplied), which may saturate the parameter servers\u2019 bandwidth.", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 910, "content": "applied), which may saturate the parameter servers\u2019 bandwidth.\nTo reduce the waiting time at each step, you could ignore the gradi\u2010\nents from the slowest few replicas (typically ~10%). For example,\nyou could run 20 replicas, but only aggregate the gradients from\nthe fastest 18 replicas at each step, and just ignore the gradients\nfrom the last 2. As soon as the parameters are updated, the first 18\nreplicas can start working again immediately, without having to\nwait for the 2 slowest replicas. This setup is generally described as\nhaving 18 replicas plus 2 spare replicas.5\nAsynchronous updates\nWith asynchronous updates, whenever a replica has finished computing the gradi\u2010\nents, it immediately uses them to update the model parameters. There is no aggrega\u2010\ntion (remove the \u201cmean\u201d step in Figure 12-17), and no synchronization. Replicas just\nwork independently of the other replicas. Since there is no waiting for the other repli\u2010", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 911, "content": "work independently of the other replicas. Since there is no waiting for the other repli\u2010\ncas, this approach runs more training steps per minute. Moreover, although the\nparameters still need to be copied to every device at every step, this happens at differ\u2010\nent times for each replica so the risk of bandwidth saturation is reduced.\nData parallelism with asynchronous updates is an attractive choice, because of its\nsimplicity, the absence of synchronization delay, and a better use of the bandwidth.\nHowever, although it works reasonably well in practice, it is almost surprising that it\nworks at all! Indeed, by the time a replica has finished computing the gradients based\non some parameter values, these parameters will have been updated several times by\nother replicas (on average N \u2013 1 times if there are N replicas) and there is no guaran\u2010\ntee that the computed gradients will still be pointing in the right direction (see", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 912, "content": "tee that the computed gradients will still be pointing in the right direction (see\nFigure 12-18). When gradients are severely out-of-date, they are called stale gradients: \nthey can slow down convergence, introducing noise and wobble effects (the learning\n348 \n| \nChapter 12: Distributing TensorFlow Across Devices and Servers\nDownload from finelybook www.finelybook.com", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 913, "content": "curve may contain temporary oscillations), or they can even make the training algo\u2010\nrithm diverge.\nFigure 12-18. Stale gradients when using asynchronous updates\nThere are a few ways to reduce the effect of stale gradients:\n\u2022 Reduce the learning rate.\n\u2022 Drop stale gradients or scale them down.\n\u2022 Adjust the mini-batch size.\n\u2022 Start the first few epochs using just one replica (this is called the warmup phase).\nStale gradients tend to be more damaging at the beginning of training, when gra\u2010\ndients are typically large and the parameters have not settled into a valley of the \ncost function yet, so different replicas may push the parameters in quite different\ndirections.\nA paper published by the Google Brain team in April 2016 benchmarked various\napproaches and found that data parallelism with synchronous updates using a few\nspare replicas was the most efficient, not only converging faster but also producing a", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 914, "content": "spare replicas was the most efficient, not only converging faster but also producing a\nbetter model. However, this is still an active area of research, so you should not rule\nout asynchronous updates quite yet.\nBandwidth saturation\nWhether you use synchronous or asynchronous updates, data parallelism still\nrequires communicating the model parameters from the parameter servers to every\nreplica at the beginning of every training step, and the gradients in the other direction\nat the end of each training step. Unfortunately, this means that there always comes a\npoint where adding an extra GPU will not improve performance at all because the\nParallelizing Neural Networks on a TensorFlow Cluster \n| \n349\nDownload from finelybook www.finelybook.com", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 915, "content": "time spent moving the data in and out of GPU RAM (and possibly across the net\u2010\nwork) will outweigh the speedup obtained by splitting the computation load. At that\npoint, adding more GPUs will just increase saturation and slow down training.\nFor some models, typically relatively small and trained on a very\nlarge training set, you are often better off training the model on a\nsingle machine with a single GPU.\nSaturation is more severe for large dense models, since they have a lot of parameters\nand gradients to transfer. It is less severe for small models (but the parallelization gain\nis small) and also for large sparse models since the gradients are typically mostly\nzeros, so they can be communicated efficiently. Jeff Dean, initiator and lead of the\nGoogle Brain project, reported typical speedups of 25\u201340x when distributing compu\u2010\ntations across 50 GPUs for dense models, and 300x speedup for sparser models", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 916, "content": "tations across 50 GPUs for dense models, and 300x speedup for sparser models\ntrained across 500 GPUs. As you can see, sparse models really do scale better. Here\nare a few concrete examples:\n\u2022 Neural Machine Translation: 6x speedup on 8 GPUs\n\u2022 Inception/ImageNet: 32x speedup on 50 GPUs\n\u2022 RankBrain: 300x speedup on 500 GPUs\nThese numbers represent the state of the art in Q1 2016. Beyond a few dozen GPUs\nfor a dense model or few hundred GPUs for a sparse model, saturation kicks in and\nperformance degrades. There is plenty of research going on to solve this problem\n(exploring peer-to-peer architectures rather than centralized parameter servers, using\nlossy model compression, optimizing when and what the replicas need to communi\u2010\ncate, and so on), so there will likely be a lot of progress in parallelizing neural net\u2010\nworks in the next few years.\nIn the meantime, here are a few simple steps you can take to reduce the saturation\nproblem:", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 917, "content": "In the meantime, here are a few simple steps you can take to reduce the saturation\nproblem:\n\u2022 Group your GPUs on a few servers rather than scattering them across many\nservers. This will avoid unnecessary network hops.\n\u2022 Shard the parameters across multiple parameter servers (as discussed earlier).\n\u2022 Drop the model parameters\u2019 float precision from 32 bits (tf.float32) to 16 bits\n(tf.bfloat16). This will cut in half the amount of data to transfer, without much\nimpact on the convergence rate or the model\u2019s performance.\n350 \n| \nChapter 12: Distributing TensorFlow Across Devices and Servers\nDownload from finelybook www.finelybook.com", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 918, "content": "Although 16-bit precision is the minimum for training neural net\u2010\nwork, you can actually drop down to 8-bit precision after training\nto reduce the size of the model and speed up computations. This is\ncalled quantizing the neural network. It is particularly useful for\ndeploying and running pretrained models on mobile phones. See\nPete Warden\u2019s great post on the subject.\nTensorFlow implementation\nTo implement data parallelism using TensorFlow, you first need to choose whether\nyou want in-graph replication or between-graph replication, and whether you want\nsynchronous updates or asynchronous updates. Let\u2019s look at how you would imple\u2010\nment each combination (see the exercises and the Jupyter notebooks for complete\ncode examples).\nWith in-graph replication + synchronous updates, you build one big graph contain\u2010\ning all the model replicas (placed on different devices), and a few nodes to aggregate\nall their gradients and feed them to an optimizer. Your code opens a session to the", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 919, "content": "all their gradients and feed them to an optimizer. Your code opens a session to the\ncluster and simply runs the training operation repeatedly.\nWith in-graph replication + asynchronous updates, you also create one big graph, but\nwith one optimizer per replica, and you run one thread per replica, repeatedly run\u2010\nning the replica\u2019s optimizer.\nWith between-graph replication + asynchronous updates, you run multiple inde\u2010\npendent clients (typically in separate processes), each training the model replica as if\nit were alone in the world, but the parameters are actually shared with other replicas\n(using a resource container).\nWith between-graph replication + synchronous updates, once again you run multiple\nclients, each training a model replica based on shared parameters, but this time you\nwrap the optimizer (e.g., a MomentumOptimizer) within a SyncReplicasOptimizer.\nEach replica uses this optimizer as it would use any other optimizer, but under the", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 920, "content": "Each replica uses this optimizer as it would use any other optimizer, but under the\nhood this optimizer sends the gradients to a set of queues (one per variable), which is\nread by one of the replica\u2019s SyncReplicasOptimizer, called the chief. The chief aggre\u2010\ngates the gradients and applies them, then writes a token to a token queue for each\nreplica, signaling it that it can go ahead and compute the next gradients. This\napproach supports having spare replicas.\nIf you go through the exercises, you will implement each of these four solutions. You\nwill easily be able to apply what you have learned to train large deep neural networks\nacross dozens of servers and GPUs! In the following chapters we will go through a\nfew more important neural network architectures before we tackle Reinforcement\nLearning.\nParallelizing Neural Networks on a TensorFlow Cluster \n| \n351\nDownload from finelybook www.finelybook.com", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 921, "content": "Exercises\n1. If you get a CUDA_ERROR_OUT_OF_MEMORY when starting your TensorFlow pro\u2010\ngram, what is probably going on? What can you do about it?\n2. What is the difference between pinning an operation on a device and placing an\noperation on a device?\n3. If you are running on a GPU-enabled TensorFlow installation, and you just use\nthe default placement, will all operations be placed on the first GPU?\n4. If you pin a variable to \"/gpu:0\", can it be used by operations placed on /gpu:1?\nOr by operations placed on \"/cpu:0\"? Or by operations pinned to devices loca\u2010\nted on other servers?\n5. Can two operations placed on the same device run in parallel?\n6. What is a control dependency and when would you want to use one?\n7. Suppose you train a DNN for days on a TensorFlow cluster, and immediately\nafter your training program ends you realize that you forgot to save the model\nusing a Saver. Is your trained model lost?", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 922, "content": "using a Saver. Is your trained model lost?\n8. Train several DNNs in parallel on a TensorFlow cluster, using different hyper\u2010\nparameter values. This could be DNNs for MNIST classification or any other task\nyou are interested in. The simplest option is to write a single client program that\ntrains only one DNN, then run this program in multiple processes in parallel,\nwith different hyperparameter values for each client. The program should have\ncommand-line options to control what server and device the DNN should be\nplaced on, and what resource container and hyperparameter values to use (make\nsure to use a different resource container for each DNN). Use a validation set or\ncross-validation to select the top three models.\n9. Create an ensemble using the top three models from the previous exercise.\nDefine it in a single graph, ensuring that each DNN runs on a different device.\nEvaluate it on the validation set: does the ensemble perform better than the indi\u2010\nvidual DNNs?", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 923, "content": "Evaluate it on the validation set: does the ensemble perform better than the indi\u2010\nvidual DNNs?\n10. Train a DNN using between-graph replication and data parallelism with asyn\u2010\nchronous updates, timing how long it takes to reach a satisfying performance.\nNext, try again using synchronous updates. Do synchronous updates produce a\nbetter model? Is training faster? Split the DNN vertically and place each vertical\nslice on a different device, and train the model again. Is training any faster? Is the\nperformance any different?\nSolutions to these exercises are available in Appendix A.\n352 \n| \nChapter 12: Distributing TensorFlow Across Devices and Servers\nDownload from finelybook www.finelybook.com", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 924, "content": "CHAPTER 13\nConvolutional Neural Networks\nAlthough IBM\u2019s Deep Blue supercomputer beat the chess world champion Garry Kas\u2010\nparov back in 1996, until quite recently computers were unable to reliably perform\nseemingly trivial tasks such as detecting a puppy in a picture or recognizing spoken\nwords. Why are these tasks so effortless to us humans? The answer lies in the fact that\nperception largely takes place outside the realm of our consciousness, within special\u2010\nized visual, auditory, and other sensory modules in our brains. By the time sensory\ninformation reaches our consciousness, it is already adorned with high-level features;\nfor example, when you look at a picture of a cute puppy, you cannot choose not to see\nthe puppy, or not to notice its cuteness. Nor can you explain how you recognize a cute\npuppy; it\u2019s just obvious to you. Thus, we cannot trust our subjective experience: per\u2010\nception is not trivial at all, and to understand it we must look at how the sensory\nmodules work.", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 925, "content": "ception is not trivial at all, and to understand it we must look at how the sensory\nmodules work.\nConvolutional neural networks (CNNs) emerged from the study of the brain\u2019s visual\ncortex, and they have been used in image recognition since the 1980s. In the last few\nyears, thanks to the increase in computational power, the amount of available training\ndata, and the tricks presented in Chapter 11 for training deep nets, CNNs have man\u2010\naged to achieve superhuman performance on some complex visual tasks. They power\nimage search services, self-driving cars, automatic video classification systems, and\nmore. Moreover, CNNs are not restricted to visual perception: they are also successful\nat other tasks, such as voice recognition or natural language processing (NLP); however,\nwe will focus on visual applications for now.\nIn this chapter we will present where CNNs came from, what their building blocks\nlook like, and how to implement them using TensorFlow. Then we will present some", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 926, "content": "look like, and how to implement them using TensorFlow. Then we will present some\nof the best CNN architectures.\n353\nDownload from finelybook www.finelybook.com", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 927, "content": "1 \u201cSingle Unit Activity in Striate Cortex of Unrestrained Cats,\u201d D. Hubel and T. Wiesel (1958).\n2 \u201cReceptive Fields of Single Neurones in the Cat\u2019s Striate Cortex,\u201d D. Hubel and T. Wiesel (1959).\n3 \u201cReceptive Fields and Functional Architecture of Monkey Striate Cortex,\u201d D. Hubel and T. Wiesel (1968).\n4 \u201cNeocognitron: A Self-organizing Neural Network Model for a Mechanism of Pattern Recognition Unaffected\nby Shift in Position,\u201d K. Fukushima (1980).\n5 \u201cGradient-Based Learning Applied to Document Recognition,\u201d Y. LeCun et al. (1998).\nThe Architecture of the Visual Cortex\nDavid H. Hubel and Torsten Wiesel performed a series of experiments on cats in\n19581 and 19592 (and a few years later on monkeys3), giving crucial insights on the\nstructure of the visual cortex (the authors received the Nobel Prize in Physiology or\nMedicine in 1981 for their work). In particular, they showed that many neurons in\nthe visual cortex have a small local receptive field, meaning they react only to visual", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 928, "content": "the visual cortex have a small local receptive field, meaning they react only to visual\nstimuli located in a limited region of the visual field (see Figure 13-1, in which the\nlocal receptive fields of five neurons are represented by dashed circles). The receptive\nfields of different neurons may overlap, and together they tile the whole visual field.\nMoreover, the authors showed that some neurons react only to images of horizontal\nlines, while others react only to lines with different orientations (two neurons may\nhave the same receptive field but react to different line orientations). They also\nnoticed that some neurons have larger receptive fields, and they react to more com\u2010\nplex patterns that are combinations of the lower-level patterns. These observations\nled to the idea that the higher-level neurons are based on the outputs of neighboring\nlower-level neurons (in Figure 13-1, notice that each neuron is connected only to a", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 929, "content": "lower-level neurons (in Figure 13-1, notice that each neuron is connected only to a\nfew neurons from the previous layer). This powerful architecture is able to detect all\nsorts of complex patterns in any area of the visual field.\nFigure 13-1. Local receptive fields in the visual cortex\nThese studies of the visual cortex inspired the neocognitron, introduced in 1980,4\nwhich gradually evolved into what we now call convolutional neural networks. An\nimportant milestone was a 1998 paper5 by Yann LeCun, L\u00e9on Bottou, Yoshua Bengio,\n354 \n| \nChapter 13: Convolutional Neural Networks\nDownload from finelybook www.finelybook.com", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 930, "content": "6 A convolution is a mathematical operation that slides one function over another and measures the integral of\ntheir pointwise multiplication. It has deep connections with the Fourier transform and the Laplace transform,\nand is heavily used in signal processing. Convolutional layers actually use cross-correlations, which are very\nsimilar to convolutions (see http://goo.gl/HAfxXd for more details).\nand Patrick Haffner, which introduced the famous LeNet-5 architecture, widely used\nto recognize handwritten check numbers. This architecture has some building blocks\nthat you already know, such as fully connected layers and sigmoid activation func\u2010\ntions, but it also introduces two new building blocks: convolutional layers and pooling\nlayers. Let\u2019s look at them now.\nWhy not simply use a regular deep neural network with fully con\u2010\nnected layers for image recognition tasks? Unfortunately, although\nthis works fine for small images (e.g., MNIST), it breaks down for", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 931, "content": "this works fine for small images (e.g., MNIST), it breaks down for\nlarger images because of the huge number of parameters it\nrequires. For example, a 100 \u00d7 100 image has 10,000 pixels, and if\nthe first layer has just 1,000 neurons (which already severely\nrestricts the amount of information transmitted to the next layer),\nthis means a total of 10 million connections. And that\u2019s just the first\nlayer. CNNs solve this problem using partially connected layers.\nConvolutional Layer\nThe most important building block of a CNN is the convolutional layer:6 neurons in\nthe first convolutional layer are not connected to every single pixel in the input image\n(like they were in previous chapters), but only to pixels in their receptive fields (see\nFigure 13-2). In turn, each neuron in the second convolutional layer is connected\nonly to neurons located within a small rectangle in the first layer. This architecture\nallows the network to concentrate on low-level features in the first hidden layer, then", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 932, "content": "allows the network to concentrate on low-level features in the first hidden layer, then\nassemble them into higher-level features in the next hidden layer, and so on. This\nhierarchical structure is common in real-world images, which is one of the reasons\nwhy CNNs work so well for image recognition.\nConvolutional Layer \n| \n355\nDownload from finelybook www.finelybook.com", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 933, "content": "Figure 13-2. CNN layers with rectangular local receptive fields\nUntil now, all multilayer neural networks we looked at had layers\ncomposed of a long line of neurons, and we had to flatten input\nimages to 1D before feeding them to the neural network. Now each\nlayer is represented in 2D, which makes it easier to match neurons\nwith their corresponding inputs.\nA neuron located in row i, column j of a given layer is connected to the outputs of the\nneurons in the previous layer located in rows i to i + fh \u2013 1, columns j to j + fw \u2013 1,\nwhere fh and fw are the height and width of the receptive field (see Figure 13-3). In\norder for a layer to have the same height and width as the previous layer, it is com\u2010\nmon to add zeros around the inputs, as shown in the diagram. This is called zero pad\u2010\nding.\nFigure 13-3. Connections between layers and zero padding\n356 \n| \nChapter 13: Convolutional Neural Networks\nDownload from finelybook www.finelybook.com", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 934, "content": "It is also possible to connect a large input layer to a much smaller layer by spacing out\nthe receptive fields, as shown in Figure 13-4. The distance between two consecutive\nreceptive fields is called the stride. In the diagram, a 5 \u00d7 7 input layer (plus zero pad\u2010\nding) is connected to a 3 \u00d7 4 layer, using 3 \u00d7 3 receptive fields and a stride of 2 (in this\nexample the stride is the same in both directions, but it does not have to be so). A\nneuron located in row i, column j in the upper layer is connected to the outputs of the\nneurons in the previous layer located in rows i \u00d7 sh to i \u00d7 sh + fh \u2013 1, columns j \u00d7 sw +\nfw \u2013 1, where sh and sw are the vertical and horizontal strides.\nFigure 13-4. Reducing dimensionality using a stride\nFilters\nA neuron\u2019s weights can be represented as a small image the size of the receptive field.\nFor example, Figure 13-5 shows two possible sets of weights, called filters (or convolu\u2010", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 935, "content": "For example, Figure 13-5 shows two possible sets of weights, called filters (or convolu\u2010\ntion kernels). The first one is represented as a black square with a vertical white line in\nthe middle (it is a 7 \u00d7 7 matrix full of 0s except for the central column, which is full of\n1s); neurons using these weights will ignore everything in their receptive field except\nfor the central vertical line (since all inputs will get multiplied by 0, except for the\nones located in the central vertical line). The second filter is a black square with a\nhorizontal white line in the middle. Once again, neurons using these weights will\nignore everything in their receptive field except for the central horizontal line.\nNow if all neurons in a layer use the same vertical line filter (and the same bias term),\nand you feed the network the input image shown in Figure 13-5 (bottom image), the\nlayer will output the top-left image. Notice that the vertical white lines get enhanced", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 936, "content": "layer will output the top-left image. Notice that the vertical white lines get enhanced\nwhile the rest gets blurred. Similarly, the upper-right image is what you get if all neu\u2010\nrons use the horizontal line filter; notice that the horizontal white lines get enhanced\nwhile the rest is blurred out. Thus, a layer full of neurons using the same filter gives \nConvolutional Layer \n| \n357\nDownload from finelybook www.finelybook.com", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 937, "content": "you a feature map, which highlights the areas in an image that are most similar to the\nfilter. During training, a CNN finds the most useful filters for its task, and it learns to\ncombine them into more complex patterns (e.g., a cross is an area in an image where\nboth the vertical filter and the horizontal filter are active).\nFigure 13-5. Applying two different filters to get two feature maps\nStacking Multiple Feature Maps\nUp to now, for simplicity, we have represented each convolutional layer as a thin 2D\nlayer, but in reality it is composed of several feature maps of equal sizes, so it is more\naccurately represented in 3D (see Figure 13-6). Within one feature map, all neurons\nshare the same parameters (weights and bias term), but different feature maps may\nhave different parameters. A neuron\u2019s receptive field is the same as described earlier,\nbut it extends across all the previous layers\u2019 feature maps. In short, a convolutional", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 938, "content": "but it extends across all the previous layers\u2019 feature maps. In short, a convolutional\nlayer simultaneously applies multiple filters to its inputs, making it capable of detect\u2010\ning multiple features anywhere in its inputs.\nThe fact that all neurons in a feature map share the same parame\u2010\nters dramatically reduces the number of parameters in the model,\nbut most importantly it means that once the CNN has learned to\nrecognize a pattern in one location, it can recognize it in any other\nlocation. In contrast, once a regular DNN has learned to recognize\na pattern in one location, it can recognize it only in that particular\nlocation.\n358 \n| \nChapter 13: Convolutional Neural Networks\nDownload from finelybook www.finelybook.com", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 939, "content": "Moreover, input images are also composed of multiple sublayers: one per color chan\u2010\nnel. There are typically three: red, green, and blue (RGB). Grayscale images have just\none channel, but some images may have much more\u2014for example, satellite images\nthat capture extra light frequencies (such as infrared).\nFigure 13-6. Convolution layers with multiple feature maps, and images with three\nchannels\nSpecifically, a neuron located in row i, column j of the feature map k in a given convo\u2010\nlutional layer l is connected to the outputs of the neurons in the previous layer l \u2013 1,\nlocated in rows i \u00d7 sw to i \u00d7 sw + fw \u2013 1 and columns j \u00d7 sh to j \u00d7 sh + fh \u2013 1, across all\nfeature maps (in layer l \u2013 1). Note that all neurons located in the same row i and col\u2010\numn j but in different feature maps are connected to the outputs of the exact same\nneurons in the previous layer.\nEquation 13-1 summarizes the preceding explanations in one big mathematical equa\u2010", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 940, "content": "Equation 13-1 summarizes the preceding explanations in one big mathematical equa\u2010\ntion: it shows how to compute the output of a given neuron in a convolutional layer.\nConvolutional Layer \n| \n359\nDownload from finelybook www.finelybook.com", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 941, "content": "It is a bit ugly due to all the different indices, but all it does is calculate the weighted\nsum of all the inputs, plus the bias term.\nEquation 13-1. Computing the output of a neuron in a convolutional layer\nzi, j, k = bk + \u2211\nu = 1\nf h\n\u2211\nv = 1\nf w\n\u2211\nk\u2032 = 1\nf n\u2032\nxi\u2032, j\u2032, k\u2032 . wu, v, k\u2032, k\nwith\ni\u2032 = u . sh + f h \u22121\nj\u2032 = v . sw + f w \u22121\n\u2022 zi, j, k is the output of the neuron located in row i, column j in feature map k of the\nconvolutional layer (layer l).\n\u2022 As explained earlier, sh and sw are the vertical and horizontal strides, fh and fw are\nthe height and width of the receptive field, and fn\u2032 is the number of feature maps\nin the previous layer (layer l \u2013 1).\n\u2022 xi\u2032, j\u2032, k\u2032 is the output of the neuron located in layer l \u2013 1, row i\u2032, column j\u2032, feature\nmap k\u2032 (or channel k\u2032 if the previous layer is the input layer).\n\u2022 bk is the bias term for feature map k (in layer l). You can think of it as a knob that\ntweaks the overall brightness of the feature map k.", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 942, "content": "tweaks the overall brightness of the feature map k.\n\u2022 wu, v, k\u2032 ,k is the connection weight between any neuron in feature map k of the layer\nl and its input located at row u, column v (relative to the neuron\u2019s receptive field),\nand feature map k\u2032.\nTensorFlow Implementation\nIn TensorFlow, each input image is typically represented as a 3D tensor of shape\n[height, width, channels]. A mini-batch is represented as a 4D tensor of shape\n[mini-batch size, height, width, channels]. The weights of a convolutional\nlayer are represented as a 4D tensor of shape [fh, fw, fn, fn\u2032]. The bias terms of a convo\u2010\nlutional layer are simply represented as a 1D tensor of shape [fn].\nLet\u2019s look at a simple example. The following code loads two sample images, using\nScikit-Learn\u2019s load_sample_images() (which loads two color images, one of a Chi\u2010\nnese temple, and the other of a flower). Then it creates two 7 \u00d7 7 filters (one with a", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 943, "content": "nese temple, and the other of a flower). Then it creates two 7 \u00d7 7 filters (one with a\nvertical white line in the middle, and the other with a horizontal white line), and\napplies them to both images using a convolutional layer built using TensorFlow\u2019s\nconv2d() function (with zero padding and a stride of 2). Finally, it plots one of the\nresulting feature maps (similar to the top-right image in Figure 13-5).\n360 \n| \nChapter 13: Convolutional Neural Networks\nDownload from finelybook www.finelybook.com", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 944, "content": "import numpy as np\nfrom sklearn.datasets import load_sample_images\n# Load sample images\ndataset = np.array(load_sample_images().images, dtype=np.float32)\nbatch_size, height, width, channels = dataset.shape\n# Create 2 filters\nfilters_test = np.zeros(shape=(7, 7, channels, 2), dtype=np.float32)\nfilters_test[:, 3, :, 0] = 1 # vertical line\nfilters_test[3, :, :, 1] = 1 # horizontal line\n# Create a graph with input X plus a convolutional layer applying the 2 filters\nX = tf.placeholder(tf.float32, shape=(None, height, width, channels))\nconvolution = tf.nn.conv2d(X, filters, strides=[1,2,2,1], padding=\"SAME\")\nwith tf.Session() as sess:\n output = sess.run(convolution, feed_dict={X: dataset})\nplt.imshow(output[0, :, :, 1]) # plot 1st image's 2nd feature map\nplt.show()\nMost of this code is self-explanatory, but the conv2d() line deserves a bit of explana\u2010\ntion:\n\u2022 X is the input mini-batch (a 4D tensor, as explained earlier).", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 945, "content": "tion:\n\u2022 X is the input mini-batch (a 4D tensor, as explained earlier).\n\u2022 filters is the set of filters to apply (also a 4D tensor, as explained earlier).\n\u2022 strides is a four-element 1D array, where the two central elements are the verti\u2010\ncal and horizontal strides (sh and sw). The first and last elements must currently\nbe equal to 1. They may one day be used to specify a batch stride (to skip some\ninstances) and a channel stride (to skip some of the previous layer\u2019s feature maps\nor channels).\n\u2022 padding must be either \"VALID\" or \"SAME\":\n\u2014 If set to \"VALID\", the convolutional layer does not use zero padding, and may\nignore some rows and columns at the bottom and right of the input image,\ndepending on the stride, as shown in Figure 13-7 (for simplicity, only the hor\u2010\nizontal dimension is shown here, but of course the same logic applies to the\nvertical dimension).\n\u2014 If set to \"SAME\", the convolutional layer uses zero padding if necessary. In this", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 946, "content": "\u2014 If set to \"SAME\", the convolutional layer uses zero padding if necessary. In this\ncase, the number of output neurons is equal to the number of input neurons\ndivided by the stride, rounded up (in this example, ceil (13 / 5) = 3). Then\nzeros are added as evenly as possible around the inputs.\nConvolutional Layer \n| \n361\nDownload from finelybook www.finelybook.com", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 947, "content": "7 A fully connected layer with 150 \u00d7 100 neurons, each connected to all 150 \u00d7 100 \u00d7 3 inputs, would have 1502\n\u00d7 1002 \u00d7 3 = 675 million parameters!\nFigure 13-7. Padding options\u2014input width: 13, filter width: 6, stride: 5\nUnfortunately, convolutional layers have quite a few hyperparameters: you must\nchoose the number of filters, their height and width, the strides, and the padding\ntype. As always, you can use cross-validation to find the right hyperparameter values,\nbut this is very time-consuming. We will discuss common CNN architectures later, to\ngive you some idea of what hyperparameter values work best in practice.\nMemory Requirements\nAnother problem with CNNs is that the convolutional layers require a huge amount\nof RAM, especially during training, because the reverse pass of backpropagation\nrequires all the intermediate values computed during the forward pass.\nFor example, consider a convolutional layer with 5 \u00d7 5 filters, outputting 200 feature", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 948, "content": "For example, consider a convolutional layer with 5 \u00d7 5 filters, outputting 200 feature\nmaps of size 150 \u00d7 100, with stride 1 and SAME padding. If the input is a 150 \u00d7 100\nRGB image (three channels), then the number of parameters is (5 \u00d7 5 \u00d7 3 + 1) \u00d7 200\n= 15,200 (the +1 corresponds to the bias terms), which is fairly small compared to a\nfully connected layer.7 However, each of the 200 feature maps contains 150 \u00d7 100 neu\u2010\nrons, and each of these neurons needs to compute a weighted sum of its 5 \u00d7 5 \u00d7 3 =\n75 inputs: that\u2019s a total of 225 million float multiplications. Not as bad as a fully con\u2010\n362 \n| \nChapter 13: Convolutional Neural Networks\nDownload from finelybook www.finelybook.com", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 949, "content": "8 1 MB = 1,024 kB = 1,024 \u00d7 1,024 bytes = 1,024 \u00d7 1,024 \u00d7 8 bits.\nnected layer, but still quite computationally intensive. Moreover, if the feature maps\nare represented using 32-bit floats, then the convolutional layer\u2019s output will occupy\n200 \u00d7 150 \u00d7 100 \u00d7 32 = 96 million bits (about 11.4 MB) of RAM.8 And that\u2019s just for\none instance! If a training batch contains 100 instances, then this layer will use up\nover 1 GB of RAM!\nDuring inference (i.e., when making a prediction for a new instance) the RAM occu\u2010\npied by one layer can be released as soon as the next layer has been computed, so you\nonly need as much RAM as required by two consecutive layers. But during training\neverything computed during the forward pass needs to be preserved for the reverse\npass, so the amount of RAM needed is (at least) the total amount of RAM required by\nall layers.\nIf training crashes because of an out-of-memory error, you can try\nreducing the mini-batch size. Alternatively, you can try reducing", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 950, "content": "reducing the mini-batch size. Alternatively, you can try reducing\ndimensionality using a stride, or removing a few layers. Or you can\ntry using 16-bit floats instead of 32-bit floats. Or you could distrib\u2010\nute the CNN across multiple devices.\nNow let\u2019s look at the second common building block of CNNs: the pooling layer.\nPooling Layer\nOnce you understand how convolutional layers work, the pooling layers are quite\neasy to grasp. Their goal is to subsample (i.e., shrink) the input image in order to\nreduce the computational load, the memory usage, and the number of parameters\n(thereby limiting the risk of overfitting). Reducing the input image size also makes\nthe neural network tolerate a little bit of image shift (location invariance).\nJust like in convolutional layers, each neuron in a pooling layer is connected to the\noutputs of a limited number of neurons in the previous layer, located within a small", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 951, "content": "outputs of a limited number of neurons in the previous layer, located within a small\nrectangular receptive field. You must define its size, the stride, and the padding type,\njust like before. However, a pooling neuron has no weights; all it does is aggregate the\ninputs using an aggregation function such as the max or mean. Figure 13-8 shows a\nmax pooling layer, which is the most common type of pooling layer. In this example,\nwe use a 2 \u00d7 2 pooling kernel, a stride of 2, and no padding. Note that only the max\ninput value in each kernel makes it to the next layer. The other inputs are dropped.\nPooling Layer \n| \n363\nDownload from finelybook www.finelybook.com", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 952, "content": "Figure 13-8. Max pooling layer (2 \u00d7 2 pooling kernel, stride 2, no padding)\nThis is obviously a very destructive kind of layer: even with a tiny 2 \u00d7 2 kernel and a\nstride of 2, the output will be two times smaller in both directions (so its area will be\nfour times smaller), simply dropping 75% of the input values.\nA pooling layer typically works on every input channel independently, so the output\ndepth is the same as the input depth. You may alternatively pool over the depth\ndimension, as we will see next, in which case the image\u2019s spatial dimensions (height\nand width) remain unchanged, but the number of channels is reduced.\nImplementing a max pooling layer in TensorFlow is quite easy. The following code\ncreates a max pooling layer using a 2 \u00d7 2 kernel, stride 2, and no padding, then\napplies it to all the images in the dataset:\n[...] # load the image dataset, just like above\n# Create a graph with input X plus a max pooling layer", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 953, "content": "# Create a graph with input X plus a max pooling layer\nX = tf.placeholder(tf.float32, shape=(None, height, width, channels))\nmax_pool = tf.nn.max_pool(X, ksize=[1,2,2,1], strides=[1,2,2,1],padding=\"VALID\")\nwith tf.Session() as sess:\n output = sess.run(max_pool, feed_dict={X: dataset})\nplt.imshow(output[0].astype(np.uint8)) # plot the output for the 1st image\nplt.show()\nThe ksize argument contains the kernel shape along all four dimensions of the input\ntensor: [batch size, height, width, channels]. TensorFlow currently does not\nsupport pooling over multiple instances, so the first element of ksize must be equal\nto 1. Moreover, it does not support pooling over both the spatial dimensions (height\nand width) and the depth dimension, so either ksize[1] and ksize[2] must both be\nequal to 1, or ksize[3] must be equal to 1.\nTo create an average pooling layer, just use the avg_pool() function instead of\nmax_pool().\n364 \n| \nChapter 13: Convolutional Neural Networks", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 954, "content": "max_pool().\n364 \n| \nChapter 13: Convolutional Neural Networks\nDownload from finelybook www.finelybook.com", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 955, "content": "Now you know all the building blocks to create a convolutional neural network. Let\u2019s\nsee how to assemble them.\nCNN Architectures\nTypical CNN architectures stack a few convolutional layers (each one generally fol\u2010\nlowed by a ReLU layer), then a pooling layer, then another few convolutional layers\n(+ReLU), then another pooling layer, and so on. The image gets smaller and smaller\nas it progresses through the network, but it also typically gets deeper and deeper (i.e.,\nwith more feature maps) thanks to the convolutional layers (see Figure 13-9). At the\ntop of the stack, a regular feedforward neural network is added, composed of a few\nfully connected layers (+ReLUs), and the final layer outputs the prediction (e.g., a\nsoftmax layer that outputs estimated class probabilities).\nFigure 13-9. Typical CNN architecture\nA common mistake is to use convolution kernels that are too large.\nYou can often get the same effect as a 9 \u00d7 9 kernel by stacking two 3", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 956, "content": "You can often get the same effect as a 9 \u00d7 9 kernel by stacking two 3\n\u00d7 3 kernels on top of each other, for a lot less compute.\nOver the years, variants of this fundamental architecture have been developed, lead\u2010\ning to amazing advances in the field. A good measure of this progress is the error rate\nin competitions such as the ILSVRC ImageNet challenge. In this competition the\ntop-5 error rate for image classification fell from over 26% to barely over 3% in just\nfive years. The top-five error rate is the number of test images for which the system\u2019s\ntop 5 predictions did not include the correct answer. The images are large (256 pixels\nhigh) and there are 1,000 classes, some of which are really subtle (try distinguishing\n120 dog breeds). Looking at the evolution of the winning entries is a good way to\nunderstand how CNNs work.\nWe will first look at the classical LeNet-5 architecture (1998), then three of the win\u2010\nners of the ILSVRC challenge: AlexNet (2012), GoogLeNet (2014), and ResNet", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 957, "content": "ners of the ILSVRC challenge: AlexNet (2012), GoogLeNet (2014), and ResNet\n(2015).\nCNN Architectures \n| \n365\nDownload from finelybook www.finelybook.com", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 958, "content": "Other Visual Tasks\nThere was stunning progress as well in other visual tasks such as object detection and\nlocalization, and image segmentation. In object detection and localization, the neural\nnetwork typically outputs a sequence of bounding boxes around various objects in\nthe image. For example, see Maxine Oquab et al.\u2019s 2015 paper that outputs a heat map\nfor each object class, or Russell Stewart et al.\u2019s 2015 paper that uses a combination of a\nCNN to detect faces and a recurrent neural network to output a sequence of bound\u2010\ning boxes around them. In image segmentation, the net outputs an image (usually of\nthe same size as the input) where each pixel indicates the class of the object to which\nthe corresponding input pixel belongs. For example, check out Evan Shelhamer et al.\u2019s\n2016 paper.\nLeNet-5\nThe LeNet-5 architecture is perhaps the most widely known CNN architecture. As\nmentioned earlier, it was created by Yann LeCun in 1998 and widely used for hand\u2010", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 959, "content": "mentioned earlier, it was created by Yann LeCun in 1998 and widely used for hand\u2010\nwritten digit recognition (MNIST). It is composed of the layers shown in Table 13-1.\nTable 13-1. LeNet-5 architecture\nLayer Type\nMaps Size\nKernel size Stride\nActivation\nOut\nFully Connected\n\u2013\n10\n\u2013\n\u2013\nRBF\nF6\nFully Connected\n\u2013\n84\n\u2013\n\u2013\ntanh\nC5\nConvolution\n120\n1 \u00d7 1\n5 \u00d7 5\n1\ntanh\nS4\nAvg Pooling\n16\n5 \u00d7 5\n2 \u00d7 2\n2\ntanh\nC3\nConvolution\n16\n10 \u00d7 10 5 \u00d7 5\n1\ntanh\nS2\nAvg Pooling\n6\n14 \u00d7 14 2 \u00d7 2\n2\ntanh\nC1\nConvolution\n6\n28 \u00d7 28 5 \u00d7 5\n1\ntanh\nIn\nInput\n1\n32 \u00d7 32 \u2013\n\u2013\n\u2013\nThere are a few extra details to be noted:\n\u2022 MNIST images are 28 \u00d7 28 pixels, but they are zero-padded to 32 \u00d7 32 pixels and\nnormalized before being fed to the network. The rest of the network does not use\nany padding, which is why the size keeps shrinking as the image progresses\nthrough the network.\n\u2022 The average pooling layers are slightly more complex than usual: each neuron\ncomputes the mean of its inputs, then multiplies the result by a learnable coeffi\u2010", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 960, "content": "computes the mean of its inputs, then multiplies the result by a learnable coeffi\u2010\ncient (one per map) and adds a learnable bias term (again, one per map), then\nfinally applies the activation function.\n366 \n| \nChapter 13: Convolutional Neural Networks\nDownload from finelybook www.finelybook.com", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 961, "content": "9 \u201cImageNet Classification with Deep Convolutional Neural Networks,\u201d A. Krizhevsky et al. (2012).\n\u2022 Most neurons in C3 maps are connected to neurons in only three or four S2\nmaps (instead of all six S2 maps). See table 1 in the original paper for details.\n\u2022 The output layer is a bit special: instead of computing the dot product of the\ninputs and the weight vector, each neuron outputs the square of the Euclidian\ndistance between its input vector and its weight vector. Each output measures\nhow much the image belongs to a particular digit class. The cross entropy cost\nfunction is now preferred, as it penalizes bad predictions much more, producing\nlarger gradients and thus converging faster.\nYann LeCun\u2019s website (\u201cLENET\u201d section) features great demos of LeNet-5 classifying \ndigits.\nAlexNet\nThe AlexNet CNN architecture9 won the 2012 ImageNet ILSVRC challenge by a large\nmargin: it achieved 17% top-5 error rate while the second best achieved only 26%! It", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 962, "content": "margin: it achieved 17% top-5 error rate while the second best achieved only 26%! It\nwas developed by Alex Krizhevsky (hence the name), Ilya Sutskever, and Geoffrey\nHinton. It is quite similar to LeNet-5, only much larger and deeper, and it was the\nfirst to stack convolutional layers directly on top of each other, instead of stacking a\npooling layer on top of each convolutional layer. Table 13-2 presents this architecture.\nTable 13-2. AlexNet architecture\nLayer Type\nMaps\nSize\nKernel size\nStride\nPadding\nActivation\nOut\nFully Connected\n\u2013\n1,000\n\u2013\n\u2013\n\u2013\nSoftmax\nF9\nFully Connected\n\u2013\n4,096\n\u2013\n\u2013\n\u2013\nReLU\nF8\nFully Connected\n\u2013\n4,096\n\u2013\n\u2013\n\u2013\nReLU\nC7\nConvolution\n256\n13 \u00d7 13\n3 \u00d7 3\n1\nSAME\nReLU\nC6\nConvolution\n384\n13 \u00d7 13\n3 \u00d7 3\n1\nSAME\nReLU\nC5\nConvolution\n384\n13 \u00d7 13\n3 \u00d7 3\n1\nSAME\nReLU\nS4\nMax Pooling\n256\n13 \u00d7 13\n3 \u00d7 3\n2\nVALID\n\u2013\nC3\nConvolution\n256\n27 \u00d7 27\n5 \u00d7 5\n1\nSAME\nReLU\nS2\nMax Pooling\n96\n27 \u00d7 27\n3 \u00d7 3\n2\nVALID\n\u2013\nC1\nConvolution\n96\n55 \u00d7 55\n11 \u00d7 11\n4\nSAME\nReLU\nIn\nInput\n3 (RGB)\n224 \u00d7 224 \u2013\n\u2013\n\u2013\n\u2013", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 963, "content": "3 \u00d7 3\n2\nVALID\n\u2013\nC1\nConvolution\n96\n55 \u00d7 55\n11 \u00d7 11\n4\nSAME\nReLU\nIn\nInput\n3 (RGB)\n224 \u00d7 224 \u2013\n\u2013\n\u2013\n\u2013\nTo reduce overfitting, the authors used two regularization techniques we discussed in\nprevious chapters: first they applied dropout (with a 50% dropout rate) during train\u2010\ning to the outputs of layers F8 and F9. Second, they performed data augmentation by\nCNN Architectures \n| \n367\nDownload from finelybook www.finelybook.com", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 964, "content": "10 \u201cGoing Deeper with Convolutions,\u201d C. Szegedy et al. (2015).\nrandomly shifting the training images by various offsets, flipping them horizontally,\nand changing the lighting conditions.\nAlexNet also uses a competitive normalization step immediately after the ReLU step\nof layers C1 and C3, called local response normalization. This form of normalization\nmakes the neurons that most strongly activate inhibit neurons at the same location\nbut in neighboring feature maps (such competitive activation has been observed in\nbiological neurons). This encourages different feature maps to specialize, pushing\nthem apart and forcing them to explore a wider range of features, ultimately improv\u2010\ning generalization. Equation 13-2 shows how to apply LRN.\nEquation 13-2. Local response normalization\nbi = ai k + \u03b1 \u2211\nj = jlow\njhigh\naj\n2\n\u2212\u03b2\nwith\njhigh = min i + r\n2, f n \u22121\njlow = max 0, i \u2212r\n2\n\u2022 bi is the normalized output of the neuron located in feature map i, at some row u", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 965, "content": "2\n\u2022 bi is the normalized output of the neuron located in feature map i, at some row u\nand column v (note that in this equation we consider only neurons located at this\nrow and column, so u and v are not shown).\n\u2022 ai is the activation of that neuron after the ReLU step, but before normalization.\n\u2022 k, \u03b1, \u03b2, and r are hyperparameters. k is called the bias, and r is called the depth\nradius.\n\u2022 fn is the number of feature maps.\nFor example, if r = 2 and a neuron has a strong activation, it will inhibit the activation\nof the neurons located in the feature maps immediately above and below its own.\nIn AlexNet, the hyperparameters are set as follows: r = 2, \u03b1 = 0.00002, \u03b2 = 0.75, and k\n= 1. This step can be implemented using TensorFlow\u2019s local_response_normaliza\ntion() operation.\nA variant of AlexNet called ZF Net was developed by Matthew Zeiler and Rob Fergus\nand won the 2013 ILSVRC challenge. It is essentially AlexNet with a few tweaked", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 966, "content": "and won the 2013 ILSVRC challenge. It is essentially AlexNet with a few tweaked \nhyperparameters (number of feature maps, kernel size, stride, etc.).\nGoogLeNet\nThe GoogLeNet architecture was developed by Christian Szegedy et al. from Google\nResearch,10 and it won the ILSVRC 2014 challenge by pushing the top-5 error rate\n368 \n| \nChapter 13: Convolutional Neural Networks\nDownload from finelybook www.finelybook.com", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 967, "content": "11 In the 2010 movie Inception, the characters keep going deeper and deeper into multiple layers of dreams,\nhence the name of these modules.\nbelow 7%. This great performance came in large part from the fact that the network\nwas much deeper than previous CNNs (see Figure 13-11). This was made possible by\nsub-networks called inception modules,11 which allow GoogLeNet to use parameters\nmuch more efficiently than previous architectures: GoogLeNet actually has 10 times\nfewer parameters than AlexNet (roughly 6 million instead of 60 million).\nFigure 13-10 shows the architecture of an inception module. The notation \u201c3 \u00d7 3 +\n2(S)\u201d means that the layer uses a 3 \u00d7 3 kernel, stride 2, and SAME padding. The input\nsignal is first copied and fed to four different layers. All convolutional layers use the\nReLU activation function. Note that the second set of convolutional layers uses differ\u2010\nent kernel sizes (1 \u00d7 1, 3 \u00d7 3, and 5 \u00d7 5), allowing them to capture patterns at different", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 968, "content": "ent kernel sizes (1 \u00d7 1, 3 \u00d7 3, and 5 \u00d7 5), allowing them to capture patterns at different\nscales. Also note that every single layer uses a stride of 1 and SAME padding (even\nthe max pooling layer), so their outputs all have the same height and width as their\ninputs. This makes it possible to concatenate all the outputs along the depth dimen\u2010\nsion in the final depth concat layer (i.e., stack the feature maps from all four top con\u2010\nvolutional layers). This concatenation layer can be implemented in TensorFlow using\nthe concat() operation, with axis=3 (axis 3 is the depth).\nFigure 13-10. Inception module\nYou may wonder why inception modules have convolutional layers with 1 \u00d7 1 ker\u2010\nnels. Surely these layers cannot capture any features since they look at only one pixel\nat a time? In fact, these layers serve two purposes:\n\u2022 First, they are configured to output many fewer feature maps than their inputs, so\nthey serve as bottleneck layers, meaning they reduce dimensionality. This is par\u2010", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 969, "content": "they serve as bottleneck layers, meaning they reduce dimensionality. This is par\u2010\nCNN Architectures \n| \n369\nDownload from finelybook www.finelybook.com", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 970, "content": "ticularly useful before the 3 \u00d7 3 and 5 \u00d7 5 convolutions, since these are very com\u2010\nputationally expensive layers.\n\u2022 Second, each pair of convolutional layers ([1 \u00d7 1, 3 \u00d7 3] and [1 \u00d7 1, 5 \u00d7 5]) acts\nlike a single, powerful convolutional layer, capable of capturing more complex\npatterns. Indeed, instead of sweeping a simple linear classifier across the image\n(as a single convolutional layer does), this pair of convolutional layers sweeps a\ntwo-layer neural network across the image.\nIn short, you can think of the whole inception module as a convolutional layer on\nsteroids, able to output feature maps that capture complex patterns at various scales.\nThe number of convolutional kernels for each convolutional layer\nis a hyperparameter. Unfortunately, this means that you have six\nmore hyperparameters to tweak for every inception layer you add.\nNow let\u2019s look at the architecture of the GoogLeNet CNN (see Figure 13-11). It is so", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 971, "content": "Now let\u2019s look at the architecture of the GoogLeNet CNN (see Figure 13-11). It is so\ndeep that we had to represent it in three columns, but GoogLeNet is actually one tall\nstack, including nine inception modules (the boxes with the spinning tops) that\nactually contain three layers each. The number of feature maps output by each convo\u2010\nlutional layer and each pooling layer is shown before the kernel size. The six numbers\nin the inception modules represent the number of feature maps output by each con\u2010\nvolutional layer in the module (in the same order as in Figure 13-10). Note that all the\nconvolutional layers use the ReLU activation function.\n370 \n| \nChapter 13: Convolutional Neural Networks\nDownload from finelybook www.finelybook.com", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 972, "content": "Figure 13-11. GoogLeNet architecture\nLet\u2019s go through this network:\n\u2022 The first two layers divide the image\u2019s height and width by 4 (so its area is divided\nby 16), to reduce the computational load.\n\u2022 Then the local response normalization layer ensures that the previous layers learn\na wide variety of features (as discussed earlier).\n\u2022 Two convolutional layers follow, where the first acts like a bottleneck layer. As\nexplained earlier, you can think of this pair as a single smarter convolutional\nlayer.\n\u2022 Again, a local response normalization layer ensures that the previous layers cap\u2010\nture a wide variety of patterns.\n\u2022 Next a max pooling layer reduces the image height and width by 2, again to speed\nup computations.\nCNN Architectures \n| \n371\nDownload from finelybook www.finelybook.com", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 973, "content": "12 \u201cDeep Residual Learning for Image Recognition,\u201d K. He (2015).\n\u2022 Then comes the tall stack of nine inception modules, interleaved with a couple\nmax pooling layers to reduce dimensionality and speed up the net.\n\u2022 Next, the average pooling layer uses a kernel the size of the feature maps with\nVALID padding, outputting 1 \u00d7 1 feature maps: this surprising strategy is called\nglobal average pooling. It effectively forces the previous layers to produce feature\nmaps that are actually confidence maps for each target class (since other kinds of\nfeatures would be destroyed by the averaging step). This makes it unnecessary to\nhave several fully connected layers at the top of the CNN (like in AlexNet), con\u2010\nsiderably reducing the number of parameters in the network and limiting the risk\nof overfitting.\n\u2022 The last layers are self-explanatory: dropout for regularization, then a fully con\u2010\nnected layer with a softmax activation function to output estimated class proba\u2010\nbilities.", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 974, "content": "nected layer with a softmax activation function to output estimated class proba\u2010\nbilities.\nThis diagram is slightly simplified: the original GoogLeNet architecture also included\ntwo auxiliary classifiers plugged on top of the third and sixth inception modules.\nThey were both composed of one average pooling layer, one convolutional layer, two\nfully connected layers, and a softmax activation layer. During training, their loss\n(scaled down by 70%) was added to the overall loss. The goal was to fight the vanish\u2010\ning gradients problem and regularize the network. However, it was shown that their\neffect was relatively minor.\nResNet\nLast but not least, the winner of the ILSVRC 2015 challenge was the Residual Network\n(or ResNet), developed by Kaiming He et al.,12 which delivered an astounding top-5\nerror rate under 3.6%, using an extremely deep CNN composed of 152 layers. The\nkey to being able to train such a deep network is to use skip connections (also called", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 975, "content": "key to being able to train such a deep network is to use skip connections (also called\nshortcut connections): the signal feeding into a layer is also added to the output of a\nlayer located a bit higher up the stack. Let\u2019s see why this is useful.\nWhen training a neural network, the goal is to make it model a target function h(x).\nIf you add the input x to the output of the network (i.e., you add a skip connection),\nthen the network will be forced to model f(x) = h(x) \u2013 x rather than h(x). This is\ncalled residual learning (see Figure 13-12).\n372 \n| \nChapter 13: Convolutional Neural Networks\nDownload from finelybook www.finelybook.com", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 976, "content": "Figure 13-12. Residual learning\nWhen you initialize a regular neural network, its weights are close to zero, so the net\u2010\nwork just outputs values close to zero. If you add a skip connection, the resulting net\u2010\nwork just outputs a copy of its inputs; in other words, it initially models the identity\nfunction. If the target function is fairly close to the identity function (which is often\nthe case), this will speed up training considerably.\nMoreover, if you add many skip connections, the network can start making progress\neven if several layers have not started learning yet (see Figure 13-13). Thanks to skip\nconnections, the signal can easily make its way across the whole network. The deep\nresidual network can be seen as a stack of residual units, where each residual unit is a\nsmall neural network with a skip connection.\nFigure 13-13. Regular deep neural network (left) and deep residual network (right)\nCNN Architectures \n| \n373\nDownload from finelybook www.finelybook.com", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 977, "content": "Now let\u2019s look at ResNet\u2019s architecture (see Figure 13-14). It is actually surprisingly\nsimple. It starts and ends exactly like GoogLeNet (except without a dropout layer),\nand in between is just a very deep stack of simple residual units. Each residual unit is\ncomposed of two convolutional layers, with Batch Normalization (BN) and ReLU\nactivation, using 3 \u00d7 3 kernels and preserving spatial dimensions (stride 1, SAME\npadding).\nFigure 13-14. ResNet architecture\nNote that the number of feature maps is doubled every few residual units, at the same\ntime as their height and width are halved (using a convolutional layer with stride 2).\nWhen this happens the inputs cannot be added directly to the outputs of the residual\nunit since they don\u2019t have the same shape (for example, this problem affects the skip\nconnection represented by the dashed arrow in Figure 13-14). To solve this problem,\nthe inputs are passed through a 1 \u00d7 1 convolutional layer with stride 2 and the right", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 978, "content": "the inputs are passed through a 1 \u00d7 1 convolutional layer with stride 2 and the right\nnumber of output feature maps (see Figure 13-15).\nFigure 13-15. Skip connection when changing feature map size and depth\n374 \n| \nChapter 13: Convolutional Neural Networks\nDownload from finelybook www.finelybook.com", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 979, "content": "13 \u201cVery Deep Convolutional Networks for Large-Scale Image Recognition,\u201d K. Simonyan and A. Zisserman\n(2015).\n14 \u201cInception-v4, Inception-ResNet and the Impact of Residual Connections on Learning,\u201d C. Szegedy et al.\n(2016).\nResNet-34 is the ResNet with 34 layers (only counting the convolutional layers and\nthe fully connected layer) containing three residual units that output 64 feature maps,\n4 RUs with 128 maps, 6 RUs with 256 maps, and 3 RUs with 512 maps.\nResNets deeper than that, such as ResNet-152, use slightly different residual units.\nInstead of two 3 \u00d7 3 convolutional layers with (say) 256 feature maps, they use three\nconvolutional layers: first a 1 \u00d7 1 convolutional layer with just 64 feature maps (4\ntimes less), which acts a a bottleneck layer (as discussed already), then a 3 \u00d7 3 layer\nwith 64 feature maps, and finally another 1 \u00d7 1 convolutional layer with 256 feature\nmaps (4 times 64) that restores the original depth. ResNet-152 contains three such", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 980, "content": "maps (4 times 64) that restores the original depth. ResNet-152 contains three such\nRUs that output 256 maps, then 8 RUs with 512 maps, a whopping 36 RUs with 1,024\nmaps, and finally 3 RUs with 2,048 maps.\nAs you can see, the field is moving rapidly, with all sorts of architectures popping out\nevery year. One clear trend is that CNNs keep getting deeper and deeper. They are\nalso getting lighter, requiring fewer and fewer parameters. At present, the ResNet\narchitecture is both the most powerful and arguably the simplest, so it is really the\none you should probably use for now, but keep looking at the ILSVRC challenge\nevery year. The 2016 winners were the Trimps-Soushen team from China with an\nastounding 2.99% error rate. To achieve this they trained combinations of the previ\u2010\nous models and joined them into an ensemble. Depending on the task, the reduced\nerror rate may or may not be worth the extra complexity.", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 981, "content": "error rate may or may not be worth the extra complexity.\nThere are a few other architectures that you may want to look at, in particular\nVGGNet13 (runner-up of the ILSVRC 2014 challenge) and Inception-v414 (which\nmerges the ideas of GoogLeNet and ResNet and achieves close to 3% top-5 error rate\non ImageNet classification).\nThere is really nothing special about implementing the various\nCNN architectures we just discussed. We saw earlier how to build\nall the individual building blocks, so now all you need is to assem\u2010\nble them to create the desired architecture. We will build ResNet-34\nin the upcoming exercises and you will find full working code in\nthe Jupyter notebooks.\nCNN Architectures \n| \n375\nDownload from finelybook www.finelybook.com", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 982, "content": "15 This name is quite misleading since this layer does not perform a deconvolution, which is a well-defined\nmathematical operation (the inverse of a convolution).\nTensorFlow Convolution Operations\nTensorFlow also offers a few other kinds of convolutional layers:\n\u2022 conv1d() creates a convolutional layer for 1D inputs. This is useful, for example,\nin natural language processing, where a sentence may be represented as a 1D\narray of words, and the receptive field covers a few neighboring words.\n\u2022 conv3d() creates a convolutional layer for 3D inputs, such as 3D PET scan.\n\u2022 atrous_conv2d() creates an atrous convolutional layer (\u201c\u00e0 trous\u201d is French for\n\u201cwith holes\u201d). This is equivalent to using a regular convolutional layer with a fil\u2010\nter dilated by inserting rows and columns of zeros (i.e., holes). For example, a 1 \u00d7\n3 filter equal to [[1,2,3]] may be dilated with a dilation rate of 4, resulting in a\ndilated filter [[1, 0, 0, 0, 2, 0, 0, 0, 3]]. This allows the convolutional", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 983, "content": "dilated filter [[1, 0, 0, 0, 2, 0, 0, 0, 3]]. This allows the convolutional\nlayer to have a larger receptive field at no computational price and using no extra\nparameters.\n\u2022 conv2d_transpose() creates a transpose convolutional layer, sometimes called a\ndeconvolutional layer,15 which upsamples an image. It does so by inserting zeros\nbetween the inputs, so you can think of this as a regular convolutional layer using\na fractional stride. Upsampling is useful, for example, in image segmentation: in a\ntypical CNN, feature maps get smaller and smaller as you progress through the\nnetwork, so if you want to output an image of the same size as the input, you\nneed an upsampling layer.\n\u2022 depthwise_conv2d() creates a depthwise convolutional layer that applies every fil\u2010\nter to every individual input channel independently. Thus, if there are fn filters\nand fn\u2032 input channels, then this will output fn \u00d7 fn\u2032 feature maps.", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 984, "content": "and fn\u2032 input channels, then this will output fn \u00d7 fn\u2032 feature maps.\n\u2022 separable_conv2d() creates a separable convolutional layer that first acts like a\ndepthwise convolutional layer, then applies a 1 \u00d7 1 convolutional layer to the\nresulting feature maps. This makes it possible to apply filters to arbitrary sets of\ninputs channels.\nExercises\n1. What are the advantages of a CNN over a fully connected DNN for image classi\u2010\nfication?\n2. Consider a CNN composed of three convolutional layers, each with 3 \u00d7 3 kernels,\na stride of 2, and SAME padding. The lowest layer outputs 100 feature maps, the\n376 \n| \nChapter 13: Convolutional Neural Networks\nDownload from finelybook www.finelybook.com", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 985, "content": "middle one outputs 200, and the top one outputs 400. The input images are RGB\nimages of 200 \u00d7 300 pixels. What is the total number of parameters in the CNN?\nIf we are using 32-bit floats, at least how much RAM will this network require\nwhen making a prediction for a single instance? What about when training on a\nmini-batch of 50 images?\n3. If your GPU runs out of memory while training a CNN, what are five things you\ncould try to solve the problem?\n4. Why would you want to add a max pooling layer rather than a convolutional\nlayer with the same stride?\n5. When would you want to add a local response normalization layer?\n6. Can you name the main innovations in AlexNet, compared to LeNet-5? What\nabout the main innovations in GoogLeNet and ResNet?\n7. Build your own CNN and try to achieve the highest possible accuracy on MNIST.\n8. Classifying large images using Inception v3.\na. Download some images of various animals. Load them in Python, for example", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 986, "content": "a. Download some images of various animals. Load them in Python, for example\nusing the matplotlib.image.mpimg.imread() function. Resize and/or crop\nthem to 299 \u00d7 299 pixels, and ensure that they have just three channels (RGB),\nwith no transparency channel.\nb. Download the latest pretrained Inception v3 model: the checkpoint is avail\u2010\nable at https://goo.gl/nxSQvl.\nc. Create the Inception v3 model by calling the inception_v3() function, as\nshown below. This must be done within an argument scope created by the\ninception_v3_arg_scope() function. Also, you must set is_training=False\nand num_classes=1001 like so:\nfrom tensorflow.contrib.slim.nets import inception\nimport tensorflow.contrib.slim as slim\nX = tf.placeholder(tf.float32, shape=[None, 299, 299, 3])\nwith slim.arg_scope(inception.inception_v3_arg_scope()):\n logits, end_points = inception.inception_v3(\n X, num_classes=1001, is_training=False)\npredictions = end_points[\"Predictions\"]", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 987, "content": "predictions = end_points[\"Predictions\"]\nsaver = tf.train.Saver()\nd. Open a session and use the Saver to restore the pretrained model checkpoint\nyou downloaded earlier.\ne. Run the model to classify the images you prepared. Display the top five pre\u2010\ndictions for each image, along with the estimated probability (the list of class\nnames is available at https://goo.gl/brXRtZ). How accurate is the model?\n9. Transfer learning for large image classification.\nExercises \n| \n377\nDownload from finelybook www.finelybook.com", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 988, "content": "a. Create a training set containing at least 100 images per class. For example, you\ncould classify your own pictures based on the location (beach, mountain, city,\netc.), or alternatively you can just use an existing dataset, such as the flowers\ndataset or MIT\u2019s places dataset (requires registration, and it is huge).\nb. Write a preprocessing step that will resize and crop the image to 299 \u00d7 299,\nwith some randomness for data augmentation.\nc. Using the pretrained Inception v3 model from the previous exercise, freeze all\nlayers up to the bottleneck layer (i.e., the last layer before the output layer),\nand replace the output layer with the appropriate number of outputs for your\nnew classification task (e.g., the flowers dataset has five mutually exclusive\nclasses so the output layer must have five neurons and use the softmax activa\u2010\ntion function).\nd. Split your dataset into a training set and a test set. Train the model on the\ntraining set and evaluate it on the test set.", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 989, "content": "training set and evaluate it on the test set.\n10. Go through TensorFlow\u2019s DeepDream tutorial. It is a fun way to familiarize your\u2010\nself with various ways of visualizing the patterns learned by a CNN, and to gener\u2010\nate art using Deep Learning.\nSolutions to these exercises are available in Appendix A.\n378 \n| \nChapter 13: Convolutional Neural Networks\nDownload from finelybook www.finelybook.com", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 990, "content": "CHAPTER 14\nRecurrent Neural Networks\nThe batter hits the ball. You immediately start running, anticipating the ball\u2019s trajec\u2010\ntory. You track it and adapt your movements, and finally catch it (under a thunder of\napplause). Predicting the future is what you do all the time, whether you are finishing\na friend\u2019s sentence or anticipating the smell of coffee at breakfast. In this chapter, we\nare going to discuss recurrent neural networks (RNN), a class of nets that can predict\nthe future (well, up to a point, of course). They can analyze time series data such as\nstock prices, and tell you when to buy or sell. In autonomous driving systems, they\ncan anticipate car trajectories and help avoid accidents. More generally, they can work\non sequences of arbitrary lengths, rather than on fixed-sized inputs like all the nets we\nhave discussed so far. For example, they can take sentences, documents, or audio\nsamples as input, making them extremely useful for natural language processing", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 991, "content": "samples as input, making them extremely useful for natural language processing\n(NLP) systems such as automatic translation, speech-to-text, or sentiment analysis \n(e.g., reading movie reviews and extracting the rater\u2019s feeling about the movie).\nMoreover, RNNs\u2019 ability to anticipate also makes them capable of surprising creativ\u2010\nity. You can ask them to predict which are the most likely next notes in a melody, then\nrandomly pick one of these notes and play it. Then ask the net for the next most likely\nnotes, play it, and repeat the process again and again. Before you know it, your net\nwill compose a melody such as the one produced by Google\u2019s Magenta project. Simi\u2010\nlarly, RNNs can generate sentences, image captions, and much more. The result is not\nexactly Shakespeare or Mozart yet, but who knows what they will produce a few years\nfrom now?\nIn this chapter, we will look at the fundamental concepts underlying RNNs, the main", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 992, "content": "from now?\nIn this chapter, we will look at the fundamental concepts underlying RNNs, the main\nproblem they face (namely, vanishing/exploding gradients, discussed in Chapter 11),\nand the solutions widely used to fight it: LSTM and GRU cells. Along the way, as\nalways, we will show how to implement RNNs using TensorFlow. Finally, we will take\na look at the architecture of a machine translation system.\n379\nDownload from finelybook www.finelybook.com", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 993, "content": "Recurrent Neurons\nUp to now we have mostly looked at feedforward neural networks, where the activa\u2010\ntions flow only in one direction, from the input layer to the output layer (except for a\nfew networks in Appendix E). A recurrent neural network looks very much like a\nfeedforward neural network, except it also has connections pointing backward. Let\u2019s\nlook at the simplest possible RNN, composed of just one neuron receiving inputs,\nproducing an output, and sending that output back to itself, as shown in Figure 14-1\n(left). At each time step t (also called a frame), this recurrent neuron receives the inputs\nx(t) as well as its own output from the previous time step, y(t\u20131). We can represent this\ntiny network against the time axis, as shown in Figure 14-1 (right). This is called\nunrolling the network through time.\nFigure 14-1. A recurrent neuron (left), unrolled through time (right)\nYou can easily create a layer of recurrent neurons. At each time step t, every neuron", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 994, "content": "You can easily create a layer of recurrent neurons. At each time step t, every neuron\nreceives both the input vector x(t) and the output vector from the previous time step\ny(t\u20131), as shown in Figure 14-2. Note that both the inputs and outputs are vectors now\n(when there was just a single neuron, the output was a scalar).\nFigure 14-2. A layer of recurrent neurons (left), unrolled through time (right)\nEach recurrent neuron has two sets of weights: one for the inputs x(t) and the other for\nthe outputs of the previous time step, y(t\u20131). Let\u2019s call these weight vectors wx and wy.\n380 \n| \nChapter 14: Recurrent Neural Networks\nDownload from finelybook www.finelybook.com", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 995, "content": "1 Note that many researchers prefer to use the hyperbolic tangent (tanh) activation function in RNNs rather\nthan the ReLU activation function. For example, take a look at by Vu Pham et al.\u2019s paper \u201cDropout Improves\nRecurrent Neural Networks for Handwriting Recognition\u201d. However, ReLU-based RNNs are also possible, as\nshown in Quoc V. Le et al.\u2019s paper \u201cA Simple Way to Initialize Recurrent Networks of Rectified Linear Units\u201d.\nThe output of a single recurrent neuron can be computed pretty much as you might\nexpect, as shown in Equation 14-1 (b is the bias term and \u03d5(\u00b7) is the activation func\u2010\ntion, e.g., ReLU1).\nEquation 14-1. Output of a single recurrent neuron for a single instance\n\ufffdt = \u03d5 \ufffdt\nT \u00b7 \ufffdx + \ufffdt \u22121\nT \u00b7 \ufffdy + b\nJust like for feedforward neural networks, we can compute a whole layer\u2019s output in\none shot for a whole mini-batch using a vectorized form of the previous equation (see\nEquation 14-2).\nEquation 14-2. Outputs of a layer of recurrent neurons for all instances in a mini-", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 996, "content": "Equation 14-2).\nEquation 14-2. Outputs of a layer of recurrent neurons for all instances in a mini-\nbatch\n\ufffdt = \u03d5 \ufffdt \u00b7 \ufffdx + \ufffdt \u22121 \u00b7 \ufffdy + \ufffd\n= \u03d5 \ufffdt\n\ufffdt \u22121\n\u00b7 \ufffd+ \ufffdwith \ufffd=\n\ufffdx\n\ufffdy\n\u2022 Y(t) is an m \u00d7 nneurons matrix containing the layer\u2019s outputs at time step t for each\ninstance in the mini-batch (m is the number of instances in the mini-batch and\nnneurons is the number of neurons).\n\u2022 X(t) is an m \u00d7 ninputs matrix containing the inputs for all instances (ninputs is the\nnumber of input features).\n\u2022 Wx is an ninputs \u00d7 nneurons matrix containing the connection weights for the inputs\nof the current time step.\n\u2022 Wy is an nneurons \u00d7 nneurons matrix containing the connection weights for the out\u2010\nputs of the previous time step.\n\u2022 The weight matrices Wx and Wy are often concatenated into a single weight\nmatrix W of shape (ninputs + nneurons) \u00d7 nneurons (see the second line of Equation\n14-2).\n\u2022 b is a vector of size nneurons containing each neuron\u2019s bias term.\nRecurrent Neurons \n| \n381", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 997, "content": "\u2022 b is a vector of size nneurons containing each neuron\u2019s bias term.\nRecurrent Neurons \n| \n381\nDownload from finelybook www.finelybook.com", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 998, "content": "Notice that Y(t) is a function of X(t) and Y(t\u20131), which is a function of X(t\u20131) and Y(t\u20132),\nwhich is a function of X(t\u20132) and Y(t\u20133), and so on. This makes Y(t) a function of all the\ninputs since time t = 0 (that is, X(0), X(1), \u2026, X(t)). At the first time step, t = 0, there are\nno previous outputs, so they are typically assumed to be all zeros.\nMemory Cells\nSince the output of a recurrent neuron at time step t is a function of all the inputs\nfrom previous time steps, you could say it has a form of memory. A part of a neural\nnetwork that preserves some state across time steps is called a memory cell (or simply\na cell). A single recurrent neuron, or a layer of recurrent neurons, is a very basic cell,\nbut later in this chapter we will look at some more complex and powerful types of\ncells.\nIn general a cell\u2019s state at time step t, denoted h(t) (the \u201ch\u201d stands for \u201chidden\u201d), is a\nfunction of some inputs at that time step and its state at the previous time step: h(t) =", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 999, "content": "function of some inputs at that time step and its state at the previous time step: h(t) =\nf(h(t\u20131), x(t)). Its output at time step t, denoted y(t), is also a function of the previous\nstate and the current inputs. In the case of the basic cells we have discussed so far, the\noutput is simply equal to the state, but in more complex cells this is not always the\ncase, as shown in Figure 14-3.\nFigure 14-3. A cell\u2019s hidden state and its output may be different\nInput and Output Sequences\nAn RNN can simultaneously take a sequence of inputs and produce a sequence of\noutputs (see Figure 14-4, top-left network). For example, this type of network is use\u2010\nful for predicting time series such as stock prices: you feed it the prices over the last N\ndays, and it must output the prices shifted by one day into the future (i.e., from N \u2013 1\ndays ago to tomorrow).\nAlternatively, you could feed the network a sequence of inputs, and ignore all outputs", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 1000, "content": "Alternatively, you could feed the network a sequence of inputs, and ignore all outputs\nexcept for the last one (see the top-right network). In other words, this is a sequence-\nto-vector network. For example, you could feed the network a sequence of words cor\u2010\n382 \n| \nChapter 14: Recurrent Neural Networks\nDownload from finelybook www.finelybook.com", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 1001, "content": "responding to a movie review, and the network would output a sentiment score (e.g.,\nfrom \u20131 [hate] to +1 [love]).\nConversely, you could feed the network a single input at the first time step (and zeros\nfor all other time steps), and let it output a sequence (see the bottom-left network).\nThis is a vector-to-sequence network. For example, the input could be an image, and\nthe output could be a caption for that image.\nLastly, you could have a sequence-to-vector network, called an encoder, followed by a\nvector-to-sequence network, called a decoder (see the bottom-right network). For\nexample, this can be used for translating a sentence from one language to another.\nYou would feed the network a sentence in one language, the encoder would convert\nthis sentence into a single vector representation, and then the decoder would decode\nthis vector into a sentence in another language. This two-step model, called an", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 1002, "content": "this vector into a sentence in another language. This two-step model, called an\nEncoder\u2013Decoder, works much better than trying to translate on the fly with a single\nsequence-to-sequence RNN (like the one represented on the top left), since the last\nwords of a sentence can affect the first words of the translation, so you need to wait\nuntil you have heard the whole sentence before translating it.\nFigure 14-4. Seq to seq (top left), seq to vector (top right), vector to seq (bottom left),\ndelayed seq to seq (bottom right)\nSounds promising, so let\u2019s start coding!\nRecurrent Neurons \n| \n383\nDownload from finelybook www.finelybook.com", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 1003, "content": "Basic RNNs in TensorFlow\nFirst, let\u2019s implement a very simple RNN model, without using any of TensorFlow\u2019s\nRNN operations, to better understand what goes on under the hood. We will create\nan RNN composed of a layer of five recurrent neurons (like the RNN represented in\nFigure 14-2), using the tanh activation function. We will assume that the RNN runs\nover only two time steps, taking input vectors of size 3 at each time step. The follow\u2010\ning code builds this RNN, unrolled through two time steps:\nn_inputs = 3\nn_neurons = 5\nX0 = tf.placeholder(tf.float32, [None, n_inputs])\nX1 = tf.placeholder(tf.float32, [None, n_inputs])\nWx = tf.Variable(tf.random_normal(shape=[n_inputs, n_neurons],dtype=tf.float32))\nWy = tf.Variable(tf.random_normal(shape=[n_neurons,n_neurons],dtype=tf.float32))\nb = tf.Variable(tf.zeros([1, n_neurons], dtype=tf.float32))\nY0 = tf.tanh(tf.matmul(X0, Wx) + b)\nY1 = tf.tanh(tf.matmul(Y0, Wy) + tf.matmul(X1, Wx) + b)\ninit = tf.global_variables_initializer()", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 1004, "content": "Y1 = tf.tanh(tf.matmul(Y0, Wy) + tf.matmul(X1, Wx) + b)\ninit = tf.global_variables_initializer()\nThis network looks much like a two-layer feedforward neural network, with a few\ntwists: first, the same weights and bias terms are shared by both layers, and second,\nwe feed inputs at each layer, and we get outputs from each layer. To run the model, we\nneed to feed it the inputs at both time steps, like so:\nimport numpy as np\n# Mini-batch: instance 0,instance 1,instance 2,instance 3\nX0_batch = np.array([[0, 1, 2], [3, 4, 5], [6, 7, 8], [9, 0, 1]]) # t = 0\nX1_batch = np.array([[9, 8, 7], [0, 0, 0], [6, 5, 4], [3, 2, 1]]) # t = 1\nwith tf.Session() as sess:\n init.run()\n Y0_val, Y1_val = sess.run([Y0, Y1], feed_dict={X0: X0_batch, X1: X1_batch})\nThis mini-batch contains four instances, each with an input sequence composed of\nexactly two inputs. At the end, Y0_val and Y1_val contain the outputs of the network\nat both time steps for all neurons and all instances in the mini-batch:", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 1005, "content": "at both time steps for all neurons and all instances in the mini-batch:\n>>> print(Y0_val) # output at t = 0\n[[-0.2964572 0.82874775 -0.34216955 -0.75720584 0.19011548] # instance 0\n [-0.12842922 0.99981797 0.84704727 -0.99570125 0.38665548] # instance 1\n [ 0.04731077 0.99999976 0.99330056 -0.999933 0.55339795] # instance 2\n [ 0.70323634 0.99309105 0.99909431 -0.85363263 0.7472108 ]] # instance 3\n>>> print(Y1_val) # output at t = 1\n[[ 0.51955646 1. 0.99999022 -0.99984968 -0.24616946] # instance 0\n [-0.70553327 -0.11918639 0.48885304 0.08917919 -0.26579669] # instance 1\n384 \n| \nChapter 14: Recurrent Neural Networks\nDownload from finelybook www.finelybook.com", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 1006, "content": "[-0.32477224 0.99996376 0.99933046 -0.99711186 0.10981458] # instance 2\n [-0.43738723 0.91517633 0.97817528 -0.91763324 0.11047263]] # instance 3\nThat wasn\u2019t too hard, but of course if you want to be able to run an RNN over 100\ntime steps, the graph is going to be pretty big. Now let\u2019s look at how to create the\nsame model using TensorFlow\u2019s RNN operations.\nStatic Unrolling Through Time\nThe static_rnn() function creates an unrolled RNN network by chaining cells. The\nfollowing code creates the exact same model as the previous one:\nX0 = tf.placeholder(tf.float32, [None, n_inputs])\nX1 = tf.placeholder(tf.float32, [None, n_inputs])\nbasic_cell = tf.contrib.rnn.BasicRNNCell(num_units=n_neurons)\noutput_seqs, states = tf.contrib.rnn.static_rnn(\n basic_cell, [X0, X1], dtype=tf.float32)\nY0, Y1 = output_seqs\nFirst we create the input placeholders, as before. Then we create a BasicRNNCell,", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 1007, "content": "First we create the input placeholders, as before. Then we create a BasicRNNCell,\nwhich you can think of as a factory that creates copies of the cell to build the unrolled\nRNN (one for each time step). Then we call static_rnn(), giving it the cell factory\nand the input tensors, and telling it the data type of the inputs (this is used to create\nthe initial state matrix, which by default is full of zeros). The static_rnn() function\ncalls the cell factory\u2019s __call__() function once per input, creating two copies of the\ncell (each containing a layer of five recurrent neurons), with shared weights and bias\nterms, and it chains them just like we did earlier. The static_rnn() function returns\ntwo objects. The first is a Python list containing the output tensors for each time step.\nThe second is a tensor containing the final states of the network. When you are using\nbasic cells, the final state is simply equal to the last output.", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 1008, "content": "basic cells, the final state is simply equal to the last output.\nIf there were 50 time steps, it would not be very convenient to have to define 50 input\nplaceholders and 50 output tensors. Moreover, at execution time you would have to\nfeed each of the 50 placeholders and manipulate the 50 outputs. Let\u2019s simplify this.\nThe following code builds the same RNN again, but this time it takes a single input\nplaceholder of shape [None, n_steps, n_inputs] where the first dimension is the\nmini-batch size. Then it extracts the list of input sequences for each time step. X_seqs\nis a Python list of n_steps tensors of shape [None, n_inputs], where once again the\nfirst dimension is the mini-batch size. To do this, we first swap the first two dimen\u2010\nsions using the transpose() function, so that the time steps are now the first dimen\u2010\nsion. Then we extract a Python list of tensors along the first dimension (i.e., one\ntensor per time step) using the unstack() function. The next two lines are the same", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 1009, "content": "tensor per time step) using the unstack() function. The next two lines are the same\nas before. Finally, we merge all the output tensors into a single tensor using the\nstack() function, and we swap the first two dimensions to get a final outputs tensor\nBasic RNNs in TensorFlow \n| \n385\nDownload from finelybook www.finelybook.com", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 1010, "content": "of shape [None, n_steps, n_neurons] (again the first dimension is the mini-batch\nsize).\nX = tf.placeholder(tf.float32, [None, n_steps, n_inputs])\nX_seqs = tf.unstack(tf.transpose(X, perm=[1, 0, 2]))\nbasic_cell = tf.contrib.rnn.BasicRNNCell(num_units=n_neurons)\noutput_seqs, states = tf.contrib.rnn.static_rnn(\n basic_cell, X_seqs, dtype=tf.float32)\noutputs = tf.transpose(tf.stack(output_seqs), perm=[1, 0, 2])\nNow we can run the network by feeding it a single tensor that contains all the mini-\nbatch sequences:\nX_batch = np.array([\n # t = 0 t = 1\n [[0, 1, 2], [9, 8, 7]], # instance 0\n [[3, 4, 5], [0, 0, 0]], # instance 1\n [[6, 7, 8], [6, 5, 4]], # instance 2\n [[9, 0, 1], [3, 2, 1]], # instance 3\n ])\nwith tf.Session() as sess:\n init.run()\n outputs_val = outputs.eval(feed_dict={X: X_batch})\nAnd we get a single outputs_val tensor for all instances, all time steps, and all neu\u2010\nrons:\n>>> print(outputs_val)", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 1011, "content": "rons:\n>>> print(outputs_val)\n[[[-0.2964572 0.82874775 -0.34216955 -0.75720584 0.19011548]\n [ 0.51955646 1. 0.99999022 -0.99984968 -0.24616946]]\n [[-0.12842922 0.99981797 0.84704727 -0.99570125 0.38665548]\n [-0.70553327 -0.11918639 0.48885304 0.08917919 -0.26579669]]\n [[ 0.04731077 0.99999976 0.99330056 -0.999933 0.55339795]\n [-0.32477224 0.99996376 0.99933046 -0.99711186 0.10981458]]\n [[ 0.70323634 0.99309105 0.99909431 -0.85363263 0.7472108 ]\n [-0.43738723 0.91517633 0.97817528 -0.91763324 0.11047263]]]\nHowever, this approach still builds a graph containing one cell per time step. If there\nwere 50 time steps, the graph would look pretty ugly. It is a bit like writing a program\nwithout ever using loops (e.g., Y0=f(0, X0); Y1=f(Y0, X1); Y2=f(Y1, X2); ...;\nY50=f(Y49, X50)). With such as large graph, you may even get out-of-memory\n(OOM) errors during backpropagation (especially with the limited memory of GPU", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 1012, "content": "(OOM) errors during backpropagation (especially with the limited memory of GPU\ncards), since it must store all tensor values during the forward pass so it can use them\nto compute gradients during the reverse pass.\nFortunately, there is a better solution: the dynamic_rnn() function.\n386 \n| \nChapter 14: Recurrent Neural Networks\nDownload from finelybook www.finelybook.com", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 1013, "content": "Dynamic Unrolling Through Time\nThe dynamic_rnn() function uses a while_loop() operation to run over the cell the\nappropriate number of times, and you can set swap_memory=True if you want it to\nswap the GPU\u2019s memory to the CPU\u2019s memory during backpropagation to avoid\nOOM errors. Conveniently, it also accepts a single tensor for all inputs at every time\nstep (shape [None, n_steps, n_inputs]) and it outputs a single tensor for all out\u2010\nputs at every time step (shape [None, n_steps, n_neurons]); there is no need to\nstack, unstack, or transpose. The following code creates the same RNN as earlier\nusing the dynamic_rnn() function. It\u2019s so much nicer!\nX = tf.placeholder(tf.float32, [None, n_steps, n_inputs])\nbasic_cell = tf.contrib.rnn.BasicRNNCell(num_units=n_neurons)\noutputs, states = tf.nn.dynamic_rnn(basic_cell, X, dtype=tf.float32)\nDuring backpropagation, the while_loop() operation does the\nappropriate magic: it stores the tensor values for each iteration dur\u2010", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 1014, "content": "appropriate magic: it stores the tensor values for each iteration dur\u2010\ning the forward pass so it can use them to compute gradients dur\u2010\ning the reverse pass.\nHandling Variable Length Input Sequences\nSo far we have used only fixed-size input sequences (all exactly two steps long). What\nif the input sequences have variable lengths (e.g., like sentences)? In this case you\nshould set the sequence_length parameter when calling the dynamic_rnn() (or\nstatic_rnn()) function; it must be a 1D tensor indicating the length of the input\nsequence for each instance. For example:\nseq_length = tf.placeholder(tf.int32, [None])\n[...]\noutputs, states = tf.nn.dynamic_rnn(basic_cell, X, dtype=tf.float32,\n sequence_length=seq_length)\nFor example, suppose the second input sequence contains only one input instead of\ntwo. It must be padded with a zero vector in order to fit in the input tensor X (because", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 1015, "content": "two. It must be padded with a zero vector in order to fit in the input tensor X (because\nthe input tensor\u2019s second dimension is the size of the longest sequence\u2014i.e., 2).\nX_batch = np.array([\n # step 0 step 1\n [[0, 1, 2], [9, 8, 7]], # instance 0\n [[3, 4, 5], [0, 0, 0]], # instance 1 (padded with a zero vector)\n [[6, 7, 8], [6, 5, 4]], # instance 2\n [[9, 0, 1], [3, 2, 1]], # instance 3\n ])\nseq_length_batch = np.array([2, 1, 2, 2])\nBasic RNNs in TensorFlow \n| \n387\nDownload from finelybook www.finelybook.com", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 1016, "content": "Of course, you now need to feed values for both placeholders X and seq_length:\nwith tf.Session() as sess:\n init.run()\n outputs_val, states_val = sess.run(\n [outputs, states], feed_dict={X: X_batch, seq_length: seq_length_batch})\nNow the RNN outputs zero vectors for every time step past the input sequence length\n(look at the second instance\u2019s output for the second time step):\n>>> print(outputs_val)\n[[[-0.2964572 0.82874775 -0.34216955 -0.75720584 0.19011548]\n [ 0.51955646 1. 0.99999022 -0.99984968 -0.24616946]] # final state\n [[-0.12842922 0.99981797 0.84704727 -0.99570125 0.38665548] # final state\n [ 0. 0. 0. 0. 0. ]] # zero vector\n [[ 0.04731077 0.99999976 0.99330056 -0.999933 0.55339795]\n [-0.32477224 0.99996376 0.99933046 -0.99711186 0.10981458]] # final state\n [[ 0.70323634 0.99309105 0.99909431 -0.85363263 0.7472108 ]", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 1017, "content": "[[ 0.70323634 0.99309105 0.99909431 -0.85363263 0.7472108 ]\n [-0.43738723 0.91517633 0.97817528 -0.91763324 0.11047263]]] # final state\nMoreover, the states tensor contains the final state of each cell (excluding the zero\nvectors):\n>>> print(states_val)\n[[ 0.51955646 1. 0.99999022 -0.99984968 -0.24616946] # t = 1\n [-0.12842922 0.99981797 0.84704727 -0.99570125 0.38665548] # t = 0 !!!\n [-0.32477224 0.99996376 0.99933046 -0.99711186 0.10981458] # t = 1\n [-0.43738723 0.91517633 0.97817528 -0.91763324 0.11047263]] # t = 1\nHandling Variable-Length Output Sequences\nWhat if the output sequences have variable lengths as well? If you know in advance\nwhat length each sequence will have (for example if you know that it will be the same\nlength as the input sequence), then you can set the sequence_length parameter as\ndescribed above. Unfortunately, in general this will not be possible: for example, the", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 1018, "content": "described above. Unfortunately, in general this will not be possible: for example, the\nlength of a translated sentence is generally different from the length of the input sen\u2010\ntence. In this case, the most common solution is to define a special output called an\nend-of-sequence token (EOS token). Any output past the EOS should be ignored (we\nwill discuss this later in this chapter).\nOkay, now you know how to build an RNN network (or more precisely an RNN net\u2010\nwork unrolled through time). But how do you train it?\n388 \n| \nChapter 14: Recurrent Neural Networks\nDownload from finelybook www.finelybook.com", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 1019, "content": "Training RNNs\nTo train an RNN, the trick is to unroll it through time (like we just did) and then\nsimply use regular backpropagation (see Figure 14-5). This strategy is called backpro\u2010\npagation through time (BPTT).\nFigure 14-5. Backpropagation through time\nJust like in regular backpropagation, there is a first forward pass through the unrolled\nnetwork (represented by the dashed arrows); then the output sequence is evaluated\nusing a cost function C \ufffdtmin\n, \ufffdtmin + 1 , \u22ef, \ufffdtmax\n (where tmin and tmax are the first\nand last output time steps, not counting the ignored outputs), and the gradients of\nthat cost function are propagated backward through the unrolled network (repre\u2010\nsented by the solid arrows); and finally the model parameters are updated using the\ngradients computed during BPTT. Note that the gradients flow backward through all\nthe outputs used by the cost function, not just through the final output (for example,", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 1020, "content": "the outputs used by the cost function, not just through the final output (for example,\nin Figure 14-5 the cost function is computed using the last three outputs of the net\u2010\nwork, Y(2), Y(3), and Y(4), so gradients flow through these three outputs, but not\nthrough Y(0) and Y(1)). Moreover, since the same parameters W and b are used at each\ntime step, backpropagation will do the right thing and sum over all time steps.\nTraining a Sequence Classifier\nLet\u2019s train an RNN to classify MNIST images. A convolutional neural network would\nbe better suited for image classification (see Chapter 13), but this makes for a simple\nexample that you are already familiar with. We will treat each image as a sequence of\n28 rows of 28 pixels each (since each MNIST image is 28 \u00d7 28 pixels). We will use\ncells of 150 recurrent neurons, plus a fully connected layer containing 10 neurons\nTraining RNNs \n| \n389\nDownload from finelybook www.finelybook.com", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 1021, "content": "(one per class) connected to the output of the last time step, followed by a softmax\nlayer (see Figure 14-6).\nFigure 14-6. Sequence classifier\nThe construction phase is quite straightforward; it\u2019s pretty much the same as the\nMNIST classifier we built in Chapter 10 except that an unrolled RNN replaces the\nhidden layers. Note that the fully connected layer is connected to the states tensor,\nwhich contains only the final state of the RNN (i.e., the 28th output). Also note that y\nis a placeholder for the target classes.\nfrom tensorflow.contrib.layers import fully_connected\nn_steps = 28\nn_inputs = 28\nn_neurons = 150\nn_outputs = 10\nlearning_rate = 0.001\nX = tf.placeholder(tf.float32, [None, n_steps, n_inputs])\ny = tf.placeholder(tf.int32, [None])\nbasic_cell = tf.contrib.rnn.BasicRNNCell(num_units=n_neurons)\noutputs, states = tf.nn.dynamic_rnn(basic_cell, X, dtype=tf.float32)\nlogits = fully_connected(states, n_outputs, activation_fn=None)", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 1022, "content": "logits = fully_connected(states, n_outputs, activation_fn=None)\nxentropy = tf.nn.sparse_softmax_cross_entropy_with_logits(\n labels=y, logits=logits)\nloss = tf.reduce_mean(xentropy)\noptimizer = tf.train.AdamOptimizer(learning_rate=learning_rate)\ntraining_op = optimizer.minimize(loss)\ncorrect = tf.nn.in_top_k(logits, y, 1)\naccuracy = tf.reduce_mean(tf.cast(correct, tf.float32))\n390 \n| \nChapter 14: Recurrent Neural Networks\nDownload from finelybook www.finelybook.com", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 1023, "content": "init = tf.global_variables_initializer()\nNow let\u2019s load the MNIST data and reshape the test data to [batch_size, n_steps,\nn_inputs] as is expected by the network. We will take care of reshaping the training\ndata in a moment.\nfrom tensorflow.examples.tutorials.mnist import input_data\nmnist = input_data.read_data_sets(\"/tmp/data/\")\nX_test = mnist.test.images.reshape((-1, n_steps, n_inputs))\ny_test = mnist.test.labels\nNow we are ready to train the RNN. The execution phase is exactly the same as for\nthe MNIST classifier in Chapter 10, except that we reshape each training batch before\nfeeding it to the network.\nn_epochs = 100\nbatch_size = 150\nwith tf.Session() as sess:\n init.run()\n for epoch in range(n_epochs):\n for iteration in range(mnist.train.num_examples // batch_size):\n X_batch, y_batch = mnist.train.next_batch(batch_size)\n X_batch = X_batch.reshape((-1, n_steps, n_inputs))\n sess.run(training_op, feed_dict={X: X_batch, y: y_batch})", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 1024, "content": "sess.run(training_op, feed_dict={X: X_batch, y: y_batch})\n acc_train = accuracy.eval(feed_dict={X: X_batch, y: y_batch})\n acc_test = accuracy.eval(feed_dict={X: X_test, y: y_test})\n print(epoch, \"Train accuracy:\", acc_train, \"Test accuracy:\", acc_test)\nThe output should look like this:\n0 Train accuracy: 0.713333 Test accuracy: 0.7299\n1 Train accuracy: 0.766667 Test accuracy: 0.7977\n...\n98 Train accuracy: 0.986667 Test accuracy: 0.9777\n99 Train accuracy: 0.986667 Test accuracy: 0.9809\nWe get over 98% accuracy\u2014not bad! Plus you would certainly get a better result by\ntuning the hyperparameters, initializing the RNN weights using He initialization,\ntraining longer, or adding a bit of regularization (e.g., dropout).\nYou can specify an initializer for the RNN by wrapping its \nconstruction \ncode \nin \na \nvariable \nscope \n(e.g., \nuse\nvariable_scope(\"rnn\", initializer=variance_scaling_ini\ntializer()) to use He initialization).\nTraining RNNs \n| \n391", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 1025, "content": "tializer()) to use He initialization).\nTraining RNNs \n| \n391\nDownload from finelybook www.finelybook.com", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 1026, "content": "Training to Predict Time Series\nNow let\u2019s take a look at how to handle time series, such as stock prices, air tempera\u2010\nture, brain wave patterns, and so on. In this section we will train an RNN to predict\nthe next value in a generated time series. Each training instance is a randomly\nselected sequence of 20 consecutive values from the time series, and the target\nsequence is the same as the input sequence, except it is shifted by one time step into\nthe future (see Figure 14-7).\nFigure 14-7. Time series (left), and a training instance from that series (right)\nFirst, let\u2019s create the RNN. It will contain 100 recurrent neurons and we will unroll it\nover 20 time steps since each training instance will be 20 inputs long. Each input will\ncontain only one feature (the value at that time). The targets are also sequences of 20\ninputs, each containing a single value. The code is almost the same as earlier:\nn_steps = 20\nn_inputs = 1\nn_neurons = 100\nn_outputs = 1", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 1027, "content": "n_steps = 20\nn_inputs = 1\nn_neurons = 100\nn_outputs = 1\nX = tf.placeholder(tf.float32, [None, n_steps, n_inputs])\ny = tf.placeholder(tf.float32, [None, n_steps, n_outputs])\ncell = tf.contrib.rnn.BasicRNNCell(num_units=n_neurons, activation=tf.nn.relu)\noutputs, states = tf.nn.dynamic_rnn(cell, X, dtype=tf.float32)\nIn general you would have more than just one input feature. For\nexample, if you were trying to predict stock prices, you would\nlikely have many other input features at each time step, such as pri\u2010\nces of competing stocks, ratings from analysts, or any other feature\nthat might help the system make its predictions.\nAt each time step we now have an output vector of size 100. But what we actually\nwant is a single output value at each time step. The simplest solution is to wrap the\ncell in an OutputProjectionWrapper. A cell wrapper acts like a normal cell, proxying\n392 \n| \nChapter 14: Recurrent Neural Networks\nDownload from finelybook www.finelybook.com", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 1028, "content": "every method call to an underlying cell, but it also adds some functionality. The Out\nputProjectionWrapper adds a fully connected layer of linear neurons (i.e., without\nany activation function) on top of each output (but it does not affect the cell state).\nAll these fully connected layers share the same (trainable) weights and bias terms.\nThe resulting RNN is represented in Figure 14-8.\nFigure 14-8. RNN cells using output projections\nWrapping a cell is quite easy. Let\u2019s tweak the preceding code by wrapping the\nBasicRNNCell into an OutputProjectionWrapper:\ncell = tf.contrib.rnn.OutputProjectionWrapper(\n tf.contrib.rnn.BasicRNNCell(num_units=n_neurons, activation=tf.nn.relu),\n output_size=n_outputs)\nSo far, so good. Now we need to define the cost function. We will use the Mean\nSquared Error (MSE), as we did in previous regression tasks. Next we will create an\nAdam optimizer, the training op, and the variable initialization op, as usual:\nlearning_rate = 0.001", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 1029, "content": "learning_rate = 0.001\nloss = tf.reduce_mean(tf.square(outputs - y))\noptimizer = tf.train.AdamOptimizer(learning_rate=learning_rate)\ntraining_op = optimizer.minimize(loss)\ninit = tf.global_variables_initializer()\nNow on to the execution phase:\nn_iterations = 10000\nbatch_size = 50\nwith tf.Session() as sess:\nTraining RNNs \n| \n393\nDownload from finelybook www.finelybook.com", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 1030, "content": "init.run()\n for iteration in range(n_iterations):\n X_batch, y_batch = [...] # fetch the next training batch\n sess.run(training_op, feed_dict={X: X_batch, y: y_batch})\n if iteration % 100 == 0:\n mse = loss.eval(feed_dict={X: X_batch, y: y_batch})\n print(iteration, \"\\tMSE:\", mse)\nThe program\u2019s output should look like this:\n0 MSE: 379.586\n100 MSE: 14.58426\n200 MSE: 7.14066\n300 MSE: 3.98528\n400 MSE: 2.00254\n[...]\nOnce the model is trained, you can make predictions:\nX_new = [...] # New sequences\ny_pred = sess.run(outputs, feed_dict={X: X_new})\nFigure 14-9 shows the predicted sequence for the instance we looked at earlier (in\nFigure 14-7), after just 1,000 training iterations.\nFigure 14-9. Time series prediction\nAlthough using an OutputProjectionWrapper is the simplest solution to reduce the\ndimensionality of the RNN\u2019s output sequences down to just one value per time step", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 1031, "content": "dimensionality of the RNN\u2019s output sequences down to just one value per time step\n(per instance), it is not the most efficient. There is a trickier but more efficient solu\u2010\ntion: you can reshape the RNN outputs from [batch_size, n_steps, n_neurons]\nto [batch_size * n_steps, n_neurons], then apply a single fully connected layer\nwith the appropriate output size (in our case just 1), which will result in an output\ntensor of shape [batch_size * n_steps, n_outputs], and then reshape this tensor\n394 \n| \nChapter 14: Recurrent Neural Networks\nDownload from finelybook www.finelybook.com", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 1032, "content": "to [batch_size, n_steps, n_outputs]. These operations are represented in\nFigure 14-10.\nFigure 14-10. Stack all the outputs, apply the projection, then unstack the result\nTo implement this solution, we first revert to a basic cell, without the OutputProjec\ntionWrapper:\ncell = tf.contrib.rnn.BasicRNNCell(num_units=n_neurons, activation=tf.nn.relu)\nrnn_outputs, states = tf.nn.dynamic_rnn(cell, X, dtype=tf.float32)\nThen we stack all the outputs using the reshape() operation, apply the fully connec\u2010\nted linear layer (without using any activation function; this is just a projection), and\nfinally unstack all the outputs, again using reshape():\nstacked_rnn_outputs = tf.reshape(rnn_outputs, [-1, n_neurons])\nstacked_outputs = fully_connected(stacked_rnn_outputs, n_outputs,\n activation_fn=None)\noutputs = tf.reshape(stacked_outputs, [-1, n_steps, n_outputs])\nTraining RNNs \n| \n395\nDownload from finelybook www.finelybook.com", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 1033, "content": "The rest of the code is the same as earlier. This can provide a significant speed boost\nsince there is just one fully connected layer instead of one per time step.\nCreative RNN\nNow that we have a model that can predict the future, we can use it to generate some\ncreative sequences, as explained at the beginning of the chapter. All we need is to pro\u2010\nvide it a seed sequence containing n_steps values (e.g., full of zeros), use the model to\npredict the next value, append this predicted value to the sequence, feed the last\nn_steps values to the model to predict the next value, and so on. This process gener\u2010\nates a new sequence that has some resemblance to the original time series (see\nFigure 14-11).\nsequence = [0.] * n_steps\nfor iteration in range(300):\n X_batch = np.array(sequence[-n_steps:]).reshape(1, n_steps, 1)\n y_pred = sess.run(outputs, feed_dict={X: X_batch})\n sequence.append(y_pred[0, -1, 0])", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 1034, "content": "y_pred = sess.run(outputs, feed_dict={X: X_batch})\n sequence.append(y_pred[0, -1, 0])\nFigure 14-11. Creative sequences, seeded with zeros (left) or with an instance (right)\nNow you can try to feed all your John Lennon albums to an RNN and see if it can\ngenerate the next \u201cImagine.\u201d However, you will probably need a much more powerful\nRNN, with more neurons, and also much deeper. Let\u2019s look at deep RNNs now.\nDeep RNNs\nIt is quite common to stack multiple layers of cells, as shown in Figure 14-12. This\ngives you a deep RNN.\n396 \n| \nChapter 14: Recurrent Neural Networks\nDownload from finelybook www.finelybook.com", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 1035, "content": "Figure 14-12. Deep RNN (left), unrolled through time (right)\nTo implement a deep RNN in TensorFlow, you can create several cells and stack them\ninto a MultiRNNCell. In the following code we stack three identical cells (but you\ncould very well use various kinds of cells with a different number of neurons):\nn_neurons = 100\nn_layers = 3\nbasic_cell = tf.contrib.rnn.BasicRNNCell(num_units=n_neurons)\nmulti_layer_cell = tf.contrib.rnn.MultiRNNCell([basic_cell] * n_layers)\noutputs, states = tf.nn.dynamic_rnn(multi_layer_cell, X, dtype=tf.float32)\nThat\u2019s all there is to it! The states variable is a tuple containing one tensor per layer,\neach representing the final state of that layer\u2019s cell (with shape [batch_size, n_neu\nrons]). If you set state_is_tuple=False when creating the MultiRNNCell, then\nstates becomes a single tensor containing the states from every layer, concatenated\nalong the column axis (i.e., its shape is [batch_size, n_layers * n_neurons]).", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 1036, "content": "along the column axis (i.e., its shape is [batch_size, n_layers * n_neurons]).\nNote that before TensorFlow 0.11.0, this behavior was the default.\nDistributing a Deep RNN Across Multiple GPUs\nChapter 12 pointed out that we can efficiently distribute deep RNNs across multiple\nGPUs by pinning each layer to a different GPU (see Figure 12-16). However, if you\ntry to create each cell in a different device() block, it will not work:\nwith tf.device(\"/gpu:0\"): # BAD! This is ignored.\n layer1 = tf.contrib.rnn.BasicRNNCell(num_units=n_neurons)\nwith tf.device(\"/gpu:1\"): # BAD! Ignored again.\n layer2 = tf.contrib.rnn.BasicRNNCell(num_units=n_neurons)\nThis fails because a BasicRNNCell is a cell factory, not a cell per se (as mentioned ear\u2010\nlier); no cells get created when you create the factory, and thus no variables do either.\nDeep RNNs \n| \n397\nDownload from finelybook www.finelybook.com", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 1037, "content": "2 This uses the decorator design pattern.\nThe device block is simply ignored. The cells actually get created later. When you call\ndynamic_rnn(), it calls the MultiRNNCell, which calls each individual BasicRNNCell, \nwhich create the actual cells (including their variables). Unfortunately, none of these\nclasses provide any way to control the devices on which the variables get created. If\nyou try to put the dynamic_rnn() call within a device block, the whole RNN gets pin\u2010\nned to a single device. So are you stuck? Fortunately not! The trick is to create your\nown cell wrapper:\nimport tensorflow as tf\nclass DeviceCellWrapper(tf.contrib.rnn.RNNCell):\n def __init__(self, device, cell):\n self._cell = cell\n self._device = device\n @property\n def state_size(self):\n return self._cell.state_size\n @property\n def output_size(self):\n return self._cell.output_size\n def __call__(self, inputs, state, scope=None):\n with tf.device(self._device):", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 1038, "content": "def __call__(self, inputs, state, scope=None):\n with tf.device(self._device):\n return self._cell(inputs, state, scope)\nThis wrapper simply proxies every method call to another cell, except it wraps the\n__call__() function within a device block.2 Now you can distribute each layer on a\ndifferent GPU:\ndevices = [\"/gpu:0\", \"/gpu:1\", \"/gpu:2\"]\ncells = [DeviceCellWrapper(dev,tf.contrib.rnn.BasicRNNCell(num_units=n_neurons))\n for dev in devices]\nmulti_layer_cell = tf.contrib.rnn.MultiRNNCell(cells)\noutputs, states = tf.nn.dynamic_rnn(multi_layer_cell, X, dtype=tf.float32)\nDo not set state_is_tuple=False, or the MultiRNNCell will con\u2010\ncatenate all the cell states into a single tensor, on a single GPU.\n398 \n| \nChapter 14: Recurrent Neural Networks\nDownload from finelybook www.finelybook.com", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 1039, "content": "Applying Dropout\nIf you build a very deep RNN, it may end up overfitting the training set. To prevent\nthat, a common technique is to apply dropout (introduced in Chapter 11). You can\nsimply add a dropout layer before or after the RNN as usual, but if you also want to\napply dropout between the RNN layers, you need to use a DropoutWrapper. The fol\u2010\nlowing code applies dropout to the inputs of each layer in the RNN, dropping each\ninput with a 50% probability:\nkeep_prob = 0.5\ncell = tf.contrib.rnn.BasicRNNCell(num_units=n_neurons)\ncell_drop = tf.contrib.rnn.DropoutWrapper(cell, input_keep_prob=keep_prob)\nmulti_layer_cell = tf.contrib.rnn.MultiRNNCell([cell_drop] * n_layers)\nrnn_outputs, states = tf.nn.dynamic_rnn(multi_layer_cell, X, dtype=tf.float32)\nNote that it is also possible to apply dropout to the outputs by setting out\nput_keep_prob.\nThe main problem with this code is that it will apply dropout not only during train\u2010", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 1040, "content": "put_keep_prob.\nThe main problem with this code is that it will apply dropout not only during train\u2010\ning but also during testing, which is not what you want (recall that dropout should be\napplied only during training). Unfortunately, the DropoutWrapper does not support\nan is_training placeholder (yet?), so you must either write your own dropout wrap\u2010\nper class, or have two different graphs: one for training, and the other for testing. The\nsecond option looks like this:\nimport sys\nis_training = (sys.argv[-1] == \"train\")\nX = tf.placeholder(tf.float32, [None, n_steps, n_inputs])\ny = tf.placeholder(tf.float32, [None, n_steps, n_outputs])\ncell = tf.contrib.rnn.BasicRNNCell(num_units=n_neurons)\nif is_training:\n cell = tf.contrib.rnn.DropoutWrapper(cell, input_keep_prob=keep_prob)\nmulti_layer_cell = tf.contrib.rnn.MultiRNNCell([cell] * n_layers)\nrnn_outputs, states = tf.nn.dynamic_rnn(multi_layer_cell, X, dtype=tf.float32)\n[...] # build the rest of the graph", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 1041, "content": "[...] # build the rest of the graph\ninit = tf.global_variables_initializer()\nsaver = tf.train.Saver()\nwith tf.Session() as sess:\n if is_training:\n init.run()\n for iteration in range(n_iterations):\n [...] # train the model\n save_path = saver.save(sess, \"/tmp/my_model.ckpt\")\n else:\n saver.restore(sess, \"/tmp/my_model.ckpt\")\n [...] # use the model\nDeep RNNs \n| \n399\nDownload from finelybook www.finelybook.com", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 1042, "content": "With that you should be able to train all sorts of RNNs! Unfortunately, if you want to\ntrain an RNN on long sequences, things will get a bit harder. Let\u2019s see why and what\nyou can do about it.\nThe Difficulty of Training over Many Time Steps\nTo train an RNN on long sequences, you will need to run it over many time steps,\nmaking the unrolled RNN a very deep network. Just like any deep neural network it\nmay suffer from the vanishing/exploding gradients problem (discussed in Chap\u2010\nter 11) and take forever to train. Many of the tricks we discussed to alleviate this\nproblem can be used for deep unrolled RNNs as well: good parameter initialization,\nnonsaturating activation functions (e.g., ReLU), Batch Normalization, Gradient Clip\u2010\nping, and faster optimizers. However, if the RNN needs to handle even moderately\nlong sequences (e.g., 100 inputs), then training will still be very slow.\nThe simplest and most common solution to this problem is to unroll the RNN only", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 1043, "content": "The simplest and most common solution to this problem is to unroll the RNN only\nover a limited number of time steps during training. This is called truncated backpro\u2010\npagation through time. In TensorFlow you can implement it simply by truncating the\ninput sequences. For example, in the time series prediction problem, you would sim\u2010\nply reduce n_steps during training. The problem, of course, is that the model will\nnot be able to learn long-term patterns. One workaround could be to make sure that\nthese shortened sequences contain both old and recent data, so that the model can\nlearn to use both (e.g., the sequence could contain monthly data for the last five\nmonths, then weekly data for the last five weeks, then daily data over the last five\ndays). But this workaround has its limits: what if fine-grained data from last year is\nactually useful? What if there was a brief but significant event that absolutely must be\ntaken into account, even years later (e.g., the result of an election)?", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 1044, "content": "taken into account, even years later (e.g., the result of an election)?\nBesides the long training time, a second problem faced by long-running RNNs is the\nfact that the memory of the first inputs gradually fades away. Indeed, due to the trans\u2010\nformations that the data goes through when traversing an RNN, some information is\nlost after each time step. After a while, the RNN\u2019s state contains virtually no trace of\nthe first inputs. This can be a showstopper. For example, say you want to perform\nsentiment analysis on a long review that starts with the four words \u201cI loved this\nmovie,\u201d but the rest of the review lists the many things that could have made the\nmovie even better. If the RNN gradually forgets the first four words, it will completely\nmisinterpret the review. To solve this problem, various types of cells with long-term\nmemory have been introduced. They have proved so successful that the basic cells are", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 1045, "content": "memory have been introduced. They have proved so successful that the basic cells are\nnot much used anymore. Let\u2019s first look at the most popular of these long memory\ncells: the LSTM cell.\n400 \n| \nChapter 14: Recurrent Neural Networks\nDownload from finelybook www.finelybook.com", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 1046, "content": "3 \u201cLong Short-Term Memory,\u201d S. Hochreiter and J. Schmidhuber (1997).\n4 \u201cLong Short-Term Memory Recurrent Neural Network Architectures for Large Scale Acoustic Modeling,\u201d H.\nSak et al. (2014).\n5 \u201cRecurrent Neural Network Regularization,\u201d W. Zaremba et al. (2015).\nLSTM Cell\nThe Long Short-Term Memory (LSTM) cell was proposed in 19973 by Sepp Hochreiter\nand J\u00fcrgen Schmidhuber, and it was gradually improved over the years by several\nresearchers, such as Alex Graves, Ha\u015fim Sak,4 Wojciech Zaremba,5 and many more. If\nyou consider the LSTM cell as a black box, it can be used very much like a basic cell,\nexcept it will perform much better; training will converge faster and it will detect\nlong-term dependencies in the data. In TensorFlow, you can simply use a BasicLSTM\nCell instead of a BasicRNNCell:\nlstm_cell = tf.contrib.rnn.BasicLSTMCell(num_units=n_neurons)\nLSTM cells manage two state vectors, and for performance reasons they are kept", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 1047, "content": "LSTM cells manage two state vectors, and for performance reasons they are kept\nseparate by default. You can change this default behavior by setting\nstate_is_tuple=False when creating the BasicLSTMCell.\nSo how does an LSTM cell work? The architecture of a basic LSTM cell is shown in\nFigure 14-13.\nFigure 14-13. LSTM cell\nLSTM Cell \n| \n401\nDownload from finelybook www.finelybook.com", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 1048, "content": "If you don\u2019t look at what\u2019s inside the box, the LSTM cell looks exactly like a regular\ncell, except that its state is split in two vectors: h(t) and c(t) (\u201cc\u201d stands for \u201ccell\u201d). You\ncan think of h(t) as the short-term state and c(t) as the long-term state.\nNow let\u2019s open the box! The key idea is that the network can learn what to store in the\nlong-term state, what to throw away, and what to read from it. As the long-term state\nc(t\u20131) traverses the network from left to right, you can see that it first goes through a\nforget gate, dropping some memories, and then it adds some new memories via the\naddition operation (which adds the memories that were selected by an input gate).\nThe result c(t) is sent straight out, without any further transformation. So, at each time\nstep, some memories are dropped and some memories are added. Moreover, after the\naddition operation, the long-term state is copied and passed through the tanh func\u2010", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 1049, "content": "addition operation, the long-term state is copied and passed through the tanh func\u2010\ntion, and then the result is filtered by the output gate. This produces the short-term\nstate h(t) (which is equal to the cell\u2019s output for this time step y(t)). Now let\u2019s look at\nwhere new memories come from and how the gates work.\nFirst, the current input vector x(t) and the previous short-term state h(t\u20131) are fed to\nfour different fully connected layers. They all serve a different purpose:\n\u2022 The main layer is the one that outputs g(t). It has the usual role of analyzing the\ncurrent inputs x(t) and the previous (short-term) state h(t\u20131). In a basic cell, there is\nnothing else than this layer, and its output goes straight out to y(t) and h(t). In con\u2010\ntrast, in an LSTM cell this layer\u2019s output does not go straight out, but instead it is\npartially stored in the long-term state.\n\u2022 The three other layers are gate controllers. Since they use the logistic activation", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 1050, "content": "\u2022 The three other layers are gate controllers. Since they use the logistic activation\nfunction, their outputs range from 0 to 1. As you can see, their outputs are fed to\nelement-wise multiplication operations, so if they output 0s, they close the gate,\nand if they output 1s, they open it. Specifically:\n\u2014 The forget gate (controlled by f(t)) controls which parts of the long-term state\nshould be erased.\n\u2014 The input gate (controlled by i(t)) controls which parts of g(t) should be added\nto the long-term state (this is why we said it was only \u201cpartially stored\u201d).\n\u2014 Finally, the output gate (controlled by o(t)) controls which parts of the long-\nterm state should be read and output at this time step (both to h(t)) and y(t).\nIn short, an LSTM cell can learn to recognize an important input (that\u2019s the role of the\ninput gate), store it in the long-term state, learn to preserve it for as long as it is\nneeded (that\u2019s the role of the forget gate), and learn to extract it whenever it is needed.", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 1051, "content": "needed (that\u2019s the role of the forget gate), and learn to extract it whenever it is needed.\nThis explains why they have been amazingly successful at capturing long-term pat\u2010\nterns in time series, long texts, audio recordings, and more.\n402 \n| \nChapter 14: Recurrent Neural Networks\nDownload from finelybook www.finelybook.com", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 1052, "content": "6 \u201cRecurrent Nets that Time and Count,\u201d F. Gers and J. Schmidhuber (2000).\nEquation 14-3 summarizes how to compute the cell\u2019s long-term state, its short-term\nstate, and its output at each time step for a single instance (the equations for a whole\nmini-batch are very similar).\nEquation 14-3. LSTM computations\n\ufffdt = \u03c3 \ufffdxi\nT \u00b7 \ufffdt + \ufffdhi\nT \u00b7 \ufffdt \u22121 + \ufffdi\n\ufffdt = \u03c3 \ufffdxf\nT \u00b7 \ufffdt + \ufffdhf\nT \u00b7 \ufffdt \u22121 + \ufffdf\n\ufffdt = \u03c3 \ufffdxo\nT \u00b7 \ufffdt + \ufffdho\nT \u00b7 \ufffdt \u22121 + \ufffdo\n\ufffdt = tanh \ufffdxg\nT \u00b7 \ufffdt + \ufffdhg\nT \u00b7 \ufffdt \u22121 + \ufffdg\n\ufffdt = \ufffdt \u2297\ufffdt \u22121 + \ufffdt \u2297\ufffdt\n\ufffdt = \ufffdt = \ufffdt \u2297tanh \ufffdt\n\u2022 Wxi, Wxf, Wxo, Wxg are the weight matrices of each of the four layers for their con\u2010\nnection to the input vector x(t).\n\u2022 Whi, Whf, Who, and Whg are the weight matrices of each of the four layers for their\nconnection to the previous short-term state h(t\u20131).\n\u2022 bi, bf, bo, and bg are the bias terms for each of the four layers. Note that Tensor\u2010\nFlow initializes bf to a vector full of 1s instead of 0s. This prevents forgetting\neverything at the beginning of training.", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 1053, "content": "everything at the beginning of training.\nPeephole Connections\nIn a basic LSTM cell, the gate controllers can look only at the input x(t) and the previ\u2010\nous short-term state h(t\u20131). It may be a good idea to give them a bit more context by\nletting them peek at the long-term state as well. This idea was proposed by Felix Gers\nand J\u00fcrgen Schmidhuber in 2000.6 They proposed an LSTM variant with extra con\u2010\nnections called peephole connections: the previous long-term state c(t\u20131) is added as an\ninput to the controllers of the forget gate and the input gate, and the current long-\nterm state c(t) is added as input to the controller of the output gate.\nTo implement peephole connections in TensorFlow, you must use the LSTMCell\ninstead of the BasicLSTMCell and set use_peepholes=True:\nlstm_cell = tf.contrib.rnn.LSTMCell(num_units=n_neurons, use_peepholes=True)\nLSTM Cell \n| \n403\nDownload from finelybook www.finelybook.com", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 1054, "content": "7 \u201cLearning Phrase Representations using RNN Encoder\u2013Decoder for Statistical Machine Translation,\u201d K. Cho\net al. (2014).\n8 A 2015 paper by Klaus Greff et al., \u201cLSTM: A Search Space Odyssey,\u201d seems to show that all LSTM variants\nperform roughly the same.\nThere are many other variants of the LSTM cell. One particularly popular variant is\nthe GRU cell, which we will look at now.\nGRU Cell\nThe Gated Recurrent Unit (GRU) cell (see Figure 14-14) was proposed by Kyunghyun\nCho et al. in a 2014 paper7 that also introduced the Encoder\u2013Decoder network we\nmentioned earlier.\nFigure 14-14. GRU cell\nThe GRU cell is a simplified version of the LSTM cell, and it seems to perform just as\nwell8 (which explains its growing popularity). The main simplifications are:\n\u2022 Both state vectors are merged into a single vector h(t).\n\u2022 A single gate controller controls both the forget gate and the input gate. If the\ngate controller outputs a 1, the input gate is open and the forget gate is closed. If\n404 \n|", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 1055, "content": "gate controller outputs a 1, the input gate is open and the forget gate is closed. If\n404 \n| \nChapter 14: Recurrent Neural Networks\nDownload from finelybook www.finelybook.com", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 1056, "content": "it outputs a 0, the opposite happens. In other words, whenever a memory must\nbe stored, the location where it will be stored is erased first. This is actually a fre\u2010\nquent variant to the LSTM cell in and of itself.\n\u2022 There is no output gate; the full state vector is output at every time step. How\u2010\never, there is a new gate controller that controls which part of the previous state\nwill be shown to the main layer.\nEquation 14-4 summarizes how to compute the cell\u2019s state at each time step for a sin\u2010\ngle instance.\nEquation 14-4. GRU computations\n\ufffdt = \u03c3 \ufffdxz\nT \u00b7 \ufffdt + \ufffdhz\nT \u00b7 \ufffdt \u22121\n\ufffdt = \u03c3 \ufffdxr\nT \u00b7 \ufffdt + \ufffdhr\nT \u00b7 \ufffdt \u22121\n\ufffdt = tanh \ufffdxg\nT \u00b7 \ufffdt + \ufffdhg\nT \u00b7 \ufffdt \u2297\ufffdt \u22121\n\ufffdt = 1 \u2212\ufffdt\n\u2297tanh \ufffdxg\nT \u00b7 \ufffdt \u22121 + \ufffdt \u2297\ufffdt\nCreating a GRU cell in TensorFlow is trivial:\ngru_cell = tf.contrib.rnn.GRUCell(num_units=n_neurons)\nLSTM or GRU cells are one of the main reasons behind the success of RNNs in recent\nyears, in particular for applications in natural language processing (NLP).\nNatural Language Processing", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 1057, "content": "Natural Language Processing\nMost of the state-of-the-art NLP applications, such as machine translation, automatic\nsummarization, parsing, sentiment analysis, and more, are now based (at least in\npart) on RNNs. In this last section, we will take a quick look at what a machine trans\u2010\nlation model looks like. This topic is very well covered by TensorFlow\u2019s awesome\nWord2Vec and Seq2Seq tutorials, so you should definitely check them out.\nWord Embeddings\nBefore we start, we need to choose a word representation. One option could be to\nrepresent each word using a one-hot vector. Suppose your vocabulary contains\n50,000 words, then the nth word would be represented as a 50,000-dimensional vector,\nfull of 0s except for a 1 at the nth position. However, with such a large vocabulary, this\nsparse representation would not be efficient at all. Ideally, you want similar words to\nhave similar representations, making it easy for the model to generalize what it learns", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 1058, "content": "have similar representations, making it easy for the model to generalize what it learns\nabout a word to all similar words. For example, if the model is told that \u201cI drink milk\u201d\nis a valid sentence, and if it knows that \u201cmilk\u201d is close to \u201cwater\u201d but far from \u201cshoes,\u201d\nNatural Language Processing \n| \n405\nDownload from finelybook www.finelybook.com", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 1059, "content": "9 For more details, check out Christopher Olah\u2019s great post, or Sebastian Ruder\u2019s series of posts.\nthen it will know that \u201cI drink water\u201d is probably a valid sentence as well, while \u201cI\ndrink shoes\u201d is probably not. But how can you come up with such a meaningful rep\u2010\nresentation?\nThe most common solution is to represent each word in the vocabulary using a fairly\nsmall and dense vector (e.g., 150 dimensions), called an embedding, and just let the\nneural network learn a good embedding for each word during training. At the begin\u2010\nning of training, embeddings are simply chosen randomly, but during training, back\u2010\npropagation automatically moves the embeddings around in a way that helps the\nneural network perform its task. Typically this means that similar words will gradu\u2010\nally cluster close to one another, and even end up organized in a rather meaningful\nway. For example, embeddings may end up placed along various axes that represent", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 1060, "content": "way. For example, embeddings may end up placed along various axes that represent\ngender, singular/plural, adjective/noun, and so on. The result can be truly amazing.9\nIn TensorFlow, you first need to create the variable representing the embeddings for\nevery word in your vocabulary (initialized randomly):\nvocabulary_size = 50000\nembedding_size = 150\nembeddings = tf.Variable(\n tf.random_uniform([vocabulary_size, embedding_size], -1.0, 1.0))\nNow suppose you want to feed the sentence \u201cI drink milk\u201d to your neural network.\nYou should first preprocess the sentence and break it into a list of known words. For\nexample you may remove unnecessary characters, replace unknown words by a pre\u2010\ndefined token word such as \u201c[UNK]\u201d, replace numerical values by \u201c[NUM]\u201d, replace\nURLs by \u201c[URL]\u201d, and so on. Once you have a list of known words, you can look up\neach word\u2019s integer identifier (from 0 to 49999) in a dictionary, for example [72, 3335,", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 1061, "content": "each word\u2019s integer identifier (from 0 to 49999) in a dictionary, for example [72, 3335,\n288]. At that point, you are ready to feed these word identifiers to TensorFlow using a\nplaceholder, and apply the embedding_lookup() function to get the corresponding\nembeddings:\ntrain_inputs = tf.placeholder(tf.int32, shape=[None]) # from ids...\nembed = tf.nn.embedding_lookup(embeddings, train_inputs) # ...to embeddings\nOnce your model has learned good word embeddings, they can actually be reused\nfairly efficiently in any NLP application: after all, \u201cmilk\u201d is still close to \u201cwater\u201d and far\nfrom \u201cshoes\u201d no matter what your application is. In fact, instead of training your own\nword embeddings, you may want to download pretrained word embeddings. Just like\nwhen reusing pretrained layers (see Chapter 11), you can choose to freeze the pre\u2010\ntrained embeddings (e.g., creating the embeddings variable using trainable=False)", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 1062, "content": "trained embeddings (e.g., creating the embeddings variable using trainable=False)\nor let backpropagation tweak them for your application. The first option will speed\nup training, but the second may lead to slightly higher performance.\n406 \n| \nChapter 14: Recurrent Neural Networks\nDownload from finelybook www.finelybook.com", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 1063, "content": "10 \u201cSequence to Sequence learning with Neural Networks,\u201d I. Sutskever et al. (2014).\nEmbeddings are also useful for representing categorical attributes\nthat can take on a large number of different values, especially when\nthere are complex similarities between values. For example, con\u2010\nsider professions, hobbies, dishes, species, brands, and so on.\nYou now have almost all the tools you need to implement a machine translation sys\u2010\ntem. Let\u2019s look at this now.\nAn Encoder\u2013Decoder Network for Machine Translation\nLet\u2019s take a look at a simple machine translation model10 that will translate English\nsentences to French (see Figure 14-15).\nFigure 14-15. A simple machine translation model\nThe English sentences are fed to the encoder, and the decoder outputs the French\ntranslations. Note that the French translations are also used as inputs to the decoder,\nbut pushed back by one step. In other words, the decoder is given as input the word", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 1064, "content": "but pushed back by one step. In other words, the decoder is given as input the word\nthat it should have output at the previous step (regardless of what it actually output).\nFor the very first word, it is given a token that represents the beginning of the sen\u2010\nNatural Language Processing \n| \n407\nDownload from finelybook www.finelybook.com", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 1065, "content": "tence (e.g., \u201c\u201d). The decoder is expected to end the sentence with an end-of-\nsequence (EOS) token (e.g., \u201c\u201d).\nNote that the English sentences are reversed before they are fed to the encoder. For\nexample \u201cI drink milk\u201d is reversed to \u201cmilk drink I.\u201d This ensures that the beginning\nof the English sentence will be fed last to the encoder, which is useful because that\u2019s\ngenerally the first thing that the decoder needs to translate.\nEach word is initially represented by a simple integer identifier (e.g., 288 for the word\n\u201cmilk\u201d). Next, an embedding lookup returns the word embedding (as explained ear\u2010\nlier, this is a dense, fairly low-dimensional vector). These word embeddings are what\nis actually fed to the encoder and the decoder.\nAt each step, the decoder outputs a score for each word in the output vocabulary (i.e.,\nFrench), and then the Softmax layer turns these scores into probabilities. For exam\u2010", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 1066, "content": "French), and then the Softmax layer turns these scores into probabilities. For exam\u2010\nple, at the first step the word \u201cJe\u201d may have a probability of 20%, \u201cTu\u201d may have a\nprobability of 1%, and so on. The word with the highest probability is output. This is\nvery much like a regular classification task, so you can train the model using the soft\nmax_cross_entropy_with_logits() function.\nNote that at inference time (after training), you will not have the target sentence to\nfeed to the decoder. Instead, simply feed the decoder the word that it output at the\nprevious step, as shown in Figure 14-16 (this will require an embedding lookup that\nis not shown on the diagram).\nFigure 14-16. Feeding the previous output word as input at inference time\nOkay, now you have the big picture. However, if you go through TensorFlow\u2019s\nsequence-to-sequence tutorial and you look at the code in rnn/translate/\nseq2seq_model.py (in the TensorFlow models), you will notice a few important differ\u2010\nences:\n408 \n|", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 1067, "content": "seq2seq_model.py (in the TensorFlow models), you will notice a few important differ\u2010\nences:\n408 \n| \nChapter 14: Recurrent Neural Networks\nDownload from finelybook www.finelybook.com", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 1068, "content": "11 The bucket sizes used in the tutorial are different.\n12 \u201cOn Using Very Large Target Vocabulary for Neural Machine Translation,\u201d S. Jean et al. (2015).\n13 \u201cNeural Machine Translation by Jointly Learning to Align and Translate,\u201d D. Bahdanau et al. (2014).\n14 \u201cLong Short-Term Memory-Networks for Machine Reading,\u201d J. Cheng (2016).\n15 \u201cShow, Attend and Tell: Neural Image Caption Generation with Visual Attention,\u201d K. Xu et al. (2015).\n\u2022 First, so far we have assumed that all input sequences (to the encoder and to the\ndecoder) have a constant length. But obviously sentence lengths may vary. There\nare several ways that this can be handled\u2014for example, using the\nsequence_length argument to the static_rnn() or dynamic_rnn() functions to\nspecify each sentence\u2019s length (as discussed earlier). However, another approach\nis used in the tutorial (presumably for performance reasons): sentences are grou\u2010\nped into buckets of similar lengths (e.g., a bucket for the 1- to 6-word sentences,", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 1069, "content": "ped into buckets of similar lengths (e.g., a bucket for the 1- to 6-word sentences,\nanother for the 7- to 12-word sentences, and so on11), and the shorter sentences\nare padded using a special padding token (e.g., \u201c\u201d). For example \u201cI drink\nmilk\u201d becomes \u201c milk drink I\u201d, and its translation becomes\n\u201cJe bois du lait \u201d. Of course, we want to ignore any output past the\nEOS token. For this, the tutorial\u2019s implementation uses a target_weights vector.\nFor example, for the target sentence \u201cJe bois du lait \u201d, the weights\nwould be set to [1.0, 1.0, 1.0, 1.0, 1.0, 0.0] (notice the weight 0.0 that\ncorresponds to the padding token in the target sentence). Simply multiplying the\nlosses by the target weights will zero out the losses that correspond to words past\nEOS tokens.\n\u2022 Second, when the output vocabulary is large (which is the case here), outputting\na probability for each and every possible word would be terribly slow. If the tar\u2010", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 1070, "content": "a probability for each and every possible word would be terribly slow. If the tar\u2010\nget vocabulary contains, say, 50,000 French words, then the decoder would out\u2010\nput 50,000-dimensional vectors, and then computing the softmax function over\nsuch a large vector would be very computationally intensive. To avoid this, one\nsolution is to let the decoder output much smaller vectors, such as 1,000-\ndimensional vectors, then use a sampling technique to estimate the loss without\nhaving to compute it over every single word in the target vocabulary. This Sam\u2010\npled Softmax technique was introduced in 2015 by S\u00e9bastien Jean et al.12 In Ten\u2010\nsorFlow you can use the sampled_softmax_loss() function.\n\u2022 Third, the tutorial\u2019s implementation uses an attention mechanism that lets the\ndecoder peek into the input sequence. Attention augmented RNNs are beyond\nthe scope of this book, but if you are interested there are helpful papers about", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 1071, "content": "the scope of this book, but if you are interested there are helpful papers about\nmachine translation,13 machine reading,14 and image captions15 using attention.\n\u2022 Finally, the tutorial\u2019s implementation makes use of the tf.nn.legacy_seq2seq\nmodule, which provides tools to build various Encoder\u2013Decoder models easily.\nNatural Language Processing \n| \n409\nDownload from finelybook www.finelybook.com", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 1072, "content": "For example, the embedding_rnn_seq2seq() function creates a simple Encoder\u2013\nDecoder model that automatically takes care of word embeddings for you, just\nlike the one represented in Figure 14-15. This code will likely be updated quickly\nto use the new tf.nn.seq2seq module.\nYou now have all the tools you need to understand the sequence-to-sequence tutor\u2010\nial\u2019s implementation. Check it out and train your own English-to-French translator!\nExercises\n1. Can you think of a few applications for a sequence-to-sequence RNN? What\nabout a sequence-to-vector RNN? And a vector-to-sequence RNN?\n2. Why do people use encoder\u2013decoder RNNs rather than plain sequence-to-\nsequence RNNs for automatic translation?\n3. How could you combine a convolutional neural network with an RNN to classify\nvideos?\n4. What are the advantages of building an RNN using dynamic_rnn() rather than\nstatic_rnn()?\n5. How can you deal with variable-length input sequences? What about variable-\nlength output sequences?", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 1073, "content": "length output sequences?\n6. What is a common way to distribute training and execution of a deep RNN\nacross multiple GPUs?\n7. Embedded Reber grammars were used by Hochreiter and Schmidhuber in their\npaper about LSTMs. They are artificial grammars that produce strings such as\n\u201cBPBTSXXVPSEPE.\u201d Check out Jenny Orr\u2019s nice introduction to this topic.\nChoose a particular embedded Reber grammar (such as the one represented on\nJenny Orr\u2019s page), then train an RNN to identify whether a string respects that\ngrammar or not. You will first need to write a function capable of generating a\ntraining batch containing about 50% strings that respect the grammar, and 50%\nthat don\u2019t.\n8. Tackle the \u201cHow much did it rain? II\u201d Kaggle competition. This is a time series\nprediction task: you are given snapshots of polarimetric radar values and asked to\npredict the hourly rain gauge total. Luis Andre Dutra e Silva\u2019s interview gives", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 1074, "content": "predict the hourly rain gauge total. Luis Andre Dutra e Silva\u2019s interview gives\nsome interesting insights into the techniques he used to reach second place in the\ncompetition. In particular, he used an RNN composed of two LSTM layers.\n9. Go through TensorFlow\u2019s Word2Vec tutorial to create word embeddings, and\nthen go through the Seq2Seq tutorial to train an English-to-French translation\nsystem.\nSolutions to these exercises are available in Appendix A.\n410 \n| \nChapter 14: Recurrent Neural Networks\nDownload from finelybook www.finelybook.com", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 1075, "content": "CHAPTER 15\nAutoencoders\nAutoencoders are artificial neural networks capable of learning efficient representa\u2010\ntions of the input data, called codings, without any supervision (i.e., the training set is\nunlabeled). These codings typically have a much lower dimensionality than the input\ndata, making autoencoders useful for dimensionality reduction (see Chapter 8). More\nimportantly, autoencoders act as powerful feature detectors, and they can be used for\nunsupervised pretraining of deep neural networks (as we discussed in Chapter 11).\nLastly, they are capable of randomly generating new data that looks very similar to the\ntraining data; this is called a generative model. For example, you could train an\nautoencoder on pictures of faces, and it would then be able to generate new faces.\nSurprisingly, autoencoders work by simply learning to copy their inputs to their out\u2010\nputs. This may sound like a trivial task, but we will see that constraining the network", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 1076, "content": "puts. This may sound like a trivial task, but we will see that constraining the network\nin various ways can make it rather difficult. For example, you can limit the size of the\ninternal representation, or you can add noise to the inputs and train the network to\nrecover the original inputs. These constraints prevent the autoencoder from trivially\ncopying the inputs directly to the outputs, which forces it to learn efficient ways of\nrepresenting the data. In short, the codings are byproducts of the autoencoder\u2019s\nattempt to learn the identity function under some constraints.\nIn this chapter we will explain in more depth how autoencoders work, what types of\nconstraints can be imposed, and how to implement them using TensorFlow, whether\nit is for dimensionality reduction, feature extraction, unsupervised pretraining, or as\ngenerative models.\n411\nDownload from finelybook www.finelybook.com", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 1077, "content": "1 \u201cPerception in chess,\u201d W. Chase and H. Simon (1973).\nEfficient Data Representations\nWhich of the following number sequences do you find the easiest to memorize?\n\u2022 40, 27, 25, 36, 81, 57, 10, 73, 19, 68\n\u2022 50, 25, 76, 38, 19, 58, 29, 88, 44, 22, 11, 34, 17, 52, 26, 13, 40, 20\nAt first glance, it would seem that the first sequence should be easier, since it is much\nshorter. However, if you look carefully at the second sequence, you may notice that it\nfollows two simple rules: even numbers are followed by their half, and odd numbers\nare followed by their triple plus one (this is a famous sequence known as the hailstone\nsequence). Once you notice this pattern, the second sequence becomes much easier to\nmemorize than the first because you only need to memorize the two rules, the first\nnumber, and the length of the sequence. Note that if you could quickly and easily\nmemorize very long sequences, you would not care much about the existence of a", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 1078, "content": "memorize very long sequences, you would not care much about the existence of a\npattern in the second sequence. You would just learn every number by heart, and that\nwould be that. It is the fact that it is hard to memorize long sequences that makes it\nuseful to recognize patterns, and hopefully this clarifies why constraining an autoen\u2010\ncoder during training pushes it to discover and exploit patterns in the data.\nThe relationship between memory, perception, and pattern matching was famously\nstudied by William Chase and Herbert Simon in the early 1970s.1 They observed that\nexpert chess players were able to memorize the positions of all the pieces in a game by\nlooking at the board for just 5 seconds, a task that most people would find impossible.\nHowever, this was only the case when the pieces were placed in realistic positions\n(from actual games), not when the pieces were placed randomly. Chess experts don\u2019t", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 1079, "content": "(from actual games), not when the pieces were placed randomly. Chess experts don\u2019t\nhave a much better memory than you and I, they just see chess patterns more easily\nthanks to their experience with the game. Noticing patterns helps them store infor\u2010\nmation efficiently.\nJust like the chess players in this memory experiment, an autoencoder looks at the\ninputs, converts them to an efficient internal representation, and then spits out some\u2010\nthing that (hopefully) looks very close to the inputs. An autoencoder is always com\u2010\nposed of two parts: an encoder (or recognition network) that converts the inputs to an\ninternal representation, followed by a decoder (or generative network) that converts\nthe internal representation to the outputs (see Figure 15-1).\nAs you can see, an autoencoder typically has the same architecture as a Multi-Layer\nPerceptron (MLP; see Chapter 10), except that the number of neurons in the output", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 1080, "content": "Perceptron (MLP; see Chapter 10), except that the number of neurons in the output\nlayer must be equal to the number of inputs. In this example, there is just one hidden\n412 \n| \nChapter 15: Autoencoders\nDownload from finelybook www.finelybook.com", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 1081, "content": "layer composed of two neurons (the encoder), and one output layer composed of\nthree neurons (the decoder). The outputs are often called the reconstructions since the\nautoencoder tries to reconstruct the inputs, and the cost function contains a recon\u2010\nstruction loss that penalizes the model when the reconstructions are different from the\ninputs.\nFigure 15-1. The chess memory experiment (left) and a simple autoencoder (right)\nBecause the internal representation has a lower dimensionality than the input data (it\nis 2D instead of 3D), the autoencoder is said to be undercomplete. An undercomplete\nautoencoder cannot trivially copy its inputs to the codings, yet it must find a way to\noutput a copy of its inputs. It is forced to learn the most important features in the\ninput data (and drop the unimportant ones).\nLet\u2019s see how to implement a very simple undercomplete autoencoder for dimension\u2010\nality reduction.\nPerforming PCA with an Undercomplete Linear\nAutoencoder", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 1082, "content": "ality reduction.\nPerforming PCA with an Undercomplete Linear\nAutoencoder\nIf the autoencoder uses only linear activations and the cost function is the Mean\nSquared Error (MSE), then it can be shown that it ends up performing Principal\nComponent Analysis (see Chapter 8).\nThe following code builds a simple linear autoencoder to perform PCA on a 3D data\u2010\nset, projecting it to 2D:\nimport tensorflow as tf\nfrom tensorflow.contrib.layers import fully_connected\nn_inputs = 3 # 3D inputs\nn_hidden = 2 # 2D codings\nPerforming PCA with an Undercomplete Linear Autoencoder \n| \n413\nDownload from finelybook www.finelybook.com", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 1083, "content": "n_outputs = n_inputs\nlearning_rate = 0.01\nX = tf.placeholder(tf.float32, shape=[None, n_inputs])\nhidden = fully_connected(X, n_hidden, activation_fn=None)\noutputs = fully_connected(hidden, n_outputs, activation_fn=None)\nreconstruction_loss = tf.reduce_mean(tf.square(outputs - X)) # MSE\noptimizer = tf.train.AdamOptimizer(learning_rate)\ntraining_op = optimizer.minimize(reconstruction_loss)\ninit = tf.global_variables_initializer()\nThis code is really not very different from all the MLPs we built in past chapters. The\ntwo things to note are:\n\u2022 The number of outputs is equal to the number of inputs.\n\u2022 To perform simple PCA, we set activation_fn=None (i.e., all neurons are linear)\nand the cost function is the MSE. We will see more complex autoencoders\nshortly.\nNow let\u2019s load the dataset, train the model on the training set, and use it to encode the\ntest set (i.e., project it to 2D):\nX_train, X_test = [...] # load the dataset\nn_iterations = 1000", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 1084, "content": "test set (i.e., project it to 2D):\nX_train, X_test = [...] # load the dataset\nn_iterations = 1000\ncodings = hidden # the output of the hidden layer provides the codings\nwith tf.Session() as sess:\n init.run()\n for iteration in range(n_iterations):\n training_op.run(feed_dict={X: X_train}) # no labels (unsupervised)\n codings_val = codings.eval(feed_dict={X: X_test})\nFigure 15-2 shows the original 3D dataset (at the left) and the output of the autoen\u2010\ncoder\u2019s hidden layer (i.e., the coding layer, at the right). As you can see, the autoen\u2010\ncoder found the best 2D plane to project the data onto, preserving as much variance\nin the data as it could (just like PCA).\n414 \n| \nChapter 15: Autoencoders\nDownload from finelybook www.finelybook.com", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 1085, "content": "Figure 15-2. PCA performed by an undercomplete linear autoencoder\nStacked Autoencoders\nJust like other neural networks we have discussed, autoencoders can have multiple\nhidden layers. In this case they are called stacked autoencoders (or deep autoencoders). \nAdding more layers helps the autoencoder learn more complex codings. However,\none must be careful not to make the autoencoder too powerful. Imagine an encoder\nso powerful that it just learns to map each input to a single arbitrary number (and the\ndecoder learns the reverse mapping). Obviously such an autoencoder will reconstruct\nthe training data perfectly, but it will not have learned any useful data representation\nin the process (and it is unlikely to generalize well to new instances).\nThe architecture of a stacked autoencoder is typically symmetrical with regards to the\ncentral hidden layer (the coding layer). To put it simply, it looks like a sandwich. For", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 1086, "content": "central hidden layer (the coding layer). To put it simply, it looks like a sandwich. For\nexample, an autoencoder for MNIST (introduced in Chapter 3) may have 784 inputs,\nfollowed by a hidden layer with 300 neurons, then a central hidden layer of 150 neu\u2010\nrons, then another hidden layer with 300 neurons, and an output layer with 784 neu\u2010\nrons. This stacked autoencoder is represented in Figure 15-3.\nFigure 15-3. Stacked autoencoder\nStacked Autoencoders \n| \n415\nDownload from finelybook www.finelybook.com", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 1087, "content": "TensorFlow Implementation\nYou can implement a stacked autoencoder very much like a regular deep MLP. In par\u2010\nticular, the same techniques we used in Chapter 11 for training deep nets can be\napplied. For example, the following code builds a stacked autoencoder for MNIST,\nusing He initialization, the ELU activation function, and \u21132 regularization. The code\nshould look very familiar, except that there are no labels (no y):\nn_inputs = 28 * 28 # for MNIST\nn_hidden1 = 300\nn_hidden2 = 150 # codings\nn_hidden3 = n_hidden1\nn_outputs = n_inputs\nlearning_rate = 0.01\nl2_reg = 0.001\nX = tf.placeholder(tf.float32, shape=[None, n_inputs])\nwith tf.contrib.framework.arg_scope(\n [fully_connected],\n activation_fn=tf.nn.elu,\n weights_initializer=tf.contrib.layers.variance_scaling_initializer(),\n weights_regularizer=tf.contrib.layers.l2_regularizer(l2_reg)):\n hidden1 = fully_connected(X, n_hidden1)\n hidden2 = fully_connected(hidden1, n_hidden2) # codings", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 1088, "content": "hidden2 = fully_connected(hidden1, n_hidden2) # codings\n hidden3 = fully_connected(hidden2, n_hidden3)\n outputs = fully_connected(hidden3, n_outputs, activation_fn=None)\nreconstruction_loss = tf.reduce_mean(tf.square(outputs - X)) # MSE\nreg_losses = tf.get_collection(tf.GraphKeys.REGULARIZATION_LOSSES)\nloss = tf.add_n([reconstruction_loss] + reg_losses)\noptimizer = tf.train.AdamOptimizer(learning_rate)\ntraining_op = optimizer.minimize(loss)\ninit = tf.global_variables_initializer()\nYou can then train the model normally. Note that the digit labels (y_batch) are\nunused:\nn_epochs = 5\nbatch_size = 150\nwith tf.Session() as sess:\n init.run()\n for epoch in range(n_epochs):\n n_batches = mnist.train.num_examples // batch_size\n for iteration in range(n_batches):\n X_batch, y_batch = mnist.train.next_batch(batch_size)\n sess.run(training_op, feed_dict={X: X_batch})\n416 \n| \nChapter 15: Autoencoders\nDownload from finelybook www.finelybook.com", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 1089, "content": "Tying Weights\nWhen an autoencoder is neatly symmetrical, like the one we just built, a common\ntechnique is to tie the weights of the decoder layers to the weights of the encoder lay\u2010\ners. This halves the number of weights in the model, speeding up training and limit\u2010\ning the risk of overfitting. Specifically, if the autoencoder has a total of N layers (not\ncounting the input layer), and WL represents the connection weights of the Lth layer\n(e.g., layer 1 is the first hidden layer, layer N\n2 is the coding layer, and layer N is the\noutput layer), then the decoder layer weights can be defined simply as: WN\u2013L+1 = WL\nT\n(with L = 1, 2, \u22ef, N\n2 ).\nUnfortunately, implementing tied weights in TensorFlow using the fully_connec\nted() function is a bit cumbersome; it\u2019s actually easier to just define the layers man\u2010\nually. The code ends up significantly more verbose:\nactivation = tf.nn.elu\nregularizer = tf.contrib.layers.l2_regularizer(l2_reg)", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 1090, "content": "activation = tf.nn.elu\nregularizer = tf.contrib.layers.l2_regularizer(l2_reg)\ninitializer = tf.contrib.layers.variance_scaling_initializer()\nX = tf.placeholder(tf.float32, shape=[None, n_inputs])\nweights1_init = initializer([n_inputs, n_hidden1])\nweights2_init = initializer([n_hidden1, n_hidden2])\nweights1 = tf.Variable(weights1_init, dtype=tf.float32, name=\"weights1\")\nweights2 = tf.Variable(weights2_init, dtype=tf.float32, name=\"weights2\")\nweights3 = tf.transpose(weights2, name=\"weights3\") # tied weights\nweights4 = tf.transpose(weights1, name=\"weights4\") # tied weights\nbiases1 = tf.Variable(tf.zeros(n_hidden1), name=\"biases1\")\nbiases2 = tf.Variable(tf.zeros(n_hidden2), name=\"biases2\")\nbiases3 = tf.Variable(tf.zeros(n_hidden3), name=\"biases3\")\nbiases4 = tf.Variable(tf.zeros(n_outputs), name=\"biases4\")\nhidden1 = activation(tf.matmul(X, weights1) + biases1)\nhidden2 = activation(tf.matmul(hidden1, weights2) + biases2)\nhidden3 = activation(tf.matmul(hidden2, weights3) + biases3)", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 1091, "content": "hidden3 = activation(tf.matmul(hidden2, weights3) + biases3)\noutputs = tf.matmul(hidden3, weights4) + biases4\nreconstruction_loss = tf.reduce_mean(tf.square(outputs - X))\nreg_loss = regularizer(weights1) + regularizer(weights2)\nloss = reconstruction_loss + reg_loss\noptimizer = tf.train.AdamOptimizer(learning_rate)\ntraining_op = optimizer.minimize(loss)\ninit = tf.global_variables_initializer()\nThis code is fairly straightforward, but there are a few important things to note:\nStacked Autoencoders \n| \n417\nDownload from finelybook www.finelybook.com", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 1092, "content": "\u2022 First, weight3 and weights4 are not variables, they are respectively the transpose\nof weights2 and weights1 (they are \u201ctied\u201d to them).\n\u2022 Second, since they are not variables, it\u2019s no use regularizing them: we only regula\u2010\nrize weights1 and weights2.\n\u2022 Third, biases are never tied, and never regularized.\nTraining One Autoencoder at a Time\nRather than training the whole stacked autoencoder in one go like we just did, it is\noften much faster to train one shallow autoencoder at a time, then stack all of them\ninto a single stacked autoencoder (hence the name), as shown on Figure 15-4. This is\nespecially useful for very deep autoencoders.\nFigure 15-4. Training one autoencoder at a time\nDuring the first phase of training, the first autoencoder learns to reconstruct the\ninputs. During the second phase, the second autoencoder learns to reconstruct the\noutput of the first autoencoder\u2019s hidden layer. Finally, you just build a big sandwich", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 1093, "content": "output of the first autoencoder\u2019s hidden layer. Finally, you just build a big sandwich\nusing all these autoencoders, as shown in Figure 15-4 (i.e., you first stack the hidden\nlayers of each autoencoder, then the output layers in reverse order). This gives you\nthe final stacked autoencoder. You could easily train more autoencoders this way,\nbuilding a very deep stacked autoencoder.\nTo implement this multiphase training algorithm, the simplest approach is to use a\ndifferent TensorFlow graph for each phase. After training an autoencoder, you just\nrun the training set through it and capture the output of the hidden layer. This output\nthen serves as the training set for the next autoencoder. Once all autoencoders have\nbeen trained this way, you simply copy the weights and biases from each autoencoder\nand use them to build the stacked autoencoder. Implementing this approach is quite\n418 \n| \nChapter 15: Autoencoders\nDownload from finelybook www.finelybook.com", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 1094, "content": "straightforward, so we won\u2019t detail it here, but please check out the code in the\nJupyter notebooks for an example.\nAnother approach is to use a single graph containing the whole stacked autoencoder,\nplus some extra operations to perform each training phase, as shown in Figure 15-5.\nFigure 15-5. A single graph to train a stacked autoencoder\nThis deserves a bit of explanation:\n\u2022 The central column in the graph is the full stacked autoencoder. This part can be\nused after training.\n\u2022 The left column is the set of operations needed to run the first phase of training.\nIt creates an output layer that bypasses hidden layers 2 and 3. This output layer\nshares the same weights and biases as the stacked autoencoder\u2019s output layer. On\ntop of that are the training operations that will aim at making the output as close\nas possible to the inputs. Thus, this phase will train the weights and biases for the\nhidden layer 1 and the output layer (i.e., the first autoencoder).", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 1095, "content": "hidden layer 1 and the output layer (i.e., the first autoencoder).\n\u2022 The right column in the graph is the set of operations needed to run the second\nphase of training. It adds the training operation that will aim at making the out\u2010\nput of hidden layer 3 as close as possible to the output of hidden layer 1. Note\nthat we must freeze hidden layer 1 while running phase 2. This phase will train\nthe weights and biases for hidden layers 2 and 3 (i.e., the second autoencoder).\nThe TensorFlow code looks like this:\n[...] # Build the whole stacked autoencoder normally.\n # In this example, the weights are not tied.\nStacked Autoencoders \n| \n419\nDownload from finelybook www.finelybook.com", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 1096, "content": "optimizer = tf.train.AdamOptimizer(learning_rate)\nwith tf.name_scope(\"phase1\"):\n phase1_outputs = tf.matmul(hidden1, weights4) + biases4\n phase1_reconstruction_loss = tf.reduce_mean(tf.square(phase1_outputs - X))\n phase1_reg_loss = regularizer(weights1) + regularizer(weights4)\n phase1_loss = phase1_reconstruction_loss + phase1_reg_loss\n phase1_training_op = optimizer.minimize(phase1_loss)\nwith tf.name_scope(\"phase2\"):\n phase2_reconstruction_loss = tf.reduce_mean(tf.square(hidden3 - hidden1))\n phase2_reg_loss = regularizer(weights2) + regularizer(weights3)\n phase2_loss = phase2_reconstruction_loss + phase2_reg_loss\n train_vars = [weights2, biases2, weights3, biases3]\n phase2_training_op = optimizer.minimize(phase2_loss, var_list=train_vars)\nThe first phase is rather straightforward: we just create an output layer that skips hid\u2010\nden layers 2 and 3, then build the training operations to minimize the distance", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 1097, "content": "den layers 2 and 3, then build the training operations to minimize the distance\nbetween the outputs and the inputs (plus some regularization).\nThe second phase just adds the operations needed to minimize the distance between\nthe output of hidden layer 3 and hidden layer 1 (also with some regularization). Most\nimportantly, we provide the list of trainable variables to the minimize() method,\nmaking sure to leave out weights1 and biases1; this effectively freezes hidden layer 1\nduring phase 2.\nDuring the execution phase, all you need to do is run the phase 1 training op for a\nnumber of epochs, then the phase 2 training op for some more epochs.\nSince hidden layer 1 is frozen during phase 2, its output will always\nbe the same for any given training instance. To avoid having to\nrecompute the output of hidden layer 1 at every single epoch, you\ncan compute it for the whole training set at the end of phase 1, then\ndirectly feed the cached output of hidden layer 1 during phase 2.", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 1098, "content": "directly feed the cached output of hidden layer 1 during phase 2.\nThis can give you a nice performance boost.\nVisualizing the Reconstructions\nOne way to ensure that an autoencoder is properly trained is to compare the inputs\nand the outputs. They must be fairly similar, and the differences should be unimpor\u2010\ntant details. Let\u2019s plot two random digits and their reconstructions:\nn_test_digits = 2\nX_test = mnist.test.images[:n_test_digits]\nwith tf.Session() as sess:\n [...] # Train the Autoencoder\n outputs_val = outputs.eval(feed_dict={X: X_test})\n420 \n| \nChapter 15: Autoencoders\nDownload from finelybook www.finelybook.com", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 1099, "content": "def plot_image(image, shape=[28, 28]):\n plt.imshow(image.reshape(shape), cmap=\"Greys\", interpolation=\"nearest\")\n plt.axis(\"off\")\nfor digit_index in range(n_test_digits):\n plt.subplot(n_test_digits, 2, digit_index * 2 + 1)\n plot_image(X_test[digit_index])\n plt.subplot(n_test_digits, 2, digit_index * 2 + 2)\n plot_image(outputs_val[digit_index])\nFigure 15-6 shows the resulting images.\nFigure 15-6. Original digits (left) and their reconstructions (right)\nLooks close enough. So the autoencoder has properly learned to reproduce its inputs,\nbut has it learned useful features? Let\u2019s take a look.\nVisualizing Features\nOnce your autoencoder has learned some features, you may want to take a look at\nthem. There are various techniques for this. Arguably the simplest technique is to\nconsider each neuron in every hidden layer, and find the training instances that acti\u2010\nvate it the most. This is especially useful for the top hidden layers since they often", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 1100, "content": "vate it the most. This is especially useful for the top hidden layers since they often\ncapture relatively large features that you can easily spot in a group of training instan\u2010\nces that contain them. For example, if a neuron strongly activates when it sees a cat in\na picture, it will be pretty obvious that the pictures that activate it the most all contain\ncats. However, for lower layers, this technique does not work so well, as the features\nare smaller and more abstract, so it\u2019s often hard to understand exactly what the neu\u2010\nron is getting all excited about.\nLet\u2019s look at another technique. For each neuron in the first hidden layer, you can cre\u2010\nate an image where a pixel\u2019s intensity corresponds to the weight of the connection to\nthe given neuron. For example, the following code plots the features learned by five\nneurons in the first hidden layer:\nwith tf.Session() as sess:\n [...] # train autoencoder\nStacked Autoencoders \n| \n421\nDownload from finelybook www.finelybook.com", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 1101, "content": "weights1_val = weights1.eval()\nfor i in range(5):\n plt.subplot(1, 5, i + 1)\n plot_image(weights1_val.T[i])\nYou may get low-level features such as the ones shown in Figure 15-7.\nFigure 15-7. Features learned by five neurons from the first hidden layer\nThe first four features seem to correspond to small patches, while the fifth feature\nseems to look for vertical strokes (note that these features come from the stacked\ndenoising autoencoder that we will discuss later).\nAnother technique is to feed the autoencoder a random input image, measure the\nactivation of the neuron you are interested in, and then perform backpropagation to\ntweak the image in such a way that the neuron will activate even more. If you iterate\nseveral times (performing gradient ascent), the image will gradually turn into the\nmost exciting image (for the neuron). This is a useful technique to visualize the kinds\nof inputs that a neuron is looking for.", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 1102, "content": "of inputs that a neuron is looking for.\nFinally, if you are using an autoencoder to perform unsupervised pretraining\u2014for\nexample, for a classification task\u2014a simple way to verify that the features learned by\nthe autoencoder are useful is to measure the performance of the classifier.\nUnsupervised Pretraining Using Stacked Autoencoders\nAs we discussed in Chapter 11, if you are tackling a complex supervised task but you\ndo not have a lot of labeled training data, one solution is to find a neural network that\nperforms a similar task, and then reuse its lower layers. This makes it possible to train\na high-performance model using only little training data because your neural net\u2010\nwork won\u2019t have to learn all the low-level features; it will just reuse the feature detec\u2010\ntors learned by the existing net.\nSimilarly, if you have a large dataset but most of it is unlabeled, you can first train a\nstacked autoencoder using all the data, then reuse the lower layers to create a neural", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 1103, "content": "stacked autoencoder using all the data, then reuse the lower layers to create a neural\nnetwork for your actual task, and train it using the labeled data. For example,\nFigure 15-8 shows how to use a stacked autoencoder to perform unsupervised pre\u2010\ntraining for a classification neural network. The stacked autoencoder itself is typically\ntrained one autoencoder at a time, as discussed earlier. When training the classifier, if\n422 \n| \nChapter 15: Autoencoders\nDownload from finelybook www.finelybook.com", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 1104, "content": "2 \u201cGreedy Layer-Wise Training of Deep Networks,\u201d Y. Bengio et al. (2007).\nyou really don\u2019t have much labeled training data, you may want to freeze the pre\u2010\ntrained layers (at least the lower ones).\nFigure 15-8. Unsupervised pretraining using autoencoders\nThis situation is actually quite common, because building a large\nunlabeled dataset is often cheap (e.g., a simple script can download\nmillions of images off the internet), but labeling them can only be\ndone reliably by humans (e.g., classifying images as cute or not).\nLabeling instances is time-consuming and costly, so it is quite com\u2010\nmon to have only a few thousand labeled instances.\nAs we discussed earlier, one of the triggers of the current Deep Learning tsunami is\nthe discovery in 2006 by Geoffrey Hinton et al. that deep neural networks can be pre\u2010\ntrained in an unsupervised fashion. They used restricted Boltzmann machines for\nthat (see Appendix E), but in 2007 Yoshua Bengio et al. showed2 that autoencoders\nworked just as well.", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 1105, "content": "worked just as well.\nThere is nothing special about the TensorFlow implementation: just train an autoen\u2010\ncoder using all the training data, then reuse its encoder layers to create a new neural\nnetwork (see Chapter 11 for more details on how to reuse pretrained layers, or check\nout the code examples in the Jupyter notebooks).\nUp to now, in order to force the autoencoder to learn interesting features, we have\nlimited the size of the coding layer, making it undercomplete. There are actually\nmany other kinds of constraints that can be used, including ones that allow the cod\u2010\nUnsupervised Pretraining Using Stacked Autoencoders \n| \n423\nDownload from finelybook www.finelybook.com", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 1106, "content": "3 \u201cExtracting and Composing Robust Features with Denoising Autoencoders,\u201d P. Vincent et al. (2008).\n4 \u201cStacked Denoising Autoencoders: Learning Useful Representations in a Deep Network with a Local Denois\u2010\ning Criterion,\u201d P. Vincent et al. (2010).\ning layer to be just as large as the inputs, or even larger, resulting in an overcomplete\nautoencoder. Let\u2019s look at some of those approaches now.\nDenoising Autoencoders\nAnother way to force the autoencoder to learn useful features is to add noise to its\ninputs, training it to recover the original, noise-free inputs. This prevents the autoen\u2010\ncoder from trivially copying its inputs to its outputs, so it ends up having to find pat\u2010\nterns in the data.\nThe idea of using autoencoders to remove noise has been around since the 1980s\n(e.g., it is mentioned in Yann LeCun\u2019s 1987 master\u2019s thesis). In a 2008 paper,3 Pascal\nVincent et al. showed that autoencoders could also be used for feature extraction. In a", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 1107, "content": "Vincent et al. showed that autoencoders could also be used for feature extraction. In a\n2010 paper,4 Vincent et al. introduced stacked denoising autoencoders.\nThe noise can be pure Gaussian noise added to the inputs, or it can be randomly\nswitched off inputs, just like in dropout (introduced in Chapter 11). Figure 15-9\nshows both options.\nFigure 15-9. Denoising autoencoders, with Gaussian noise (left) or dropout (right)\n424 \n| \nChapter 15: Autoencoders\nDownload from finelybook www.finelybook.com", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 1108, "content": "TensorFlow Implementation\nImplementing denoising autoencoders in TensorFlow is not too hard. Let\u2019s start with\nGaussian noise. It\u2019s really just like training a regular autoencoder, except you add\nnoise to the inputs, and the reconstruction loss is calculated based on the original\ninputs:\nX = tf.placeholder(tf.float32, shape=[None, n_inputs])\nX_noisy = X + tf.random_normal(tf.shape(X))\n[...]\nhidden1 = activation(tf.matmul(X_noisy, weights1) + biases1)\n[...]\nreconstruction_loss = tf.reduce_mean(tf.square(outputs - X)) # MSE\n[...]\nSince the shape of X is only partially defined during the construc\u2010\ntion phase, we cannot know in advance the shape of the noise that\nwe must add to X. We cannot call X.get_shape() because this\nwould just return the partially defined shape of X ([None,\nn_inputs]), and random_normal() expects a fully defined shape so\nit would raise an exception. Instead, we call tf.shape(X), which\ncreates an operation that will return the shape of X at runtime,", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 1109, "content": "creates an operation that will return the shape of X at runtime,\nwhich will be fully defined at that point.\nImplementing the dropout version, which is more common, is not much harder:\nfrom tensorflow.contrib.layers import dropout\nkeep_prob = 0.7\nis_training = tf.placeholder_with_default(False, shape=(), name='is_training')\nX = tf.placeholder(tf.float32, shape=[None, n_inputs])\nX_drop = dropout(X, keep_prob, is_training=is_training)\n[...]\nhidden1 = activation(tf.matmul(X_drop, weights1) + biases1)\n[...]\nreconstruction_loss = tf.reduce_mean(tf.square(outputs - X)) # MSE\n[...]\nDuring training we must set is_training to True (as explained in Chapter 11) using\nthe feed_dict:\nsess.run(training_op, feed_dict={X: X_batch, is_training: True})\nHowever, during testing it is not necessary to set is_training to False, since we set\nthat as the default in the call to the placeholder_with_default() function.\nDenoising Autoencoders \n| \n425\nDownload from finelybook www.finelybook.com", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 1110, "content": "Sparse Autoencoders\nAnother kind of constraint that often leads to good feature extraction is sparsity: by\nadding an appropriate term to the cost function, the autoencoder is pushed to reduce\nthe number of active neurons in the coding layer. For example, it may be pushed to\nhave on average only 5% significantly active neurons in the coding layer. This forces\nthe autoencoder to represent each input as a combination of a small number of acti\u2010\nvations. As a result, each neuron in the coding layer typically ends up representing a\nuseful feature (if you could speak only a few words per month, you would probably\ntry to make them worth listening to).\nIn order to favor sparse models, we must first measure the actual sparsity of the cod\u2010\ning layer at each training iteration. We do so by computing the average activation of\neach neuron in the coding layer, over the whole training batch. The batch size must\nnot be too small, or else the mean will not be accurate.", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 1111, "content": "not be too small, or else the mean will not be accurate.\nOnce we have the mean activation per neuron, we want to penalize the neurons that\nare too active by adding a sparsity loss to the cost function. For example, if we meas\u2010\nure that a neuron has an average activation of 0.3, but the target sparsity is 0.1, it must\nbe penalized to activate less. One approach could be simply adding the squared error\n(0.3 \u2013 0.1)2 to the cost function, but in practice a better approach is to use the Kull\u2010\nback\u2013Leibler divergence (briefly discussed in Chapter 4), which has much stronger\ngradients than the Mean Squared Error, as you can see in Figure 15-10.\nFigure 15-10. Sparsity loss\n426 \n| \nChapter 15: Autoencoders\nDownload from finelybook www.finelybook.com", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 1112, "content": "Given two discrete probability distributions P and Q, the KL divergence between\nthese distributions, noted DKL(P \u2225 Q), can be computed using Equation 15-1.\nEquation 15-1. Kullback\u2013Leibler divergence\nDKL P \u2225Q = \u2211\ni P i log P i\nQ i\nIn our case, we want to measure the divergence between the target probability p that a\nneuron in the coding layer will activate, and the actual probability q (i.e., the mean\nactivation over the training batch). So the KL divergence simplifies to Equation 15-2.\nEquation 15-2. KL divergence between the target sparsity p and the actual sparsity q\nDKL p \u2225q = p log p\nq + 1 \u2212p log 1 \u2212p\n1 \u2212q\nOnce we have computed the sparsity loss for each neuron in the coding layer, we just\nsum up these losses, and add the result to the cost function. In order to control the\nrelative importance of the sparsity loss and the reconstruction loss, we can multiply\nthe sparsity loss by a sparsity weight hyperparameter. If this weight is too high, the", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 1113, "content": "the sparsity loss by a sparsity weight hyperparameter. If this weight is too high, the\nmodel will stick closely to the target sparsity, but it may not reconstruct the inputs\nproperly, making the model useless. Conversely, if it is too low, the model will mostly\nignore the sparsity objective and it will not learn any interesting features.\nTensorFlow Implementation\nWe now have all we need to implement a sparse autoencoder using TensorFlow:\ndef kl_divergence(p, q):\n return p * tf.log(p / q) + (1 - p) * tf.log((1 - p) / (1 - q))\nlearning_rate = 0.01\nsparsity_target = 0.1\nsparsity_weight = 0.2\n[...] # Build a normal autoencoder (in this example the coding layer is hidden1)\noptimizer = tf.train.AdamOptimizer(learning_rate)\nhidden1_mean = tf.reduce_mean(hidden1, axis=0) # batch mean\nsparsity_loss = tf.reduce_sum(kl_divergence(sparsity_target, hidden1_mean))\nreconstruction_loss = tf.reduce_mean(tf.square(outputs - X)) # MSE\nloss = reconstruction_loss + sparsity_weight * sparsity_loss", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 1114, "content": "loss = reconstruction_loss + sparsity_weight * sparsity_loss\ntraining_op = optimizer.minimize(loss)\nAn important detail is the fact that the activations of the coding layer must be\nbetween 0 and 1 (but not equal to 0 or 1), or else the KL divergence will return NaN\nSparse Autoencoders \n| \n427\nDownload from finelybook www.finelybook.com", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 1115, "content": "5 \u201cAuto-Encoding Variational Bayes,\u201d D. Kingma and M. Welling (2014).\n(Not a Number). A simple solution is to use the logistic activation function for the\ncoding layer:\nhidden1 = tf.nn.sigmoid(tf.matmul(X, weights1) + biases1)\nOne simple trick can speed up convergence: instead of using the MSE, we can choose\na reconstruction loss that will have larger gradients. Cross entropy is often a good\nchoice. To use it, we must normalize the inputs to make them take on values from 0\nto 1, and use the logistic activation function in the output layer so the outputs also\ntake on values from 0 to 1. TensorFlow\u2019s sigmoid_cross_entropy_with_logits()\nfunction takes care of efficiently applying the logistic (sigmoid) activation function to\nthe outputs and computing the cross entropy:\n[...]\nlogits = tf.matmul(hidden1, weights2) + biases2)\noutputs = tf.nn.sigmoid(logits)\nreconstruction_loss = tf.reduce_sum(\n tf.nn.sigmoid_cross_entropy_with_logits(labels=X, logits=logits))", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 1116, "content": "tf.nn.sigmoid_cross_entropy_with_logits(labels=X, logits=logits))\nNote that the outputs operation is not needed during training (we use it only when\nwe want to look at the reconstructions).\nVariational Autoencoders\nAnother important category of autoencoders was introduced in 2014 by Diederik\nKingma and Max Welling,5 and has quickly become one of the most popular types of\nautoencoders: variational autoencoders.\nThey are quite different from all the autoencoders we have discussed so far, in partic\u2010\nular:\n\u2022 They are probabilistic autoencoders, meaning that their outputs are partly deter\u2010\nmined by chance, even after training (as opposed to denoising autoencoders,\nwhich use randomness only during training).\n\u2022 Most importantly, they are generative autoencoders, meaning that they can gener\u2010\nate new instances that look like they were sampled from the training set.\nBoth these properties make them rather similar to RBMs (see Appendix E), but they", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 1117, "content": "Both these properties make them rather similar to RBMs (see Appendix E), but they\nare easier to train and the sampling process is much faster (with RBMs you need to\nwait for the network to stabilize into a \u201cthermal equilibrium\u201d before you can sample a\nnew instance).\n428 \n| \nChapter 15: Autoencoders\nDownload from finelybook www.finelybook.com", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 1118, "content": "6 Variational autoencoders are actually more general; the codings are not limited to Gaussian distributions.\nLet\u2019s take a look at how they work. Figure 15-11 (left) shows a variational autoen\u2010\ncoder. You can recognize, of course, the basic structure of all autoencoders, with an\nencoder followed by a decoder (in this example, they both have two hidden layers),\nbut there is a twist: instead of directly producing a coding for a given input, the\nencoder produces a mean coding \u03bc and a standard deviation \u03c3. The actual coding is\nthen sampled randomly from a Gaussian distribution with mean \u03bc and standard devi\u2010\nation \u03c3. After that the decoder just decodes the sampled coding normally. The right\npart of the diagram shows a training instance going through this autoencoder. First,\nthe encoder produces \u03bc and \u03c3, then a coding is sampled randomly (notice that it is\nnot exactly located at \u03bc), and finally this coding is decoded, and the final output\nresembles the training instance.", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 1119, "content": "resembles the training instance.\nFigure 15-11. Variational autoencoder (left), and an instance going through it (right)\nAs you can see on the diagram, although the inputs may have a very convoluted dis\u2010\ntribution, a variational autoencoder tends to produce codings that look as though\nthey were sampled from a simple Gaussian distribution:6 during training, the cost\nfunction (discussed next) pushes the codings to gradually migrate within the coding\nspace (also called the latent space) to occupy a roughly (hyper)spherical region that\nlooks like a cloud of Gaussian points. One great consequence is that after training a\nVariational Autoencoders \n| \n429\nDownload from finelybook www.finelybook.com", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 1120, "content": "7 For more mathematical details, check out the original paper on variational autoencoders, or Carl Doersch\u2019s\ngreat tutorial (2016).\nvariational autoencoder, you can very easily generate a new instance: just sample a\nrandom coding from the Gaussian distribution, decode it, and voil\u00e0!\nSo let\u2019s look at the cost function. It is composed of two parts. The first is the usual \nreconstruction loss that pushes the autoencoder to reproduce its inputs (we can use\ncross entropy for this, as discussed earlier). The second is the latent loss that pushes\nthe autoencoder to have codings that look as though they were sampled from a simple\nGaussian distribution, for which we use the KL divergence between the target distri\u2010\nbution (the Gaussian distribution) and the actual distribution of the codings. The\nmath is a bit more complex than earlier, in particular because of the Gaussian noise,\nwhich limits the amount of information that can be transmitted to the coding layer", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 1121, "content": "which limits the amount of information that can be transmitted to the coding layer\n(thus pushing the autoencoder to learn useful features). Luckily, the equations sim\u2010\nplify to the following code for the latent loss:7\neps = 1e-10 # smoothing term to avoid computing log(0) which is NaN\nlatent_loss = 0.5 * tf.reduce_sum(\n tf.square(hidden3_sigma) + tf.square(hidden3_mean)\n - 1 - tf.log(eps + tf.square(hidden3_sigma)))\nOne common variant is to train the encoder to output \u03b3 = log(\u03c32) rather than \u03c3.\nWherever we need \u03c3 we can just compute \u03c3 = exp\n\u03b3\n2 . This makes it a bit easier for\nthe encoder to capture sigmas of different scales, and thus it helps speed up conver\u2010\ngence. The latent loss ends up a bit simpler:\nlatent_loss = 0.5 * tf.reduce_sum(\n tf.exp(hidden3_gamma) + tf.square(hidden3_mean) - 1 - hidden3_gamma)\nThe following code builds the variational autoencoder shown in Figure 15-11 (left), \nusing the log(\u03c32) variant:\nn_inputs = 28 * 28 # for MNIST\nn_hidden1 = 500", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 1122, "content": "using the log(\u03c32) variant:\nn_inputs = 28 * 28 # for MNIST\nn_hidden1 = 500\nn_hidden2 = 500\nn_hidden3 = 20 # codings\nn_hidden4 = n_hidden2\nn_hidden5 = n_hidden1\nn_outputs = n_inputs\nlearning_rate = 0.001\nwith tf.contrib.framework.arg_scope(\n [fully_connected],\n activation_fn=tf.nn.elu,\n weights_initializer=tf.contrib.layers.variance_scaling_initializer()):\n X = tf.placeholder(tf.float32, [None, n_inputs])\n430 \n| \nChapter 15: Autoencoders\nDownload from finelybook www.finelybook.com", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 1123, "content": "hidden1 = fully_connected(X, n_hidden1)\n hidden2 = fully_connected(hidden1, n_hidden2)\n hidden3_mean = fully_connected(hidden2, n_hidden3, activation_fn=None)\n hidden3_gamma = fully_connected(hidden2, n_hidden3, activation_fn=None)\n hidden3_sigma = tf.exp(0.5 * hidden3_gamma)\n noise = tf.random_normal(tf.shape(hidden3_sigma), dtype=tf.float32)\n hidden3 = hidden3_mean + hidden3_sigma * noise\n hidden4 = fully_connected(hidden3, n_hidden4)\n hidden5 = fully_connected(hidden4, n_hidden5)\n logits = fully_connected(hidden5, n_outputs, activation_fn=None)\n outputs = tf.sigmoid(logits)\nreconstruction_loss = tf.reduce_sum(\n tf.nn.sigmoid_cross_entropy_with_logits(labels=X, logits=logits))\nlatent_loss = 0.5 * tf.reduce_sum(\n tf.exp(hidden3_gamma) + tf.square(hidden3_mean) - 1 - hidden3_gamma)\ncost = reconstruction_loss + latent_loss\noptimizer = tf.train.AdamOptimizer(learning_rate=learning_rate)\ntraining_op = optimizer.minimize(cost)", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 1124, "content": "training_op = optimizer.minimize(cost)\ninit = tf.global_variables_initializer()\nGenerating Digits\nNow let\u2019s use this variational autoencoder to generate images that look like handwrit\u2010\nten digits. All we need to do is train the model, then sample random codings from a \nGaussian distribution and decode them.\nimport numpy as np\nn_digits = 60\nn_epochs = 50\nbatch_size = 150\nwith tf.Session() as sess:\n init.run()\n for epoch in range(n_epochs):\n n_batches = mnist.train.num_examples // batch_size\n for iteration in range(n_batches):\n X_batch, y_batch = mnist.train.next_batch(batch_size)\n sess.run(training_op, feed_dict={X: X_batch})\n codings_rnd = np.random.normal(size=[n_digits, n_hidden3])\n outputs_val = outputs.eval(feed_dict={hidden3: codings_rnd})\nThat\u2019s it. Now we can see what the \u201chandwritten\u201d digits produced by the autoencoder\nlook like (see Figure 15-12):\nfor iteration in range(n_digits):\n plt.subplot(n_digits, 10, iteration + 1)", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 1125, "content": "for iteration in range(n_digits):\n plt.subplot(n_digits, 10, iteration + 1)\n plot_image(outputs_val[iteration])\nVariational Autoencoders \n| \n431\nDownload from finelybook www.finelybook.com", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 1126, "content": "8 \u201cContractive Auto-Encoders: Explicit Invariance During Feature Extraction,\u201d S. Rifai et al. (2011).\nFigure 15-12. Images of handwritten digits generated by the variational autoencoder\nA majority of these digits look pretty convincing, while a few are rather \u201ccreative.\u201d But\ndon\u2019t be too harsh on the autoencoder\u2014it only started learning less than an hour ago.\nGive it a bit more training time, and those digits will look better and better.\nOther Autoencoders\nThe amazing successes of supervised learning in image recognition, speech recogni\u2010\ntion, text translation, and more have somewhat overshadowed unsupervised learning,\nbut it is actually booming. New architectures for autoencoders and other unsuper\u2010\nvised learning algorithms are invented regularly, so much so that we cannot cover\nthem all in this book. Here is a brief (by no means exhaustive) overview of a few more\ntypes of autoencoders that you may want to check out:\nContractive autoencoder (CAE)8", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 1127, "content": "types of autoencoders that you may want to check out:\nContractive autoencoder (CAE)8\nThe autoencoder is constrained during training so that the derivatives of the cod\u2010\nings with regards to the inputs are small. In other words, two similar inputs must\nhave similar codings.\n432 \n| \nChapter 15: Autoencoders\nDownload from finelybook www.finelybook.com", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 1128, "content": "9 \u201cStacked Convolutional Auto-Encoders for Hierarchical Feature Extraction,\u201d J. Masci et al. (2011).\n10 \u201cGSNs: Generative Stochastic Networks,\u201d G. Alain et al. (2015).\n11 \u201cWinner-Take-All Autoencoders,\u201d A. Makhzani and B. Frey (2015).\n12 \u201cAdversarial Autoencoders,\u201d A. Makhzani et al. (2016).\nStacked convolutional autoencoders9\nAutoencoders that learn to extract visual features by reconstructing images pro\u2010\ncessed through convolutional layers.\nGenerative stochastic network (GSN)10\nA generalization of denoising autoencoders, with the added capability to generate\ndata.\nWinner-take-all (WTA) autoencoder11\nDuring training, after computing the activations of all the neurons in the coding\nlayer, only the top k% activations for each neuron over the training batch are pre\u2010\nserved, and the rest are set to zero. Naturally this leads to sparse codings. More\u2010\nover, a similar WTA approach can be used to produce sparse convolutional\nautoencoders.\nAdversarial autoencoders12", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 1129, "content": "autoencoders.\nAdversarial autoencoders12\nOne network is trained to reproduce its inputs, and at the same time another is\ntrained to find inputs that the first network is unable to properly reconstruct.\nThis pushes the first autoencoder to learn robust codings.\nExercises\n1. What are the main tasks that autoencoders are used for?\n2. Suppose you want to train a classifier and you have plenty of unlabeled training\ndata, but only a few thousand labeled instances. How can autoencoders help?\nHow would you proceed?\n3. If an autoencoder perfectly reconstructs the inputs, is it necessarily a good\nautoencoder? How can you evaluate the performance of an autoencoder?\n4. What are undercomplete and overcomplete autoencoders? What is the main risk\nof an excessively undercomplete autoencoder? What about the main risk of an\novercomplete autoencoder?\n5. How do you tie weights in a stacked autoencoder? What is the point of doing so?", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 1130, "content": "5. How do you tie weights in a stacked autoencoder? What is the point of doing so?\n6. What is a common technique to visualize features learned by the lower layer of a\nstacked autoencoder? What about higher layers?\n7. What is a generative model? Can you name a type of generative autoencoder?\nExercises \n| \n433\nDownload from finelybook www.finelybook.com", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 1131, "content": "13 \u201cSemantic Hashing,\u201d R. Salakhutdinov and G. Hinton (2008).\n8. Let\u2019s use a denoising autoencoder to pretrain an image classifier:\n\u2022 You can use MNIST (simplest), or another large set of images such as CIFAR10\nif you want a bigger challenge. If you choose CIFAR10, you need to write code\nto load batches of images for training. If you want to skip this part, Tensor\u2010\nFlow\u2019s model zoo contains tools to do just that.\n\u2022 Split the dataset into a training set and a test set. Train a deep denoising\nautoencoder on the full training set.\n\u2022 Check that the images are fairly well reconstructed, and visualize the low-level\nfeatures. Visualize the images that most activate each neuron in the coding\nlayer.\n\u2022 Build a classification deep neural network, reusing the lower layers of the\nautoencoder. Train it using only 10% of the training set. Can you get it to per\u2010\nform as well as the same classifier trained on the full training set?", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 1132, "content": "form as well as the same classifier trained on the full training set?\n9. Semantic hashing, introduced in 2008 by Ruslan Salakhutdinov and Geoffrey\nHinton,13 is a technique used for efficient information retrieval: a document (e.g.,\nan image) is passed through a system, typically a neural network, which outputs a\nfairly low-dimensional binary vector (e.g., 30 bits). Two similar documents are\nlikely to have identical or very similar hashes. By indexing each document using\nits hash, it is possible to retrieve many documents similar to a particular docu\u2010\nment almost instantly, even if there are billions of documents: just compute the\nhash of the document and look up all documents with that same hash (or hashes\ndiffering by just one or two bits). Let\u2019s implement semantic hashing using a\nslightly tweaked stacked autoencoder:\n\u2022 Create a stacked autoencoder containing two hidden layers below the coding\nlayer, and train it on the image dataset you used in the previous exercise. The", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 1133, "content": "layer, and train it on the image dataset you used in the previous exercise. The\ncoding layer should contain 30 neurons and use the logistic activation function\nto output values between 0 and 1. After training, to produce the hash of an\nimage, you can simply run it through the autoencoder, take the output of the\ncoding layer, and round every value to the closest integer (0 or 1).\n\u2022 One neat trick proposed by Salakhutdinov and Hinton is to add Gaussian\nnoise (with zero mean) to the inputs of the coding layer, during training only.\nIn order to preserve a high signal-to-noise ratio, the autoencoder will learn to\nfeed large values to the coding layer (so that the noise becomes negligible). In\nturn, this means that the logistic function of the coding layer will likely satu\u2010\nrate at 0 or 1. As a result, rounding the codings to 0 or 1 won\u2019t distort them too\nmuch, and this will improve the reliability of the hashes.\n434 \n| \nChapter 15: Autoencoders\nDownload from finelybook www.finelybook.com", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 1134, "content": "14 \u201cCNN Based Hashing for Image Retrieval,\u201d J. Gua and J. Li (2015).\n\u2022 Compute the hash of every image, and see if images with identical hashes look\nalike. Since MNIST and CIFAR10 are labeled, a more objective way to measure\nthe performance of the autoencoder for semantic hashing is to ensure that\nimages with the same hash generally have the same class. One way to do this is\nto measure the average Gini purity (introduced in Chapter 6) of the sets of\nimages with identical (or very similar) hashes.\n\u2022 Try fine-tuning the hyperparameters using cross-validation.\n\u2022 Note that with a labeled dataset, another approach is to train a convolutional\nneural network (see Chapter 13) for classification, then use the layer below the\noutput layer to produce the hashes. See Jinma Gua and Jianmin Li\u2019s 2015\npaper.14 See if that performs better.\n10. Train a variational autoencoder on the image dataset used in the previous exerci\u2010", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 1135, "content": "10. Train a variational autoencoder on the image dataset used in the previous exerci\u2010\nses (MNIST or CIFAR10), and make it generate images. Alternatively, you can try\nto find an unlabeled dataset that you are interested in and see if you can generate\nnew samples.\nSolutions to these exercises are available in Appendix A.\nExercises \n| \n435\nDownload from finelybook www.finelybook.com", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 1136, "content": "Download from finelybook www.finelybook.com", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 1137, "content": "1 For more details, be sure to check out Richard Sutton and Andrew Barto\u2019s book on RL, Reinforcement Learn\u2010\ning: An Introduction (MIT Press), or David Silver\u2019s free online RL course at University College London.\n2 \u201cPlaying Atari with Deep Reinforcement Learning,\u201d V. Mnih et al. (2013).\n3 \u201cHuman-level control through deep reinforcement learning,\u201d V. Mnih et al. (2015).\n4 Check out the videos of DeepMind\u2019s system learning to play Space Invaders, Breakout, and more at https://\ngoo.gl/yTsH6X.\nCHAPTER 16\nReinforcement Learning\nReinforcement Learning (RL) is one of the most exciting fields of Machine Learning\ntoday, and also one of the oldest. It has been around since the 1950s, producing many\ninteresting applications over the years,1 in particular in games (e.g., TD-Gammon, a\nBackgammon playing program) and in machine control, but seldom making the\nheadline news. But a revolution took place in 2013 when researchers from an English", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 1138, "content": "headline news. But a revolution took place in 2013 when researchers from an English\nstartup called DeepMind demonstrated a system that could learn to play just about\nany Atari game from scratch,2 eventually outperforming humans3 in most of them,\nusing only raw pixels as inputs and without any prior knowledge of the rules of the\ngames.4 This was the first of a series of amazing feats, culminating in March 2016\nwith the victory of their system AlphaGo against Lee Sedol, the world champion of\nthe game of Go. No program had ever come close to beating a master of this game, let\nalone the world champion. Today the whole field of RL is boiling with new ideas, with\na wide range of applications. DeepMind was bought by Google for over 500 million\ndollars in 2014.\nSo how did they do it? With hindsight it seems rather simple: they applied the power\nof Deep Learning to the field of Reinforcement Learning, and it worked beyond their", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 1139, "content": "of Deep Learning to the field of Reinforcement Learning, and it worked beyond their\nwildest dreams. In this chapter we will first explain what Reinforcement Learning is\nand what it is good at, and then we will present two of the most important techniques\nin deep Reinforcement Learning: policy gradients and deep Q-networks (DQN),\n437\nDownload from finelybook www.finelybook.com", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 1140, "content": "including a discussion of Markov decision processes (MDP). We will use these techni\u2010\nques to train a model to balance a pole on a moving cart, and another to play Atari\ngames. The same techniques can be used for a wide variety of tasks, from walking\nrobots to self-driving cars.\nLearning to Optimize Rewards\nIn Reinforcement Learning, a software agent makes observations and takes actions\nwithin an environment, and in return it receives rewards. Its objective is to learn to act\nin a way that will maximize its expected long-term rewards. If you don\u2019t mind a bit of\nanthropomorphism, you can think of positive rewards as pleasure, and negative\nrewards as pain (the term \u201creward\u201d is a bit misleading in this case). In short, the agent\nacts in the environment and learns by trial and error to maximize its pleasure and\nminimize its pain.\nThis is quite a broad setting, which can apply to a wide variety of tasks. Here are a few\nexamples (see Figure 16-1):", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 1141, "content": "examples (see Figure 16-1):\na. The agent can be the program controlling a walking robot. In this case, the envi\u2010\nronment is the real world, the agent observes the environment through a set of\nsensors such as cameras and touch sensors, and its actions consist of sending sig\u2010\nnals to activate motors. It may be programmed to get positive rewards whenever\nit approaches the target destination, and negative rewards whenever it wastes\ntime, goes in the wrong direction, or falls down.\nb. The agent can be the program controlling Ms. Pac-Man. In this case, the environ\u2010\nment is a simulation of the Atari game, the actions are the nine possible joystick\npositions (upper left, down, center, and so on), the observations are screenshots,\nand the rewards are just the game points.\nc. Similarly, the agent can be the program playing a board game such as the game of\nGo.\nd. The agent does not have to control a physically (or virtually) moving thing. For", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 1142, "content": "Go.\nd. The agent does not have to control a physically (or virtually) moving thing. For\nexample, it can be a smart thermostat, getting rewards whenever it is close to the\ntarget temperature and saves energy, and negative rewards when humans need to\ntweak the temperature, so the agent must learn to anticipate human needs.\ne. The agent can observe stock market prices and decide how much to buy or sell\nevery second. Rewards are obviously the monetary gains and losses.\n438 \n| \nChapter 16: Reinforcement Learning\nDownload from finelybook www.finelybook.com", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 1143, "content": "5 Images (a), (c), and (d) are reproduced from Wikipedia. (a) and (d) are in the public domain. (c) was created\nby user Stevertigo and released under Creative Commons BY-SA 2.0. (b) is a screenshot from the Ms. Pac-\nMan game, copyright Atari (the author believes it to be fair use in this chapter). (e) was reproduced from Pix\u2010\nabay, released under Creative Commons CC0.\nFigure 16-1. Reinforcement Learning examples: (a) walking robot, (b) Ms. Pac-Man, (c)\nGo player, (d) thermostat, (e) automatic trader5\nNote that there may not be any positive rewards at all; for example, the agent may\nmove around in a maze, getting a negative reward at every time step, so it better find\nthe exit as quickly as possible! There are many other examples of tasks where Rein\u2010\nforcement Learning is well suited, such as self-driving cars, placing ads on a web\npage, or controlling where an image classification system should focus its attention.\nLearning to Optimize Rewards \n| \n439", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 1144, "content": "Learning to Optimize Rewards \n| \n439\nDownload from finelybook www.finelybook.com", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 1145, "content": "6 It is often better to give the poor performers a slight chance of survival, to preserve some diversity in the \u201cgene\npool.\u201d\nPolicy Search\nThe algorithm used by the software agent to determine its actions is called its policy. \nFor example, the policy could be a neural network taking observations as inputs and\noutputting the action to take (see Figure 16-2).\nFigure 16-2. Reinforcement Learning using a neural network policy\nThe policy can be any algorithm you can think of, and it does not even have to be\ndeterministic. For example, consider a robotic vacuum cleaner whose reward is the\namount of dust it picks up in 30 minutes. Its policy could be to move forward with\nsome probability p every second, or randomly rotate left or right with probability 1 \u2013\np. The rotation angle would be a random angle between \u2013r and +r. Since this policy\ninvolves some randomness, it is called a stochastic policy. The robot will have an", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 1146, "content": "involves some randomness, it is called a stochastic policy. The robot will have an\nerratic trajectory, which guarantees that it will eventually get to any place it can reach\nand pick up all the dust. The question is: how much dust will it pick up in 30\nminutes?\nHow would you train such a robot? There are just two policy parameters you can\ntweak: the probability p and the angle range r. One possible learning algorithm could\nbe to try out many different values for these parameters, and pick the combination\nthat performs best (see Figure 16-3). This is an example of policy search, in this case\nusing a brute force approach. However, when the policy space is too large (which is\ngenerally the case), finding a good set of parameters this way is like searching for a\nneedle in a gigantic haystack.\nAnother way to explore the policy space is to use genetic algorithms. For example, you\ncould randomly create a first generation of 100 policies and try them out, then \u201ckill\u201d", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 1147, "content": "could randomly create a first generation of 100 policies and try them out, then \u201ckill\u201d\nthe 80 worst policies6 and make the 20 survivors produce 4 offspring each. An off\u2010\n440 \n| \nChapter 16: Reinforcement Learning\nDownload from finelybook www.finelybook.com", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 1148, "content": "7 If there is a single parent, this is called asexual reproduction. With two (or more) parents, it is called sexual\nreproduction. An offspring\u2019s genome (in this case a set of policy parameters) is randomly composed of parts of\nits parents\u2019 genomes.\nspring is just a copy of its parent7 plus some random variation. The surviving policies\nplus their offspring together constitute the second generation. You can continue to\niterate through generations this way, until you find a good policy.\nFigure 16-3. Four points in policy space and the agent\u2019s corresponding behavior\nYet another approach is to use optimization techniques, by evaluating the gradients of\nthe rewards with regards to the policy parameters, then tweaking these parameters by\nfollowing the gradient toward higher rewards (gradient ascent). This approach is\ncalled policy gradients (PG), which we will discuss in more detail later in this chapter.\nFor example, going back to the vacuum cleaner robot, you could slightly increase p", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 1149, "content": "For example, going back to the vacuum cleaner robot, you could slightly increase p\nand evaluate whether this increases the amount of dust picked up by the robot in 30\nminutes; if it does, then increase p some more, or else reduce p. We will implement a\npopular PG algorithm using TensorFlow, but before we do we need to create an envi\u2010\nronment for the agent to live in, so it\u2019s time to introduce OpenAI gym.\nIntroduction to OpenAI Gym\nOne of the challenges of Reinforcement Learning is that in order to train an agent,\nyou first need to have a working environment. If you want to program an agent that\nwill learn to play an Atari game, you will need an Atari game simulator. If you want to\nprogram a walking robot, then the environment is the real world and you can directly\ntrain your robot in that environment, but this has its limits: if the robot falls off a cliff,\nyou can\u2019t just click \u201cundo.\u201d You can\u2019t speed up time either; adding more computing\nIntroduction to OpenAI Gym \n| \n441", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 1150, "content": "Introduction to OpenAI Gym \n| \n441\nDownload from finelybook www.finelybook.com", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 1151, "content": "8 OpenAI is a nonprofit artificial intelligence research company, funded in part by Elon Musk. Its stated goal is\nto promote and develop friendly AIs that will benefit humanity (rather than exterminate it).\npower won\u2019t make the robot move any faster. And it\u2019s generally too expensive to train\n1,000 robots in parallel. In short, training is hard and slow in the real world, so you\ngenerally need a simulated environment at least to bootstrap training.\nOpenAI gym8 is a toolkit that provides a wide variety of simulated environments\n(Atari games, board games, 2D and 3D physical simulations, and so on), so you can\ntrain agents, compare them, or develop new RL algorithms.\nLet\u2019s install OpenAI gym. For a minimal OpenAI gym installation, simply use pip:\n$ pip3 install --upgrade gym\nNext open up a Python shell or a Jupyter notebook and create your first environment:\n>>> import gym\n>>> env = gym.make(\"CartPole-v0\")\n[2016-10-14 16:03:23,199] Making new env: MsPacman-v0\n>>> obs = env.reset()", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 1152, "content": "[2016-10-14 16:03:23,199] Making new env: MsPacman-v0\n>>> obs = env.reset()\n>>> obs\narray([-0.03799846, -0.03288115, 0.02337094, 0.00720711])\n>>> env.render()\nThe make() function creates an environment, in this case a CartPole environment.\nThis is a 2D simulation in which a cart can be accelerated left or right in order to bal\u2010\nance a pole placed on top of it (see Figure 16-4). After the environment is created, we\nmust initialize it using the reset() method. This returns the first observation. Obser\u2010\nvations depend on the type of environment. For the CartPole environment, each\nobservation is a 1D NumPy array containing four floats: these floats represent the\ncart\u2019s horizontal position (0.0 = center), its velocity, the angle of the pole (0.0 = verti\u2010\ncal), and its angular velocity. Finally, the render() method displays the environment\nas shown in Figure 16-4.\nFigure 16-4. The CartPole environment\n442 \n| \nChapter 16: Reinforcement Learning\nDownload from finelybook www.finelybook.com", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 1153, "content": "If you want render() to return the rendered image as a NumPy array, you can set the\nmode parameter to rgb_array (note that other environments may support different\nmodes):\n>>> img = env.render(mode=\"rgb_array\")\n>>> img.shape # height, width, channels (3=RGB)\n(400, 600, 3)\nUnfortunately, the CartPole (and a few other environments) ren\u2010\nders the image to the screen even if you set the mode to\n\"rgb_array\". The only way to avoid this is to use a fake X server\nsuch as Xvfb or Xdummy. For example, you can install Xvfb and\nstart Python using the following command: xvfb-run -s \"-\nscreen 0 1400x900x24\" python. Or use the xvfbwrapper package.\nLet\u2019s ask the environment what actions are possible:\n>>> env.action_space\nDiscrete(2)\nDiscrete(2) means that the possible actions are integers 0 and 1, which represent\naccelerating left (0) or right (1). Other environments may have more discrete actions,\nor other kinds of actions (e.g., continuous). Since the pole is leaning toward the right,", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 1154, "content": "or other kinds of actions (e.g., continuous). Since the pole is leaning toward the right,\nlet\u2019s accelerate the cart toward the right:\n>>> action = 1 # accelerate right\n>>> obs, reward, done, info = env.step(action)\n>>> obs\narray([-0.03865608, 0.16189797, 0.02351508, -0.27801135])\n>>> reward\n1.0\n>>> done\nFalse\n>>> info\n{}\nThe step() method executes the given action and returns four values:\nobs\nThis is the new observation. The cart is now moving toward the right (obs[1]>0).\nThe pole is still tilted toward the right (obs[2]>0), but its angular velocity is now\nnegative (obs[3]<0), so it will likely be tilted toward the left after the next step.\nreward\nIn this environment, you get a reward of 1.0 at every step, no matter what you do,\nso the goal is to keep running as long as possible.\nIntroduction to OpenAI Gym \n| \n443\nDownload from finelybook www.finelybook.com", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 1155, "content": "done\nThis value will be True when the episode is over. This will happen when the pole\ntilts too much. After that, the environment must be reset before it can be used\nagain.\ninfo\nThis dictionary may provide extra debug information in other environments.\nThis data should not be used for training (it would be cheating).\nLet\u2019s hardcode a simple policy that accelerates left when the pole is leaning toward the\nleft and accelerates right when the pole is leaning toward the right. We will run this\npolicy to see the average rewards it gets over 500 episodes:\ndef basic_policy(obs):\n angle = obs[2]\n return 0 if angle < 0 else 1\ntotals = []\nfor episode in range(500):\n episode_rewards = 0\n obs = env.reset()\n for step in range(1000): # 1000 steps max, we don't want to run forever\n action = basic_policy(obs)\n obs, reward, done, info = env.step(action)\n episode_rewards += reward\n if done:\n break\n totals.append(episode_rewards)", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 1156, "content": "if done:\n break\n totals.append(episode_rewards)\nThis code is hopefully self-explanatory. Let\u2019s look at the result:\n>>> import numpy as np\n>>> np.mean(totals), np.std(totals), np.min(totals), np.max(totals)\n(42.125999999999998, 9.1237121830974033, 24.0, 68.0)\nEven with 500 tries, this policy never managed to keep the pole upright for more than\n68 consecutive steps. Not great. If you look at the simulation in the Jupyter note\u2010\nbooks, you will see that the cart oscillates left and right more and more strongly until\nthe pole tilts too much. Let\u2019s see if a neural network can come up with a better policy.\nNeural Network Policies\nLet\u2019s create a neural network policy. Just like the policy we hardcoded earlier, this\nneural network will take an observation as input, and it will output the action to be\nexecuted. More precisely, it will estimate a probability for each action, and then we\nwill select an action randomly according to the estimated probabilities (see", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 1157, "content": "will select an action randomly according to the estimated probabilities (see\nFigure 16-5). In the case of the CartPole environment, there are just two possible\nactions (left or right), so we only need one output neuron. It will output the probabil\u2010\nity p of action 0 (left), and of course the probability of action 1 (right) will be 1 \u2013 p.\n444 \n| \nChapter 16: Reinforcement Learning\nDownload from finelybook www.finelybook.com", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 1158, "content": "For example, if it outputs 0.7, then we will pick action 0 with 70% probability, and\naction 1 with 30% probability.\nFigure 16-5. Neural network policy\nYou may wonder why we are picking a random action based on the probability given\nby the neural network, rather than just picking the action with the highest score. This\napproach lets the agent find the right balance between exploring new actions and\nexploiting the actions that are known to work well. Here\u2019s an analogy: suppose you go\nto a restaurant for the first time, and all the dishes look equally appealing so you ran\u2010\ndomly pick one. If it turns out to be good, you can increase the probability to order it\nnext time, but you shouldn\u2019t increase that probability up to 100%, or else you will\nnever try out the other dishes, some of which may be even better than the one you\ntried.\nAlso note that in this particular environment, the past actions and observations can", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 1159, "content": "tried.\nAlso note that in this particular environment, the past actions and observations can\nsafely be ignored, since each observation contains the environment\u2019s full state. If there\nwere some hidden state, then you may need to consider past actions and observations\nas well. For example, if the environment only revealed the position of the cart but not\nits velocity, you would have to consider not only the current observation but also the\nprevious observation in order to estimate the current velocity. Another example is\nwhen the observations are noisy; in that case, you generally want to use the past few\nobservations to estimate the most likely current state. The CartPole problem is thus as\nNeural Network Policies \n| \n445\nDownload from finelybook www.finelybook.com", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 1160, "content": "simple as can be; the observations are noise-free and they contain the environment\u2019s\nfull state.\nHere is the code to build this neural network policy using TensorFlow:\nimport tensorflow as tf\nfrom tensorflow.contrib.layers import fully_connected\n# 1. Specify the neural network architecture\nn_inputs = 4 # == env.observation_space.shape[0]\nn_hidden = 4 # it's a simple task, we don't need more hidden neurons\nn_outputs = 1 # only outputs the probability of accelerating left\ninitializer = tf.contrib.layers.variance_scaling_initializer()\n# 2. Build the neural network\nX = tf.placeholder(tf.float32, shape=[None, n_inputs])\nhidden = fully_connected(X, n_hidden, activation_fn=tf.nn.elu,\n weights_initializer=initializer)\nlogits = fully_connected(hidden, n_outputs, activation_fn=None,\n weights_initializer=initializer)\noutputs = tf.nn.sigmoid(logits)\n# 3. Select a random action based on the estimated probabilities", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 1161, "content": "outputs = tf.nn.sigmoid(logits)\n# 3. Select a random action based on the estimated probabilities\np_left_and_right = tf.concat(axis=1, values=[outputs, 1 - outputs])\naction = tf.multinomial(tf.log(p_left_and_right), num_samples=1)\ninit = tf.global_variables_initializer()\nLet\u2019s go through this code:\n1. After the imports, we define the neural network architecture. The number of\ninputs is the size of the observation space (which in the case of the CartPole is\nfour), we just have four hidden units and no need for more, and we have just one\noutput probability (the probability of going left).\n2. Next we build the neural network. In this example, it\u2019s a vanilla Multi-Layer Per\u2010\nceptron, with a single output. Note that the output layer uses the logistic (sig\u2010\nmoid) activation function in order to output a probability from 0.0 to 1.0. If there\nwere more than two possible actions, there would be one output neuron per\naction, and you would use the softmax activation function instead.", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 1162, "content": "action, and you would use the softmax activation function instead.\n3. Lastly, we call the multinomial() function to pick a random action. This func\u2010\ntion independently samples one (or more) integers, given the log probability of\neach integer. For example, if you call it with the array [np.log(0.5),\nnp.log(0.2), np.log(0.3)] and with num_samples=5, then it will output five\nintegers, each of which will have a 50% probability of being 0, 20% of being 1,\nand 30% of being 2. In our case we just need one integer representing the action\nto take. Since the outputs tensor only contains the probability of going left, we\nmust first concatenate 1-outputs to it to have a tensor containing the probability\n446 \n| \nChapter 16: Reinforcement Learning\nDownload from finelybook www.finelybook.com", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 1163, "content": "of both left and right actions. Note that if there were more than two possible\nactions, the neural network would have to output one probability per action so\nyou would not need the concatenation step.\nOkay, we now have a neural network policy that will take observations and output\nactions. But how do we train it?\nEvaluating Actions: The Credit Assignment Problem\nIf we knew what the best action was at each step, we could train the neural network as\nusual, by minimizing the cross entropy between the estimated probability and the tar\u2010\nget probability. It would just be regular supervised learning. However, in Reinforce\u2010\nment Learning the only guidance the agent gets is through rewards, and rewards are\ntypically sparse and delayed. For example, if the agent manages to balance the pole\nfor 100 steps, how can it know which of the 100 actions it took were good, and which\nof them were bad? All it knows is that the pole fell after the last action, but surely this", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 1164, "content": "of them were bad? All it knows is that the pole fell after the last action, but surely this\nlast action is not entirely responsible. This is called the credit assignment problem:\nwhen the agent gets a reward, it is hard for it to know which actions should get credi\u2010\nted (or blamed) for it. Think of a dog that gets rewarded hours after it behaved well;\nwill it understand what it is rewarded for?\nTo tackle this problem, a common strategy is to evaluate an action based on the sum\nof all the rewards that come after it, usually applying a discount rate r at each step. For\nexample (see Figure 16-6), if an agent decides to go right three times in a row and gets\n+10 reward after the first step, 0 after the second step, and finally \u201350 after the third\nstep, then assuming we use a discount rate r = 0.8, the first action will have a total\nscore of 10 + r \u00d7 0 + r2 \u00d7 (\u201350) = \u201322. If the discount rate is close to 0, then future", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 1165, "content": "score of 10 + r \u00d7 0 + r2 \u00d7 (\u201350) = \u201322. If the discount rate is close to 0, then future\nrewards won\u2019t count for much compared to immediate rewards. Conversely, if the \ndiscount rate is close to 1, then rewards far into the future will count almost as much\nas immediate rewards. Typical discount rates are 0.95 or 0.99. With a discount rate of\n0.95, rewards 13 steps into the future count roughly for half as much as immediate\nrewards (since 0.9513 \u2248 0.5), while with a discount rate of 0.99, rewards 69 steps into\nthe future count for half as much as immediate rewards. In the CartPole environ\u2010\nment, actions have fairly short-term effects, so choosing a discount rate of 0.95 seems\nreasonable.\nEvaluating Actions: The Credit Assignment Problem \n| \n447\nDownload from finelybook www.finelybook.com", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 1166, "content": "9 \u201cSimple Statistical Gradient-Following Algorithms for Connectionist Reinforcement Learning,\u201d R. Williams\n(1992).\nFigure 16-6. Discounted rewards\nOf course, a good action may be followed by several bad actions that cause the pole to\nfall quickly, resulting in the good action getting a low score (similarly, a good actor\nmay sometimes star in a terrible movie). However, if we play the game enough times,\non average good actions will get a better score than bad ones. So, to get fairly reliable\naction scores, we must run many episodes and normalize all the action scores (by\nsubtracting the mean and dividing by the standard deviation). After that, we can rea\u2010\nsonably assume that actions with a negative score were bad while actions with a posi\u2010\ntive score were good. Perfect\u2014now that we have a way to evaluate each action, we are\nready to train our first agent using policy gradients. Let\u2019s see how.\nPolicy Gradients", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 1167, "content": "ready to train our first agent using policy gradients. Let\u2019s see how.\nPolicy Gradients\nAs discussed earlier, PG algorithms optimize the parameters of a policy by following\nthe gradients toward higher rewards. One popular class of PG algorithms, called\nREINFORCE algorithms, was introduced back in 19929 by Ronald Williams. Here is\none common variant:\n1. First, let the neural network policy play the game several times and at each step\ncompute the gradients that would make the chosen action even more likely, but\ndon\u2019t apply these gradients yet.\n448 \n| \nChapter 16: Reinforcement Learning\nDownload from finelybook www.finelybook.com", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 1168, "content": "10 We already did something similar in Chapter 11 when we discussed Gradient Clipping: we first computed the\ngradients, then we clipped them, and finally we applied the clipped gradients.\n2. Once you have run several episodes, compute each action\u2019s score (using the\nmethod described in the previous paragraph).\n3. If an action\u2019s score is positive, it means that the action was good and you want to\napply the gradients computed earlier to make the action even more likely to be\nchosen in the future. However, if the score is negative, it means the action was\nbad and you want to apply the opposite gradients to make this action slightly less\nlikely in the future. The solution is simply to multiply each gradient vector by the\ncorresponding action\u2019s score.\n4. Finally, compute the mean of all the resulting gradient vectors, and use it to per\u2010\nform a Gradient Descent step.\nLet\u2019s implement this algorithm using TensorFlow. We will train the neural network", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 1169, "content": "Let\u2019s implement this algorithm using TensorFlow. We will train the neural network\npolicy we built earlier so that it learns to balance the pole on the cart. Let\u2019s start by\ncompleting the construction phase we coded earlier to add the target probability, the \ncost function, and the training operation. Since we are acting as though the chosen\naction is the best possible action, the target probability must be 1.0 if the chosen\naction is action 0 (left) and 0.0 if it is action 1 (right):\ny = 1. - tf.to_float(action)\nNow that we have a target probability, we can define the cost function (cross entropy) \nand compute the gradients:\nlearning_rate = 0.01\ncross_entropy = tf.nn.sigmoid_cross_entropy_with_logits(\n labels=y, logits=logits)\noptimizer = tf.train.AdamOptimizer(learning_rate)\ngrads_and_vars = optimizer.compute_gradients(cross_entropy)\nNote that we are calling the optimizer\u2019s compute_gradients() method instead of the", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 1170, "content": "Note that we are calling the optimizer\u2019s compute_gradients() method instead of the\nminimize() method. This is because we want to tweak the gradients before we apply\nthem.10 The compute_gradients() method returns a list of gradient vector/variable\npairs (one pair per trainable variable). Let\u2019s put all the gradients in a list, to make it\nmore convenient to obtain their values:\ngradients = [grad for grad, variable in grads_and_vars]\nOkay, now comes the tricky part. During the execution phase, the algorithm will run\nthe policy and at each step it will evaluate these gradient tensors and store their val\u2010\nues. After a number of episodes it will tweak these gradients as explained earlier (i.e.,\nmultiply them by the action scores and normalize them) and compute the mean of\nthe tweaked gradients. Next, it will need to feed the resulting gradients back to the\nPolicy Gradients \n| \n449\nDownload from finelybook www.finelybook.com", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 1171, "content": "optimizer so that it can perform an optimization step. This means we need one place\u2010\nholder per gradient vector. Moreover, we must create the operation that will apply the\nupdated gradients. For this we will call the optimizer\u2019s apply_gradients() function,\nwhich takes a list of gradient vector/variable pairs. Instead of giving it the original\ngradient vectors, we will give it a list containing the updated gradients (i.e., the ones\nfed through the gradient placeholders):\ngradient_placeholders = []\ngrads_and_vars_feed = []\nfor grad, variable in grads_and_vars:\n gradient_placeholder = tf.placeholder(tf.float32, shape=grad.get_shape())\n gradient_placeholders.append(gradient_placeholder)\n grads_and_vars_feed.append((gradient_placeholder, variable))\ntraining_op = optimizer.apply_gradients(grads_and_vars_feed)\nLet\u2019s step back and take a look at the full construction phase:\nn_inputs = 4\nn_hidden = 4\nn_outputs = 1\ninitializer = tf.contrib.layers.variance_scaling_initializer()", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 1172, "content": "n_hidden = 4\nn_outputs = 1\ninitializer = tf.contrib.layers.variance_scaling_initializer()\nlearning_rate = 0.01\nX = tf.placeholder(tf.float32, shape=[None, n_inputs])\nhidden = fully_connected(X, n_hidden, activation_fn=tf.nn.elu,\n weights_initializer=initializer)\nlogits = fully_connected(hidden, n_outputs, activation_fn=None,\n weights_initializer=initializer)\noutputs = tf.nn.sigmoid(logits)\np_left_and_right = tf.concat(axis=1, values=[outputs, 1 - outputs])\naction = tf.multinomial(tf.log(p_left_and_right), num_samples=1)\ny = 1. - tf.to_float(action)\ncross_entropy = tf.nn.sigmoid_cross_entropy_with_logits(\n labels=y, logits=logits)\noptimizer = tf.train.AdamOptimizer(learning_rate)\ngrads_and_vars = optimizer.compute_gradients(cross_entropy)\ngradients = [grad for grad, variable in grads_and_vars]\ngradient_placeholders = []\ngrads_and_vars_feed = []\nfor grad, variable in grads_and_vars:", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 1173, "content": "gradient_placeholders = []\ngrads_and_vars_feed = []\nfor grad, variable in grads_and_vars:\n gradient_placeholder = tf.placeholder(tf.float32, shape=grad.get_shape())\n gradient_placeholders.append(gradient_placeholder)\n grads_and_vars_feed.append((gradient_placeholder, variable))\ntraining_op = optimizer.apply_gradients(grads_and_vars_feed)\ninit = tf.global_variables_initializer()\nsaver = tf.train.Saver()\n450 \n| \nChapter 16: Reinforcement Learning\nDownload from finelybook www.finelybook.com", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 1174, "content": "On to the execution phase! We will need a couple of functions to compute the total\ndiscounted rewards, given the raw rewards, and to normalize the results across multi\u2010\nple episodes:\ndef discount_rewards(rewards, discount_rate):\n discounted_rewards = np.empty(len(rewards))\n cumulative_rewards = 0\n for step in reversed(range(len(rewards))):\n cumulative_rewards = rewards[step] + cumulative_rewards * discount_rate\n discounted_rewards[step] = cumulative_rewards\n return discounted_rewards\ndef discount_and_normalize_rewards(all_rewards, discount_rate):\n all_discounted_rewards = [discount_rewards(rewards)\n for rewards in all_rewards]\n flat_rewards = np.concatenate(all_discounted_rewards)\n reward_mean = flat_rewards.mean()\n reward_std = flat_rewards.std()\n return [(discounted_rewards - reward_mean)/reward_std\n for discounted_rewards in all_discounted_rewards]\nLet\u2019s check that this works:", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 1175, "content": "for discounted_rewards in all_discounted_rewards]\nLet\u2019s check that this works:\n>>> discount_rewards([10, 0, -50], discount_rate=0.8)\narray([-22., -40., -50.])\n>>> discount_and_normalize_rewards([[10, 0, -50], [10, 20]], discount_rate=0.8)\n[array([-0.28435071, -0.86597718, -1.18910299]),\n array([ 1.26665318, 1.0727777 ])]\nThe call to discount_rewards() returns exactly what we expect (see Figure 16-6).\nYou can verify that the function discount_and_normalize_rewards() does indeed\nreturn the normalized scores for each action in both episodes. Notice that the first\nepisode was much worse than the second, so its normalized scores are all negative; all\nactions from the first episode would be considered bad, and conversely all actions\nfrom the second episode would be considered good.\nWe now have all we need to train the policy:\nn_iterations = 250 # number of training iterations\nn_max_steps = 1000 # max steps per episode", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 1176, "content": "n_max_steps = 1000 # max steps per episode\nn_games_per_update = 10 # train the policy every 10 episodes\nsave_iterations = 10 # save the model every 10 training iterations\ndiscount_rate = 0.95\nwith tf.Session() as sess:\n init.run()\n for iteration in range(n_iterations):\n all_rewards = [] # all sequences of raw rewards for each episode\n all_gradients = [] # gradients saved at each step of each episode\n for game in range(n_games_per_update):\n current_rewards = [] # all raw rewards from the current episode\n current_gradients = [] # all gradients from the current episode\nPolicy Gradients \n| \n451\nDownload from finelybook www.finelybook.com", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 1177, "content": "obs = env.reset()\n for step in range(n_max_steps):\n action_val, gradients_val = sess.run(\n [action, gradients],\n feed_dict={X: obs.reshape(1, n_inputs)}) # one obs\n obs, reward, done, info = env.step(action_val[0][0])\n current_rewards.append(reward)\n current_gradients.append(gradients_val)\n if done:\n break\n all_rewards.append(current_rewards)\n all_gradients.append(current_gradients)\n # At this point we have run the policy for 10 episodes, and we are\n # ready for a policy update using the algorithm described earlier.\n all_rewards = discount_and_normalize_rewards(all_rewards)\n feed_dict = {}\n for var_index, grad_placeholder in enumerate(gradient_placeholders):\n # multiply the gradients by the action scores, and compute the mean", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 1178, "content": "# multiply the gradients by the action scores, and compute the mean\n mean_gradients = np.mean(\n [reward * all_gradients[game_index][step][var_index]\n for game_index, rewards in enumerate(all_rewards)\n for step, reward in enumerate(rewards)],\n axis=0)\n feed_dict[grad_placeholder] = mean_gradients\n sess.run(training_op, feed_dict=feed_dict)\n if iteration % save_iterations == 0:\n saver.save(sess, \"./my_policy_net_pg.ckpt\")\nEach training iteration starts by running the policy for 10 episodes (with maximum\n1,000 steps per episode, to avoid running forever). At each step, we also compute the\ngradients, pretending that the chosen action was the best. After these 10 episodes\nhave been run, we compute the action scores using the discount_and_normal\nize_rewards() function; we go through each trainable variable, across all episodes", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 1179, "content": "ize_rewards() function; we go through each trainable variable, across all episodes\nand all steps, to multiply each gradient vector by its corresponding action score; and\nwe compute the mean of the resulting gradients. Finally, we run the training opera\u2010\ntion, feeding it these mean gradients (one per trainable variable). We also save the\nmodel every 10 training operations.\nAnd we\u2019re done! This code will train the neural network policy, and it will success\u2010\nfully learn to balance the pole on the cart (you can try it out in the Jupyter note\u2010\nbooks). Note that there are actually two ways the agent can lose the game: either the\npole can tilt too much, or the cart can go completely off the screen. With 250 training\niterations, the policy learns to balance the pole quite well, but it is not yet good\nenough at avoiding going off the screen. A few hundred more training iterations will\nfix that.\n452 \n| \nChapter 16: Reinforcement Learning\nDownload from finelybook www.finelybook.com", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 1180, "content": "Researchers try to find algorithms that work well even when the\nagent initially knows nothing about the environment. However,\nunless you are writing a paper, you should inject as much prior\nknowledge as possible into the agent, as it will speed up training\ndramatically. For example, you could add negative rewards propor\u2010\ntional to the distance from the center of the screen, and to the pole\u2019s\nangle. Also, if you already have a reasonably good policy (e.g.,\nhardcoded), you may want to train the neural network to imitate it\nbefore using policy gradients to improve it.\nDespite its relative simplicity, this algorithm is quite powerful. You can use it to tackle\nmuch harder problems than balancing a pole on a cart. In fact, AlphaGo was based\non a similar PG algorithm (plus Monte Carlo Tree Search, which is beyond the scope\nof this book).\nWe will now look at another popular family of algorithms. Whereas PG algorithms", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 1181, "content": "of this book).\nWe will now look at another popular family of algorithms. Whereas PG algorithms\ndirectly try to optimize the policy to increase rewards, the algorithms we will look at\nnow are less direct: the agent learns to estimate the expected sum of discounted future\nrewards for each state, or the expected sum of discounted future rewards for each\naction in each state, then uses this knowledge to decide how to act. To understand\nthese algorithms, we must first introduce Markov decision processes (MDP).\nMarkov Decision Processes\nIn the early 20th century, the mathematician Andrey Markov studied stochastic pro\u2010\ncesses with no memory, called Markov chains. Such a process has a fixed number of\nstates, and it randomly evolves from one state to another at each step. The probability\nfor it to evolve from a state s to a state s\u2032 is fixed, and it depends only on the pair (s,s\u2032),\nnot on past states (the system has no memory).", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 1182, "content": "not on past states (the system has no memory).\nFigure 16-7 shows an example of a Markov chain with four states. Suppose that the\nprocess starts in state s0, and there is a 70% chance that it will remain in that state at\nthe next step. Eventually it is bound to leave that state and never come back since no\nother state points back to s0. If it goes to state s1, it will then most likely go to state s2\n(90% probability), then immediately back to state s1 (with 100% probability). It may\nalternate a number of times between these two states, but eventually it will fall into\nstate s3 and remain there forever (this is a terminal state). Markov chains can have\nvery different dynamics, and they are heavily used in thermodynamics, chemistry,\nstatistics, and much more.\nMarkov Decision Processes \n| \n453\nDownload from finelybook www.finelybook.com", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 1183, "content": "11 \u201cA Markovian Decision Process,\u201d R. Bellman (1957).\nFigure 16-7. Example of a Markov chain\nMarkov decision processes were first described in the 1950s by Richard Bellman.11\nThey resemble Markov chains but with a twist: at each step, an agent can choose one\nof several possible actions, and the transition probabilities depend on the chosen\naction. Moreover, some state transitions return some reward (positive or negative),\nand the agent\u2019s goal is to find a policy that will maximize rewards over time.\nFor example, the MDP represented in Figure 16-8 has three states and up to three\npossible discrete actions at each step. If it starts in state s0, the agent can choose\nbetween actions a0, a1, or a2. If it chooses action a1, it just remains in state s0 with cer\u2010\ntainty, and without any reward. It can thus decide to stay there forever if it wants. But\nif it chooses action a0, it has a 70% probability of gaining a reward of +10, and", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 1184, "content": "if it chooses action a0, it has a 70% probability of gaining a reward of +10, and\nremaining in state s0. It can then try again and again to gain as much reward as possi\u2010\nble. But at one point it is going to end up instead in state s1. In state s1 it has only two\npossible actions: a0 or a1. It can choose to stay put by repeatedly choosing action a1, or\nit can choose to move on to state s2 and get a negative reward of \u201350 (ouch). In state s3\nit has no other choice than to take action a1, which will most likely lead it back to\nstate s0, gaining a reward of +40 on the way. You get the picture. By looking at this\nMDP, can you guess which strategy will gain the most reward over time? In state s0 it\nis clear that action a0 is the best option, and in state s3 the agent has no choice but to\ntake action a1, but in state s1 it is not obvious whether the agent should stay put (a0) or\ngo through the fire (a2).\n454 \n| \nChapter 16: Reinforcement Learning\nDownload from finelybook www.finelybook.com", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 1185, "content": "Figure 16-8. Example of a Markov decision process\nBellman found a way to estimate the optimal state value of any state s, noted V*(s),\nwhich is the sum of all discounted future rewards the agent can expect on average\nafter it reaches a state s, assuming it acts optimally. He showed that if the agent acts\noptimally, then the Bellman Optimality Equation applies (see Equation 16-1). This\nrecursive equation says that if the agent acts optimally, then the optimal value of the\ncurrent state is equal to the reward it will get on average after taking one optimal\naction, plus the expected optimal value of all possible next states that this action can\nlead to.\nEquation 16-1. Bellman Optimality Equation\nV* s = maxa \u2211s\u2032T s, a, s\u2032 R s, a, s\u2032 + \u03b3 . V* s\u2032\nfor all s\n\u2022 T(s, a, s\u2032) is the transition probability from state s to state s\u2032, given that the agent\nchose action a.\n\u2022 R(s, a, s\u2032) is the reward that the agent gets when it goes from state s to state s\u2032,\ngiven that the agent chose action a.", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 1186, "content": "given that the agent chose action a.\n\u2022 \u03b3 is the discount rate.\nThis equation leads directly to an algorithm that can precisely estimate the optimal\nstate value of every possible state: you first initialize all the state value estimates to\nzero, and then you iteratively update them using the Value Iteration algorithm (see\nEquation 16-2). A remarkable result is that, given enough time, these estimates are\nMarkov Decision Processes \n| \n455\nDownload from finelybook www.finelybook.com", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 1187, "content": "guaranteed to converge to the optimal state values, corresponding to the optimal pol\u2010\nicy.\nEquation 16-2. Value Iteration algorithm\nVk + 1 s\nmax\na \u2211\ns\u2032\nT s, a, s\u2032 R s, a, s\u2032 + \u03b3 . Vk s\u2032\nfor all s\n\u2022 Vk(s) is the estimated value of state s at the kth iteration of the algorithm.\nThis algorithm is an example of Dynamic Programming, which\nbreaks down a complex problem (in this case estimating a poten\u2010\ntially infinite sum of discounted future rewards) into tractable sub-\nproblems that can be tackled iteratively (in this case finding the\naction that maximizes the average reward plus the discounted next\nstate value).\nKnowing the optimal state values can be useful, in particular to evaluate a policy, but\nit does not tell the agent explicitly what to do. Luckily, Bellman found a very similar\nalgorithm to estimate the optimal state-action values, generally called Q-Values. The\noptimal Q-Value of the state-action pair (s,a), noted Q*(s,a), is the sum of discounted", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 1188, "content": "optimal Q-Value of the state-action pair (s,a), noted Q*(s,a), is the sum of discounted\nfuture rewards the agent can expect on average after it reaches the state s and chooses\naction a, but before it sees the outcome of this action, assuming it acts optimally after\nthat action.\nHere is how it works: once again, you start by initializing all the Q-Value estimates to\nzero, then you update them using the Q-Value Iteration algorithm (see Equation\n16-3).\nEquation 16-3. Q-Value Iteration algorithm\nQk + 1 s, a\n\u2211\ns\u2032\nT s, a, s\u2032 R s, a, s\u2032 + \u03b3 . max\na\u2032\nQk s\u2032, a\u2032\nfor all s, a\nOnce you have the optimal Q-Values, defining the optimal policy, noted \u03c0*(s), is triv\u2010\nial: when the agent is in state s, it should choose the action with the highest Q-Value\nfor that state: \u03c0* s = argmax\na\nQ* s, a .\nLet\u2019s apply this algorithm to the MDP represented in Figure 16-8. First, we need to\ndefine the MDP:\nnan=np.nan # represents impossible actions\nT = np.array([ # shape=[s, a, s']", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 1189, "content": "define the MDP:\nnan=np.nan # represents impossible actions\nT = np.array([ # shape=[s, a, s']\n [[0.7, 0.3, 0.0], [1.0, 0.0, 0.0], [0.8, 0.2, 0.0]],\n456 \n| \nChapter 16: Reinforcement Learning\nDownload from finelybook www.finelybook.com", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 1190, "content": "[[0.0, 1.0, 0.0], [nan, nan, nan], [0.0, 0.0, 1.0]],\n [[nan, nan, nan], [0.8, 0.1, 0.1], [nan, nan, nan]],\n ])\nR = np.array([ # shape=[s, a, s']\n [[10., 0.0, 0.0], [0.0, 0.0, 0.0], [0.0, 0.0, 0.0]],\n [[10., 0.0, 0.0], [nan, nan, nan], [0.0, 0.0, -50.]],\n [[nan, nan, nan], [40., 0.0, 0.0], [nan, nan, nan]],\n ])\npossible_actions = [[0, 1, 2], [0, 2], [1]]\nNow let\u2019s run the Q-Value Iteration algorithm:\nQ = np.full((3, 3), -np.inf) # -inf for impossible actions\nfor state, actions in enumerate(possible_actions):\n Q[state, actions] = 0.0 # Initial value = 0.0, for all possible actions\nlearning_rate = 0.01\ndiscount_rate = 0.95\nn_iterations = 100\nfor iteration in range(n_iterations):\n Q_prev = Q.copy()\n for s in range(3):\n for a in possible_actions[s]:\n Q[s, a] = np.sum([\n T[s, a, sp] * (R[s, a, sp] + discount_rate * np.max(Q_prev[sp]))\n for sp in range(3)\n ])", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 1191, "content": "for sp in range(3)\n ])\nThe resulting Q-Values look like this:\n>>> Q\narray([[ 21.89498982, 20.80024033, 16.86353093],\n [ 1.11669335, -inf, 1.17573546],\n [ -inf, 53.86946068, -inf]])\n>>> np.argmax(Q, axis=1) # optimal action for each state\narray([0, 2, 1])\nThis gives us the optimal policy for this MDP, when using a discount rate of 0.95: in\nstate s0 choose action a0, in state s1 choose action a2 (go through the fire!), and in state\ns2 choose action a1 (the only possible action). Interestingly, if you reduce the discount\nrate to 0.9, the optimal policy changes: in state s1 the best action becomes a0 (stay put;\ndon\u2019t go through the fire). It makes sense because if you value the present much more\nthan the future, then the prospect of future rewards is not worth immediate pain.\nTemporal Difference Learning and Q-Learning\nReinforcement Learning problems with discrete actions can often be modeled as", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 1192, "content": "Reinforcement Learning problems with discrete actions can often be modeled as\nMarkov decision processes, but the agent initially has no idea what the transition\nprobabilities are (it does not know T(s, a, s\u2032)), and it does not know what the rewards\nare going to be either (it does not know R(s, a, s\u2032)). It must experience each state and\nTemporal Difference Learning and Q-Learning \n| \n457\nDownload from finelybook www.finelybook.com", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 1193, "content": "each transition at least once to know the rewards, and it must experience them multi\u2010\nple times if it is to have a reasonable estimate of the transition probabilities.\nThe Temporal Difference Learning (TD Learning) algorithm is very similar to the\nValue Iteration algorithm, but tweaked to take into account the fact that the agent has\nonly partial knowledge of the MDP. In general we assume that the agent initially\nknows only the possible states and actions, and nothing more. The agent uses an\nexploration policy\u2014for example, a purely random policy\u2014to explore the MDP, and as\nit progresses the TD Learning algorithm updates the estimates of the state values\nbased on the transitions and rewards that are actually observed (see Equation 16-4).\nEquation 16-4. TD Learning algorithm\nVk + 1 s\n1 \u2212\u03b1 Vk s + \u03b1 r + \u03b3 . Vk s\u2032\n\u2022 \u03b1 is the learning rate (e.g., 0.01).\nTD Learning has many similarities with Stochastic Gradient\nDescent, in particular the fact that it handles one sample at a time.", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 1194, "content": "Descent, in particular the fact that it handles one sample at a time.\nJust like SGD, it can only truly converge if you gradually reduce the\nlearning rate (otherwise it will keep bouncing around the opti\u2010\nmum).\nFor each state s, this algorithm simply keeps track of a running average of the imme\u2010\ndiate rewards the agent gets upon leaving that state, plus the rewards it expects to get\nlater (assuming it acts optimally).\nSimilarly, the Q-Learning algorithm is an adaptation of the Q-Value Iteration algo\u2010\nrithm to the situation where the transition probabilities and the rewards are initially\nunknown (see Equation 16-5).\nEquation 16-5. Q-Learning algorithm\nQk + 1 s, a\n1 \u2212\u03b1 Qk s, a + \u03b1 r + \u03b3 . max\na\u2032\nQk s\u2032, a\u2032\nFor each state-action pair (s, a), this algorithm keeps track of a running average of the\nrewards r the agent gets upon leaving the state s with action a, plus the rewards it\nexpects to get later. Since the target policy would act optimally, we take the maximum", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 1195, "content": "expects to get later. Since the target policy would act optimally, we take the maximum\nof the Q-Value estimates for the next state.\nHere is how Q-Learning can be implemented:\n458 \n| \nChapter 16: Reinforcement Learning\nDownload from finelybook www.finelybook.com", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 1196, "content": "import numpy.random as rnd\nlearning_rate0 = 0.05\nlearning_rate_decay = 0.1\nn_iterations = 20000\ns = 0 # start in state 0\nQ = np.full((3, 3), -np.inf) # -inf for impossible actions\nfor state, actions in enumerate(possible_actions):\n Q[state, actions] = 0.0 # Initial value = 0.0, for all possible actions\nfor iteration in range(n_iterations):\n a = rnd.choice(possible_actions[s]) # choose an action (randomly)\n sp = rnd.choice(range(3), p=T[s, a]) # pick next state using T[s, a]\n reward = R[s, a, sp]\n learning_rate = learning_rate0 / (1 + iteration * learning_rate_decay)\n Q[s, a] = learning_rate * Q[s, a] + (1 - learning_rate) * (\n reward + discount_rate * np.max(Q[sp])\n )\n s = sp # move to next state\nGiven enough iterations, this algorithm will converge to the optimal Q-Values. This is\ncalled an off-policy algorithm because the policy being trained is not the one being", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 1197, "content": "called an off-policy algorithm because the policy being trained is not the one being\nexecuted. It is somewhat surprising that this algorithm is capable of learning the opti\u2010\nmal policy by just watching an agent act randomly (imagine learning to play golf\nwhen your teacher is a drunken monkey). Can we do better?\nExploration Policies\nOf course Q-Learning can work only if the exploration policy explores the MDP\nthoroughly enough. Although a purely random policy is guaranteed to eventually\nvisit every state and every transition many times, it may take an extremely long time\nto do so. Therefore, a better option is to use the \u03b5-greedy policy: at each step it acts\nrandomly with probability \u03b5, or greedily (choosing the action with the highest Q-\nValue) with probability 1-\u03b5. The advantage of the \u03b5-greedy policy (compared to a\ncompletely random policy) is that it will spend more and more time exploring the\ninteresting parts of the environment, as the Q-Value estimates get better and better,", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 1198, "content": "interesting parts of the environment, as the Q-Value estimates get better and better,\nwhile still spending some time visiting unknown regions of the MDP. It is quite com\u2010\nmon to start with a high value for \u03b5 (e.g., 1.0) and then gradually reduce it (e.g., down\nto 0.05).\nAlternatively, rather than relying on chance for exploration, another approach is to\nencourage the exploration policy to try actions that it has not tried much before. This\ncan be implemented as a bonus added to the Q-Value estimates, as shown in Equation\n16-6.\nTemporal Difference Learning and Q-Learning \n| \n459\nDownload from finelybook www.finelybook.com", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 1199, "content": "Equation 16-6. Q-Learning using an exploration function\nQ s, a\n1 \u2212\u03b1 Q s, a + \u03b1 r + \u03b3 . max\n\u03b1\u2032\nf Q s\u2032, a\u2032 , N s\u2032, a\u2032\n\u2022 N(s\u2032, a\u2032) counts the number of times the action a\u2032 was chosen in state s\u2032.\n\u2022 f(q, n) is an exploration function, such as f(q, n) = q + K/(1 + n), where K is a\ncuriosity hyperparameter that measures how much the agent is attracted to to the\nunknown.\nApproximate Q-Learning\nThe main problem with Q-Learning is that it does not scale well to large (or even\nmedium) MDPs with many states and actions. Consider trying to use Q-Learning to\ntrain an agent to play Ms. Pac-Man. There are over 250 pellets that Ms. Pac-Man can\neat, each of which can be present or absent (i.e., already eaten). So the number of pos\u2010\nsible states is greater than 2250 \u2248 1075 (and that\u2019s considering the possible states only of\nthe pellets). This is way more than atoms in the observable universe, so there\u2019s abso\u2010\nlutely no way you can keep track of an estimate for every single Q-Value.", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 1200, "content": "lutely no way you can keep track of an estimate for every single Q-Value.\nThe solution is to find a function that approximates the Q-Values using a manageable\nnumber of parameters. This is called Approximate Q-Learning. For years it was rec\u2010\nommended to use linear combinations of hand-crafted features extracted from the\nstate (e.g., distance of the closest ghosts, their directions, and so on) to estimate Q-\nValues, but DeepMind showed that using deep neural networks can work much bet\u2010\nter, especially for complex problems, and it does not require any feature engineering.\nA DNN used to estimate Q-Values is called a deep Q-network (DQN), and using a\nDQN for Approximate Q-Learning is called Deep Q-Learning.\nIn the rest of this chapter, we will use Deep Q-Learning to train an agent to play Ms.\nPac-Man, much like DeepMind did in 2013. The code can easily be tweaked to learn\nto play the majority of Atari games quite well. It can achieve superhuman skill at most", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 1201, "content": "to play the majority of Atari games quite well. It can achieve superhuman skill at most\naction games, but it is not so good at games with long-running storylines.\nLearning to Play Ms. Pac-Man Using Deep Q-Learning\nSince we will be using an Atari environment, we must first install OpenAI gym\u2019s Atari\ndependencies. While we\u2019re at it, we will also install dependencies for other OpenAI\ngym environments that you may want to play with. On macOS, assuming you have\ninstalled Homebrew, you need to run:\n$ brew install cmake boost boost-python sdl2 swig wget\n460 \n| \nChapter 16: Reinforcement Learning\nDownload from finelybook www.finelybook.com", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 1202, "content": "On Ubuntu, type the following command (replacing python3 with python if you are\nusing Python 2):\n$ apt-get install -y python3-numpy python3-dev cmake zlib1g-dev libjpeg-dev\\\n xvfb libav-tools xorg-dev python3-opengl libboost-all-dev libsdl2-dev swig\nThen install the extra Python modules:\n$ pip3 install --upgrade 'gym[all]'\nIf everything went well, you should be able to create a Ms. Pac-Man environment:\n>>> env = gym.make(\"MsPacman-v0\")\n>>> obs = env.reset()\n>>> obs.shape # [height, width, channels]\n(210, 160, 3)\n>>> env.action_space\nDiscrete(9)\nAs you can see, there are nine discrete actions available, which correspond to the nine\npossible positions of the joystick (left, right, up, down, center, upper left, and so on),\nand the observations are simply screenshots of the Atari screen (see Figure 16-9, left),\nrepresented as 3D NumPy arrays. These images are a bit large, so we will create a\nsmall preprocessing function that will crop the image and shrink it down to 88 \u00d7 80", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 1203, "content": "small preprocessing function that will crop the image and shrink it down to 88 \u00d7 80\npixels, convert it to grayscale, and improve the contrast of Ms. Pac-Man. This will\nreduce the amount of computations required by the DQN, and speed up training.\nmspacman_color = np.array([210, 164, 74]).mean()\ndef preprocess_observation(obs):\n img = obs[1:176:2, ::2] # crop and downsize\n img = img.mean(axis=2) # to greyscale\n img[img==mspacman_color] = 0 # improve contrast\n img = (img - 128) / 128 - 1 # normalize from -1. to 1.\n return img.reshape(88, 80, 1)\nThe result of preprocessing is shown in Figure 16-9 (right).\nLearning to Play Ms. Pac-Man Using Deep Q-Learning \n| \n461\nDownload from finelybook www.finelybook.com", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 1204, "content": "Figure 16-9. Ms. Pac-Man observation, original (left) and after preprocessing (right)\nNext, let\u2019s create the DQN. It could just take a state-action pair (s,a) as input, and out\u2010\nput an estimate of the corresponding Q-Value Q(s,a), but since the actions are dis\u2010\ncrete it is more convenient to use a neural network that takes only a state s as input\nand outputs one Q-Value estimate per action. The DQN will be composed of three\nconvolutional layers, followed by two fully connected layers, including the output\nlayer (see Figure 16-10).\nFigure 16-10. Deep Q-network to play Ms. Pac-Man\n462 \n| \nChapter 16: Reinforcement Learning\nDownload from finelybook www.finelybook.com", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 1205, "content": "As we will see, the training algorithm we will use requires two DQNs with the same\narchitecture (but different parameters): one will be used to drive Ms. Pac-Man during\ntraining (the actor), and the other will watch the actor and learn from its trials and\nerrors (the critic). At regular intervals we will copy the critic to the actor. Since we\nneed two identical DQNs, we will create a q_network() function to build them:\nfrom tensorflow.contrib.layers import convolution2d, fully_connected\ninput_height = 88\ninput_width = 80\ninput_channels = 1\nconv_n_maps = [32, 64, 64]\nconv_kernel_sizes = [(8,8), (4,4), (3,3)]\nconv_strides = [4, 2, 1]\nconv_paddings = [\"SAME\"]*3\nconv_activation = [tf.nn.relu]*3\nn_hidden_in = 64 * 11 * 10 # conv3 has 64 maps of 11x10 each\nn_hidden = 512\nhidden_activation = tf.nn.relu\nn_outputs = env.action_space.n # 9 discrete actions are available\ninitializer = tf.contrib.layers.variance_scaling_initializer()\ndef q_network(X_state, scope):\n prev_layer = X_state", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 1206, "content": "def q_network(X_state, scope):\n prev_layer = X_state\n conv_layers = []\n with tf.variable_scope(scope) as scope:\n for n_maps, kernel_size, stride, padding, activation in zip(\n conv_n_maps, conv_kernel_sizes, conv_strides,\n conv_paddings, conv_activation):\n prev_layer = convolution2d(\n prev_layer, num_outputs=n_maps, kernel_size=kernel_size,\n stride=stride, padding=padding, activation_fn=activation,\n weights_initializer=initializer)\n conv_layers.append(prev_layer)\n last_conv_layer_flat = tf.reshape(prev_layer, shape=[-1, n_hidden_in])\n hidden = fully_connected(\n last_conv_layer_flat, n_hidden, activation_fn=hidden_activation,\n weights_initializer=initializer)\n outputs = fully_connected(\n hidden, n_outputs, activation_fn=None,\n weights_initializer=initializer)", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 1207, "content": "hidden, n_outputs, activation_fn=None,\n weights_initializer=initializer)\n trainable_vars = tf.get_collection(tf.GraphKeys.TRAINABLE_VARIABLES,\n scope=scope.name)\n trainable_vars_by_name = {var.name[len(scope.name):]: var\n for var in trainable_vars}\n return outputs, trainable_vars_by_name\nThe first part of this code defines the hyperparameters of the DQN architecture.\nThen the q_network() function creates the DQN, taking the environment\u2019s state\nX_state as input, and the name of the variable scope. Note that we will just use one\nLearning to Play Ms. Pac-Man Using Deep Q-Learning \n| \n463\nDownload from finelybook www.finelybook.com", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 1208, "content": "observation to represent the environment\u2019s state since there\u2019s almost no hidden state\n(except for blinking objects and the ghosts\u2019 directions).\nThe trainable_vars_by_name dictionary gathers all the trainable variables of this\nDQN. It will be useful in a minute when we create operations to copy the critic DQN\nto the actor DQN. The keys of the dictionary are the names of the variables, stripping\nthe part of the prefix that just corresponds to the scope\u2019s name. It looks like this:\n>>> trainable_vars_by_name\n{'/Conv/biases:0': ,\n '/Conv/weights:0': ,\n '/Conv_1/biases:0': ,\n '/Conv_1/weights:0': ,\n '/Conv_2/biases:0': ,\n '/Conv_2/weights:0': ,", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 1209, "content": "'/Conv_2/weights:0': ,\n '/fully_connected/biases:0': ,\n '/fully_connected/weights:0': ,\n '/fully_connected_1/biases:0': ,\n '/fully_connected_1/weights:0': }\nNow let\u2019s create the input placeholder, the two DQNs, and the operation to copy the\ncritic DQN to the actor DQN:\nX_state = tf.placeholder(tf.float32, shape=[None, input_height, input_width,\n input_channels])\nactor_q_values, actor_vars = q_network(X_state, scope=\"q_networks/actor\")\ncritic_q_values, critic_vars = q_network(X_state, scope=\"q_networks/critic\")\ncopy_ops = [actor_var.assign(critic_vars[var_name])\n for var_name, actor_var in actor_vars.items()]\ncopy_critic_to_actor = tf.group(*copy_ops)", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 1210, "content": "copy_critic_to_actor = tf.group(*copy_ops)\nLet\u2019s step back for a second: we now have two DQNs that are both capable of taking\nan environment state (i.e., a preprocessed observation) as input and outputting an\nestimated Q-Value for each possible action in that state. Plus we have an operation\ncalled copy_critic_to_actor to copy all the trainable variables of the critic DQN to\nthe actor DQN. We use TensorFlow\u2019s tf.group() function to group all the assign\u2010\nment operations into a single convenient operation.\nThe actor DQN can be used to play Ms. Pac-Man (initially very badly). As discussed\nearlier, you want it to explore the game thoroughly enough, so you generally want to\ncombine it with an \u03b5-greedy policy or another exploration strategy.\nBut what about the critic DQN? How will it learn to play the game? The short answer\nis that it will try to make its Q-Value predictions match the Q-Values estimated by the", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 1211, "content": "is that it will try to make its Q-Value predictions match the Q-Values estimated by the\nactor through its experience of the game. Specifically, we will let the actor play for a\nwhile, storing all its experiences in a replay memory. Each memory will be a 5-tuple\n(state, action, next state, reward, continue), where the \u201ccontinue\u201d item will be equal to\n0.0 when the game is over, or 1.0 otherwise. Next, at regular intervals we will sample a\n464 \n| \nChapter 16: Reinforcement Learning\nDownload from finelybook www.finelybook.com", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 1212, "content": "batch of memories from the replay memory, and we will estimate the Q-Values from\nthese memories. Finally, we will train the critic DQN to predict these Q-Values using\nregular supervised learning techniques. Once every few training iterations, we will\ncopy the critic DQN to the actor DQN. And that\u2019s it! Equation 16-7 shows the cost\nfunction used to train the critic DQN:\nEquation 16-7. Deep Q-Learning cost function\nJ \u03b8critic = 1\nm \u2211\ni = 1\nm\ny i \u2212Q s i , a i , \u03b8critic\n2\nwith y i = r i + \u03b3 . max\na\u2032\nQ s\u2032 i , a\u2032, \u03b8actor\n\u2022 s(i), a(i), r(i) and s\u2032(i) are respectively the state, action, reward, and next state of the\nith memory sampled from the replay memory.\n\u2022 m is the size of the memory batch.\n\u2022 \u03b8critic and \u03b8actor are the critic and the actor\u2019s parameters.\n\u2022 Q(s(i),a(i),\u03b8critic) is the critic DQN\u2019s prediction of the ith memorized state-action\u2019s Q-\nValue.\n\u2022 Q(s\u2032(i), a\u2032, \u03b8actor) is the actor DQN\u2019s prediction of the Q-Value it can expect from\nthe next state s\u2032(i) if it chooses action a\u2032.", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 1213, "content": "the next state s\u2032(i) if it chooses action a\u2032.\n\u2022 y(i) is the target Q-Value for the ith memory. Note that it is equal to the reward\nactually observed by the actor, plus the actor\u2019s prediction of what future rewards it\nshould expect if it were to play optimally (as far as it knows).\n\u2022 J(\u03b8critic) is the cost function used to train the critic DQN. As you can see, it is just\nthe Mean Squared Error between the target Q-Values y(i) as estimated by the actor\nDQN, and the critic DQN\u2019s predictions of these Q-Values.\nThe replay memory is optional, but highly recommended. Without\nit, you would train the critic DQN using consecutive experiences\nthat may be very correlated. This would introduce a lot of bias and\nslow down the training algorithm\u2019s convergence. By using a replay\nmemory, we ensure that the memories fed to the training algorithm\ncan be fairly uncorrelated.\nLet\u2019s add the critic DQN\u2019s training operations. First, we need to be able to compute its", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 1214, "content": "Let\u2019s add the critic DQN\u2019s training operations. First, we need to be able to compute its\npredicted Q-Values for each state-action in the memory batch. Since the DQN out\u2010\nputs one Q-Value for every possible action, we need to keep only the Q-Value that\ncorresponds to the action that was actually chosen in this memory. For this, we will\nconvert the action to a one-hot vector (recall that this is a vector full of 0s except for a\nLearning to Play Ms. Pac-Man Using Deep Q-Learning \n| \n465\nDownload from finelybook www.finelybook.com", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 1215, "content": "1 at the ith index), and multiply it by the Q-Values: this will zero out all Q-Values\nexcept for the one corresponding to the memorized action. Then just sum over the\nfirst axis to obtain only the desired Q-Value prediction for each memory.\nX_action = tf.placeholder(tf.int32, shape=[None])\nq_value = tf.reduce_sum(critic_q_values * tf.one_hot(X_action, n_outputs),\n axis=1, keep_dims=True)\nNext let\u2019s add the training operations, assuming the target Q-Values will be fed\nthrough a placeholder. We also create a nontrainable variable called global_step. \nThe optimizer\u2019s minimize() operation will take care of incrementing it. Plus we cre\u2010\nate the usual init operation and a Saver.\ny = tf.placeholder(tf.float32, shape=[None, 1])\ncost = tf.reduce_mean(tf.square(y - q_value))\nglobal_step = tf.Variable(0, trainable=False, name='global_step')\noptimizer = tf.train.AdamOptimizer(learning_rate)\ntraining_op = optimizer.minimize(cost, global_step=global_step)", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 1216, "content": "training_op = optimizer.minimize(cost, global_step=global_step)\ninit = tf.global_variables_initializer()\nsaver = tf.train.Saver()\nThat\u2019s it for the construction phase. Before we look at the execution phase, we will\nneed a couple of tools. First, let\u2019s start by implementing the replay memory. We will\nuse a deque list since it is very efficient at pushing items to the queue and popping\nthem out from the end of the list when the maximum memory size is reached. We\nwill also write a small function to randomly sample a batch of experiences from the\nreplay memory:\nfrom collections import deque\nreplay_memory_size = 10000\nreplay_memory = deque([], maxlen=replay_memory_size)\ndef sample_memories(batch_size):\n indices = rnd.permutation(len(replay_memory))[:batch_size]\n cols = [[], [], [], [], []] # state, action, reward, next_state, continue\n for idx in indices:\n memory = replay_memory[idx]\n for col, value in zip(cols, memory):\n col.append(value)", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 1217, "content": "for col, value in zip(cols, memory):\n col.append(value)\n cols = [np.array(col) for col in cols]\n return (cols[0], cols[1], cols[2].reshape(-1, 1), cols[3],\n cols[4].reshape(-1, 1))\nNext, we will need the actor to explore the game. We will use the \u03b5-greedy policy, and\ngradually decrease \u03b5 from 1.0 to 0.05, in 50,000 training steps:\neps_min = 0.05\neps_max = 1.0\neps_decay_steps = 50000\n466 \n| \nChapter 16: Reinforcement Learning\nDownload from finelybook www.finelybook.com", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 1218, "content": "def epsilon_greedy(q_values, step):\n epsilon = max(eps_min, eps_max - (eps_max-eps_min) * step/eps_decay_steps)\n if rnd.rand() < epsilon:\n return rnd.randint(n_outputs) # random action\n else:\n return np.argmax(q_values) # optimal action\nThat\u2019s it! We have all we need to start training. The execution phase does not contain\nanything too complex, but it is a bit long, so take a deep breath. Ready? Let\u2019s go! First,\nlet\u2019s initialize a few variables:\nn_steps = 100000 # total number of training steps\ntraining_start = 1000 # start training after 1,000 game iterations\ntraining_interval = 3 # run a training step every 3 game iterations\nsave_steps = 50 # save the model every 50 training steps\ncopy_steps = 25 # copy the critic to the actor every 25 training steps\ndiscount_rate = 0.95\nskip_start = 90 # skip the start of every game (it's just waiting time)\nbatch_size = 50\niteration = 0 # game iterations\ncheckpoint_path = \"./my_dqn.ckpt\"", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 1219, "content": "batch_size = 50\niteration = 0 # game iterations\ncheckpoint_path = \"./my_dqn.ckpt\"\ndone = True # env needs to be reset\nNext, let\u2019s open the session and run the main training loop:\nwith tf.Session() as sess:\n if os.path.isfile(checkpoint_path):\n saver.restore(sess, checkpoint_path)\n else:\n init.run()\n while True:\n step = global_step.eval()\n if step >= n_steps:\n break\n iteration += 1\n if done: # game over, start again\n obs = env.reset()\n for skip in range(skip_start): # skip the start of each game\n obs, reward, done, info = env.step(0)\n state = preprocess_observation(obs)\n # Actor evaluates what to do\n q_values = actor_q_values.eval(feed_dict={X_state: [state]})\n action = epsilon_greedy(q_values, step)\n # Actor plays\n obs, reward, done, info = env.step(action)\n next_state = preprocess_observation(obs)", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 1220, "content": "obs, reward, done, info = env.step(action)\n next_state = preprocess_observation(obs)\n # Let's memorize what just happened\n replay_memory.append((state, action, reward, next_state, 1.0 - done))\n state = next_state\nLearning to Play Ms. Pac-Man Using Deep Q-Learning \n| \n467\nDownload from finelybook www.finelybook.com", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 1221, "content": "if iteration < training_start or iteration % training_interval != 0:\n continue\n # Critic learns\n X_state_val, X_action_val, rewards, X_next_state_val, continues = (\n sample_memories(batch_size))\n next_q_values = actor_q_values.eval(\n feed_dict={X_state: X_next_state_val})\n max_next_q_values = np.max(next_q_values, axis=1, keepdims=True)\n y_val = rewards + continues * discount_rate * max_next_q_values\n training_op.run(feed_dict={X_state: X_state_val,\n X_action: X_action_val, y: y_val})\n # Regularly copy critic to actor\n if step % copy_steps == 0:\n copy_critic_to_actor.run()\n # And save regularly\n if step % save_steps == 0:\n saver.save(sess, checkpoint_path)\nWe start by restoring the models if a checkpoint file exists, or else we just initialize the", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 1222, "content": "We start by restoring the models if a checkpoint file exists, or else we just initialize the\nvariables normally. Then the main loop starts, where iteration counts the total\nnumber of game steps we have gone through since the program started, and step\ncounts the total number of training steps since training started (if a checkpoint is\nrestored, the global step is restored as well). Then the code resets the game (and skips\nthe first boring game steps, where nothing happens). Next, the actor evaluates what to\ndo, and plays the game, and its experience is memorized in replay memory. Then, at\nregular intervals (after a warmup period), the critic goes through a training step. It\nsamples a batch of memories and asks the actor to estimate the Q-Values of all actions\nfor the next state, and it applies Equation 16-7 to compute the target Q-Value y_val.\nThe only tricky part here is that we must multiply the next state\u2019s Q-Values by the", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 1223, "content": "The only tricky part here is that we must multiply the next state\u2019s Q-Values by the\ncontinues vector to zero out the Q-Values corresponding to memories where the\ngame was over. Next we run a training operation to improve the critic\u2019s ability to pre\u2010\ndict Q-Values. Finally, at regular intervals we copy the critic to the actor, and we save\nthe model.\n468 \n| \nChapter 16: Reinforcement Learning\nDownload from finelybook www.finelybook.com", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 1224, "content": "Unfortunately, training is very slow: if you use your laptop for\ntraining, it will take days before Ms. Pac-Man gets any good, and if\nyou look at the learning curve, measuring the average rewards per\nepisode, you will notice that it is extremely noisy. At some points\nthere may be no apparent progress for a very long time until sud\u2010\ndenly the agent learns to survive a reasonable amount of time. As\nmentioned earlier, one solution is to inject as much prior knowl\u2010\nedge as possible into the model (e.g., through preprocessing,\nrewards, and so on), and you can also try to bootstrap the model by\nfirst training it to imitate a basic strategy. In any case, RL still\nrequires quite a lot of patience and tweaking, but the end result is\nvery exciting.\nExercises\n1. How would you define Reinforcement Learning? How is it different from regular\nsupervised or unsupervised learning?\n2. Can you think of three possible applications of RL that were not mentioned in", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 1225, "content": "2. Can you think of three possible applications of RL that were not mentioned in\nthis chapter? For each of them, what is the environment? What is the agent?\nWhat are possible actions? What are the rewards?\n3. What is the discount rate? Can the optimal policy change if you modify the dis\u2010\ncount rate?\n4. How do you measure the performance of a Reinforcement Learning agent?\n5. What is the credit assignment problem? When does it occur? How can you allevi\u2010\nate it?\n6. What is the point of using a replay memory?\n7. What is an off-policy RL algorithm?\n8. Use Deep Q-Learning to tackle OpenAI gym\u2019s \u201cBypedalWalker-v2.\u201d The Q-\nnetworks do not need to be very deep for this task.\n9. Use policy gradients to train an agent to play Pong, the famous Atari game (Pong-\nv0 in the OpenAI gym). Beware: an individual observation is insufficient to tell\nthe direction and speed of the ball. One solution is to pass two observations at a", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 1226, "content": "the direction and speed of the ball. One solution is to pass two observations at a\ntime to the neural network policy. To reduce dimensionality and speed up train\u2010\ning, you should definitely preprocess these images (crop, resize, and convert\nthem to black and white), and possibly merge them into a single image (e.g., by\noverlaying them).\n10. If you have about $100 to spare, you can purchase a Raspberry Pi 3 plus some\ncheap robotics components, install TensorFlow on the Pi, and go wild! For an\nexample, check out this fun post by Lukas Biewald, or take a look at GoPiGo or\nBrickPi. Why not try to build a real-life cartpole by training the robot using pol\u2010\nExercises \n| \n469\nDownload from finelybook www.finelybook.com", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 1227, "content": "icy gradients? Or build a robotic spider that learns to walk; give it rewards any\ntime it gets closer to some objective (you will need sensors to measure the dis\u2010\ntance to the objective). The only limit is your imagination.\nSolutions to these exercises are available in Appendix A.\nThank You!\nBefore we close the last chapter of this book, I would like to thank you for reading it\nup to the last paragraph. I truly hope that you had as much pleasure reading this book\nas I had writing it, and that it will be useful for your projects, big or small.\nIf you find errors, please send feedback. More generally, I would love to know what\nyou think, so please don\u2019t hesitate to contact me via O\u2019Reilly, or through the ageron/\nhandson-ml GitHub project.\nGoing forward, my best advice to you is to practice and practice: try going through all\nthe exercises if you have not done so already, play with the Jupyter notebooks, join\nKaggle.com or some other ML community, watch ML courses, read papers, attend", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 1228, "content": "Kaggle.com or some other ML community, watch ML courses, read papers, attend\nconferences, meet experts. You may also want to study some topics that we did not\ncover in this book, including recommender systems, clustering algorithms, anomaly\ndetection algorithms, and genetic algorithms.\nMy greatest hope is that this book will inspire you to build a wonderful ML applica\u2010\ntion that will benefit all of us! What will it be?\nAur\u00e9lien G\u00e9ron, November 26th, 2016\n470 \n| \nChapter 16: Reinforcement Learning\nDownload from finelybook www.finelybook.com", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 1229, "content": "APPENDIX A\nExercise Solutions\nSolutions to the coding exercises are available in the online Jupyter\nnotebooks at https://github.com/ageron/handson-ml.\nChapter 1: The Machine Learning Landscape\n1. Machine Learning is about building systems that can learn from data. Learning\nmeans getting better at some task, given some performance measure.\n2. Machine Learning is great for complex problems for which we have no algorith\u2010\nmic solution, to replace long lists of hand-tuned rules, to build systems that adapt\nto fluctuating environments, and finally to help humans learn (e.g., data mining).\n3. A labeled training set is a training set that contains the desired solution (a.k.a. a\nlabel) for each instance.\n4. The two most common supervised tasks are regression and classification.\n5. Common unsupervised tasks include clustering, visualization, dimensionality\nreduction, and association rule learning.\n6. Reinforcement Learning is likely to perform best if we want a robot to learn to", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 1230, "content": "6. Reinforcement Learning is likely to perform best if we want a robot to learn to\nwalk in various unknown terrains since this is typically the type of problem that\nReinforcement Learning tackles. It might be possible to express the problem as a\nsupervised or semisupervised learning problem, but it would be less natural.\n7. If you don\u2019t know how to define the groups, then you can use a clustering algo\u2010\nrithm (unsupervised learning) to segment your customers into clusters of similar\ncustomers. However, if you know what groups you would like to have, then you\n471\nDownload from finelybook www.finelybook.com", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 1231, "content": "can feed many examples of each group to a classification algorithm (supervised\nlearning), and it will classify all your customers into these groups.\n8. Spam detection is a typical supervised learning problem: the algorithm is fed\nmany emails along with their label (spam or not spam).\n9. An online learning system can learn incrementally, as opposed to a batch learn\u2010\ning system. This makes it capable of adapting rapidly to both changing data and\nautonomous systems, and of training on very large quantities of data.\n10. Out-of-core algorithms can handle vast quantities of data that cannot fit in a\ncomputer\u2019s main memory. An out-of-core learning algorithm chops the data into\nmini-batches and uses online learning techniques to learn from these mini-\nbatches.\n11. An instance-based learning system learns the training data by heart; then, when\ngiven a new instance, it uses a similarity measure to find the most similar learned\ninstances and uses them to make predictions.", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 1232, "content": "instances and uses them to make predictions.\n12. A model has one or more model parameters that determine what it will predict\ngiven a new instance (e.g., the slope of a linear model). A learning algorithm tries\nto find optimal values for these parameters such that the model generalizes well\nto new instances. A hyperparameter is a parameter of the learning algorithm\nitself, not of the model (e.g., the amount of regularization to apply).\n13. Model-based learning algorithms search for an optimal value for the model\nparameters such that the model will generalize well to new instances. We usually\ntrain such systems by minimizing a cost function that measures how bad the sys\u2010\ntem is at making predictions on the training data, plus a penalty for model com\u2010\nplexity if the model is regularized. To make predictions, we feed the new\ninstance\u2019s features into the model\u2019s prediction function, using the parameter val\u2010\nues found by the learning algorithm.", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 1233, "content": "ues found by the learning algorithm.\n14. Some of the main challenges in Machine Learning are the lack of data, poor data\nquality, nonrepresentative data, uninformative features, excessively simple mod\u2010\nels that underfit the training data, and excessively complex models that overfit\nthe data.\n15. If a model performs great on the training data but generalizes poorly to new\ninstances, the model is likely overfitting the training data (or we got extremely\nlucky on the training data). Possible solutions to overfitting are getting more\ndata, simplifying the model (selecting a simpler algorithm, reducing the number\nof parameters or features used, or regularizing the model), or reducing the noise\nin the training data.\n16. A test set is used to estimate the generalization error that a model will make on\nnew instances, before the model is launched in production.\n472 \n| \nAppendix A: Exercise Solutions\nDownload from finelybook www.finelybook.com", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 1234, "content": "1 If you draw a straight line between any two points on the curve, the line never crosses the curve.\n17. A validation set is used to compare models. It makes it possible to select the best\nmodel and tune the hyperparameters.\n18. If you tune hyperparameters using the test set, you risk overfitting the test set,\nand the generalization error you measure will be optimistic (you may launch a\nmodel that performs worse than you expect).\n19. Cross-validation is a technique that makes it possible to compare models (for\nmodel selection and hyperparameter tuning) without the need for a separate vali\u2010\ndation set. This saves precious training data.\nChapter 2: End-to-End Machine Learning Project\nSee the Jupyter notebooks available at https://github.com/ageron/handson-ml.\nChapter 3: Classification\nSee the Jupyter notebooks available at https://github.com/ageron/handson-ml.\nChapter 4: Training Linear Models\n1. If you have a training set with millions of features you can use Stochastic Gradi\u2010", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 1235, "content": "1. If you have a training set with millions of features you can use Stochastic Gradi\u2010\nent Descent or Mini-batch Gradient Descent, and perhaps Batch Gradient\nDescent if the training set fits in memory. But you cannot use the Normal Equa\u2010\ntion because the computational complexity grows quickly (more than quadrati\u2010\ncally) with the number of features.\n2. If the features in your training set have very different scales, the cost function will\nhave the shape of an elongated bowl, so the Gradient Descent algorithms will take\na long time to converge. To solve this you should scale the data before training\nthe model. Note that the Normal Equation will work just fine without scaling.\n3. Gradient Descent cannot get stuck in a local minimum when training a Logistic\nRegression model because the cost function is convex.1\n4. If the optimization problem is convex (such as Linear Regression or Logistic\nRegression), and assuming the learning rate is not too high, then all Gradient", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 1236, "content": "Regression), and assuming the learning rate is not too high, then all Gradient\nDescent algorithms will approach the global optimum and end up producing\nfairly similar models. However, unless you gradually reduce the learning rate,\nStochastic GD and Mini-batch GD will never truly converge; instead, they will\nkeep jumping back and forth around the global optimum. This means that even\nExercise Solutions \n| \n473\nDownload from finelybook www.finelybook.com", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 1237, "content": "2 Moreover, the Normal Equation requires computing the inverse of a matrix, but that matrix is not always\ninvertible. In contrast, the matrix for Ridge Regression is always invertible.\nif you let them run for a very long time, these Gradient Descent algorithms will\nproduce slightly different models.\n5. If the validation error consistently goes up after every epoch, then one possibility\nis that the learning rate is too high and the algorithm is diverging. If the training\nerror also goes up, then this is clearly the problem and you should reduce the\nlearning rate. However, if the training error is not going up, then your model is\noverfitting the training set and you should stop training.\n6. Due to their random nature, neither Stochastic Gradient Descent nor Mini-batch\nGradient Descent is guaranteed to make progress at every single training itera\u2010\ntion. So if you immediately stop training when the validation error goes up, you", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 1238, "content": "tion. So if you immediately stop training when the validation error goes up, you\nmay stop much too early, before the optimum is reached. A better option is to\nsave the model at regular intervals, and when it has not improved for a long time\n(meaning it will probably never beat the record), you can revert to the best saved\nmodel.\n7. Stochastic Gradient Descent has the fastest training iteration since it considers\nonly one training instance at a time, so it is generally the first to reach the vicinity\nof the global optimum (or Mini-batch GD with a very small mini-batch size).\nHowever, only Batch Gradient Descent will actually converge, given enough\ntraining time. As mentioned, Stochastic GD and Mini-batch GD will bounce\naround the optimum, unless you gradually reduce the learning rate.\n8. If the validation error is much higher than the training error, this is likely because\nyour model is overfitting the training set. One way to try to fix this is to reduce", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 1239, "content": "your model is overfitting the training set. One way to try to fix this is to reduce\nthe polynomial degree: a model with fewer degrees of freedom is less likely to\noverfit. Another thing you can try is to regularize the model\u2014for example, by\nadding an \u21132 penalty (Ridge) or an \u21131 penalty (Lasso) to the cost function. This\nwill also reduce the degrees of freedom of the model. Lastly, you can try to\nincrease the size of the training set.\n9. If both the training error and the validation error are almost equal and fairly\nhigh, the model is likely underfitting the training set, which means it has a high\nbias. You should try reducing the regularization hyperparameter \u03b1.\n10. Let\u2019s see:\n\u2022 A model with some regularization typically performs better than a model\nwithout any regularization, so you should generally prefer Ridge Regression\nover plain Linear Regression.2\n\u2022 Lasso Regression uses an \u21131 penalty, which tends to push the weights down to", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 1240, "content": "\u2022 Lasso Regression uses an \u21131 penalty, which tends to push the weights down to\nexactly zero. This leads to sparse models, where all weights are zero except for\n474 \n| \nAppendix A: Exercise Solutions\nDownload from finelybook www.finelybook.com", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 1241, "content": "the most important weights. This is a way to perform feature selection auto\u2010\nmatically, which is good if you suspect that only a few features actually matter.\nWhen you are not sure, you should prefer Ridge Regression.\n\u2022 Elastic Net is generally preferred over Lasso since Lasso may behave erratically\nin some cases (when several features are strongly correlated or when there are\nmore features than training instances). However, it does add an extra hyper\u2010\nparameter to tune. If you just want Lasso without the erratic behavior, you can\njust use Elastic Net with an l1_ratio close to 1.\n11. If you want to classify pictures as outdoor/indoor and daytime/nighttime, since\nthese are not exclusive classes (i.e., all four combinations are possible) you should\ntrain two Logistic Regression classifiers.\n12. See the Jupyter notebooks available at https://github.com/ageron/handson-ml.\nChapter 5: Support Vector Machines\n1. The fundamental idea behind Support Vector Machines is to fit the widest possi\u2010", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 1242, "content": "1. The fundamental idea behind Support Vector Machines is to fit the widest possi\u2010\nble \u201cstreet\u201d between the classes. In other words, the goal is to have the largest pos\u2010\nsible margin between the decision boundary that separates the two classes and\nthe training instances. When performing soft margin classification, the SVM\nsearches for a compromise between perfectly separating the two classes and hav\u2010\ning the widest possible street (i.e., a few instances may end up on the street).\nAnother key idea is to use kernels when training on nonlinear datasets.\n2. After training an SVM, a support vector is any instance located on the \u201cstreet\u201d (see\nthe previous answer), including its border. The decision boundary is entirely\ndetermined by the support vectors. Any instance that is not a support vector (i.e.,\noff the street) has no influence whatsoever; you could remove them, add more\ninstances, or move them around, and as long as they stay off the street they won\u2019t", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 1243, "content": "instances, or move them around, and as long as they stay off the street they won\u2019t\naffect the decision boundary. Computing the predictions only involves the sup\u2010\nport vectors, not the whole training set.\n3. SVMs try to fit the largest possible \u201cstreet\u201d between the classes (see the first\nanswer), so if the training set is not scaled, the SVM will tend to neglect small\nfeatures (see Figure 5-2).\n4. An SVM classifier can output the distance between the test instance and the deci\u2010\nsion boundary, and you can use this as a confidence score. However, this score\ncannot be directly converted into an estimation of the class probability. If you set\nprobability=True when creating an SVM in Scikit-Learn, then after training it\nwill calibrate the probabilities using Logistic Regression on the SVM\u2019s scores\n(trained by an additional five-fold cross-validation on the training data). This\nwill add the predict_proba() and predict_log_proba() methods to the SVM.\nExercise Solutions \n| \n475", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 1244, "content": "will add the predict_proba() and predict_log_proba() methods to the SVM.\nExercise Solutions \n| \n475\nDownload from finelybook www.finelybook.com", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 1245, "content": "3 log2 is the binary log, log2(m) = log(m) / log(2).\n5. This question applies only to linear SVMs since kernelized can only use the dual\nform. The computational complexity of the primal form of the SVM problem is\nproportional to the number of training instances m, while the computational\ncomplexity of the dual form is proportional to a number between m2 and m3. So\nif there are millions of instances, you should definitely use the primal form,\nbecause the dual form will be much too slow.\n6. If an SVM classifier trained with an RBF kernel underfits the training set, there\nmight be too much regularization. To decrease it, you need to increase gamma or C\n(or both).\n7. Let\u2019s call the QP parameters for the hard-margin problem H\u2032, f\u2032, A\u2032 and b\u2032 (see\n\u201cQuadratic Programming\u201d on page 159). The QP parameters for the soft-margin\nproblem have m additional parameters (np = n + 1 + m) and m additional con\u2010\nstraints (nc = 2m). They can be defined like so:", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 1246, "content": "straints (nc = 2m). They can be defined like so:\n\u2022 H is equal to H\u2032, plus m columns of 0s on the right and m rows of 0s at the\nbottom: \ufffd=\n\ufffd\u2032 0 \u22ef\n0 0\n\u22ee\n\u22f1\n\u2022 f is equal to f\u2032 with m additional elements, all equal to the value of the hyper\u2010\nparameter C.\n\u2022 b is equal to b\u2032 with m additional elements, all equal to 0.\n\u2022 A is equal to A\u2032, with an extra m \u00d7 m identity matrix Im appended to the right,\n\u2013 Im just below it, and the rest filled with zeros: \ufffd=\n\ufffd\u2032 \ufffdm\n0 \u2212\ufffdm\nFor the solutions to exercises 8, 9, and 10, please see the Jupyter notebooks available\nat https://github.com/ageron/handson-ml.\nChapter 6: Decision Trees\n1. The depth of a well-balanced binary tree containing m leaves is equal to log2(m)3,\nrounded up. A binary Decision Tree (one that makes only binary decisions, as is\nthe case of all trees in Scikit-Learn) will end up more or less well balanced at the\nend of training, with one leaf per training instance if it is trained without restric\u2010", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 1247, "content": "end of training, with one leaf per training instance if it is trained without restric\u2010\ntions. Thus, if the training set contains one million instances, the Decision Tree\nwill have a depth of log2(106) \u2248 20 (actually a bit more since the tree will generally\nnot be perfectly well balanced).\n476 \n| \nAppendix A: Exercise Solutions\nDownload from finelybook www.finelybook.com", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 1248, "content": "2. A node\u2019s Gini impurity is generally lower than its parent\u2019s. This is ensured by the\nCART training algorithm\u2019s cost function, which splits each node in a way that\nminimizes the weighted sum of its children\u2019s Gini impurities. However, if one\nchild is smaller than the other, it is possible for it to have a higher Gini impurity\nthan its parent, as long as this increase is more than compensated for by a\ndecrease of the other child\u2019s impurity. For example, consider a node containing\nfour instances of class A and 1 of class B. Its Gini impurity is 1 \u22121\n5\n2 \u22124\n5\n2 = 0.32.\nNow suppose the dataset is one-dimensional and the instances are lined up in the\nfollowing order: A, B, A, A, A. You can verify that the algorithm will split this\nnode after the second instance, producing one child node with instances A, B,\nand the other child node with instances A, A, A. The first child node\u2019s Gini\nimpurity is 1 \u22121\n2\n2 \u22121\n2\n2 = 0.5, which is higher than its parent. This is compensated", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 1249, "content": "impurity is 1 \u22121\n2\n2 \u22121\n2\n2 = 0.5, which is higher than its parent. This is compensated\nfor by the fact that the other node is pure, so the overall weighted Gini impurity\nis 2\n5 \u00d7 0.5 + 3\n5 \u00d7 0 = 0.2 , which is lower than the parent\u2019s Gini impurity.\n3. If a Decision Tree is overfitting the training set, it may be a good idea to decrease\nmax_depth, since this will constrain the model, regularizing it.\n4. Decision Trees don\u2019t care whether or not the training data is scaled or centered;\nthat\u2019s one of the nice things about them. So if a Decision Tree underfits the train\u2010\ning set, scaling the input features will just be a waste of time.\n5. The computational complexity of training a Decision Tree is O(n \u00d7 m log(m)). So\nif you multiply the training set size by 10, the training time will be multiplied by\nK = (n \u00d7 10m \u00d7 log(10m)) / (n \u00d7 m \u00d7 log(m)) = 10 \u00d7 log(10m) / log(m). If m =\n106, then K \u2248 11.7, so you can expect the training time to be roughly 11.7 hours.", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 1250, "content": "106, then K \u2248 11.7, so you can expect the training time to be roughly 11.7 hours.\n6. Presorting the training set speeds up training only if the dataset is smaller than a\nfew thousand instances. If it contains 100,000 instances, setting presort=True\nwill considerably slow down training.\nFor the solutions to exercises 7 and 8, please see the Jupyter notebooks available at\nhttps://github.com/ageron/handson-ml.\nChapter 7: Ensemble Learning and Random Forests\n1. If you have trained five different models and they all achieve 95% precision, you\ncan try combining them into a voting ensemble, which will often give you even\nbetter results. It works better if the models are very different (e.g., an SVM classi\u2010\nfier, a Decision Tree classifier, a Logistic Regression classifier, and so on). It is\neven better if they are trained on different training instances (that\u2019s the whole\npoint of bagging and pasting ensembles), but if not it will still work as long as the\nmodels are very different.", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 1251, "content": "models are very different.\nExercise Solutions \n| \n477\nDownload from finelybook www.finelybook.com", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 1252, "content": "2. A hard voting classifier just counts the votes of each classifier in the ensemble\nand picks the class that gets the most votes. A soft voting classifier computes the\naverage estimated class probability for each class and picks the class with the\nhighest probability. This gives high-confidence votes more weight and often per\u2010\nforms better, but it works only if every classifier is able to estimate class probabil\u2010\nities \n(e.g., \nfor \nthe \nSVM \nclassifiers \nin \nScikit-Learn \nyou \nmust \nset\nprobability=True).\n3. It is quite possible to speed up training of a bagging ensemble by distributing it\nacross multiple servers, since each predictor in the ensemble is independent of\nthe others. The same goes for pasting ensembles and Random Forests, for the\nsame reason. However, each predictor in a boosting ensemble is built based on\nthe previous predictor, so training is necessarily sequential, and you will not gain\nanything by distributing training across multiple servers. Regarding stacking", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 1253, "content": "anything by distributing training across multiple servers. Regarding stacking\nensembles, all the predictors in a given layer are independent of each other, so\nthey can be trained in parallel on multiple servers. However, the predictors in one\nlayer can only be trained after the predictors in the previous layer have all been\ntrained.\n4. With out-of-bag evaluation, each predictor in a bagging ensemble is evaluated\nusing instances that it was not trained on (they were held out). This makes it pos\u2010\nsible to have a fairly unbiased evaluation of the ensemble without the need for an\nadditional validation set. Thus, you have more instances available for training,\nand your ensemble can perform slightly better.\n5. When you are growing a tree in a Random Forest, only a random subset of the\nfeatures is considered for splitting at each node. This is true as well for Extra-\nTrees, but they go one step further: rather than searching for the best possible", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 1254, "content": "Trees, but they go one step further: rather than searching for the best possible\nthresholds, like regular Decision Trees do, they use random thresholds for each\nfeature. This extra randomness acts like a form of regularization: if a Random\nForest overfits the training data, Extra-Trees might perform better. Moreover,\nsince Extra-Trees don\u2019t search for the best possible thresholds, they are much\nfaster to train than Random Forests. However, they are neither faster nor slower\nthan Random Forests when making predictions.\n6. If your AdaBoost ensemble underfits the training data, you can try increasing the\nnumber of estimators or reducing the regularization hyperparameters of the base\nestimator. You may also try slightly increasing the learning rate.\n7. If your Gradient Boosting ensemble overfits the training set, you should try\ndecreasing the learning rate. You could also use early stopping to find the right\nnumber of predictors (you probably have too many).", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 1255, "content": "number of predictors (you probably have too many).\nFor the solutions to exercises 8 and 9, please see the Jupyter notebooks available at\nhttps://github.com/ageron/handson-ml.\n478 \n| \nAppendix A: Exercise Solutions\nDownload from finelybook www.finelybook.com", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 1256, "content": "Chapter 8: Dimensionality Reduction\n1. Motivations and drawbacks:\n\u2022 The main motivations for dimensionality reduction are:\n\u2014 To speed up a subsequent training algorithm (in some cases it may even\nremove noise and redundant features, making the training algorithm per\u2010\nform better).\n\u2014 To visualize the data and gain insights on the most important features.\n\u2014 Simply to save space (compression).\n\u2022 The main drawbacks are:\n\u2014 Some information is lost, possibly degrading the performance of subse\u2010\nquent training algorithms.\n\u2014 It can be computationally intensive.\n\u2014 It adds some complexity to your Machine Learning pipelines.\n\u2014 Transformed features are often hard to interpret.\n2. The curse of dimensionality refers to the fact that many problems that do not\nexist in low-dimensional space arise in high-dimensional space. In Machine\nLearning, one common manifestation is the fact that randomly sampled high-\ndimensional vectors are generally very sparse, increasing the risk of overfitting", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 1257, "content": "dimensional vectors are generally very sparse, increasing the risk of overfitting\nand making it very difficult to identify patterns in the data without having plenty\nof training data.\n3. Once a dataset\u2019s dimensionality has been reduced using one of the algorithms we\ndiscussed, it is almost always impossible to perfectly reverse the operation,\nbecause some information gets lost during dimensionality reduction. Moreover,\nwhile some algorithms (such as PCA) have a simple reverse transformation pro\u2010\ncedure that can reconstruct a dataset relatively similar to the original, other algo\u2010\nrithms (such as T-SNE) do not.\n4. PCA can be used to significantly reduce the dimensionality of most datasets, even\nif they are highly nonlinear, because it can at least get rid of useless dimensions.\nHowever, if there are no useless dimensions\u2014for example, the Swiss roll\u2014then\nreducing dimensionality with PCA will lose too much information. You want to\nunroll the Swiss roll, not squash it.", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 1258, "content": "unroll the Swiss roll, not squash it.\n5. That\u2019s a trick question: it depends on the dataset. Let\u2019s look at two extreme exam\u2010\nples. First, suppose the dataset is composed of points that are almost perfectly\naligned. In this case, PCA can reduce the dataset down to just one dimension\nwhile still preserving 95% of the variance. Now imagine that the dataset is com\u2010\nposed of perfectly random points, scattered all around the 1,000 dimensions. In\nExercise Solutions \n| \n479\nDownload from finelybook www.finelybook.com", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 1259, "content": "this case all 1,000 dimensions are required to preserve 95% of the variance. So the\nanswer is, it depends on the dataset, and it could be any number between 1 and\n1,000. Plotting the explained variance as a function of the number of dimensions\nis one way to get a rough idea of the dataset\u2019s intrinsic dimensionality.\n6. Regular PCA is the default, but it works only if the dataset fits in memory. Incre\u2010\nmental PCA is useful for large datasets that don\u2019t fit in memory, but it is slower\nthan regular PCA, so if the dataset fits in memory you should prefer regular\nPCA. Incremental PCA is also useful for online tasks, when you need to apply\nPCA on the fly, every time a new instance arrives. Randomized PCA is useful\nwhen you want to considerably reduce dimensionality and the dataset fits in\nmemory; in this case, it is much faster than regular PCA. Finally, Kernel PCA is\nuseful for nonlinear datasets.\n7. Intuitively, a dimensionality reduction algorithm performs well if it eliminates a", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 1260, "content": "7. Intuitively, a dimensionality reduction algorithm performs well if it eliminates a\nlot of dimensions from the dataset without losing too much information. One\nway to measure this is to apply the reverse transformation and measure the\nreconstruction error. However, not all dimensionality reduction algorithms pro\u2010\nvide a reverse transformation. Alternatively, if you are using dimensionality\nreduction as a preprocessing step before another Machine Learning algorithm\n(e.g., a Random Forest classifier), then you can simply measure the performance\nof that second algorithm; if dimensionality reduction did not lose too much\ninformation, then the algorithm should perform just as well as when using the\noriginal dataset.\n8. It can absolutely make sense to chain two different dimensionality reduction\nalgorithms. A common example is using PCA to quickly get rid of a large num\u2010\nber of useless dimensions, then applying another much slower dimensionality", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 1261, "content": "ber of useless dimensions, then applying another much slower dimensionality\nreduction algorithm, such as LLE. This two-step approach will likely yield the\nsame performance as using LLE only, but in a fraction of the time.\nFor the solutions to exercises 9 and 10, please see the Jupyter notebooks available at\nhttps://github.com/ageron/handson-ml.\nChapter 9: Up and Running with TensorFlow\n1. Main benefits and drawbacks of creating a computation graph rather than\ndirectly executing the computations:\n\u2022 Main benefits:\n\u2014 TensorFlow can automatically compute the gradients for you (using\nreverse-mode autodiff).\n\u2014 TensorFlow can take care of running the operations in parallel in different\nthreads.\n480 \n| \nAppendix A: Exercise Solutions\nDownload from finelybook www.finelybook.com", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 1262, "content": "\u2014 It makes it easier to run the same model across different devices.\n\u2014 It simplifies introspection\u2014for example, to view the model in TensorBoard.\n\u2022 Main drawbacks:\n\u2014 It makes the learning curve steeper.\n\u2014 It makes step-by-step debugging harder.\n2. Yes, the statement a_val = a.eval(session=sess) is indeed equivalent to a_val\n= sess.run(a).\n3. No, the statement a_val, b_val = a.eval(session=sess), b.eval(ses\nsion=sess) is not equivalent to a_val, b_val = sess.run([a, b]). Indeed, the\nfirst statement runs the graph twice (once to compute a, once to compute b),\nwhile the second statement runs the graph only once. If any of these operations\n(or the ops they depend on) have side effects (e.g., a variable is modified, an item\nis inserted in a queue, or a reader reads a file), then the effects will be different. If\nthey don\u2019t have side effects, both statements will return the same result, but the\nsecond statement will be faster than the first.", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 1263, "content": "second statement will be faster than the first.\n4. No, you cannot run two graphs in the same session. You would have to merge the\ngraphs into a single graph first.\n5. In local TensorFlow, sessions manage variable values, so if you create a graph g\ncontaining a variable w, then start two threads and open a local session in each\nthread, both using the same graph g, then each session will have its own copy of\nthe variable w. However, in distributed TensorFlow, variable values are stored in\ncontainers managed by the cluster, so if both sessions connect to the same cluster\nand use the same container, then they will share the same variable value for w.\n6. A variable is initialized when you call its initializer, and it is destroyed when the\nsession ends. In distributed TensorFlow, variables live in containers on the clus\u2010\nter, so closing a session will not destroy the variable. To destroy a variable, you\nneed to clear its container.", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 1264, "content": "need to clear its container.\n7. Variables and placeholders are extremely different, but beginners often confuse\nthem:\n\u2022 A variable is an operation that holds a value. If you run the variable, it returns\nthat value. Before you can run it, you need to initialize it. You can change the\nvariable\u2019s value (for example, by using an assignment operation). It is stateful:\nthe variable keeps the same value upon successive runs of the graph. It is typi\u2010\ncally used to hold model parameters but also for other purposes (e.g., to count\nthe global training step).\n\u2022 Placeholders technically don\u2019t do much: they just hold information about the\ntype and shape of the tensor they represent, but they have no value. In fact, if\nExercise Solutions \n| \n481\nDownload from finelybook www.finelybook.com", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 1265, "content": "you try to evaluate an operation that depends on a placeholder, you must feed\nTensorFlow the value of the placeholder (using the feed_dict argument) or\nelse you will get an exception. Placeholders are typically used to feed training\nor test data to TensorFlow during the execution phase. They are also useful to\npass a value to an assignment node, to change the value of a variable (e.g.,\nmodel weights).\n8. If you run the graph to evaluate an operation that depends on a placeholder but\nyou don\u2019t feed its value, you get an exception. If the operation does not depend\non the placeholder, then no exception is raised.\n9. When you run a graph, you can feed the output value of any operation, not just\nthe value of placeholders. In practice, however, this is rather rare (it can be useful,\nfor example, when you are caching the output of frozen layers; see Chapter 11).\n10. You can specify a variable\u2019s initial value when constructing the graph, and it will", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 1266, "content": "10. You can specify a variable\u2019s initial value when constructing the graph, and it will\nbe initialized later when you run the variable\u2019s initializer during the execution\nphase. If you want to change that variable\u2019s value to anything you want during the\nexecution phase, then the simplest option is to create an assignment node (dur\u2010\ning the graph construction phase) using the tf.assign() function, passing the\nvariable and a placeholder as parameters. During the execution phase, you can\nrun the assignment operation and feed the variable\u2019s new value using the place\u2010\nholder.\nimport tensorflow as tf\nx = tf.Variable(tf.random_uniform(shape=(), minval=0.0, maxval=1.0))\nx_new_val = tf.placeholder(shape=(), dtype=tf.float32)\nx_assign = tf.assign(x, x_new_val)\nwith tf.Session():\n x.initializer.run() # random number is sampled *now*\n print(x.eval()) # 0.646157 (some random number)\n x_assign.eval(feed_dict={x_new_val: 5.0})\n print(x.eval()) # 5.0", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 1267, "content": "x_assign.eval(feed_dict={x_new_val: 5.0})\n print(x.eval()) # 5.0\n11. Reverse-mode autodiff (implemented by TensorFlow) needs to traverse the graph\nonly twice in order to compute the gradients of the cost function with regards to\nany number of variables. On the other hand, forward-mode autodiff would need\nto run once for each variable (so 10 times if we want the gradients with regards to\n10 different variables). As for symbolic differentiation, it would build a different\ngraph to compute the gradients, so it would not traverse the original graph at all\n(except when building the new gradients graph). A highly optimized symbolic\ndifferentiation system could potentially run the new gradients graph only once to\ncompute the gradients with regards to all variables, but that new graph may be\nhorribly complex and inefficient compared to the original graph.\n482 \n| \nAppendix A: Exercise Solutions\nDownload from finelybook www.finelybook.com", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 1268, "content": "12. See the Jupyter notebooks available at https://github.com/ageron/handson-ml.\nChapter 10: Introduction to Artificial Neural Networks\n1. Here is a neural network based on the original artificial neurons that computes A\n\u2295 B (where \u2295 represents the exclusive OR), using the fact that A \u2295 B = (A \u2227 \u00ac B)\n\u2228 (\u00ac A \u2227 B). There are other solutions\u2014for example, using the fact that A \u2295 B =\n(A \u2228 B) \u2227 \u00ac(A \u2227 B), or the fact that A \u2295 B = (A \u2228 B) \u2227 (\u00ac A \u2228 \u2227 B), and so on.\n2. A classical Perceptron will converge only if the dataset is linearly separable, and it\nwon\u2019t be able to estimate class probabilities. In contrast, a Logistic Regression\nclassifier will converge to a good solution even if the dataset is not linearly sepa\u2010\nrable, and it will output class probabilities. If you change the Perceptron\u2019s activa\u2010\ntion function to the logistic activation function (or the softmax activation\nfunction if there are multiple neurons), and if you train it using Gradient Descent", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 1269, "content": "function if there are multiple neurons), and if you train it using Gradient Descent\n(or some other optimization algorithm minimizing the cost function, typically\ncross entropy), then it becomes equivalent to a Logistic Regression classifier.\n3. The logistic activation function was a key ingredient in training the first MLPs\nbecause its derivative is always nonzero, so Gradient Descent can always roll\ndown the slope. When the activation function is a step function, Gradient\nDescent cannot move, as there is no slope at all.\n4. The step function, the logistic function, the hyperbolic tangent, the rectified lin\u2010\near unit (see Figure 10-8). See Chapter 11 for other examples, such as ELU and\nvariants of the ReLU.\n5. Considering the MLP described in the question: suppose you have an MLP com\u2010\nposed of one input layer with 10 passthrough neurons, followed by one hidden\nlayer with 50 artificial neurons, and finally one output layer with 3 artificial neu\u2010", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 1270, "content": "layer with 50 artificial neurons, and finally one output layer with 3 artificial neu\u2010\nrons. All artificial neurons use the ReLU activation function.\nExercise Solutions \n| \n483\nDownload from finelybook www.finelybook.com", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 1271, "content": "4 When the values to predict can vary by many orders of magnitude, then you may want to predict the loga\u2010\nrithm of the target value rather than the target value directly. Simply computing the exponential of the neural\nnetwork\u2019s output will give you the estimated value (since exp(log v) = v).\n\u2022 The shape of the input matrix X is m \u00d7 10, where m represents the training\nbatch size.\n\u2022 The shape of the hidden layer\u2019s weight vector Wh is 10 \u00d7 50 and the length of\nits bias vector bh is 50.\n\u2022 The shape of the output layer\u2019s weight vector Wo is 50 \u00d7 3, and the length of its\nbias vector bo is 3.\n\u2022 The shape of the network\u2019s output matrix Y is m \u00d7 3.\n\u2022 Y = (X \u00b7 Wh + bh) \u00b7 Wo + bo. Note that when you are adding a bias vector to a\nmatrix, it is added to every single row in the matrix, which is called broadcast\u2010\ning.\n6. To classify email into spam or ham, you just need one neuron in the output layer\nof a neural network\u2014for example, indicating the probability that the email is", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 1272, "content": "of a neural network\u2014for example, indicating the probability that the email is\nspam. You would typically use the logistic activation function in the output layer\nwhen estimating a probability. If instead you want to tackle MNIST, you need 10\nneurons in the output layer, and you must replace the logistic function with the\nsoftmax activation function, which can handle multiple classes, outputting one\nprobability per class. Now, if you want your neural network to predict housing\nprices like in Chapter 2, then you need one output neuron, using no activation\nfunction at all in the output layer.4\n7. Backpropagation is a technique used to train artificial neural networks. It first\ncomputes the gradients of the cost function with regards to every model parame\u2010\nter (all the weights and biases), and then it performs a Gradient Descent step\nusing these gradients. This backpropagation step is typically performed thou\u2010\nsands or millions of times, using many training batches, until the model parame\u2010", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 1273, "content": "sands or millions of times, using many training batches, until the model parame\u2010\nters converge to values that (hopefully) minimize the cost function. To compute\nthe gradients, backpropagation uses reverse-mode autodiff (although it wasn\u2019t\ncalled that when backpropagation was invented, and it has been reinvented sev\u2010\neral times). Reverse-mode autodiff performs a forward pass through a computa\u2010\ntion graph, computing every node\u2019s value for the current training batch, and then\nit performs a reverse pass, computing all the gradients at once (see Appendix D\nfor more details). So what\u2019s the difference? Well, backpropagation refers to the\nwhole process of training an artificial neural network using multiple backpropa\u2010\ngation steps, each of which computes gradients and uses them to perform a Gra\u2010\ndient Descent step. In contrast, reverse-mode autodiff is a simply a technique to\ncompute gradients efficiently, and it happens to be used by backpropagation.\n484 \n| \nAppendix A: Exercise Solutions", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 1274, "content": "484 \n| \nAppendix A: Exercise Solutions\nDownload from finelybook www.finelybook.com", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 1275, "content": "5 In Chapter 11 we discuss many techniques that introduce additional hyperparameters: type of weight initiali\u2010\nzation, activation function hyperparameters (e.g., amount of leak in leaky ReLU), Gradient Clipping thres\u2010\nhold, type of optimizer and its hyperparameters (e.g., the momentum hyperparameter when using a\nMomentumOptimizer), type of regularization for each layer, and the regularization hyperparameters (e.g., drop\u2010\nout rate when using dropout) and so on.\n8. Here is a list of all the hyperparameters you can tweak in a basic MLP: the num\u2010\nber of hidden layers, the number of neurons in each hidden layer, and the activa\u2010\ntion function used in each hidden layer and in the output layer.5 In general, the\nReLU activation function (or one of its variants; see Chapter 11) is a good default\nfor the hidden layers. For the output layer, in general you will want the logistic\nactivation function for binary classification, the softmax activation function for", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 1276, "content": "activation function for binary classification, the softmax activation function for\nmulticlass classification, or no activation function for regression.\nIf the MLP overfits the training data, you can try reducing the number of hidden\nlayers and reducing the number of neurons per hidden layer.\n9. See the Jupyter notebooks available at https://github.com/ageron/handson-ml.\nChapter 11: Training Deep Neural Nets\n1. No, all weights should be sampled independently; they should not all have the\nsame initial value. One important goal of sampling weights randomly is to break\nsymmetries: if all the weights have the same initial value, even if that value is not\nzero, then symmetry is not broken (i.e., all neurons in a given layer are equiva\u2010\nlent), and backpropagation will be unable to break it. Concretely, this means that\nall the neurons in any given layer will always have the same weights. It\u2019s like hav\u2010\ning just one neuron per layer, and much slower. It is virtually impossible for such", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 1277, "content": "ing just one neuron per layer, and much slower. It is virtually impossible for such\na configuration to converge to a good solution.\n2. It is perfectly fine to initialize the bias terms to zero. Some people like to initialize\nthem just like weights, and that\u2019s okay too; it does not make much difference.\n3. A few advantages of the ELU function over the ReLU function are:\n\u2022 It can take on negative values, so the average output of the neurons in any\ngiven layer is typically closer to 0 than when using the ReLU activation func\u2010\ntion (which never outputs negative values). This helps alleviate the vanishing\ngradients problem.\n\u2022 It always has a nonzero derivative, which avoids the dying units issue that can\naffect ReLU units.\nExercise Solutions \n| \n485\nDownload from finelybook www.finelybook.com", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 1278, "content": "\u2022 It is smooth everywhere, whereas the ReLU\u2019s slope abruptly jumps from 0 to 1\nat z = 0. Such an abrupt change can slow down Gradient Descent because it\nwill bounce around z = 0.\n4. The ELU activation function is a good default. If you need the neural network to\nbe as fast as possible, you can use one of the leaky ReLU variants instead (e.g., a\nsimple leaky ReLU using the default hyperparameter value). The simplicity of the\nReLU activation function makes it many people\u2019s preferred option, despite the\nfact that they are generally outperformed by the ELU and leaky ReLU. However,\nthe ReLU activation function\u2019s capability of outputting precisely zero can be use\u2010\nful in some cases (e.g., see Chapter 15). The hyperbolic tangent (tanh) can be use\u2010\nful in the output layer if you need to output a number between \u20131 and 1, but\nnowadays it is not used much in hidden layers. The logistic activation function is\nalso useful in the output layer when you need to estimate a probability (e.g., for", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 1279, "content": "also useful in the output layer when you need to estimate a probability (e.g., for\nbinary classification), but it is also rarely used in hidden layers (there are excep\u2010\ntions\u2014for example, for the coding layer of variational autoencoders; see Chap\u2010\nter 15). Finally, the softmax activation function is useful in the output layer to\noutput probabilities for mutually exclusive classes, but other than that it is rarely\n(if ever) used in hidden layers.\n5. If you set the momentum hyperparameter too close to 1 (e.g., 0.99999) when using\na MomentumOptimizer, then the algorithm will likely pick up a lot of speed, hope\u2010\nfully roughly toward the global minimum, but then it will shoot right past the\nminimum, due to its momentum. Then it will slow down and come back, accel\u2010\nerate again, overshoot again, and so on. It may oscillate this way many times\nbefore converging, so overall it will take much longer to converge than with a\nsmaller momentum value.", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 1280, "content": "smaller momentum value.\n6. One way to produce a sparse model (i.e., with most weights equal to zero) is to\ntrain the model normally, then zero out tiny weights. For more sparsity, you can\napply \u21131 regularization during training, which pushes the optimizer toward spar\u2010\nsity. A third option is to combine \u21131 regularization with dual averaging, using\nTensorFlow\u2019s FTRLOptimizer class.\n7. Yes, dropout does slow down training, in general roughly by a factor of two.\nHowever, it has no impact on inference since it is only turned on during training.\nFor the solutions to exercises 8, 9, and 10, please see the Jupyter notebooks available\nat https://github.com/ageron/handson-ml.\n486 \n| \nAppendix A: Exercise Solutions\nDownload from finelybook www.finelybook.com", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 1281, "content": "Chapter 12: Distributing TensorFlow Across Devices and\nServers\n1. When a TensorFlow process starts, it grabs all the available memory on all GPU\ndevices that are visible to it, so if you get a CUDA_ERROR_OUT_OF_MEMORY when\nstarting your TensorFlow program, it probably means that other processes are\nrunning that have already grabbed all the memory on at least one visible GPU\ndevice (most likely it is another TensorFlow process). To fix this problem, a triv\u2010\nial solution is to stop the other processes and try again. However, if you need all\nprocesses to run simultaneously, a simple option is to dedicate different devices\nto each process, by setting the CUDA_VISIBLE_DEVICES environment variable\nappropriately for each device. Another option is to configure TensorFlow to grab\nonly part of the GPU memory, instead of all of it, by creating a ConfigProto, set\u2010\nting its gpu_options.per_process_gpu_memory_fraction to the proportion of", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 1282, "content": "ting its gpu_options.per_process_gpu_memory_fraction to the proportion of\nthe total memory that it should grab (e.g., 0.4), and using this ConfigProto when\nopening a session. The last option is to tell TensorFlow to grab memory only\nwhen it needs it by setting the gpu_options.allow_growth to True. However,\nthis last option is usually not recommended because any memory that Tensor\u2010\nFlow grabs is never released, and it is harder to guarantee a repeatable behavior\n(there may be race conditions depending on which processes start first, how\nmuch memory they need during training, and so on).\n2. By pinning an operation on a device, you are telling TensorFlow that this is\nwhere you would like this operation to be placed. However, some constraints may\nprevent TensorFlow from honoring your request. For example, the operation\nmay have no implementation (called a kernel) for that particular type of device.\nIn this case, TensorFlow will raise an exception by default, but you can configure", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 1283, "content": "In this case, TensorFlow will raise an exception by default, but you can configure\nit to fall back to the CPU instead (this is called soft placement). Another example\nis an operation that can modify a variable; this operation and the variable need to\nbe collocated. So the difference between pinning an operation and placing an\noperation is that pinning is what you ask TensorFlow (\u201cPlease place this opera\u2010\ntion on GPU #1\u201d) while placement is what TensorFlow actually ends up doing\n(\u201cSorry, falling back to the CPU\u201d).\n3. If you are running on a GPU-enabled TensorFlow installation, and you just use\nthe default placement, then if all operations have a GPU kernel (i.e., a GPU\nimplementation), yes, they will all be placed on the first GPU. However, if one or\nmore operations do not have a GPU kernel, then by default TensorFlow will raise\nan exception. If you configure TensorFlow to fall back to the CPU instead (soft\nplacement), then all operations will be placed on the first GPU except the ones", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 1284, "content": "placement), then all operations will be placed on the first GPU except the ones\nwithout a GPU kernel and all the operations that must be collocated with them\n(see the answer to the previous exercise).\nExercise Solutions \n| \n487\nDownload from finelybook www.finelybook.com", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 1285, "content": "4. Yes, if you pin a variable to \"/gpu:0\", it can be used by operations placed\non /gpu:1. TensorFlow will automatically take care of adding the appropriate\noperations to transfer the variable\u2019s value across devices. The same goes for devi\u2010\nces located on different servers (as long as they are part of the same cluster).\n5. Yes, two operations placed on the same device can run in parallel: TensorFlow\nautomatically takes care of running operations in parallel (on different CPU\ncores or different GPU threads), as long as no operation depends on another\noperation\u2019s output. Moreover, you can start multiple sessions in parallel threads\n(or processes), and evaluate operations in each thread. Since sessions are inde\u2010\npendent, TensorFlow will be able to evaluate any operation from one session in\nparallel with any operation from another session.\n6. Control dependencies are used when you want to postpone the evaluation of an", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 1286, "content": "6. Control dependencies are used when you want to postpone the evaluation of an\noperation X until after some other operations are run, even though these opera\u2010\ntions are not required to compute X. This is useful in particular when X would\noccupy a lot of memory and you only need it later in the computation graph, or if\nX uses up a lot of I/O (for example, it requires a large variable value located on a\ndifferent device or server) and you don\u2019t want it to run at the same time as other\nI/O-hungry operations, to avoid saturating the bandwidth.\n7. You\u2019re in luck! In distributed TensorFlow, the variable values live in containers\nmanaged by the cluster, so even if you close the session and exit the client pro\u2010\ngram, the model parameters are still alive and well on the cluster. You simply\nneed to open a new session to the cluster and save the model (make sure you\ndon\u2019t call the variable initializers or restore a previous model, as this would\ndestroy your precious new model!).", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 1287, "content": "destroy your precious new model!).\nFor the solutions to exercises 8, 9, and 10, please see the Jupyter notebooks available\nat https://github.com/ageron/handson-ml.\nChapter 13: Convolutional Neural Networks\n1. These are the main advantages of a CNN over a fully connected DNN for image\nclassification:\n\u2022 Because consecutive layers are only partially connected and because it heavily\nreuses its weights, a CNN has many fewer parameters than a fully connected\nDNN, which makes it much faster to train, reduces the risk of overfitting, and\nrequires much less training data.\n\u2022 When a CNN has learned a kernel that can detect a particular feature, it can\ndetect that feature anywhere on the image. In contrast, when a DNN learns a\nfeature in one location, it can detect it only in that particular location. Since\nimages typically have very repetitive features, CNNs are able to generalize\n488 \n| \nAppendix A: Exercise Solutions\nDownload from finelybook www.finelybook.com", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 1288, "content": "much better than DNNs for image processing tasks such as classification, using\nfewer training examples.\n\u2022 Finally, a DNN has no prior knowledge of how pixels are organized; it does not\nknow that nearby pixels are close. A CNN\u2019s architecture embeds this prior\nknowledge. Lower layers typically identify features in small areas of the images,\nwhile higher layers combine the lower-level features into larger features. This\nworks well with most natural images, giving CNNs a decisive head start com\u2010\npared to DNNs.\n2. Let\u2019s compute how many parameters the CNN has. Since its first convolutional\nlayer has 3 \u00d7 3 kernels, and the input has three channels (red, green, and blue),\nthen each feature map has 3 \u00d7 3 \u00d7 3 weights, plus a bias term. That\u2019s 28 parame\u2010\nters per feature map. Since this first convolutional layer has 100 feature maps, it\nhas a total of 2,800 parameters. The second convolutional layer has 3 \u00d7 3 kernels,", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 1289, "content": "has a total of 2,800 parameters. The second convolutional layer has 3 \u00d7 3 kernels,\nand its input is the set of 100 feature maps of the previous layer, so each feature\nmap has 3 \u00d7 3 \u00d7 100 = 900 weights, plus a bias term. Since it has 200 feature\nmaps, this layer has 901 \u00d7 200 = 180,200 parameters. Finally, the third and last\nconvolutional layer also has 3 \u00d7 3 kernels, and its input is the set of 200 feature\nmaps of the previous layers, so each feature map has 3 \u00d7 3 \u00d7 200 = 1,800 weights,\nplus a bias term. Since it has 400 feature maps, this layer has a total of 1,801 \u00d7 400\n= 720,400 parameters. All in all, the CNN has 2,800 + 180,200 + 720,400 =\n903,400 parameters.\nNow let\u2019s compute how much RAM this neural network will require (at least)\nwhen making a prediction for a single instance. First let\u2019s compute the feature\nmap size for each layer. Since we are using a stride of 2 and SAME padding, the\nhorizontal and vertical size of the feature maps are divided by 2 at each layer", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 1290, "content": "horizontal and vertical size of the feature maps are divided by 2 at each layer\n(rounding up if necessary), so as the input channels are 200 \u00d7 300 pixels, the first\nlayer\u2019s feature maps are 100 \u00d7 150, the second layer\u2019s feature maps are 50 \u00d7 75,\nand the third layer\u2019s feature maps are 25 \u00d7 38. Since 32 bits is 4 bytes and the first\nconvolutional layer has 100 feature maps, this first layer takes up 4 x 100 \u00d7 150 \u00d7\n100 = 6 million bytes (about 5.7 MB, considering that 1 MB = 1,024 KB and 1 KB\n= 1,024 bytes). The second layer takes up 4 \u00d7 50 \u00d7 75 \u00d7 200 = 3 million bytes\n(about 2.9 MB). Finally, the third layer takes up 4 \u00d7 25 \u00d7 38 \u00d7 400 = 1,520,000\nbytes (about 1.4 MB). However, once a layer has been computed, the memory\noccupied by the previous layer can be released, so if everything is well optimized,\nonly 6 + 9 = 15 million bytes (about 14.3 MB) of RAM will be required (when the\nsecond layer has just been computed, but the memory occupied by the first layer", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 1291, "content": "second layer has just been computed, but the memory occupied by the first layer\nis not released yet). But wait, you also need to add the memory occupied by the\nCNN\u2019s parameters. We computed earlier that it has 903,400 parameters, each\nusing up 4 bytes, so this adds 3,613,600 bytes (about 3.4 MB). The total RAM\nrequired is (at least) 18,613,600 bytes (about 17.8 MB).\nExercise Solutions \n| \n489\nDownload from finelybook www.finelybook.com", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 1292, "content": "Lastly, let\u2019s compute the minimum amount of RAM required when training the\nCNN on a mini-batch of 50 images. During training TensorFlow uses backpropa\u2010\ngation, which requires keeping all values computed during the forward pass until\nthe reverse pass begins. So we must compute the total RAM required by all layers\nfor a single instance and multiply that by 50! At that point let\u2019s start counting in\nmegabytes rather than bytes. We computed before that the three layers require\nrespectively 5.7, 2.9, and 1.4 MB for each instance. That\u2019s a total of 10.0 MB per\ninstance. So for 50 instances the total RAM is 500 MB. Add to that the RAM\nrequired by the input images, which is 50 \u00d7 4 \u00d7 200 \u00d7 300 \u00d7 3 = 36 million bytes\n(about 34.3 MB), plus the RAM required for the model parameters, which is\nabout 3.4 MB (computed earlier), plus some RAM for the gradients (we will\nneglect them since they can be released gradually as backpropagation goes down", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 1293, "content": "neglect them since they can be released gradually as backpropagation goes down\nthe layers during the reverse pass). We are up to a total of roughly 500.0 + 34.3 +\n3.4 = 537.7 MB. And that\u2019s really an optimistic bare minimum.\n3. If your GPU runs out of memory while training a CNN, here are five things you\ncould try to solve the problem (other than purchasing a GPU with more RAM):\n\u2022 Reduce the mini-batch size.\n\u2022 Reduce dimensionality using a larger stride in one or more layers.\n\u2022 Remove one or more layers.\n\u2022 Use 16-bit floats instead of 32-bit floats.\n\u2022 Distribute the CNN across multiple devices.\n4. A max pooling layer has no parameters at all, whereas a convolutional layer has\nquite a few (see the previous questions).\n5. A local response normalization layer makes the neurons that most strongly acti\u2010\nvate inhibit neurons at the same location but in neighboring feature maps, which\nencourages different feature maps to specialize and pushes them apart, forcing", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 1294, "content": "encourages different feature maps to specialize and pushes them apart, forcing\nthem to explore a wider range of features. It is typically used in the lower layers to\nhave a larger pool of low-level features that the upper layers can build upon.\n6. The main innovations in AlexNet compared to LeNet-5 are (1) it is much larger\nand deeper, and (2) it stacks convolutional layers directly on top of each other,\ninstead of stacking a pooling layer on top of each convolutional layer. The main\ninnovation in GoogLeNet is the introduction of inception modules, which make it\npossible to have a much deeper net than previous CNN architectures, with fewer\nparameters. Finally, ResNet\u2019s main innovation is the introduction of skip connec\u2010\ntions, which make it possible to go well beyond 100 layers. Arguably, its simplic\u2010\nity and consistency are also rather innovative.\nFor the solutions to exercises 7, 8, 9, and 10, please see the Jupyter notebooks avail\u2010\nable at https://github.com/ageron/handson-ml.\n490", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 1295, "content": "able at https://github.com/ageron/handson-ml.\n490 \n| \nAppendix A: Exercise Solutions\nDownload from finelybook www.finelybook.com", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 1296, "content": "Chapter 14: Recurrent Neural Networks\n1. Here are a few RNN applications:\n\u2022 For a sequence-to-sequence RNN: predicting the weather (or any other time\nseries), machine translation (using an encoder\u2013decoder architecture), video\ncaptioning, speech to text, music generation (or other sequence generation),\nidentifying the chords of a song.\n\u2022 For a sequence-to-vector RNN: classifying music samples by music genre, ana\u2010\nlyzing the sentiment of a book review, predicting what word an aphasic patient\nis thinking of based on readings from brain implants, predicting the probabil\u2010\nity that a user will want to watch a movie based on her watch history (this is\none of many possible implementations of collaborative filtering).\n\u2022 For a vector-to-sequence RNN: image captioning, creating a music playlist\nbased on an embedding of the current artist, generating a melody based on a\nset of parameters, locating pedestrians in a picture (e.g., a video frame from a\nself-driving car\u2019s camera).", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 1297, "content": "self-driving car\u2019s camera).\n2. In general, if you translate a sentence one word at a time, the result will be terri\u2010\nble. For example, the French sentence \u201cJe vous en prie\u201d means \u201cYou are welcome,\u201d\nbut if you translate it one word at a time, you get \u201cI you in pray.\u201d Huh? It is much\nbetter to read the whole sentence first and then translate it. A plain sequence-to-\nsequence RNN would start translating a sentence immediately after reading the\nfirst word, while an encoder\u2013decoder RNN will first read the whole sentence and\nthen translate it. That said, one could imagine a plain sequence-to-sequence\nRNN that would output silence whenever it is unsure about what to say next (just\nlike human translators do when they must translate a live broadcast).\n3. To classify videos based on the visual content, one possible architecture could be\nto take (say) one frame per second, then run each frame through a convolutional\nneural network, feed the output of the CNN to a sequence-to-vector RNN, and", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 1298, "content": "neural network, feed the output of the CNN to a sequence-to-vector RNN, and\nfinally run its output through a softmax layer, giving you all the class probabili\u2010\nties. For training you would just use cross entropy as the cost function. If you\nwanted to use the audio for classification as well, you could convert every second\nof audio to a spectrograph, feed this spectrograph to a CNN, and feed the output\nof this CNN to the RNN (along with the corresponding output of the other\nCNN).\n4. Building an RNN using dynamic_rnn() rather than static_rnn() offers several \nadvantages:\n\u2022 It is based on a while_loop() operation that is able to swap the GPU\u2019s memory\nto the CPU\u2019s memory during backpropagation, avoiding out-of-memory\nerrors.\nExercise Solutions \n| \n491\nDownload from finelybook www.finelybook.com", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 1299, "content": "\u2022 It is arguably easier to use, as it can directly take a single tensor as input and\noutput (covering all time steps), rather than a list of tensors (one per time\nstep). No need to stack, unstack, or transpose.\n\u2022 It generates a smaller graph, easier to visualize in TensorBoard.\n5. To handle variable length input sequences, the simplest option is to set the\nsequence_length parameter when calling the static_rnn() or dynamic_rnn()\nfunctions. Another option is to pad the smaller inputs (e.g., with zeros) to make\nthem the same size as the largest input (this may be faster than the first option if\nthe input sequences all have very similar lengths). To handle variable-length out\u2010\nput sequences, if you know in advance the length of each output sequence, you\ncan use the sequence_length parameter (for example, consider a sequence-to-\nsequence RNN that labels every frame in a video with a violence score: the output\nsequence will be exactly the same length as the input sequence). If you don\u2019t", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 1300, "content": "sequence will be exactly the same length as the input sequence). If you don\u2019t\nknow in advance the length of the output sequence, you can use the padding\ntrick: always output the same size sequence, but ignore any outputs that come\nafter the end-of-sequence token (by ignoring them when computing the cost\nfunction).\n6. To distribute training and execution of a deep RNN across multiple GPUs, a\ncommon technique is simply to place each layer on a different GPU (see Chap\u2010\nter 12).\nFor the solutions to exercises 7, 8, and 9, please see the Jupyter notebooks available at\nhttps://github.com/ageron/handson-ml.\nChapter 15: Autoencoders\n1. Here are some of the main tasks that autoencoders are used for:\n\u2022 Feature extraction\n\u2022 Unsupervised pretraining\n\u2022 Dimensionality reduction\n\u2022 Generative models\n\u2022 Anomaly detection (an autoencoder is generally bad at reconstructing outliers)\n2. If you want to train a classifier and you have plenty of unlabeled training data,", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 1301, "content": "2. If you want to train a classifier and you have plenty of unlabeled training data,\nbut only a few thousand labeled instances, then you could first train a deep\nautoencoder on the full dataset (labeled + unlabeled), then reuse its lower half for\nthe classifier (i.e., reuse the layers up to the codings layer, included) and train the\nclassifier using the labeled data. If you have little labeled data, you probably want\nto freeze the reused layers when training the classifier.\n492 \n| \nAppendix A: Exercise Solutions\nDownload from finelybook www.finelybook.com", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 1302, "content": "3. The fact that an autoencoder perfectly reconstructs its inputs does not necessarily\nmean that it is a good autoencoder; perhaps it is simply an overcomplete autoen\u2010\ncoder that learned to copy its inputs to the codings layer and then to the outputs.\nIn fact, even if the codings layer contained a single neuron, it would be possible\nfor a very deep autoencoder to learn to map each training instance to a different\ncoding (e.g., the first instance could be mapped to 0.001, the second to 0.002, the\nthird to 0.003, and so on), and it could learn \u201cby heart\u201d to reconstruct the right\ntraining instance for each coding. It would perfectly reconstruct its inputs\nwithout really learning any useful pattern in the data. In practice such a mapping\nis unlikely to happen, but it illustrates the fact that perfect reconstructions are not\na guarantee that the autoencoder learned anything useful. However, if it produces\nvery bad reconstructions, then it is almost guaranteed to be a bad autoencoder.", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 1303, "content": "very bad reconstructions, then it is almost guaranteed to be a bad autoencoder.\nTo evaluate the performance of an autoencoder, one option is to measure the\nreconstruction loss (e.g., compute the MSE, the mean square of the outputs\nminus the inputs). Again, a high reconstruction loss is a good sign that the\nautoencoder is bad, but a low reconstruction loss is not a guarantee that it is\ngood. You should also evaluate the autoencoder according to what it will be used\nfor. For example, if you are using it for unsupervised pretraining of a classifier,\nthen you should also evaluate the classifier\u2019s performance.\n4. An undercomplete autoencoder is one whose codings layer is smaller than the\ninput and output layers. If it is larger, then it is an overcomplete autoencoder.\nThe main risk of an excessively undercomplete autoencoder is that it may fail to\nreconstruct the inputs. The main risk of an overcomplete autoencoder is that it", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 1304, "content": "reconstruct the inputs. The main risk of an overcomplete autoencoder is that it\nmay just copy the inputs to the outputs, without learning any useful feature.\n5. To tie the weights of an encoder layer and its corresponding decoder layer, you\nsimply make the decoder weights equal to the transpose of the encoder weights.\nThis reduces the number of parameters in the model by half, often making train\u2010\ning converge faster with less training data, and reducing the risk of overfitting the\ntraining set.\n6. To visualize the features learned by the lower layer of a stacked autoencoder, a\ncommon technique is simply to plot the weights of each neuron, by reshaping\neach weight vector to the size of an input image (e.g., for MNIST, reshaping a\nweight vector of shape [784] to [28, 28]). To visualize the features learned by\nhigher layers, one technique is to display the training instances that most activate\neach neuron.\n7. A generative model is a model capable of randomly generating outputs that", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 1305, "content": "each neuron.\n7. A generative model is a model capable of randomly generating outputs that\nresemble the training instances. For example, once trained successfully on the\nMNIST dataset, a generative model can be used to randomly generate realistic\nimages of digits. The output distribution is typically similar to the training data.\nFor example, since MNIST contains many images of each digit, the generative\nmodel would output roughly the same number of images of each digit. Some\nExercise Solutions \n| \n493\nDownload from finelybook www.finelybook.com", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 1306, "content": "generative models can be parametrized\u2014for example, to generate only some\nkinds of outputs. An example of a generative autoencoder is the variational\nautoencoder.\nFor the solutions to exercises 8, 9, and 10, please see the Jupyter notebooks available\nat https://github.com/ageron/handson-ml.\nChapter 16: Reinforcement Learning\n1. Reinforcement Learning is an area of Machine Learning aimed at creating agents\ncapable of taking actions in an environment in a way that maximizes rewards\nover time. There are many differences between RL and regular supervised and\nunsupervised learning. Here are a few:\n\u2022 In supervised and unsupervised learning, the goal is generally to find patterns\nin the data. In Reinforcement Learning, the goal is to find a good policy.\n\u2022 Unlike in supervised learning, the agent is not explicitly given the \u201cright\u201d\nanswer. It must learn by trial and error.\n\u2022 Unlike in unsupervised learning, there is a form of supervision, through", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 1307, "content": "\u2022 Unlike in unsupervised learning, there is a form of supervision, through\nrewards. We do not tell the agent how to perform the task, but we do tell it\nwhen it is making progress or when it is failing.\n\u2022 A Reinforcement Learning agent needs to find the right balance between\nexploring the environment, looking for new ways of getting rewards, and\nexploiting sources of rewards that it already knows. In contrast, supervised and\nunsupervised learning systems generally don\u2019t need to worry about explora\u2010\ntion; they just feed on the training data they are given.\n\u2022 In supervised and unsupervised learning, training instances are typically inde\u2010\npendent (in fact, they are generally shuffled). In Reinforcement Learning, con\u2010\nsecutive observations are generally not independent. An agent may remain in\nthe same region of the environment for a while before it moves on, so consecu\u2010\ntive observations will be very correlated. In some cases a replay memory is", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 1308, "content": "tive observations will be very correlated. In some cases a replay memory is\nused to ensure that the training algorithm gets fairly independent observa\u2010\ntions.\n2. Here are a few possible applications of Reinforcement Learning, other than those\nmentioned in Chapter 16:\nMusic personalization\nThe environment is a user\u2019s personalized web radio. The agent is the software\ndeciding what song to play next for that user. Its possible actions are to play\nany song in the catalog (it must try to choose a song the user will enjoy) or to\nplay an advertisement (it must try to choose an ad that the user will be inter\u2010\n494 \n| \nAppendix A: Exercise Solutions\nDownload from finelybook www.finelybook.com", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 1309, "content": "ested in). It gets a small reward every time the user listens to a song, a larger\nreward every time the user listens to an ad, a negative reward when the user\nskips a song or an ad, and a very negative reward if the user leaves.\nMarketing\nThe environment is your company\u2019s marketing department. The agent is the\nsoftware that defines which customers a mailing campaign should be sent to,\ngiven their profile and purchase history (for each customer it has two possi\u2010\nble actions: send or don\u2019t send). It gets a negative reward for the cost of the\nmailing campaign, and a positive reward for estimated revenue generated\nfrom this campaign.\nProduct delivery\nLet the agent control a fleet of delivery trucks, deciding what they should\npick up at the depots, where they should go, what they should drop off, and\nso on. They would get positive rewards for each product delivered on time,\nand negative rewards for late deliveries.", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 1310, "content": "and negative rewards for late deliveries.\n3. When estimating the value of an action, Reinforcement Learning algorithms typ\u2010\nically sum all the rewards that this action led to, giving more weight to immediate\nrewards, and less weight to later rewards (considering that an action has more\ninfluence on the near future than on the distant future). To model this, a discount\nrate is typically applied at each time step. For example, with a discount rate of 0.9,\na reward of 100 that is received two time steps later is counted as only 0.92 \u00d7 100\n= 81 when you are estimating the value of the action. You can think of the dis\u2010\ncount rate as a measure of how much the future is valued relative to the present:\nif it is very close to 1, then the future is valued almost as much as the present. If it\nis close to 0, then only immediate rewards matter. Of course, this impacts the\noptimal policy tremendously: if you value the future, you may be willing to put", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 1311, "content": "optimal policy tremendously: if you value the future, you may be willing to put\nup with a lot of immediate pain for the prospect of eventual rewards, while if you\ndon\u2019t value the future, you will just grab any immediate reward you can find,\nnever investing in the future.\n4. To measure the performance of a Reinforcement Learning agent, you can simply\nsum up the rewards it gets. In a simulated environment, you can run many epi\u2010\nsodes and look at the total rewards it gets on average (and possibly look at the\nmin, max, standard deviation, and so on).\n5. The credit assignment problem is the fact that when a Reinforcement Learning\nagent receives a reward, it has no direct way of knowing which of its previous\nactions contributed to this reward. It typically occurs when there is a large delay\nbetween an action and the resulting rewards (e.g., during a game of Atari\u2019s Pong,\nthere may be a few dozen time steps between the moment the agent hits the ball", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 1312, "content": "there may be a few dozen time steps between the moment the agent hits the ball\nand the moment it wins the point). One way to alleviate it is to provide the agent\nwith shorter-term rewards, when possible. This usually requires prior knowledge\nExercise Solutions \n| \n495\nDownload from finelybook www.finelybook.com", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 1313, "content": "about the task. For example, if we want to build an agent that will learn to play\nchess, instead of giving it a reward only when it wins the game, we could give it a\nreward every time it captures one of the opponent\u2019s pieces.\n6. An agent can often remain in the same region of its environment for a while, so\nall of its experiences will be very similar for that period of time. This can intro\u2010\nduce some bias in the learning algorithm. It may tune its policy for this region of\nthe environment, but it will not perform well as soon as it moves out of this\nregion. To solve this problem, you can use a replay memory; instead of using\nonly the most immediate experiences for learning, the agent will learn based on a\nbuffer of its past experiences, recent and not so recent (perhaps this is why we\ndream at night: to replay our experiences of the day and better learn from them?).\n7. An off-policy RL algorithm learns the value of the optimal policy (i.e., the sum of", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 1314, "content": "7. An off-policy RL algorithm learns the value of the optimal policy (i.e., the sum of\ndiscounted rewards that can be expected for each state if the agent acts opti\u2010\nmally), independently of how the agent actually acts. Q-Learning is a good exam\u2010\nple of such an algorithm. In contrast, an on-policy algorithm learns the value of\nthe policy that the agent actually executes, including both exploration and exploi\u2010\ntation.\nFor the solutions to exercises 8, 9, and 10, please see the Jupyter notebooks available\nat https://github.com/ageron/handson-ml.\n496 \n| \nAppendix A: Exercise Solutions\nDownload from finelybook www.finelybook.com", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 1315, "content": "APPENDIX B\nMachine Learning Project Checklist\nThis checklist can guide you through your Machine Learning projects. There are\neight main steps:\n1. Frame the problem and look at the big picture.\n2. Get the data.\n3. Explore the data to gain insights.\n4. Prepare the data to better expose the underlying data patterns to Machine Learn\u2010\ning algorithms.\n5. Explore many different models and short-list the best ones.\n6. Fine-tune your models and combine them into a great solution.\n7. Present your solution.\n8. Launch, monitor, and maintain your system.\nObviously, you should feel free to adapt this checklist to your needs.\nFrame the Problem and Look at the Big Picture\n1. Define the objective in business terms.\n2. How will your solution be used?\n3. What are the current solutions/workarounds (if any)?\n4. How should you frame this problem (supervised/unsupervised, online/offline,\netc.)?\n5. How should performance be measured?\n6. Is the performance measure aligned with the business objective?\n497", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 1316, "content": "6. Is the performance measure aligned with the business objective?\n497\nDownload from finelybook www.finelybook.com", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 1317, "content": "7. What would be the minimum performance needed to reach the business objec\u2010\ntive?\n8. What are comparable problems? Can you reuse experience or tools?\n9. Is human expertise available?\n10. How would you solve the problem manually?\n11. List the assumptions you (or others) have made so far.\n12. Verify assumptions if possible.\nGet the Data\nNote: automate as much as possible so you can easily get fresh data.\n1. List the data you need and how much you need.\n2. Find and document where you can get that data.\n3. Check how much space it will take.\n4. Check legal obligations, and get authorization if necessary.\n5. Get access authorizations.\n6. Create a workspace (with enough storage space).\n7. Get the data.\n8. Convert the data to a format you can easily manipulate (without changing the\ndata itself).\n9. Ensure sensitive information is deleted or protected (e.g., anonymized).\n10. Check the size and type of data (time series, sample, geographical, etc.).", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 1318, "content": "10. Check the size and type of data (time series, sample, geographical, etc.).\n11. Sample a test set, put it aside, and never look at it (no data snooping!).\nExplore the Data\nNote: try to get insights from a field expert for these steps.\n1. Create a copy of the data for exploration (sampling it down to a manageable size\nif necessary).\n2. Create a Jupyter notebook to keep a record of your data exploration.\n3. Study each attribute and its characteristics:\n\u2022 Name\n\u2022 Type (categorical, int/float, bounded/unbounded, text, structured, etc.)\n498 \n| \nAppendix B: Machine Learning Project Checklist\nDownload from finelybook www.finelybook.com", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 1319, "content": "\u2022 % of missing values\n\u2022 Noisiness and type of noise (stochastic, outliers, rounding errors, etc.)\n\u2022 Possibly useful for the task?\n\u2022 Type of distribution (Gaussian, uniform, logarithmic, etc.)\n4. For supervised learning tasks, identify the target attribute(s).\n5. Visualize the data.\n6. Study the correlations between attributes.\n7. Study how you would solve the problem manually.\n8. Identify the promising transformations you may want to apply.\n9. Identify extra data that would be useful (go back to \u201cGet the Data\u201d on page 498).\n10. Document what you have learned.\nPrepare the Data\nNotes:\n\u2022 Work on copies of the data (keep the original dataset intact).\n\u2022 Write functions for all data transformations you apply, for five reasons:\n\u2014 So you can easily prepare the data the next time you get a fresh dataset\n\u2014 So you can apply these transformations in future projects\n\u2014 To clean and prepare the test set\n\u2014 To clean and prepare new data instances once your solution is live", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 1320, "content": "\u2014 To clean and prepare new data instances once your solution is live\n\u2014 To make it easy to treat your preparation choices as hyperparameters\n1. Data cleaning:\n\u2022 Fix or remove outliers (optional).\n\u2022 Fill in missing values (e.g., with zero, mean, median\u2026) or drop their rows (or\ncolumns).\n2. Feature selection (optional):\n\u2022 Drop the attributes that provide no useful information for the task.\n3. Feature engineering, where appropriate:\n\u2022 Discretize continuous features.\nMachine Learning Project Checklist \n| \n499\nDownload from finelybook www.finelybook.com", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 1321, "content": "\u2022 Decompose features (e.g., categorical, date/time, etc.).\n\u2022 Add promising transformations of features (e.g., log(x), sqrt(x), x^2, etc.).\n\u2022 Aggregate features into promising new features.\n4. Feature scaling: standardize or normalize features.\nShort-List Promising Models\nNotes:\n\u2022 If the data is huge, you may want to sample smaller training sets so you can train\nmany different models in a reasonable time (be aware that this penalizes complex\nmodels such as large neural nets or Random Forests).\n\u2022 Once again, try to automate these steps as much as possible.\n1. Train many quick and dirty models from different categories (e.g., linear, naive\nBayes, SVM, Random Forests, neural net, etc.) using standard parameters.\n2. Measure and compare their performance.\n\u2022 For each model, use N-fold cross-validation and compute the mean and stan\u2010\ndard deviation of the performance measure on the N folds.\n3. Analyze the most significant variables for each algorithm.", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 1322, "content": "3. Analyze the most significant variables for each algorithm.\n4. Analyze the types of errors the models make.\n\u2022 What data would a human have used to avoid these errors?\n5. Have a quick round of feature selection and engineering.\n6. Have one or two more quick iterations of the five previous steps.\n7. Short-list the top three to five most promising models, preferring models that\nmake different types of errors.\nFine-Tune the System\nNotes:\n\u2022 You will want to use as much data as possible for this step, especially as you move\ntoward the end of fine-tuning.\n\u2022 As always automate what you can.\n500 \n| \nAppendix B: Machine Learning Project Checklist\nDownload from finelybook www.finelybook.com", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 1323, "content": "1 \u201cPractical Bayesian Optimization of Machine Learning Algorithms,\u201d J. Snoek, H. Larochelle, R. Adams (2012).\n1. Fine-tune the hyperparameters using cross-validation.\n\u2022 Treat your data transformation choices as hyperparameters, especially when\nyou are not sure about them (e.g., should I replace missing values with zero or\nwith the median value? Or just drop the rows?).\n\u2022 Unless there are very few hyperparameter values to explore, prefer random\nsearch over grid search. If training is very long, you may prefer a Bayesian\noptimization approach (e.g., using Gaussian process priors, as described by\nJasper Snoek, Hugo Larochelle, and Ryan Adams).1\n2. Try Ensemble methods. Combining your best models will often perform better\nthan running them individually.\n3. Once you are confident about your final model, measure its performance on the\ntest set to estimate the generalization error.\nDon\u2019t tweak your model after measuring the generalization error:", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 1324, "content": "Don\u2019t tweak your model after measuring the generalization error:\nyou would just start overfitting the test set.\nPresent Your Solution\n1. Document what you have done.\n2. Create a nice presentation.\n\u2022 Make sure you highlight the big picture first.\n3. Explain why your solution achieves the business objective.\n4. Don\u2019t forget to present interesting points you noticed along the way.\n\u2022 Describe what worked and what did not.\n\u2022 List your assumptions and your system\u2019s limitations.\n5. Ensure your key findings are communicated through beautiful visualizations or\neasy-to-remember statements (e.g., \u201cthe median income is the number-one pre\u2010\ndictor of housing prices\u201d).\nMachine Learning Project Checklist \n| \n501\nDownload from finelybook www.finelybook.com", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 1325, "content": "Launch!\n1. Get your solution ready for production (plug into production data inputs, write\nunit tests, etc.).\n2. Write monitoring code to check your system\u2019s live performance at regular inter\u2010\nvals and trigger alerts when it drops.\n\u2022 Beware of slow degradation too: models tend to \u201crot\u201d as data evolves.\n\u2022 Measuring performance may require a human pipeline (e.g., via a crowdsourc\u2010\ning service).\n\u2022 Also monitor your inputs\u2019 quality (e.g., a malfunctioning sensor sending ran\u2010\ndom values, or another team\u2019s output becoming stale). This is particularly\nimportant for online learning systems.\n3. Retrain your models on a regular basis on fresh data (automate as much as possi\u2010\nble).\n502 \n| \nAppendix B: Machine Learning Project Checklist\nDownload from finelybook www.finelybook.com", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 1326, "content": "APPENDIX C\nSVM Dual Problem\nTo understand duality, you first need to understand the Lagrange multipliers method.\nThe general idea is to transform a constrained optimization objective into an uncon\u2010\nstrained one, by moving the constraints into the objective function. Let\u2019s look at a\nsimple example. Suppose you want to find the values of x and y that minimize the\nfunction f(x,y) = x2 + 2y, subject to an equality constraint: 3x + 2y + 1 = 0. Using the\nLagrange multipliers method, we start by defining a new function called the Lagran\u2010\ngian (or Lagrange function): g(x, y, \u03b1) = f(x, y) \u2013 \u03b1(3x + 2y + 1). Each constraint (in\nthis case just one) is subtracted from the original objective, multiplied by a new vari\u2010\nable called a Lagrange multiplier.\nJoseph-Louis Lagrange showed that if x, y is a solution to the constrained optimiza\u2010\ntion problem, then there must exist an \u03b1 such that x, y, \u03b1 is a stationary point of the", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 1327, "content": "tion problem, then there must exist an \u03b1 such that x, y, \u03b1 is a stationary point of the\nLagrangian (a stationary point is a point where all partial derivatives are equal to\nzero). In other words, we can compute the partial derivatives of g(x, y, \u03b1) with regards\nto x, y, and \u03b1; we can find the points where these derivatives are all equal to zero; and\nthe solutions to the constrained optimization problem (if they exist) must be among\nthese stationary points.\nIn this example the partial derivatives are: \n\u2202\n\u2202x g x, y, \u03b1 = 2x \u22123\u03b1\n\u2202\n\u2202y g x, y, \u03b1 = 2 \u22122\u03b1\n\u2202\n\u2202\u03b1 g x, y, \u03b1 = \u22123x \u22122y \u22121\nWhen \nall \nthese \npartial \nderivatives \nare \nequal \nto \n0, \nwe \nfind \nthat\n2x \u22123\u03b1 = 2 \u22122\u03b1 = \u22123x \u22122y \u22121 = 0, from which we can easily find that x = 3\n2,\ny = \u221211\n4 , and \u03b1 = 1. This is the only stationary point, and as it respects the con\u2010\nstraint, it must be the solution to the constrained optimization problem.\n503\nDownload from finelybook www.finelybook.com", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 1328, "content": "However, this method applies only to equality constraints. Fortunately, under some\nregularity conditions (which are respected by the SVM objectives), this method can\nbe generalized to inequality constraints as well (e.g., 3x + 2y + 1 \u2265 0). The generalized\nLagrangian for the hard margin problem is given by Equation C-1, where the \u03b1(i) vari\u2010\nables are called the Karush\u2013Kuhn\u2013Tucker (KKT) multipliers, and they must be greater\nor equal to zero.\nEquation C-1. Generalized Lagrangian for the hard margin problem\n\u2112\ufffd, b, \u03b1 = 1\n2\ufffdT \u00b7 \ufffd\u2212\u2211\ni = 1\nm\n\u03b1 i t i \ufffdT \u00b7 \ufffdi + b \u22121\nwith\n\u03b1 i \u22650\nfor i = 1, 2, \u22ef, m\nJust like with the Lagrange multipliers method, you can compute the partial deriva\u2010\ntives and locate the stationary points. If there is a solution, it will necessarily be\namong the stationary points \ufffd, b, \u03b1 that respect the KKT conditions:\n\u2022 Respect the problem\u2019s constraints: t i\n\ufffdT \u00b7 \ufffdi + b \u22651\nfor i = 1, 2, \u22ef, m,\n\u2022 Verify \u03b1 i \u22650\nfor i = 1, 2, \u22ef, m,", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 1329, "content": "\ufffdT \u00b7 \ufffdi + b \u22651\nfor i = 1, 2, \u22ef, m,\n\u2022 Verify \u03b1 i \u22650\nfor i = 1, 2, \u22ef, m,\n\u2022 Either \u03b1 i = 0 or the ith constraint must be an active constraint, meaning it must\nhold by equality: t i\n\ufffdT \u00b7 \ufffdi + b = 1. This condition is called the complemen\u2010\ntary slackness condition. It implies that either \u03b1 i = 0 or the ith instance lies on the\nboundary (it is a support vector).\nNote that the KKT conditions are necessary conditions for a stationary point to be a\nsolution of the constrained optimization problem. Under some conditions, they are\nalso sufficient conditions. Luckily, the SVM optimization problem happens to meet\nthese conditions, so any stationary point that meets the KKT conditions is guaranteed\nto be a solution to the constrained optimization problem.\nWe can compute the partial derivatives of the generalized Lagrangian with regards to\nw and b with Equation C-2.\nEquation C-2. Partial derivatives of the generalized Lagrangian\n\u2207\ufffd\u2112\ufffd, b, \u03b1 = \ufffd\u2212\u2211\ni = 1\nm\n\u03b1 i t i \ufffdi\n\u2202\n\u2202b\u2112\ufffd, b, \u03b1 = \u2212\u2211\ni = 1\nm\n\u03b1 i t i", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 1330, "content": "\u2207\ufffd\u2112\ufffd, b, \u03b1 = \ufffd\u2212\u2211\ni = 1\nm\n\u03b1 i t i \ufffdi\n\u2202\n\u2202b\u2112\ufffd, b, \u03b1 = \u2212\u2211\ni = 1\nm\n\u03b1 i t i\n504 \n| \nAppendix C: SVM Dual Problem\nDownload from finelybook www.finelybook.com", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 1331, "content": "When these partial derivatives are equal to 0, we have Equation C-3.\nEquation C-3. Properties of the stationary points\n\ufffd= \u2211\ni = 1\nm\n\u03b1 i t i \ufffdi\n\u2211\ni = 1\nm\n\u03b1 i t i = 0\nIf we plug these results into the definition of the generalized Lagrangian, some terms\ndisappear and we find Equation C-4.\nEquation C-4. Dual form of the SVM problem\n\u2112\ufffd, b, \u03b1 = 1\n2 \u2211\ni = 1\nm\n\u2211\nj = 1\nm\n\u03b1 i \u03b1 j t i t j \ufffdi T \u00b7 \ufffdj\n\u2212\n\u2211\ni = 1\nm\n\u03b1 i\nwith\n\u03b1 i \u22650\nfor i = 1, 2, \u22ef, m\nThe goal is now to find the vector \u03b1 that minimizes this function, with \u03b1 i \u22650 for all\ninstances. This constrained optimization problem is the dual problem we were look\u2010\ning for.\nOnce you find the optimal \u03b1, you can compute \ufffd using the first line of Equation C-3.\nTo compute b, you can use the fact that a support vector verifies t(i)(wT \u00b7 x(i) + b) = 1,\nso if the kth instance is a support vector (i.e., \u03b1k > 0), you can use it to compute\nb = 1 \u2212t k \ufffdT \u00b7 \ufffdk . However, it is often prefered to compute the average over all", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 1332, "content": "b = 1 \u2212t k \ufffdT \u00b7 \ufffdk . However, it is often prefered to compute the average over all\nsupport vectors to get a more stable and precise value, as in Equation C-5.\nEquation C-5. Bias term estimation using the dual form\nb = 1\nns \u2211\ni = 1\n\u03b1 i > 0\nm\n1 \u2212t i \ufffdT \u00b7 \ufffdi\nSVM Dual Problem \n| \n505\nDownload from finelybook www.finelybook.com", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 1333, "content": "Download from finelybook www.finelybook.com", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 1334, "content": "APPENDIX D\nAutodiff\nThis appendix explains how TensorFlow\u2019s autodiff feature works, and how it com\u2010\npares to other solutions.\nSuppose you define a function f(x,y) = x2y + y + 2, and you need its partial derivatives\n\u2202f\n\u2202x and \u2202f\n\u2202y, typically to perform Gradient Descent (or some other optimization algo\u2010\nrithm). Your main options are manual differentiation, symbolic differentiation,\nnumerical differentiation, forward-mode autodiff, and finally reverse-mode autodiff.\nTensorFlow implements this last option. Let\u2019s go through each of these options.\nManual Differentiation\nThe first approach is to pick up a pencil and a piece of paper and use your calculus\nknowledge to derive the partial derivatives manually. For the function f(x,y) just\ndefined, it is not too hard; you just need to use five rules:\n\u2022 The derivative of a constant is 0.\n\u2022 The derivative of \u03bbx is \u03bb (where \u03bb is a constant).\n\u2022 The derivative of x\u03bb is \u03bbx\u03bb \u2013 1, so the derivative of x2 is 2x.", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 1335, "content": "\u2022 The derivative of x\u03bb is \u03bbx\u03bb \u2013 1, so the derivative of x2 is 2x.\n\u2022 The derivative of a sum of functions is the sum of these functions\u2019 derivatives.\n\u2022 The derivative of \u03bb times a function is \u03bb times its derivative.\n507\nDownload from finelybook www.finelybook.com", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 1336, "content": "From these rules, you can derive Equation D-1:\nEquation D-1. Partial derivatives of f(x,y)\n\u2202f\n\u2202x =\n\u2202x2y\n\u2202x\n+ \u2202y\n\u2202x + \u22022\n\u2202x = y\n\u2202x2\n\u2202x\n+ 0 + 0 = 2xy\n\u2202f\n\u2202y =\n\u2202x2y\n\u2202y\n+ \u2202y\n\u2202y + \u22022\n\u2202y = x2 + 1 + 0 = x2 + 1\nThis approach can become very tedious for more complex functions, and you run the\nrisk of making mistakes. The good news is that deriving the mathematical equations\nfor the partial derivatives like we just did can be automated, through a process called\nsymbolic differentiation.\nSymbolic Differentiation\nFigure D-1 shows how symbolic differentiation works on an even simpler function,\ng(x,y) = 5 + xy. The graph for that function is represented on the left. After symbolic\ndifferentiation, we get the graph on the right, which represents the partial derivative\n\u2202g\n\u2202x = 0 + 0 \u00d7 x + y \u00d7 1 = y (we could similarly obtain the partial derivative with\nregards to y).\nFigure D-1. Symbolic differentiation\nThe algorithm starts by getting the partial derivative of the leaf nodes. The constant", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 1337, "content": "The algorithm starts by getting the partial derivative of the leaf nodes. The constant\nnode (5) returns the constant 0, since the derivative of a constant is always 0. The\n508 \n| \nAppendix D: Autodiff\nDownload from finelybook www.finelybook.com", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 1338, "content": "variable x returns the constant 1 since \u2202x\n\u2202x = 1, and the variable y returns the constant\n0 since \u2202y\n\u2202x = 0 (if we were looking for the partial derivative with regards to y, it would\nbe the reverse).\nNow we have all we need to move up the graph to the multiplication node in function\ng. Calculus tells us that the derivative of the product of two functions u and v is\n\u2202u \u00d7 v\n\u2202x\n= \u2202v\n\u2202x \u00d7 u + \u2202u\n\u2202x \u00d7 u. We can therefore construct a large part of the graph on the\nright, representing 0 \u00d7 x + y \u00d7 1.\nFinally, we can go up to the addition node in function g. As mentioned, the derivative\nof a sum of functions is the sum of these functions\u2019 derivatives. So we just need to\ncreate an addition node and connect it to the parts of the graph we have already com\u2010\nputed. We get the correct partial derivative: \u2202g\n\u2202x = 0 + 0 \u00d7 x + y \u00d7 1 .\nHowever, it can be simplified (a lot). A few trivial pruning steps can be applied to this", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 1339, "content": "However, it can be simplified (a lot). A few trivial pruning steps can be applied to this\ngraph to get rid of all unnecessary operations, and we get a much smaller graph with\njust one node: \u2202g\n\u2202x = y.\nIn this case, simplification is fairly easy, but for a more complex function, symbolic\ndifferentiation can produce a huge graph that may be tough to simplify and lead to\nsuboptimal performance. Most importantly, symbolic differentiation cannot deal with\nfunctions defined with arbitrary code\u2014for example, the following function discussed\nin Chapter 9:\ndef my_func(a, b):\n z = 0\n for i in range(100):\n z = a * np.cos(z + i) + z * np.sin(b - i)\n return z\nNumerical Differentiation\nThe simplest solution is to compute an approximation of the derivatives, numerically.\nRecall that the derivative h\u2032(x0) of a function h(x) at a point x0 is the slope of the func\u2010\ntion at that point, or more precisely Equation D-2.\nEquation D-2. Derivative of a function h(x) at point x0\nh\u2032 x =\nlim\nx\nx0", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 1340, "content": "Equation D-2. Derivative of a function h(x) at point x0\nh\u2032 x =\nlim\nx\nx0\nh x \u2212h x0\nx \u2212x0\n=\nlim\n\ufffd\n0\nh x0 + \ufffd\u2212h x0\n\ufffd\nAutodiff \n| \n509\nDownload from finelybook www.finelybook.com", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 1341, "content": "So if we want to calculate the partial derivative of f(x,y) with regards to x, at x = 3 and\ny = 4, we can simply compute f(3 + \u03f5, 4) \u2013 f(3, 4) and divide the result by \u03f5, using a\nvery small value for \u03f5. That\u2019s exactly what the following code does:\ndef f(x, y):\n return x**2*y + y + 2\ndef derivative(f, x, y, x_eps, y_eps):\n return (f(x + x_eps, y + y_eps) - f(x, y)) / (x_eps + y_eps)\ndf_dx = derivative(f, 3, 4, 0.00001, 0)\ndf_dy = derivative(f, 3, 4, 0, 0.00001)\nUnfortunately, the result is imprecise (and it gets worse for more complex functions).\nThe correct results are respectively 24 and 10, but instead we get:\n>>> print(df_dx)\n24.000039999805264\n>>> print(df_dy)\n10.000000000331966\nNotice that to compute both partial derivatives, we have to call f() at least three times\n(we called it four times in the preceding code, but it could be optimized). If there\nwere 1,000 parameters, we would need to call f() at least 1,001 times. When you are", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 1342, "content": "were 1,000 parameters, we would need to call f() at least 1,001 times. When you are\ndealing with large neural networks, this makes numerical differentiation way too\ninefficient.\nHowever, numerical differentiation is so simple to implement that it is a great tool to\ncheck that the other methods are implemented correctly. For example, if it disagrees\nwith your manually derived function, then your function probably contains a mis\u2010\ntake.\nForward-Mode Autodiff\nForward-mode autodiff is neither numerical differentiation nor symbolic differentia\u2010\ntion, but in some ways it is their love child. It relies on dual numbers, which are\n(weird but fascinating) numbers of the form a + b\u03f5 where a and b are real numbers\nand \u03f5 is an infinitesimal number such that \u03f52 = 0 (but \u03f5 \u2260 0). You can think of the\ndual number 42 + 24\u03f5 as something akin to 42.0000\u22ef000024 with an infinite num\u2010\nber of 0s (but of course this is simplified just to give you some idea of what dual num\u2010", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 1343, "content": "ber of 0s (but of course this is simplified just to give you some idea of what dual num\u2010\nbers are). A dual number is represented in memory as a pair of floats. For example, 42\n+ 24\u03f5 is represented by the pair (42.0, 24.0).\n510 \n| \nAppendix D: Autodiff\nDownload from finelybook www.finelybook.com", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 1344, "content": "Dual numbers can be added, multiplied, and so on, as shown in Equation D-3.\nEquation D-3. A few operations with dual numbers\n\u03bb a + b\ufffd= \u03bba + \u03bbb\ufffd\na + b\ufffd+ c + d\ufffd= a + c + b + d \ufffd\na + b\ufffd\u00d7 c + d\ufffd= ac + ad + bc \ufffd+ bd \ufffd2 = ac + ad + bc \ufffd\nMost importantly, it can be shown that h(a + b\u03f5) = h(a) + b \u00d7 h\u2032(a)\u03f5, so computing\nh(a + \u03f5) gives you both h(a) and the derivative h\u2032(a) in just one shot. Figure D-2\nshows how forward-mode autodiff computes the partial derivative of f(x,y) with\nregards to x at x = 3 and y = 4. All we need to do is compute f(3 + \u03f5, 4); this will\noutput a dual number whose first component is equal to f(3, 4) and whose second\ncomponent is equal to \u2202f\n\u2202x 3, 4 .\nFigure D-2. Forward-mode autodiff\nAutodiff \n| \n511\nDownload from finelybook www.finelybook.com", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 1345, "content": "To compute \u2202f\n\u2202y 3, 4 we would have to go through the graph again, but this time with\nx = 3 and y = 4 + \u03f5.\nSo forward-mode autodiff is much more accurate than numerical differentiation, but\nit suffers from the same major flaw: if there were 1,000 parameters, it would require\n1,000 passes through the graph to compute all the partial derivatives. This is where\nreverse-mode autodiff shines: it can compute all of them in just two passes through\nthe graph.\nReverse-Mode Autodiff\nReverse-mode autodiff is the solution implemented by TensorFlow. It first goes\nthrough the graph in the forward direction (i.e., from the inputs to the output) to\ncompute the value of each node. Then it does a second pass, this time in the reverse\ndirection (i.e., from the output to the inputs) to compute all the partial derivatives.\nFigure D-3 represents the second pass. During the first pass, all the node values were\ncomputed, starting from x = 3 and y = 4. You can see those values at the bottom right", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 1346, "content": "computed, starting from x = 3 and y = 4. You can see those values at the bottom right\nof each node (e.g., x \u00d7 x = 9). The nodes are labeled n1 to n7 for clarity. The output\nnode is n7: f(3,4) = n7 = 42.\nFigure D-3. Reverse-mode autodiff\n512 \n| \nAppendix D: Autodiff\nDownload from finelybook www.finelybook.com", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 1347, "content": "The idea is to gradually go down the graph, computing the partial derivative of f(x,y)\nwith regards to each consecutive node, until we reach the variable nodes. For this,\nreverse-mode autodiff relies heavily on the chain rule, shown in Equation D-4.\nEquation D-4. Chain rule\n\u2202f\n\u2202x = \u2202f\n\u2202ni\n\u00d7\n\u2202ni\n\u2202x\nSince n7 is the output node, f = n7 so trivially \u2202f\n\u2202n7 = 1.\nLet\u2019s continue down the graph to n5: how much does f vary when n5 varies? The\nanswer is \u2202f\n\u2202n5 = \u2202f\n\u2202n7 \u00d7\n\u2202n7\n\u2202n5. We already know that \u2202f\n\u2202n7 = 1, so all we need is \n\u2202n7\n\u2202n5. Since\nn7 simply performs the sum n5 + n6, we find that \n\u2202n7\n\u2202n5 = 1, so \u2202f\n\u2202n5 = 1 \u00d7 1 = 1.\nNow we can proceed to node n4: how much does f vary when n4 varies? The answer is\n\u2202f\n\u2202n4 = \u2202f\n\u2202n5 \u00d7\n\u2202n5\n\u2202n4. Since n5 = n4 \u00d7 n2, we find that \n\u2202n5\n\u2202n4 = n2, so \u2202f\n\u2202n4 = 1 \u00d7 n2 = 4.\nThe process continues until we reach the bottom of the graph. At that point we will\nhave calculated all the partial derivatives of f(x,y) at the point x = 3 and y = 4. In this", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 1348, "content": "have calculated all the partial derivatives of f(x,y) at the point x = 3 and y = 4. In this\nexample, we find \u2202f\n\u2202x = 24 and \u2202f\n\u2202y = 10. Sounds about right!\nReverse-mode autodiff is a very powerful and accurate technique, especially when\nthere are many inputs and few outputs, since it requires only one forward pass plus\none reverse pass per output to compute all the partial derivatives for all outputs with\nregards to all the inputs. Most importantly, it can deal with functions defined by arbi\u2010\ntrary code. It can also handle functions that are not entirely differentiable, as long as\nyou ask it to compute the partial derivatives at points that are differentiable.\nIf you implement a new type of operation in TensorFlow and you\nwant to make it compatible with autodiff, then you need to provide\na function that builds a subgraph to compute its partial derivatives\nwith regards to its inputs. For example, suppose you implement a", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 1349, "content": "with regards to its inputs. For example, suppose you implement a\nfunction that computes the square of its input f(x) = x2. In that case\nyou would need to provide the corresponding derivative function f\u2032\n(x) = 2x. Note that this function does not compute a numerical\nresult, but instead builds a subgraph that will (later) compute the\nresult. This is very useful because it means that you can compute\ngradients of gradients (to compute second-order derivatives, or\neven higher-order derivatives).\nAutodiff \n| \n513\nDownload from finelybook www.finelybook.com", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 1350, "content": "Download from finelybook www.finelybook.com", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 1351, "content": "APPENDIX E\nOther Popular ANN Architectures\nIn this appendix we will give a quick overview of a few historically important neural\nnetwork architectures that are much less used today than deep Multi-Layer Percep\u2010\ntrons (Chapter 10), convolutional neural networks (Chapter 13), recurrent neural\nnetworks (Chapter 14), or autoencoders (Chapter 15). They are often mentioned in\nthe literature, and some are still used in many applications, so it is worth knowing\nabout them. Moreover, we will discuss deep belief nets (DBNs), which were the state of\nthe art in Deep Learning until the early 2010s. They are still the subject of very active\nresearch, so they may well come back with a vengeance in the near future.\nHopfield Networks\nHopfield networks were first introduced by W. A. Little in 1974, then popularized by J.\nHopfield in 1982. They are associative memory networks: you first teach them some\npatterns, and then when they see a new pattern they (hopefully) output the closest", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 1352, "content": "patterns, and then when they see a new pattern they (hopefully) output the closest\nlearned pattern. This has made them useful in particular for character recognition\nbefore they were outperformed by other approaches. You first train the network by\nshowing it examples of character images (each binary pixel maps to one neuron), and\nthen when you show it a new character image, after a few iterations it outputs the\nclosest learned character.\nThey are fully connected graphs (see Figure E-1); that is, every neuron is connected\nto every other neuron. Note that on the diagram the images are 6 \u00d7 6 pixels, so the\nneural network on the left should contain 36 neurons (and 648 connections), but for\nvisual clarity a much smaller network is represented.\n515\nDownload from finelybook www.finelybook.com", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 1353, "content": "Figure E-1. Hopfield network\nThe training algorithm works by using Hebb\u2019s rule: for each training image, the\nweight between two neurons is increased if the corresponding pixels are both on or\nboth off, but decreased if one pixel is on and the other is off.\nTo show a new image to the network, you just activate the neurons that correspond to\nactive pixels. The network then computes the output of every neuron, and this gives\nyou a new image. You can then take this new image and repeat the whole process.\nAfter a while, the network reaches a stable state. Generally, this corresponds to the\ntraining image that most resembles the input image.\nA so-called energy function is associated with Hopfield nets. At each iteration, the\nenergy decreases, so the network is guaranteed to eventually stabilize to a low-energy\nstate. The training algorithm tweaks the weights in a way that decreases the energy\nlevel of the training patterns, so the network is likely to stabilize in one of these low-", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 1354, "content": "level of the training patterns, so the network is likely to stabilize in one of these low-\nenergy configurations. Unfortunately, some patterns that were not in the training set\nalso end up with low energy, so the network sometimes stabilizes in a configuration\nthat was not learned. These are called spurious patterns.\nAnother major flaw with Hopfield nets is that they don\u2019t scale very well\u2014their mem\u2010\nory capacity is roughly equal to 14% of the number of neurons. For example, to clas\u2010\nsify 28 \u00d7 28 images, you would need a Hopfield net with 784 fully connected neurons\nand 306,936 weights. Such a network would only be able to learn about 110 different\ncharacters (14% of 784). That\u2019s a lot of parameters for such a small memory.\nBoltzmann Machines\nBoltzmann machines were invented in 1985 by Geoffrey Hinton and Terrence Sejnow\u2010\nski. Just like Hopfield nets, they are fully connected ANNs, but they are based on sto\u2010\n516 \n| \nAppendix E: Other Popular ANN Architectures", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 1355, "content": "516 \n| \nAppendix E: Other Popular ANN Architectures\nDownload from finelybook www.finelybook.com", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 1356, "content": "chastic neurons: instead of using a deterministic step function to decide what value to\noutput, these neurons output 1 with some probability, and 0 otherwise. The probabil\u2010\nity function that these ANNs use is based on the Boltzmann distribution (used in\nstatistical mechanics) hence their name. Equation E-1 gives the probability that a par\u2010\nticular neuron will output a 1.\nEquation E-1. Probability that the ith neuron will output 1\np si\nnext step = 1 = \u03c3\n\u2211j = 1\nN\nwi, jsj + bi\nT\n\u2022 sj is the jth neuron\u2019s state (0 or 1).\n\u2022 wi,j is the connection weight between the ith and jth neurons. Note that wi,i = 0.\n\u2022 bi is the ith neuron\u2019s bias term. We can implement this term by adding a bias neu\u2010\nron to the network.\n\u2022 N is the number of neurons in the network.\n\u2022 T is a number called the network\u2019s temperature; the higher the temperature, the\nmore random the output is (i.e., the more the probability approaches 50%).\n\u2022 \u03c3 is the logistic function.", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 1357, "content": "\u2022 \u03c3 is the logistic function.\nNeurons in Boltzmann machines are separated into two groups: visible units and hid\u2010\nden units (see Figure E-2). All neurons work in the same stochastic way, but the visi\u2010\nble units are the ones that receive the inputs and from which outputs are read.\nFigure E-2. Boltzmann machine\nOther Popular ANN Architectures \n| \n517\nDownload from finelybook www.finelybook.com", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 1358, "content": "Because of its stochastic nature, a Boltzmann machine will never stabilize into a fixed\nconfiguration, but instead it will keep switching between many configurations. If it is\nleft running for a sufficiently long time, the probability of observing a particular con\u2010\nfiguration will only be a function of the connection weights and bias terms, not of the\noriginal configuration (similarly, after you shuffle a deck of cards for long enough, the\nconfiguration of the deck does not depend on the initial state). When the network\nreaches this state where the original configuration is \u201cforgotten,\u201d it is said to be in\nthermal equilibrium (although its configuration keeps changing all the time). By set\u2010\nting the network parameters appropriately, letting the network reach thermal equili\u2010\nbrium, and then observing its state, we can simulate a wide range of probability\ndistributions. This is called a generative model.\nTraining a Boltzmann machine means finding the parameters that will make the net\u2010", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 1359, "content": "Training a Boltzmann machine means finding the parameters that will make the net\u2010\nwork approximate the training set\u2019s probability distribution. For example, if there are\nthree visible neurons and the training set contains 75% (0, 1, 1) triplets, 10% (0, 0, 1)\ntriplets, and 15% (1, 1, 1) triplets, then after training a Boltzmann machine, you could\nuse it to generate random binary triplets with about the same probability distribu\u2010\ntion. For example, about 75% of the time it would output the (0, 1, 1) triplet.\nSuch a generative model can be used in a variety of ways. For example, if it is trained\non images, and you provide an incomplete or noisy image to the network, it will\nautomatically \u201crepair\u201d the image in a reasonable way. You can also use a generative\nmodel for classification. Just add a few visible neurons to encode the training image\u2019s\nclass (e.g., add 10 visible neurons and turn on only the fifth neuron when the training", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 1360, "content": "class (e.g., add 10 visible neurons and turn on only the fifth neuron when the training\nimage represents a 5). Then, when given a new image, the network will automatically\nturn on the appropriate visible neurons, indicating the image\u2019s class (e.g., it will turn\non the fifth visible neuron if the image represents a 5).\nUnfortunately, there is no efficient technique to train Boltzmann machines. However,\nfairly efficient algorithms have been developed to train restricted Boltzmann machines\n(RBM).\nRestricted Boltzmann Machines\nAn RBM is simply a Boltzmann machine in which there are no connections between\nvisible units or between hidden units, only between visible and hidden units. For\nexample, Figure E-3 represents an RBM with three visible units and four hidden\nunits.\n518 \n| \nAppendix E: Other Popular ANN Architectures\nDownload from finelybook www.finelybook.com", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 1361, "content": "1 \u201cOn Contrastive Divergence Learning,\u201d M. \u00c1. Carreira-Perpi\u00f1\u00e1n and G. Hinton (2005).\nFigure E-3. Restricted Boltzmann machine\nA very efficient training algorithm, called Contrastive Divergence, was introduced in\n2005 by Miguel \u00c1. Carreira-Perpi\u00f1\u00e1n and Geoffrey Hinton.1 Here is how it works: for\neach training instance x, the algorithm starts by feeding it to the network by setting\nthe state of the visible units to x1, x2, \u22ef, xn. Then you compute the state of the hidden\nunits by applying the stochastic equation described before (Equation E-1). This gives\nyou a hidden vector h (where hi is equal to the state of the ith unit). Next you compute\nthe state of the visible units, by applying the same stochastic equation. This gives you\na vector \ufffd\u02d9. Then once again you compute the state of the hidden units, which gives\nyou a vector \ufffd\u02d9. Now you can update each connection weight by applying the rule in\nEquation E-2.\nEquation E-2. Contrastive divergence weight update\nwi, j", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 1362, "content": "Equation E-2.\nEquation E-2. Contrastive divergence weight update\nwi, j\nnext step = wi, j + \u03b7 \ufffd\ufffdT \u2212\ufffd\u02d9\ufffd\u02d9T\nThe great benefit of this algorithm it that it does not require waiting for the network\nto reach thermal equilibrium: it just goes forward, backward, and forward again, and\nthat\u2019s it. This makes it incomparably more efficient than previous algorithms, and it\nwas a key ingredient to the first success of Deep Learning based on multiple stacked\nRBMs.\nDeep Belief Nets\nSeveral layers of RBMs can be stacked; the hidden units of the first-level RBM serves\nas the visible units for the second-layer RBM, and so on. Such an RBM stack is called\na deep belief net (DBN).\nYee-Whye Teh, one of Geoffrey Hinton\u2019s students, observed that it was possible to\ntrain DBNs one layer at a time using Contrastive Divergence, starting with the lower\nOther Popular ANN Architectures \n| \n519\nDownload from finelybook www.finelybook.com", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 1363, "content": "2 \u201cA Fast Learning Algorithm for Deep Belief Nets,\u201d G. Hinton, S. Osindero, Y. Teh (2006).\nlayers and then gradually moving up to the top layers. This led to the groundbreaking\narticle that kickstarted the Deep Learning tsunami in 2006.2\nJust like RBMs, DBNs learn to reproduce the probability distribution of their inputs,\nwithout any supervision. However, they are much better at it, for the same reason that\ndeep neural networks are more powerful than shallow ones: real-world data is often\norganized in hierarchical patterns, and DBNs take advantage of that. Their lower lay\u2010\ners learn low-level features in the input data, while higher layers learn high-level fea\u2010\ntures.\nJust like RBMs, DBNs are fundamentally unsupervised, but you can also train them\nin a supervised manner by adding some visible units to represent the labels. More\u2010\nover, one great feature of DBNs is that they can be trained in a semisupervised fash\u2010", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 1364, "content": "over, one great feature of DBNs is that they can be trained in a semisupervised fash\u2010\nion. Figure E-4 represents such a DBN configured for semisupervised learning.\nFigure E-4. A deep belief network configured for semisupervised learning\nFirst, the RBM 1 is trained without supervision. It learns low-level features in the\ntraining data. Then RBM 2 is trained with RBM 1\u2019s hidden units as inputs, again\nwithout supervision: it learns higher-level features (note that RBM 2\u2019s hidden units\ninclude only the three rightmost units, not the label units). Several more RBMs could\nbe stacked this way, but you get the idea. So far, training was 100% unsupervised.\n520 \n| \nAppendix E: Other Popular ANN Architectures\nDownload from finelybook www.finelybook.com", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 1365, "content": "3 See this video by Geoffrey Hinton for more details and a demo: http://goo.gl/7Z5QiS.\nLastly, RBM 3 is trained using both RBM 2\u2019s hidden units as inputs, as well as extra\nvisible units used to represent the target labels (e.g., a one-hot vector representing the\ninstance class). It learns to associate high-level features with training labels. This is\nthe supervised step.\nAt the end of training, if you feed RBM 1 a new instance, the signal will propagate up\nto RBM 2, then up to the top of RBM 3, and then back down to the label units; hope\u2010\nfully, the appropriate label will light up. This is how a DBN can be used for classifica\u2010\ntion.\nOne great benefit of this semisupervised approach is that you don\u2019t need much\nlabeled training data. If the unsupervised RBMs do a good enough job, then only a\nsmall amount of labeled training instances per class will be necessary. Similarly, a\nbaby learns to recognize objects without supervision, so when you point to a chair", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 1366, "content": "baby learns to recognize objects without supervision, so when you point to a chair\nand say \u201cchair,\u201d the baby can associate the word \u201cchair\u201d with the class of objects it has\nalready learned to recognize on its own. You don\u2019t need to point to every single chair\nand say \u201cchair\u201d; only a few examples will suffice (just enough so the baby can be sure\nthat you are indeed referring to the chair, not to its color or one of the chair\u2019s parts).\nQuite amazingly, DBNs can also work in reverse. If you activate one of the label units,\nthe signal will propagate up to the hidden units of RBM 3, then down to RBM 2, and\nthen RBM 1, and a new instance will be output by the visible units of RBM 1. This\nnew instance will usually look like a regular instance of the class whose label unit you\nactivated. This generative capability of DBNs is quite powerful. For example, it has\nbeen used to automatically generate captions for images, and vice versa: first a DBN is", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 1367, "content": "been used to automatically generate captions for images, and vice versa: first a DBN is\ntrained (without supervision) to learn features in images, and another DBN is trained\n(again without supervision) to learn features in sets of captions (e.g., \u201ccar\u201d often\ncomes with \u201cautomobile\u201d). Then an RBM is stacked on top of both DBNs and trained\nwith a set of images along with their captions; it learns to associate high-level features\nin images with high-level features in captions. Next, if you feed the image DBN an\nimage of a car, the signal will propagate through the network, up to the top-level\nRBM, and back down to the bottom of the caption DBN, producing a caption. Due to\nthe stochastic nature of RBMs and DBNs, the caption will keep changing randomly,\nbut it will generally be appropriate for the image. If you generate a few hundred cap\u2010\ntions, the most frequently generated ones will likely be a good description of the\nimage.3\nSelf-Organizing Maps", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 1368, "content": "image.3\nSelf-Organizing Maps\nSelf-organizing maps (SOM) are quite different from all the other types of neural net\u2010\nworks we have discussed so far. They are used to produce a low-dimensional repre\u2010\nOther Popular ANN Architectures \n| \n521\nDownload from finelybook www.finelybook.com", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 1369, "content": "sentation of a high-dimensional dataset, generally for visualization, clustering, or\nclassification. The neurons are spread across a map (typically 2D for visualization,\nbut it can be any number of dimensions you want), as shown in Figure E-5, and each\nneuron has a weighted connection to every input (note that the diagram shows just\ntwo inputs, but there are typically a very large number, since the whole point of\nSOMs is to reduce dimensionality).\nFigure E-5. Self-organizing maps\nOnce the network is trained, you can feed it a new instance and this will activate only\none neuron (i.e., hence one point on the map): the neuron whose weight vector is\nclosest to the input vector. In general, instances that are nearby in the original input\nspace will activate neurons that are nearby on the map. This makes SOMs useful for\nvisualization (in particular, you can easily identify clusters on the map), but also for\napplications like speech recognition. For example, if each instance represents the", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 1370, "content": "applications like speech recognition. For example, if each instance represents the\naudio recording of a person pronouncing a vowel, then different pronunciations of\nthe vowel \u201ca\u201d will activate neurons in the same area of the map, while instances of the\nvowel \u201ce\u201d will activate neurons in another area, and intermediate sounds will gener\u2010\nally activate intermediate neurons on the map.\nOne important difference with the other dimensionality reduction\ntechniques discussed in Chapter 8 is that all instances get mapped\nto a discrete number of points in the low-dimensional space (one\npoint per neuron). When there are very few neurons, this techni\u2010\nque is better described as clustering rather than dimensionality\nreduction.\n522 \n| \nAppendix E: Other Popular ANN Architectures\nDownload from finelybook www.finelybook.com", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 1371, "content": "4 You can imagine a class of young children with roughly similar skills. One child happens to be slightly better\nat basketball. This motivates her to practice more, especially with her friends. After a while, this group of\nfriends gets so good at basketball that other kids cannot compete. But that\u2019s okay, because the other kids spe\u2010\ncialize in other topics. After a while, the class is full of little specialized groups.\nThe training algorithm is unsupervised. It works by having all the neurons compete\nagainst each other. First, all the weights are initialized randomly. Then a training\ninstance is picked randomly and fed to the network. All neurons compute the dis\u2010\ntance between their weight vector and the input vector (this is very different from the\nartificial neurons we have seen so far). The neuron that measures the smallest dis\u2010\ntance wins and tweaks its weight vector to be even slightly closer to the input vector,", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 1372, "content": "tance wins and tweaks its weight vector to be even slightly closer to the input vector,\nmaking it more likely to win future competitions for other inputs similar to this one.\nIt also recruits its neighboring neurons, and they too update their weight vector to be\nslightly closer to the input vector (but they don\u2019t update their weights as much as the\nwinner neuron). Then the algorithm picks another training instance and repeats the\nprocess, again and again. This algorithm tends to make nearby neurons gradually\nspecialize in similar inputs.4\nOther Popular ANN Architectures \n| \n523\nDownload from finelybook www.finelybook.com", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 1373, "content": "Download from finelybook www.finelybook.com", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 1374, "content": "Index\nSymbols\n__call__(), 385\n\u03b5-greedy policy, 459, 464\n\u03b5-insensitive, 155\n\u03c7 2 test (see chi square test)\n\u2113 0 norm, 39\n\u2113 1 and \u2113 2 regularization, 303-304\n\u2113 1 norm, 39, 130, 139, 300, 303\n\u2113 2 norm, 39, 128-130, 139, 142, 303, 307\n\u2113 k norm, 39\n\u2113 \u221e norm, 39\nA\naccuracy, 4, 83-84\nactions, evaluating, 447-448\nactivation functions, 262-264\nactive constraints, 504\nactors, 463\nactual class, 85\nAdaBoost, 192-195\nAdagrad, 296-298\nAdam optimization, 293, 298-300\nadaptive learning rate, 297\nadaptive moment optimization, 298\nagents, 438\nAlexNet architecture, 367-368\nalgorithms\npreparing data for, 59-68\nAlphaGo, 14, 253, 437, 453\nAnaconda, 41\nanomaly detection, 12\nApple\u2019s Siri, 253\napply_gradients(), 286, 450\narea under the curve (AUC), 92\narg_scope(), 285\narray_split(), 217\nartificial neural networks (ANNs), 253-274\nBoltzmann Machines, 516-518\ndeep belief networks (DBNs), 519-521\nevolution of, 254\nHopfield Networks, 515-516\nhyperparameter fine-tuning, 270-272\noverview, 253-255", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 1375, "content": "evolution of, 254\nHopfield Networks, 515-516\nhyperparameter fine-tuning, 270-272\noverview, 253-255\nPerceptrons, 257-264\nself-organizing maps, 521-523\ntraining a DNN with TensorFlow, 265-270\nartificial neuron, 256\n(see also artificial neural network (ANN))\nassign(), 237\nassociation rule learning, 12\nassociative memory networks, 515\nassumptions, checking, 40\nasynchronous updates, 348-349\nasynchrous communication, 329-334\natrous_conv2d(), 376\nattention mechanism, 409\nattributes, 9, 45-48\n(see also data structure)\ncombinations of, 58-59\npreprocessed, 48\ntarget, 48\nautodiff, 238-239, 507-513\nforward-mode, 510-512\nmanual differentiation, 507\nnumerical differentiation, 509\nreverse-mode, 512-513\nsymbolic differentiation, 508-509\nautoencoders, 411-435\n525\nDownload from finelybook www.finelybook.com", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 1376, "content": "adversarial, 433\ncontractive, 432\ndenoising, 424-425\nefficient data representations, 412\ngenerative stochastic network (GSN), 433\novercomplete, 424\nPCA with undercomplete linear autoen\u2010\ncoder, 413\nreconstructions, 413\nsparse, 426-428\nstacked, 415-424\nstacked convolutional, 433\nundercomplete, 413\nvariational, 428-432\nvisualizing features, 421-422\nwinner-take-all (WTA), 433\nautomatic differentiating, 231\nautonomous driving systems, 379\nAverage Absolute Deviation, 39\naverage pooling layer, 364\navg_pool(), 364\nB\nbackpropagation, 261-262, 275, 291, 422\nbackpropagation through time (BPTT), 389\nbagging and pasting, 185-188\nout-of-bag evaluation, 187-188\nin Scikit-Learn, 186-187\nbandwidth saturation, 349-351\nBasicLSTMCell, 401\nBasicRNNCell, 397-398\nBatch Gradient Descent, 114-117, 130\nbatch learning, 14-15\nBatch Normalization, 282-286, 374\noperation summary, 282\nwith TensorFlow, 284-286\nbatch(), 341\nbatch_join(), 341\nbatch_norm(), 284-285\nBellman Optimality Equation, 455", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 1377, "content": "batch(), 341\nbatch_join(), 341\nbatch_norm(), 284-285\nBellman Optimality Equation, 455\nbetween-graph replication, 344\nbias neurons, 258\nbias term, 106\nbias/variance tradeoff, 126\nbiases, 267\nbinary classifiers, 82, 134\nbiological neurons, 254-256\nblack box models, 170\nblending, 200-203\nBoltzmann Machines, 516-518\n(see also restricted Boltzman machines\n(RBMs))\nboosting, 191-200\nAdaBoost, 192-195\nGradient Boosting, 195-200\nbootstrap aggregation (see bagging)\nbootstrapping, 72, 185, 442, 469\nbottleneck layers, 369\nbrew, 202\nC\nCaffe model zoo, 291\ncall__(), 398\nCART (Classification and Regression Tree)\nalgorithm, 170-171, 176\ncategorical attributes, 62-64\ncell wrapper, 392\nchi square test, 174\nclassification versus regression, 8, 101\nclassifiers\nbinary, 82\nerror analysis, 96-99\nevaluating, 96\nMNIST dataset, 79-81\nmulticlass, 93-96\nmultilabel, 100-101\nmultioutput, 101-102\nperformance measures, 82-93\nprecision of, 85\nvoting, 181-184\nclip_by_value(), 286\nclosed-form equation, 105, 128, 136", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 1378, "content": "precision of, 85\nvoting, 181-184\nclip_by_value(), 286\nclosed-form equation, 105, 128, 136\ncluster specification, 324\nclustering algorithms, 10\nclusters, 323\ncoding space, 429\ncodings, 411\ncomplementary slackness condition, 504\ncomponents_, 214\ncomputational complexity, 110, 153, 172\ncompute_gradients(), 286, 449\nconcat(), 369\nconfig.gpu_options, 318\nConfigProto, 317\nconfusion matrix, 84-86, 96-99\nconnectionism, 260\nconstrained optimization, 158, 503\nContrastive Divergence, 519\n526 \n| \nIndex\nDownload from finelybook www.finelybook.com", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 1379, "content": "control dependencies, 323\nconv1d(), 376\nconv2d_transpose(), 376\nconv3d(), 376\nconvergence rate, 117\nconvex function, 113\nconvolution kernels, 357, 365, 370\nconvolutional neural networks (CNNs),\n353-378\narchitectures, 365-376\nAlexNet, 367-368\nGoogleNet, 368-372\nLeNet5, 366-367\nResNet, 372-375\nconvolutional layer, 355-363, 370, 376\nfeature maps, 358-360\nfilters, 357\nmemory requirement, 362-363\nevolution of, 354\npooling layer, 363-365\nTensorFlow implementation, 360-362\nCoordinator class, 338-340\ncorrelation coefficient, 55-58\ncorrelations, finding, 55-58\ncost function, 20, 39\nin AdaBoost, 193\nin adagrad, 297\nin artificial neural networks, 264, 267-268\nin autodiff, 238\nin batch normalization, 285\ncross entropy, 367\ndeep Q-Learning, 465\nin Elastic Net, 132\nin Gradient Descent, 105, 111-112, 114,\n117-119, 200, 275\nin Logistic Regression, 135-136\nin PG algorithms, 449\nin variational autoencoders, 430\nin Lasso Regression, 130-131\nin Linear Regression, 108, 113", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 1380, "content": "in variational autoencoders, 430\nin Lasso Regression, 130-131\nin Linear Regression, 108, 113\nin Momentum optimization, 294-295\nin pretrained layers reuse, 293\nin ridge regression, 127-129\nin RNNs, 389, 393\nstale gradients and, 349\ncreative sequences, 396\ncredit assignment problem, 447-448\ncritics, 463\ncross entropy, 140-141, 264, 428, 449\ncross-validation, 30, 69-71, 83-84\nCUDA library, 315\ncuDNN library, 315\ncurse of dimensionality, 205-207\n(see also dimensionality reduction)\ncustom transformers, 64-65\nD\ndata, 30\n(see also test data; training data)\ncreating workspace for, 40-43\ndownloading, 43-45\nfinding correlations in, 55-58\nmaking assumptions about, 30\npreparing for Machine Learning algorithms,\n59-68\ntest-set creation, 49-53\nworking with real data, 33\ndata augmentation, 309-310\ndata cleaning, 60-62\ndata mining, 6\ndata parallelism, 347-351\nasynchronous updates, 348-349\nbandwidth saturation, 349-351\nsynchronous updates, 348\nTensorFlow implementation, 351\ndata pipeline, 36", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 1381, "content": "synchronous updates, 348\nTensorFlow implementation, 351\ndata pipeline, 36\ndata snooping bias, 49\ndata structure, 45-48\ndata visualization, 53-55\nDataFrame, 60\ndataquest, xvi\ndecay, 284\ndecision boundaries, 136-139, 142, 170\ndecision function, 87, 156-157\nDecision Stumps, 195\ndecision threshold, 87\nDecision Trees, 69-70, 167-179, 181\nbinary trees, 170\nclass probability estimates, 171\ncomputational complexity, 172\ndecision boundaries, 170\nGINI impurity, 172\ninstability with, 177-178\nnumbers of children, 170\npredictions, 169-171\nRandom Forests (see Random Forests)\nregression tasks, 175-176\nregularization hyperparameters, 173-174\nIndex \n| \n527\nDownload from finelybook www.finelybook.com", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 1382, "content": "training and visualizing, 167-169\ndecoder, 412\ndeconvolutional layer, 376\ndeep autoencoders (see stacked autoencoders)\ndeep belief networks (DBNs), 13, 519-521\nDeep Learning, 437\n(see also Reinforcement Learning; Tensor\u2010\nFlow)\nabout, xiii, xvi\nlibraries, 230-231\ndeep neural networks (DNNs), 261, 275-312\n(see also Multi-Layer Perceptrons (MLP))\nfaster optimizers for, 293-302\nregularization, 302-310\nreusing pretrained layers, 286-293\ntraining guidelines overview, 310\ntraining with TensorFlow, 265-270\ntraining with TF.Learn, 264\nunstable gradients, 276\nvanishing and exploding gradients, 275-286\nDeep Q-Learning, 460-469\nMs. Pac Man example, 460-469\ndeep Q-network, 460\ndeep RNNs, 396-400\napplying dropout, 399\ndistributing across multiple GPUs, 397\nlong sequence difficulties, 400\ntruncated backpropagation through time,\n400\nDeepMind, 14, 253, 437, 460\ndegrees of freedom, 27, 126\ndenoising autoencoders, 424-425\ndepth concat layer, 369\ndepth radius, 368\ndepthwise_conv2d(), 376\ndequeue(), 332", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 1383, "content": "depth concat layer, 369\ndepth radius, 368\ndepthwise_conv2d(), 376\ndequeue(), 332\ndequeue_many(), 332, 334\ndequeue_up_to(), 333-334\ndequeuing data, 331\ndescribe(), 46\ndevice blocks, 327\ndevice(), 319\ndimensionality reduction, 12, 205-225, 411\napproaches to\nManifold Learning, 210\nprojection, 207-209\nchoosing the right number of dimensions,\n215\ncurse of dimensionality, 205-207\nand data visualization, 205\nIsomap, 224\nLLE (Locally Linear Embedding), 221-223\nMultidimensional Scaling, 223-224\nPCA (Principal Component Analysis),\n211-218\nt-Distributed Stochastic Neighbor Embed\u2010\nding (t-SNE), 224\ndiscount rate, 447\ndistributed computing, 229\ndistributed sessions, 328-329\nDNNClassifier, 264\ndrop(), 60\ndropconnect, 307\ndropna(), 60\ndropout, 272, 399\ndropout rate, 304\ndropout(), 306\nDropoutWrapper, 399\nDRY (Don\u2019t Repeat Yourself), 247\nDual Averaging, 300\ndual numbers, 510\ndual problem, 160\nduality, 503\ndying ReLUs, 279\ndynamic placements, 320\ndynamic placer, 318\nDynamic Programming, 456", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 1384, "content": "duality, 503\ndying ReLUs, 279\ndynamic placements, 320\ndynamic placer, 318\nDynamic Programming, 456\ndynamic unrolling through time, 387\ndynamic_rnn(), 387, 398, 409\nE\nearly stopping, 133-134, 198, 272, 303\nElastic Net, 132\nembedded device blocks, 327\nEmbedded Reber grammars, 410\nembeddings, 405-407\nembedding_lookup(), 406\nencoder, 412\nEncoder\u2013Decoder, 383\nend-of-sequence (EOS) token, 388\nenergy functions, 516\nenqueuing data, 330\nEnsemble Learning, 70, 74, 181-203\nbagging and pasting, 185-188\nboosting, 191-200\nin-graph versus between-graph replication,\n343-345\nRandom Forests, 189-191\n528 \n| \nIndex\nDownload from finelybook www.finelybook.com", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 1385, "content": "(see also Random Forests)\nrandom patches and random subspaces, 188\nstacking, 200-202\nentropy impurity measure, 172\nenvironments, in reinforcement learning,\n438-447, 459, 464\nepisodes (in RL), 444, 448-449, 451-452, 469\nepochs, 118\n\u03b5-insensitive, 155\nequality contraints, 504\nerror analysis, 96-99\nestimators, 61\nEuclidian norm, 39\neval(), 240\nevaluating models, 29-31\nexplained variance, 215\nexplained variance ratio, 214\nexploding gradients, 276\n(see also gradients, vanishing and explod\u2010\ning)\nexploration policies, 459\nexponential decay, 284\nexponential linear unit (ELU), 280-281\nexponential scheduling, 301\nExtra-Trees, 190\nF\nF-1 score, 86-87\nface-recognition, 100\nfake X server, 443\nfalse positive rate (FPR), 91-93\nfan-in, 277, 279\nfan-out, 277, 279\nfeature detection, 411\nfeature engineering, 25\nfeature extraction, 12\nfeature importance, 190-191\nfeature maps, 220, 357-360, 374\nfeature scaling, 65\nfeature selection, 26, 74, 130, 191, 499\nfeature space, 218, 220", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 1386, "content": "feature scaling, 65\nfeature selection, 26, 74, 130, 191, 499\nfeature space, 218, 220\nfeature vector, 39, 107, 156, 237\nfeatures, 9\nFeatureUnion, 66\nfeedforward neural network (FNN), 263\nfeed_dict, 240\nFIFOQueue, 330, 333\nfillna(), 60\nfirst-in first-out (FIFO) queues, 330\nfirst-order partial derivatives (Jacobians), 300\nfit(), 61, 66, 217\nfitness function, 20\nfit_inverse_transform=, 221\nfit_transform(), 61, 66\nfolds, 69, 81, 83-84\nFollow The Regularized Leader (FTRL), 300\nforget gate, 402\nforward-mode autodiff, 510-512\nframing a problem, 35-37\nfrozen layers, 289-290\nfully_connected(), 267, 278, 284-285, 417\nG\ngame play (see reinforcement learning)\ngamma value, 152\ngate controllers, 402\nGaussian distribution, 37, 429, 431\nGaussian RBF, 151\nGaussian RBF kernel, 152-153, 163\ngeneralization error, 29\ngeneralized Lagrangian, 504-505\ngenerative autoencoders, 428\ngenerative models, 411, 518\ngenetic algorithms, 440\ngeodesic distance, 224\nget_variable(), 249-250\nGINI impurity, 169, 172", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 1387, "content": "genetic algorithms, 440\ngeodesic distance, 224\nget_variable(), 249-250\nGINI impurity, 169, 172\nglobal average pooling, 372\nglobal_step, 466\nglobal_variables(), 308\nglobal_variables_initializer(), 233\nGlorot initialization, 276-279\nGoogle, 230\nGoogle Images, 253\nGoogle Photos, 13\nGoogleNet architecture, 368-372\ngpu_options.per_process_gpu_memory_frac\u2010\ntion, 317\ngradient ascent, 441\nGradient Boosted Regression Trees (GBRT),\n195\nGradient Boosting, 195-200\nGradient Descent (GD), 105, 111-121, 164, 275,\n294, 296\nalgorithm comparisons, 119-121\nautomatically computing gradients, 238-239\nBatch GD, 114-117, 130\ndefining, 111\nIndex \n| \n529\nDownload from finelybook www.finelybook.com", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 1388, "content": "local minimum versus global minimum, 112\nmanually computing gradients, 237\nMini-batch GD, 119-121, 239-241\noptimizer, 239\nStochastic GD, 117-119, 148\nwith TensorFlow, 237-239\nGradient Tree Boosting, 195\nGradientDescentOptimizer, 268\ngradients(), 238\ngradients, vanishing and exploding, 275-286,\n400\nBatch Normalization, 282-286\nGlorot and He initialization, 276-279\ngradient clipping, 286\nnonsaturating activation functions, 279-281\ngraphviz, 168\ngreedy algorithm, 172\ngrid search, 71-74, 151\ngroup(), 464\nGRU (Gated Recurrent Unit) cell, 404-405\nH\nhailstone sequence, 412\nhard margin classification, 146-147\nhard voting classifiers, 181-184\nharmonic mean, 86\nHe initialization, 276-279\nHeaviside step function, 257\nHebb's rule, 258, 516\nHebbian learning, 259\nhidden layers, 261\nhierarchical clustering, 10\nhinge loss function, 164\nhistograms, 47-48\nhold-out sets, 200\n(see also blenders)\nHopfield Networks, 515-516\nhyperbolic tangent (htan activation function),\n262, 272, 276, 278, 381", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 1389, "content": "Hopfield Networks, 515-516\nhyperbolic tangent (htan activation function),\n262, 272, 276, 278, 381\nhyperparameters, 28, 65, 72-74, 76, 111, 151,\n154, 270\n(see also neural network hyperparameters)\nhyperplane, 157, 210-211, 213, 224\nhypothesis, 39\nmanifold, 210\nhypothesis boosting (see boosting)\nhypothesis function, 107\nhypothesis, null, 174\nI\nidentity matrix, 128, 160\nILSVRC ImageNet challenge, 365\nimage classification, 365\nimpurity measures, 169, 172\nin-graph replication, 343\ninception modules, 369\nInception-v4, 375\nincremental learning, 16, 217\ninequality constraints, 504\ninference, 22, 311, 363, 408\ninfo(), 45\ninformation gain, 173\ninformation theory, 172\ninit node, 241\ninput gate, 402\ninput neurons, 258\ninput_put_keep_prob, 399\ninstance-based learning, 17, 21\nInteractiveSession, 233\nintercept term, 106\nInternal Covariate Shift problem, 282\ninter_op_parallelism_threads, 322\nintra_op_parallelism_threads, 322\ninverse_transform(), 221\nin_top_k(), 268\nirreducible error, 127", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 1390, "content": "intra_op_parallelism_threads, 322\ninverse_transform(), 221\nin_top_k(), 268\nirreducible error, 127\nisolated environment, 41-42\nIsomap, 224\nis_training, 284-285, 399\nJ\njobs, 323\njoin(), 325, 339\nJupyter, 40, 42, 48\nK\nK-fold cross-validation, 69-71, 83\nk-Nearest Neighbors, 21, 100\nKarush\u2013Kuhn\u2013Tucker (KKT) conditions, 504\nkeep probability, 306\nKeras, 231\nKernel PCA (kPCA), 218-221\nkernel trick, 150, 152, 161-164, 218\nkernelized SVM, 161-164\nkernels, 150-153, 321\nKullback\u2013Leibler divergence, 141, 426\n530 \n| \nIndex\nDownload from finelybook www.finelybook.com", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 1391, "content": "L\nl1_l2_regularizer(), 303\nLabelBinarizer, 66\nlabels, 8, 37\nLagrange function, 504-505\nLagrange multiplier, 503\nlandmarks, 151-152\nlarge margin classification, 145-146\nLasso Regression, 130-132\nlatent loss, 430\nlatent space, 429\nlaw of large numbers, 183\nleaky ReLU, 279\nlearning rate, 16, 111, 115-118\nlearning rate scheduling, 118, 300-302\nLeNet-5 architecture, 355, 366-367\nLevenshtein distance, 153\nliblinear library, 153\nlibsvm library, 154\nLinear Discriminant Analysis (LDA), 224\nlinear models\nearly stopping, 133-134\nElastic Net, 132\nLasso Regression, 130-132\nLinear Regression (see Linear Regression)\nregression (see Linear Regression)\nRidge Regression, 127-129, 132\nSVM, 145-148\nLinear Regression, 20, 68, 105-121, 132\ncomputational complexity, 110\nGradient Descent in, 111-121\nlearning curves in, 123-127\nNormal Equation, 108-110\nregularizing models (see regularization)\nusing Stochastic Gradient Descent (SGD),\n119\nwith TensorFlow, 235-236\nlinear SVM classification, 145-148", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 1392, "content": "119\nwith TensorFlow, 235-236\nlinear SVM classification, 145-148\nlinear threshold units (LTUs), 257\nLipschitz continuous, 113\nLLE (Locally Linear Embedding), 221-223\nload_sample_images(), 360\nlocal receptive field, 354\nlocal response normalization, 368\nlocal sessions, 328\nlocation invariance, 363\nlog loss, 136\nlogging placements, 320-320\nlogistic function, 134\nLogistic Regression, 9, 134-142\ndecision boundaries, 136-139\nestimating probablities, 134-135\nSoftmax Regression model, 139-142\ntraining and cost function, 135-136\nlog_device_placement, 320\nLSTM (Long Short-Term Memory) cell,\n401-405\nM\nmachine control (see reinforcement learning)\nMachine Learning\nlarge-scale projects (see TensorFlow)\nnotations, 38-39\nprocess example, 33-77\nproject checklist, 35, 497-502\nresources on, xvi-xvii\nuses for, xiii-xiv\nMachine Learning basics\nattributes, 9\nchallenges, 22-29\nalgorithm problems, 26-28\ntraining data problems, 25\ndefinition, 4\nfeatures, 9\noverview, 3\nreasons for using, 4-7", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 1393, "content": "training data problems, 25\ndefinition, 4\nfeatures, 9\noverview, 3\nreasons for using, 4-7\nspam filter example, 4-6\nsummary, 28\ntesting and validating, 29-31\ntypes of systems, 7-22\nbatch and online learning, 14-17\ninstance-based versus model-based\nlearning, 17-22\nsupervised/unsupervised learning, 8-14\nworkflow example, 18-22\nmachine translation (see natural language pro\u2010\ncessing (NLP))\nmake(), 442\nManhattan norm, 39\nmanifold assumption/hypothesis, 210\nManifold Learning, 210, 221\n(see also LLE (Locally Linear Embedding)\nMapReduce, 37\nmargin violations, 147\nMarkov chains, 453\nMarkov decision processes, 453-457\nmaster service, 325\nMatplotlib, 40, 48, 91, 97\nIndex \n| \n531\nDownload from finelybook www.finelybook.com", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 1394, "content": "max margin learning, 293\nmax pooling layer, 363\nmax-norm regularization, 307-308\nmax_norm(), 308\nmax_norm_regularizer(), 308\nmax_pool(), 364\nMean Absolute Error (MAE), 39-40\nmean coding, 429\nMean Square Error (MSE), 107, 237, 426\nmeasure of similarity, 17\nmemmap, 217\nmemory cells, 346, 382\nMercer's theorem, 163\nmeta learner (see blending)\nmin-max scaling, 65\nMini-batch Gradient Descent, 119-121, 136,\n239-241\nmini-batches, 15\nminimize(), 286, 289, 449, 466\nmin_after_dequeue, 333\nMNIST dataset, 79-81\nmodel parallelism, 345-347\nmodel parameters, 114, 116, 133, 156, 159, 234,\n268, 389\ndefining, 19\nmodel selection, 19\nmodel zoos, 291\nmodel-based learning, 18-22\nmodels\nanalyzing, 74-75\nevaluating on test set, 75-76\nmoments, 298\nMomentum optimization, 294-295\nMonte Carlo tree search, 453\nMulti-Layer Perceptrons (MLP), 253, 260-263,\n446\ntraining with TF.Learn, 264\nmulticlass classifiers, 93-96\nMultidimensional Scaling (MDS), 223\nmultilabel classifiers, 100-101", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 1395, "content": "multiclass classifiers, 93-96\nMultidimensional Scaling (MDS), 223\nmultilabel classifiers, 100-101\nMultinomial Logistic Regression (see Softmax\nRegression)\nmultinomial(), 446\nmultioutput classifiers, 101-102\nMultiRNNCell, 398\nmultithreaded readers, 338-340\nmultivariate regression, 37\nN\nnaive Bayes classifiers, 94\nname scopes, 245\nnatural language processing (NLP), 379,\n405-410\nencoder-decoder network for machine\ntranslation, 407-410\nTensorFlow tutorials, 405, 408\nword embeddings, 405-407\nNesterov Accelerated Gradient (NAG), 295-296\nNesterov momentum optimization, 295-296\nnetwork topology, 270\nneural network hyperparameters, 270-272\nactivation functions, 272\nneurons per hidden layer, 272\nnumber of hidden layers, 270-271\nneural network policies, 444-447\nneurons\nbiological, 254-256\nlogical computations with, 256\nneuron_layer(), 267\nnext_batch(), 269\nNo Free Lunch theorem, 30\nnode edges, 244\nnonlinear dimensionality reduction (NLDR),\n221\n(see also Kernel PCA; LLE (Locally Linear", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 1396, "content": "nonlinear dimensionality reduction (NLDR),\n221\n(see also Kernel PCA; LLE (Locally Linear\nEmbedding))\nnonlinear SVM classification, 149-154\ncomputational complexity, 153\nGaussian RBF kernel, 152-153\nwith polynomial features, 149-150\npolynomial kernel, 150-151\nsimilarity features, adding, 151-152\nnonparametric models, 173\nnonresponse bias, 25\nnonsaturating activation functions, 279-281\nnormal distribution (see Gaussian distribution)\nNormal Equation, 108-110\nnormalization, 65\nnormalized exponential, 139\nnorms, 39\nnotations, 38-39\nNP-Complete problems, 172\nnull hypothesis, 174\nnumerical differentiation, 509\nNumPy, 40\nNumPy arrays, 63\nNVidia Compute Capability, 314\n532 \n| \nIndex\nDownload from finelybook www.finelybook.com", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 1397, "content": "nvidia-smi, 318\nn_components, 215\nO\nobservation space, 446\noff-policy algorithm, 459\noffline learning, 14\none-hot encoding, 63\none-versus-all (OvA) strategy, 94, 141, 165\none-versus-one (OvO) strategy, 94\nonline learning, 15-17\nonline SVMs, 164-165\nOpenAI Gym, 441-444\noperation_timeout_in_ms, 345\nOptical Character Recognition (OCR), 3\noptimal state value, 455\noptimizers, 293-302\nAdaGrad, 296-298\nAdam optimization, 293, 298-300\nGradient Descent (see Gradient Descent\noptimizer)\nlearning rate scheduling, 300-302\nMomentum optimization, 294-295\nNesterov Accelerated Gradient (NAG),\n295-296\nRMSProp, 298\nout-of-bag evaluation, 187-188\nout-of-core learning, 16\nout-of-memory (OOM) errors, 386\nout-of-sample error, 29\nOutOfRangeError, 337, 339\noutput gate, 402\noutput layer, 261\nOutputProjectionWrapper, 392-395\noutput_put_keep_prob, 399\novercomplete autoencoder, 424\noverfitting, 26-28, 49, 147, 152, 173, 176, 272\navoiding through regularization, 302-310\nP\np-value, 174\nPaddingFIFOQueue, 334", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 1398, "content": "avoiding through regularization, 302-310\nP\np-value, 174\nPaddingFIFOQueue, 334\nPandas, 40, 44\nscatter_matrix, 56-57\nparallel distributed computing, 313-352\ndata parallelism, 347-351\nin-graph versus between-graph replication,\n343-345\nmodel parallelism, 345-347\nmultiple devices across multiple servers,\n323-342\nasynchronous communication using\nqueues, 329-334\nloading training data, 335-342\nmaster and worker services, 325\nopening a session, 325\npinning operations across tasks, 326\nsharding variables, 327\nsharing state across sessions, 328-329\nmultiple devices on a single machine,\n314-323\ncontrol dependencies, 323\ninstallation, 314-316\nmanaging the GPU RAM, 317-318\nparallel execution, 321-322\nplacing operations on devices, 318-321\none neural network per device, 342-343\nparameter efficiency, 271\nparameter matrix, 139\nparameter server (ps), 324\nparameter space, 114\nparameter vector, 107, 111, 135, 139\nparametric models, 173\npartial derivative, 114\npartial_fit(), 217\nPearson's r, 55", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 1399, "content": "parametric models, 173\npartial derivative, 114\npartial_fit(), 217\nPearson's r, 55\npeephole connections, 403\npenalties (see rewards, in RL)\npercentiles, 46\nPerceptron convergence theorem, 259\nPerceptrons, 257-264\nversus Logistic Regression, 260\ntraining, 258-259\nperformance measures, 37-40\nconfusion matrix, 84-86\ncross-validation, 83-84\nprecision and recall, 86-90\nROC (receiver operating characteristic)\ncurve, 91-93\nperformance scheduling, 301\npermutation(), 49\nPG algorithms, 448\nphoto-hosting services, 13\npinning operations, 326\npip, 41\nPipeline constructor, 66-68\npipelines, 36\nplaceholder nodes, 239\nIndex \n| \n533\nDownload from finelybook www.finelybook.com", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 1400, "content": "placers (see simple placer; dynamic placer)\npolicy, 440\npolicy gradients, 441 (see PG algorithms)\npolicy space, 440\npolynomial features, adding, 149-150\npolynomial kernel, 150-151, 162\nPolynomial Regression, 106, 121-123\nlearning curves in, 123-127\npooling kernel, 363\npooling layer, 363-365\npower scheduling, 301\nprecision, 85\nprecision and recall, 86-90\nF-1 score, 86-87\nprecision/recall (PR) curve, 92\nprecision/recall tradeoff, 87-90\npredetermined piecewise constant learning\nrate, 301\npredict(), 62\npredicted class, 85\npredictions, 84-86, 156-157, 169-171\npredictors, 8, 62\npreloading training data, 335\nPReLU (parametric leaky ReLU), 279\npreprocessed attributes, 48\npretrained layers reuse, 286-293\nauxiliary task, 292-293\ncaching frozen layers, 290\nfreezing lower layers, 289\nmodel zoos, 291\nother frameworks, 288\nTensorFlow model, 287-288\nunsupervised pretraining, 291-292\nupper layers, 290\nPretty Tensor, 231\nprimal problem, 160\nprincipal component, 212", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 1401, "content": "upper layers, 290\nPretty Tensor, 231\nprimal problem, 160\nprincipal component, 212\nPrincipal Component Analysis (PCA), 211-218\nexplained variance ratios, 214\nfinding principal components, 212-213\nfor compression, 216-217\nIncremental PCA, 217-218\nKernel PCA (kPCA), 218-221\nprojecting down to d dimensions, 213\nRandomized PCA, 218\nScikit Learn for, 214\nvariance, preserving, 211-212\nprobabilistic autoencoders, 428\nprobabilities, estimating, 134-135, 171\nproducer functions, 341\nprojection, 207-209\npropositional logic, 254\npruning, 174, 509\nPython\nisolated environment in, 41-42\nnotebooks in, 42-43\npickle, 71\npip, 41\nQ\nQ-Learning algorithm, 458-469\napproximate Q-Learning, 460\ndeep Q-Learning, 460-469\nQ-Value Iteration Algorithm, 456\nQ-Values, 456\nQuadratic Programming (QP) Problems,\n159-160\nquantizing, 351\nqueries per second (QPS), 343\nQueueRunner, 338-340\nqueues, 329-334\nclosing, 333\ndequeuing data, 331\nenqueuing data, 330\nfirst-in first-out (FIFO), 330\nof tuples, 332\nPaddingFIFOQueue, 334", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 1402, "content": "enqueuing data, 330\nfirst-in first-out (FIFO), 330\nof tuples, 332\nPaddingFIFOQueue, 334\nRandomShuffleQueue, 333\nq_network(), 463\nR\nRadial Basis Function (RBF), 151\nRandom Forests, 70-72, 94, 167, 178, 181,\n189-191\nExtra-Trees, 190\nfeature importance, 190-191\nrandom initialization, 111, 116, 118, 276\nRandom Patches and Random Subspaces, 188\nrandomized leaky ReLU (RReLU), 279\nRandomized PCA, 218\nrandomized search, 74, 270\nRandomShuffleQueue, 333, 337\nrandom_uniform(), 237\nreader operations, 335\nrecall, 85\nrecognition network, 412\nreconstruction error, 216\nreconstruction loss, 413, 428, 430\n534 \n| \nIndex\nDownload from finelybook www.finelybook.com", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 1403, "content": "reconstruction pre-image, 220\nreconstructions, 413\nrecurrent neural networks (RNNs), 379-410\ndeep RNNs, 396-400\nexploration policies, 459\nGRU cell, 404-405\ninput and output sequences, 382-383\nLSTM cell, 401-405\nnatural language processing (NLP), 405-410\nin TensorFlow, 384-388\ndynamic unrolling through time, 387\nstatic unrolling through time, 385-386\nvariable length input sequences, 387\nvariable length output sequences, 388\ntraining, 389-396\nbackpropagation through time (BPTT),\n389\ncreative sequences, 396\nsequence classifiers, 389-391\ntime series predictions, 392-396\nrecurrent neurons, 380-383\nmemory cells, 382\nreduce_mean(), 268\nreduce_sum(), 427-428, 430, 466\nregression, 8\nDecision Trees, 175-176\nregression models\nlinear, 68\nregression versus classification, 101\nregularization, 27-28, 30, 127-134\ndata augmentation, 309-310\nDecision Trees, 173-174\ndropout, 304-307\nearly stopping, 133-134, 303\nElastic Net, 132\nLasso Regression, 130-132\nmax-norm, 307-308\nRidge Regression, 127-129", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 1404, "content": "Elastic Net, 132\nLasso Regression, 130-132\nmax-norm, 307-308\nRidge Regression, 127-129\nshrinkage, 197\n\u2113 1 and \u2113 2 regularization, 303-304\nREINFORCE algorithms, 448\nReinforcement Learning (RL), 13-14, 437-470\nactions, 447-448\ncredit assignment problem, 447-448\ndiscount rate, 447\nexamples of, 438\nMarkov decision processes, 453-457\nneural network policies, 444-447\nOpenAI gym, 441-444\nPG algorithms, 448-453\npolicy search, 440-441\nQ-Learning algorithm, 458-469\nrewards, learning to optimize, 438-439\nTemporal Difference (TD) Learning,\n457-458\nReLU (rectified linear units), 246-248\nReLU activation, 374\nReLU function, 262, 272, 278-281\nrelu(z), 266\nrender(), 442\nreplay memory, 464\nreplica_device_setter(), 327\nrequest_stop(), 339\nreset(), 442\nreset_default_graph(), 234\nreshape(), 395\nresidual errors, 195-196\nresidual learning, 372\nresidual network (ResNet), 291, 372-375\nresidual units, 373\nResNet, 372-375\nresource containers, 328-329\nrestore(), 241\nrestricted Boltzmann machines (RBMs), 13,", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 1405, "content": "resource containers, 328-329\nrestore(), 241\nrestricted Boltzmann machines (RBMs), 13,\n291, 518\nreuse_variables(), 249\nreverse-mode autodiff, 512-513\nrewards, in RL, 438-439\nrgb_array, 443\nRidge Regression, 127-129, 132\nRMSProp, 298\nROC (receiver operating characteristic) curve,\n91-93\nRoot Mean Square Error (RMSE), 37-40, 107\nRReLU (randomized leaky ReLU), 279\nrun(), 233, 345\nS\nSampled Softmax, 409\nsampling bias, 24-25, 51\nsampling noise, 24\nsave(), 241\nSaver node, 241\nScikit Flow, 231\nScikit-Learn, 40\nabout, xiv\nbagging and pasting in, 186-187\nCART algorithm, 170-171, 176\nIndex \n| \n535\nDownload from finelybook www.finelybook.com", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 1406, "content": "cross-validation, 69-71\ndesign principles, 61-62\nimputer, 60-62\nLinearSVR class, 156\nMinMaxScaler, 65\nmin_ and max_ hyperparameters, 173\nPCA implementation, 214\nPerceptron class, 259\nPipeline constructor, 66-68, 149\nRandomized PCA, 218\nRidge Regression with, 129\nSAMME, 195\nSGDClassifier, 82, 87-88, 94\nSGDRegressor, 119\nsklearn.base.BaseEstimator, 64, 67, 84\nsklearn.base.clone(), 83, 133\nsklearn.base.TransformerMixin, 64, 67\nsklearn.datasets.fetch_california_housing(),\n236\nsklearn.datasets.fetch_mldata(), 79\nsklearn.datasets.load_iris(), 137, 148, 167,\n190, 259\nsklearn.datasets.load_sample_images(),\n360-361\nsklearn.datasets.make_moons(), 149, 178\nsklearn.decomposition.IncrementalPCA,\n217\nsklearn.decomposition.KernelPCA,\n218-219, 221\nsklearn.decomposition.PCA, 214\nsklearn.ensemble.AdaBoostClassifier, 195\nsklearn.ensemble.BaggingClassifier, 186-189\nsklearn.ensemble.GradientBoostingRegres\u2010\nsor, 196, 198-199\nsklearn.ensemble.RandomForestClassifier,\n92, 95, 184", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 1407, "content": "sor, 196, 198-199\nsklearn.ensemble.RandomForestClassifier,\n92, 95, 184\nsklearn.ensemble.RandomForestRegressor,\n70, 72-74, 189-190, 196\nsklearn.ensemble.VotingClassifier, 184\nsklearn.externals.joblib, 71\nsklearn.linear_model.ElasticNet, 132\nsklearn.linear_model.Lasso, 132\nsklearn.linear_model.LinearRegression,\n20-21, 62, 68, 110, 120, 122, 124-125\nsklearn.linear_model.LogisticRegression,\n137, 139, 141, 184, 219\nsklearn.linear_model.Perceptron, 259\nsklearn.linear_model.Ridge, 129\nsklearn.linear_model.SGDClassifier, 82\nsklearn.linear_model.SGDRegressor,\n119-120, 129, 132-133\nsklearn.manifold.LocallyLinearEmbedding,\n221-222\nsklearn.metrics.accuracy_score(), 184, 188,\n264\nsklearn.metrics.confusion_matrix(), 85, 96\nsklearn.metrics.f1_score(), 87, 100\nsklearn.metrics.mean_squared_error(),\n68-69, 76, 124, 133, 198-199, 221\nsklearn.metrics.precision_recall_curve(), 88\nsklearn.metrics.precision_score(), 86, 90\nsklearn.metrics.recall_score(), 86, 90\nsklearn.metrics.roc_auc_score(), 92-93", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 1408, "content": "sklearn.metrics.recall_score(), 86, 90\nsklearn.metrics.roc_auc_score(), 92-93\nsklearn.metrics.roc_curve(), 91-92\nsklearn.model_selection.cross_val_pre\u2010\ndict(), 84, 88, 92, 96, 100\nsklearn.model_selection.cross_val_score(),\n69-70, 83-84\nsklearn.model_selection.GridSearchCV,\n72-74, 77, 96, 179, 219\nsklearn.model_selection.StratifiedKFold, 83\nsklearn.model_selection.StratifiedShuffleS\u2010\nplit, 52\nsklearn.model_selection.train_test_split(),\n50, 69, 124, 178, 198\nsklearn.multiclass.OneVsOneClassifier, 95\nsklearn.neighbors.KNeighborsClassifier,\n100, 102\nsklearn.neighbors.KNeighborsRegressor, 22\nsklearn.pipeline.FeatureUnion, 66\nsklearn.pipeline.Pipeline, 66, 125, 148-149,\n219\nsklearn.preprocessing.Imputer, 60, 66\nsklearn.preprocessing.LabelBinarizer, 64, 66\nsklearn.preprocessing.LabelEncoder, 62\nsklearn.preprocessing.OneHotEncoder, 63\nsklearn.preprocessing.PolynomialFeatures,\n122-123, 125, 128, 149\nsklearn.preprocessing.StandardScaler,\n65-66, 96, 114, 128, 146, 148-150, 152,\n237, 264", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 1409, "content": "sklearn.preprocessing.StandardScaler,\n65-66, 96, 114, 128, 146, 148-150, 152,\n237, 264\nsklearn.svm.LinearSVC, 147-149, 153-154,\n156, 165\nsklearn.svm.LinearSVR, 155-156\nsklearn.svm.SVC, 148, 150, 152-154, 156,\n165, 184\nsklearn.svm.SVR, 77, 156\n536 \n| \nIndex\nDownload from finelybook www.finelybook.com", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 1410, "content": "sklearn.tree.DecisionTreeClassifier, 173,\n179, 186-187, 189, 195\nsklearn.tree.DecisionTreeRegressor, 69, 167,\n175, 195-196\nsklearn.tree.export_graphviz(), 168\nStandardScaler, 114, 237, 264\nSVM classification classes, 154\nTF.Learn, 231\nuser guide, xvi\nscore(), 62\nsearch space, 74, 270\nsecond-order partial derivatives (Hessians), 300\nself-organizing maps (SOMs), 521-523\nsemantic hashing, 434\nsemisupervised learning, 13\nsensitivity, 85, 91\nsentiment analysis, 379\nseparable_conv2d(), 376\nsequences, 379\nsequence_length, 387-388, 409\nShannon's information theory, 172\nshortcut connections, 372\nshow(), 48\nshow_graph(), 245\nshrinkage, 197\nshuffle_batch(), 341\nshuffle_batch_join(), 341\nsigmoid function, 134\nsigmoid_cross_entropy_with_logits(), 428\nsimilarity function, 151-152\nsimulated annealing, 118\nsimulated environments, 442\n(see also OpenAI Gym)\nSingular Value Decomposition (SVD), 213\nskewed datasets, 84\nskip connections, 310, 372\nslack variable, 158\nsmoothing terms, 283, 297, 299, 430", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 1411, "content": "skip connections, 310, 372\nslack variable, 158\nsmoothing terms, 283, 297, 299, 430\nsoft margin classification, 146-148\nsoft placements, 321\nsoft voting, 184\nsoftmax function, 139, 263, 264\nSoftmax Regression, 139-142\nsource ops, 236, 322\nspam filters, 3-6, 8\nsparse autoencoders, 426-428\nsparse matrix, 63\nsparse models, 130, 300\nsparse_softmax_cross_entropy_with_logits(),\n268\nsparsity loss, 426\nspecificity, 91\nspeech recognition, 6\nspurious patterns, 516\nstack(), 385\nstacked autoencoders, 415-424\nTensorFlow implementation, 416\ntraining one-at-a-time, 418-420\ntying weights, 417-418\nunsupervised pretraining with, 422-424\nvisualizing the reconstructions, 420-421\nstacked denoising autoencoders, 422, 424\nstacked denoising encoders, 424\nstacked generalization (see stacking)\nstacking, 200-202\nstale gradients, 348\nstandard correlation coefficient, 55\nstandard deviation, 37\nstandardization, 65\nStandardScaler, 66, 237, 264\nstate-action values, 456\nstates tensor, 388\nstate_is_tuple, 398, 401", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 1412, "content": "StandardScaler, 66, 237, 264\nstate-action values, 456\nstates tensor, 388\nstate_is_tuple, 398, 401\nstatic unrolling through time, 385-386\nstatic_rnn(), 385-386, 409\nstationary point, 503-505\nstatistical mode, 185\nstatistical significance, 174\nstemming, 103\nstep functions, 257\nstep(), 443\nStochastic Gradient Boosting, 199\nStochastic Gradient Descent (SGD), 117-119,\n148, 260\ntraining, 136\nStochastic Gradient Descent (SGD) classifier,\n82, 129\nstochastic neurons, 516\nstochastic policy, 440\nstratified sampling, 51-53, 83\nstride, 357\nstring kernels, 153\nstring_input_producer(), 341\nstrong learners, 182\nsubderivatives, 164\nsubgradient vector, 131\nsubsample, 199, 363\nIndex \n| \n537\nDownload from finelybook www.finelybook.com", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 1413, "content": "supervised learning, 8-9\nSupport Vector Machines (SVMs), 94, 145-166\ndecision function and predictions, 156-157\ndual problem, 503-505\nkernelized SVM, 161-164\nlinear classification, 145-148\nmechanics of, 156-165\nnonlinear classification, 149-154\nonline SVMs, 164-165\nQuadratic Programming (QP) problems,\n159-160\nSVM regression, 154-165\nthe dual problem, 160\ntraining objective, 157-159\nsupport vectors, 146\nsvd(), 213\nsymbolic differentiation, 238, 508-509\nsynchronous updates, 348\nT\nt-Distributed Stochastic Neighbor Embedding\n(t-SNE), 224\ntail heavy, 48\ntarget attributes, 48\ntarget_weights, 409\ntasks, 323\nTemporal Difference (TD) Learning, 457-458\ntensor processing units (TPUs), 315\nTensorBoard, 231\nTensorFlow, 229-252\nabout, xiv\nautodiff, 238-239, 507-513\nBatch Normalization with, 284-286\nconstruction phase, 234\ncontrol dependencies, 323\nconvenience functions, 341\nconvolutional layers, 376\nconvolutional neural networks and, 360-362\ndata parallelism and, 351", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 1414, "content": "convolutional layers, 376\nconvolutional neural networks and, 360-362\ndata parallelism and, 351\ndenoising autoencoders, 425-425\ndropout with, 306\ndynamic placer, 318\nexecution phase, 234\nfeeding data to the training algorithm,\n239-241\nGradient Descent with, 237-239\ngraphs, managing, 234\ninitial graph creation and session run,\n232-234\ninstallation, 232\nl1 and l2 regularization with, 303\nlearning schedules in, 302\nLinear Regression with, 235-236\nmax pooling layer in, 364\nmax-norm regularization with, 307\nmodel zoo, 291\nmodularity, 246-248\nMomentum optimization in, 295\nname scopes, 245\nneural network policies, 446\nNLP tutorials, 405, 408\nnode value lifecycle, 235\noperations (ops), 235\noptimizer, 239\noverview, 229-231\nparallel distributed computing (see parallel\ndistributed computing with TensorFlow)\nPython API\nconstruction, 265-269\nexecution, 269\nusing the neural network, 270\nqueues (see queues)\nreusing pretrained layers, 287-288\nRNNs in, 384-388\n(see also recurrent neural networks", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 1415, "content": "reusing pretrained layers, 287-288\nRNNs in, 384-388\n(see also recurrent neural networks\n(RNNs))\nsaving and restoring models, 241-242\nsharing variables, 248-251\nsimple placer, 318\nsklearn.metrics.accuracy_score(), 286\nsparse autoencoders with, 427\nand stacked autoencoders, 416\nTensorBoard, 242-245\ntf.abs(), 303\ntf.add(), 246, 303-304\ntf.add_n(), 247-248, 250-251\ntf.add_to_collection(), 308\ntf.assign(), 237, 288, 307-308, 482\ntf.bfloat16, 350\ntf.bool, 284, 306\ntf.cast(), 268, 391\ntf.clip_by_norm(), 307-308\ntf.clip_by_value(), 286\ntf.concat(), 312, 369, 446, 450\ntf.ConfigProto, 317, 320-321, 345, 487\ntf.constant(), 235-237, 319-320, 323,\n325-326\ntf.constant_initializer(), 249-251\n538 \n| \nIndex\nDownload from finelybook www.finelybook.com", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 1416, "content": "tf.container(), 328-330, 351-352, 481\ntf.contrib.framework.arg_scope(), 285, 416,\n430\ntf.contrib.layers.batch_norm(), 284-285\ntf.contrib.layers.convolution2d(), 463\ntf.contrib.layers.fully_connected(), 267\ntf.contrib.layers.l1_regularizer(), 303, 308\ntf.contrib.layers.l2_regularizer(), 303,\n416-417\ntf.contrib.layers.variance_scaling_initial\u2010\nizer(), 278-279, 391, 416-417, 430, 446,\n450, 463\ntf.contrib.learn.DNNClassifier, 264\ntf.contrib.learn.infer_real_valued_col\u2010\numns_from_input(), 264\ntf.contrib.rnn.BasicLSTMCell, 401, 403\ntf.contrib.rnn.BasicRNNCell, 385-387, 390,\n392-393, 395, 397-399, 401\ntf.contrib.rnn.DropoutWrapper, 399\ntf.contrib.rnn.GRUCell, 405\ntf.contrib.rnn.LSTMCell, 403\ntf.contrib.rnn.MultiRNNCell, 397-399\ntf.contrib.rnn.OutputProjectionWrapper,\n392-394\ntf.contrib.rnn.RNNCell, 398\ntf.contrib.rnn.static_rnn(), 385-387,\n409-410, 491-492\ntf.contrib.slim module, 231, 377\ntf.contrib.slim.nets module (nets), 377\ntf.control_dependencies(), 323\ntf.decode_csv(), 336, 340", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 1417, "content": "tf.contrib.slim.nets module (nets), 377\ntf.control_dependencies(), 323\ntf.decode_csv(), 336, 340\ntf.device(), 319-321, 326-327, 397-398\ntf.exp(), 430-431\ntf.FIFOQueue, 330, 332-333, 336, 340\ntf.float32, 236, 482\ntf.get_collection(), 288-289, 304, 308, 416,\n463\ntf.get_default_graph(), 234, 242\ntf.get_default_session(), 233\ntf.get_variable(), 249-251, 288, 303-308\ntf.global_variables(), 308\ntf.global_variables_initializer(), 233, 237\ntf.gradients(), 238\ntf.Graph, 232, 234, 242, 335, 343\ntf.GraphKeys.REGULARIZATION_LOS\u2010\nSES, 304, 416\ntf.GraphKeys.TRAINABLE_VARIABLES,\n288-289, 463\ntf.group(), 464\ntf.int32, 321-332, 337, 387, 390, 406, 466\ntf.int64, 265\ntf.InteractiveSession, 233\nTF.Learn, 264\ntf.log(), 427, 430, 446, 450\ntf.matmul(), 236-237, 246, 265, 384, 417,\n420, 425, 427-428\ntf.matrix_inverse(), 236\ntf.maximum(), 246, 248-251, 281\ntf.multinomial(), 446, 450\ntf.name_scope(), 245, 248-249, 265,\n267-268, 419-420\ntf.nn.conv2d(), 360-361\ntf.nn.dynamic_rnn(), 386-387, 390, 392,", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 1418, "content": "267-268, 419-420\ntf.nn.conv2d(), 360-361\ntf.nn.dynamic_rnn(), 386-387, 390, 392,\n395, 397-399, 409-410, 491-492\ntf.nn.elu(), 281, 416-417, 430, 446, 450\ntf.nn.embedding_lookup(), 406\ntf.nn.in_top_k(), 268, 391\ntf.nn.max_pool(), 364-365\ntf.nn.relu(), 265, 392-393, 395, 463\ntf.nn.sigmoid_cross_entropy_with_logits(),\n428, 431, 449-450\ntf.nn.sparse_soft\u2010\nmax_cross_entropy_with_logits(),\n267-268, 390\ntf.one_hot(), 466\ntf.PaddingFIFOQueue, 334\ntf.placeholder(), 239-240, 482\ntf.placeholder_with_default(), 425\ntf.RandomShuffleQueue, 333, 337-338,\n340-341\ntf.random_normal(), 246, 384, 425, 430\ntf.random_uniform(), 237, 241, 406, 482\ntf.reduce_mean(), 237, 245, 267-268, 303,\n390-391, 414, 416, 418, 420, 425, 427,\n466\ntf.reduce_sum(), 303, 427-428, 430-431,\n465-466\ntf.reset_default_graph(), 234\ntf.reshape(), 395, 463\ntf.RunOptions, 345\ntf.Session, 233, 482\ntf.shape(), 425, 430\ntf.square(), 237, 245, 393, 414, 416, 418, 420,\n425, 427, 430-431, 466\ntf.stack(), 336, 340, 386\ntf.string, 336, 340", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 1419, "content": "425, 427, 430-431, 466\ntf.stack(), 336, 340, 386\ntf.string, 336, 340\ntf.summary.FileWriter, 242-243\ntf.summary.scalar(), 242\nIndex \n| \n539\nDownload from finelybook www.finelybook.com", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 1420, "content": "tf.tanh(), 384\ntf.TextLineReader, 336, 340\ntf.to_float(), 449-450\ntf.train.AdamOptimizer, 293, 299, 390, 393,\n414, 416-417, 419, 427, 431, 449-450, 466\ntf.train.ClusterSpec, 324\ntf.train.Coordinator, 338-340\ntf.train.exponential_decay(), 302\ntf.train.GradientDescentOptimizer, 239,\n268, 286, 293, 295\ntf.train.MomentumOptimizer, 239, 295-296,\n302, 311, 351, 485-486\ntf.train.QueueRunner, 338-341\ntf.train.replica_device_setter(), 327-328\ntf.train.RMSPropOptimizer, 298\ntf.train.Saver, 241-242, 268, 377, 399, 450,\n466\ntf.train.Server, 324\ntf.train.start_queue_runners(), 341\ntf.transpose(), 236-237, 386, 417\ntf.truncated_normal(), 265\ntf.unstack(), 385-387, 395, 492\ntf.Variable, 232, 482\ntf.variable_scope(), 249-251, 288, 307-308,\n328, 391, 463\ntf.zeros(), 265, 384, 417\ntruncated backpropagation through time,\n400\nvisualizing graph and training curves,\n242-245\nTensorFlow Serving, 343\ntensorflow.contrib, 267\ntest set, 29, 49-53, 81\ntesting and validating, 29-31\ntext attributes, 62-64", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 1421, "content": "test set, 29, 49-53, 81\ntesting and validating, 29-31\ntext attributes, 62-64\nTextLineReader, 336\nTF-slim, 231\nTF.Learn, 231, 264\nthermal equilibrium, 518\nthread pools (inter-op/intra-op, in TensorFlow,\n322\nthreshold variable, 248-251\nTikhonov regularization, 127\ntime series data, 379\ntoarray(), 63\ntolerance hyperparameter, 154\ntrainable, 288\ntraining data, 4\ninsufficient quantities, 22\nirrelevant features, 25\nloading, 335-342\nnonrepresentative, 24\noverfitting, 26-28\npoor quality, 25\nunderfitting, 28\ntraining instance, 4\ntraining models, 20, 105-143\nlearning curves in, 123-127\nLinear Regression, 105, 106-121\nLogistic Regression, 134-142\noverview, 105-106\nPolynomial Regression, 106, 121-123\ntraining objectives, 157-159\ntraining set, 4, 29, 53, 60, 68-69\ncost function of, 135-136\nshuffling, 81\ntransfer learning, 286-293\n(see also pretrained layers reuse)\ntransform(), 61, 66\ntransformation pipelines, 66-68\ntransformers, 61\ntransformers, custom, 64-65\ntranspose(), 385", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 1422, "content": "transformation pipelines, 66-68\ntransformers, 61\ntransformers, custom, 64-65\ntranspose(), 385\ntrue negative rate (TNR), 91\ntrue positive rate (TPR), 85, 91\ntruncated backpropagation through time, 400\ntuples, 332\ntying weights, 417\nU\nunderfitting, 28, 68, 152\nunivariate regression, 37\nunstack(), 385\nunsupervised learning, 10-12\nanomaly detection, 12\nassociation rule learning, 10, 12\nclustering, 10\ndimensionality reduction algorithm, 12\nvisualization algorithms, 11\nunsupervised pretraining, 291-292, 422-424\nupsampling, 376\nutility function, 20\nV\nvalidation set, 30\nValue Iteration, 455\nvalue_counts(), 46\nvanishing gradients, 276\n540 \n| \nIndex\nDownload from finelybook www.finelybook.com", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 1423, "content": "(see also gradients, vanishing and explod\u2010\ning)\nvariables, sharing, 248-251\nvariable_scope(), 249-250\nvariance\nbias/variance tradeoff, 126\nvariance preservation, 211-212\nvariance_scaling_initializer(), 278\nvariational autoencoders, 428-432\nVGGNet, 375\nvisual cortex, 354\nvisualization, 242-245\nvisualization algorithms, 11-12\nvoice recognition, 353\nvoting classifiers, 181-184\nW\nwarmup phase, 349\nweak learners, 182\nweight-tying, 417\nweights, 267, 288\nfreezing, 289\nwhile_loop(), 387\nwhite box models, 170\nworker, 324\nworker service, 325\nworker_device, 327\nworkspace directory, 40-43\nX\nXavier initialization, 276-279\nY\nYouTube, 253\nZ\nzero padding, 356, 361\nIndex \n| \n541\nDownload from finelybook www.finelybook.com", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 1424, "content": "About the Author\nAur\u00e9lien G\u00e9ron is a Machine Learning consultant. A former Googler, he led the You\u2010\nTube video classification team from 2013 to 2016. He was also a founder and CTO of\nWifirst from 2002 to 2012, a leading Wireless ISP in France; and a founder and CTO\nof Polyconseil in 2001, the firm that now manages the electric car sharing service\nAutolib\u2019.\nBefore this he worked as an engineer in a variety of domains: finance (JP Morgan and\nSoci\u00e9t\u00e9 G\u00e9n\u00e9rale), defense (Canada\u2019s DOD), and healthcare (blood transfusion). He\npublished a few technical books (on C++, WiFi, and internet architectures), and was\na Computer Science lecturer in a French engineering school.\nA few fun facts: he taught his three children to count in binary with their fingers (up\nto 1023), he studied microbiology and evolutionary genetics before going into soft\u2010\nware engineering, and his parachute didn\u2019t open on the second jump.\nColophon\nThe animal on the cover of Hands-On Machine Learning with Scikit-Learn and Ten\u2010", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 1425, "content": "Colophon\nThe animal on the cover of Hands-On Machine Learning with Scikit-Learn and Ten\u2010\nsorFlow is the far eastern fire salamander (Salamandra infraimmaculata), an amphib\u2010\nian found in the Middle East. They have black skin featuring large yellow spots on\ntheir back and head. These spots are a warning coloration meant to keep predators at\nbay. Full-grown salamanders can be over a foot in length.\nFar eastern fire salamanders live in subtropical shrubland and forests near rivers or\nother freshwater bodies. They spend most of their life on land, but lay their eggs in\nthe water. They subsist mostly on a diet of insects, worms, and small crustaceans, but\noccasionally eat other salamanders. Males of the species have been known to live up\nto 23 years, while females can live up to 21 years.\nAlthough not yet endangered, the far eastern fire salamander population is in decline.\nPrimary threats include damming of rivers (which disrupts the salamander\u2019s breed\u2010", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}} {"index": 1426, "content": "Primary threats include damming of rivers (which disrupts the salamander\u2019s breed\u2010\ning) and pollution. They are also threatened by the recent introduction of predatory\nfish, such as the mosquitofish. These fish were intended to control the mosquito pop\u2010\nulation, but they also feed on young salamanders.\nMany of the animals on O\u2019Reilly covers are endangered; all of them are important to\nthe world. To learn more about how you can help, go to animals.oreilly.com.\nThe cover image is from Wood\u2019s Illustrated Natural History. The cover fonts are URW\nTypewriter and Guardian Sans. The text font is Adobe Minion Pro; the heading font\nis Adobe Myriad Condensed; and the code font is Dalton Maag\u2019s Ubuntu Mono.\nDownload from finelybook www.finelybook.com", "metadata": {"source": "Hands on Machine Learning with Scikit Learn and TensorFlow.pdf"}}