Machine-Learning.md

Machine Learning Resources

General Machine Learning

Official Documentation and Tutorials

• Scikit-Learn Official Documentation
  • Scikit-learn Tutorials
    • An Introduction to Machine Learning with Scikit-learn
    • A Tutorial on Statistical-Learning for Scientific Data Processing
  • Scikit-learn User Guide
  • Scikit-Learn Examples
  • Scikit-Learn - Choosing the Right Estimator - a flowchart designed to give users a bit of a rough guide on how to approach problems with regard to which estimators to try on your data.
  • Sklearn Documentation on Feature Selection
  • Scikit-Learn - Model Evaluation - quantifying the quality of predictions.

Other Tutorials and Guides

• scikit-learn Beginner Tutorials - abridged implementation of the official scikit-learn beginner tutorials.
• Scikit-learn Tutorial - this repository contains notebooks and other files associated with Jake Vanderplas's Scikit-learn tutorial.
  • Index Notebook on GitHub
  • Index Notebook on nbviewer
• Chris Albon's Machine Learning Tutorials - detailed guides on how to do everything data science and machine learning related.
• Practical Machine Learning with Python - an excellent ebook with a detailed Machine Learning guide, including real world examples. "Master the essential skills needed to recognize and solve complex real-world problems with Machine Learning and Deep Learning by leveraging the highly popular Python Machine Learning Eco-system."
• Machine Learning in Python - chapter 5, Machine Learning in Python from Python Data Science Handbook.
• From Data Science IPython Notebooks:
  • Scikit-Learn notebooks - IPython Notebook(s) demonstrating scikit-learn functionality.
  • SciPy Notebooks - IPython Notebook(s) demonstrating statistical inference with SciPy functionality.
• Learn Data Science - a collection of Data Science Learning materials in the form of Jupyter Notebooks.
• AI with Python Tutorial - machine learning in Python tutorial.
• Machine learning in Python with scikit-learn - a video series that explains how solve machine learning problems using Python's popular scikit-learn library.
• A Gentle Introduction to Scikit-Learn: A Python Machine Learning Library
• Scipy Lecture Notes - one document to learn numerics, science, and data with Python.

Feature Selection

• Feature Selection For Machine Learning in Python
• Side-by-side Comparison of Feature Selection Tactics
• Walkthrough on Feature Selection in Python
• An Overview of Feature Selection Strategies
• Part 1 and Part 2 on Feature Engineering for Recommendation Systems

Supervised Learning

• Scikit-Learn Supervised Learning Documentation
• Scikit-learn Basics Tutorial - a quick tutorial of how to visualize, estimate, and classify with the Abalone dataset.
• Introduction to Machine Learning in Python with Scikit-Learn (video series) - a series of scikit-learn video tutorials, from Data School. There's also a GitHub repository with Jupyter Notebooks.
• An Introduction to Statistical Learning with Python Code - his repository contains Python code for a selection of tables, figures and LAB sections from the book 'An Introduction to Statistical Learning with Applications in R' by James, Witten, Hastie, Tibshirani (2013). The book was originally written with examples in R, Jordi Warmenhoven "translates" it into Python code.

Regression

• Scikit-Learn Generalized Linear Models Documentation
  • Scikit-Learn Regression Metrics
  • Scikit-learn documentation for Lasso, Ridge, and Elastic Net Regression.
  • Scikit-learn examples for Lasso and Ridge Regression.
• Linear Regression with Statsmodels and Scikit-Learn - a Jupyter Notebook with linear regression examples in Statsmodels and Scikit-Learn.
• Simple and Multiple Linear Regression in Python - a quick introduction to linear regression in Python I wrote.
• Scikit-Learn Regularization and Gradient Descent - examples of regularization of linear models.
• Data School's "Friendly Introduction to Linear Regression" using Python
• A Comprehensive Beginners Guide for Linear, Ridge and Lasso Regression
• A Complete Tutorial on Ridge and Lasso Regression in Python
• Introduction to Regression Splines (with Python codes)

Classification

• SKLearn Classifier Comparison
• Introduction to Implementing scikit-learn Classifiers - an introduction to implementing a number of scikit-learn classifiers, along with some data exploration. Excellent resources, contains examples in Jupyter Notebooks and many useful links.

Logistic Regression

• Logistic Regression Documentation
• Logistic Regression in SKLearn User Guide
• Logistic Regression with Scikit-Learn - a Jupyter Notebook with a logistic regression example.
• "Don’t Sweat the Solver Stuff" - "Tips for Better Logistic Regression Models in Scikit-Learn"
• Logistic Regression with Statsmodels - another Jupyter Notebook with a logistic regression example.

K-Nearest Neighbors

• SKLearn Documentation on Implementing KNN - K-Neighbors Classifier
• Nearest Neighbors Classification - SKLearn User Guide
• K-Nearest Neighbors Examples - an example of K-Nearest Neighbors Classifier (in a Jupyter Notebook).
• Model Training and Prediction with K-Nearest Neighbors
• Tutorial To Implement K-Nearest Neighbors in Python From Scratch

Classification Models Evaluation

• Precision-Recall
• Receiver Operating Characteristic Curves Demystified (in Python) - how to plot an ROC curve using Python and an explanation of the characteristics of a basic ROC curve.

Train-Test Split, Cross Validation and Bias-Variance Tradeoff

• Sklearn Documentation on Train Test Split
• Sklearn Documentation on Cross Validation
• KFold Cross Validation - from the Sklearn documentation.
• Train/Test Split and Cross Validation in Python - a blog post I wrote about the Train-Test Split and Cross Validation in Python.
• StackExchange Discussion on Approximate TTS, validation Set Sizes

Tree Based Methods

• Decision Trees in Scikit-learn
  • Decision Trees Classifier Documentation
  • Decision Tree Regressor Documentation
• Forests of Randomized Trees Documentation:
  • Random Forest Regressor Documentation
  • Random Forest Classifier Documentation
• A Complete Tutorial on Tree Based Modeling from Scratch
• "Decision Trees - Decoded" - a blog post on tree based methods, which consists of Decision Trees, Random Forest and Boosting methods.
• "Bagging - Unraveled" - a blog post on Bagging or Bootstrap Aggregating.
• Selecting Good Features: Random Forests
• Feature Importances with Forests of Trees - this example shows the use of forests of trees to evaluate the importance of features on an artificial classification task.

Support Vector Machines

• SVM Documentation on SKLearn
• Iris Example on SKLearn - plot different SVM classifiers in the iris dataset (from SKLearn).
• Hyperplane Walkthrough on SKLearn - SVM: Maximum margin separating hyperplane.

Unsupervised Learning

• Scikit-Learn Unsupervised Learning Documentation
• ISL Chapter 10 - Unsupervised Learning - examples of Unsupervised Learning: Principal Component Analysis, K-Means Clustering and Hierarchical Clustering. Chapter 10 out of An Introduction to Statistical Learning with Python Code.

Clustering

• Scikit-Learn Clustering Overview
  • Scikit-Learn K-means Overview
    • Scikit-Learn K-means Documentation
  • Scikit-Learn DBSCAN Overview
    • Scikit-Learn DBSCAN Documentation
  • Scikit-Learn Hierarchical Clustering Overview
  • Evaluating Clusters - clustering performance evaluation
  • Silhouette Score - measure of how far apart clusters are.
    • Silhouette Score Documentation
• Clustering with Sklearn - a Jupyter Notebook with clustering algorithms in Scikit-Learn.
• "The 5 Clustering Algorithms Data Scientists Need to Know" - a blog post on clustering algorithms.
• An Introduction to K-means Clustering Analysis - contains a theoretical explanation, pseudocode and Python code examples.

Principal Component Analysis

• Principal component analysis (PCA) - SKLearn documentation on PCA.
• SKLearn Documentation on PCA
• A One-Stop Shop for Principal Component Analysis - a blog post with an excellent introduction to Principal Component Analysis.
• Principal Component Analysis in 3 Simple Steps
• Practical Guide to Principal Component Analysis (PCA) in R & Python

Books

• Statistics and Machine Learning in Python
• Data Science Python Notebooks - deep learning (TensorFlow, Theano, Caffe, Keras), scikit-learn, Kaggle, big data (Spark, Hadoop MapReduce, HDFS), matplotlib, pandas, NumPy, SciPy, Python essentials, AWS, and various command line notebooks.
• Python Machine Learning (2nd Ed.) Code Repository - The "Python Machine Learning (2nd edition)" book code repository and info resource
• Hands-on Machine Learning with Scikit-Learn and TensorFlow - a series of Jupyter notebooks that walk you through the fundamentals of Machine Learning and Deep Learning in python using Scikit-Learn (and TensorFlow). This is the index notebook and the relevant chapters/notebooks are 1 to 8.
• Understanding Machine Learning: From Theory to Algorithms - a textbook by By Shai Shalev-Shwartz and Shai Ben-David from The School of Computer Science & Engineering at the Hebrew University. The aim of this textbook is to introduce machine learning, and the algorithmic paradigms it offers, in a principled way. The book provides a theoretical account of the fundamentals underlying machine learning and the mathematical derivations that transform these principles into practical algorithms.
• Machine Learning: An Algorithmic Perspective - a Machine Learning and Artificial Inteligence book by Stephen Marsland, from the School of Engineering and Advanced Technology (SEAT) at Massey University in Palmerston North, New Zealand.
• Mining of Massive Datasets - a book aimed to teach you Data Mining and Machine Learning techniques to process large datasets and extract valuable knowledge from them.