BigdataEngineer

Overview

What is Spark ML?

Introduction to Machine Learning

Introduction to Spark

Free Account creation in Databricks

Provisioning a Spark Cluster

Basics about notebooks

Why we should learn Apache Spark?

Spark RDD (Create and Display Practical)

Spark Dataframe (Create and Display Practical)

Anonymus Functions in Scala

Extra (Optional on Spark DataFrame)

Extra (Optional on Spark DataFrame) in Details

Spark Datasets (Create and Display Practical)

Types of Machine Learning

Steps Involved in Machine Learning Program

Spark MLlib

Importing Notebook and Data Upload

Data Sources

Data Source JSON File

Data Source LIBSVM File

Data Source Arvo File

Data Source Parquet File

Machine Learning Data Pipeline Overview

Machine Learning Project as an Example (Just for Basic Idea)

Machine Learning Pipeline Example Project (Will it Rain Tomorrow in Australia) 2

Machine Learning Pipeline Example Project (Will it Rain Tomorrow in Australia) 3

Components of a Machine Learning Pipeline

Extracting, transforming and selecting features

TF-IDF (Feature Extractor)

Word2Vec (Feature Extractor)

CountVectorizer (Feature Extractor)

FeatureHasher (Feature Extractor)

Tokenizer (Feature Transformers)

StopWordsRemover (Feature Transformers)

n-gram (Feature Transformers)

Binarizer (Feature Transformers)

PCA (Feature Transformers)

Polynomial Expansion (Feature Transformers)

Discrete Cosine Transform (DCT) (Feature Transformers)

StringIndexer (Feature Transformers)

IndexToString (Feature Transformers)

OneHotEncoder (Feature Transformers)

SQLTransformer (Feature Transformers)

VectorAssembler (Feature Transformers)

RFormula (Feature Selector)

ChiSqSelector (Feature Selector)

Classification Model

Decision tree classifier Project

Logistic regression Model (Classification Model It has regression in the name)

Naive Bayes Project (Iris flower class prediction)

Random Forest Classifier Project

Gradient-boosted tree classifier Project

Linear Support Vector Machine Project

One-vs-Rest classifier (a.k.a. One-vs-All) Project

Regression Model

Linear Regression Model Project

Decision tree regression Model Project

Random forest regression Model Project

Gradient-boosted tree regression Model Project

Clustering KMeans Project (Mall Customer Segmentation)

Explanation of few terms used in Model

Download Resources

Thank you