Big Data Analytics

Simple linear regression

Fit a simple linear regression between two variables in R;Interpret output from R;Use models to predict a response variable;Validate the assumptions of the model.

Modelling data

Adapt the simple linear regression model in R to deal with multiple variables;Incorporate continuous and categorical variables in their models;Select the best-fitting model by inspecting the R output.

Many models

Manipulate nested dataframes in R;Use R to apply simultaneous linear models to large data frames by stratifying the data;Interpret the output of learner models.

Classification

Adapt linear models to take into account when the response is a categorical variable;Implement Logistic regression (LR) in R;Implement Generalised linear models (GLMs) in R;Implement Linear discriminant analysis (LDA) in R.

Prediction using models

Implement the principles of building a model to do prediction using classification;Split data into training and test sets, perform cross validation and model evaluation metrics;Use model selection for explaining data with models;Analyse the overfitting and bias-variance trade-off in prediction problems.

Getting bigger

Set up and apply sparklyr;Use logical verbs in R by applying native sparklyr versions of the verbs.

Supervised machine learning with sparklyr

Apply sparklyr to machine learning regression and classification models;Use machine learning models for prediction;Illustrate how distributed computing techniques can be used for “bigger” problems.

Deep learning

Use massive amounts of data to train multi-layer networks for classification;Understand some of the guiding principles behind training deep networks, including the use of autoencoders, dropout, regularization, and early termination;Use sparklyr and H2O to train deep networks.

Deep learning applications and scaling up

Understand some of the ways in which massive amounts of unlabelled data, and partially labelled data, is used to train neural network models;Leverage existing trained networks for targeting new applications;Implement architectures for object classification and object detection and assess their effectiveness.

Bringing it all together

Consolidate your understanding of relationships between the methodologies presented in this course, theirrelative strengths, weaknesses and range of applicability of these methods.