The vanilla Perceptron algorithm for binary classification can be used for multiclass classification using a one-versus all approach: for each class, train a Perceptron classifier with that class as +1 and other classes as -1. For classification, calculcate the value of each class using the corresponding Perceptron, and then predict the class with the highest score. In fact, sklearn.linear_model.Perceptron is an implementation of this approach.
(a) Load the digits dataset in sklearn, and construct a 70/30 train/test split.
Answer. [Write your solution here. Add cells as needed.]
(b) Train a multilclass Perceptron on the training set, and report the trained model's training and test accuracies.
Answer. [Write your solution here. Add cells as needed.]
(c) Train a logistic regression model on the training set, and report the trained model's training and test accuracies.
Answer. [Write your solution here. Add cells as needed.]
(d) Compare the accuracy and efficiency of Perceptron against logistic regression.
Answer. [Write your solution here. Add cells as needed.]
Adaline essentially treats a binary classification problem as a regression problem with -1 and +1 as the only possible outputs. Certainly, other regression algorithms can be used to solve the regression problem too. We compare Adaline and OLS in this question.
(a)
Load the breast cancer dataset using the
sklearn.datasets.load_breast_cancer() function.
Construct a random 70/30 train/test split, and make necessary changes to labels of the dataset.
Answer. [Write your solution here. Add cells as needed.]
(b) Train an OLS model on the training set. Report the model parameters, and the training and test set MSEs and accuracies.
Answer. [Write your solution here. Add cells as needed.]
(c) Implement the mini-batch version of Adaline. Your implementation should allow users to specify the learning rate parameter, the batch size, and the maximum number of iterations. You can additionally implement additional options such as the specification of a termination criterion if you want.
Answer. [Write your solution here. Add cells as needed.]
(d) Train your best Adaline model on the training set . Tune the learning rate, the batch size, and the maximum number of epochs (one epoch is one pass through the same number of examples as the entire dataset). Report the model parameters, and the training and test set MSEs and accuracies.
Answer. [Write your solution here. Add cells as needed.]
(e) Comment on how your Adaline model compares with the OLS model.
Answer. [Write your solution here. Add cells as needed.]