Use PyTorch to compute the partial derivatives in Tutorial 5 Q1(b).
Answer. [Write your solution here. Add cells as needed.]
We implement OLS in PyTorch in this question. The short introduction to PyTorch in lecture 12 will be useful. In addition, you may find the official PyTorch tutorials (https://pytorch.org/tutorials/) helpful. For computation using GPU, you can use Google's Colab as described in the Prac 1.
(a)
Load the diabetes dataset using the
sklearn.datasets.load_diabetes() function, and construct a random
70/30 train/test split for the dataset.
Answer. [Write your solution here. Add cells as needed.]
(b) Convert your training and test data to PyTorch tensors.
Answer. [Write your solution here. Add cells as needed.]
(c) Implement the closed-form formula for OLS in PyTorch, and use it to find the OLS model learned on the training set. Report the training set MSE.
Answer. [Write your solution here. Add cells as needed.]
(d) Now follow Lecture 12 to use PyTorch to use gradient descent to train an OLS model on CPU. Can you obtain the same model as in (a)? Try different learning rate and number of iterations if needed. Generate a plot for the test set accuracy against the number of iterations. You may need to use a large number of iterations.
Answer. [Write your solution here. Add cells as needed.]
(e) Repeat (b) but train the model on a GPU.
Answer. [Write your solution here. Add cells as needed.]
(f) Train the OLS model using batch size of 10. How many epochs do you need to get a good fit?
Answer. [Write your solution here. Add cells as needed.]