STAT3007/7007 Deep Learning, Prac 8

2022 Semester 1

Q1. Making your neural nets work

Making your neural net work usually requires informed trial-and-error: we need to be aware of the decisions that can affect a neural net's performance, and how they qualitatively affect the neural net's performance, and then use such knowledge to narrow down the candidate design decisions that we need to explore.

The following checklist includes some typical things that you need to check when trying to make your neural nets work:

• Architecture has enough but not too much capacity?
• Input normalized?
• Good initialization?
• Suitable loss function?
• Suitable optimization algorithm with suitable hyperparameters?
• Trained for long enough?

In this question, we do a case study on how to make a faulty neural net work. The implementation of the neural net is in the file diabetes_net.py.

(a) Are there any obvious issues with the code?

Answer. [Write your solution here. Add cells as needed.]

(b) What can you do to improve the performance?

Answer. [Write your solution here. Add cells as needed.]