- Concepts that apply to machine learning also apply to deep learning (#37)
- Garbage In/Garbage Out (#26)
- Normalize your data (#7)
- Make sure data is not biased/skewed (#43)
- How to deal with class imbalance
- Make it reproducible (#21)
- Validate it (#27)
- Test/train/validate splitting (#20, #19)
- Test the robustness of your DL model in a simulation framework in which the ground truth is known (#49)
- Sanity checks, good coding practices, design and run experiments systematically (#52, #35)
- Use traditional methods to establish performance baselines
- Use a traditional machine learning algorithm (e.g., an off-the-shelf ensemble method like random forest that doesn't require extensive hyperparameter tuning) to assess whether it already solves the problem/addresses the research question adequately before moving to DL #11 #10)
- Implement a simple (generalized) linear model like multinomial logistic regression in the same software framework that is being used for DL, to establish a performance baseline early on in the project to help with debugging more complex DL implementations #41
- Understand the complexities of training deep neural networks
- Rerun multiple times different initial weight settings (e.g., avg. top 3 out of 5 performance) for fair comparison (#42)
- More extensive model selection: architecture as well as hyperparameter search required (#42)
- Deep learning really shines on unstructured, not structured data (#22)
- Know your data and your question
- Choose an appropriate neural network architecture and data representation (#29)
- A bit of discussion on DL architectures and how they apply to different problems here would be helpful
- Again, discussion on DL specific tuning would be helpful. i.e. layers, dropout, activation functions etc.
- Address deep neural networks' increased tendency to overfit the dataset (#28)
- Emphasize the added dangers of deep models with greater representational capacity with respect to overfitting
- Some discussion on how not to overfit particularly in the context of DL methods.
- Overfitting as a symptom of fitting to confounding effects (#55)
- Do not necessarily consider a DL model as a black box
- How to interpret DL models (#36)
- Similar to #6. Check if DL is actually a significant improvement in performance over a more 'interpretable' model (#25)
- Interpret predictions in the correct manner
- Don't share models trained on sensitive data.
- Talk about machine learning security and privacy concerns, especially with respect to PII and PHI. (#4)
- Emphasize the added dangers of deep models with greater representational capacity with respect to recovering sensitive data