Deep Learning Tricks
Papers
Practical recommendations for gradient-based training of deep architectures
- author: Yoshua Bengio
- arxiv: http://arxiv.org/abs/1206.5533
Blogs
Efficient BackProp
- intro: Neural Networks: Tricks of the Trade, 2nd
- blog: http://blog.csdn.net/zouxy09/article/details/45288129
Deep Learning for Vision: Tricks of the Trade
- intro: CVPR. Marc’Aurelio Ranzato
- slides: http://bavm2013.splashthat.com/img/events/46439/assets/34a7.ranzato.pdf
Optimizing RNN performance
- intro: Silicon Valley AI Lab
- keywords: Optimize GEMM, parallel GPU, GRU and LSTM…
- blog: http://svail.github.io/
Must Know Tips/Tricks in Deep Neural Networks
- intro: by Xiu-Shen Wei, NJU LAMDA
- blog: http://lamda.nju.edu.cn/weixs/project/CNNTricks/CNNTricks.html
- slides: http://lamda.nju.edu.cn/weixs/slide/CNNTricks_slide.pdf
Training Tricks from Deeplearning4j
http://deeplearning4j.org/trainingtricks.html
Suggestions for DL from Llya Sutskeve
- intro: data, preprocessing, mini-batch, gradient normalization, learning rate, weight initialization, data augmentation, dropout and ensemble
- blog: http://yyue.blogspot.com/2015/01/a-brief-overview-of-deep-learning.html
Efficient Training Strategies for Deep Neural Network Language Models
- intro: batch-size, initial learning rate, network initialization
- blog: https://fb56552f-a-62cb3a1a-s-sites.googlegroups.com/site/deeplearningworkshopnips2014/71.pdf?attachauth=ANoY7cp_eDwTXPm6iWHdBRhlIsgPASEAwkW-exLSOsz467mge7zLCkBMWznOu_G90vGVtqNvXOusc4z6cC6hEnHk6YzHtuEr_kyU0fyme7asaECN0zvoNwDk5258CueoB6fY3WtLvbJzYok1xiIeWSFYtk5mKXCXFDMI6djwhjCX1xi0GEEv_x7uMQwTdQlDItZ3kgLnZ2RjctQmIXDCu58fS3Wby4vWX3CkhMIf_EpCXx7jDn_M2SM%3D&attredirects=0
Neural Networks Best Practice
- intro: Uber
- paper: http://www.kentran.net/2013/04/neural-network-best-practices.html
Dark Knowledge from Hinton
- youtube: https://www.youtube.com/watch?v=EK61htlw8hY
- slides: http://www.ttic.edu/dl/dark14.pdf
- notes: http://deepdish.io/2014/10/28/hintons-dark-knowledge/
- notes: http://fastml.com/geoff-hintons-dark-knowledge/
Stochastic Gradient Descent Tricks(Leon Bottou)
http://leon.bottou.org/publications/pdf/tricks-2012.pdf
Advice for applying Machine Learning
https://jmetzen.github.io/2015-01-29/ml_advice.html
How to Debug Learning Algorithm for Regression Model
http://vitalflux.com/machine-learning-debug-learning-algorithm-regression-model/
Large-scale L-BFGS using MapReduce
- intro: NIPS 2014
- paper: http://papers.nips.cc/paper/5333-large-scale-l-bfgs-using-mapreduce.pdf
Selecting good features
– Part I: univariate selection: http://blog.datadive.net/selecting-good-features-part-i-univariate-selection/ – Part II: linear models and regularization: http://blog.datadive.net/selecting-good-features-part-ii-linear-models-and-regularization/ – Part III: random forests: http://blog.datadive.net/selecting-good-features-part-iii-random-forests/ – Part IV: stability selection, RFE and everything side by side: http://blog.datadive.net/selecting-good-features-part-iv-stability-selection-rfe-and-everything-side-by-side/
机器学习代码心得之有监督学习的模块
http://www.weibo.com/p/1001603795687165852957
Stochastic Gradient Boosting: Choosing the Best Number of Iterations
- intro: Kaggle winner YANIR SEROUSSI
- blog: http://yanirseroussi.com/2014/12/29/stochastic-gradient-boosting-choosing-the-best-number-of-iterations/
Large-Scale High-Precision Topic Modeling on Twitter
- intro: Twitter senior researcher. KDD 2014
- paper: http://www.eeshyang.com/papers/KDD14Jubjub.pdf
H2O World - Top 10 Deep Learning Tips & Tricks - Arno Candel
http://www.slideshare.net/0xdata/h2o-world-top-10-deep-learning-tips-tricks-arno-candel
How To Improve Deep Learning Performance: 20 Tips, Tricks and Techniques That You Can Use To Fight Overfitting and Get Better Generalization
http://machinelearningmastery.com/improve-deep-learning-performance/
Neural Network Training Speed Trick
The Black Magic of Deep Learning - Tips and Tricks for the practitioner
http://nmarkou.blogspot.ru/2017/02/the-black-magic-of-deep-learning-tips.html
