Generative Adversarial Networks

Generative Adversarial Nets

• arxiv: http://arxiv.org/abs/1406.2661
• paper: https://papers.nips.cc/paper/5423-generative-adversarial-nets.pdf
• github: https://github.com/goodfeli/adversarial
• github: https://github.com/aleju/cat-generator

Adversarial Feature Learning

• intro: ICLR 2017
• arxiv: https://arxiv.org/abs/1605.09782
• github: https://github.com/jeffdonahue/bigan

Generative Adversarial Networks

• intro: by Ian Goodfellow, NIPS 2016 tutorial
• arxiv: https://arxiv.org/abs/1701.00160
• slides: http://www.iangoodfellow.com/slides/2016-12-04-NIPS.pdf
• mirror: https://pan.baidu.com/s/1gfBNYW7

Adversarial Examples and Adversarial Training

• intro: NIPS 2016, Ian Goodfellow OpenAI
• slides: http://www.iangoodfellow.com/slides/2016-12-9-AT.pdf

How to Train a GAN? Tips and tricks to make GANs work

• github: https://github.com/soumith/ganhacks

Unsupervised and Semi-supervised Learning with Categorical Generative Adversarial Networks

• intro: CatGAN
• arxiv: http://arxiv.org/abs/1511.06390

Unsupervised Representation Learning with Deep Convolutional Generative Adversarial Networks

• intro: DCGAN
• arxiv: http://arxiv.org/abs/1511.06434
• github: https://github.com/jazzsaxmafia/dcgan_tensorflow
• github: https://github.com/Newmu/dcgan_code
• github: https://github.com/mattya/chainer-DCGAN
• github: https://github.com/soumith/dcgan.torch
• github: https://github.com/carpedm20/DCGAN-tensorflow

InfoGAN: Interpretable Representation Learning by Information Maximizing Generative Adversarial Nets

• arxiv: https://arxiv.org/abs/1606.03657
• github: https://github.com/openai/InfoGAN
• github(Tensorflow): https://github.com/buriburisuri/supervised_infogan

Learning Interpretable Latent Representations with InfoGAN: A tutorial on implementing InfoGAN in Tensorflow

• blog: https://medium.com/@awjuliani/learning-interpretable-latent-representations-with-infogan-dd710852db46#.r0kur3aum
• github: https://gist.github.com/awjuliani/c9ecd8b37d33d6855cd4ed9aa16ce89f#file-infogan-tutorial-ipynb

Coupled Generative Adversarial Networks

• arxiv: https://arxiv.org/abs/1606.07536

Energy-based Generative Adversarial Network

• intro: EBGAN
• author: Junbo Zhao, Michael Mathieu, Yann LeCun
• arxiv: http://arxiv.org/abs/1609.03126
• github(Tensorflow): https://github.com/buriburisuri/ebgan

SeqGAN: Sequence Generative Adversarial Nets with Policy Gradient

• github: https://github.com/LantaoYu/SeqGAN

Connecting Generative Adversarial Networks and Actor-Critic Methods

• arxiv: https://arxiv.org/abs/1610.01945

Generative Adversarial Nets from a Density Ratio Estimation Perspective

• arxiv: https://arxiv.org/abs/1610.02920

Unrolled Generative Adversarial Networks

• paper: http://openreview.net/pdf?id=BydrOIcle
• github: https://github.com/bstriner/keras-adversarial

Generative Adversarial Networks as Variational Training of Energy Based Models

• arxiv: https://arxiv.org/abs/1611.01799
• github: https://github.com/Shuangfei/vgan

Multi-class Generative Adversarial Networks with the L2 Loss Function

Least Squares Generative Adversarial Networks

• arxiv: https://arxiv.org/abs/1611.04076

Inverting The Generator Of A Generative Adversarial Networ

• intro: NIPS 2016 Workshop on Adversarial Training
• arxiv: https://arxiv.org/abs/1611.05644

ml4a-invisible-cities

• project page: https://opendot.github.io/ml4a-invisible-cities/
• arxiv: https://github.com/opendot/ml4a-invisible-cities

Semi-Supervised Learning with Context-Conditional Generative Adversarial Networks

• arxiv: https://arxiv.org/abs/1611.06430

Associative Adversarial Networks

• intro: NIPS 2016 Workshop on Adversarial Training
• arxiv: https://arxiv.org/abs/1611.06953

Temporal Generative Adversarial Nets

• arxiv: https://arxiv.org/abs/1611.06624

Handwriting Profiling using Generative Adversarial Networks

• intro: Accepted at The Thirty-First AAAI Conference on Artificial Intelligence (AAAI-17 Student Abstract and Poster Program)
• arxiv: https://arxiv.org/abs/1611.08789

C-RNN-GAN: Continuous recurrent neural networks with adversarial training

• intro: Constructive Machine Learning Workshop (CML) at NIPS 2016
• project page: http://mogren.one/publications/2016/c-rnn-gan/
• arxiv: https://arxiv.org/abs/1611.09904
• github: https://github.com/olofmogren/c-rnn-gan

Ensembles of Generative Adversarial Networks

• intro: NIPS 2016 Workshop on Adversarial Training
• arxiv: https://arxiv.org/abs/1612.00991

Improved generator objectives for GANs

• intro: NIPS 2016 Workshop on Adversarial Training
• arxiv: https://arxiv.org/abs/1612.02780

Stacked Generative Adversarial Networks

• intro: SGAN
• arxiv: https://arxiv.org/abs/1612.04357
• github: https://github.com/xunhuang1995/SGAN

Unsupervised Pixel-Level Domain Adaptation with Generative Adversarial Networks

• intro: Google Brain & Google Research
• arxiv: https://arxiv.org/abs/1612.05424

AdaGAN: Boosting Generative Models

• intro: Max Planck Institute for Intelligent Systems & Google Brain
• arxiv: https://arxiv.org/abs/1701.02386

Towards Principled Methods for Training Generative Adversarial Networks

• intro: Courant Institute of Mathematical Sciences & Facebook AI Research
• arxiv: https://arxiv.org/abs/1701.04862

Wasserstein GAN

• intro: Courant Institute of Mathematical Sciences & Facebook AI Research
• arxiv: https://arxiv.org/abs/1701.07875
• github: https://github.com/martinarjovsky/WassersteinGAN
• github: https://github.com/Zardinality/WGAN-tensorflow
• github(Tensorflow/Keras): https://github.com/kuleshov/tf-wgan
• github: https://github.com/shekkizh/WassersteinGAN.tensorflow
• gist: https://gist.github.com/soumith/71995cecc5b99cda38106ad64503cee3
• reddit: https://www.reddit.com/r/MachineLearning/comments/5qxoaz/r_170107875_wasserstein_gan/

Improved Training of Wasserstein GANs

• intro: NIPS 2017
• arxiv: https://arxiv.org/abs/1704.00028
• github(TensorFlow): https://github.com/igul222/improved_wgan_training
• github: https://github.com/jalola/improved-wgan-pytorch

On the effect of Batch Normalization and Weight Normalization in Generative Adversarial Networks

On the Effects of Batch and Weight Normalization in Generative Adversarial Networks

• arxiv: https://arxiv.org/abs/1704.03971
• github: https://github.com/stormraiser/GAN-weight-norm

Controllable Generative Adversarial Network

• intro: Korea University
• arxiv: https://arxiv.org/abs/1708.00598

Generative Adversarial Networks: An Overview

• intro: Imperial College London & Victoria University of Wellington & University of Montreal & Cortexica Vision Systems Ltd
• intro: IEEE Signal Processing Magazine Special Issue on Deep Learning for Visual Understanding
• arxiv: https://arxiv.org/abs/1710.07035

CyCADA: Cycle-Consistent Adversarial Domain Adaptation

https://arxiv.org/abs/1711.03213

Spectral Normalization for Generative Adversarial Networks

https://openreview.net/forum?id=B1QRgziT-

Are GANs Created Equal? A Large-Scale Study

• intro: Google Brain
• arxiv: https://arxiv.org/abs/1711.10337
• reddit: https://www.reddit.com/r/MachineLearning/comments/7gwip3/d_googles_large_scale_gantuning_paper_unfairly/

GAGAN: Geometry-Aware Generative Adverserial Networks

https://arxiv.org/abs/1712.00684

CycleGAN: a Master of Steganography

• intro: NIPS 2017, workshop on Machine Deception
• arxiv: https://arxiv.org/abs/1712.02950

PacGAN: The power of two samples in generative adversarial networks

• intro: CMU & University of Illinois at Urbana-Champaign
• arxiv: https://arxiv.org/abs/1712.04086

ComboGAN: Unrestrained Scalability for Image Domain Translation

• arxiv: https://arxiv.org/abs/1712.06909
• github: https://github.com/AAnoosheh/ComboGAN

Decoupled Learning for Conditional Adversarial Networks

https://arxiv.org/abs/1801.06790

No Modes left behind: Capturing the data distribution effectively using GANs

• intro: AAAI 2018
• arxiv: https://arxiv.org/abs/1802.00771

Image-to-Image Translation

Pix2Pix

Image-to-Image Translation with Conditional Adversarial Networks

• intro: CVPR 2017
• project page: https://phillipi.github.io/pix2pix/
• arxiv: https://arxiv.org/abs/1611.07004
• github: https://github.com/phillipi/pix2pix
• github(TensorFlow): https://github.com/yenchenlin/pix2pix-tensorflow
• github(Chainer): https://github.com/mattya/chainer-pix2pix
• github(PyTorch): https://github.com/mrzhu-cool/pix2pix-pytorch
• github(Chainer): https://github.com/wuhuikai/chainer-pix2pix

Remastering Classic Films in Tensorflow with Pix2Pix

• blog: https://hackernoon.com/remastering-classic-films-in-tensorflow-with-pix2pix-f4d551fa0503#.6dmahnt8n
• github: https://github.com/awjuliani/Pix2Pix-Film
• model: https://drive.google.com/file/d/0B8x0IeJAaBccNFVQMkQ0QW15TjQ/view

Image-to-Image Translation in Tensorflow

• blog: http://affinelayer.com/pix2pix/index.html
• github: https://github.com/affinelayer/pix2pix-tensorflow

webcam pix2pix

https://github.com/memo/webcam-pix2pix-tensorflow

Unsupervised Image-to-Image Translation with Generative Adversarial Networks

• intro: Imperial College London & Indian Institute of Technology
• arxiv: https://arxiv.org/abs/1701.02676

Unsupervised Image-to-Image Translation Networks

• intro: NIPS 2017 Spotlight
• intro: unsupervised/unpaired image-to-image translation using coupled GANs
• project page: http://research.nvidia.com/publication/2017-12_Unsupervised-Image-to-Image-Translation
• arxiv: https://arxiv.org/abs/1703.00848
• github: https://github.com/mingyuliutw/UNIT

Unpaired Image-to-Image Translation using Cycle-Consistent Adversarial Networks

• intro: UC Berkeley
• project page: https://junyanz.github.io/CycleGAN/
• arxiv: https://arxiv.org/abs/1703.10593
• github(official, Torch): https://github.com/junyanz/CycleGAN
• github(official, PyTorch): https://github.com/junyanz/pytorch-CycleGAN-and-pix2pix
• github(PyTorch): https://github.com/eveningglow/semi-supervised-CycleGAN
• github(Chainer): https://github.com/Aixile/chainer-cyclegan

CycleGAN and pix2pix in PyTorch

• intro: Image-to-image translation in PyTorch (e.g. horse2zebra, edges2cats, and more)
• github: https://github.com/junyanz/pytorch-CycleGAN-and-pix2pix

Perceptual Adversarial Networks for Image-to-Image Transformation

https://arxiv.org/abs/1706.09138

XGAN: Unsupervised Image-to-Image Translation for many-to-many Mappings

• intro: IST Austria & Google Brain & Google Research
• arxiv: https://arxiv.org/abs/1711.05139

In2I : Unsupervised Multi-Image-to-Image Translation Using Generative Adversarial Networks

https://arxiv.org/abs/1711.09334

StarGAN: Unified Generative Adversarial Networks for Multi-Domain Image-to-Image Translation

• intro: Korea University & Clova AI Research
• arxiv: https://arxiv.org/abs/1711.09020
• github: https://github.com//yunjey/StarGAN

Discriminative Region Proposal Adversarial Networks for High-Quality Image-to-Image Translation

https://arxiv.org/abs/1711.09554

Toward Multimodal Image-to-Image Translation

• intro: NIPS 2017. BicycleGAN
• project page: https://junyanz.github.io/BicycleGAN/
• arxiv: https://arxiv.org/abs/1711.11586
• github(official, PyTorch): https://github.com//junyanz/BicycleGAN
• github: https://github.com/gitlimlab/BicycleGAN-Tensorflow
• github: https://github.com/kvmanohar22/img2imgGAN
• github: https://github.com/eveningglow/BicycleGAN-pytorch

Face Translation between Images and Videos using Identity-aware CycleGAN

https://arxiv.org/abs/1712.00971

Unsupervised Multi-Domain Image Translation with Domain-Specific Encoders/Decoders

https://arxiv.org/abs/1712.02050

High-Resolution Image Synthesis and Semantic Manipulation with Conditional GANs

• intro: NVIDIA Corporation, UC Berkeley
• project page: https://tcwang0509.github.io/pix2pixHD/
• arxiv: https://arxiv.org/abs/1711.11585
• github: https://github.com/NVIDIA/pix2pixHD
• youtube: https://www.youtube.com/watch?v=3AIpPlzM_qs&feature=youtu.be

On the Effectiveness of Least Squares Generative Adversarial Networks

https://arxiv.org/abs/1712.06391

GANs for Limited Labeled Data

• intro: Ian Goodfellow
• slides: http://www.iangoodfellow.com/slides/2017-12-09-label.pdf

Defending Against Adversarial Examples

• intro: Ian Goodfellow
• slides: http://www.iangoodfellow.com/slides/2017-12-08-defending.pdf

Conditional Image-to-Image Translation

• intro: CVPR 2018
• arxiv: https://arxiv.org/abs/1805.00251

Projects

Generative Adversarial Networks with Keras

• github: https://github.com/phreeza/keras-GAN

Generative Adversarial Network Demo for Fresh Machine Learning #2

• youtube: https://www.youtube.com/watch?v=deyOX6Mt_As&feature=em-uploademail
• github: https://github.com/llSourcell/Generative-Adversarial-Network-Demo
• demo: http://cs.stanford.edu/people/karpathy/gan/

TextGAN: A generative adversarial network for text generation, written in TensorFlow.

• github: https://github.com/AustinStoneProjects/TextGAN

cleverhans v0.1: an adversarial machine learning library

• arxiv: https://arxiv.org/abs/1610.00768
• github: https://github.com/openai/cleverhans

Deep Convolutional Variational Autoencoder w/ Adversarial Network

• intro: An implementation of the deep convolutional generative adversarial network, combined with a varational autoencoder
• github: https://github.com/staturecrane/dcgan_vae_torch

A versatile GAN(generative adversarial network) implementation. Focused on scalability and ease-of-use.

• github: https://github.com/255BITS/HyperGAN

AdaGAN: Boosting Generative Models

• intro: AdaGAN: greedy iterative procedure to train mixtures of GANs
• intro: Max Planck Institute for Intelligent Systems & Google Brain
• arxiv: https://arxiv.org/abs/1701.02386
• github: https://github.com/tolstikhin/adagan

TensorFlow-GAN (TFGAN)

• intro: TFGAN: A Lightweight Library for Generative Adversarial Networks
• github: https://github.com//tensorflow/tensorflow/tree/master/tensorflow/contrib/gan
• blog: https://research.googleblog.com/2017/12/tfgan-lightweight-library-for.html

Blogs

Generative Adversial Networks Explained

• blog: http://kvfrans.com/generative-adversial-networks-explained/

Generative Adversarial Autoencoders in Theano

• blog: https://swarbrickjones.wordpress.com/2016/01/24/generative-adversarial-autoencoders-in-theano/
• github: https://github.com/mikesj-public/dcgan-autoencoder

An introduction to Generative Adversarial Networks (with code in TensorFlow)

• blog: http://blog.aylien.com/introduction-generative-adversarial-networks-code-tensorflow/
• github: https://github.com/AYLIEN/gan-intro

Difficulties training a Generative Adversarial Network

• github: https://github.com/shekkizh/neuralnetworks.thought-experiments/blob/master/Generative%20Models/GAN/Readme.md

Are Energy-Based GANs any more energy-based than normal GANs?

http://www.inference.vc/are-energy-based-gans-actually-energy-based/

Generative Adversarial Networks Explained with a Classic Spongebob Squarepants Episode: Plus a Tensorflow tutorial for implementing your own GAN

• blog: https://medium.com/@awjuliani/generative-adversarial-networks-explained-with-a-classic-spongebob-squarepants-episode-54deab2fce39#.rpiunhdjh
• gist: https://gist.github.com/awjuliani/8ebf356d03ffee139659807be7fa2611

Deep Learning Research Review Week 1: Generative Adversarial Nets

• blog: https://adeshpande3.github.io/adeshpande3.github.io/Deep-Learning-Research-Review-Week-1-Generative-Adversarial-Nets

Stability of Generative Adversarial Networks

• blog: http://www.araya.org/archives/1183

Instance Noise: A trick for stabilising GAN training

• blog: http://www.inference.vc/instance-noise-a-trick-for-stabilising-gan-training/

Generating Fine Art in 300 Lines of Code

• intro: DCGAN
• blog: https://medium.com/@richardherbert/generating-fine-art-in-300-lines-of-code-4d37218216a6#.63qm8ef9g

Talks / Videos

Generative Adversarial Network visualization

• youtube: https://www.youtube.com/watch?v=mObnwR-u8pc

Resources

The GAN Zoo

• intro: A list of all named GANs!
• github: https://github.com/hindupuravinash/the-gan-zoo

AdversarialNetsPapers: The classical Papers about adversial nets

• github: https://github.com/zhangqianhui/AdversarialNetsPapers

GAN Timeline

• intro: A timeline showing the development of Generative Adversarial Networks (GAN)
• github: https://github.com//dongb5/GAN-Timeline

Painting

Artist Agent: A Reinforcement Learning Approach to Automatic Stroke Generation in Oriental Ink Painting

• intro: ICML 2012
• arxiv: https://arxiv.org/abs/1206.4634

Neural Art

A Neural Algorithm of Artistic Style

• arxiv: http://arxiv.org/abs/1508.06576
• gitxiv: http://gitxiv.com/posts/jG46ukGod8R7Rdtud/a-neural-algorithm-of-artistic-style
• github: https://github.com/kaishengtai/neuralart
• github: https://github.com/jcjohnson/neural-style
• github: https://github.com/andersbll/neural_artistic_style
• ipn: http://nbviewer.ipython.org/github/Lasagne/Recipes/blob/master/examples/styletransfer/Art%20Style%20Transfer.ipynb
• github: https://github.com/mbartoli/neural-animation
• github: https://github.com/memisevic/artify
• github: https://github.com/mattya/chainer-gogh
• github(TensorFlow): https://github.com/anishathalye/neural-style
• github: https://github.com/woodrush/neural-art-tf
• github: https://github.com/dmlc/mxnet/tree/master/example/neural-style
• demo: http://deepart.io/
• github: https://github.com/Teaonly/easyStyle
• github: https://github.com/ckmarkoh/neuralart_tensorflow
• github: https://github.com/fzliu/style-transfer
• github: https://github.com/titu1994/Neural-Style-Transfer
• github: https://github.com/saikatbsk/Vincent-AI-Artist
• github: https://github.com/zhaw/neural_style

Image Style Transfer Using Convolutional Neural Networks

• paper: www.cv-foundation.org/openaccess/content_cvpr_2016/papers/Gatys_Image_Style_Transfer_CVPR_2016_paper.pdf

Artificial Startup Style: Neural art about startup fashion

• blog: https://medium.com/data-engineering/artificial-startup-style-437f6090b1f7#.8u06gq42e

From Pixels to Paragraphs: How artistic experiments with deep learning guard us from hype

• blog: https://medium.com/@genekogan/from-pixels-to-paragraphs-eb2763da0e9b#.er3djn9z9

Experiments with style transfer

• website: http://mtyka.github.io/code/2015/10/02/experiments-with-style-transfer.html

Style Transfer for Headshot Portraits (SIGGRAPH 2014)

• project: https://people.csail.mit.edu/yichangshih/portrait_web/

Teaching recurrent Neural Networks about Monet

• blog: http://blog.manugarri.com/teaching-recurrent-neural-networks-about-monet/
• github: https://github.com/manugarri/keras_monet

Content Aware Neural Style Transfer

• arxiv: http://arxiv.org/abs/1601.04568

Combining Markov Random Fields and Convolutional Neural Networks for Image Synthesis

• arxiv: http://arxiv.org/abs/1601.04589
• github: https://github.com/chuanli11/CNNMRF

Stylenet: Neural Network with Style Synthesis

• github: https://github.com/machrisaa/stylenet

Ostagram

• intro: This program presents web-service for algorithm combining the content of one image with the style of another image using convolutional neural networks
• github: https://github.com/SergeyMorugin/ostagram

Exploring the Neural Algorithm of Artistic Style

• intro: A short class project report
• arxiv: http://arxiv.org/abs/1602.07188

Perceptual Losses for Real-Time Style Transfer and Super-Resolution

• intro: Justin Johnson, Alexandre Alahi, Li Fei-Fei. ECCV 2016
• arxiv: http://arxiv.org/abs/1603.08155
• github: https://github.com/jcjohnson/fast-neural-style
• github: https://github.com/yusuketomoto/chainer-fast-neuralstyle
• github: https://github.com/awentzonline/keras-rtst
• github(Keras): https://github.com/titu1994/Fast-Neural-Style
• gtihub(Tensorflow): [https://github.com/junrushao1994/fast-neural-style.tf] (https://github.com/junrushao1994/fast-neural-style.tf)
• github(PyTorch): https://github.com/bengxy/FastNeuralStyle

Image transformation networks with fancy loss functions

• intro: Fast neural style in tensorflow based on http://arxiv.org/abs/1603.08155
• blog: http://olavnymoen.com/2016/07/07/image-transformation-network
• github: https://github.com/OlavHN/fast-neural-style

Improving the Neural Algorithm of Artistic Style

• arxiv: http://arxiv.org/abs/1605.04603

CubistMirror: an openframeworks app which repeatedly applies real-time style transfer on a webcam

• github: https://github.com/genekogan/CubistMirror

Transfer Style But Not Color

• blog: http://blog.deepart.io/2016/06/04/color-independent-style-transfer/
• github: https://github.com/pavelgonchar/color-independent-style-transfer

neural-art-mini: Lightweight version of mxnet neural art implementation

• intro: Lightweight version of mxnet neural art implementation using ~4.8M SqueezeNet model. Compressed model is less than 500KB
• github: https://github.com/pavelgonchar/neural-art-mini

Preserving Color in Neural Artistic Style Transfer

• arxiv: http://arxiv.org/abs/1606.05897

End to End Neural Art with Generative Models

• blog: http://dmlc.ml/mxnet/2016/06/20/end-to-end-neural-style.html
• github: https://github.com/dmlc/mxnet/tree/master/example/neural-style

Neural Style Explained

• blog: http://kvfrans.com/neural-style-explained/
• github: https://github.com/kvfrans/neural-style

Texture Networks: Feed-forward Synthesis of Textures and Stylized Images

• intro: IMCL 2016
• arxiv: http://arxiv.org/abs/1603.03417
• github: https://github.com/DmitryUlyanov/texture_nets
• notes: https://blog.acolyer.org/2016/09/23/texture-networks-feed-forward-synthesis-of-textures-and-stylized-images/
• github: https://github.com/tgyg-jegli/tf_texture_net

Learning Typographic Style

• arxiv: https://arxiv.org/abs/1603.04000

Instance Normalization: The Missing Ingredient for Fast Stylization

• arxiv: http://arxiv.org/abs/1607.08022

Painting style transfer for head portraits using convolutional neural networks

• paper: http://dl.acm.org/citation.cfm?id=2925968
• sci-hub: http://dl.acm.org.sci-hub.cc/citation.cfm?doid=2897824.2925968

Style-Transfer via Texture-Synthesis

• arxiv: http://arxiv.org/abs/1609.03057

neural-style-tf: TensorFlow implementation of Neural Style

• github: https://github.com/cysmith/neural-style-tf

Deep Convolutional Networks as Models of Generalization and Blending Within Visual Creativity

• intro: In Proceedings of the 7th International Conference on Computational Creativity. Palo Alto: Association for the Advancement of Artificial Intelligence (AAAI) Press (2016)
• arxiv: https://arxiv.org/abs/1610.02478

A Learned Representation For Artistic Style

• intro: Google Brain
• arxiv: https://arxiv.org/abs/1610.07629
• blog: https://research.googleblog.com/2016/10/supercharging-style-transfer.html
• github: https://github.com/tensorflow/magenta/tree/master/magenta/models/image_stylization
• github(Tensorflow): https://github.com/Heumi/Fast_Multi_Style_Transfer-tf

How to Fake It As an Artist with Docker, AWS and Deep Learning

• blog: https://medium.com/@lherrera/how-to-fake-it-as-an-artist-with-docker-aws-and-deep-learning-6d42f4acd890#.eq9gcgkzh

Multistyle Pastiche Generator

• blog: https://magenta.tensorflow.org/2016/11/01/multistyle-pastiche-generator/

Fast Style Transfer in TensorFlow

• intro: Video Stylization, Image Stylization
• github: https://github.com/lengstrom/fast-style-transfer

Neural Style Transfer For Chinese Fonts

• github: https://github.com/kaonashi-tyc/Rewrite

Neural Style Representations and the Large-Scale Classification of Artistic Style

• arxiv: https://arxiv.org/abs/1611.05368

Controlling Perceptual Factors in Neural Style Transfer

• intro: University of Tubingen & Adobe Research
• arxiv: https://arxiv.org/abs/1611.07865

Awesome Typography: Statistics-Based Text Effects Transfer

• arxiv: https://arxiv.org/abs/1611.09026

Fast Patch-based Style Transfer of Arbitrary Style

• arxiv: https://arxiv.org/abs/1612.04337
• github: https://github.com/rtqichen/style-swap

Demystifying Neural Style Transfer

• intro: IJCAI 2017
• arxiv: https://arxiv.org/abs/1701.01036
• github: https://github.com/lyttonhao/Neural-Style-MMD

Son of Zorn’s Lemma: Targeted Style Transfer Using Instance-aware Semantic Segmentation

• arxiv: https://arxiv.org/abs/1701.02357

Bringing Impressionism to Life with Neural Style Transfer in Come Swim

• intro: a case study of how Neural Style Transfer can be used in a movie production context
• keywords: Kristen Stewart !
• arxiv: https://arxiv.org/abs/1701.04928

Pytorch tutorials for Neural Style transfert

• github: https://github.com/alexis-jacq/Pytorch-Tutorials

Stable and Controllable Neural Texture Synthesis and Style Transfer Using Histogram Losses

• arxiv: https://arxiv.org/abs/1701.08893

Arbitrary Style Transfer In Real-Time With Adaptive Instance Normalization

• intro: ICCV 2017. Cornell University
• paper: https://openreview.net/pdf?id=B1fUVMzKg
• github(Torch): https://github.com/xunhuang1995/AdaIN-style
• github(TensorFlow): https://github.com/elleryqueenhomels/arbitrary_style_transfer

Picking an optimizer for Style Transfer

• blog: https://medium.com/slavv/picking-an-optimizer-for-style-transfer-86e7b8cba84b#.cgv2oreaq
• github: https://github.com/slavivanov/Style-Tranfer

Multi-style Generative Network for Real-time Transfer

https://arxiv.org/abs/1703.06953

Deep Photo Style Transfer

• arxiv: https://arxiv.org/abs/1703.07511
• github(Torch): https://github.com/luanfujun/deep-photo-styletransfer
• github(Docker): https://github.com/martinbenson/deep-photo-styletransfer

Lightweight Neural Style on Pytorch

https://github.com/lizeng614/SqueezeNet-Neural-Style-Pytorch

StyleBank: An Explicit Representation for Neural Image Style Transfer

• intro: CVPR 2017
• arxiv: https://arxiv.org/abs/1703.09210

How to Make an Image More Memorable? A Deep Style Transfer Approach

• intro: ACM ICMR 2017
• arxiv: https://arxiv.org/abs/1704.01745

Visual Attribute Transfer through Deep Image Analogy

• intro: SIGGRAPH 2017
• keywords: Deep Image Analogy
• arxiv: https://arxiv.org/abs/1705.01088
• github: https://github.com/msracver/Deep-Image-Analogy

Characterizing and Improving Stability in Neural Style Transfer

https://arxiv.org/abs/1705.02092

Towards Metamerism via Foveated Style Transfer

https://arxiv.org/abs/1705.10041

Style Transfer for Sketches with Enhanced Residual U-net and Auxiliary Classifier GAN

https://arxiv.org/abs/1706.03319

Meta Networks for Neural Style Transfer

• arxiv: https://arxiv.org/abs/1709.04111
• github: https://github.com/FalongShen/styletransfer

Neural Color Transfer between Images

• intro: Hong Kong University of Science and Technology & Microsoft Research
• arxiv: https://arxiv.org/abs/1710.00756

Improved Style Transfer by Respecting Inter-layer Correlations

https://arxiv.org/abs/1801.01933

Face Destylization

https://arxiv.org/abs/1802.01237

Unsupervised Typography Transfer

https://arxiv.org/abs/1802.02595

Stereoscopic Neural Style Transfer

• intro: CVPR 2018
• arxiv: https://arxiv.org/abs/1802.10591

Arbitrary Style Transfer with Deep Feature Reshuffle

https://arxiv.org/abs/1805.04103

Neural Art On Audio

MSc AI Project on generative deep networks and neural style transfer for audio

• github: https://github.com/Fr-d-rik/generative_audio
• project report: https://github.com/Fr-d-rik/generative_audio/blob/master/docs/project_report.pdf

Neural Song Style

• intro: Audio style transfer AI
• github: https://github.com/rupeshs/neuralsongstyle

Time Domain Neural Audio Style Transfer

• intro: NIPS 2017
• arxiv: https://arxiv.org/abs/1711.11160
• github: https://github.com//pkmital/time-domain-neural-audio-style-transfer

Neural Art On Video

neural-style-video

• blog: http://larseidnes.com/2015/12/18/painting-videos-with-neural-networks/
• github: https://github.com/larspars/neural-style-video

Instructions for making a Neural-Style movie

• website: https://gist.github.com/genekogan/d61c8010d470e1dbe15d
• sample: https://vimeo.com/139123754

Artistic style transfer for videos

• arxiv: http://arxiv.org/abs/1604.08610
• github: https://github.com/manuelruder/artistic-videos

Artistic style transfer for videos and spherical images

https://arxiv.org/abs/1708.04538

How Deep Learning Can Paint Videos in the Style of Art’s Great Masters

• blog: https://blogs.nvidia.com/blog/2016/05/25/deep-learning-paints-videos/

DeepMovie: Using Optical Flow and Deep Neural Networks to Stylize Movies

• arxiv: http://arxiv.org/abs/1605.08153

Coherent Online Video Style Transfer

https://arxiv.org/abs/1703.09211

Laplacian-Steered Neural Style Transfer

• intro: ACM Multimedia Conference (MM) 2017
• arxiv: https://arxiv.org/abs/1707.01253

Real-Time Neural Style Transfer for Videos

• intro: Tsinghua University & Tencent AI Lab
• paper: http://openaccess.thecvf.com/content_cvpr_2017/papers/Huang_Real-Time_Neural_Style_CVPR_2017_paper.pdf

Multi-Content GAN for Few-Shot Font Style Transfer

https://arxiv.org/abs/1712.00516

Neural Doodle

Semantic Style Transfer and Turning Two-Bit Doodles into Fine Artworks

• paper: http://nucl.ai/semantic-style-transfer.pdf
• reddit: https://www.reddit.com/r/MachineLearning/comments/48zstj/my_wip_implementation_of_neural_image_analogies/
• github: https://github.com/alexjc/neural-doodle

Neural Doodle

• github: https://github.com/alexjc/neural-doodle

Faster neural doodle

• github: https://github.com/DmitryUlyanov/fast-neural-doodle

Feed-forward neural doodle

• blog: http://dmitryulyanov.github.io/feed-forward-neural-doodle/
• github: https://github.com/DmitryUlyanov/online-neural-doodle
• demo: http://likemo.net/

neural image analogies: Generate image analogies using neural matching and blending

• github: https://github.com/awentzonline/image-analogies

Neural doodle with Keras

https://github.com/fchollet/keras/blob/master/examples/neural_doodle.py

Deep Dreams

deepdream

• github: https://github.com/google/deepdream

cnn-vis: Use CNNs to generate images

• github: https://github.com/jcjohnson/cnn-vis

bat-country: A lightweight, extendible, easy to use Python package for deep dreaming and image generation with Caffe and CNNs

• github: https://github.com/jrosebr1/bat-country

DeepDreaming with TensorFlow

• ipn: http://nbviewer.jupyter.org/github/tensorflow/tensorflow/blob/master/tensorflow/examples/tutorials/deepdream/deepdream.ipynb

deepdraw

• github: https://github.com/auduno/deepdraw

Understanding Deep Dreams

• blog: http://www.alanzucconi.com/2015/07/06/live-your-deepdream-how-to-recreate-the-inceptionism-effect/

Generating Deep Dreams

• blog：http://www.alanzucconi.com/2016/05/25/generating-deep-dreams/

Audio Deepdream: Optimizing Raw Audio With Convolutional Networks

• intro: Google Brain
• paper: https://wp.nyu.edu/ismir2016/wp-content/uploads/sites/2294/2016/08/ardila-audio.pdf
• examples: https://drive.google.com/drive/folders/0B7NUtSSG_AgqZWF1ajdhSjhEMlk

Emoji

Brewing EmojiNet

• blog: http://engineering.curalate.com/2016/01/20/emojinet.html
• website: https://emojini.curalate.com/

Image2Emoji: Zero-shot Emoji Prediction for Visual Media

• paper: http://isis-data.science.uva.nl/cgmsnoek/pub/cappallo-image2emoji-mm2015.pdf

Teaching Robots to Feel: Emoji & Deep Learning 👾 💭 💕

• blog: http://getdango.com/emoji-and-deep-learning.html
• app: https://play.google.com/store/apps/details?id=co.dango.emoji.gif

Text input with relevant emoji sorted with deeplearning

• homepage: http://codepen.io/Idlework/pen/xOgGqM

Sketch

Sketch-a-Net that Beats Humans

• project page: http://www.eecs.qmul.ac.uk/~tmh/downloads.html
• arxiv: http://arxiv.org/abs/1501.07873
• paper: http://www.eecs.qmul.ac.uk/~tmh/papers/yu2015sketchanet.pdf
• code: http://www.eecs.qmul.ac.uk/~tmh/downloads/SketchANet_Code.zip

How Do Humans Sketch Objects?

• project page: http://cybertron.cg.tu-berlin.de/eitz/projects/classifysketch/
• paper: http://cybertron.cg.tu-berlin.de/eitz/pdf/2012_siggraph_classifysketch.pdf
• github: https://github.com/Zebreu/SketchingAI
• gitxiv: http://gitxiv.com/posts/ZBCxEc9g3Fg5xCQ6n/sketchingai

Learning to Simplify: Fully Convolutional Networks for Rough Sketch Cleanup (SIGGRAPH 2016)

• homepage: http://hi.cs.waseda.ac.jp/~esimo/en/research/sketch/
• paper: http://hi.cs.waseda.ac.jp/~esimo/publications/SimoSerraSIGGRAPH2016.pdf

Convolutional Sketch Inversion

• arxiv: http://arxiv.org/abs/1606.03073
• review: https://www.technologyreview.com/s/601684/machine-vision-algorithm-learns-to-transform-hand-drawn-sketches-into-photorealistic-images/
• review: https://techcrunch.com/2016/07/24/researchers-use-neural-networks-to-turn-face-sketches-into-photos/

Sketch Me That Shoe (CVPR 2016)

• project page: http://www.eecs.qmul.ac.uk/~qian/Project_cvpr16.html
• paper: http://www.eecs.qmul.ac.uk/~qian/SketchMeThatShoe.pdf
• github: https://github.com/seuliufeng/DeepSBIR

Mastering Sketching: Adversarial Augmentation for Structured Prediction

• keywords: Sketch Simplification
• project page: http://hi.cs.waseda.ac.jp/~esimo/en/research/sketch_master/
• arxiv: https://arxiv.org/abs/1703.08966
• github: https://github.com/bobbens/sketch_simplification

SketchyGAN: Towards Diverse and Realistic Sketch to Image Synthesis

• intro: Georgia Institute of Technology
• arxiv: https://arxiv.org/abs/1801.02753

Image Stylization

Automatic Portrait Segmentation for Image Stylization

• intro: The Chinese University of Hong Kong & Adobe Research
• project page(data+code): http://xiaoyongshen.me/webpage_portrait/index.html
• paper: http://www.cse.cuhk.edu.hk/leojia/papers/portrait_eg16.pdf
• github(Tensorflow): https://github.com/PetroWu/AutoPortraitMatting

Transfiguring Portraits

• paper: http://homes.cs.washington.edu/~kemelmi/Transfiguring_Portraits_Kemelmacher_SIGGRAPH2016.pdf

Stylize Aesthetic QR Code

• intro: Zhengzhou University & Zhejiang University
• arxiv: https://arxiv.org/abs/1803.01146

Image Colorization

Deep Colorization

• paper: http://www.cs.cityu.edu.hk/~qiyang/publications/iccv-15.pdf

Learning Large-Scale Automatic Image Colorization

• paper: http://www.cv-foundation.org/openaccess/content_iccv_2015/papers/Deshpande_Learning_Large-Scale_Automatic_ICCV_2015_paper.pdf

Learning Representations for Automatic Colorization

• homepage: http://people.cs.uchicago.edu/~larsson/colorization/
• arxiv: http://arxiv.org/abs/1603.06668
• github: https://github.com/gustavla/autocolorize

Colorful Image Colorization

• intro: ECCV 2016
• project page: http://richzhang.github.io/colorization/
• arxiv: http://arxiv.org/abs/1603.08511
• github: https://github.com/richzhang/colorization
• demo: http://demos.algorithmia.com/colorize-photos/
• github: https://github.com/tdeboissiere/DeepLearningImplementations/tree/master/Colorful github(Tensorflow): https://github.com/nilboy/colorization-tf

Colorising Black & White Photos using Deep Learning

https://hackernoon.com/colorising-black-white-photos-using-deep-learning-4da22a05f531

Automatic Colorization (Tensorflow + VGG)

• blog: http://tinyclouds.org/colorize/

colornet: Neural Network to colorize grayscale images

https://github.com/pavelgonchar/colornet

Let there be Color!: Joint End-to-end Learning of Global and Local Image Priors for Automatic Image Colorization with Simultaneous Classification (SIGGRAPH 2016)

• homepage: http://hi.cs.waseda.ac.jp/~iizuka/projects/colorization/
• paper: http://hi.cs.waseda.ac.jp/~iizuka/projects/colorization/data/colorization_sig2016.pdf
• github(Torch7): https://github.com/satoshiiizuka/siggraph2016_colorization

Convolutional autoencoder to colorize greyscale images

• github: https://github.com/YerevaNN/neural-colorizer

Image-Color: A deep learning approach to colorizing images

• github: https://github.com/cameronfabbri/Colorful-Image-Colorization

Creating an artificial artist: Color your photos using Neural Networks

• blog: https://www.analyticsvidhya.com/blog/2016/11/creating-an-artificial-artist-color-your-photos-using-neural-networks/

Paints Chainer: line drawing colorization using chainer

• github: https://github.com/pfnet/PaintsChainer
• demo: http://paintschainer.preferred.tech/

Unsupervised Diverse Colorization via Generative Adversarial Networks

• arxiv: https://arxiv.org/abs/1702.06674

(DE)^2 CO: Deep Depth Colorization

https://arxiv.org/abs/1703.10881

A Neural Representation of Sketch Drawings

• intro: Google Brain
• arxiv: https://arxiv.org/abs/1704.03477

Real-Time User-Guided Image Colorization with Learned Deep Priors

• intro: SIGGRAPH 2017
• project page: https://richzhang.github.io/ideepcolor/
• arxiv: https://arxiv.org/abs/1705.02999
• github(official, Caffe): https://github.com/junyanz/interactive-deep-colorization

PixColor: Pixel Recursive Colorization

• intro: Google Research
• arxiv: https://arxiv.org/abs/1705.07208

cGAN-based Manga Colorization Using a Single Training Image

• intro: University of Tokyo
• arxiv: https://arxiv.org/abs/1706.06918

Interactive Deep Colorization With Simultaneous Global and Local Inputs

https://arxiv.org/abs/1801.09083

Image Colorization with Generative Adversarial Networks

https://arxiv.org/abs/1803.05400

Learning to Color from Language

• intro: Allen Institute of Artificial Intelligence & University of Massachusetts
• arxiv: https://arxiv.org/abs/1804.06026

Sounds

Visually Indicated Sounds

• project page: http://vis.csail.mit.edu/
• arxiv: http://arxiv.org/abs/1512.08512

Music

GRUV: Algorithmic Music Generation using Recurrent Neural Networks

• github: https://github.com/MattVitelli/GRUV

DeepHear - Composing and harmonizing music with neural networks

• website: http://web.mit.edu/felixsun/www/neural-music.html
• github: https://github.com/fephsun/neuralnetmusic

Using AutoHarp and a Character-Based RNN to Create MIDI Drum Loops

• website: http://www.inquisitivists.com/2015/09/16/using-autoharp-and-a-character-based-rnn-to-create-midi-drum-loops

Musical Audio Synthesis Using Autoencoding Neural Nets

• paper: http://www.cs.dartmouth.edu/~sarroff/papers/sarroff2014a.pdf
• github: https://github.com/woodshop/deepAutoController/tree/icmc_smc_2014
• video: https://vimeo.com/121827215

sound-rnn: Generating sound using recurrent neural networks

• github: https://github.com/johnglover/sound-rnn
• blog: http://www.johnglover.net/blog/generating-sound-with-rnns.html

Using LSTM Recurrent Neural Networks for Music Generation (Project for AI Prac Fall 2015 at Cornell)

• youtube: https://www.youtube.com/watch?v=aSr8_QQYpYM
• video: http://video.weibo.com/show?fid=1034:4be01d679bb1a68a634fe0f589caa779

Visually Indicated Sounds (MIT. 2015)

• arxiv: http://arxiv.org/abs/1512.08512

Training a Recurrent Neural Network to Compose Music

• blog: https://maraoz.com/2016/02/02/abc-rnn/

LSTM Realbook

• blog: https://keunwoochoi.wordpress.com/2016/02/19/lstm-realbook/
• github: https://github.com/keunwoochoi/lstm_real_book

LSTMetallica: Generation drum tracks by learning the drum tracks of 60 Metallica songs

• blog: https://keunwoochoi.wordpress.com/2016/02/23/lstmetallica/

deepjazz: Deep learning driven jazz generation using Keras & Theano!

• homepage: https://jisungk.github.io/deepjazz/
• github：https://github.com/jisungk/deepjazz

Magenta: Music and Art Generation with Machine Intelligence

• homepage: http://magenta.tensorflow.org/
• github: https://github.com/tensorflow/magenta

Music Transcription with Convolutional Neural Networks

• blog: https://www.lunaverus.com/cnn
• download: https://www.lunaverus.com/download

Long Short-Term Memory Recurrent Neural Network Architectures for Generating Music and Japanese Lyrics

• paper: http://cslab1.bc.edu/~csacademics/pdf/16Mikami.pdf
• github: https://github.com/mikamia

BachBot: Use deep learning to generate and harmonize music in the style of Bach

• intro: BachBot is a research project utilizing long short term memory (LSTMs) to generate Bach compositions
• homepage: http://bachbot.com/
• github: https://github.com/feynmanliang/bachbot

Generate Music in TensorFlow

• youtube: https://www.youtube.com/watch?v=ZE7qWXX05T0
• github: https://github.com/llSourcell/Music_Generator_Demo

Generate new lyrics in the style of any artist using LSTMs and TensorFlow

• github: https://github.com/dyelax/encore.ai

sound-GAN: Generative Adversial Network for music composition

• github: https://github.com/jacotar/sound-GAN

Analyzing Six Deep Learning Tools for Music Generation

• intro: Magenta / DeepJazz / BachBot / FlowMachines / WaveNet / GRUV
• blog: http://www.asimovinstitute.org/notes-vs-waves/

WIMP2: Creating Music with AI: Highlights of Current Research

• youtube: https://www.youtube.com/watch?v=5K4hn6cBUPU

Song From PI: A Musically Plausible Network for Pop Music Generation

• paper: http://openreview.net/pdf?id=ByBwSPcex
• project page: http://www.cs.toronto.edu/songfrompi/

Grammar Argumented LSTM Neural Networks with Note-Level Encoding for Music Composition

• arxiv: https://arxiv.org/abs/1611.05416

用TensorFlow生成周杰伦歌词

• blog: http://leix.me/2016/11/28/tensorflow-lyrics-generation/
• github: https://github.com/leido/char-rnn-cn

Hip-Hop - Generating lyrics with RNNs

• blog: http://affinelayer.com/hiphop/index.html

Metis Final Project: Music Composition with LSTMs

http://blog.naoya.io/metis-final-project-music-composition-with-lstms/

Neural Translation of Musical Style

• blog: http://imanmalik.com/cs/2017/06/05/neural-style.html
• github: https://github.com/imalikshake/StyleNet

Poetry

NeuralSnap: Generates poetry from images using convolutional and recurrent neural networks

• github: https://github.com/rossgoodwin/neuralsnap

Generating Chinese Classical Poems with RNN Encoder-Decoder

• arxiv: http://arxiv.org/abs/1604.01537

Chinese Poetry Generation with Planning based Neural Network

• intro: COLING 2016. University of Science and Technology of China & Baidu
• arxiv: https://arxiv.org/abs/1610.09889
• blog: http://freecoder.me/archives/213.html

Weiqi (Go)

Teaching Deep Convolutional Neural Networks to Play Go

• arxiv: http://arxiv.org/abs/1412.3409
• demo: https://chrisc36.github.io/deep-go/

Move Evaluation in Go Using Deep Convolutional Neural Networks(Google DeepMind, Google Brain)

• arxiv: http://arxiv.org/abs/1412.6564

Training Deep Convolutional Neural Networks to Play Go

• paper: http://jmlr.org/proceedings/papers/v37/clark15.pdf

Computer Go Research - The Challenges Ahead (Martin Müller. IEEE CIG 2015)

• homepage: https://webdocs.cs.ualberta.ca/~mmueller/talks.html

GoCNN: Using CNN for Go (Weiqi/Baduk) board evaluation with tensorflow

• github: https://github.com/jmgilmer/GoCNN

DarkGo: Go in Darknet

• homepage: http://pjreddie.com/darknet/darkgo-go-in-darknet/

BetaGo: Go bots for the people

• homepage: http://maxpumperla.github.io/betago/
• github: https://github.com/maxpumperla/betago

Deep Learning and the Game of Go

• book: https://www.manning.com/books/deep-learning-and-the-game-of-go
• github: https://github.com//maxpumperla/deep_learning_and_the_game_of_go

DarkForest

Better Computer Go Player with Neural Network and Long-term Prediction (Facebook AI Research)

• arxiv: http://arxiv.org/abs/1511.06410
• github: https://github.com/facebookresearch/darkforestGo
• MIT tech review: http://www.technologyreview.com/view/544181/how-facebooks-ai-researchers-built-a-game-changing-go-engine/

AlphaGo

Mastering the game of Go with deep neural networks and tree search

• intro: AlphaGo. Google DeepMind
• homepage: http://www.deepmind.com/alpha-go.html
• paper: https://storage.googleapis.com/deepmind-data/assets/papers/deepmind-mastering-go.pdf
• naturep page: http://www.nature.com/nature/journal/v529/n7587/full/nature16961.html
• paper: https://gogameguru.com/i/2016/03/deepmind-mastering-go.pdf
• slides: http://www.bioinfo.org.cn/~casp/temp/alphago_slides.pdf
• blog: http://www.furidamu.org/blog/2016/01/26/mastering-the-game-of-go-with-deep-neural-networks-and-tree-search/
• blog(“AlphaGo: From Intuitive Learning to Holistic Knowledge”): https://caminao.wordpress.com/2016/02/01/alphago/
• github: https://github.com/Rochester-NRT/AlphaGo

AlphaGo Teach

• intro: Let the AlphaGo Teaching Tool help you find new and creative ways of playing Go
• homepage: https://alphagoteach.deepmind.com/

AlphaGo的分析

• intro: by 田渊栋
• blog: http://zhuanlan.zhihu.com/yuandong/20607684

How Alphago Works

• slides: http://www.slideshare.net/ShaneSeungwhanMoon/how-alphago-works
• slides: http://pan.baidu.com/s/1qXwagGW

AlphaGo in Depth

• intro: by Mark Chang
• slides: http://www.slideshare.net/ckmarkohchang/alphago-in-depth?qid=283ab3bc-7d04-4e14-a205-b0b671ca4099
• mirror: https://pan.baidu.com/s/1i5JNeRj

Leela

• intro: Leela is a strong Go playing program combining advances in Go programming and further original research into a small, easy to use graphical interface.
• homepage: https://sjeng.org/leela.html

Mastering the game of Go without human knowledge

• nature page: http://www.nature.com/nature/journal/v550/n7676/full/nature24270.html
• paper: https://deepmind.com/documents/119/agz_unformatted_nature.pdf
• notes: https://blog.acolyer.org/2017/11/17/mastering-the-game-of-go-without-human-knowledge/

Computer Go & AlphaGo Zero

• youtube: https://www.youtube.com/watch?v=6fKG4wJ7uBk
• mirror: https://www.bilibili.com/video/av16428694/
• slides: https://drive.google.com/file/d/1rmUyIitEmAtMUKdKEnlHRfmXtpyoxxey/view

AlphaZero: Mastering Games without Human Knowledge - NIPS 2017

• intro: Keynote by David Silver on AlphaGo, AlphaGo Zero and AlphaZero, at the 2017 NIPS Deep Reinforcement Learning Symposium, 6 Dec, Long Beach, CA
• youtube: https://www.youtube.com/watch?v=A3ekFcZ3KNw
• mirror: https://www.bilibili.com/video/av17210816/

The future is here – AlphaZero learns chess

https://en.chessbase.com/post/the-future-is-here-alphazero-learns-chess

**AlphaGo Zero Cheat Sheet

https://applied-data.science/static/main/res/alpha_go_zero_cheat_sheet.png

Chess

Giraffe: Using Deep Reinforcement Learning to Play Chess

• intro: MSc thesis
• arxiv: http://arxiv.org/abs/1509.01549

Spawkfish: neural network based chess engine

• homepage: http://spawk.fish/

Chess position evaluation with convolutional neural network in Julia

• blog: http://int8.io/chess-position-evaluation-with-convolutional-neural-networks-in-julia/

Deep Learning for … Chess

• blog: http://blog.yhat.com/posts/deep-learning-chess.html
• github: https://github.com/erikbern/deep-pink

DeepChess: End-to-End Deep Neural Network for Automatic Learning in Chess

• intro: Winner of Best Paper Award in ICANN 2016
• arxiv: https://arxiv.org/abs/1711.09667
• paper: http://www.cs.tau.ac.il/~wolf/papers/deepchess.pdf
• github: https://github.com/mr-press/DeepChess

Mastering Chess and Shogi by Self-Play with a General Reinforcement Learning Algorithm

• intro: DeepMind
• arxiv: https://arxiv.org/abs/1712.01815

Game

Learning Game of Life with a Convolutional Neural Network

• blog: http://danielrapp.github.io/cnn-gol/
• github: https://github.com/DanielRapp/cnn-gol

Reinforcement Learning using Tensor Flow: A deep Q learning demonstration using Google Tensorflow

• github: https://github.com/nivwusquorum/tensorflow-deepq

Poker-CNN: A Pattern Learning Strategy for Making Draws and Bets in Poker Games Using Convolutional Networks

• arxiv: http://arxiv.org/abs/1509.06731
• paper: http://colinraffel.com/publications/aaai2016poker.pdf
• github: https://github.com/moscow25/deep_draw
• slides: https://drive.google.com/file/d/0B5eOIUHA0khiMjN1YnEtZHMwams/view
• slides: http://pan.baidu.com/s/1nu5zpZ7

TorchCraft: a Library for Machine Learning Research on Real-Time Strategy Games

• intro: Connecting Torch to StarCraft
• arxiv: https://arxiv.org/abs/1611.00625
• github: https://github.com/TorchCraft/TorchCraft

BlizzCon 2016 DeepMind and StarCraft II Deep Learning Panel Transcript

• part 1: http://starcraft.blizzplanet.com/blog/comments/blizzcon-2016-deepmind-and-starcraft-ii-deep-learning-panel-transcript
• part 2: http://starcraft.blizzplanet.com/blog/comments/blizzcon-2016-deepmind-and-starcraft-ii-deep-learning-panel-transcript/2

DeepStack: Expert-Level Artificial Intelligence in No-Limit Poker

• arxiv: https://arxiv.org/abs/1701.01724
• github: https://github.com/lifrordi/DeepStack-Leduc

Gym StarCraft: StarCraft environment for OpenAI Gym, based on Facebook’s TorchCraft

• intro: Gym StarCraft is an environment bundle for OpenAI Gym. It is based on Facebook’s TorchCraft, which is a bridge between Torch and StarCraft for AI research.
• github: https://github.com/deepcraft/gym-starcraft

Multiagent Bidirectionally-Coordinated Nets for Learning to Play StarCraft Combat Games

https://arxiv.org/abs/1703.10069

Learning Macromanagement in StarCraft from Replays using Deep Learning

• intro: CIG 2017. IT University of Copenhagen
• arxiv: https://arxiv.org/abs/1707.03743

Multi-platform Version of StarCraft: Brood War in a Docker Container: Technical Report

• intro: Czech Technical University in Prague
• arxiv: https://arxiv.org/abs/1801.02193
• gihtub: https://github.com/Games-and-Simulations/sc-docker

DeepLeague

DeepLeague: leveraging computer vision and deep learning on the League of Legends mini map + giving away a dataset of over 100,000 labeled images to further esports analytics research

• blog: https://medium.com/@farzatv/deepleague-leveraging-computer-vision-and-deep-learning-on-the-league-of-legends-mini-map-giving-d275fd17c4e0

DeepLeague (Part 2): The Technical Details

• blog: https://medium.com/@farzatv/deepleague-part-2-the-technical-details-374439e7e09a
• github: https://github.com/farzaa/DeepLeague

Courses

Learning Machines

http://www.patrickhebron.com/learning-machines/

Learning Bit by Bit

https://itp.nyu.edu/varwiki/Syllabus/LearningBitbyBitS10

MACHINE LEARNING FOR MUSICIANS AND ARTISTS (Course opens January 2016)

https://www.kadenze.com/courses/machine-learning-for-musicians-and-artists/info

Machine learning for artists @ ITP-NYU, Spring 2016

• videos/lectures/course notes: http://ml4a.github.io/classes/itp-S16/
• index: http://ml4a.github.io/index/
• github: https://github.com/ml4a/ml4a.github.io
• notes: http://www.kdnuggets.com/2016/04/machine-learning-artists-video-lectures-notes.html
• blog: https://medium.com/@genekogan/machine-learning-for-artists-e93d20fdb097#.25w95beqb

Machine Learning for Artists @ OpenDot, November 2016

• homepage: http://ml4a.github.io/classes/opendot/

The Neural Aesthetic @ SchoolOfMa, Summer 2016

http://ml4a.github.io/classes/neural-aesthetic/

Blogs

Review of machine / deep learning in an artistic context

https://medium.com/@memoakten/machine-deep-learning-in-an-artistic-context-441f28774bcc#.gegpq99ag

Apprentice Work

https://www.technologyreview.com/s/600762/apprentice-work/

Exploring the Intersection of Art and Machine Intelligence

http://googleresearch.blogspot.jp/2016/02/exploring-intersection-of-art-and.html

Using machine learning to generate music

http://www.datasciencecentral.com/profiles/blogs/using-machine-learning-to-generate-music

art in the age of machine intelligence

https://medium.com/artists-and-machine-intelligence/what-is-ami-ccd936394a83#.hyt4ei9a9

Understanding Aesthetics with Deep Learning

https://devblogs.nvidia.com/parallelforall/understanding-aesthetics-deep-learning/

Go, Marvin Minsky, and the Chasm that AI Hasn’t Yet Crossed

blog: https://medium.com/backchannel/has-deepmind-really-passed-go-adc85e256bec#.inx8nfid0

A Return to Machine Learning

• intro: This post is aimed at artists and other creative people who are interested in a survey of recent developments in machine learning research that intersect with art and culture.
• blog: https://medium.com/@kcimc/a-return-to-machine-learning-2de3728558eb#.bp2b1ax2x

Resources

Music, Art and Machine Intelligence Workshop 2016

• youtube: https://www.youtube.com/playlist?list=PL8h7Hk91ScJ2NKFS_i8y4DYk4Zg1LHn0a

Papers

Bayesian

Tutorials

Deep learning

• intro: From Wikipedia, the free encyclopedia
• blog: https://www.wikiwand.com/en/Deep_learning

VGG Convolutional Neural Networks Practical

• homepage: http://www.robots.ox.ac.uk/~vgg/practicals/cnn/index.html
• github: https://github.com/vedaldi/practical-cnn

Hacker’s guide to Neural Networks

http://karpathy.github.io/neuralnets/

Deep Learning Tutorials

• website: http://deeplearning.net/tutorial/
• code: https://github.com/lisa-lab/DeepLearningTutorials

Deep Learning in a Nutshell: Core Concepts

http://devblogs.nvidia.com/parallelforall/deep-learning-nutshell-core-concepts/

Deep Learning in a Nutshell: History and Training

http://devblogs.nvidia.com/parallelforall/deep-learning-nutshell-history-training/

A Deep Learning Tutorial: From Perceptrons to Deep Networks

http://www.toptal.com/machine-learning/an-introduction-to-deep-learning-from-perceptrons-to-deep-networks

Deep Neural Networks (with Python code)

• paper: http://scholarbank.nus.edu.sg/bitstream/handle/10635/120564/DeepNeuralNetworks.pdf?sequence=1

Three Classes of Deep Learning Architectures and Their Applications: A Tutorial Survey

• paper: http://research.microsoft.com/pubs/192937/Transactions-APSIPA.pdf

Stanford Unsupervised Feature Learning and Deep Learning Tutorial: UFLDL Tutorial

• homepage: http://ufldl.stanford.edu/tutorial/
• programming exercises: https://github.com/amaas/stanford_dl_ex

The Unreasonable Effectiveness of Deep Learning (LeCun)

• slides: http://www.ee.ucl.ac.uk/sahd2014/resources/LeCun.pdf

Deep learning from the bottom up

• blog: https://www.metacademy.org/roadmaps/rgrosse/deep_learning

Introduction to Deep Learning with Python (By Alec Radford. Theano)

• youtube: https://www.youtube.com/watch?v=S75EdAcXHKk&hd=1

New to deep learning? Here are 4 easy lessons from Google

• blog: https://gigaom.com/2015/01/29/new-to-deep-learning-here-are-4-easy-lessons-from-google/

Deep Learning 101

• blog: http://markus.com/deep-learning-101/

Neural Networks Demystified

• Part 1: Data and Architecture: https://www.youtube.com/watch?v=bxe2T-V8XRs
• Part 2: Forward Propagation: https://www.youtube.com/watch?v=UJwK6jAStmg
• Part 3: Gradient Descent: https://www.youtube.com/watch?v=5u0jaA3qAGk
• Part 4: Backpropagation: https://www.youtube.com/watch?v=GlcnxUlrtek
• Part 5: Numerical Gradient Checking: https://www.youtube.com/watch?v=pHMzNW8Agq4
• Part 6: Training: https://www.youtube.com/watch?v=9KM9Td6RVgQ

• Part 7: Overfitting, Testing, and Regularization: https://www.youtube.com/watch?v=S4ZUwgesjS8

• all-pack: http://pan.baidu.com/s/1dDq5oNB

Deep Learning SIMPLIFIED

• playlist: https://www.youtube.com/playlist?list=PLjJh1vlSEYgvGod9wWiydumYl8hOXixNu

A ‘Brief’ History of Neural Nets and Deep Learning

• part 1: http://www.andreykurenkov.com/writing/a-brief-history-of-neural-nets-and-deep-learning/
• part 2: http://www.andreykurenkov.com/writing/a-brief-history-of-neural-nets-and-deep-learning-part-2/
• part 3: http://www.andreykurenkov.com/writing/a-brief-history-of-neural-nets-and-deep-learning-part-3/
• part 4: http://www.andreykurenkov.com/writing/a-brief-history-of-neural-nets-and-deep-learning-part-4/

Deep Neural Networks — An Overview

https://medium.com/@asjad/deep-neural-networks-an-overview-480112b12a13#.i7apzmnso

A Tutorial on Deep Neural Networks for Intelligent Systems

• arxiv: http://arxiv.org/abs/1603.07249

Deep Learning for Computer Vision – Introduction to Convolution Neural Networks

• blog: http://www.analyticsvidhya.com/blog/2016/04/deep-learning-computer-vision-introduction-convolution-neural-networks/

BI Lab Deep Learning Tutorial

• github: https://github.com/bi-lab/deeplearning_tutorial

Deep Learning Tutorials

• github: https://github.com/sjchoi86/Deep-Learning-101

Neural Network Architectures

• blog: http://culurciello.github.io/tech/2016/06/04/nets.html

A Practical Introduction to Deep Learning with Caffe and Python

• blog: http://adilmoujahid.com/posts/2016/06/introduction-deep-learning-python-caffe/

Notes on Convolutional Neural Networks

• homepage: http://cogprints.org/5869/
• paper: http://cogprints.org/5869/1/cnn_tutorial.pdf

Feed Forward and Backward Run in Deep Convolution Neural Network

• intro: 20th International Conference on Computer Vision and Image Processing
• arxiv: https://arxiv.org/abs/1711.03278

Convolutional Networks

http://deeplearning4j.org/convolutionalnets.html

Exploring convolutional neural networks with DL4J

• blog: http://brooksandrew.github.io/simpleblog/articles/convolutional-neural-network-training-with-dl4j/

Understanding Convolutional Neural Networks

• arxiv: http://arxiv.org/abs/1605.09081

Laws, Sausages and ConvNets

• blog: http://www.trivialorwrong.com/2016/06/01/laws-sausages-and-convnets.html

Convolutional Neural Networks (CNNs): An Illustrated Explanation

• blog: http://xrds.acm.org/blog/2016/06/convolutional-neural-networks-cnns-illustrated-explanation/

intro_deep: Introduction tutorials to deep learning with Theano and OpenDeep

• slides: https://docs.google.com/presentation/d/1cg9Tn2wWwqJmaSSDnlBDBEETD5SyV6TJSD8qiDJFgEM
• mirror: http://pan.baidu.com/s/1hqIR0yC
• youtube: https://www.youtube.com/watch?v=afUvcD3tEoQ
• mirror: http://pan.baidu.com/s/1qWHp7xa
• github: https://github.com/mbeissinger/intro_deep

Deep Learning on Java by Breandan Considine

• video: https://speakerdeck.com/breandan/deep-learning-on-java

Using Convolutional Neural Networks and TensorFlow for Image Classification (NYC TensorFlow meetup)

• blog: http://blog.altoros.com/using-convolutional-neural-networks-and-tensorflow-for-image-classification-and-search.html

Neural networks with Theano and Lasagne

• youtube: https://www.youtube.com/watch?v=dtGhSE1PFh0
• mirror: http://pan.baidu.com/s/1kUl3PvL
• github: https://github.com/ebenolson/pydata2015

Introduction to Deep Learning

• github: https://github.com/rouseguy/intro2deeplearning
• slides: https://speakerdeck.com/bargava/introduction-to-deep-learning

Introduction to Deep Learning for Image Recognition - SciPy US 2016

• github: https://github.com/rouseguy/scipyUS2016_dl-image
• slides: https://speakerdeck.com/bargava/introduction-to-deep-learning-for-image-processing

Deep learning tutorials (2nd ed.)

• github: https://github.com/sjchoi86/dl_tutorials

A Beginner’s Guide To Understanding Convolutional Neural Networks

• part 1: https://adeshpande3.github.io/adeshpande3.github.io/A-Beginner’s-Guide-To-Understanding-Convolutional-Neural-Networks/

A Beginner’s Guide To Understanding Convolutional Neural Networks Part 2

• part 2: https://adeshpande3.github.io/adeshpande3.github.io/A-Beginner’s-Guide-To-Understanding-Convolutional-Neural-Networks-Part-2/

The 9 Deep Learning Papers You Need To Know About (Understanding CNNs Part 3)

• part 3: https://adeshpande3.github.io/adeshpande3.github.io/The-9-Deep-Learning-Papers-You-Need-To-Know-About.html

Deep Learning Part 1: Comparison of Symbolic Deep Learning Frameworks

• blog: http://blog.revolutionanalytics.com/2016/08/deep-learning-part-1.html

Deep Learning Part 2: Transfer Learning and Fine-tuning Deep Convolutional Neural Networks

• blog: http://blog.revolutionanalytics.com/2016/08/deep-learning-part-2.html

Deep Learning Part 3: Combining Deep Convolutional Neural Network with Recurrent Neural Network

• blog: http://blog.revolutionanalytics.com/2016/09/deep-learning-part-3.html

Introduction to Deep Learning for Image Processing

• slides: https://speakerdeck.com/bargava/introduction-to-deep-learning-for-image-processing

The best explanation of Convolutional Neural Networks on the Internet!

• blog: https://medium.com/technologymadeeasy/the-best-explanation-of-convolutional-neural-networks-on-the-internet-fbb8b1ad5df8

The Evolution and Core Concepts of Deep Learning & Neural Networks

• blog: https://www.analyticsvidhya.com/blog/2016/08/evolution-core-concepts-deep-learning-neural-networks/

An Intuitive Explanation of Convolutional Neural Networks

• blog: https://ujjwalkarn.me/2016/08/11/intuitive-explanation-convnets/

How Convolutional Neural Networks Work

• blog: http://brohrer.github.io/how_convolutional_neural_networks_work.html

Preliminary Note on the Complexity of a Neural Network

• blog: http://r2rt.com/preliminary-note-on-the-complexity-of-a-neural-network.html

Deep Learning Tutorial

• intro: Hung-yi Lee. 李宏毅
• slides: http://www.slideshare.net/tw_dsconf/ss-62245351?qid=c0f0f97a-6ca8-4df0-97e2-984452215ee7&v=&b=&from_search=1
• mirror: https://pan.baidu.com/s/1mhMhuFQ

Jupyter notebooks and code for Intro to DL talk at Genesys

• blog: http://sujitpal.blogspot.com/2016/08/kerasjupyter-notebooks-for-my.html
• github: https://github.com/sujitpal/intro-dl-talk-code

Learn Deep Learning the Hard Way

• blog: https://medium.com/artifacia/learn-deep-learning-the-hard-way-e5d844f9fbc1#.yitf25xg5

A Complete Guide on Getting Started with Deep Learning in Python

• blog: https://www.analyticsvidhya.com/blog/2016/08/deep-learning-path/

Deep learning for complete beginners: Recognising handwritten digits

• blog: http://online.cambridgecoding.com/notebooks/cca_admin/deep-learning-for-complete-beginners-recognising-handwritten-digits

Deep learning for complete beginners: Using convolutional nets to recognise images

• blog: http://online.cambridgecoding.com/notebooks/cca_admin/convolutional-neural-networks-with-keras

Deep learning for complete beginners: neural network fine-tuning techniques

• blog: http://online.cambridgecoding.com/notebooks/cca_admin/neural-networks-tuning-techniques

How do Convolutional Neural Networks work?

http://brohrer.github.io/how_convolutional_neural_networks_work.html

Creating a Neural Network That Can Tell if a Name Is Male or Female, in JavaScript

• blog: https://medium.com/@hathibel/creating-a-neural-network-that-can-tell-if-a-name-is-male-or-female-in-javascript-3061029be396#.89l3znc2l

Softmax Classifiers Explained

• blog: http://www.pyimagesearch.com/2016/09/12/softmax-classifiers-explained/

The Softmax function and its derivative

• blog: http://eli.thegreenplace.net/2016/the-softmax-function-and-its-derivative/

How an algorithm behind Deep Learning works

• video: http://blog.revolutionanalytics.com/2016/09/how-the-algorithm-behind-deep-learning-works.html
• slides: https://github.com/brohrer/public-hosting/raw/master/How_CNNs_work.pdf
• blog: http://www.kdnuggets.com/2016/08/brohrer-convolutional-neural-networks-explanation.html
• mirror: http://v.youku.com/v_show/id_XMTcyNTgwNDQyOA==.html

The Neural Network Zoo

http://www.asimovinstitute.org/neural-network-zoo/

Recognising Beer with TensorFlow

• blog: https://medium.com/@chrishawkins/recognising-beer-with-tensorflow-9dedfee3c3c0#.pn5gm3fgc
• gist: https://gist.github.com/chrishawkins/177e37756c833768a21d446cc4921c6e

Deep learning architecture diagrams

• intro: LSTM diagrams
• blog: http://fastml.com/deep-learning-architecture-diagrams/

Getting Started with Deep Learning and Python

• blog: http://www.pyimagesearch.com/2014/09/22/getting-started-deep-learning-python

Deep Learning Practicals

• intro: Video playlist of Torch Video Tutorials
• youtube: https://www.youtube.com/playlist?list=PLLHTzKZzVU9ebuL6DCclzI54MrPNFGqbW
• mirror: https://pan.baidu.com/s/1skMFGkt

A simple workflow for deep learning

• blog: https://cartesianfaith.com/2016/09/29/a-simple-workflow-for-deep-learning/
• github: https://github.com/zatonovo/deep_learning_ex

A primer on universal function approximation with deep learning (in Torch and R)

• blog: https://cartesianfaith.com/2016/09/23/a-primer-on-universal-function-approximation-with-deep-learning-in-torch-and-r/

An Introduction to Implementing Neural Networks using TensorFlow

https://www.analyticsvidhya.com/blog/2016/10/an-introduction-to-implementing-neural-networks-using-tensorflow/

A Gentle Introduction to Convolutional Neural Networks

• blog: https://miguelgfierro.com/blog/2016/a-gentle-introduction-to-convolutional-neural-networks/

Beginning Machine Learning with Keras and TensorFlow

• blog: http://blog.thoughtram.io/machine-learning/2016/09/23/beginning-ml-with-keras-and-tensorflow.html

Shortest Way to Deep Learning

• blog: https://blog.asadmemon.com/shortest-way-to-deep-learning-41e704d65ef#.19iowho8t

Deep learning with Matlab

• intro: Covered topics of the presentation: Machine learning workflow, Extracting feaures from images (colours, edges, corners, etc.)
• youtube: https://www.youtube.com/watch?v=r4D3NxQ0Xhg

Convolutional neural networks for computer vision with Matlab

• youtube: https://www.youtube.com/watch?v=MwUic9FCBJI

Neural Net Computing Explodes

• blog: http://semiengineering.com/neural-net-computing-explodes/

Tutorial: Optimizing Neural Networks using Keras (with Image recognition case study)

https://www.analyticsvidhya.com/blog/2016/10/tutorial-optimizing-neural-networks-using-keras-with-image-recognition-case-study/

15 Deep Learning Tutorials

• blog: http://www.datasciencecentral.com/profiles/blogs/15-deep-learning-tutorials

Deep Learning Episode 1: Optimizing DeepMind’s A3C on Torch

http://www.allinea.com/blog/201607/deep-learning-episode-1-optimizing-deepminds-a3c-torch

Deep Learning Episode 2: Scaling TensorFlow over multiple EC2 GPU nodes

http://www.allinea.com/blog/201608/deep-learning-episode-2-scaling-tensorflow-over-multiple-ec2-gpu-nodes

Deep Learning Episode 3: Supercomputer vs Pong

http://www.allinea.com/blog/201610/deep-learning-episode-3-supercomputer-vs-pong

Deep Learning Episode 4: Supercomputer vs Pong II

http://www.allinea.com/blog/201610/deep-learning-episode-4-supercomputer-vs-pong-ii

Nuts and Bolts of Applying Deep Learning — Summary

• blog: https://medium.com/@aniketvartak/nuts-and-bolts-of-applying-deep-learning-summary-84b8a8e873d5#.y7xidlhb4

Intro to Deep Learning for Computer Vision

http://chaosmail.github.io/deeplearning/2016/10/22/intro-to-deep-learning-for-computer-vision/

If I Can Learn to Play Atari, I Can Learn TensorFlow

• intro: Here is a summary of new deep learning libraries, tools, and updates to existing frameworks.
• blog: https://dzone.com/articles/deep-learning-resources

TensorFlow workshop materials

• github: https://github.com/amygdala/tensorflow-workshop

Some theorems on deep learning

• intro: Tomaso Poggio [MIT]
• youtube: https://www.youtube.com/watch?v=YVjvRvvVn4w
• mirror: https://pan.baidu.com/s/1o8o7LjW

Pokemon, Colors, and Deep Learning

• blog: https://juandes.com/pokemon-colors-and-deep-learning-95fb715be46
• github: https://github.com/juandes/PokemonTypesDeepLearning

Why Deep Learning is Radically Different from Machine Learning

• blog: https://medium.com/intuitionmachine/why-deep-learning-is-radically-different-from-machine-learning-945a4a65da4d#.yuqxqxpsz

Deep Learning: The Unreasonable Effectiveness of Randomness

• blog: https://medium.com/intuitionmachine/deep-learning-the-unreasonable-effectiveness-of-randomness-14d5aef13f87#.1n775yl4i

Deep Meta-Learning : Machines now Bootstrap Themselves

• blog: https://medium.com/intuitionmachine/deep-learning-can-now-create-itself-92e7ff0d59a7#.ml0dy8m9a

Are Deep Neural Networks Creative?

• blog: http://www.kdnuggets.com/2016/05/deep-neural-networks-creative-deep-learning-art.html

Are Deep Neural Networks Creative? v2

• blog: http://approximatelycorrect.com/2016/11/11/are-deep-neural-networks-creative/

Develop/Train A Convolutional Neural Netwok For MNIST Dataset

• github: https://github.com/mirjalil/DataScience/blob/master/notebooks/deeplearning/tensorflow_03_CNN.ipynb

Rethinking Generalization in Deep Learning

https://medium.com/intuitionmachine/rethinking-generalization-in-deep-learning-ec66ed684ace#.tcnsqik5w

The hard thing about deep learning

• blog: https://www.oreilly.com/ideas/the-hard-thing-about-deep-learning

The hard thing about deep learning

• blog: https://www.oreilly.com/ideas/the-hard-thing-about-deep-learning

Introduction to Autoencoders

• blog: https://pgaleone.eu/neural-networks/2016/11/18/introduction-to-autoencoders/

Two Days to a Demo

• blog: https://developer.nvidia.com/embedded/twodaystoademo

Deep Learning Tutorials for 10 Weeks

• github: https://github.com/sjchoi86/dl_tutorials_4th

Deep Learning in Clojure With Cortex

• blog: http://gigasquidsoftware.com/blog/2016/12/27/deep-learning-in-clojure-with-cortex/

A Guide to Deep Learning by YerevaNN

• blog: http://yerevann.com/a-guide-to-deep-learning/

Learning to Learn, to Program, to Explore and to Seek Knowledge

• intro: Nando de Freitas, NIPS 2016
• youtube: https://www.youtube.com/watch?v=tPWGGwmgwG0
• mirror: https://pan.baidu.com/s/1b2VZsE

Have Fun with Machine Learning: A Guide for Beginners

• intro: An absolute beginner’s guide to Machine Learning and Image Classification with Neural Networks
• github: https://github.com/humphd/have-fun-with-machine-learning

Deep Learning Cheat Sheet

• blog: https://hackernoon.com/deep-learning-cheat-sheet-25421411e460#.e0jwzxpkm

How to train your Deep Neural Network

http://rishy.github.io//ml/2017/01/05/how-to-train-your-dnn/

A deep learning traffic light detector using dlib and a few images from Google street view

• blog: https://sagivtech.com/2016/11/10/post-2/

Recognizing Traffic Lights With Deep Learning

• blog: https://medium.com/@davidbrai/recognizing-traffic-lights-with-deep-learning-23dae23287cc#.k22tnf37a
• github: https://github.com/davidbrai/deep-learning-traffic-lights

Tutorials for deep learning

• github: https://github.com/oduerr/dl_tutorial

The Holographic Principle: Why Deep Learning Works

https://medium.com/intuitionmachine/the-holographic-principle-and-deep-learning-52c2d6da8d9

Deep Neural Networks - A Brief History

• arxiv: https://arxiv.org/abs/1701.05549

Fundamental Deep Learning code in TFLearn, Keras, Theano and TensorFlow

• blog: https://insights.untapt.com/fundamental-deep-learning-code-in-tflearn-keras-theano-and-tensorflow-66be10a03227#.hoaw8fp9p
• slides: https://static1.squarespace.com/static/5362fa11e4b035b5651b7f7e/t/588fb378cd0f687201a2e317/1485812622873/Jon_Krohn_NYHackR_Deep_Learning_2017_01_30.pdf

Deep Neural Network from scratch

https://matrices.io/deep-neural-network-from-scratch/

Convolutional Neural Networks

https://github.com/Alfredvc/cnn_workshop

Exploring Optimizers

https://github.com//KeremTurgutlu/deeplearning/blob/master/Exploring%20Optimizers.ipynb

A Gentle Introduction to Exploding Gradients in Neural Networks

https://machinelearningmastery.com/exploding-gradients-in-neural-networks/

Only Numpy: (Why I do Manual Back Propagation) Implementing Multi Channel/Layer Convolution Neural Network on Numpy with Interactive Code

https://medium.com/swlh/only-numpy-why-i-do-manual-back-propagation-implementing-multi-channel-layer-convolution-neural-7d83242fcc24

92.45% on CIFAR-10 in Torch

• intro: Dropout after Convolution
• blog: http://torch.ch/blog/2015/07/30/cifar.html

Convolution

Understanding Convolutions

• blog: http://colah.github.io/posts/2014-07-Understanding-Convolutions/

Note on the implementation of a convolutional neural networks

• intro: CS231n, Convolutional layer, Pooling layer, Forward pass, Backward pass
• blog: http://cthorey.github.io./backprop_conv/
• github: https://github.com/cthorey/CS231

Convolution in Caffe: a memo

• blog: https://github.com/Yangqing/caffe/wiki/Convolution-in-Caffe:-a-memo

我对卷积的理解

• blog: http://mengqi92.github.io/2015/10/06/convolution/
• blog: https://segmentfault.com/a/1190000004706582

An Analysis of Convolution for Inference

http://www.slideshare.net/nervanasys/an-analysis-of-convolution-for-inference

Understanding Convolution in Deep Learning

• blog: http://timdettmers.com/2015/03/26/convolution-deep-learning/

A guide to convolution arithmetic for deep learning

• arxiv: http://arxiv.org/abs/1603.07285
• github: https://github.com/vdumoulin/conv_arithmetic

Going beyond full utilization: The inside scoop on Nervana’s Winograd kernels

• blog: https://www.nervanasys.com/winograd-2/

Playing with convolutions in TensorFlow: From a short introduction to convolution to a complete model

• blog: http://mourafiq.com/2016/08/10/playing-with-convolutions-in-tensorflow.html
• github: https://github.com/mouradmourafiq/tensorflow-convolution-models

How convolutional neural networks see the world: An exploration of convnet filters with Keras

• blog: https://blog.keras.io/how-convolutional-neural-networks-see-the-world.html

One by One [ 1 x 1 ] Convolution - counter-intuitively useful

http://iamaaditya.github.io/2016/03/one-by-one-convolution/

Checkerboard artifact free sub-pixel convolution: A note on sub-pixel convolution, resize convolution and convolution resize

• intro: Twitter
• arxiv: https://arxiv.org/abs/1707.02937

Receptive Field

A guide to receptive field arithmetic for Convolutional Neural Networks

https://medium.com/@nikasa1889/a-guide-to-receptive-field-arithmetic-for-convolutional-neural-networks-e0f514068807

Momentum

Why Momentum Really Works

• blog: http://distill.pub/2017/momentum/
• github: https://github.com/distillpub/post–momentum

maxDNN

maxDNN: An Efficient Convolution Kernel for Deep Learning with Maxwell GPUs

• arxiv: http://arxiv.org/abs/1501.06633
• github: https://github.com/eBay/maxDNN

GEMM (General Matrix Matrix Multiply)

Why GEMM is at the heart of deep learning

• blog: http://petewarden.com/2015/04/20/why-gemm-is-at-the-heart-of-deep-learning/

A full walk through of the SGEMM implementation

• github-wiki: https://github.com/NervanaSystems/maxas/wiki/SGEMM

Backpropagation

Learning representations by back-propagating errors

Learning Internal Representations by Error Propagating

• author: David E. Rumelhart, Geoffrey E. Hinton & Ronald J. Williams. 1986
• paper: http://www.nature.com/nature/journal/v323/n6088/pdf/323533a0.pdf
• mirror: http://pan.baidu.com/s/1bo30gHp
• mirror: http://pan.baidu.com/s/1kVfJ4of

Calculus on Computational Graphs: Backpropagation

• blog: http://colah.github.io/posts/2015-08-Backprop/

Styles of Truncated Backpropagation

• blog: http://r2rt.com/styles-of-truncated-backpropagation.html

Is BackPropagation Necessary?

• blog: http://deliprao.com/archives/191

Backpropagation In Convolutional LSTMs

https://www.doc.ic.ac.uk/~ahanda/ConvLSTMs.pdf

Backward Pass on Conv Layer

Convolutional Neural Networks backpropagation: from intuition to derivation

• blog: https://grzegorzgwardys.wordpress.com/2016/04/22/8/

Backpropagation In Convolutional Neural Networks

• blog: http://www.jefkine.com/general/2016/09/05/backpropagation-in-convolutional-neural-networks/

Why do we rotate weights when computing the gradients in a convolution layer of a convolution network?

http://soumith.ch/ex/pages/2014/08/07/why-rotate-weights-convolution-gradient/

Note on the implementation of a convolutional neural networks

http://cthorey.github.io./backprop_conv/

Attention

Attention in a Convolutional Neural Net

• blog: http://www.danvatterott.com/blog/2016/09/20/attention-in-a-convolutional-neural-net/

Attention-based Networks

• intro: M. Malinowski. Max Planck Institut Informatik
• slides: http://download.mpi-inf.mpg.de/d2/mmalinow-slides/attention_networks.pdf

Attention in Neural Networks and How to Use It

http://akosiorek.github.io/ml/2017/10/14/visual-attention.html

Softmax

Hierarchical softmax and negative sampling: short notes worth telling

https://towardsdatascience.com/hierarchical-softmax-and-negative-sampling-short-notes-worth-telling-2672010dbe08

Caffe

DIY Deep Learning for Vision: a Hands-On Tutorial with Caffe

• homepage: http://tutorial.caffe.berkeleyvision.org/
• slides: https://docs.google.com/presentation/d/1UeKXVgRvvxg9OUdh_UiC5G71UMscNPlvArsWER41PsU/edit#slide=id.gc2fcdcce7_216_0

Deep learning tutorial on Caffe technology : basic commands, Python and C++ code

http://christopher5106.github.io/deep/learning/2015/09/04/Deep-learning-tutorial-on-Caffe-Technology.html

Using Caffe with your own dataset

https://medium.com/@alexrachnog/using-caffe-with-your-own-dataset-b0ade5d71233

OpenCV 3.0.0-dev: Load Caffe framework models

http://docs.opencv.org/master/d5/de7/tutorial_dnn_googlenet.html#gsc.tab=0

Chainer

Chainer Info

https://github.com/hidetomasuoka/chainer-info

Keras

Keras tutorial

• intro: Tutorial teaching the basics of Keras and some deep learning concepts
• github: https://github.com/jfsantos/keras-tutorial

Keras Tutorial: The Ultimate Beginner’s Guide to Deep Learning in Python

https://elitedatascience.com/keras-tutorial-deep-learning-in-python

Deep Learning with Keras: Tutorial @ EuroScipy 2016

• github: https://github.com/leriomaggio/deep-learning-keras-tensorflow

Transfer Learning and Fine Tuning for Cross Domain Image Classification with Keras

• slides: http://www.slideshare.net/sujitpal/transfer-learning-and-fine-tuning-for-cross-domain-image-classification-with-keras
• mirror: https://pan.baidu.com/s/1gfn1xuj
• github: https://github.com/sujitpal/fttl-with-keras

MXNet

10 Deep Learning projects based on Apache MXNet

https://medium.com/@julsimon/10-deep-learning-projects-based-on-apache-mxnet-8231109f3f64

Awesome MXNet(Beta)

https://github.com/chinakook/Awesome-MXNet

TVM

Optimize Deep Learning GPU Operators with TVM: A Depthwise Convolution Example

http://tvmlang.org/2017/08/22/Optimize-Deep-Learning-GPU-Operators-with-TVM-A-Depthwise-Convolution-Example.html

Theano

Theano Tutorial @ LTI, Carnegie Mellon University

• github: https://github.com/k-kawakami/Theano_Tutorial/blob/master/Theano_Tutorial.ipynb

An Introduction to MXNet/Gluon

• intro: @李沐
• github: https://github.com/mli/cvpr17

TensorFlow

LearningTensorFlow.com: A beginners guide to a powerful framework.

• homepge: http://learningtensorflow.com/index.html

TensorFlow Examples: TensorFlow tutorials and code examples for beginners

• github: https://github.com/aymericdamien/TensorFlow-Examples

Awesome TensorFlow: A curated list of awesome TensorFlow experiments, libraries, and projects

• github: https://github.com/jtoy/awesome-tensorflow/

The Good, Bad, & Ugly of TensorFlow: A survey of six months rapid evolution (+ tips/hacks and code to fix the ugly stuff)

• blog: https://indico.io/blog/the-good-bad-ugly-of-tensorflow/

Tensorflow Tutorials using Jupyter Notebook

• github: https://github.com/sjchoi86/Tensorflow-101

TensorFlow Tutorial

• homepage: http://terryum.io/ml_practice/2016/05/28/TFIntroSlides/
• slides: https://s3.amazonaws.com/www.terryum.io/images/TensorFlow_Intro_160529.pptx
• mirror: http://pan.baidu.com/s/1c5cICY
• github: https://github.com/terryum/TensorFlow_Exercises

FIRST CONTACT WITH TENSORFLOW

• ebook: http://www.jorditorres.org/first-contact-with-tensorflow/

Introduction to TensorFlow

• github: https://github.com/nihit/TensorFlow101

TensorFlow-Tutorials: Simple tutorials using Google’s TensorFlow Framework

• github: https://github.com/nlintz/TensorFlow-Tutorials

Neural Network Toolbox on TensorFlow

• github: https://github.com/ppwwyyxx/tensorpack

Awesome Tensorflow Implementations

• github: https://github.com/TensorFlowKR/awesome_tensorflow_implementations

The Ultimate List of TensorFlow Resources: Books, Tutorials & More

• blog: https://hackerlists.com/tensorflow-resources/

Install TensorFlow: Slides and code from our TensorFlow Workshop

• github: https://github.com/random-forests/tensorflow-workshop

A Tour of TensorFlow

• arxiv: https://arxiv.org/abs/1610.01178

TensorFlow Tutorials

• youtube: https://www.youtube.com/playlist?list=PL9Hr9sNUjfsmEu1ZniY0XpHSzl5uihcXZ
• github: https://github.com/Hvass-Labs/TensorFlow-Tutorials

Shapes and dynamic dimensions in TensorFlow

• blog: https://blog.metaflow.fr/shapes-and-dynamic-dimensions-in-tensorflow-7b1fe79be363#.1293uf94t

TensorFlow saving/restoring and mixing multiple models

https://blog.metaflow.fr/tensorflow-saving-restoring-and-mixing-multiple-models-c4c94d5d7125#.242xy4d46

Getting to Know TensorFlow

• blog: https://hackernoon.com/machine-learning-with-tensorflow-8873fdee2b68#.90j2jz5av

Image Classification and Segmentation with Tensorflow and TF-Slim http://warmspringwinds.github.io/tensorflow/tf-slim/2016/10/30/image-classification-and-segmentation-using-tensorflow-and-tf-slim/

Not another MNIST tutorial with TensorFlow

• blog: https://www.oreilly.com/learning/not-another-mnist-tutorial-with-tensorflow

Dive Into TensorFlow

• Part I: Getting Started with TensorFlow: http://textminingonline.com/dive-into-tensorflow-part-i-getting-started-with-tensorflow
• Part II: Basic Concepts: http://textminingonline.com/dive-into-tensorflow-part-ii-basic-concepts
• Part III: GTX 1080+Ubuntu16.04+CUDA8.0+cuDNN5.0+TensorFlow: http://textminingonline.com/dive-into-tensorflow-part-iii-gtx-1080-ubuntu16-04-cuda8-0-cudnn5-0-tensorflow
• Part IV: Hello MNIST: http://textminingonline.com/dive-into-tensorflow-part-iv-hello-mnist
• Part V: Deep MNIST: http://textminingonline.com/dive-into-tensorflow-part-v-deep-mnist
• Part VI: Beyond Deep Learning: http://textminingonline.com/dive-into-tensorflow-part-vi-beyond-deep-learning

TensorFlow Exercises - focusing on the comparison with NumPy.

• github: https://github.com/Kyubyong/tensorflow-exercises

A Gentle Guide to Using Batch Normalization in Tensorflow

• blog: http://ruishu.io/2016/12/27/batchnorm/

Using TensorFlow in Windows with a GPU

• blog: http://www.heatonresearch.com/2017/01/01/tensorflow-windows-gpu.html

Tensorflow and deep learning - without a PhD

• youtube: https://www.youtube.com/watch?v=vq2nnJ4g6N0
• mirror: https://pan.baidu.com/s/1o8HF9R8
• blog: https://codelabs.developers.google.com/codelabs/cloud-tensorflow-mnist/#0

4 Steps To Learn TensorFlow When You Already Know scikit-learn https://medium.com/@Zelros/4-steps-to-learn-tensorflow-when-you-already-know-scikit-learn-3cd0340456b5#.q206au7u9

Gentlest Introduction to Tensorflow

• part 1: https://medium.com/all-of-us-are-belong-to-machines/the-gentlest-introduction-to-tensorflow-248dc871a224#.fxyclr1ui
• part 2: https://medium.com/all-of-us-are-belong-to-machines/gentlest-introduction-to-tensorflow-part-2-ed2a0a7a624f#.vf7p9upg2
• part 3: https://medium.com/all-of-us-are-belong-to-machines/gentlest-intro-to-tensorflow-part-3-matrices-multi-feature-linear-regression-30a81ebaaa6c#.bvjru1f88
• part 4: https://medium.com/all-of-us-are-belong-to-machines/gentlest-intro-to-tensorflow-4-logistic-regression-2afd0cabc54#.seh1fbr24

learn code with tensorflow

• github: https://github.com/burness/tensorflow-101

TensorFlow Machine Learning Cookbook

• book: https://www.packtpub.com/big-data-and-business-intelligence/tensorflow-machine-learning-cookbook
• github: https://github.com/nfmcclure/tensorflow_cookbook

TensorFlow Image Recognition on a Raspberry Pi

http://svds.com/tensorflow-image-recognition-raspberry-pi/

TensorFlow For Machine Intelligence

• book: https://bleedingedgepress.com/tensor-flow-for-machine-intelligence/
• github: https://github.com/backstopmedia/tensorflowbook

Installing TensorFlow on Raspberry Pi 3 (and probably 2 as well)

• intro: TensorFlow for Raspberry Pi
• github: https://github.com/samjabrahams/tensorflow-on-raspberry-pi

CodinGame: Deep Learning - TensorFlow

• homepage: https://www.codingame.com/games/machine-learning?utm_source=CodinGame&utm_medium=Email&utm_campaign=tensorflow

A Practical Guide for Debugging Tensorflow Codes

Debugging Tips on TensorFlow

• slides: https://wookayin.github.io/TensorflowKR-2016-talk-debugging
• github: https://github.com/wookayin/TensorflowKR-2016-talk-debugging

Tensorflow Projects: Deep learning using tensorflow

• intro: A repo of everything deep and neurally related. Implementations and ideas are largely based on papers from arxiv and implementations, tutorials from the internet.
• github: https://github.com/shekkizh/TensorflowProjects

Machine Learning with TensorFlow

• homepage: http://www.tensorflowbook.com/
• github: https://github.com/BinRoot/TensorFlow-Book
• blog: https://www.manning.com/books/machine-learning-with-tensorflow

Convolutional Networks: from TensorFlow to iOS BNNS

• blog: https://paiv.github.io/blog/2016/09/25/tensorflow-to-bnns.html
• github: https://github.com/paiv/mnist-bnns

Android TensorFlow Machine Learning Example

• blog: https://blog.mindorks.com/android-tensorflow-machine-learning-example-ff0e9b2654cc#.ysg0ss9r2
• github: https://github.com/MindorksOpenSource/AndroidTensorFlowMachineLearningExample

TensorFlow and Deep Learning Tutorials

https://github.com/wagamamaz/tensorflow-tutorial

Finetuning AlexNet with TensorFlow

• blog: https://kratzert.github.io/kratzert.github.io/2017/02/24/finetuning-alexnet-with-tensorflow.html
• github: https://github.com/kratzert/finetune_alexnet_with_tensorflow

Deep Learning examples using Tensorflow

https://github.com/aditya101993/Deep-Learning

How To Write Your Own Tensorflow in C++

https://oneraynyday.github.io/ml/2017/10/20/Tensorflow-C++/

Tensorflow on Android

A Guide to Running Tensorflow Models on Android

• youtube: https://www.youtube.com/watch?v=kFWKdLOxykE
• mirror: http://www.bilibili.com/video/av9806881/index_10.html
• github: https://github.com/llSourcell/A_Guide_to_Running_Tensorflow_Models_on_Android

TensorFlow Android stand-alone demo

• intro: Android demo source files extracted from original TensorFlow source. (TensorFlow r0.10)
• github: https://github.com/miyosuda/TensorFlowAndroidDemo

Torch

Torch Developer Guide

• github: https://github.com/Atcold/Torch-Developer-Guide

PyTorch

Practical PyTorch tutorials

• github: https://github.com/spro/practical-pytorch

The Incredible PyTorch

• github: https://github.com/ritchieng/the-incredible-pytorch

PyTorch quick start: Classifying an image

• blog: http://blog.outcome.io/pytorch-quick-start-classifying-an-image/
• ipn: https://gist.github.com/jbencook/9918217f866c1aa9967391ba62d123b5

tutorial for researchers to learn deep learning with pytorch.

https://github.com/yunjey/pytorch-tutorial

Building a System for Fun!

Facial Recognition On A Jetson TX1 In Tensorflow

• blog: http://www.mattkrzus.com/face.html

Build an AI Cat Chaser with Jetson TX1 and Caffe

• blog: https://devblogs.nvidia.com/parallelforall/ai-cat-chaser-jetson-tx1-caffe/

Deep Learning in Aerial Systems Using Jetson

• blog: https://devblogs.nvidia.com/parallelforall/deep-learning-in-aerial-systems-jetson/
• github: https://github.com/amitibo/auvsi-targets

Cherry Autonomous Racecar (CAR): NCAT ECE Senior Design Project

• intro: Implementation of the CNN from End to End Learning for Self-Driving Cars on a Nvidia Jetson TX1 using Tensorflow and ROS
• github: https://github.com/DJTobias/Cherry-Autonomous-Racecar

Traffic Signs Classification

Traffic signs classification with Deep Learning.

• blog: https://hackernoon.com/traffic-signs-classification-with-deep-learning-b0cb03e23efb#.n0fjehwo6
• github: https://github.com/MehdiSv/TrafficSignsRecognition/

Traffic Sign Recognition with TensorFlow

• blog: https://medium.com/@waleedka/traffic-sign-recognition-with-tensorflow-629dffc391a6#.2khbv6a9a

Traffic signs classification with a convolutional network

http://navoshta.com/traffic-signs-classification/

Convolutional Neural Network for Traffic Sign Classification — CarND

• blog: https://medium.com/@gruby/convolutional-neural-network-for-traffic-sign-classification-carnd-e46e95453899#.lfutfs21v

Talks

A Tour of Deep Learning With C++

• intro: CppCon 2017, Peter Goldsborough
• youtube: https://www.youtube.com/watch?v=9-1lcss0NMg
• bilibili: https://www.bilibili.com/video/av20675156/

Papers

Practical recommendations for gradient-based training of deep architectures

• author: Yoshua Bengio
• arxiv: http://arxiv.org/abs/1206.5533

Blogs

Efficient BackProp

• intro: Neural Networks: Tricks of the Trade, 2nd
• blog: http://blog.csdn.net/zouxy09/article/details/45288129

Deep Learning for Vision: Tricks of the Trade

• intro: CVPR. Marc’Aurelio Ranzato
• slides: http://bavm2013.splashthat.com/img/events/46439/assets/34a7.ranzato.pdf

Optimizing RNN performance

• intro: Silicon Valley AI Lab
• keywords: Optimize GEMM, parallel GPU, GRU and LSTM…
• blog: http://svail.github.io/

Must Know Tips/Tricks in Deep Neural Networks

• intro: by Xiu-Shen Wei, NJU LAMDA
• blog: http://lamda.nju.edu.cn/weixs/project/CNNTricks/CNNTricks.html
• slides: http://lamda.nju.edu.cn/weixs/slide/CNNTricks_slide.pdf

Training Tricks from Deeplearning4j

http://deeplearning4j.org/trainingtricks.html

Suggestions for DL from Llya Sutskeve

• intro: data, preprocessing, mini-batch, gradient normalization, learning rate, weight initialization, data augmentation, dropout and ensemble
• blog: http://yyue.blogspot.com/2015/01/a-brief-overview-of-deep-learning.html

Efficient Training Strategies for Deep Neural Network Language Models

• intro: batch-size, initial learning rate, network initialization
• blog: https://fb56552f-a-62cb3a1a-s-sites.googlegroups.com/site/deeplearningworkshopnips2014/71.pdf?attachauth=ANoY7cp_eDwTXPm6iWHdBRhlIsgPASEAwkW-exLSOsz467mge7zLCkBMWznOu_G90vGVtqNvXOusc4z6cC6hEnHk6YzHtuEr_kyU0fyme7asaECN0zvoNwDk5258CueoB6fY3WtLvbJzYok1xiIeWSFYtk5mKXCXFDMI6djwhjCX1xi0GEEv_x7uMQwTdQlDItZ3kgLnZ2RjctQmIXDCu58fS3Wby4vWX3CkhMIf_EpCXx7jDn_M2SM%3D&attredirects=0

Neural Networks Best Practice

• intro: Uber
• paper: http://www.kentran.net/2013/04/neural-network-best-practices.html

Dark Knowledge from Hinton

• youtube: https://www.youtube.com/watch?v=EK61htlw8hY
• slides: http://www.ttic.edu/dl/dark14.pdf
• notes: http://deepdish.io/2014/10/28/hintons-dark-knowledge/
• notes: http://fastml.com/geoff-hintons-dark-knowledge/

Stochastic Gradient Descent Tricks(Leon Bottou)

http://leon.bottou.org/publications/pdf/tricks-2012.pdf

Advice for applying Machine Learning

https://jmetzen.github.io/2015-01-29/ml_advice.html

How to Debug Learning Algorithm for Regression Model

http://vitalflux.com/machine-learning-debug-learning-algorithm-regression-model/

Large-scale L-BFGS using MapReduce

• intro: NIPS 2014
• paper: http://papers.nips.cc/paper/5333-large-scale-l-bfgs-using-mapreduce.pdf

Selecting good features

– Part I: univariate selection: http://blog.datadive.net/selecting-good-features-part-i-univariate-selection/ – Part II: linear models and regularization: http://blog.datadive.net/selecting-good-features-part-ii-linear-models-and-regularization/ – Part III: random forests: http://blog.datadive.net/selecting-good-features-part-iii-random-forests/ – Part IV: stability selection, RFE and everything side by side: http://blog.datadive.net/selecting-good-features-part-iv-stability-selection-rfe-and-everything-side-by-side/

机器学习代码心得之有监督学习的模块

http://www.weibo.com/p/1001603795687165852957

Stochastic Gradient Boosting: Choosing the Best Number of Iterations

• intro: Kaggle winner YANIR SEROUSSI
• blog: http://yanirseroussi.com/2014/12/29/stochastic-gradient-boosting-choosing-the-best-number-of-iterations/

Large-Scale High-Precision Topic Modeling on Twitter

• intro: Twitter senior researcher. KDD 2014
• paper: http://www.eeshyang.com/papers/KDD14Jubjub.pdf

H2O World - Top 10 Deep Learning Tips & Tricks - Arno Candel

http://www.slideshare.net/0xdata/h2o-world-top-10-deep-learning-tips-tricks-arno-candel

How To Improve Deep Learning Performance: 20 Tips, Tricks and Techniques That You Can Use To Fight Overfitting and Get Better Generalization

http://machinelearningmastery.com/improve-deep-learning-performance/

Neural Network Training Speed Trick

https://medium.com/machine-learning-at-petiteprogrammer/neural-network-training-speed-trick-92d6b22a7754#.4v6qukpn7

The Black Magic of Deep Learning - Tips and Tricks for the practitioner

http://nmarkou.blogspot.ru/2017/02/the-black-magic-of-deep-learning-tips.html

Papers

Accelerating Deep Convolutional Neural Networks Using Specialized Hardware

• paper: http://research.microsoft.com/pubs/240715/CNN%20Whitepaper.pdf

Installation / Deploying

Setting up a Deep Learning Machine from Scratch (Software): Instructions for setting up the software on your deep learning machine

• intro: A detailed guide to setting up your machine for deep learning research. Includes instructions to install drivers, tools and various deep learning frameworks. This was tested on a 64 bit machine with Nvidia Titan X, running Ubuntu 14.04
• github: https://github.com/saiprashanths/dl-setup

How to install CUDA Toolkit and cuDNN for deep learning

• blog: http://www.pyimagesearch.com/2016/07/04/how-to-install-cuda-toolkit-and-cudnn-for-deep-learning/

Deploying Deep Learning: Guide to deploying deep-learning inference networks and realtime object detection with TensorRT and Jetson TX1.

• github: https://github.com/dusty-nv/jetson-inference

Install Log

• intro: setting up Caffe on a cluster running Redhat 6.3 (Santiago) without having root
• github: https://github.com/yosinski/caffe/blob/jason_public/doc/linux-no-root-install-log.md

Lessons Learned from Deploying Deep Learning at Scale

• blog: http://blog.algorithmia.com/deploying-deep-learning-cloud-services/

Docker

All-in-one Docker image for Deep Learning

• intro: An all-in-one Docker image for deep learning. Contains all the popular DL frameworks (TensorFlow, Theano, Torch, Caffe, etc.)
• github: https://github.com/saiprashanths/dl-docker

NVIDIA Docker: GPU Server Application Deployment Made Easy

• blog: https://devblogs.nvidia.com/parallelforall/nvidia-docker-gpu-server-application-deployment-made-easy/
• github: https://github.com/NVIDIA/nvidia-docker

Deep learning base image for Docker (Tensorflow, Caffe, MXNet, Torch, Openface, etc.)

https://github.com/dominiek/deep-base

Deepo: a Docker image with a full reproducible deep learning research environment

• intro: A Docker image containing almost all popular deep learning frameworks: theano, tensorflow, sonnet, pytorch, keras, lasagne, mxnet, cntk, chainer, caffe, torch.
• project page: https://hub.docker.com/r/ufoym/deepo/
• github: https://github.com//ufoym/deepo

Cloud

SuperVessel Cloud for POWER/OpenPOWER LoginRegisterTutorials

http://www.ptopenlab.com/

Building Deep Neural Networks in the Cloud with Azure GPU VMs, MXNet and Microsoft R Server

https://blogs.technet.microsoft.com/machinelearning/2016/09/15/building-deep-neural-networks-in-the-cloud-with-azure-gpu-vms-mxnet-and-microsoft-r-server/

Microsoft open sources its next-gen cloud hardware design

• blog: https://techcrunch.com/2016/10/31/microsoft-open-sources-its-next-gen-cloud-hardware-design/

Google Taps AMD For Accelerating Machine Learning In The Cloud

http://www.forbes.com/sites/aarontilley/2016/11/15/google-taps-amd-for-accelerating-machine-learning-in-the-cloud/#3549d8554181

Amazon EC2

Deep Learning AMI on AWS Marketplace

https://aws.amazon.com/marketplace/pp/B01M0AXXQB

We Have To Go Deeper: AWS p2.xlarge GPU optimized deep learning cluster-grenade

• github: https://github.com/Miej/GoDeeper

A GPU enabled AMI for Deep Learning

• blog: https://blog.empiricalci.com/a-gpu-enabled-ami-for-deep-learning-5aa3d694b630#.9339zxm4e

Keras with GPU on Amazon EC2 – a step-by-step instruction

https://medium.com/@mateuszsieniawski/keras-with-gpu-on-amazon-ec2-a-step-by-step-instruction-4f90364e49ac#.k27d0mqir

Microsoft R Server

Training Deep Neural Networks on ImageNet Using Microsoft R Server and Azure GPU VMs

• blog: https://blogs.technet.microsoft.com/machinelearning/2016/11/15/imagenet-deep-neural-network-training-using-microsoft-r-server-and-azure-gpu-vms/

Hardware System

I: Building a Deep Learning (Dream) Machine

• blog: http://graphific.github.io/posts/building-a-deep-learning-dream-machine/
• slides: http://www.slideshare.net/roelofp/building-a-deep-learning-dream-machine

II: Running a Deep Learning (Dream) Machine

• blog: http://graphific.github.io/posts/running-a-deep-learning-dream-machine/

A Full Hardware Guide to Deep Learning

• blog: http://timdettmers.com/2015/03/09/deep-learning-hardware-guide/

Build your own Deep Learning Box

• blog: https://annalyzin.wordpress.com/2016/05/19/build-a-deep-learning-box/

32-TFLOP Deep Learning GPU Box: A super-fast linux-based machine with multiple GPUs for training deep neural nets

https://hackaday.io/project/12070-32-tflop-deep-learning-gpu-box

Hands-on with the NVIDIA DIGITS DevBox for Deep Learning

• blog: http://www.pyimagesearch.com/2016/06/06/hands-on-with-the-nvidia-digits-devbox-for-deep-learning/

Considerations when setting up deep learning hardware

• blog: http://www.pyimagesearch.com/2016/06/13/considerations-when-setting-up-deep-learning-hardware/

Building a Workstation for Deep Learning

• slides: http://www.slideshare.net/PetteriTeikariPhD/deep-learning-workstation

Deep Learning Machine: First build experience

• blog: https://medium.com/@vivek.yadav/deep-learning-machine-first-build-experience-d04abf198831#.1d6q5mw9m

Building a machine learning/deep learning workstation for under $5000

• blog: https://www.analyticsvidhya.com/blog/2016/11/building-a-machine-learning-deep-learning-workstation-for-under-5000/

Hardware Guide: Neural Networks on GPUs (Updated 2016-1-30)

• intro: by Joseph Redmon
• blog: http://pjreddie.com/darknet/hardware-guide/

Building Your Own Deep Learning Box

https://medium.com/@bfortuner/building-your-own-deep-learning-box-47b918aea1eb#.4r5zchk4f

Setting up a Deep learning machine in a lazy yet quick way https://medium.com/@sravsatuluri/setting-up-a-deep-learning-machine-in-a-lazy-yet-quick-way-be2642318850#.jrxrkfxa2

Deep Confusion: Misadventures In Building A Deep Learning Machine

http://www.topbots.com/deep-confusion-misadventures-in-building-a-machine-learning-server/

DIY-Deep-Learning-Workstation

• intro: Build a deep learning workstation from scratch (HW & SW).
• github: https://github.com/charlesq34/DIY-Deep-Learning-Workstation

GPU

Which GPU(s) to Get for Deep Learning: My Experience and Advice for Using GPUs in Deep Learning

• blog: http://timdettmers.com/2017/04/09/which-gpu-for-deep-learning/

从深度学习选择什么样的gpu来谈谈gpu的硬件架构

• blog: http://chenrudan.github.io/blog/2015/12/20/introductionofgpuhardware.html

GPU折腾手记——2015 (by 李沐)

• blog: http://mli.github.io/gpu/2016/01/17/build-gpu-clusters/

HPC, Deep Learning and GPUs(2016 Stanford HPC Conference)

• youtube: https://www.youtube.com/watch?v=JwgoC-1V_38
• video: http://pan.baidu.com/s/1pKrSvOZ

Modern GPU 2.0: Design patterns for GPU computing

• intro: Modern GPU is code and commentary intended to promote new and productive ways of thinking about GPU computing.
• homepage: http://nvlabs.github.io/moderngpu/
• github: https://github.com/nvlabs/moderngpu

CuMF: CUDA-Acclerated ALS on mulitple GPUs.

• github: https://github.com/wei-tan/CuMF

Basic Performance Analysis of NVIDIA GPU Accelerator Cards for Deep Learning Applications

• wihte paper: https://www.amax.com/enterprise/pdfs/Deep%20Learning%20Performance%20Analysis.pdf

CuPy : NumPy-like API accelerated with CUDA

• github: https://github.com/pfnet/cupy

NumPy GPU acceleration

• blog: http://scottsievert.com/blog/2016/07/01/numpy-gpu/

Efficient Convolutional Neural Network Inference on Mobile GPUs (Embedded Vision Summit)

• youtube: https://www.youtube.com/watch?v=ximyhmm17UM

Deep Learning with Multiple GPUs on Rescale: Torch

• blog: https://blog.rescale.com/deep-learning-with-multiple-gpus-on-rescale-torch/

GPU-accelerated Theano & Keras on Windows 10 native

• arxiv: https://github.com/philferriere/dlwin

NVIDIA Announces Quadro GP100 - Big Pascal Comes to Workstations

http://www.anandtech.com/show/11102/nvidia-announces-quadro-gp100

FPGA

Recurrent Neural Networks Hardware Implementation on FPGA

• arxiv: http://arxiv.org/abs/1511.05552

Is implementing deep learning on FPGAs a natural next step after the success with GPUs?

• quora: https://www.quora.com/Is-implementing-deep-learning-on-FPGAs-a-natural-next-step-after-the-success-with-GPUs

Efficient Implementation of Neural Network Systems Built on FPGAs, Programmed with OpenCL

• paper: https://www.altera.com/content/dam/altera-www/global/en_US/pdfs/literature/solution-sheets/efficient_neural_networks.pdf?utm_source=Altera&utm_medium=link&utm_campaign=OpenCL_15_1&utm_content=NA_efficient-neural-networks-solution-sheet-download-link

Deep Learning on FPGAs: Past, Present, and Future

• arxiv: http://arxiv.org/abs/1602.04283

FPGAs Challenge GPUs as a Platform for Deep Learning

• blog: https://www.tractica.com/automation-robotics/fpgas-challenge-gpus-as-a-platform-for-deep-learning/

Convolution Neural Network CNN Implementation on Altera FPGA using OpenCL

• youtube: https://www.youtube.com/watch?v=78Qd5t-Mn0s

Accelerating Deep Learning Using Altera FPGAs (Embedded Vision Summit)

• youtube: https://www.youtube.com/watch?v=HlBC9qBqZRs
• slides: http://www.slideshare.net/embeddedvision/accelerating-deep-learning-using-altera-fpgas-a-presentation-from-intel

Machine Learning on FPGAs: Neural Networks

• youtube: https://www.youtube.com/watch?v=3iCifD8gZ0Q

Comprehensive Evaluation of OpenCL-based Convolutional Neural Network Accelerators in Xilinx and Altera FPGAs

• arxiv: https://arxiv.org/abs/1609.09296

Microsoft Goes All in for FPGAs to Build Out AI Cloud

• blog: https://www.top500.org/news/microsoft-goes-all-in-for-fpgas-to-build-out-cloud-based-ai/

Caffeinated FPGAs: FPGA Framework For Convolutional Neural Networks

• arxiv: https://arxiv.org/abs/1609.09671
• github: https://github.com/dicecco1/fpga_caffe

Intel Unveils FPGA to Accelerate Neural Networks

http://datacenterfrontier.com/intel-unveils-fpga-to-accelerate-ai-neural-networks/

Deep Learning with FPGA

• blog: https://amundtveit.com/2016/11/23/deep-learning-with-fpga/

A General Neural Network Hardware Architecture on FPGA

• intro: University of Birmingham
• arxiv: https://arxiv.org/abs/1711.05860

Approximate FPGA-based LSTMs under Computation Time Constraints

• intro: ARC 2018
• arxiv: https://arxiv.org/abs/1801.02190

ARM / Processor

‘Neural network’ spotted deep inside Samsung’s Galaxy S7 silicon brain: Secrets of Exynos M1 cores spilled

• blog: http://www.theregister.co.uk/2016/08/22/samsung_m1_core/?mt=1471918256061

Intel will add deep-learning instructions to its processors

• blog: http://lemire.me/blog/2016/10/14/intel-will-add-deep-learning-instructions-to-its-processors/

SRAM

ShiDianNao: Shifting Vision Processing Closer to the Sensor http://lap.epfl.ch/files/content/sites/lap/files/shared/publications/DuJun15_ShiDianNaoShiftingVisionProcessingCloserToTheSensor_ISCA15.pdf

Blogs

Emerging “Universal” FPGA, GPU Platform for Deep Learning

• blog: http://www.nextplatform.com/2016/06/29/universal-fpga-gpu-platform-deep-learning/

An Early Look at Startup Graphcore’s Deep Learning Chip

https://www.nextplatform.com/2017/03/09/early-look-startup-graphcores-deep-learning-chip/

Hardware for Deep Learning

https://medium.com/towards-data-science/hardware-for-deep-learning-8d9b03df41a

Videos

Energy-efficient Hardware for Embedded Vision and Deep Convolutional Neural Networks

• intro: September 2016 Embedded Vision Alliance Member Meeting Presentation: MIT
• youtube: https://www.youtube.com/watch?v=dO_lHz87DVM

ImageNet