Deep Learning Resources

Published: 09 Oct 2015 Category: deep_learning

ImageNet

Single-model on 224x224

Method top1 top5 Model Size Speed
ResNet-101 78.0% 94.0%    
ResNet-200 78.3% 94.2%    
Inception-v3        
Inception-v4        
Inception-ResNet-v2        
ResNet-50 77.8%      
ResNet-101 79.6% 94.7%    

Single-model on 320×320 / 299×299

Method top1 top5 Model Size Speed
ResNet-101        
ResNet-200 79.9% 95.2%    
Inception-v3 78.8% 94.4%    
Inception-v4 80.0% 95.0%    
Inception-ResNet-v2 80.1% 95.1%    
ResNet-50        
ResNet-101 80.9% 95.6%    

AlexNet

ImageNet Classification with Deep Convolutional Neural Networks

Network In Network

Network In Network

Batch-normalized Maxout Network in Network

GoogLeNet (Inception V1)

Going Deeper with Convolutions

Building a deeper understanding of images

VGGNet

Very Deep Convolutional Networks for Large-Scale Image Recognition

Tensorflow VGG16 and VGG19

Inception-V2

Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift

ImageNet pre-trained models with batch normalization

Inception-V3

Inception-V3 = Inception-V2 + BN-auxiliary (fully connected layer of the auxiliary classifier is also batch-normalized, not just the convolutions)

Rethinking the Inception Architecture for Computer Vision

Inception in TensorFlow

Train your own image classifier with Inception in TensorFlow

Notes on the TensorFlow Implementation of Inception v3

https://pseudoprofound.wordpress.com/2016/08/28/notes-on-the-tensorflow-implementation-of-inception-v3/

Training an InceptionV3-based image classifier with your own dataset

Inception-BN full for Caffe: Inception-BN ImageNet (21K classes) model for Caffe

ResNet

Deep Residual Learning for Image Recognition

Third-party re-implementations

https://github.com/KaimingHe/deep-residual-networks#third-party-re-implementations

Training and investigating Residual Nets

resnet.torch: an updated version of fb.resnet.torch with many changes.

Highway Networks and Deep Residual Networks

Interpretating Deep Residual Learning Blocks as Locally Recurrent Connections

Lab41 Reading Group: Deep Residual Learning for Image Recognition

50-layer ResNet, trained on ImageNet, classifying webcam

Reproduced ResNet on CIFAR-10 and CIFAR-100 dataset.

ResNet-V2

Identity Mappings in Deep Residual Networks

Deep Residual Networks for Image Classification with Python + NumPy

Inception-V4 / Inception-ResNet-V2

Inception-V4, Inception-Resnet And The Impact Of Residual Connections On Learning

The inception-resnet-v2 models trained from scratch via torch

Inception v4 in Keras

ResNeXt

Aggregated Residual Transformations for Deep Neural Networks

Residual Networks Variants

Resnet in Resnet: Generalizing Residual Architectures

Residual Networks are Exponential Ensembles of Relatively Shallow Networks

Wide Residual Networks

Residual Networks of Residual Networks: Multilevel Residual Networks

Multi-Residual Networks

Deep Pyramidal Residual Networks

Learning Identity Mappings with Residual Gates

Wider or Deeper: Revisiting the ResNet Model for Visual Recognition

Deep Pyramidal Residual Networks with Separated Stochastic Depth

Spatially Adaptive Computation Time for Residual Networks

ShaResNet: reducing residual network parameter number by sharing weights

Sharing Residual Units Through Collective Tensor Factorization in Deep Neural Networks

Residual Attention Network for Image Classification

Dilated Residual Networks

Dynamic Steerable Blocks in Deep Residual Networks

Learning Deep ResNet Blocks Sequentially using Boosting Theory

Learning Strict Identity Mappings in Deep Residual Networks

Spiking Deep Residual Network

https://arxiv.org/abs/1805.01352

DenseNet

Densely Connected Convolutional Networks

Memory-Efficient Implementation of DenseNets

DenseNet 2.0

CondenseNet: An Efficient DenseNet using Learned Group Convolutions

Xception

Deep Learning with Separable Convolutions

Xception: Deep Learning with Depthwise Separable Convolutions

Towards a New Interpretation of Separable Convolutions

MobileNets

MobileNets: Efficient Convolutional Neural Networks for Mobile Vision Applications

MobileNets: Open-Source Models for Efficient On-Device Vision

Google’s MobileNets on the iPhone

ShuffleNet: An Extremely Efficient Convolutional Neural Network for Mobile Devices

Depth_conv-for-mobileNet

https://github.com//LamHoCN/Depth_conv-for-mobileNet

The Enhanced Hybrid MobileNet

https://arxiv.org/abs/1712.04698

FD-MobileNet: Improved MobileNet with a Fast Downsampling Strategy

https://arxiv.org/abs/1802.03750

A Quantization-Friendly Separable Convolution for MobileNets

MobileNetV2

Inverted Residuals and Linear Bottlenecks: Mobile Networks forClassification, Detection and Segmentation

SENet

Squeeze-and-Excitation Networks

ImageNet Projects

Training an Object Classifier in Torch-7 on multiple GPUs over ImageNet

Semi-Supervised Learning

Semi-Supervised Learning with Graphs

Semi-Supervised Learning with Ladder Networks

Semi-supervised Feature Transfer: The Practical Benefit of Deep Learning Today?

Temporal Ensembling for Semi-Supervised Learning

Semi-supervised Knowledge Transfer for Deep Learning from Private Training Data

Infinite Variational Autoencoder for Semi-Supervised Learning

Transfer Learning

Discriminative Transfer Learning with Tree-based Priors

How transferable are features in deep neural networks?

Learning and Transferring Mid-Level Image Representations using Convolutional Neural Networks

Learning Transferable Features with Deep Adaptation Networks

Transferring Knowledge from a RNN to a DNN

Simultaneous Deep Transfer Across Domains and Tasks

Net2Net: Accelerating Learning via Knowledge Transfer

Transfer Learning from Deep Features for Remote Sensing and Poverty Mapping

A theoretical framework for deep transfer learning

Transfer learning using neon

Hyperparameter Transfer Learning through Surrogate Alignment for Efficient Deep Neural Network Training

What makes ImageNet good for transfer learning?

Fine-tuning a Keras model using Theano trained Neural Network & Introduction to Transfer Learning

Multi-source Transfer Learning with Convolutional Neural Networks for Lung Pattern Analysis

Borrowing Treasures from the Wealthy: Deep Transfer Learning through Selective Joint Fine-tuning

Optimal Transport for Deep Joint Transfer Learning

https://arxiv.org/abs/1709.02995

Transfer Learning with Binary Neural Networks

Gradual Tuning: a better way of Fine Tuning the parameters of a Deep Neural Network

Born Again Neural Networks

One Shot Learning

One-shot Learning with Memory-Augmented Neural Networks

Matching Networks for One Shot Learning

Learning feed-forward one-shot learners [NIPS 2016] [VALSE seminar]

Generative Adversarial Residual Pairwise Networks for One Shot Learning

Few-Shot Learning

Optimization as a Model for Few-Shot Learning

Learning to Compare: Relation Network for Few-Shot Learning

Unleashing the Potential of CNNs for Interpretable Few-Shot Learning

Low-Shot Learning from Imaginary Data

Semantic Feature Augmentation in Few-shot Learning

Multi-label Learning

CNN: Single-label to Multi-label

Deep Learning for Multi-label Classification

Predicting Unseen Labels using Label Hierarchies in Large-Scale Multi-label Learning

Learning with a Wasserstein Loss

From Softmax to Sparsemax: A Sparse Model of Attention and Multi-Label Classification

CNN-RNN: A Unified Framework for Multi-label Image Classification

Improving Multi-label Learning with Missing Labels by Structured Semantic Correlations

Extreme Multi-label Loss Functions for Recommendation, Tagging, Ranking & Other Missing Label Applications

Multi-Label Image Classification with Regional Latent Semantic Dependencies

Privileged Multi-label Learning

Multi-task Learning

Multitask Learning / Domain Adaptation

multi-task learning

Learning and Transferring Multi-task Deep Representation for Face Alignment

Multi-task learning of facial landmarks and expression

Multi-Task Deep Visual-Semantic Embedding for Video Thumbnail Selection

Learning Multiple Tasks with Deep Relationship Networks

Learning deep representation of multityped objects and tasks

Cross-stitch Networks for Multi-task Learning

Multi-Task Learning in Tensorflow (Part 1)

Deep Multi-Task Learning with Shared Memory

Learning to Push by Grasping: Using multiple tasks for effective learning

Identifying beneficial task relations for multi-task learning in deep neural networks

Multi-Task Learning Using Uncertainty to Weigh Losses for Scene Geometry and Semantics

One Model To Learn Them All

MultiModel: Multi-Task Machine Learning Across Domains

https://research.googleblog.com/2017/06/multimodel-multi-task-machine-learning.html

An Overview of Multi-Task Learning in Deep Neural Networks

PackNet: Adding Multiple Tasks to a Single Network by Iterative Pruning

End-to-End Multi-Task Learning with Attention

Multi-modal Learning

Multimodal Deep Learning

Multimodal Convolutional Neural Networks for Matching Image and Sentence

A C++ library for Multimodal Deep Learning

Multimodal Learning for Image Captioning and Visual Question Answering

Multi modal retrieval and generation with deep distributed models

Learning Aligned Cross-Modal Representations from Weakly Aligned Data

Variational methods for Conditional Multimodal Deep Learning

Training and Evaluating Multimodal Word Embeddings with Large-scale Web Annotated Images

Deep Multi-Modal Image Correspondence Learning

Multimodal Deep Learning (D4L4 Deep Learning for Speech and Language UPC 2017)

Debugging Deep Learning

Some tips for debugging deep learning

Introduction to debugging neural networks

How to Visualize, Monitor and Debug Neural Network Learning

Learning from learning curves

Understanding CNN

Understanding the Effective Receptive Field in Deep Convolutional Neural Networks

Deep Learning Networks

PCANet: A Simple Deep Learning Baseline for Image Classification?

Convolutional Kernel Networks

Deeply-supervised Nets

FitNets: Hints for Thin Deep Nets

Striving for Simplicity: The All Convolutional Net

How these researchers tried something unconventional to come out with a smaller yet better Image Recognition.

Pointer Networks

Pointer Networks in TensorFlow (with sample code)

Rectified Factor Networks

Correlational Neural Networks

Diversity Networks

Competitive Multi-scale Convolution

A Unified Approach for Learning the Parameters of Sum-Product Networks (SPN)

Awesome Sum-Product Networks

Recombinator Networks: Learning Coarse-to-Fine Feature Aggregation

Dynamic Capacity Networks

Bitwise Neural Networks

Learning Discriminative Features via Label Consistent Neural Network

A Theory of Generative ConvNet

How to Train Deep Variational Autoencoders and Probabilistic Ladder Networks

Group Equivariant Convolutional Networks (G-CNNs)

Deep Spiking Networks

Low-rank passthrough neural networks

Single Image 3D Interpreter Network

Deeply-Fused Nets

SNN: Stacked Neural Networks

Universal Correspondence Network

Progressive Neural Networks

Holistic SparseCNN: Forging the Trident of Accuracy, Speed, and Size

Mollifying Networks

Domain Separation Networks

Local Binary Convolutional Neural Networks

CliqueCNN: Deep Unsupervised Exemplar Learning

Convexified Convolutional Neural Networks

Multi-scale brain networks

https://arxiv.org/abs/1711.11473

Input Convex Neural Networks

HyperNetworks

HyperLSTM

X-CNN: Cross-modal Convolutional Neural Networks for Sparse Datasets

Tensor Switching Networks

BranchyNet: Fast Inference via Early Exiting from Deep Neural Networks

Spectral Convolution Networks

DelugeNets: Deep Networks with Massive and Flexible Cross-layer Information Inflows

PolyNet: A Pursuit of Structural Diversity in Very Deep Networks

Weakly Supervised Cascaded Convolutional Networks

DeepSetNet: Predicting Sets with Deep Neural Networks

Steerable CNNs

Feedback Networks

Oriented Response Networks

OptNet: Differentiable Optimization as a Layer in Neural Networks

A fast and differentiable QP solver for PyTorch

Meta Networks

https://arxiv.org/abs/1703.00837

Deformable Convolutional Networks

Second-order Convolutional Neural Networks

https://arxiv.org/abs/1703.06817

Gabor Convolutional Networks

https://arxiv.org/abs/1705.01450

Deep Rotation Equivariant Network

https://arxiv.org/abs/1705.08623

Dense Transformer Networks

Deep Complex Networks

Deep Quaternion Networks

DiracNets: Training Very Deep Neural Networks Without Skip-Connections

Dual Path Networks

Primal-Dual Group Convolutions for Deep Neural Networks

Interleaved Group Convolutions for Deep Neural Networks

IGCV2: Interleaved Structured Sparse Convolutional Neural Networks

Sensor Transformation Attention Networks

https://arxiv.org/abs/1708.01015

Sparsity Invariant CNNs

https://arxiv.org/abs/1708.06500

SPARCNN: SPAtially Related Convolutional Neural Networks

https://arxiv.org/abs/1708.07522

BranchyNet: Fast Inference via Early Exiting from Deep Neural Networks

https://arxiv.org/abs/1709.01686

Polar Transformer Networks

https://arxiv.org/abs/1709.01889

Tensor Product Generation Networks

https://arxiv.org/abs/1709.09118

Deep Competitive Pathway Networks

Context Embedding Networks

https://arxiv.org/abs/1710.01691

Generalization in Deep Learning

Understanding Deep Learning Generalization by Maximum Entropy

Do Convolutional Neural Networks Learn Class Hierarchy?

Deep Hyperspherical Learning

Beyond Sparsity: Tree Regularization of Deep Models for Interpretability

Neural Motifs: Scene Graph Parsing with Global Context

Priming Neural Networks

https://arxiv.org/abs/1711.05918

Three Factors Influencing Minima in SGD

https://arxiv.org/abs/1711.04623

BPGrad: Towards Global Optimality in Deep Learning via Branch and Pruning

https://arxiv.org/abs/1711.06959

BlockDrop: Dynamic Inference Paths in Residual Networks

Wasserstein Introspective Neural Networks

https://arxiv.org/abs/1711.08875

SkipNet: Learning Dynamic Routing in Convolutional Networks

https://arxiv.org/abs/1711.09485

Do Convolutional Neural Networks act as Compositional Nearest Neighbors?

ConvNets and ImageNet Beyond Accuracy: Explanations, Bias Detection, Adversarial Examples and Model Criticism

Broadcasting Convolutional Network

https://arxiv.org/abs/1712.02517

Point-wise Convolutional Neural Network

ScreenerNet: Learning Curriculum for Neural Networks

  • intro: Intel Corporation & Allen Institute for Artificial Intelligence
  • keywords: curricular learning, deep learning, deep q-learning
  • arxiv: https://arxiv.org/abs/1801.00904

Sparsely Connected Convolutional Networks

https://arxiv.org/abs/1801.05895

Spherical CNNs

Going Deeper in Spiking Neural Networks: VGG and Residual Architectures

Rotate your Networks: Better Weight Consolidation and Less Catastrophic Forgetting

https://arxiv.org/abs/1802.02950

Convolutional Neural Networks with Alternately Updated Clique

Decoupled Networks

Convolutions / Filters

Warped Convolutions: Efficient Invariance to Spatial Transformations

Coordinating Filters for Faster Deep Neural Networks

Shift: A Zero FLOP, Zero Parameter Alternative to Spatial Convolutions

Spatially-Adaptive Filter Units for Deep Neural Networks

clcNet: Improving the Efficiency of Convolutional Neural Network using Channel Local Convolutions

https://arxiv.org/abs/1712.06145

DCFNet: Deep Neural Network with Decomposed Convolutional Filters

https://arxiv.org/abs/1802.04145

Fast End-to-End Trainable Guided Filter

Diagonalwise Refactorization: An Efficient Training Method for Depthwise Convolutions

Highway Networks

Highway Networks

Highway Networks with TensorFlow

Very Deep Learning with Highway Networks

Training Very Deep Networks

Spatial Transformer Networks

Spatial Transformer Networks

The power of Spatial Transformer Networks

Recurrent Spatial Transformer Networks

Deep Learning Paper Implementations: Spatial Transformer Networks - Part I

Top-down Flow Transformer Networks

https://arxiv.org/abs/1712.02400

Non-Parametric Transformation Networks

Hierarchical Spatial Transformer Network

https://arxiv.org/abs/1801.09467

FractalNet

FractalNet: Ultra-Deep Neural Networks without Residuals

Architecture Search for Convolutional Neural Networks

Neural Architecture Search with Reinforcement Learning

Neural Optimizer Search with Reinforcement Learning

Learning Transferable Architectures for Scalable Image Recognition

The First Step-by-Step Guide for Implementing Neural Architecture Search with Reinforcement Learning Using TensorFlow

Practical Network Blocks Design with Q-Learning

https://arxiv.org/abs/1708.05552

Simple And Efficient Architecture Search for Convolutional Neural Networks

Progressive Neural Architecture Search

Finding Competitive Network Architectures Within a Day Using UCT

Regularized Evolution for Image Classifier Architecture Search

https://arxiv.org/abs/1802.01548

Efficient Neural Architecture Search via Parameters Sharing

Neural Architecture Search with Bayesian Optimisation and Optimal Transport

Graph Convolutional Networks

Learning Convolutional Neural Networks for Graphs

Convolutional Neural Networks on Graphs with Fast Localized Spectral Filtering

Semi-Supervised Classification with Graph Convolutional Networks

Graph Based Convolutional Neural Network

How powerful are Graph Convolutions? (review of Kipf & Welling, 2016)

http://www.inference.vc/how-powerful-are-graph-convolutions-review-of-kipf-welling-2016-2/

Graph Convolutional Networks

DeepGraph: Graph Structure Predicts Network Growth

Deep Learning with Sets and Point Clouds

Deep Learning on Graphs

Robust Spatial Filtering with Graph Convolutional Neural Networks

https://arxiv.org/abs/1703.00792

Modeling Relational Data with Graph Convolutional Networks

https://arxiv.org/abs/1703.06103

Distance Metric Learning using Graph Convolutional Networks: Application to Functional Brain Networks

Deep Learning on Graphs with Graph Convolutional Networks

Deep Learning on Graphs with Keras

Learning Graph While Training: An Evolving Graph Convolutional Neural Network

https://arxiv.org/abs/1708.04675

Residual Gated Graph ConvNets

https://arxiv.org/abs/1711.07553

Probabilistic and Regularized Graph Convolutional Networks

Generative Models

Max-margin Deep Generative Models

Discriminative Regularization for Generative Models

Auxiliary Deep Generative Models

Sampling Generative Networks: Notes on a Few Effective Techniques

Conditional Image Synthesis With Auxiliary Classifier GANs

On the Quantitative Analysis of Decoder-Based Generative Models

Boosted Generative Models

An Architecture for Deep, Hierarchical Generative Models

Deep Learning and Hierarchal Generative Models

Probabilistic Torch

Tutorial on Deep Generative Models

A Note on the Inception Score

Gradient Layer: Enhancing the Convergence of Adversarial Training for Generative Models

Deep Learning and Robots

Robot Learning Manipulation Action Plans by “Watching” Unconstrained Videos from the World Wide Web

End-to-End Training of Deep Visuomotor Policies

Comment on Open AI’s Efforts to Robot Learning

The Curious Robot: Learning Visual Representations via Physical Interactions

How to build a robot that “sees” with $100 and TensorFlow

Deep Visual Foresight for Planning Robot Motion

Sim-to-Real Robot Learning from Pixels with Progressive Nets

Towards Lifelong Self-Supervision: A Deep Learning Direction for Robotics

A Differentiable Physics Engine for Deep Learning in Robotics

Deep-learning in Mobile Robotics - from Perception to Control Systems: A Survey on Why and Why not

Deep Robotic Learning

Deep Learning in Robotics: A Review of Recent Research

https://arxiv.org/abs/1707.07217

Deep Learning for Robotics

DroNet: Learning to Fly by Driving

A Survey on Deep Learning Methods for Robot Vision

https://arxiv.org/abs/1803.10862

Deep Learning on Mobile / Embedded Devices

Convolutional neural networks on the iPhone with VGGNet

TensorFlow for Mobile Poets

The Convolutional Neural Network(CNN) for Android

TensorFlow on Android

Experimenting with TensorFlow on Android

XNOR.ai frees AI from the prison of the supercomputer

Embedded Deep Learning with NVIDIA Jetson

Embedded and mobile deep learning research resources

https://github.com/csarron/emdl

Modeling the Resource Requirements of Convolutional Neural Networks on Mobile Devices

https://arxiv.org/abs/1709.09118

Benchmarks

Deep Learning’s Accuracy

Benchmarks for popular CNN models

Deep Learning Benchmarks

http://add-for.com/deep-learning-benchmarks/

cudnn-rnn-benchmarks

Papers

Reweighted Wake-Sleep

Probabilistic Backpropagation for Scalable Learning of Bayesian Neural Networks

Deeply-Supervised Nets

Deep learning

On the Expressive Power of Deep Learning: A Tensor Analysis

Understanding and Predicting Image Memorability at a Large Scale

Towards Open Set Deep Networks

Structured Prediction Energy Networks

Deep Neural Networks predict Hierarchical Spatio-temporal Cortical Dynamics of Human Visual Object Recognition

Recent Advances in Convolutional Neural Networks

Understanding Deep Convolutional Networks

DeepCare: A Deep Dynamic Memory Model for Predictive Medicine

Exploiting Cyclic Symmetry in Convolutional Neural Networks

Cross-dimensional Weighting for Aggregated Deep Convolutional Features

Understanding Visual Concepts with Continuation Learning

Learning Efficient Algorithms with Hierarchical Attentive Memory

DrMAD: Distilling Reverse-Mode Automatic Differentiation for Optimizing Hyperparameters of Deep Neural Networks

Do Deep Convolutional Nets Really Need to be Deep (Or Even Convolutional)?

Harnessing Deep Neural Networks with Logic Rules

Degrees of Freedom in Deep Neural Networks

Deep Networks with Stochastic Depth

LIFT: Learned Invariant Feature Transform

Bridging the Gaps Between Residual Learning, Recurrent Neural Networks and Visual Cortex

Understanding How Image Quality Affects Deep Neural Networks

Deep Embedding for Spatial Role Labeling

Unreasonable Effectiveness of Learning Neural Nets: Accessible States and Robust Ensembles

Learning Deep Representation for Imbalanced Classification

Newtonian Image Understanding: Unfolding the Dynamics of Objects in Static Images

DeepMath - Deep Sequence Models for Premise Selection

Convolutional Neural Networks Analyzed via Convolutional Sparse Coding

Systematic evaluation of CNN advances on the ImageNet

Why does deep and cheap learning work so well?

A scalable convolutional neural network for task-specified scenarios via knowledge distillation

Alternating Back-Propagation for Generator Network

A Novel Representation of Neural Networks

Optimization of Convolutional Neural Network using Microcanonical Annealing Algorithm

Uncertainty in Deep Learning

Deep Convolutional Neural Network Design Patterns

Extensions and Limitations of the Neural GPU

Neural Functional Programming

Deep Information Propagation

Compressed Learning: A Deep Neural Network Approach

A backward pass through a CNN using a generative model of its activations

Understanding deep learning requires rethinking generalization

Learning the Number of Neurons in Deep Networks

Survey of Expressivity in Deep Neural Networks

  • intro: Presented at NIPS 2016 Workshop on Interpretable Machine Learning in Complex Systems
  • intro: Google Brain & Cornell University & Stanford University
  • arxiv: https://arxiv.org/abs/1611.08083

Designing Neural Network Architectures using Reinforcement Learning

Towards Robust Deep Neural Networks with BANG

Deep Quantization: Encoding Convolutional Activations with Deep Generative Model

A Probabilistic Theory of Deep Learning

A Probabilistic Framework for Deep Learning

Paying More Attention to Attention: Improving the Performance of Convolutional Neural Networks via Attention Transfer

Risk versus Uncertainty in Deep Learning: Bayes, Bootstrap and the Dangers of Dropout

Outrageously Large Neural Networks: The Sparsely-Gated Mixture-of-Experts Layer

Deep Network Guided Proof Search

PathNet: Evolution Channels Gradient Descent in Super Neural Networks

Reluplex: An Efficient SMT Solver for Verifying Deep Neural Networks

The Power of Sparsity in Convolutional Neural Networks

Learning across scales - A multiscale method for Convolution Neural Networks

Stacking-based Deep Neural Network: Deep Analytic Network on Convolutional Spectral Histogram Features

A Compositional Object-Based Approach to Learning Physical Dynamics

Genetic CNN

Deep Sets

  • intro: Amazon Web Services & CMU
  • keywords: statistic estimation, point cloud classification, set expansion, and image tagging
  • arxiv: https://arxiv.org/abs/1703.06114

Multiscale Hierarchical Convolutional Networks

https://arxiv.org/abs/1703.04140 https://github.com/jhjacobsen/HierarchicalCNN

Deep Neural Networks Do Not Recognize Negative Images

https://arxiv.org/abs/1703.06857

Failures of Deep Learning

Multi-Scale Dense Convolutional Networks for Efficient Prediction

Scaling the Scattering Transform: Deep Hybrid Networks

Deep Learning is Robust to Massive Label Noise

https://arxiv.org/abs/1705.10694

Input Fast-Forwarding for Better Deep Learning

Deep Mutual Learning

Automated Problem Identification: Regression vs Classification via Evolutionary Deep Networks

Revisiting Unreasonable Effectiveness of Data in Deep Learning Era

Deep Layer Aggregation

Improving Robustness of Feature Representations to Image Deformations using Powered Convolution in CNNs

https://arxiv.org/abs/1707.07830

Learning uncertainty in regression tasks by deep neural networks

Generalizing the Convolution Operator in Convolutional Neural Networks

https://arxiv.org/abs/1707.09864

Convolution with Logarithmic Filter Groups for Efficient Shallow CNN

https://arxiv.org/abs/1707.09855

Deep Multi-View Learning with Stochastic Decorrelation Loss

https://arxiv.org/abs/1707.09669

Take it in your stride: Do we need striding in CNNs?

https://arxiv.org/abs/1712.02502

Security Risks in Deep Learning Implementation

Online Learning with Gated Linear Networks

On the Information Bottleneck Theory of Deep Learning

https://openreview.net/forum?id=ry_WPG-A-&noteId=ry_WPG-A

The Unreasonable Effectiveness of Deep Features as a Perceptual Metric

Less is More: Culling the Training Set to Improve Robustness of Deep Neural Networks

Towards an Understanding of Neural Networks in Natural-Image Spaces

https://arxiv.org/abs/1801.09097

Deep Private-Feature Extraction

https://arxiv.org/abs/1802.03151

Not All Samples Are Created Equal: Deep Learning with Importance Sampling

Tutorials and Surveys

A Survey: Time Travel in Deep Learning Space: An Introduction to Deep Learning Models and How Deep Learning Models Evolved from the Initial Ideas

On the Origin of Deep Learning

Efficient Processing of Deep Neural Networks: A Tutorial and Survey

The History Began from AlexNet: A Comprehensive Survey on Deep Learning Approaches

{https://arxiv.org/abs/1803.01164}(https://arxiv.org/abs/1803.01164)

Mathematics of Deep Learning

A Mathematical Theory of Deep Convolutional Neural Networks for Feature Extraction

Mathematics of Deep Learning

Local Minima

Local minima in training of deep networks

Deep linear neural networks with arbitrary loss: All local minima are global

Gradient Descent Learns One-hidden-layer CNN: Don’t be Afraid of Spurious Local Minima

CNNs are Globally Optimal Given Multi-Layer Support

Spurious Local Minima are Common in Two-Layer ReLU Neural Networks

https://arxiv.org/abs/1712.08968

Dive Into CNN

Structured Receptive Fields in CNNs

How ConvNets model Non-linear Transformations

Separable Convolutions / Grouped Convolutions

Factorized Convolutional Neural Networks

Design of Efficient Convolutional Layers using Single Intra-channel Convolution, Topological Subdivisioning and Spatial “Bottleneck” Structure

STDP

A biological gradient descent for prediction through a combination of STDP and homeostatic plasticity

An objective function for STDP

Towards a Biologically Plausible Backprop

Target Propagation

How Auto-Encoders Could Provide Credit Assignment in Deep Networks via Target Propagation

Difference Target Propagation

Zero Shot Learning

Learning a Deep Embedding Model for Zero-Shot Learning

Zero-Shot (Deep) Learning

https://amundtveit.com/2016/11/18/zero-shot-deep-learning/

Zero-shot learning experiments by deep learning.

https://github.com/Elyorcv/zsl-deep-learning

Zero-Shot Learning - The Good, the Bad and the Ugly

Semantic Autoencoder for Zero-Shot Learning

Zero-Shot Learning via Category-Specific Visual-Semantic Mapping

https://arxiv.org/abs/1711.06167

Zero-Shot Learning via Class-Conditioned Deep Generative Models

Feature Generating Networks for Zero-Shot Learning

https://arxiv.org/abs/1712.00981

Zero-Shot Visual Recognition using Semantics-Preserving Adversarial Embedding Network

https://arxiv.org/abs/1712.01928

Combining Deep Universal Features, Semantic Attributes, and Hierarchical Classification for Zero-Shot Learning

Multi-Context Label Embedding

Incremental Learning

iCaRL: Incremental Classifier and Representation Learning

FearNet: Brain-Inspired Model for Incremental Learning

https://arxiv.org/abs/1711.10563

Incremental Learning in Deep Convolutional Neural Networks Using Partial Network Sharing

Incremental Classifier Learning with Generative Adversarial Networks

https://arxiv.org/abs/1802.00853

Ensemble Deep Learning

Convolutional Neural Fabrics

Stochastic Multiple Choice Learning for Training Diverse Deep Ensembles

Snapshot Ensembles: Train 1, Get M for Free

Ensemble Deep Learning

Domain Adaptation

Adversarial Discriminative Domain Adaptation

Parameter Reference Loss for Unsupervised Domain Adaptation

https://arxiv.org/abs/1711.07170

Residual Parameter Transfer for Deep Domain Adaptation

https://arxiv.org/abs/1711.07714

Adversarial Feature Augmentation for Unsupervised Domain Adaptation

https://arxiv.org/abs/1711.08561

Image to Image Translation for Domain Adaptation

https://arxiv.org/abs/1712.00479

Incremental Adversarial Domain Adaptation

https://arxiv.org/abs/1712.07436

Deep Visual Domain Adaptation: A Survey

https://arxiv.org/abs/1802.03601

Unsupervised Domain Adaptation: A Multi-task Learning-based Method

https://arxiv.org/abs/1803.09208

Importance Weighted Adversarial Nets for Partial Domain Adaptation

https://arxiv.org/abs/1803.09210

Open Set Domain Adaptation by Backpropagation

https://arxiv.org/abs/1804.10427

Embedding

Learning Deep Embeddings with Histogram Loss

Full-Network Embedding in a Multimodal Embedding Pipeline

https://arxiv.org/abs/1707.09872

Clustering-driven Deep Embedding with Pairwise Constraints

https://arxiv.org/abs/1803.08457

Regression

A Comprehensive Analysis of Deep Regression

https://arxiv.org/abs/1803.08450

CapsNets

Dynamic Routing Between Capsules

Capsule Networks (CapsNets) – Tutorial

Improved Explainability of Capsule Networks: Relevance Path by Agreement

Computer Vision

A Taxonomy of Deep Convolutional Neural Nets for Computer Vision

On the usability of deep networks for object-based image analysis

Learning Recursive Filters for Low-Level Vision via a Hybrid Neural Network

DSAC - Differentiable RANSAC for Camera Localization

Toward Geometric Deep SLAM

All-In-One Network

HyperFace: A Deep Multi-task Learning Framework for Face Detection, Landmark Localization, Pose Estimation, and Gender Recognition

UberNet: Training a `Universal’ Convolutional Neural Network for Low-, Mid-, and High-Level Vision using Diverse Datasets and Limited Memory

An All-In-One Convolutional Neural Network for Face Analysis

  • intro: simultaneous face detection, face alignment, pose estimation, gender recognition, smile detection, age estimation and face recognition
  • arxiv: https://arxiv.org/abs/1611.00851

MultiNet: Real-time Joint Semantic Reasoning for Autonomous Driving

Deep Learning for Data Structures

The Case for Learned Index Structures

Projects

Top Deep Learning Projects

deepnet: Implementation of some deep learning algorithms

DeepNeuralClassifier(Julia): Deep neural network using rectified linear units to classify hand written digits from the MNIST dataset

Clarifai Node.js Demo

Deep Learning in Rust

Implementation of state-of-art models in Torch

Deep Learning (Python, C, C++, Java, Scala, Go)

deepmark: THE Deep Learning Benchmarks

Siamese Net

  • intro: “This package shows how to train a siamese network using Lasagne and Theano and includes network definitions for state-of-the-art networks including: DeepID, DeepID2, Chopra et. al, and Hani et. al. We also include one pre-trained model using a custom convolutional network.”
  • github: https://github.com/Kadenze/siamese_net

PRE-TRAINED CONVNETS AND OBJECT LOCALISATION IN KERAS

Deep Learning algorithms with TensorFlow: Ready to use implementations of various Deep Learning algorithms using TensorFlow

Fast Multi-threaded VGG 19 Feature Extractor

Live demo of neural network classifying images

http://ml4a.github.io/dev/demos/cifar_confusion.html#

mojo cnn: c++ convolutional neural network

DeepHeart: Neural networks for monitoring cardiac data

Deep Water: Deep Learning in H2O using Native GPU Backends

Greentea LibDNN: Greentea LibDNN - a universal convolution implementation supporting CUDA and OpenCL

Dracula: A spookily good Part of Speech Tagger optimized for Twitter

Trained image classification models for Keras

PyCNN: Cellular Neural Networks Image Processing Python Library

regl-cnn: Digit recognition with Convolutional Neural Networks in WebGL

dagstudio: Directed Acyclic Graph Studio with Javascript D3

NEUGO: Neural Networks in Go

gvnn: Neural Network Library for Geometric Computer Vision

DeepForge: A development environment for deep learning

Implementation of recent Deep Learning papers

GPU-accelerated Theano & Keras on Windows 10 native

Head Pose and Gaze Direction Estimation Using Convolutional Neural Networks

Intel(R) Math Kernel Library for Deep Neural Networks (Intel(R) MKL-DNN)

Deep CNN and RNN - Deep convolution/recurrent neural network project with TensorFlow

Experimental implementation of novel neural network structures

WaterNet: A convolutional neural network that identifies water in satellite images

Kur: Descriptive Deep Learning

Development of JavaScript-based deep learning platform and application to distributed training

NewralNet

FeatherCNN

Readings and Questions

What you wanted to know about AI

http://fastml.com/what-you-wanted-to-know-about-ai/

Epoch vs iteration when training neural networks

Questions to Ask When Applying Deep Learning

http://deeplearning4j.org/questions.html

How can I know if Deep Learning works better for a specific problem than SVM or random forest?

What is the difference between deep learning and usual machine learning?

Resources

Awesome Deep Learning

Awesome-deep-vision: A curated list of deep learning resources for computer vision

Applied Deep Learning Resources: A collection of research articles, blog posts, slides and code snippets about deep learning in applied settings.

Deep Learning Libraries by Language

Deep Learning Resources

http://yanirseroussi.com/deep-learning-resources/

Deep Learning Resources

https://omtcyfz.github.io/2016/08/29/Deep-Learning-Resources.html

Turing Machine: musings on theory & code(DEEP LEARNING REVOLUTION, summer 2015, state of the art & topnotch links)

https://vzn1.wordpress.com/2015/09/01/deep-learning-revolution-summer-2015-state-of-the-art-topnotch-links/

BICV Group: Biologically Inspired Computer Vision research group

http://www.bicv.org/deep-learning/

Learning Deep Learning

http://rt.dgyblog.com/ref/ref-learning-deep-learning.html

Summaries and notes on Deep Learning research papers

Deep Learning Glossary

The Deep Learning Playbook

https://medium.com/@jiefeng/deep-learning-playbook-c5ebe34f8a1a#.eg9cdz5ak

Deep Learning Study: Study of HeXA@UNIST in Preparation for Submission

Deep Learning Books

awesome-very-deep-learning: A curated list of papers and code about very deep neural networks (50+ layers)

Deep Learning Resources and Tutorials using Keras and Lasagne

Deep Learning: Definition, Resources, Comparison with Machine Learning

Awesome - Most Cited Deep Learning Papers

The most cited papers in computer vision and deep learning

deep learning papers: A place to collect papers that are related to deep learning and computational biology

papers-I-read

LEARNING DEEP LEARNING - MY TOP-FIVE LIST

awesome-free-deep-learning-papers

DeepLearningBibliography: Bibliography for Publications about Deep Learning using GPU

Deep Learning Papers Reading Roadmap

deep-learning-papers

Deep Learning and applications in Startups, CV, Text Mining, NLP

ml4a-guides - a collection of practical resources for working with machine learning software, including code and tutorials

http://ml4a.github.io/guides/

deep-learning-resources

21 Deep Learning Videos, Tutorials & Courses on Youtube from 2016

https://www.analyticsvidhya.com/blog/2016/12/21-deep-learning-videos-tutorials-courses-on-youtube-from-2016/

Awesome Deep learning papers and other resources

awesome-deep-vision-web-demo

Summaries of machine learning papers

https://github.com/aleju/papers

Awesome Deep Learning Resources

https://github.com/guillaume-chevalier/awesome-deep-learning-resources

Virginia Tech Vision and Learning Reading Group

https://github.com//vt-vl-lab/reading_group

MEGALODON: ML/DL Resources At One Place

Arxiv Pages

Neural and Evolutionary Computing

https://arxiv.org/list/cs.NE/recent

Learning

https://arxiv.org/list/cs.LG/recent

Computer Vision and Pattern Recognition

https://arxiv.org/list/cs.CV/recent

Arxiv Sanity Preserver

Today’s Deep Learning

http://todaysdeeplearning.com/

arXiv Analytics

http://arxitics.com/

Tools

DNNGraph - A deep neural network model generation DSL in Haskell

Deep playground: an interactive visualization of neural networks, written in typescript using d3.js

Neural Network Package

  • intro: This package provides an easy and modular way to build and train simple or complex neural networks using Torch
  • github: https://github.com/torch/nn

deepdish: Deep learning and data science tools from the University of Chicago deepdish: Serving Up Chicago-Style Deep Learning

AETROS CLI: Console application to manage deep neural network training in AETROS Trainer

Deep Learning Studio: Cloud platform for designing Deep Learning AI without programming

http://deepcognition.ai/

cuda-on-cl: Build NVIDIA® CUDA™ code for OpenCL™ 1.2 devices

Receptive Field Calculator

receptivefield

Challenges / Hackathons

VisionHack 2017

  • intro: 10 - 14 Sep 2017, Moscow, Russia
  • intro: a full-fledged hackathon that will last three full days
  • homepage: http://visionhack.misis.ru/

NVIDIA AI City Challenge Workshop at CVPR 2018

http://www.aicitychallenge.org/

Books

Deep Learning

Fundamentals of Deep Learning: Designing Next-Generation Artificial Intelligence Algorithms

FIRST CONTACT WITH TENSORFLOW: Get started with with Deep Learning programming

《解析卷积神经网络—深度学习实践手册》

Make Your Own Neural Network: IPython Neural Networks on a Raspberry Pi Zero

Blogs

Neural Networks and Deep Learning

http://neuralnetworksanddeeplearning.com

Deep Learning Reading List

http://deeplearning.net/reading-list/

WILDML: A BLOG ABOUT MACHINE LEARNING, DEEP LEARNING AND NLP.

http://www.wildml.com/

Andrej Karpathy blog

http://karpathy.github.io/

Rodrigob’s github page

http://rodrigob.github.io/

colah’s blog

http://colah.github.io/

What My Deep Model Doesn’t Know…

http://mlg.eng.cam.ac.uk/yarin/blog_3d801aa532c1ce.html

Christoph Feichtenhofer

Image recognition is not enough: As with language, photos need contextual intelligence

https://medium.com/@ken_getquik/image-recognition-is-not-enough-293cd7d58004#.dex817l2z

ResNets, HighwayNets, and DenseNets, Oh My!

The Frontiers of Memory and Attention in Deep Learning

Design Patterns for Deep Learning Architectures

http://www.deeplearningpatterns.com/doku.php

Building a Deep Learning Powered GIF Search Engine

850k Images in 24 hours: Automating Deep Learning Dataset Creation

https://gab41.lab41.org/850k-images-in-24-hours-automating-deep-learning-dataset-creation-60bdced04275#.xhq9feuxx

How six lines of code + SQL Server can bring Deep Learning to ANY App

Neural Network Architectures

About me

Hi world~

Recent Posts

Links

A blog template forked from zJiaJun. Powered by Jekyll.