Unsupervised Feature Learning and Deep Learning Tutorial

http://ufldl.stanford.edu/tutorial/

Description: This tutorial will teach you the main ideas of Unsupervised Feature Learning and Deep Learning.

ディープ・ラーニングだけではなく、多層ニューラルネットなどの基礎技術の解説もある。

[1502.02551] Deep Learning with Limited Numerical Precision

http://arxiv.org/abs/1502.02551

Our results show that deep networks can be trained using only 16-bit wide fixed-point number representation when using stochastic rounding, and incur little to no degradation in the classification accuracy.

16ビットの浮動小数点数（半精度浮動小数点数）でも確率的に丸め処理をすれば使い物になるらしい。

GTC 2015 - Googleが「ディープラーニング」に対する取り組みを紹介 | マイナビニュース

http://news.mynavi.jp/articles/2015/03/20/gtc02/

Long short term memory - Wikipedia, the free encyclopedia

http://en.wikipedia.org/wiki/Long_short_term_memory

Long short term memory (LSTM) is a recurrent neural network (RNN) architecture

[1503.04069] LSTM: A Search Space Odyssey

http://arxiv.org/abs/1503.04069

• The most commonly used LSTM architecture (vanilla LSTM) performs reasonably well on various datasets and using any of eight possible modifications does not significantly improve the LSTM performance.
• Certain modifications such as coupling the input and forget gates or removing peephole connections simplify LSTM without significantly hurting performance.
• The forget gate and the output activation function are the critical components of the LSTM block. While the first is crucial for LSTM performance, the second is necessary whenever the cell state is unbounded.

CS231n Convolutional Neural Networks for Visual Recognition

http://cs231n.github.io/

These notes accompany the Stanford CS class CS231n: Convolutional Neural Networks for Visual Recognition.

Stanford University CS231n: Convolutional Neural Networks for Visual Recognition

http://vision.stanford.edu/teaching/cs231n/

Rectifier (neural networks) - Wikipedia, the free encyclopedia

http://en.wikipedia.org/wiki/Rectifier_%28neural_networks%29#Leaky_ReLUs

Leaky ReLUs allow a small, non-zero gradient when the unit is not active.

CIFAR-10 Competition Winners: Interviews with Dr. Ben Graham, Phil Culliton, & Zygmunt Zając | no free hunch

http://blog.kaggle.com/2015/01/02/cifar-10-competition-winners-interviews-with-dr-ben-graham-phil-culliton-zygmunt-zajac/

The "MP" entries above denote max pooling, and "VeryLeakyReLU" denotes a "leaky" ReLU with a fairly large (alpha was 0.33) non-zero gradient.

[1412.6980] Adam: A Method for Stochastic Optimization

http://arxiv.org/abs/1412.6980

We introduce Adam, an algorithm for first-order gradient-based optimization of stochastic objective functions.

意思決定を行う脳の組織構造 | 沖縄科学技術大学院大学 OIST

https://www.oist.jp/ja/news-center/news/2015/2/26/18669

腹側線条体は、ラットが試行をいつ開始するかを決める早い段階で最も活性化しました。背内側線条体では、左右いずれかの穴に向かう前、左右の選択の結果得られる報酬の予測に応じて発火レベルが変化しました。背外側線条体は、試行の様々なタイミングで短い発火を示し、細かな運動制御に関係していると考えられます。

DEEP LEARNING An MIT Press book in preparation

http://www-labs.iro.umontreal.ca/~bengioy/dlbook/

An MIT Press book in preparation
Yoshua Bengio, Ian Goodfellow and Aaron Courville