現在のDNNにおける未解決問題

http://www.slideshare.net/DaisukeOkanohara/dnn-62218756

脳型計算機雑談会での資料です

1. 大きなNNの学習はなぜ一様に成功するか
2. 敵対的生成ネットワーク(GAN)の解析
3. seq2seqによる可変長情報の埋め込み

Stanford University CS224d: Deep Learning for Natural Language Processing

http://cs224d.stanford.edu/

Through lectures and programming assignments students will learn the necessary engineering tricks for making neural networks work on practical problems.

http://cs224d.stanford.edu/syllabus.html

[1502.03167] Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift

http://arxiv.org/abs/1502.03167

Our method draws its strength from making normalization a part of the model architecture and performing the normalization for each training mini-batch.

Automatic Colorization

http://tinyclouds.org/colorize/

This project is an attempt to use modern deep learning techniques to automatically colorize black and white photos.

http://tinyclouds.org/

Ryan Dahl

Computational Network Toolkit (CNTK) - Home

https://cntk.codeplex.com/

CNTK, the Computational Network Toolkit by Microsoft Research, is a unified deep-learning toolkit

CNTK allows to easily realize and combine popular model types such as feed-forward DNNs, convolutional nets (CNNs), and recurrent networks (RNNs/LSTMs). It implements stochastic gradient descent (SGD, error backpropagation) learning with automatic differentiation and parallelization across multiple GPUs and servers.

Microsoft Research による、Windows と Linux に対応した deep learning のツールキット。
下記ページに速度性能を比較した記事がある。

Microsoft Computational Network Toolkit offers most efficient distributed deep learning computational performance
http://blogs.technet.com/b/inside_microsoft_research/archive/2015/12/07/microsoft-computational-network-toolkit-offers-most-efficient-distributed-deep-learning-computational-performance.aspx

Teaching Machines to Read and Comprehend (slide)

http://lxmls.it.pt/2015/lxmls15.pdf

Conclusion

Summary
* supervised machine reading is a viable research direction with the available data,
* LSTM based recurrent networks constantly surprise with their ability to encode dependencies in sequences,
* attention is a very effective and exible modelling technique.

Future directions
* more and better data, corpus querying, and cross document queries,
* recurrent networks incorporating long term and working memory are well suited to NLU task.

Lisbon Machine Learning School 2015 のスライド。トピックは自然言語処理。

Computational Linguistics and Deep Learning

http://www.mitpressjournals.org/doi/abs/10.1162/COLI_a_00239

Christopher D. Manning
Stanford University
MIT Press Journals - Computational Linguistics - Early Access - Citation

Deep Learningの教材 - こんな夢を見た

http://hytae.hatenablog.com/entry/2015/11/14/Deep_Learning%E3%81%AE%E6%95%99%E6%9D%90

Deep Learningを勉強するにあたって内容がまとまっている教材についてリスト化しました。

Marvin: Deep Learning in N Dimensions

http://marvin.is/

Marvin was born to be hacked, relying on few dependencies and basic C++. All code lives in two files (marvin.hpp and marvin.cu) and all numbers take up two bytes (FP16).

Marvin’s life depends on an NVIDIA GPU with CUDA 7.5 and cuDNN 3.

https://github.com/PrincetonVision/marvin/

Marvin is a GPU-only neural network framework made with simplicity, hackability, speed, memory consumption, and high dimensional data in mind.

The MIT License (MIT)
Copyright (c) 2015 Princeton Vision Group

Understanding Convolutional Neural Networks for NLP | WildML

http://www.wildml.com/2015/11/understanding-convolutional-neural-networks-for-nlp/

In this post I’ll try to summarize what CNNs are, and how they’re used in NLP.

Deep Learningの基礎と応用

http://www.slideshare.net/beam2d/deep-learning-52872945

第14回情報科学技術フォーラム (FIT2015) のイベント企画「ビッグデータ解析のための機械学習技術」におけるチュートリアル講演資料です。ニューラルネットの基礎（計算グラフとしての定式化、勾配法、誤差逆伝播法）をさらったあと、最近画像や自然言語などで注目されている応用・手法について広く浅く紹介しています。

DEEP LEARNING: Methods and Applications - Microsoft Research

http://research.microsoft.com/apps/pubs/default.aspx?id=209355

Abstract

This book is aimed to provide an overview of general deep learning methodology and its applications to a variety of signal and information processing tasks.

26 Things I Learned in the Deep Learning Summer School - Marek Rei

http://www.marekrei.com/blog/26-things-i-learned-in-the-deep-learning-summer-school/

Neural networks and deep learning

http://neuralnetworksanddeeplearning.com/

Neural Networks and Deep Learning is a free online book.

Chainer: A flexible framework of neural networks

http://chainer.org/

A Powerful, Flexible, and Intuitive Framework of Neural Networks

ライセンスは MIT License。

Deep Learning のフレームワーク Chainer を公開しました | Preferred Research

http://research.preferred.jp/2015/06/deep-learning-chainer/

本日、Deep Learning の新しいフレームワークである Chainer を公開しました。

Chainer は、ニューラルネットを誤差逆伝播法で学習するためのフレームワークです。以下のような特徴を持っています。

* Python のライブラリとして提供（要 Python 2.7+）
* あらゆるニューラルネットの構造に柔軟に対応
* 動的な計算グラフ構築による直感的なコード
* GPU をサポートし、複数 GPU をつかった学習も直感的に記述可能

ConvNetJS MNIST demo

http://cs.stanford.edu/people/karpathy/convnetjs/demo/mnist.html

This demo trains a Convolutional Neural Network on the MNIST digits dataset in your browser, with nothing but Javascript.

手書き数字の画像データを JavaScript のプログラムで学習するデモ。
学習中の各層（conv, pool, softmax）の重みやアクティベーション値をリアルタイムで見ることができる。

karpathy/convnetjs · GitHub

https://github.com/karpathy/convnetjs

ConvNetJS is a Javascript implementation of Neural networks, together with nice browser-based demos.

ConvNetJS: Deep Learning in your browser

http://cs.stanford.edu/people/karpathy/convnetjs/

ConvNetJS is a Javascript library for training Deep Learning models (mainly Neural Networks) entirely in your browser. Open a tab and you're training. No software requirements, no compilers, no installations, no GPUs, no sweat.

ディープラーニング チュートリアル（もしくは研究動向報告）

http://www.vision.is.tohoku.ac.jp/index.php/download_file/view/15/137/

岡谷貴之　齋藤真樹
東北大学

日本語のスライド。