福島邦彦: Deep CNN ネオコグニトロンの学習

https://kaigi.org/jsai/webprogram/2016/pdf/130.pdf

最近のネオコグニトロンでは，中間層のS 細胞の学習に，Add-if-Silent (AiS) と名付けた学習則を用いている．この学習則は，S 細胞の入力結合の強度を定めるだけでなく，細胞そのものの発生も制御している．

Performance Benefits of Half Precision Floats | Intel® Software

https://software.intel.com/en-us/articles/performance-benefits-of-half-precision-floats/

3.2. Results

Using half-floats provides a performance benefit over 32-bit floats when 32-bit float data does not fit into the L1 cache. Specifically, half-floats provide an average speedup of 1.05x when 32-bit data would fit in the L2 cache, an average speedup of 1.3x when 32-bit data would fit in the L3 cache, and an average speedup of 1.6x when the 32-bit data would fit into memory. Additionally, while half-floats may not provide a direct performance benefit when 32-bit data would fit into the L1 cache, you may still experience an auxiliary benefit when using half-floats in your program because half-floats will use half as much space, which allows for significantly more of your programs data to reside in L1.

half-float (FP16) を使えば、使用するメモリがCPUのキャッシュに載らない場合では 1.6倍くらい高速化できるらしい。

Intel Intrinsics Guide

https://software.intel.com/sites/landingpage/IntrinsicsGuide/

The Intel Intrinsics Guide is an interactive reference tool for Intel intrinsic instructions, which are C style functions that provide access to many Intel instructions - including Intel® SSE, AVX, AVX-512, and more - without the need to write assembly code.