Baki Gökgöz

Activation function saturation determines robustness to conductance nonlinearity in analog in-memory learning

In analog memristive neural networks the weighted summation is performed efficiently in the crossbar, but how the activation function interacts with the imperfect, non-linear conductance updates of the devices remains poorly understood. We compare five activation functions—Sigmoid, Tanh, ReLU, GELU and SiLU—on MNIS...