Approximating Hessians for Neural Networks
Speaker: Maciej Skorski (Faculty of Science, Technology and Medicine; University of Luxembourg)
Title: Approximating Hessians for Neural Networks
Time: Wednesday, 2021.06.30, 10:00 a.m. (CET)
Place: fully virtual (contact Jakub Lengiewicz to register)
Format: 30 min. presentation + 30 min. discussion
Abstract: Second-order optimization techniques have been proven successful, but cannot be fully leveraged for neural networks due to infeasibility of hessian calculations. I will discuss an approximated way of computing the hessian for neural networks, which emerges from a careful analysis of the hessian chain rule and activation functions.