❓

degeneracy 와 degradation 의 차이?

상태

해결되지 않은 궁금증

생성일

2021/08/10 12:07

아예 다른거임.

degradation

Vanishing gradient emerged earlier than degradation. Vanishing gradient is the phenomenon that with the increase of the neural network layers, the gradient become too small to allow network evolve. This problem is largely addressed by renormalisation and ReLU activation function. After people tackled that problem and then tried to make neural network deeper, the degradation problem emerged. People found that with deeper layers, network gets higher error rate. ResNet is introduced to address that problem and got big success.