我正在学习来自deeplearning.ai
课程中神经网络的正则化。在这里 ,在dropout正则化中,教授说如果应用了dropout,则计算得出的激活值将比不应用dropout时小(在测试时)。因此,我们需要缩放激活值以使测试阶段更简单。
我理解了这一事实,但我不明白如何进行缩放。这是一个用于实现反向dropout的代码示例。
keep_prob = 0.8 # 0 <= keep_prob <= 1
l = 3 # this code is only for layer 3
# the generated number that are less than 0.8 will be dropped. 80% stay, 20% dropped
d3 = np.random.rand(a[l].shape[0], a[l].shape[1]) < keep_prob
a3 = np.multiply(a3,d3) # keep only the values in d3
# increase a3 to not reduce the expected value of output
# (ensures that the expected value of a3 remains the same) - to solve the scaling problem
a3 = a3 / keep_prob
在上述代码中,为什么要将激活值除以0.8或保留层中节点的概率(keep_prob)?任何数值示例都将有所帮助。