[Neural Networks] Drawbacks of Backpropagation
[Neural Networks] Drawbacks of Backpropagation

BP algorithm has been described in 'An Introduction to Backpropagation and Multilayer Perceptrons'. And the implement of the BP algorithm has been recorded at 'The Backpropagation Algorithm (Part I)' and 'The Backpropagation Algorithm (Part II)'. BP has worked in many applications, but there are too many drawbacks in the process. The basic BP algorithm is too slow for most practical applications that it might take days or even weeks in training. And these following 4 posts are some investigations to make the BP algorithm more practical and to speed it up.

[Neural Networks] The Backpropagation Algorithm (Part I)
[Neural Networks] The Backpropagation Algorithm (Part I)

We have seen a three-layer network is flexible in approximating functions. If we had a more-than-three-layer network, it could be used to approximate any functions as close as we want. However, another trouble came to us is how to train these networks. This problem almost killed neural networks in the 1970s. Until backpropagation(BP for short) algorithm was found that it is an efficient algorithm in training multiple layers networks.