Formula
w -= rate x g
Where W is weight, Rate is learning rate, G is gradient. It is minus to reduce the overshot of output value, and the delta at output should be Out-Ytrue instead of the other way round.
Training
Logging
In supervised learning, the trainer programme should log out loss value to see whether it is reducing. Also log out accuracy of the Validation Set (Test Set).