learngd (Neural Network Toolbox)

Neural Network Toolbox

Search Help Desk

learngd

Examples See Also

Gradient descent weight/bias learning function

Syntax

[dW,LS] = learngd(W,P,Z,N,A,T,E,gW,gA,D,LP,LS)

[db,LS] = learngd(b,ones(1,Q),Z,N,A,T,E,gW,gA,D,LP,LS)

info = learngd(code)

Description

learngd is the gradient descent weight/bias learning function.

learngd(W,P,Z,N,A,T,E,gW,gA,D,LP,LS) takes several inputs,

W - S x R weight matrix (or S x 1 bias vector).

P - R x Q input vectors (or ones(1,Q)).

Z - S x Q weighted input vectors.

N - S x Q net input vectors.

A - S x Q output vectors.

T - S x Q layer target vectors.

E - S x Q layer error vectors.

gW - S x R gradient with respect to performance.

gA - S x Q output gradient with respect to performance.

D - S x S neuron distances.

LP - Learning parameters, none, LP = [].

LS - Learning state, initially should be = [].

and returns,

dW - S x R weight (or bias) change matrix.

LS - New learning state.

Learning occurs according to learngd's learning parameter shown here with its default value.

LP.lr - 0.01 - Learning rate.

learngd(code) returns useful information for each code string:

'pnames' - Names of learning parameters.

'pdefaults' - Default learning parameters.

'needg' - Returns 1 if this function uses gW or gA.

Examples

Here we define a random gradient gW for a weight going to a layer with 3 neurons, from an input with 2 elements. We also define a learning rate of 0.5.

gW = rand(3,2);
lp.lr = 0.5;

Since learngd only needs these values to calculate a weight change (see algorithim below), we will use them to do so.

dW = learngd([],[],[],[],[],[],[],gW,[],[],lp,[])

Network Use

You can create a standard network that uses learngd with newff, newcf, or newelm. To prepare the weights and the bias of layer i of a custom network to adapt with learngd:

1.: Set net.adaptFcn to 'adaptwb'. net.adaptParam will automatically become trainwb's default parameters.

2.: Set each net.inputWeights{i,j}.learnFcn to 'learngd'. Set each net.layerWeights{i,j}.learnFcn to 'learngd'. Set net.biases{i}.learnFcn to 'learngd'. Each weight and bias learning parameter property will automatically be set to learngd's default parameters.

To allow the network to adapt:

1.: Set net.adaptParam properties to desired values.

2.: Call adapt with the network.

See newff or newcf for examples.

Algoritm

learngd calculates the weight change dW for a given neuron from the neuron's input P and error E, and the weight (or bias) learning rate LR, according to the gradient descent: dw = lr*gW.