srchbre (Neural Network Toolbox)

One-dimensional interval location using Brent's method

[a,gX,perf,retcode,delta,tol] = srchbre(net,X,Pd,Tl,Ai,Q,TS,dX,gX,perf,dperf,delta,tol,ch_perf)

srchbre is a linear search routine. It searches in a given direction to locate the minimum of the performance function in that direction. It uses a technique called Brent's technique.

srchbre(net,X,Pd,Tl,Ai,Q,TS,dX,gX,perf,dperf,delta,tol,ch_perf) takes these inputs,

net - Neural network.

X - Vector containing current values of weights and biases.

Pd - Delayed input vectors.

Tl - Layer target vectors.

Ai - Initial input delay conditions.

Q - Batch size.

TS - Time steps.

dX - Search direction vector.

gX - Gradient vector.

perf - Performance value at current X.

dperf - Slope of performance value at current X in direction of dX.

delta - Initial step size.

tol - Tolerance on search.

ch_perf - Change in performance on previous step.

and returns,

A - Step size which minimizes performance.

gX - Gradient at new minimum point.

perf - Performance value at new minimum point.

retcode - Return code, which has three elements. The first two elements correspond to the number of function evaluations in the two stages of the search. The third element is a return code. These will have different meanings for different search algorithms. Some may not be used in this function.

0 - normal; 1 - minimum step taken;

2 - maximum step taken; 3 - beta condition not met.

delta - New initial step size. Based on the current step size.

tol - New tolerance on search.

Parameters used for the brent algorithm are:

alpha - Scale factor which determines sufficient reduction in perf.

beta - Scale factor which determines sufficiently large step size.

bmax - Largest step size.

scale_tol - Parameter which relates the tolerance tol to the initial step size delta. Usually set to 20.

The defaults for these parameters are set in the training function which calls it. See traincgf, traincgb, traincgp, trainbfg, trainoss.

Dimensions for these variables are:

Pd - No x Ni x TS cell array, each element P{i,j,ts} is a Dij x Q matrix.

Tl - Nl x TS cell array, each element P{i,ts} is an Vi x Q matrix.

Ai - Nl x LD cell array, each element Ai{i,k} is an Si x Q matrix.

where

Ni = net.numInputs

Nl = net.numLayers

LD = net.numLayerDelays

Ri = net.inputs{i}.size

Si = net.layers{i}.size

Vi = net.targets{i}.size

Dij = Ri * length(net.inputWeights{i,j}.delays)

Examples

Here is a problem consisting of inputs P and targets T that we would like to solve with a network.

P = [0 1 2 3 4 5];
T = [0 0 0 1 1 1];

Here a two-layer feed-forward network is created. The network's input ranges from [0 to 10]. The first layer has two tansig neurons, and the second layer has one logsig neuron. The traincgf network training function and the srchbac search function are to be used.

Create and Test a Network

net = newff([0 5],[2 1],{'tansig','logsig'},'traincgf');
a = sim(net,p)

Train and Retest the Network

net.trainParam.searchFcn = 'srchbre';
net.trainParam.epochs = 50;
net.trainParam.show = 10;
net.trainParam.goal = 0.1;
net = train(net,p,t);
a = sim(net,p)

Network Use

You can create a standard network that uses srchbre with newff, newcf, or newelm.

To prepare a custom network to be trained with traincgf, using the line search function srchbre:

1.: Set net.trainFcn to 'traincgf'. This will set net.trainParam to traincgf's default parameters.

2.: Set net.trainParam.searchFcn to 'srchbre'.

The srchbre function can be used with any of the following training functions: traincgf, traincgb, traincgp, trainbfg, trainoss.

Algorithm

srchbre brackets the minimum of the performance function in the search direction dX, using Brent's algorithm described on page 46 of Scales (Introduction to Non-Linear Estimation 1985). It is a hybrid algorithm based on the golden section search and the quadratic approximation.