       # Q: Differentiation of a neural network function

• To: mathgroup@smc.vnet.net
• Subject: [mg10865] Q: Differentiation of a neural network function
• Date: Wed, 11 Feb 1998 18:32:34 -0500
• Organization: Posted via ULCC Internet Services

```Hello,
Can anybody help me on this:?

I would like to obtain an expression for the derivative of the function
implemented by a fully connected feed forward neural network. a feed
forward neural network is often viewed as a black box of n real inputs
(X) and m real outputs (Y). *** FOR MY PROBLEM ASSUME m = 1 ***

Hence, this black box represents a mapping from R^n to R^m. This mapping
depends on a number of parameters, called the weights (like for example
the coefficients of a polyonym)

Training is the process of finding a particular value for each of these
parameters - the weights - so that a specified mapping can be
implemented.

After training, the black box represents the specific mapping which is
of the form:

Phi : R^n -> R | y = W_(N+1) x f(W_(N) x f(W_(N-1) x f( ... x f(W_2 x
f(W_1 x X))))))))

where:	W1 ... W_(N+1) are matrices containing the parameters (weights)
which are real numbers
f(a) = 1 / (1 + exp(-a))
and f(A), A is a matrix of a_ij, is the new matrix AA whose each
element aa_ij is equal
to f(a_ij).
and `x' is the cross product.

If all the stuff regarding the neural network are a bit unclear please
ignore them and just tell me how to obtain an expression for the
derivative of a function of your choice, i will try and work from
there.

for example you might tell me how to obtain the derivative of f(x) = a /
(b + c*exp(-x)) when a, b and c are general parameters (not
instantiated to a specific value).

Also, if mathematica can not do that, could you suggest some other
method to do it, other than by hand?

thank you very much,