[Date Index] [Thread Index] [Author Index]

Q: Differentiation of a neural network function

• To: mathgroup@smc.vnet.net
• Subject: [mg10865] Q: Differentiation of a neural network function
• From: livantes@cs.city.ac.uk (Andreas Hadjiprocopis)
• Date: Wed, 11 Feb 1998 18:32:34 -0500
• Organization: Posted via ULCC Internet Services

Hello,
Can anybody help me on this:?

I would like to obtain an expression for the derivative of the function
implemented by a fully connected feed forward neural network. a feed
forward neural network is often viewed as a black box of n real inputs
(X) and m real outputs (Y). *** FOR MY PROBLEM ASSUME m = 1 ***

Hence, this black box represents a mapping from R^n to R^m. This mapping
depends on a number of parameters, called the weights (like for example
the coefficients of a polyonym)

Training is the process of finding a particular value for each of these
parameters - the weights - so that a specified mapping can be
implemented.

After training, the black box represents the specific mapping which is
of the form:

Phi : R^n -> R | y = W_(N+1) x f(W_(N) x f(W_(N-1) x f( ... x f(W_2 x
f(W_1 x X))))))))

where:	W1 ... W_(N+1) are matrices containing the parameters (weights)
which are real numbers
	f(a) = 1 / (1 + exp(-a))
	and f(A), A is a matrix of a_ij, is the new matrix AA whose each
element aa_ij is equal
	to f(a_ij).
	and `x' is the cross product.

If all the stuff regarding the neural network are a bit unclear please
ignore them and just tell me how to obtain an expression for the
derivative of a function of your choice, i will try and work from
there.

for example you might tell me how to obtain the derivative of f(x) = a /
(b + c*exp(-x)) when a, b and c are general parameters (not
instantiated to a specific value).

Also, if mathematica can not do that, could you suggest some other
method to do it, other than by hand?

thank you very much,
(please use email if possible)
--
Andreas Hadjiprocopis		livantes@soi.city.ac.uk Computer Science
Department	http://www.soi.city.ac.uk/~livantes/home.html Room A528,
City University	+44 71  477 8551 (telephone) London, UK, EC1V 0HB		+44
71  477 8587 (fax)