Q: Differentiation of a neural network function
- To: mathgroup@smc.vnet.net
- Subject: [mg10865] Q: Differentiation of a neural network function
- From: livantes@cs.city.ac.uk (Andreas Hadjiprocopis)
- Date: Wed, 11 Feb 1998 18:32:34 -0500
- Organization: Posted via ULCC Internet Services
Hello, Can anybody help me on this:? I would like to obtain an expression for the derivative of the function implemented by a fully connected feed forward neural network. a feed forward neural network is often viewed as a black box of n real inputs (X) and m real outputs (Y). *** FOR MY PROBLEM ASSUME m = 1 *** Hence, this black box represents a mapping from R^n to R^m. This mapping depends on a number of parameters, called the weights (like for example the coefficients of a polyonym) Training is the process of finding a particular value for each of these parameters - the weights - so that a specified mapping can be implemented. After training, the black box represents the specific mapping which is of the form: Phi : R^n -> R | y = W_(N+1) x f(W_(N) x f(W_(N-1) x f( ... x f(W_2 x f(W_1 x X)))))))) where: W1 ... W_(N+1) are matrices containing the parameters (weights) which are real numbers f(a) = 1 / (1 + exp(-a)) and f(A), A is a matrix of a_ij, is the new matrix AA whose each element aa_ij is equal to f(a_ij). and `x' is the cross product. If all the stuff regarding the neural network are a bit unclear please ignore them and just tell me how to obtain an expression for the derivative of a function of your choice, i will try and work from there. for example you might tell me how to obtain the derivative of f(x) = a / (b + c*exp(-x)) when a, b and c are general parameters (not instantiated to a specific value). Also, if mathematica can not do that, could you suggest some other method to do it, other than by hand? thank you very much, (please use email if possible) -- Andreas Hadjiprocopis livantes@soi.city.ac.uk Computer Science Department http://www.soi.city.ac.uk/~livantes/home.html Room A528, City University +44 71 477 8551 (telephone) London, UK, EC1V 0HB +44 71 477 8587 (fax)