[solved] Re: FindMinimum v. NMinimize and an external program

*To*: mathgroup at smc.vnet.net*Subject*: [mg125191] [solved] Re: FindMinimum v. NMinimize and an external program*From*: Ian <carroll.ian at gmail.com>*Date*: Mon, 27 Feb 2012 06:41:57 -0500 (EST)*Delivered-to*: l-mathgroup@mail-archive0.wolfram.com*References*: <jia0qc$1c4$1@smc.vnet.net> <jictj6$emj$1@smc.vnet.net>

Thanks to Szabolcs for the guidance on LibraryLink. It does look like the smart way to go. However, I did figure out how to fix my (non-portable) use of pipes. For the record, the problem appears to just be that I was reading and writing for matted data. On Mathematica's side, replace calls using Read and Write with BinaryRead and BinaryWrite. On the C side, use fread and fwrite rather than scanf and printf. Get your data types lined up correctly, and it works as expected. My guess is that the "zero gradient" warning Mathematica gave using formatted data was accurate: not enough precision was included in the formatted strings to reflect the small changes Mathematica was looking for. -Ian On Sunday, February 26, 2012 1:21:42 AM UTC-8, Szabolcs wrote: > On Saturday, 25 February 2012 08:58:20 UTC+2, Ian wrote: > > Can anyone say whether the following behavior is expected and why? I'd like to avoid learning how to use MathLink, but I need to use FindMinimum on a large external calculation and my way is broken. > > > > Say there's a command line program called 'prog' that calculates y = (x-1)^2, reading x from a named pipe called 'in' and writing y to a named pipe called 'out'. Mathematica interacts with prog by reading and writing to the pipes. Like ... > > > > > in = OpenWrite["in"]; > > > f[x_?NumericQ]:=(Write[in, CForm[x]]; First[ReadList["!cat out", Number]]) > > > > Calling NMinimize on f gives the correct answer, but FindMinimum doesn't. Specifically, FindMinimum[f[x], {x, 0}] returns the message FindMinimum:= :fmgz. That is, Mathematica thinks the gradient is zero. > > > > I can't find a reason the two functions should behave differently. Any thoughts? > > > > A couple details for the DIYers. Here's prog: > > #include <stdio.h> > > #include <math.h> > > int main() > > { > > FILE * io; > > int j; > > double x,y; > > > > while((j = scanf("%lf", &x)) != EOF) > > { > > y = pow((x-1.0), 2); > > io = fopen("out", "w"); > > fprintf(io, "%e\n", y); > > fclose(io); > > } > > return; > > } > > > > Run prog with a redirect for stdin, as in > > /prog < in > > Not a direct answer, but I very much recommend learning LibraryLink (not MathLink). Please see the answers to a question I asked about which is the easiest way to integrate Mathematica with C/C++ code: > > http://stackoverflow.com/questions/8140869/minimal-effort-method-for-integrating-c-functions-into-mathematica > > There's a short but complete guide to setting up a LibraryLink program in Arnoud Buzing's answer. Please take a look at it. Your situation will be even easier because both your input and output are scalars. > > Advantages of LibraryLink compared to pipes: > > * fast -- you have direct access to Mathematica's memory (no need for I/O, converting between strings and numbers, slow process launching) > > * more robust (no need to create named pipes---I don't even know how to do that on Windows) > > * cross platform > > I find it's less work and less worry than hackish approaches such as pipes, also it probably won't take you more than 30 minutes to get it working the first time. On Sunday, February 26, 2012 1:21:42 AM UTC-8, Szabolcs wrote: > On Saturday, 25 February 2012 08:58:20 UTC+2, Ian wrote: > > Can anyone say whether the following behavior is expected and why? I'd like to avoid learning how to use MathLink, but I need to use FindMinimum on a large external calculation and my way is broken. > > > > Say there's a command line program called 'prog' that calculates y = (x-1)^2, reading x from a named pipe called 'in' and writing y to a named pipe called 'out'. Mathematica interacts with prog by reading and writing to the pipes. Like ... > > > > > in = OpenWrite["in"]; > > > f[x_?NumericQ]:=(Write[in, CForm[x]]; First[ReadList["!cat out", Number]]) > > > > Calling NMinimize on f gives the correct answer, but FindMinimum doesn't. Specifically, FindMinimum[f[x], {x, 0}] returns the message FindMinimum:= :fmgz. That is, Mathematica thinks the gradient is zero. > > > > I can't find a reason the two functions should behave differently. Any thoughts? > > > > A couple details for the DIYers. Here's prog: > > #include <stdio.h> > > #include <math.h> > > int main() > > { > > FILE * io; > > int j; > > double x,y; > > > > while((j = scanf("%lf", &x)) != EOF) > > { > > y = pow((x-1.0), 2); > > io = fopen("out", "w"); > > fprintf(io, "%e\n", y); > > fclose(io); > > } > > return; > > } > > > > Run prog with a redirect for stdin, as in > > /prog < in > > Not a direct answer, but I very much recommend learning LibraryLink (not MathLink). Please see the answers to a question I asked about which is the easiest way to integrate Mathematica with C/C++ code: > > http://stackoverflow.com/questions/8140869/minimal-effort-method-for-integrating-c-functions-into-mathematica > > There's a short but complete guide to setting up a LibraryLink program in Arnoud Buzing's answer. Please take a look at it. Your situation will be even easier because both your input and output are scalars. > > Advantages of LibraryLink compared to pipes: > > * fast -- you have direct access to Mathematica's memory (no need for I/O, converting between strings and numbers, slow process launching) > > * more robust (no need to create named pipes---I don't even know how to do that on Windows) > > * cross platform > > I find it's less work and less worry than hackish approaches such as pipes, also it probably won't take you more than 30 minutes to get it working the first time. On Sunday, February 26, 2012 1:21:42 AM UTC-8, Szabolcs wrote: > On Saturday, 25 February 2012 08:58:20 UTC+2, Ian wrote: > > Can anyone say whether the following behavior is expected and why? I'd like to avoid learning how to use MathLink, but I need to use FindMinimum on a large external calculation and my way is broken. > > > > Say there's a command line program called 'prog' that calculates y = (x-1)^2, reading x from a named pipe called 'in' and writing y to a named pipe called 'out'. Mathematica interacts with prog by reading and writing to the pipes. Like ... > > > > > in = OpenWrite["in"]; > > > f[x_?NumericQ]:=(Write[in, CForm[x]]; First[ReadList["!cat out", Number]]) > > > > Calling NMinimize on f gives the correct answer, but FindMinimum doesn't. Specifically, FindMinimum[f[x], {x, 0}] returns the message FindMinimum:= :fmgz. That is, Mathematica thinks the gradient is zero. > > > > I can't find a reason the two functions should behave differently. Any thoughts? > > > > A couple details for the DIYers. Here's prog: > > #include <stdio.h> > > #include <math.h> > > int main() > > { > > FILE * io; > > int j; > > double x,y; > > > > while((j = scanf("%lf", &x)) != EOF) > > { > > y = pow((x-1.0), 2); > > io = fopen("out", "w"); > > fprintf(io, "%e\n", y); > > fclose(io); > > } > > return; > > } > > > > Run prog with a redirect for stdin, as in > > /prog < in > > Not a direct answer, but I very much recommend learning LibraryLink (not MathLink). Please see the answers to a question I asked about which is the easiest way to integrate Mathematica with C/C++ code: > > http://stackoverflow.com/questions/8140869/minimal-effort-method-for-integrating-c-functions-into-mathematica > > There's a short but complete guide to setting up a LibraryLink program in Arnoud Buzing's answer. Please take a look at it. Your situation will be even easier because both your input and output are scalars. > > Advantages of LibraryLink compared to pipes: > > * fast -- you have direct access to Mathematica's memory (no need for I/O, converting between strings and numbers, slow process launching) > > * more robust (no need to create named pipes---I don't even know how to do that on Windows) > > * cross platform > > I find it's less work and less worry than hackish approaches such as pipes, also it probably won't take you more than 30 minutes to get it working the first time.