Re: Newbie Question
- To: mathgroup at smc.vnet.net
- Subject: [mg32728] Re: [mg32686] Newbie Question
- From: Sseziwa Mukasa <mukasa at jeol.com>
- Date: Thu, 7 Feb 2002 05:10:27 -0500 (EST)
- Organization: JEOL (USA) Ltd.
- References: <200202060841.DAA02223@smc.vnet.net>
- Sender: owner-wri-mathgroup at wolfram.com
"Brunsman, Kenneth J" wrote:
>
> I can parse out the column I need using Take [data [[All, Column #]]] --- no
> big deal. So far, so good.
>
I'm not sure what you're using Take for, data[[All,Column #]] should be
sufficient.
>
> Now here's the problem --- How do I get % Differences between any of 10,000
> items in that list? I need to take % Differences between adjacent pairs,
> i.e. n and n-1, as well as items n and n-m where m can range from 1 to
> 10,000.
>
Do you want all the differences n-m (m = 1..n-1) at once or just for a
particular m? For an individual m I got good results with
f[x_,m_Integer]:=Block[{a=Drop[x,m]},(a-Drop[x,-m])/a]
On a PowerMac with an 800MHz G4 processor it takes 0.02 seconds to evaluate
f[x,1] where x is a list of length 10000. Table[f[x,i],{i,Length[x]-1}] will
of course give you all the differences for m = 1..n-1.
>
> Further, I need to make sure that this code runs fast because I'm about to
> run this on a data set of 10^8 data points (financial times series). I
> could do this standing on my head in Fortran, but I'm bound and determined
> to learn Mathematica if it takes the rest of my unnatural life.
>
The key difference between Fortran programming and Mathematica in my opinion is
that writing explicit loops in Mathematica is generally inefficient, so is
indexing elements of lists and arrays. Take advantage of the fact that many
operators will automatically apply themselves to every element in a list. Also
avoid making copies as much as possible by using lambda functions and mapping.
Finally, variables are best used to eliminate common subexpressions as in the
Block statement in the function above.
Incidentally though, keeping a list of 10^8 elements in memory is probably not
a good idea, the list should probably be broken up and processed in smaller
pieces to avoid spending all your time swapping memory to disk.
> This is my first time attempting list processing and all I'm doing is
> screwing up royally.
>
Functional programming and pattern matching are not generally practiced outside
of academic programming exercises and they are a very different paradigm from
traditional imperative programming (object oriented techniques being a
different kettle of fish entirely). Practice makes perfect of course. There
are many excellent books that not only teach efficient programming style in
Mathematica but generally do so while applying to techniques to domain specific
problems. I don't do any financial series analysis myself but a quick search
on Amazon.com for "mathematica finance" turns up 6 titles on using Mathematica
for economic and financial modeling.
Regards,
Sseziwa Mukasa
- References:
- Newbie Question
- From: "Brunsman, Kenneth J" <kbrunsman@ou.edu>
- Newbie Question