[Date Index]
[Thread Index]
[Author Index]
Re: Importing large file into table and calculating takes a long time. How to improve efficiency?
*To*: mathgroup at smc.vnet.net
*Subject*: [mg126333] Re: Importing large file into table and calculating takes a long time. How to improve efficiency?
*From*: Šerých Jakub <Serych at panska.cz>
*Date*: Tue, 1 May 2012 14:58:14 -0400 (EDT)
*Delivered-to*: l-mathgroup@mail-archive0.wolfram.com
I haven't studied your code, but definitely the first error I can see is, that normal asignment is "=" not "==" in Mathematica.
Jakub
> -----Original Message-----
> From: Gangerolv [mailto:ipschka at yahoo.com]
> Sent: Tuesday, May 01, 2012 11:24 AM
> To: mathgroup at smc.vnet.net
> Subject: Importing large file into table and calculating takes a
> long time. How to improve efficiency?
>
> (First my disclaimer, I'm new to mathematica)
>
> I'm importing a file with three values, x,y,z in hexadecimal.
> Sample of the input data: {0., 24, "009d"}, {0., 28, 9}, {0., 28, 99}, {"00dc",
> 27, 98}, {0., 29, 95},...
>
> This set is converted to integer: {0., 36, 157}, {0., 40, 9}, {0., 40, 153}, {220,
> 39, 152}, {0., 41, 149},...
>
> A final vector is calculated using: g==sqrt(x^2 + y^2 + z^2) for each data set.
>
> The data is then plotted.
>
> All of this is good an works great for a small file (2000 data sets). But when I
> try to import and calculate a larger file (over 100k data sets), it seems to take
> forever. Either my methods are not efficient (use of Table), or I'm not using
> correct settings for importing of the data. The file is only 2Mb so I know
> mathematica should be able to handle it.
>
> Here's what I'm doing:
>
> =========================================================================
> dataHex ==
> Import["C:\\Projects\\Mathematica\\test.csv"]
> points == Length[dataHex]
> dataDec == Table[
> ToExpression["16^^" <> #] & /@ {
> ToString[dataHex[[i, 1]]],
> ToString[dataHex[[i, 2]]],
> ToString[dataHex[[i, 3]]]},
> {i, 1, points}]
> dataDecCompl == Table[{
> If[dataDec[[i, 1]] > 32768, dataDec[[i, 1]] - 65536, dataDec[[i, 1]]],
> If[dataDec[[i, 2]] > 32768, dataDec[[i, 2]] - 65536, dataDec[[i, 2]]],
> If[dataDec[[i, 3]] > 32768, dataDec[[i, 2]] - 65536, dataDec[[i, 3]]]},
> {i, 1, points}];
>
> dataG == MovingAverage[Table[
> Sqrt[dataX[[i]]^2 + dataY[[i]]^2 + dataZ[[i]]^2] // N,
> {i, 1, points - filter}],
> 3];
> plotG == ListLinePlot[{dataG}, PlotRange -> All];
> =========================================================================
>
> Any suggestions would be highly appreciated. Using M 8.0.4 on Win7.
>
> Cheers!
>
> jan
>
Prev by Date:
**Re: Struggling to prove simple triangle inequality**
Next by Date:
**Re: Importing large file into table and calculating takes a long time. How to improve efficiency?**
Previous by thread:
**Re: Importing large file into table and calculating takes a long time. How to improve efficiency?**
Next by thread:
**Re: Importing large file into table and calculating takes a long time. How to improve efficiency?**
| |