Re: Dealing with large data sets
- To: mathgroup at smc.vnet.net
- Subject: [mg42994] Re: [mg42961] Dealing with large data sets
- From: William K Reinholtz <kirk.reinholtz at jpl.nasa.gov>
- Date: Thu, 7 Aug 2003 00:53:56 -0400 (EDT)
- Sender: owner-wri-mathgroup at wolfram.com
I'd try a couple of simple things to sort this out. Trick is to NEVER
tempt the front end to print your whole test vector...
1. Kill kernel
2. Reload your data
3. First@testdata to make sure you've really got what you think you've
got.
4. Length@testdata, again make sure it's the big number you expect
5. Take[testdata,10]//TableForm still look right?
6. Mean[Take[testdata,10]] look right?
7. Mean[testdata] should work once whatever went wrong above was fixed
I've found this usually gets to the bottom of whatever stupid mistake
I've made.
On Wednesday, Aug 6, 2003, at 00:16 US/Pacific, Rees wrote:
> Dear all,
>
> I have a large dataset (156381 points) and I wish to fit a
> distribution to the dataset. However I cannot seem to get Mathematica
> to
> recognise the dataset. I imported the data as "testdata" then did
> Mean[testdata]. Unfortunately this command just repeated the data and
> failed to give me the mean value.
>
> I repeated everything using another data format:
> {0.217,0.434,0.652 etc...} with the same response.
>
>
> Any help greatly appreciated.
>
> Rees
>
> DATASET:
>
> 0.217
> 0.434
> 0.652
> 0.217
> 0.869
> 0.268
> 1.303
> 0.433
> 4.118
> 0.561
> 0.18
> 0.359
> 0.539
> 0.18
> 0.718
> 0.139
> 1.077
> 0.848
> 0.696
> 3.976
> 0.549
> 0.509
> 0.125
> 2.498
> 0.625
> 0.213
> 0.427
> 0.64
> 0.213
> 0.854
> 0.229
> 1.28
> 1.273
> 0.392
> 1.28
> 0.764
> 0.196
> 3.922
> 0.98
> 0.183
> 0.367
> 0.55
> 0.183
> 0.734
> 0.157
> 1.101
> 1.003
> 0.602
> 3.007
> 0.392
> 0.602
> 0.154
> 3.084
> 0.771
> data continues down 156381st row
>