Re: Obtaining Random LIne from A file
- To: mathgroup at smc.vnet.net
- Subject: [mg129873] Re: Obtaining Random LIne from A file
- From: Bill Rowe <readnews at sbcglobal.net>
- Date: Wed, 20 Feb 2013 22:26:29 -0500 (EST)
- Delivered-to: l-mathgroup@mail-archive0.wolfram.com
- Delivered-to: l-mathgroup@wolfram.com
- Delivered-to: mathgroup-newout@smc.vnet.net
- Delivered-to: mathgroup-newsend@smc.vnet.net
On 2/19/13 at 1:00 AM, ramiro.barrantes at gmail.com (Ramiro) wrote: >Thank you so much for the reply. My files are 50MB each, I don't >think ReadList would work for my purposes, it would be too slow. Have you tried and found ReadList to be too slow? If you haven't you should try it. You might well find it is faster than you think. Specifically, I was using Mathematica version 7 running on OS X 10.5 using a MacBook Pro with 8GB of installed memory and a 2.53 GHz Intel Core Duo processor to analyze files consisting of 12 channels of 32 bit data sampled at 200KSPS over a typical sampling period of 5 sec. If you do the math, you will see these were 48MB data files. Each test run would consist of 10-20 such files. I was able to complete the analysis needed in 1-2 hours. The time was not spent waiting for my MacBook Pro to read the data. Instead, it was spent reviewing plots etc and adjusting analysis steps based on the plots and other results. The analysis involved a rather complex non-linear fitting routine which could not be relied on to find an optimal solution without some tweaking.