Re: General question on handling very large datasets in Mathematica
- To: mathgroup at smc.vnet.net
- Subject: [mg48844] Re: General question on handling very large datasets in Mathematica
- From: Jens-Peer Kuska <kuska at informatik.uni-leipzig.de>
- Date: Sat, 19 Jun 2004 04:30:44 -0400 (EDT)
- Organization: Universitaet Leipzig
- References: <firstname.lastname@example.org>
- Reply-to: kuska at informatik.uni-leipzig.de
- Sender: owner-wri-mathgroup at wolfram.com
I have only large images with up to 512 MByte data.
You should use the 64 versions of Mathematica because
a 64 operating system is not limited to 4 GByte memory.
and you should try to avoid to copy the data to often
when you call your functions (Hold attribute)
Mark Coleman wrote:
> I am about to begin some research work with a very large dataset --
> several million records (though each record is relatively simple). I
> plan to use Mathematica to estimate a number of statistical models using
> various subsets of this data. I was wondering if other MathGroup have
> any experiences they could share using Mathematica to handle large datasets. My
> particular concern is Import-Export and general reading of large data
> sets in Mathematica. I can have this data in a variety of formats, from pure
> ASCII to any number of RDBS formats.
> I apologize for the vagueness of this posting.
Prev by Date:
AW: Simplifying a large matrix into an equation using M
Next by Date:
Net/Link: Problem with DLL (1)
Previous by thread:
General question on handling very large datasets in Mathematica
Next by thread:
Announcement: Calculus with Mathematica