Services & Resources / Wolfram Forums
-----
 /
MathGroup Archive
2004
*January
*February
*March
*April
*May
*June
*July
*August
*September
*October
*November
*December
*Archive Index
*Ask about this page
*Print this page
*Give us feedback
*Sign up for the Wolfram Insider

MathGroup Archive 2004

[Date Index] [Thread Index] [Author Index]

Search the Archive

Re: General question on handling very large datasets in Mathematica

  • To: mathgroup at smc.vnet.net
  • Subject: [mg48844] Re: General question on handling very large datasets in Mathematica
  • From: Jens-Peer Kuska <kuska at informatik.uni-leipzig.de>
  • Date: Sat, 19 Jun 2004 04:30:44 -0400 (EDT)
  • Organization: Universitaet Leipzig
  • References: <cau1o5$akh$1@smc.vnet.net>
  • Reply-to: kuska at informatik.uni-leipzig.de
  • Sender: owner-wri-mathgroup at wolfram.com

Hi,

I have only large images with up to 512 MByte data.
You should use the 64 versions of Mathematica because
a 64 operating system is not limited to 4 GByte memory.
and you should try to avoid to copy the data to often
when you call your functions (Hold attribute)

Regards
  Jens

Mark Coleman wrote:
> 
> Greetings,
> 
> I am about to begin some research work with a very large dataset --
> several million records (though each record is relatively simple). I
> plan to use Mathematica to estimate a number of statistical models using
> various subsets of this data. I was wondering if other MathGroup have
> any experiences they could share using Mathematica to handle large datasets. My
> particular concern is Import-Export and general reading of large data
> sets in Mathematica. I can have this data in a variety of formats, from pure
> ASCII to any number of RDBS formats.
> 
> I apologize for the vagueness of this posting.
> 
> Regards,
> 
> Mark


  • Prev by Date: AW: Simplifying a large matrix into an equation using M
  • Next by Date: Net/Link: Problem with DLL (1)
  • Previous by thread: General question on handling very large datasets in Mathematica
  • Next by thread: Announcement: Calculus with Mathematica