Re: negative FileByteCount
- To: mathgroup at smc.vnet.net
- Subject: [mg83201] Re: negative FileByteCount
- From: Jean-Marc Gulliet <jeanmarc.gulliet at gmail.com>
- Date: Thu, 15 Nov 2007 05:30:07 -0500 (EST)
- Organization: The Open University, Milton Keynes, UK
- References: <fh1c35$boh$1@smc.vnet.net><fh3rfa$115$1@smc.vnet.net> <fheg8a$lvg$1@smc.vnet.net>
[My reply is at the bottom of this email.] rych wrote: > Thanks for response > Mathematica 6.0.1.0 > Windows XP Pro SP2 32 bit > 2GB of RAM, Commit charge: 2113/5977M. > Perhaps, I should increase the page file, say to 2x3GB? It's if the > FileByteCount actually loads the whole file into memory to find out > about its size. I'll report later whether it worked. Ultimately, I'd > like to know the size limit and how to load such big files by parts. > Igor > > On 10 Nov, 08:51, Jean-Marc Gulliet <jeanmarc.gull... at gmail.com> > wrote: >> rych wrote: >>> Is there a maximum file size permitted in Mathematica? Because it >>> seems I can't work with my 3GB file, >> <snip> >> >> It all depends on what version of Mathematica you are using (32 or 64 >> bits), your operating system, the available memory you have when >> importing the file, possibly the type of data imported... >> >> Thus the following questions: What version of Mathematica do you use? >> What platform (hardware and operating system do you use? How much memory >> is available for Mathematica (or are you running many applications >> simultaneously or just Mathematica)? Summary: if Mathematica, or any other applications, must load 3GB of data into memory, this is doomed to failed on a 32-bit system. The address space of any 32-bit system, regardless of its operating system (Windows, Linux, MacOs, Solaris, UNIX, etc.) or chip maker (Intel, AMD, PowerPC, etc.), is limited to 4GB (i.e. the smallest address is 0 and the largest is (2^32)-1 = 4,294,967,295). This limit is due to the size of the address bus (32-bit width), bus that allows the exchange of data between the processor and the main memory. On Windows, the main memory is managed as virtual memory, i.e. the main memory is made of physical ram and a page file. Note that the addressable virtual memory cannot exceeds 4GB even though the page file may be created as large as you wish. In addition of this limitation to a maximum of 4GB of addressable memory, the address space is variously divided between the operating system and the user(s). Still on Windows XP, the kernel (system memory) gets 2GB and the applications get the other 2GB. (It is possible on certain circumstances to increase this ratio in favor of the user applications up to 3GB or so.) Also, applications running concurrently must share this 2GB space and some applications (or at least specific functions of these applications) may require continuous block of memory (similarly to a hard disk, the main memory will become fragmented during a work session). Conclusion: if Mathematica (or any other applications) must load 3GB of data into memory, this is doomed to failed on a 32-bit system. The following pages may be particularly worth reading: "Windows XP Professional x64 Edition, From Wikipedia, the free encyclopedia" http://en.wikipedia.org/wiki/Windows_XP_Professional_x64_Edition particularly the first section is of uttermost importance regarding memory differences between 32- and 64-bit architectures. "RAM, Virtual Memory, Pagefile and all that stuff" http://support.microsoft.com/default.aspx/kb/555223 Regards, -- Jean-Marc
- Follow-Ups:
- Re: Re: negative FileByteCount
- From: "Jean-Marc Gulliet" <jeanmarc.gulliet@gmail.com>
- Re: Re: negative FileByteCount