Re: Is it possible to create a single csv file one row at a time using
- To: mathgroup at smc.vnet.net
- Subject: [mg103554] Re: Is it possible to create a single csv file one row at a time using
- From: Bill Rowe <readnews at sbcglobal.net>
- Date: Sun, 27 Sep 2009 23:06:38 -0400 (EDT)
On 9/27/09 at 7:30 AM, phineas.q.butterfats at gmail.com (Phineas Q. Butterfats) wrote: >I'm running an algorithm that outputs a list of numbers each time it >runs. I need to collect this data so I can analyze it in >Mathematica, but I have to run my algorithm about 14 million >times. So far I've been creating tables with 100,000 rows (one >for each run >of the algorithm) and then exporting them to csv files. My question >is: can I create a csv file one row at a time? Is there a way to >use Export (or something else) so that the file isn't overwritten >each time I run my algorithm, but instead builds a single csv file >row by row? It's less taxing on my system if I can create the csv >file "on the fly," as opposed to creating such big tables first, and >then exporting them. Of course, I could just have 14 million csv >files, each with one row, but I'd prefer to group them 100,000 at a >time to begin with. The following idea should work for you. First, use OpenWrite to open a file for collecting the data. Use ExportString to convert the output of your algorithm to a CSV string. Then use Write to write the resulting string to the output stream returned by OpenWrite. Finally, simply close all the file to finish up.