Services & ResourcesWolfram ForumsMathGroup Archive

[Date Index] [Thread Index] [Author Index]

Re: how to ListPlot3D large data sets

• To: mathgroup at smc.vnet.net
• Subject: [mg120555] Re: how to ListPlot3D large data sets
• From: Oliver Ruebenkoenig <ruebenko at wolfram.com>
• Date: Fri, 29 Jul 2011 04:41:10 -0400 (EDT)
• Delivered-to: l-mathgroup@mail-archive0.wolfram.com
• References: <201107281153.HAA04026@smc.vnet.net>

On Thu, 28 Jul 2011, Oliver Ruebenkoenig wrote:

> On Wed, 27 Jul 2011, Ted Sariyski wrote:
>
>> Hi,
>>
>> In one attempt MaxPlotPoints did not help, but I was not persistent
>> because I have to resolve other issues before I return to ListPlot3D.
>> One issue is that I cannot pack data. On the big data set I verify that
>> it is numeric array and still  I cannot get it packed. Here is an example:
>>
>> mydata={{0.00100,10.,0.},{0.00100,10.,0.00100},{0.00100,10.,0.00200},{0.00100,10.,0.00300},{0.00100,10.,0.00400}};
>>
>> MatrixQ[mydata,NumericQ]
>> True
>>
>> myPackedData=ToPackedArray[mydata,Real];
>>
>> MatrixQ[myPackedData, NumericQ]
>> True
>>
>> PackedArrayQ[myPackedData]
>> False
>>
>> What I am doing wrong?
>> Thanks,
>> --Ted
>
>
> Ted, possibly ToPackedArray are not on you context path
>
> either try
>
> <<Developer`
>
> or
>
> mydata = {{0.00100, 10., 0.}, {0.00100, 10., 0.00100}, {0.00100, 10.,
>     0.00200}, {0.00100, 10., 0.00300}, {0.00100, 10., 0.00400}};
> myPackedData = Developer`ToPackedArray[mydata, Real];
> Developer`PackedArrayQ[myPackedData]
>
> Oliver


Daniel Lichtblau actually pointed out to me that I might have 
misunderstood the implications of you result.

Some comments: Entering a list like mydata by "hand" (tying, 
cut&copy&paste) will always be an unpacked list.

You'd have to check PackedArrayQ directly on the import of your large data 
set, assuming that returned False, you could wrap a 
N[yourLargeDataImport] around it and see if it then is packed. Should that 
still not be the case, there might be non numerical stuff in your imported 
list and you could give it a shot at removing that and the see if and that 
affects performance for you plot.

I hope this helps in a better way.

Oliver



>
>>
>> On 7/25/2011 7:30 AM, Oliver Ruebenkoenig wrote:
>>> On Sat, 23 Jul 2011, Ted Sariyski wrote:
>>>
>>>> Hi,
>>>> I guess I am pushing the limits of ListPlot3D. I try to ListPlot3d a data
>>>> set with many millions of records. I was not able to get an image from the
>>>> full dataset, it takes forever. If I use e.g. every fifth record, although
>>>> slow, I get an image. The machine running Mathematica is Windows 7 (64 bit),
>>>> has 24 GB RAM and there was no swapping. I wonder what is considered as a
>>>> reasonable data size for ListPlot3D and are there other tools in Mathematica
>>>> for visualization of large data sets?
>>>> Thanks in advance,
>>>> --Ted
>>>>
>>>>
>>>>
>>> Ted,
>>>
>>> is your data packed?
>>>
>>> Developer`PackedArrayQ[yourData]
>>>
>>> Oliver
>>>
>>
>>
>>
>
>

• References:
  • Re: how to ListPlot3D large data sets
    • From: Oliver Ruebenkoenig <ruebenko@wolfram.com>