RunEncode

• To: mathgroup at christensen.Cybernetics.NET
• Subject: [mg214] RunEncode
• From: rrigon at zeus.tamu.edu (Riccardo Rigon)
• Date: Tue, 22 Nov 1994 16:43:32 +0100

```This is the result of my inquiry about the RunEncode problem. Thank you for
the collaboration.:

>> I have a matrix of 258x258 integer numbers and I want to count how many times
>> the same number occurs. TO perform this task I have used so far
>> a combination of Sort and the RunEncode routine
>> ( D. Jacobson - The Mathematica J., vol 1, Issue 4, pg. 26 ) which showed
>> to be very efficient.
>> Let A be the matrix. Then the Mathematica command work this way:
>>
>> B=RunEncode[Sort[Flatten[A]]];
>>
>> (*
>> RunEncode[x_,wht_:0]:=
>> Module[{diffpositions=Flatten[Append[
>> Position[Drop[x,1]-Drop[x,-1],_?(#!=wht&)],
>> {Length[x]}]]},
>> Transpose[{Map[x[[#]]&,diffpositions],
>> diffpositions-Prepend[Drop[diffpositions,-1],0]}]];
>> *)
>>
>> Unfortunately with such a big matrix something goes wrong. In particular
>> some of the length become negative. Why ? In some other cases
>> an error message  containing the misterious keyword "Toobig" appears.
>>
>> Does anybody knows what is happening?
>

FIRST ANSWER FROM DAVID JACOBSON HIMSELF:

>It's a bug in Mathematica.  Drop[xxx,-1] does not seem to work
>correctly when xxx is very large.
>
>Here is an edited transscript that demonstrates the problem.
>
>In[3]:= a = Table[Random[Integer,{1,10}],{256},{256}];
>
>In[5]:= afs = Sort[Flatten[a]];
>
>In[19]:= Drop[afs,1]//Short
>
>Out[19]//Short=
>
>>   {1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, <<65516>>, 10, 10, 10, 10, 10, 10}
>
>In[20]:= Drop[afs,-1]//Short
>
>Out[20]//Short= {}
>

LATER THE PROBLEM WITH Drop WAS FOUND NOT SO REPRODUCIBLE BUT THE BUG
INDEED EXISTS.
DEFINIG BETTER THE PROBLEM:

>
In[10]:= bear=Table[Random[Integer,{1,10}],{258},{258}];
In[11]:= yogi=Sort[Flatten[bear]];

In[12]:=RunEncode[yogi]

Drop::drop:
Cannot drop positions 1 through 6545 in
{6545, 13220, 19954, 26654, <<4>>, 59862, 66564}.
Drop::argrx:
Drop called with 3 arguments; 2 arguments are expected.

Out[12]=
{{1, 6545 + -Drop[0, {6545, 13220, 19954, 26654, 33277,

39920, 46556, 53159, 59862, 66564}, 6545]},

{2, 13220 + -Drop[0, {6545, 13220, 19954, 26654, 33277,

.....

In[13]:=RunEncode[yogi]
Thread::tdlen:
Objects of unequal length in TooBig cannot be combined.

----------------------------------------------------------------------------
THE ANSWER FROM THE WRI SUPPORT
(*
Alan DeGuzman
Technical Support
Wolfram Research, Inc.
*)

.....

I have been able to reproduce your problem. It seems as though Drop[] breaks
after an evaluatation of Drop[x,-1] (where x is a long list).

---------------------------------------------------------------------------
In[1]:= Short[ yogi = Sort[Table[Random[Integer,{1,10}],{66564}]] , 1]

Out[1]//Short= {1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, <<66545>>, 10, 10, 10,

>    10, 10, 10}

In[2]:= diffpositions = Flatten[Append[ Position[Drop[yogi,1] -
Drop[yogi,-1],_?(#!=0&)],{Length[yogi]}]]

Out[2]= {6597, 13266, 19952, 26613, 33320, 39902, 46594, 53291, 59890, 66564}
---------------------------------------------------------------------------

Now we try to drop the last element.

---------------------------------------------------------------------------
In[3]:= Drop[diffpositions,-1]

Drop::drop: Cannot drop positions 1 through 6597 in
{6597, 13266, 19952, 26613, 33320, 39902, <<1>>, 53291, 59890, 66564}.

Out[3]= Drop[{6597, 13266, 19952, 26613, 33320, 39902, 46594, 53291, 59890,

>     66564}, 6597]
---------------------------------------------------------------------------

Note that the -1 parsed into the first element of the list, namely 6597.
The value 6597 is held for the Drop[_,-1] of any list.

---------------------------------------------------------------------------
In[4]:= Drop[{1,2,3,4,5},-1]

Drop::drop: Cannot drop positions 1 through 6597 in {1, 2, 3, 4, 5}.

Out[4]= Drop[{1, 2, 3, 4, 5}, 6597]
---------------------------------------------------------------------------

Dropping the last two elements still work.

---------------------------------------------------------------------------
In[5]:= Drop[{1,2,3,4,5},-2]

Out[5]= {1, 2, 3}
---------------------------------------------------------------------------

I have forwarded this problem to our developers as bug report. Thank you
for taking the time to bring this problem to our attention.

I tried to develope a workaround by redefining Drop[_,-1] using Take[],
but I ran into the same problem.

---------------------------------------------------------------------------
In[1]:= Unprotect[Drop]

Out[1]= {Drop}

In[2]:= Drop[l_List,-1] := Module[{len = Length[l]},Take[l,len-1]]

In[3]:= yogi = Sort[Table[Random[Integer,{1,10}],{66564}]] ;

In[4]:= diffpositions = Flatten[Append[ Position[Drop[yogi,1] -
Drop[yogi,-1],_?(#!=0&)],{Length[yogi]}]]

Out[4]= {6500, 13094, 19756, 26347, 33037, 39735, 46421, 53180, 59924, 66564}

In[5]:= Drop[diffpositions,-1]

Take::take: Cannot take positions 1 through 6510 in
{6500, 13094, 19756, 26347, 33037, 39735, <<1>>, 53180, 59924, 66564}.

Out[5]= Take[{6500, 13094, 19756, 26347, 33037, 39735, 46421, 53180, 59924,

>     66564}, 6510]
---------------------------------------------------------------------------

But I finally figured out that using Reverse[] and Rest[] seems to work
(although since it must Reverse[] the list twice, it will be slower).
Use this redefinition of Drop[], and your code should work okay.

---------------------------------------------------------------------------
In[1]:= Unprotect[Drop]

Out[1]= {Drop}

In[2]:= Drop[l_List,-1] := Reverse[Rest[Reverse[l]]]

In[3]:= Drop[{1,2,3,4,5},-1]

Out[3]= {1, 2, 3, 4}

In[4]:= yogi = Sort[Table[Random[Integer,{1,10}],{66564}]] ;

In[5]:= diffpositions = Flatten[Append[ Position[Drop[yogi,1] -
Drop[yogi,-1],_?(#!=0&)],{Length[yogi]}]]

Out[5]= {6695, 13350, 20105, 26771, 33441, 40110, 46817, 53417, 59962, 66564}

In[6]:= Drop[diffpositions,-1]

Out[6]= {6695, 13350, 20105, 26771, 33441, 40110, 46817, 53417, 59962}
---------------------------------------------------------------------------
I had also a mail from

Stig.F.Mjolsnes at delab.sintef.no

who suggested to use Count  to go around the problem. (This system is
indeed a lot slower than using RunEncode:

A = Table[Random[Integer,{0,2}],{10},{10}]

....

%  //Flatten

Table[Count[%4,i],{i,0,2}]

FURTHERMORE:

No RunEncode used here, but if you want to do that, there is a
very elegant one-liner described in Ilan Vardi: Computational
Recreations page 5.

THA's ALL
------------------------------------------------------------------------------

Riccardo Rigon, Ph. D.

Civil Egineering Department
Texas A&M University
College Station, TX 77843
ph:  (409)-845-0335
fax: (409)-862-1542
e-mail: rrigon at zeus.tamu.edu

```

• Prev by Date: [no subject]
• Next by Date: ListContourPlot for irregular sampling
• Previous by thread: [no subject]
• Next by thread: ListContourPlot for irregular sampling