RunEncode
- To: mathgroup at christensen.Cybernetics.NET
- Subject: [mg214] RunEncode
- From: rrigon at zeus.tamu.edu (Riccardo Rigon)
- Date: Tue, 22 Nov 1994 16:43:32 +0100
This is the result of my inquiry about the RunEncode problem. Thank you for
the collaboration.:
>> I have a matrix of 258x258 integer numbers and I want to count how many times
>> the same number occurs. TO perform this task I have used so far
>> a combination of Sort and the RunEncode routine
>> ( D. Jacobson - The Mathematica J., vol 1, Issue 4, pg. 26 ) which showed
>> to be very efficient.
>> Let A be the matrix. Then the Mathematica command work this way:
>>
>> B=RunEncode[Sort[Flatten[A]]];
>>
>> (*
>> RunEncode[x_,wht_:0]:=
>> Module[{diffpositions=Flatten[Append[
>> Position[Drop[x,1]-Drop[x,-1],_?(#!=wht&)],
>> {Length[x]}]]},
>> Transpose[{Map[x[[#]]&,diffpositions],
>> diffpositions-Prepend[Drop[diffpositions,-1],0]}]];
>> *)
>>
>> Unfortunately with such a big matrix something goes wrong. In particular
>> some of the length become negative. Why ? In some other cases
>> an error message containing the misterious keyword "Toobig" appears.
>>
>> Does anybody knows what is happening?
>
FIRST ANSWER FROM DAVID JACOBSON HIMSELF:
>It's a bug in Mathematica. Drop[xxx,-1] does not seem to work
>correctly when xxx is very large.
>
>Here is an edited transscript that demonstrates the problem.
>
>In[3]:= a = Table[Random[Integer,{1,10}],{256},{256}];
>
>In[5]:= afs = Sort[Flatten[a]];
>
>In[19]:= Drop[afs,1]//Short
>
>Out[19]//Short=
>
>> {1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, <<65516>>, 10, 10, 10, 10, 10, 10}
>
>In[20]:= Drop[afs,-1]//Short
>
>Out[20]//Short= {}
>
LATER THE PROBLEM WITH Drop WAS FOUND NOT SO REPRODUCIBLE BUT THE BUG
INDEED EXISTS.
DEFINIG BETTER THE PROBLEM:
>
In[10]:= bear=Table[Random[Integer,{1,10}],{258},{258}];
In[11]:= yogi=Sort[Flatten[bear]];
In[12]:=RunEncode[yogi]
Drop::drop:
Cannot drop positions 1 through 6545 in
{6545, 13220, 19954, 26654, <<4>>, 59862, 66564}.
Drop::argrx:
Drop called with 3 arguments; 2 arguments are expected.
Out[12]=
{{1, 6545 + -Drop[0, {6545, 13220, 19954, 26654, 33277,
39920, 46556, 53159, 59862, 66564}, 6545]},
{2, 13220 + -Drop[0, {6545, 13220, 19954, 26654, 33277,
.....
In[13]:=RunEncode[yogi]
Thread::tdlen:
Objects of unequal length in TooBig cannot be combined.
----------------------------------------------------------------------------
THE ANSWER FROM THE WRI SUPPORT
(*
Alan DeGuzman
Technical Support
Wolfram Research, Inc.
*)
.....
I have been able to reproduce your problem. It seems as though Drop[] breaks
after an evaluatation of Drop[x,-1] (where x is a long list).
---------------------------------------------------------------------------
In[1]:= Short[ yogi = Sort[Table[Random[Integer,{1,10}],{66564}]] , 1]
Out[1]//Short= {1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, <<66545>>, 10, 10, 10,
> 10, 10, 10}
In[2]:= diffpositions = Flatten[Append[ Position[Drop[yogi,1] -
Drop[yogi,-1],_?(#!=0&)],{Length[yogi]}]]
Out[2]= {6597, 13266, 19952, 26613, 33320, 39902, 46594, 53291, 59890, 66564}
---------------------------------------------------------------------------
Now we try to drop the last element.
---------------------------------------------------------------------------
In[3]:= Drop[diffpositions,-1]
Drop::drop: Cannot drop positions 1 through 6597 in
{6597, 13266, 19952, 26613, 33320, 39902, <<1>>, 53291, 59890, 66564}.
Out[3]= Drop[{6597, 13266, 19952, 26613, 33320, 39902, 46594, 53291, 59890,
> 66564}, 6597]
---------------------------------------------------------------------------
Note that the -1 parsed into the first element of the list, namely 6597.
The value 6597 is held for the Drop[_,-1] of any list.
---------------------------------------------------------------------------
In[4]:= Drop[{1,2,3,4,5},-1]
Drop::drop: Cannot drop positions 1 through 6597 in {1, 2, 3, 4, 5}.
Out[4]= Drop[{1, 2, 3, 4, 5}, 6597]
---------------------------------------------------------------------------
Dropping the last two elements still work.
---------------------------------------------------------------------------
In[5]:= Drop[{1,2,3,4,5},-2]
Out[5]= {1, 2, 3}
---------------------------------------------------------------------------
I have forwarded this problem to our developers as bug report. Thank you
for taking the time to bring this problem to our attention.
I tried to develope a workaround by redefining Drop[_,-1] using Take[],
but I ran into the same problem.
---------------------------------------------------------------------------
In[1]:= Unprotect[Drop]
Out[1]= {Drop}
In[2]:= Drop[l_List,-1] := Module[{len = Length[l]},Take[l,len-1]]
In[3]:= yogi = Sort[Table[Random[Integer,{1,10}],{66564}]] ;
In[4]:= diffpositions = Flatten[Append[ Position[Drop[yogi,1] -
Drop[yogi,-1],_?(#!=0&)],{Length[yogi]}]]
Out[4]= {6500, 13094, 19756, 26347, 33037, 39735, 46421, 53180, 59924, 66564}
In[5]:= Drop[diffpositions,-1]
Take::take: Cannot take positions 1 through 6510 in
{6500, 13094, 19756, 26347, 33037, 39735, <<1>>, 53180, 59924, 66564}.
Out[5]= Take[{6500, 13094, 19756, 26347, 33037, 39735, 46421, 53180, 59924,
> 66564}, 6510]
---------------------------------------------------------------------------
But I finally figured out that using Reverse[] and Rest[] seems to work
(although since it must Reverse[] the list twice, it will be slower).
Use this redefinition of Drop[], and your code should work okay.
---------------------------------------------------------------------------
In[1]:= Unprotect[Drop]
Out[1]= {Drop}
In[2]:= Drop[l_List,-1] := Reverse[Rest[Reverse[l]]]
In[3]:= Drop[{1,2,3,4,5},-1]
Out[3]= {1, 2, 3, 4}
In[4]:= yogi = Sort[Table[Random[Integer,{1,10}],{66564}]] ;
In[5]:= diffpositions = Flatten[Append[ Position[Drop[yogi,1] -
Drop[yogi,-1],_?(#!=0&)],{Length[yogi]}]]
Out[5]= {6695, 13350, 20105, 26771, 33441, 40110, 46817, 53417, 59962, 66564}
In[6]:= Drop[diffpositions,-1]
Out[6]= {6695, 13350, 20105, 26771, 33441, 40110, 46817, 53417, 59962}
---------------------------------------------------------------------------
I had also a mail from
Stig.F.Mjolsnes at delab.sintef.no
who suggested to use Count to go around the problem. (This system is
indeed a lot slower than using RunEncode:
A = Table[Random[Integer,{0,2}],{10},{10}]
....
% //Flatten
Table[Count[%4,i],{i,0,2}]
FURTHERMORE:
No RunEncode used here, but if you want to do that, there is a
very elegant one-liner described in Ilan Vardi: Computational
Recreations page 5.
THA's ALL
------------------------------------------------------------------------------
Riccardo Rigon, Ph. D.
Civil Egineering Department
Texas A&M University
College Station, TX 77843
ph: (409)-845-0335
fax: (409)-862-1542
e-mail: rrigon at zeus.tamu.edu