Services & ResourcesWolfram ForumsMathGroup Archive

[Date Index] [Thread Index] [Author Index]

Re: Re: Sorting a list of pairs on the second elements

• To: mathgroup at smc.vnet.net
• Subject: [mg51425] Re: Re: Sorting a list of pairs on the second elements
• From: "Peltio" <peltio at twilight.zone>
• Date: Sun, 17 Oct 2004 03:05:10 -0400 (EDT)
• References: <cklmi7$f1o$1@smc.vnet.net> <200410150647.CAA05265@smc.vnet.net> <ckqmu1$nf5$1@smc.vnet.net>
• Reply-to: "Peltio" <peltioNOSP at Mdespammed.com.invalid>
• Sender: owner-wri-mathgroup at wolfram.com

"János" wrote

[snip]
    {tgaaaattttccggttt,13311}
[snip]

>Maybe because they are Strings ?  I am just guessing...

I think you guessed right. I did not read your original post with due
attention. You clearly stated that the elements were strings, but I took
your data sample as 'real' Mathematica code and, since I did not see any quotes
to delimit those strings, I surmised they were symbols and numbers.
With String the ordering function using OrderedQ posted by other people
should do the work, but I fear it would be very slow when compared with
     Transpose@Reverse@Transpose@Sort@Transpose@Reverse@Transpose[ data ]

Perhaps coding the second element of your list as Integer could speed
things up  a bit (moreover, such a conversion would allow you to exploit
Ted Ersek's trick).

> the data I had to sort got so big that I even did not
>get to that point to able to sort because the Kernel just quitted on me
>half way.

You mean that you are not even able to Sort your raw data?
Or that the whole swapping+sorting+swapping procedure is too much a burden
for your machine?

In the latter case, I wonder if clearing variables at intermediate steps
could help you somehow.  The above approach is quite fast but could be
building a lot of intermediate data. (I suppose that a oneliner keeps
every intermediate result in memory before passing it to the next
wrapper...) I won't be able to test this until next week, but can
this "intermediate clearing" reduce the amount of memory swallowed
by Mathematica?

    MemoryInUse[]
    temp = Transpose[collectedDnaBin]; Clear[collectedDnaBin];
    MemoryInUse[]
    temp1 = Reverse[temp]; Clear[temp];
    MemoryInUse[]
    temp = Transpose[temp1]; Clear[temp1];
    MemoryInUse[]
    temp1 = Sort[temp]; Clear[temp];
    MemoryInUse[]
    temp = Transpose[temp1]; Clear[temp1];
    MemoryInUse[]
    temp1 = Reverse[temp]; Clear[temp];
    MemoryInUse[]
    sortedDnaBin = Transpose[temp1]; Clear[temp1];
    MemoryInUse[]

If clearing a variable frees the memory previously allocated for it, this
step-by-step approach should ask less memory than the oneliner.

If clearing a variable does not free any memory (is there a command
that frees allocated memory, or is this a strictly OS task?) you might
want to quit the kernel between every single step, saving the data to
a file that will be read in for the subsequent step.
If killing the kernel won't free the memory previously allocated
(it would not surprise me if certain OSes wouldn't free any memory at all)
you could try by killing the kernel, the front end, turning off the power,
unplugging the machine from the outlet and only then trying the next step.

Obviously this would give you benefits only if your system is able to
sort data that big without quitting the kernel. If that is not the case,
well, you might want to consider writing a C routine to be interfaced
to Mathematica via Mathlink.
Sorry I couldnt' help more.

>> Invalid address in reply-to. Crafty demunging required to mail me.
>
>I tried to employ Mathematica in "Crafty demunging" but I failed, so I
>send it "as is".  Now I embrace for an "Addressee not found..."
>thunder... :)

Munging is *very powerful* in keeping spam away from my mailbox.
Unfortunately it kills a lot of the privately sent answers I could get from
newsgroups. My munged address peltioNOSP at Mdespammed.com.invalid requires
capital letters and ".invalid" suffix to be removed in order to turn into a
valid address. A Mathematica routine to demung it should read something like...

    demung[ address_String ] :=
      FromCharacterCode[ Drop[ DeleteCases[ Flatten[
        ToCharacterCode[ Characters[ address]]],_?CapitalQ], -8]]
    CapitalQ[ code_Integer ] := (64 < code < 91)

: )
cheers,
Peltio
Invalid address etc. etc.

• References:
  • Re: Sorting a list of pairs on the second elements
    • From: "Peltio" <peltio@twilight.zone>