Fun with MIDI, Music, and J/Link (long)
- To: mathgroup at smc.vnet.net
- Subject: [mg27851] Fun with MIDI, Music, and J/Link (long)
- From: tgayley at wolfram.com (Todd Gayley)
- Date: Thu, 22 Mar 2001 04:29:56 -0500 (EST)
- Organization: Wolfram Research, Inc.
- Sender: owner-wri-mathgroup at wolfram.com
Several messages asking about MIDI have appeared recently. I hope their authors excuse me
for starting a new thread instead of replying directly, but I wanted to catch the
attention of readers who might not have any clear interest in MIDI export.
Alan asks:
>I know what MIDI is and what I am trying to do is convert, for example,
>a list of values containing a frequency to play on the synthesizer and
>the duration for how long to play it. Is there a Mathematica package to
>do so?
>The list may look like:
>{{note,duration},{note,duration},{note,duration}...}
This is just the sort of thing that J/Link is ideal for. If someone has already written a
package, then great, but maybe not. Then what do you do? Write your own (ugh)? Find a C
library for MIDI and MathLink to it? You gotta be kidding! You say to yourself, "I bet
that can be done easily in Java, and therefore I can probably do it in a few lines of
Mathematica." You would be correct.
Java has a standard extension called the Java Sound API, and it handles MIDI. Java Sound
comes standard with the Java 1.3 version, so those with Windows, Linux, Solaris, and some
other UNIX platforms can do this right now. Mac users will have to wait for OS X (and
until Mathematica and J/Link are available for OS X). The Java Sound home page is
http://java.sun.com/products/java-media/sound/index.html, and Windows users will have to
download a soundbank, as the default distribution does not include one. You do not need
any special hardware for this--the typical sound card in a PC can handle it. I don't know
about Sun platforms.
The latest version of J/Link (1.1.2) can be downloaded from
www.wolfram.com/solutions/mathlink/jlink.
Now that everybody is ready, let's look at some code. Note that everything we will see is
Mathematica code. No Java code whatsoever needed to be written for anything I present
here. We use J/Link's ability to let you call Java from Mathematica to do everything in
the Mathematica language.
Let me start off by saying that I don't know anything about MIDI. This is cobbled together
from looking at one small example Java program I found and skimming the Java Sound
programming docs.
Instead of diving right into the program to write MIDI files that Alan wanted, let's see
how easy it is to start playing with MIDI in Mathematica. We start by loading JLink.m and
starting Java:
In[1]:= <<JLink`
In[2]:= InstallJava[];
In[3]:= LoadClass["javax.sound.midi.MidiSystem"];
synth = MidiSystem`getSynthesizer[];
synth@open[];
channel = First[synth@getChannels[]];
If the LoadClass fails, you don't have Java Sound installed. Now we have a channel for
sending commands to. Let's change the instrument from a grand piano (MIDI instrument 1) to
a church organ (instrument 20). The Java code indexes from 0, so we need to subtract 1
from the standard MIDI instrument values:
In[7]:= channel@programChange[19]; (* church organ *)
In[8]:= channel@noteOn[60, 80] (* 60 is middle C, 80 is a velocity parameter (volume). *)
Now you are hearing an organ playing C. Add some more notes.
In[9]:= channel@noteOn[64, 80] (* E *)
In[10]:= channel@noteOn[67, 80] (* G *)
Now you are hearing a major triad. Turn off the music and close the synthesizer:
In[11]:= channel@allNotesOff[]
In[12]:= synth@close[]
What could be easier? You could have a lot of fun dynamically controlling sound from
Mathematica code in this way. At the very end of this message, I present a complete
Mathematica program that displays a keyboard and lets you use the mouse to play it, change
instruments, etc.
Now on to the question about writing MIDI files from pitch and duration data. It is easy
to write functions that write files and play a given MIDI sequence, so we'll first create
a separate function that creates the sequence and we'll use it for both writing and
playing.
CreateSequence[data:{{_Integer, _Integer}..}] :=
Module[{seq, track, msg, time = 0},
InstallJava[];
(* 100 pulses per quarter-note (that is, durations of 100 will last 1/2 sec.) *)
seq = JavaNew["javax.sound.midi.Sequence", Sequence`PPQ, 100];
JavaBlock[
track = seq@createTrack[];
msg = JavaNew["javax.sound.midi.ShortMessage"];
(* This sets the instrument. The second 0 is the instrument slot.
0 is for grand piano, and other numbers will give different sounds.
*)
msg@setMessage[ShortMessage`PROGRAMUCHANGE, 0, 0, 0];
track@add[JavaNew["javax.sound.midi.MidiEvent", msg, 0]];
(* This function applied to the data list creates the notes in the sequence.*)
Function[{pitch, duration},
msg = JavaNew["javax.sound.midi.ShortMessage"];
(* The 80 below specifies the velocity with which the key is struck
(thus it is a volume).
*)
msg@setMessage[ShortMessage`NOTEUON, 0, pitch, 80];
track@add[JavaNew["javax.sound.midi.MidiEvent", msg, time]];
msg = JavaNew["javax.sound.midi.ShortMessage"];
(* The 75 below is a not-so-fast release of the piano key. *)
msg@setMessage[ShortMessage`NOTEUOFF, 0, pitch, 75];
track@add[JavaNew["javax.sound.midi.MidiEvent", msg, time += duration]];
] @@@ data
];
seq
]
This plays sequences (I've made $Sequencer a global variable because we need to manually
close it after the playing is over):
PlaySequence[seq_?JavaObjectQ] :=
(
LoadClass["javax.sound.midi.MidiSystem"];
$Sequencer = MidiSystem`getSequencer[];
$Sequencer@open[];
$Sequencer@setSequence[seq];
$Sequencer@start[];
)
This writes sequences into MIDI files:
WriteSequence[file_String, seq_?JavaObjectQ] :=
JavaBlock[
Module[{supportedFileTypes},
LoadClass["javax.sound.midi.MidiSystem"];
supportedFileTypes = MidiSystem`getMidiFileTypes[seq];
(* The first supported file type is often suitable only for one-track seqs. *)
MidiSystem`write[seq, First[supportedFileTypes],
JavaNew["java.io.File", file]]
]
]
Now we have what Alan wanted. The notes in the data list below are the standard MIDI pitch
sequence, with 60 = middle C. The second number is the duration, with 100 = quarter note =
1/2 sec. This tempo scaling comes from the 100 in the CreateSequence program, and it could
be easily changed.
In[16]:= Cscale = {{60, 100}, {62, 100}, {64, 100}, {65, 100}, {67, 100}, {69, 100},
{71, 100}, {72, 100}};
In[17]:= WriteSequence["test.mid", CreateSequence[Cscale]]
To play it:
In[18]:= PlaySequence[CreateSequence[Cscale]]
Then, close the snythesizer so other applications can use it:
In[19]:= $Sequencer@close[]
I'm sure that readers can imagine lots of interesting ways to algorithmically generate
sound data of this type. It would also be straightforward to write an import function that
read in MIDI files and produced a format like this.
I was quite entertained playing with sound in Mathematica, so I went on and produced a
keyboard program. I will present the code without commentary below, but here is how you
would use it:
In[20]:= Piano[]
This displays the keyboard and starts it running. It will take a few seconds to start up,
mostly because of the time it takes to populate the listbox with all the different
instruments. The Piano[] program will not end until you close the keyboard window. Play
around with all the different instrument sounds. This is a good example of a reasonably
complicated J/Link program that uses many of the facilities that Java provides, although
of course it is written in Mathematica.
--Todd Gayley
Wolfram Research
(****************************** Piano program listing ********************************)
Needs["JLink`"]
Piano[instrument_Integer:1] :=
JavaBlock[
Module[{frm, label1, label2, pianoKeyListener, synth, instMenu, instListener},
InstallJava[];
LoadClass["java.awt.Color"];
frm = JavaNew["com.wolfram.jlink.MathFrame", "Mathematica Keyboard"];
frm@setSize[800, 240];
frm@setLayout[Null];
label1 = JavaNew["java.awt.Label",
"Click to play. Hold Shift key down to drag from"];
label2 = JavaNew["java.awt.Label",
"key to key (shift must be down before clicking)."];
frm@add[label1];
frm@add[label2];
label1@setBounds[20, 25, 300, 15];
label2@setBounds[20, 40, 300, 15];
keyPressListener = JavaNew["com.wolfram.jlink.MathMouseListener"];
keyPressListener@setHandler["mousePressed", "keyPressed"];
keyPressListener@setHandler["mouseReleased", "keyReleased"];
keyPressListener@setHandler["mouseEntered", "keyEntered"];
keyPressListener@setHandler["mouseExited", "keyExited"];
LoadClass["javax.sound.midi.MidiSystem"];
synth = MidiSystem`getSynthesizer[];
synth@open[];
instMenu = JavaNew["java.awt.Choice"];
MapIndexed[instMenu@add[ToString[First[#2]] <> ". " <> #1@getName[]]&,
synth@getAvailableInstruments[]];
frm@add[instMenu];
instMenu@setBounds[340, 30, 160, 20];
instMenu@select[instrument - 1]; (* Convert to 0-based index. *)
instListener = JavaNew["com.wolfram.jlink.MathItemListener", "instChanged"];
instMenu@addItemListener[instListener];
JavaShow[frm];
Map[
createKey[frm, keyPressListener, #]&,
{{"C", 60}, {"C#", 61}, {"D", 62}, {"D#", 63}, {"E", 64}, {"F", 65},
{"F#", 66}, {"G", 67}, {"G#", 68}, {"A", 69}, {"A#", 70}, {"B", 71},
{"C", 72}, {"C#", 73}, {"D", 74}, {"D#", 75}, {"E", 76}, {"F", 77},
{"F#", 78}, {"G", 79}, {"G#", 80}, {"A", 81}, {"A#", 82}, {"B", 83},
{"C", 84}}
];
frm@setModal[];
Block[{$Channel, $IsShiftDown, $IsMouseDown = False},
$Channel = First[synth@getChannels[]];
$Channel@programChange[instrument - 1];
DoModal[]
];
synth@close[];
]
]
$WhiteKeyWidth = 50;
$WhiteKeyHeight = 150;
$BlackKeyWidth = 30;
$BlackKeyHeight = 100;
createKey[frm_, listener_, {noteName_, pitch_}] :=
Module[{key, lastWhiteKey, leftEdge},
key = JavaNew["java.awt.Button", noteName];
key@setName[ToString[pitch]];
key@addMouseListener[listener];
If[StringTake[noteName, -1] != "#",
(* White keys *)
If[frm@getComponentCount[] == 3, (* 3 components exist before first key. *)
leftEdge = 25,
(* else *)
lastWhiteKey = frm@getComponent[frm@getComponentCount[] - 1];
leftEdge = lastWhiteKey@getLocation[]@x + $WhiteKeyWidth
];
frm@add[key];
key@setBounds[leftEdge, 70, $WhiteKeyWidth, $WhiteKeyHeight];
key@setBackground[Color`white],
(* else *)
(* Black keys *)
lastWhiteKey = frm@getComponent[frm@getComponentCount[] - 1];
frm@add[key, 0];
key@setBounds[lastWhiteKey@getLocation[]@x + $WhiteKeyWidth -
$BlackKeyWidth/2, 70, $BlackKeyWidth, $BlackKeyHeight];
key@setBackground[Color`black];
key@setForeground[Color`white]
]
]
restoreKeyColor[comp_] :=
If[StringTake[comp@getLabel[], -1] == "#",
comp@setBackground[Color`black],
(* else *)
comp@setBackground[Color`white]
]
(** These are the event handler callback functions. **)
keyPressed[evt_, _, _, _] :=
JavaBlock[
$Channel@noteOn[ToExpression[evt@getSource[]@getName[]], 80];
$IsMouseDown = True;
$IsShiftDown = evt@isShiftDown[];
evt@getSource[]@setBackground[Color`gray];
ReleaseObject[evt]
]
keyReleased[evt_, _, _, _] :=
JavaBlock[
$Channel@allNotesOff[];
$IsMouseDown = False;
restoreKeyColor[evt@getSource[]];
ReleaseObject[evt]
]
keyEntered[evt_, _, _, _] :=
JavaBlock[
If[$IsMouseDown && $IsShiftDown,
$Channel@allNotesOff[];
$Channel@noteOn[ToExpression[evt@getSource[]@getName[]], 80];
evt@getSource[]@setBackground[Color`gray]
];
ReleaseObject[evt]
]
keyExited[evt_, _, _, _] :=
JavaBlock[
restoreKeyColor[evt@getSource[]];
ReleaseObject[evt]
]
instChanged[evt_, state_] :=
JavaBlock[
If[state === ItemEvent`SELECTED,
$Channel@programChange[evt@getSource[]@getSelectedIndex[]]
];
ReleaseObject[evt]
]
(******************************** End Piano Listing **********************************)