Flash Audio

Build a MIDI Synth with Flash

by Martin Wilde

page 1 2 3 4 5

Creating the Sequencer

In Step 3 of the requirements, we use the underlying Flash digital audio player to play out the sounds.  To understand how this works, let’s take a detour into some of the operational details of the MIDI interpreter.

SMFs are organized as a stream of bytes which represent commands to do such things as turn a note on or off, or change the volume or pan of a sound.  A SMF sequencer plays the files back by stepping through time and seeing what commands have to be issued at each successive moment.  The setInterval timer is the Flash mechanism we will use to drive that progression of time through the MIDI score.  We specify the desired period of the timer and the ActionScript method to process the MIDI commands of the score at the start of SMF playback.  We also keep track of the timer’s ID to stop it later:

TARGETPERIOD = 5; //5 mS target timer period
function StartTimer()
{
	timerID = setInterval(HandleTimer, TARGETPERIOD);
	startTime = getTimer(); //Get initial starting time
}

A MIDI ‘frame’ is the MIDI interpretation completed on each timer interrupt, and the inverse of this timer period is known as the ‘frame rate’ of the MIDI interpreter.  Do not confuse this with the frames and frame rate of Flash movies.  In fact, I should point out that the entirety of this processing occurs in a single Flash movie frame.  The result of the above ActionScript code is that the MIDI synth calls the HandleTimer method, ideally, every 5 milliseconds (equivalent to a frame rate of 200 Hertz).  The HandleTimer method in turn calls the ‘UpdatePlayers’ method which is what reads through the SMF array and executes the MIDI commands for that frame:

function HandleTimer()
{
	updateAfterEvent();
	now = getTimer(); //Get current time
	thisPeriod = (now-startTime); //Calculate mS duration of current period
	startTime = now; //Save current time for next timer interrupt
	UpdatePlayers(); //Do the MIDI interpretation
}

Accomplishing the MIDI command execution requires a reliable, steady timer.  Unfortunately, this is a point where Flash is lacking.  It is the timer that governs how frequently we look to see if there is something to do in the SMF, observing that the higher the frame rate, the better the musical timing.  But we don’t have a steady timer in Flash, at least not on the PC (Macs seem a little better).  I have found given a 5 mS desired period, over 98% of the timer periods recorded are late, and none are early.  However, only a small percentage of timer interrupts are more than 10 mS late, though it does happen with unsatisfactory frequency.  A 200 Hz frame rate may seem too aggressive, even for a MIDI interpreter in a real programming language such as C.  But through trial and error, I have found that this value gives the best performance on my PC.

So the question becomes, how do we compensate for this observed, erratic timer behavior?  It is possible to start a sound with a certain offset in Flash.  Since the majority of the timer frames are less than 10 mS late, we can borrow an idea from a previous column on echo and reverberation effects (see Remixology #17) and pad the head of each of sound with 10 mS of silence to counterbalance that lateness.  The underlying theory is this:  if the timer is right on, the MIDI synth starts the sound at it’s beginning and plays the full 10 mS silence pad.  If the timer is anywhere up to 10 mS late, the MIDI synth can start the sound at an appropriate offset into that silence pad:  we play less of the silence at the head of the sound, and the sound appears in the output mix at the intended time.  This explains my earlier directive to pad each sound’s head with 10 mS of silence before importing.  Of course, the timer could be more than 10 mS late, in which case the beginning part of the sound itself could get chopped off.  Therefore, we limit the maximum start offset for each sound to the duration of the pad.

You may be tempted to pad the sounds with more silence to compensate for very long delays.  But in this case, more is not better because the silence pad pushes the audible portion of the sound out in time.  In fast musical passages, that sound may get cut short, or not be heard at all, if the silence pad is too long.

This technique does a good job of taking care of the notes within an individual MIDI frame.  But the timer may be multiple frames late.  Therefore, we have to keep track of the total elapsed time and play catch up in our MIDI score when the timer finally returns.

The combination of these two compensation techniques goes a long way toward smoothing out the Flash timer’s idiosyncratic behavior.  But, as you can hear in the sample SWF, there is still a ‘herky-jerky'-ness in the final playback.

As I mentioned before, Flash can play a maximum of 8 notes simultaneously.  However, there are often more than 8 pitches sounding at the same time in a piece of music.  Rather than analyze the MIDI scores to make sure no more than 8 notes are played at a time, we employ a simple note-stealing scheme.  When it comes time to play a new note and all 8 Flash channels are playing, the note-stealing scheme stops the oldest note to start the new one.  We implement this note-stealing mechanism using a rudimentary note table.  We define a two-dimensional array called NoteTable to store the channel number, note number and time on for all 8 notes:

var NTCHAN = 0;
var NTNOTE = 1;
var NTTIMEON = 2;
var NTFREE = -1;
var NoteTable =[[0,0,NTFREE], [0,0,NTFREE], [0,0,NTFREE], [0,0,NTFREE],
				[0,0,NTFREE], [0,0,NTFREE], [0,0,NTFREE], [0,0,NTFREE]];

We set the NTTIMEON field of the NoteTable array to NTFREE as a flag to indicate that particular note is currently available.  We set this field to a real start time when we start a sound, and reset it to NTFREE when we stop that sound.

When all 8 NoteTable entries are full and another note in the score has to play, we find the note with the oldest start time by looking through the NTTIMEON fields of the array.  We stop the sound with the oldest start time and fill that NoteTable entry with the new note values.  The new sound is then started with the appropriate offset, effectively stealing that old note.

page 1 2 3 4 5