Roland Boutique JU-08 MIDI System Exclusive

The Roland Boutique line of synthesizer modules are known to support only a few of the generic MIDI control messages. The MIDI Implementation Chart of the JU-06 lists the following MIDI CC messages:

1	MOD
11	EXPRESSION PEDAL
64	HOLD

However, Boutique modules of the same kind can be chained in order to increase polyphony. To accomplish this, it is necessary that the master module (the first in the chain) is able to control parameters of the slave modules. It does so by transmitting system exclusive ("sysex") messages whenever the master's state is changed.

1. If the master is switched to a different patch, a dump of all its parameters is transmitted via several system exclusive messages. A dump of the current patch is also sent, when the master is switched to chain mode.
2. If voice parameters are changed via panel controls, appropriate system exclusive messages are sent.

Note, that these patch dumps or panel controller messages are sent only if the module is in chain mode. But a module will always respond to such system exclusive messages, irrespective of the mode it is in.

So, while a module that is not in chain mode will not send parameters or dumps on its own, you can still request that it sends these parameter values by sending it parameter request messages.

A parameter request message has the following form:

f0 41 10 00 00 00 1d 11 [address] [length] xx f7
 |  |  |  |  |  |  |  |                     |  |
 |  |  |  ?  ?  ?  |  Request parameter     |  End of Exclusive
 |  |  |           Product code for JU-06   Checksum
 |  |  Device ID
 |  Roland
 Begin System Exclusive

Address and length fields are 4 bytes each, highest byte first. Because parameter values are sent half-byte wise (4 bits per sysex byte, higher bits first), lengths and addresses are multiples of two. Note that with the Boutiques you cannot request multiple values at once (by specifying an appropriate length), therefore the length field is usually 2 (most parameter values are 8-bit values = 2 times 4bits). The checksum covers only the address and length (or data, see below) fields.

The response sent by a module looks as follows:

f0 41 10 00 00 00 1d 12  [address] [data]  xx f7
 |  |  |           |  |                     |  |
 |  |  |           |  Parameter value       |  End of Exclusive
 |  |  |           Product code for JU-06   Checksum
 |  |  Device ID
 |  Roland
 Begin System Exclusive

Because data bytes in a MIDI message must have their highest bit=0, a data byte can contain only values from 0 to127. A 8-bit parameter value is therefore split into two half-bytes, each containing 4 bits of parameter value data, and sent as two MIDI data bytes, with the highest 4 bits in the first byte.

aaaabbbb -> 0000aaaa 0000bbbb   d[i]=(v>>4)&0x0f; d[i+1]=v&0x0f;

In a response, two such successive MIDI bytes are combined in the following manner to reconstruct a parameter value byte.

0000xxxx 0000yyyy -> xxxxyyyy   (d[i]<<4)|(d[i+1]&0x0f)

The following table shows all known parameters of the JU-06. The top 2 bytes of the address field are always 03 00 , so the table lists only the high (columns) and low (rows) bytes of the remaining 2 address bytes. The address ranges (indicated by the high byte, i.e. columns) appear to relate to a particular function (LFO, DCO, ...).

	06	07	08	09	0A	10	11
00	Rate	Range	HPF	Mode	Attack	Chorus Switch	Porta Switch
02	Delay Time	LFO Mod	Cutoff	Level	Decay	Delay Level	Porta Time
04		PWM	Reso		Sustain	Delay Time	Assign Mode
06		PWM Source	Env Polarity		Release	Delay Feedb	???
08		Square Switch	Env Mod				Bend Range
0A		Saw Switch	LFO Mod
0C		Sub Level	Key Follow
0E		Noise Level
	LFO	DCO	VCF	VCA	ENV	Effect	Controller

Example: VCF Cutoff has address [03 00] 08 02

The patch dump that you get when you change voices or switch to chain mode is split by function. So you get a separate sysex message for each of the address ranges starting with 06 00 (LFO), 07 00 (DCO), and so on. What's interesting, is that each message contains more entries than there are parameters in the JU-06. So while the LFO has two parameters, the corresponding sysex message has 7 entries! I wonder what they are for...

Address	06	07	08	09	0A	10	11
Function	LFO	DCO	VCF	VCA	ENV	Effect	Controller
Parameters	2	8	7	2	4	4	4 (5)
Sysex entries	7	13	12	7	9	10	11

When you supply the wrong number of 'length' bytes (2 or 3) in a parameter request, yet still the correct check sum, the JU-06 responds with a dump of the whole data set (as in a voice dump), but with more entries. This way, I discovered that the patch name starts at address (03 00) 11 16. A patch name appears to be 16 characters long -- at least that's the number of characters in the file dumps that you get when you follow the data backup procedure of your module. In the sysex dump there are a lot more characters, but I don't know what they're for -- yet.

If you want to query the voice name, you have to include 2 bytes before the voice name. When I supplied address 11 16, the module consistently failed to respond. However, when giving address 11 14, I can get as many characters as I want. Of course, with the lower address the characters start at index 2 of the sysex data field, with 1 character per byte.

This is as far as I got, but there's certainly more to discover. Stay tuned!

Sources and Further reading

• Roland System Exclusive Messages
• Reverse-engineering the Roland JU-06 synth
• Roland Boutique JU-08 Data File Format
• Summary Of Roland Boutique SYSEX messages [JPG]

Updated: 6.1.2016

Roland Boutique JU-06 Data File Format

In my YoshiMusix music blog, I've written two short reviews [one in english, der andere in deutsch] on the new Boutique synth modules by Roland, in particular the JU-06 that I bought. At the moment, I'm also fiddling with the new Web MIDI API by the World Wide Web Consortium, because you have to be up-to-date with stuff like that. So, while poking around and checking this and that, I thought I could check the file format used by the Boutiques, because ... why not? The first thing you do, or rather, the first thing I do is to drag the file into Windows Notepad. That'll give you a broad idea on what's inside a file, because among all the garbage you can read printable characters.

What a surprise I got, when I saw that the data files for the sequencer and voice data are actually ASCII text files! Here's a dump of the first sequencer line, and below of voice 1-1. I think the dumps speak for themselves (lines are separated by CR LF).

Contents of the first sequencer data (BQ_PTN1.PRM):

SCALE(1);
LASTSTEP(15);
SW1(1);
SW2(1);
SW3(1);
SW4(1);
SW5(1);
SW6(1);
SW7(1);
SW8(1);
SW9(1);
SW10(1);
SW11(1);
SW12(1);
SW13(1);
SW14(1);
SW15(1);
SW16(1);
NOTE1(48);
NOTE2(72);
NOTE3(61);
NOTE4(60);
NOTE5(70);
NOTE6(75);
NOTE7(60);
NOTE8(73);
NOTE9(60);
NOTE10(48);
NOTE11(60);
NOTE12(72);
NOTE13(60);
NOTE14(70);
NOTE15(60);
NOTE16(72);
VELOCITY1(70);
VELOCITY2(83);
VELOCITY3(72);
VELOCITY4(64);
VELOCITY5(79);
VELOCITY6(49);
VELOCITY7(51);
VELOCITY8(82);
VELOCITY9(44);
VELOCITY10(38);
VELOCITY11(44);
VELOCITY12(50);
VELOCITY13(71);
VELOCITY14(72);
VELOCITY15(67);
VELOCITY16(67);
GATETIME1(100);
GATETIME2(100);
GATETIME3(100);
GATETIME4(100);
GATETIME5(100);
GATETIME6(100);
GATETIME7(100);
GATETIME8(100);
GATETIME9(100);
GATETIME10(100);
GATETIME11(100);
GATETIME12(100);
GATETIME13(100);
GATETIME14(100);
GATETIME15(100);
GATETIME16(100);
PTN_NAME(PATTERN 01                      );

 And here the voice data of preset 1-1 (JU06_PATCH1.PRM)

LFO RATE        (30);
LFO DELAY TIME  (59);
OSC RANGE       (0);
OSC LFO MOD     (0);
PWM             (253);
PWM SOURCE      (0);
SQR SW          (0);
SAW SW          (1);
SUB LEVEL       (0);
NOISE LEVEL     (0);
HPF             (41);
CUTOFF          (102);
RESONANCE       (17);
ENV POLARITY    (1);
ENV MOD         (100);
FLT LFO MOD     (0);
FLT KEY FOLLOW  (194);
AMP MODE        (1);
AMP LEVEL       (255);
ATTACK          (32);
DECAY           (110);
SUSTAIN         (119);
RELEASE         (60);
CHORUS SW       (1);
DELAY LEVEL     (0);
DELAY TIME      (0);
DELAY FEEDBACK  (0);
PORTA SW        (0);
PORTA TIME      (100);
ASSIGN MODE     (0);
BEND RANGE      (2);
PATCH_NAME(Brass           );

So, if you're unhappy with the vintage analog slider-and-knob-interfaces of the Boutique synths, all you need to edit a patch is Notepad!

Polyphonic Auduino

If you're into Arduino hacking and are also interested in synthesizers, you've probably heard of the Auduino granular synthesizer by Peter Knight of tinker.it. It's a simple noise maker built using a 16MHz Arduino board and a handful of components. And it's a lot of fun to play with.

If you want to build one, you can head over to Notes and Volts where there's a multi-part video explaining how to do it, starting with a standalone Arduino board, the actual Auduino, and then two videos describing a line level and finally a MIDI mod (links to the various parts are provided in the video).

The synthesis engine is only a few lines of code and invites hacking it ... to boldly go where no man has gone before. So, what about a polyphonic Auduino?

To generate sounds, the synth employs a method called granular synthesis. I don't know anything about granular synthesis (and haven't found a simple introduction on the intertubes), but from reading the code I was able to dissect the basic operation of the synth. There's a two-part counter called synthPhase (with an Accumulator and an Increment) that controls the pitch of the sound. It does so by determining the rate at which two triangle wave samples -- the grains that give the method its name -- are replayed (i.e. restarted). The two triangle wave oscillators that generate these samples consist each of a two-part counter grainPhase (grain2Phase, respectively, each with an Acc and Inc) and a decaying amplitude grainAmp (grain2Amp, respectively). The speed and decaying rate of these two oscillators is determined solely by the parameters read in the main loop. In other words they are quasi free-running oscillators that are independent of the note being played. All in all there is one counter for the note and two for the grains. In principle, adding another voice would mean adding another counter. Does the Arduino have enough power to allow adding more voices?

If you inspect the code (I'm talking of the midified code by David Benn of Notes and Volts) you can see that it consists of two loops. Actually, you don't, but it does. The main loop is represented by the function called loop, and the second by the timer interrupt routine. The loop function is called repeatedly by the Arduino main program and contains all time insensitive operations such as the processing of MIDI events and the reading of the potentiometers that control the sound of the synth. The second one is called repeatedly by a timer and is responsible for computing the actual sound samples at a rate of 31.25kHz. If you consider that the Arduino's processor is running at 16MHz, this leaves at most 512 cycles for the interrupt routine. Otherwise it won't be able to finish before another interrupt is initiated. That's not a lot. In order to provide, say, four independent voices we need to severely limit what needs to be computed for a sound sample.

Before starting to tinker with the code I measured how long it takes the interrupt routine to compute a sample. I built a version of the Auduino code with a free digital pin being set when the interrupt routine is entered and cleared before it is exited. A plot of the corresponding signal on an oscilloscope shows a square wave with a duty cycle of about 26%. In other words, the processor spends a bit more than a quarter of the time in the interrupt routine. Upon closer inspection, I found a few places where the compiler produces sub-optimal code, especially with regard to multiplications (a fact that has not been lost on others). By working around that and also reformulating a further two lines of code, I was able to reduce the duty cycle to less than 22% without even beginning to change the algorithm.

In order to see, how far I can push the envelope with a simplistic approach, I copied the wave generating code within the interrupt routine three more times, added some simple 4 notes on/off logic and burned the whole contraption to the Arduino. Lo and behold, I was able to play 4-note chords on the Auduino with the duty cycle only going a tad above 75%.

Spending three quarters of the time in an interrupt routine is a lot, but perhaps just about acceptable. I analyzed the code (including the generated assembly code), and I can confidently say that 4 voice polyphony is quite the limit on a 16MHz Arduino, if one wants to keep the algorithm as it is. Of course, one needn't do that. I wonder, if you can hear any difference, if the grain waves are of a different shape. After all, the ATmega contains two hardware counters (one 16 and one 8 bits) which could be used instead of the software counters to generate the grain waves.

So there you go. A 16MHz Arduino is perfectly capable of doing 4-voice polyphonic grain synthesis. Four voices is not a lot, of course. But considering that you can buy brand new professional equipment with just 4-voice polyphony, we need not be embarrassed.

DX7 Voice Banks on the Web

If you've got a Yamaha DX7 you know that it's a nightmare to program, perhaps unless you've got one of these. For the rest of us, there are DX7 voice banks galore that can be googled and downloaded from all over the place. Or, if you can't be bothered, you can go to Bobby Blues' All the Web Patches Collection where apparently he assembled and zipped into a single archive all DX7 voices that he could find on the web.

If you download and unzip the archive, you get about 3000 files in 79 folders, each file containing a sysex-dump of a DX7 voice bank consisting of 32 voices. That's a lot of voices to sift through!

After listening to these voices for a while, you get the impression that they all sound the same. There are many reasons for that. Among the more prominent ones is the fact, that, chiefly, most DX7 sounds fall into one of three categories:

1. E-Piano
2. Brassy
3. Other

Therefore you shouldn't be surprised that many of them sound very similar. But there's also a more mundane possibility: They sound alike because they are the same. In a collection of this size you are bound to find a lot of duplicates.

In order to test that proposition (and to avoid having to check the same voice over and over again), I wrote a small tool to help me identifying duplicate voices in a collection of banks. You can point it at a folder and it will traverse all sub-folders and read all *.syx files that it finds along the way. At the same time it culls all init and null voices. Init voices are those that you get when you initialize a voice slot for editing on a DX7. Zero voices are those that are just a stream of 0s. Most likely they are originating from unused slots in a software librarian.

The voices are then "sanitized", i.e the voice parameter values are clamped to their legal ranges. This is necessary because voice authors sometimes watermark their creations by specifying bigger-than-legal values for certain parameters. Because such values cannot be specified ordinarily, the voice must have been copied (as in "pilfered"). Or so. In any case, the DX7 will simply clamp a value to the legal range (e.g. 0-99 for an operator's output level) when reading such a voice, so we'll do the same. A sanitized voice (118 bytes = 128 minus 10 for the voice name which we ignore) is then interpreted as a binary (118*8=944bit) number and inserted into a sorted list. This way, duplicates flock together and can easily be identified. Note that a "voice" in this sense is simply a certain configuration of (legal) voice parameter values. It is possible that many of these configurations sound exactly the same (e.g. if the parameters of the first two "stacks" of algorithm 5 are swapped). But they are different configurations and hence treated as different voices.

If we do that on Bobby Blues' DX7_AllTheWeb archive we find that there are in total 95936 (non-null, non-init) voices in 2998 banks. Of these nearly 96000 voices, 26761 are distinct, and among them 11937 unique. Thus, on average, a single voice appears 3.5 times, or more precisely (because 11937 of them appear exactly once), the ones that are not unique appear on average 5.7 times. Indeed, some voices appear as many as 92 times across various banks! There are also several duplicate banks, i.e two or more banks that contain exactly the same voices (though not necessarily in the same order). But there is also a sizable number of unique banks, i.e. banks that contain only unique voices.

Screen shot of librarian showing all copies of the voice selected.
Note that the same voice is stored under various names in different banks.
Voices in orange in the left pane are unique (1 copy in the full archive).

Now all that duplication leads naturally to a question: how many of these banks can we delete in order to get the smallest number of banks still containing all voices? Good question! This is an instance of the set cover problem, where you want the smallest number of subsets covering the whole set. You can compute the exact number(s) using a branch-and-bound algorithm, but that is too much of a hassle. You get a good approximation by continuously adding those sets (banks) that contain the largest number of yet uncovered elements (that would be voices in our case), and then deleting those voices in all other banks that contain them. If a bank loses its last voice, it is empty and can be deleted. This algorithm assumes, of course, that we don't rearrange voices (move between banks to fill them up). We only want to eliminate complete banks that we don't need because all the voices therein are already contained in other banks.

Banks in blue are those with more than one copy (selected has five).
Banks in orange are unique (contain only unique voices).
Note that one library contains the same bank under 3 names (Cart4, Ray-1 and Ray-5).

Because unique banks contain only unique voices, we can move them immediately to our final set. Next follow all those non-unique banks that contain one or more unique voices. Because these voices can't be anywhere else we must include these banks trivially, too. For the remainder of the banks the algorithm proceeds as specified above by finding and including those banks that contain the largest number of yet not-included voices, etc. In the end, 1382 banks remain. That's about half the number in the archive, the other half is unnecessary -- coverage-wise.

Still, there are 26761 distinct voices, and if you want to audition them all they should keep you busy for a while. Well, at least you shouldn't expect too many duplicates anymore...

V/Oct Calibration Tool

I'm currently upgrading a few of my first DIY synthesizers by adding MIDI control and perhaps also renovating the front panel. I also want to re-calibrate the tracking of all the oscillators of the first synths I built in order to have a proper volt/octave scale for the MIDI/CV converters to work with. To that end, I built a small V/Oct calibration tool based on an old circuit by Ray Wilson of Music From Outer Space that he mentions in one of his earlier VCO circuits. In contrast to Ray's tool, mine spans a 7 octave range by supplying voltages from 0 to 7 volts.

V/Oct calibration tool. Soon with enclosure.

I started by selecting a couple of resistors from a batch of 50 or so (10k 1% metal film) for equal resistance using a Wheatstone bridge. I managed to find two groups of 3 and 4 resistors that among them produced a midpoint voltage of 0.0mV (when measured using my cheapo DMM), with the two groups being 0.1mV apart. I don't know how that difference translates to resistance, but in any case they all now seem pretty equal to me. :-)

I used these resistors to build a multi-level voltage divider to produce the voltage steps starting with the 7 volts on top of the chain. Incidentally, I get the top 7 volts by simply swapping the 47k and 68k resistors in Ray's circuit (trimmed to precision using a 10k multi-turn trimpot). The calibration tool's power is taken from the synth's power supply. Two LEDs on the front panel show proper connection to both negative and positive supply voltages.

It took a bit more time to build than I anticipated (I did it on perfbord with the component leads providing the connections between parts), but now that it's finished I actually look forward to tediously calibrating all the oscillators.

Mutable-Instruments Shruthi-1

22. May 2015
I've visited the site of Mutable Instruments many times in the last two years or so, and pondered and contemplated buying one of their fine synthesizer kits. But then I paused visiting them for a while, and while I was absent they appear to have stopped offering some of them. You can still download and manufacture on your own their PCB designs, but ready-for-assembly kits are gone. The only synthesizer that they still sell kits for is the Shruthi-1, and nobody knows how long stocks will last. I wonder if you can guess what happened next...

If what you guessed rhymes with "bought a kit" you're on the right track. Indeed, I just received the kit by DHL, and here's what I've got.

Full kit with PCBs, parts and metal enclosure.

Filter board. Pretty little details...

Really nice and solid metal case.

As I have written elsewhere I prefer interesting standalone synthesizers to giant modular racks, and that's why I couldn't resist adding a Shruthi to my growing collection of DIY gear. Building the device shouldn't take too long -- a few hours they say on the web site. That'll give me something to work on while I'm sourcing all the necessary parts for the ASM-2 that I also just bought a PCB for.

25. May 2015
I started stuffing the filter board. I finished the power supply and soldered all resistors. The board pretty much looks like step 10 on the very detailed step-by-step Shruti assembly instructions page. Which is why posting any pictures of my progress seems pretty pointless, come to think of it.

26. May 2015
Done! After having completed the filter board I started stuffing the control board. It has far fewer components and was completed in half the time it took to stuff the filter board. Some care is required, though, because there are parts on both sides of the board, and you need to be careful that they don't interfere with one another, especially with the display. Before mounting the display I made sure the micro controller booted correctly, as suggested in the building instructions, because it'll obscure parts of the micro controller's solder pins and render them inaccessible.

After having soldered the display in place and also the controller elements (buttons, potentiometers and the encoder), I was ready to calibrate the filter to a V/oct scale. I did it "by ear" using my DX7 as a sine wave source. It's not the most accurate way of doing this, but I managed to match the filter's resonance sine wave to that of the DX's to within a fraction of a cycle. That's good enough for me. Mounting the PCB stack into the metal case completed the project.

Building the Shruthi-1 took me about a day, and I'm not the most efficient builder. Anybody more experienced should be able to complete it in a few hours. And what does it sound like? What I've heard so far sounds very cool, especially the built-in arpeggio and sequencer. I'll make a YouTube video to show off the synth but first I'll have to waste some time with it.