DX7 Voice Banks on the Web

If you've got a Yamaha DX7 you know that it's a nightmare to program, perhaps unless you've got one of these. For the rest of us, there are DX7 voice banks galore that can be googled and downloaded from all over the place. Or, if you can't be bothered, you can go to Bobby Blues' All the Web Patches Collection where apparently he assembled and zipped into a single archive all DX7 voices that he could find on the web.

If you download and unzip the archive, you get about 3000 files in 79 folders, each file containing a sysex-dump of a DX7 voice bank consisting of 32 voices. That's a lot of voices to sift through!

After listening to these voices for a while, you get the impression that they all sound the same. There are many reasons for that. Among the more prominent ones is the fact, that, chiefly, most DX7 sounds fall into one of three categories:

1. E-Piano
2. Brassy
3. Other

Therefore you shouldn't be surprised that many of them sound very similar. But there's also a more mundane possibility: They sound alike because they are the same. In a collection of this size you are bound to find a lot of duplicates.

In order to test that proposition (and to avoid having to check the same voice over and over again), I wrote a small tool to help me identifying duplicate voices in a collection of banks. You can point it at a folder and it will traverse all sub-folders and read all *.syx files that it finds along the way. At the same time it culls all init and null voices. Init voices are those that you get when you initialize a voice slot for editing on a DX7. Zero voices are those that are just a stream of 0s. Most likely they are originating from unused slots in a software librarian.

The voices are then "sanitized", i.e the voice parameter values are clamped to their legal ranges. This is necessary because voice authors sometimes watermark their creations by specifying bigger-than-legal values for certain parameters. Because such values cannot be specified ordinarily, the voice must have been copied (as in "pilfered"). Or so. In any case, the DX7 will simply clamp a value to the legal range (e.g. 0-99 for an operator's output level) when reading such a voice, so we'll do the same. A sanitized voice (118 bytes = 128 minus 10 for the voice name which we ignore) is then interpreted as a binary (118*8=944bit) number and inserted into a sorted list. This way, duplicates flock together and can easily be identified. Note that a "voice" in this sense is simply a certain configuration of (legal) voice parameter values. It is possible that many of these configurations sound exactly the same (e.g. if the parameters of the first two "stacks" of algorithm 5 are swapped). But they are different configurations and hence treated as different voices.

If we do that on Bobby Blues' DX7_AllTheWeb archive we find that there are in total 95936 (non-null, non-init) voices in 2998 banks. Of these nearly 96000 voices, 26761 are distinct, and among them 11937 unique. Thus, on average, a single voice appears 3.5 times, or more precisely (because 11937 of them appear exactly once), the ones that are not unique appear on average 5.7 times. Indeed, some voices appear as many as 92 times across various banks! There are also several duplicate banks, i.e two or more banks that contain exactly the same voices (though not necessarily in the same order). But there is also a sizable number of unique banks, i.e. banks that contain only unique voices.

Screen shot of librarian showing all copies of the voice selected.
Note that the same voice is stored under various names in different banks.
Voices in orange in the left pane are unique (1 copy in the full archive).

Now all that duplication leads naturally to a question: how many of these banks can we delete in order to get the smallest number of banks still containing all voices? Good question! This is an instance of the set cover problem, where you want the smallest number of subsets covering the whole set. You can compute the exact number(s) using a branch-and-bound algorithm, but that is too much of a hassle. You get a good approximation by continuously adding those sets (banks) that contain the largest number of yet uncovered elements (that would be voices in our case), and then deleting those voices in all other banks that contain them. If a bank loses its last voice, it is empty and can be deleted. This algorithm assumes, of course, that we don't rearrange voices (move between banks to fill them up). We only want to eliminate complete banks that we don't need because all the voices therein are already contained in other banks.

Banks in blue are those with more than one copy (selected has five).
Banks in orange are unique (contain only unique voices).
Note that one library contains the same bank under 3 names (Cart4, Ray-1 and Ray-5).

Because unique banks contain only unique voices, we can move them immediately to our final set. Next follow all those non-unique banks that contain one or more unique voices. Because these voices can't be anywhere else we must include these banks trivially, too. For the remainder of the banks the algorithm proceeds as specified above by finding and including those banks that contain the largest number of yet not-included voices, etc. In the end, 1382 banks remain. That's about half the number in the archive, the other half is unnecessary -- coverage-wise.

Still, there are 26761 distinct voices, and if you want to audition them all they should keep you busy for a while. Well, at least you shouldn't expect too many duplicates anymore...

RAM Cartridge for the Yamaha DX7 Synthesizer (Part V: Success)

This is part five of my report on building a RAM cartridge for the Yamaha DX7 digital synthesizer, and finally I've got some positive news: I've got a prototype that works! Since my last failure report I ordered another prototype batch, this time using seeedstudio's Fusion PCP service, and a 1.2mm thick board. I also ordered on eBay a replacement cartridge socket so I could wire the connector out of the synthesizer and in the future plug the prototype panels in without wearing out the cartridge socket of the DX7.

The layout of the new PCBs is the nearly same as that of the ones manufactured at OSH Park. They're only thinner, i.e. 1.2mm instead of 1.6mm, and this time I even added a pad for the (necessary?) 2.2uF decoupling capacitor. Soldering the handful of components was quick and painless. When I plugged it into the DX7 for the first time (the external connector isn't set up yet), it didn't work. The synthesizer simply didn't recognize the cartridge. To my utter dismay it appears that the PCB is too thin now, as it slipped right into the socket without any force. I could make the DX7 recognize it, however, by pressing it slightly backwards. Still, it didn't work: I consistently got a Write Error. I was quite convinced by then that there was a fundamental problem with my design.

I studied the circuit diagrams again for hours, even wrote a quick'n'dirty disassembler for the DX7's main CPU, the HD63B03X, to disassemble the system ROM and inspect the routines for writing the cartridge. I also started making a connector probe using one of the PCBs with wires soldered to the EEPROM solder pads. This would allow me to signal-trace the various EEPROM bus signals, especially the two chip select lines /CE1 and /CE2 and the write enable line /WE. Because the PCBs are too thin I planned to cover the connector pads with some solder to make them slightly thicker. For good measure I did the same thing with the prototype PCB for a final test to make sure it wasn't a dodgy connection that caused the failure.

I plugged it in, and it went quite nicely into the socket, requiring some but not excessive force. Then I pressed the cartridge voice selector. The display read FORMAT CONFLICT! So I pressed function 8: format cartridge. The display read CARTRIDGE FORM ? I pressed yes. ARE YOU SURE ? Yes. MEMORY PROTECTED. Oh, sure, I forgot. I pressed the cartridge protect selector. MEMORY PROTECT CARTRIDGE ON. I pressed off, then function 8. CARTRIDGE FORM ? Yes. ARE YOU SURE ? Yes. UNDER WRITING ! I stopped breathing. Then, after a few long seconds: FORMATTING END. I took a breath. It worked? I pressed the cartridge voice selector, then voice 1: INIT VOICE. Voice 17: INIT VOICE. It worked?? I saved the internal memory onto the cartridge, and after a few seconds of UNDER WRITING I found all my voices on the cartridge, and they sounded exactly as they're supposed to. I even switched the cartridge to the second bank, ran the same formatting exercise and ended up with a cartridge that had my voices on one bank and init voices on the other. It worked! Success! Oh, sweet success!! How can I ruin it. I need some rest. And another breath.

RAM Cartridge for the Yamaha DX7 Synthesizer (Part IV: Reboot)

This is part four of my report on building a RAM cartridge for the Yamaha DX7 digital synthesizer. It took a while since the last installment, because I've got several hard- and software projects that run in parallel. One is a hardware programmer for the DX7, a bit like this Jellinghaus DX programmer, but with far fewer knobs, but a graphical display instead. Another is a user interface (including graphical display) for a Yamaha XG synthesizer based on a DB50XG daughter board, and the third is a software librarian and editor for the DX7.

Anyway, I completely redesigned the RAM cartridge, this time using a correctly sized edge connector, correct pin assignment, and a through-hole EEPROM.

Furthermore, I tried to stay within the confines of the original cartridge board area. The packing is dense but not too crowded. Soldering the components was quick and painless. The board is suitable for both 64kbits (suitable for 2 DX7 voice banks) and 256kbits (8 banks) EEPROMs. To test the board I use the simpler 64kbit device and a an ordinary sliding switch in the position of the rotary coding switch (top left corner of the board), and only one additional resistor. I didn't include a pad for the 2.2uF polar capacitor, but managed to mount it between 5V and ground by soldering it to the power leg of the EEPROM and the bottom pad of R1 to the left of the memory chip. The finished board looked quite neat and tidy, a far cry from the mess of the previous prototypes. I couldn't wait to test it in my DX7.

Well, make that last one "I couldn't wait to test it in my DX7". It seems the board is too thick. I could possibly force it into the connector, but I don't want to ruin my DX7. The boards do have a solid if not heavy feel. Nonetheless, OSH Park's pricing and spec page states that the boards are 1.6mm thick. They're closer to 1.8mm. When I tested my previous prototypes I thought that even my 1.6mm thick hand crafted adapter was close to the limit regarding the force required to set the board into the socket, but these are just too thick.

This is very unfortunate, because I like the quality, price and ease of use of OSH Park, but for this project I can't use their service. I need to find a different prototyping service that allows specifying thinner boards, perhaps 1.4mm or so.

RAM Cartridge for the Yamaha DX7 Synthesizer (Part III: Testing)

DX7 RAM cartridge version 1 (left) and 2 (right). Surprisingly(?) neither works.

My DIY project to build my own RAM cartridge for the Yamaha DX7 digital synthesizer is currently in limbo. I have fixed everything that I found to have made wrong with my prototype board, and it still does not work. I had the following problems with my board:

• Edge connector is too narrow. It appears to have a 3+ mm pitch. Mine has a tenth inch (2.54mm) pitch. I fixed it by gluing a separate correctly sized/pitched double sided  PCB to the board. At the same time I also fixed...
• Swapped connectors. There was a 50-50 chance as to which side of the edge connector faces which way. My guess was wrong: the side with the power supply faces backwards. To fix that, I had to wire the pins of my glued-on PCP to the corresponding pins on the prototype PCB crosswise back-to front and front-to-back.
• Missing pull-up resistors. For a reason that totally eludes me, I forgot to add pull-up resistors for address lines 8 to 10. I soldered them at the back of the PCP using the power pin on a near resistor network.
• Missing 2.2uF/16 polar bypass capacitor. I know from various synth DIY projects that you're supposed to add bypass capacitors near the power supply of ICs. I didn't add one on the prototype board, because the design on yates.ca doesn't include one either.

Despite all of the above, the board fails consistently with a Write Error! message. I can briefly see the message Under Writing flash by, which led me to the missing address pull-up resistors. That didn't fix it, though. I'm a bit at a loss, because the circuit isn't that complicated, and I can see no reason why it shouldn't work. Perhaps I have simply fried the 28C64 EEPROM? It is a surface mount device, and I'm not the most expert solderer. For me, it means to go back to the drawing board.

In the meantime, you might be interested in the DX7 RAM Cartridge project by blogger Brian Durocher.

RAM Cartridge for the Yamaha DX7 Synthesizer (Part II: Prototyping)

You knew it! If you followed my Les Paul DIY build log, you're not surprised to hear that I managed to bungle the DX7 cartridge project in a similarly spectacular fashion, and it's all my fault.

The prototype boards arrived, and they look awesome! The PCB material is of a dark violet color, and all the solder pads are golden. I ordered them at OSH Park, where you'll always get (multiples of) 3 boards.

DX7 cartridge prototype boards, rightmost already populated.

When I checked them I saw instantly that I made the connector part too small. Instead of measuring the proper connector spacing, I violated Sommerer's Law ("your assumptions are wrong") and assumed them to be the usual one tenth of an inch or 2.54mm. Well, its closer to 3 mm, so they don't fit. Oh, well!

Now, instead of throwing them away and order another batch with the proper dimensions, I'll use them as tinkering material. I already ordered a small sheet of blank double sided circuit board, which I'll cut to size using a high-tech template that I made from an obsolete phone card.

I'll make the properly spaced connector traces by filing or scratching gaps into the copper layer, and then glue the contraption as a new connector onto the PCB. All that's needed then is pieces of wire connecting the pads on the cartridge PCB with the corresponding pads on the new connector. It'll look hideous, but serve its purpose.

But first I have to wait for the blank PCB to arrive.

RAM Cartridge for the Yamaha DX7 Synthesizer (Part I: Introduction)

When I was a teenager I spent hours on end in the keyboard section of music instrument shops and played those keyboards and synthesizers that I knew from the back of album covers, where musicians used to list the equipment they used. Of course, the famous flagship synthesizers like the Roland Jupiter-8, the Oberheim OB-8 and the likes were completely out of my price range. Finally, a friend of mine who owned a small monophonic synthesizer himself told me that there's a new synthesizer that's perfect for me: the Yamaha DX7. A few weeks later my mom drove me across the border into Germany where I bought a DX7 for under CHF 3000. Despite being over 30 years old, it still works perfectly, and I still play it frequently.

The Yamaha DX7 was the dominant synthesizer of the 1980s with its idiosyncratic sounds such as the electronic piano, bass, and marimba voices. If you happen to see a music video of that era and watch closely what keyboards the guys with the strange hairdos mime playing on, 9 out of 10 times it will be a DX7.

The DX7 has internal memory for 32 voices, which even for the 80s was relatively little. It has an expansion slot, however, where cartridges can be plugged in to provide direct access to further 32 voices. The DX7 shipped with two ROM cartridges, each containing 2 preset voice banks (2 x 32 voices) selectable by a switch on the cartridge. At the time, Yamaha also sold RAM cartridges onto which single voices or the full contents of the internal memory could be stored, sort of an external RAM extension. Unfortunately, the cartridges are very hard to find these days, and even the used ones sold on eBay are very expensive.

The publicly available circuit diagrams [PDF] of the DX7, e.g. on Dave Benson's DX7 page, reveal that the cartridge design is very simple. It is really only an external memory chip, and the pins on the cartridge slot are just the address, data and signal buses to control the reading and/or writing of data from or to an EEPROM.

The simplicity of the circuit suggests that it should be possible to build oneself such a memory cartridge. EEPROMS with 64kbits (8k x 8), that's room for 2 voice banks, can be found for less than USD 5. The other parts (resistors, switches and an AND gate for the combined chip enable lines) cost perhaps a dollar or two. The PCB is another story, but there are prototype PCB manufacturers like OSH Park that do them for USD 10 apiece or so. Don't even ask about the enclosure.

As you can guess, I'm going to try to design and build such a cartridge. In the following weeks I will be reporting on the progress of this project. Peek preview: The prototype boards have already been sent away for manufacture.

Update [19.08.2014 08:42 CEST]: I just got a notice from OSH Park that the PCBs have been shipped!

So stay tuned!