Tuesday, 25 August 2015

DX7 Voice Banks on the Web

If you've got a Yamaha DX7 you know that it's a nightmare to program, perhaps unless you've got one of these. For the rest of us, there are DX7 voice banks galore that can be googled and downloaded from all over the place. Or, if you can't be bothered, you can go to Bobby Blues' All the Web Patches Collection where apparently he assembled and zipped into a single archive all DX7 voices that he could find on the web.

If you download and unzip the archive, you get about 3000 files in 79 folders, each file containing a sysex-dump of a DX7 voice bank consisting of 32 voices. That's a lot of voices to sift through!

After listening to these voices for a while, you get the impression that they all sound the same. There are many reasons for that. Among the more prominent ones is the fact, that, chiefly, most DX7 sounds fall into one of three categories:
  1. E-Piano
  2. Brassy
  3. Other
Therefore you shouldn't be surprised that many of them sound very similar. But there's also a more mundane possibility: They sound alike because they are the same. In a collection of this size you are bound to find a lot of duplicates.

In order to test that proposition (and to avoid having to check the same voice over and over again), I wrote a small tool to help me identifying duplicate voices in a collection of banks. You can point it at a folder and it will traverse all sub-folders and read all *.syx files that it finds along the way. At the same time it culls all init and null voices. Init voices are those that you get when you initialize a voice slot for editing on a DX7. Zero voices are those that are just a stream of 0s. Most likely they are originating from unused slots in a software librarian.

The voices are then "sanitized", i.e the voice parameter values are clamped to their legal ranges. This is necessary because voice authors sometimes watermark their creations by specifying bigger-than-legal values for certain parameters. Because such values cannot be specified ordinarily, the voice must have been copied (as in "pilfered"). Or so. In any case, the DX7 will simply clamp a value to the legal range (e.g. 0-99 for an operator's output level) when reading such a voice, so we'll do the same. A sanitized voice (118 bytes = 128 minus 10 for the voice name which we ignore) is then interpreted as a binary (118*8=944bit) number and inserted into a sorted list. This way, duplicates flock together and can easily be identified. Note that a "voice" in this sense is simply a certain configuration of (legal) voice parameter values. It is possible that many of these configurations sound exactly the same (e.g. if the parameters of the first two "stacks" of algorithm 5 are swapped). But they are different configurations and hence treated as different voices.

If we do that on Bobby Blues' DX7_AllTheWeb archive we find that there are in total 95936 (non-null, non-init) voices in 2998 banks. Of these nearly 96000 voices, 26761 are distinct, and among them 11937 unique. Thus, on average, a single voice appears 3.5 times, or more precisely (because 11937 of them appear exactly once), the ones that are not unique appear on average 5.7 times. Indeed, some voices appear as many as 92 times across various banks! There are also several duplicate banks, i.e two or more banks that contain exactly the same voices (though not necessarily in the same order). But there is also a sizable number of unique banks, i.e. banks that contain only unique voices.
Screen shot of librarian showing all copies of the voice selected.
Note that the same voice is stored under various names in different banks.
Voices in orange in the left pane are unique (1 copy in the full archive).
Now all that duplication leads naturally to a question: how many of these banks can we delete in order to get the smallest number of banks still containing all voices? Good question! This is an instance of the set cover problem, where you want the smallest number of subsets covering the whole set. You can compute the exact number(s) using a branch-and-bound algorithm, but that is too much of a hassle. You get a good approximation by continuously adding those sets (banks) that contain the largest number of yet uncovered elements (that would be voices in our case), and then deleting those voices in all other banks that contain them. If a bank loses its last voice, it is empty and can be deleted. This algorithm assumes, of course, that we don't rearrange voices (move between banks to fill them up). We only want to eliminate complete banks that we don't need because all the voices therein are already contained in other banks.
Banks in blue are those with more than one copy (selected has five).
Banks in orange are unique (contain only unique voices).
Note that one library contains the same bank under 3 names (Cart4, Ray-1 and Ray-5).
Because unique banks contain only unique voices, we can move them immediately to our final set. Next follow all those non-unique banks that contain one or more unique voices. Because these voices can't be anywhere else we must include these banks trivially, too. For the remainder of the banks the algorithm proceeds as specified above by finding and including those banks that contain the largest number of yet not-included voices, etc. In the end, 1382 banks remain. That's about half the number in the archive, the other half is unnecessary -- coverage-wise.

Still, there are 26761 distinct voices, and if you want to audition them all they should keep you busy for a while. Well, at least you shouldn't expect too many duplicates anymore...

1 comment:

  1. HI, Is the software tool and/or the source code for it available? Thanks.