Well actually you could. You would have to rename your audio file to match the name of one of the voice set, then copy your audio files over the old one, overrighting them. This way, you would hear your own voice set, but the other people on the server would hear the old voice set. After that, if you wish some people to hear your voice set, you would only have to distribute your files to those people.
The problem with that, is that will be a complete overright, and means that if someone else is using the original files, you will hear your audio file. Example, you would hear a dwarf (if you chose to overight a drawf voice st) speaking a girly dark elf voice set.
Worth the incovenience? probably not, but this would be a way of doing it, without having a massive amount of audio files to be added to the hack.
-edits: I should really check the date of the last post :p