Audio works but only for one sound at a time

I've managed to get audio playback working for AIFF, OGGV, and MOD, but this only works for one sound being played at a time. According to the specs, AIFF can be played at the same time as an OGGV or MOD, but at no time may an OGGV and MOD play at the same time. My code seems to enforce this restriction, but I am having lots of trouble getting AIFF and OGGV/MOD streams mixed. When I start a long-lasting OGGV or MOD clip, then cause an AIFF to play, I get distortion, slowdown, and finally a segfault. I can't figure out what the problem is or even any reference code on doing this the right way.

@erikd, this is what I described to you around a month ago in #xiph. Please take a look at it. The relevant branch is ao-curses.