pouët.net

Synchronizing sound with graphics in 4k intros

category: code [glöplog]
 
Hi ! I'm pretty new to 4k and making my way through my first one (Windows intro).

I'm facing a sync issue between sound and graphics. The graphics is a GLSL shader and the music is a generated WAV buffer that I start playing with sndPlaySound (SND_ASYNC mode) just before the rendering loop (basically like in iq's 4k framework). The WAV has the same duration as the intro, and the rendering loop simply runs until timeGetTime() - startTime > introDuration.

Ideally, the audio and graphics should start roughly at the same time (without much of a difference at least). However, I notice a random delay (about 0.1s) between the moment the window/graphics start and the sound starts playing, resulting in wrong graphics/audio timing and the intro terminating a bit before the music finishes.

Does anyone know why it is happening and if/how it can be fixed ? Is there a correct way to ensure graphics/audio sync ?
Maybe my approach at 4k is completely outdated, haha...

Thanks !
added on the 2023-12-15 19:36:34 by Krafpy Krafpy
sndPlaySound is cheap but not reliable, sadly. If you have a few more bytes, you should probably be better off with just a single-buffer DirectSound setup (that you can render directly into) and then using the GetCurrentPosition to tell where your play cursor is.
(Upside: you can also #ifdef _DEBUG yourself seeking in the intro.)
added on the 2023-12-15 19:50:05 by Gargaj Gargaj
Thank you for the advice ! I'll try using DirectSound then.

Just out of curiosity, the Microsoft's documentation mentions WASAPI as a better alternative to DirectSound (which is apparently legacy). Have there been any attempt at using it, or does it add too much of an overhead for 4k compared to DirectSound ?
added on the 2023-12-15 20:24:26 by Krafpy Krafpy
You are probably using mmsystem.h, as it was used in Leviathan and IQ's framework. With some Windows update, this was of playing sound started to have this annoying delay. The delay even seems to be machine-specific. I worked so hard to figure out what the delay was on my machine (in the 4k intro Adam), but when the intro played on compo machine, the sync was off again.

Thus, don't use it. Instead, use DirectSound, as Gargaj suggested, with DSBCAPS_TRUEPLAYPOSITION to syncs land right on time. See here for a barebones examples and here how it was used with 4klang in an intro.
added on the 2023-12-15 20:31:30 by pestis pestis
Alright, thank you for the links !
added on the 2023-12-15 20:51:11 by Krafpy Krafpy
Don't use timeGetTime(), it's not synced with music playback, and nobody guarantees that visuals and audio start at the same time even if the start commands are located close to each other.

If you use waveOutOpen() to play music, you ask the system "what time is it now in audio land", and it returns you the current time position. You do this with the waveOutGetPosition() function.

Compofiller Studio's Templates/4k_intro/exemain.asm contains a working example how you play stuff.


Code: ... global MMTime ; MMTIME is a "union" struct that can be used for many different content types based on the first field MMTime: dd 2 ; wType: TIME_SAMPLES = 2 MMTime_sample: dd 0 ; actual position value db 0, 0, 0, 0 ; some room for other cases ... extern _waveOutOpen@24 extern __imp__waveOutOpen@24 extern _waveOutPrepareHeader@12 extern __imp__waveOutPrepareHeader@12 extern _waveOutWrite@12 extern __imp__waveOutWrite@12 extern _waveOutGetPosition@12 extern __imp__waveOutGetPosition@12 section .code2 code align=1 ; ---- GetPosition returns the current audio playback position ---- section .code3 code align=1 global GetPosition GetPosition: push 12 ; sizeof MMTIME struct push MMTime push DWORD [hWaveOut] call [__imp__waveOutGetPosition@12] mov eax, [MMTime_sample] ret ... call GetPosition ; <-- get the time to eax ; did we reach the end of the prod? cmp eax, %[PROD_END_TIME] jae loppu


Loppu means end in Finnish. PROD_END_TIME is the end-time of your prod in samples.
added on the 2023-12-16 12:35:14 by yzi yzi
@yzi: you don't need WaveOutPrepareHeader@12 btw
added on the 2023-12-16 12:57:53 by NR4 NR4
Ok. I think I tried leaving out everything one by one, and left the waveOutPrepareHeader call in place for some reason.
added on the 2023-12-16 13:27:32 by yzi yzi
I tried leaving out waveOutPrepareHeader() and then audio doesn't start. Maybe you could hard-code some stuff in the struct, and then some bytes gained by leaving out the function call would be lost by the added data values.

Have you tested this and how much does it save?
added on the 2023-12-16 13:45:26 by yzi yzi
it boils down to this: https://github.com/vsariola/sointu/blob/master/examples/code/asm/386/asmplay.win32.asm#L37

I always use it this way in my intros and didn't notice anything breaking yet
added on the 2023-12-16 14:34:42 by NR4 NR4
it saves the symbol + 3 * (push instruction + operand) + call instruction. So overall maybe not that much, but any byte you can save there adds to what you can add content-wise
added on the 2023-12-16 14:37:41 by NR4 NR4
I tested it, it worked and saved 5 bytes in the final Crinklerized output.

To apply the change, edit exemain.asm and write 2 in the dwFlags field of WaveHDR:

Code: global WaveHDR WaveHDR: WaveHDR_lpData: dd rendered_audio_data WaveHDR_dwBufferLength: dd (AUDIO_BUFFER_ALLOCATED_LENGTH * 4) ; audio buffer length in bytes WaveHDR_dwBytesRecorded: dd 0 WaveHDR_dwUser: dd 0 WaveHDR_dwFlags: dd 2 ; <----- wave header is declared as prepared WaveHDR_dwLoops: dd 0 WaveHDR_lpNext: dd 0 WaveHDR_reserved: dd 0


And comment out the waveOutPrepareHeader call

Code: ; push 32 ; sizeof(WaveHDR) ; push WaveHDR ; push DWORD [hWaveOut] ; call [__imp__waveOutPrepareHeader@12]


Nice!
added on the 2023-12-16 18:02:59 by yzi yzi

login