Emulated GB(C) video capture for YouTube etc.
category: offtopic [glöplog]
Background: If you want to rcord GB(C) demos, you've got a couple of choices. Record from a Super Gameboy (plugs into a SNES) or a Gameboy Player for the Gamecube. Or you can do screen recording of an emulator, which may result in skipped frames and/or audio. A more deterministic way would be to use VisualBoy Advance, which can output AVI files. However, this still leads to audio/video desynch, and VBA's sound isn't especially the best, even in VBA-m/VBA-rr.
So, what I've wanted to do is to use BGB. Lately, beware, the author of the emulator, has made a number of accuracy improvements to BGB. (More accuracy+a lot of improvements to the debugger will come in BGB 1.4 which is still under development. Soon you can say hello to the definitive no$gmb killer.) It will also have "high quality" audio rendering for all 4 channels (meaning here that the waveforms are built up from its partials with an inverse Fourier transform, so you never go above Nyquist.)
Lately, I've been nagging beware to implement some form of video output from BGB. The upcoming BGB 1.4 does have improvements in this regard, even if it's not as easy to use as VBA for video output. The feature lets you output screenshots as BMP files for each frame, and sync this exactly with the audio. The BMP method is a bit lo-fi, but whole thing can easily be reassembled with AviSynth.
So far everything seems ok. I've uploaded a video of Cenotaph, freshly released on Revision (link me beautiful). So, as for my questions:
The Gameboy's screen resolution is 160*144. I want to scale this up a bit, in part to control the aesthetics, in part to enable YouTube's higher bitrate audio, which is not available if the resolution is too low, and so everything sound like shit.
Since I'm merging the video and audio with AviSynth, I can easily resize in various steps. 144*5=720 so I could very easily just do nearest neighbor *5, and pillarbox the sides to fit 960*720, and call it a day. This would preserve the pixelation. However, I decided to go the fancy route and first scale *2 with nearest neighbor, and then *2.5 (or *5 of the original size) with bicubic. (See video linked above for the result.) I find that this both preserves the hard edges, and gives a smooth feel. I also tried various other combinations, and using something (maybe something with Lanczos resize) produced ringing artifacts which accidentally emulated the looked of a pixel raster for some graphics in the demo.
In short, does anyone sit on a magic formula for the "perfect" upscale of pixel graphics? Opinions on how the current video looks?
Close-up of the current upscale method:
Frame rates and audio desync... My first strategy was to try and match a 44100 Hz samplerate to a 60 FPS (as opposed to the 59.73 Hz vertical update rate of the real thing) This gives you a nice integer ratio of 44100/60=735 audio samples per frame. VLC confirms that this is the case. Awesome.
However, when I uploaded this to YouTube, they decided to do two things:
1) YT drops the frame rate in half.
2) YT adjusts it to not 30 but 29.970029. NTSC-ish. Yuck!
Should I try to embrace teh NTSC-ish framerate, or stick with an integer one? Is it better to trash every other frame myself before uploading? Can I trick YouTube into accepting 60 FPS by, I don't know, uploading a 120 FPS file, or doing something else?
I have not yet tested conclusively whether this framerate change causes desync... (will prepare tests for this at some point.)
How does d.tv handle framerates? Is 60 FPS even possible for Flash video?
And lastly, any GB(C) demos you would like to see captured this way? Any other comments?
So, what I've wanted to do is to use BGB. Lately, beware, the author of the emulator, has made a number of accuracy improvements to BGB. (More accuracy+a lot of improvements to the debugger will come in BGB 1.4 which is still under development. Soon you can say hello to the definitive no$gmb killer.) It will also have "high quality" audio rendering for all 4 channels (meaning here that the waveforms are built up from its partials with an inverse Fourier transform, so you never go above Nyquist.)
Lately, I've been nagging beware to implement some form of video output from BGB. The upcoming BGB 1.4 does have improvements in this regard, even if it's not as easy to use as VBA for video output. The feature lets you output screenshots as BMP files for each frame, and sync this exactly with the audio. The BMP method is a bit lo-fi, but whole thing can easily be reassembled with AviSynth.
So far everything seems ok. I've uploaded a video of Cenotaph, freshly released on Revision (link me beautiful). So, as for my questions:
The Gameboy's screen resolution is 160*144. I want to scale this up a bit, in part to control the aesthetics, in part to enable YouTube's higher bitrate audio, which is not available if the resolution is too low, and so everything sound like shit.
Since I'm merging the video and audio with AviSynth, I can easily resize in various steps. 144*5=720 so I could very easily just do nearest neighbor *5, and pillarbox the sides to fit 960*720, and call it a day. This would preserve the pixelation. However, I decided to go the fancy route and first scale *2 with nearest neighbor, and then *2.5 (or *5 of the original size) with bicubic. (See video linked above for the result.) I find that this both preserves the hard edges, and gives a smooth feel. I also tried various other combinations, and using something (maybe something with Lanczos resize) produced ringing artifacts which accidentally emulated the looked of a pixel raster for some graphics in the demo.
In short, does anyone sit on a magic formula for the "perfect" upscale of pixel graphics? Opinions on how the current video looks?
Close-up of the current upscale method:
Frame rates and audio desync... My first strategy was to try and match a 44100 Hz samplerate to a 60 FPS (as opposed to the 59.73 Hz vertical update rate of the real thing) This gives you a nice integer ratio of 44100/60=735 audio samples per frame. VLC confirms that this is the case. Awesome.
However, when I uploaded this to YouTube, they decided to do two things:
1) YT drops the frame rate in half.
2) YT adjusts it to not 30 but 29.970029. NTSC-ish. Yuck!
Should I try to embrace teh NTSC-ish framerate, or stick with an integer one? Is it better to trash every other frame myself before uploading? Can I trick YouTube into accepting 60 FPS by, I don't know, uploading a 120 FPS file, or doing something else?
I have not yet tested conclusively whether this framerate change causes desync... (will prepare tests for this at some point.)
How does d.tv handle framerates? Is 60 FPS even possible for Flash video?
And lastly, any GB(C) demos you would like to see captured this way? Any other comments?
i didnt notice any framedropping. demo is okish, good fpr gbc i suppose.
whats that with bgb? shame its windows only. i tried running the rom with VisualBoyAdvance version 1.8.0 [SDL] --throttle=100 under ubuntu 10.04 with athlon 64 3000+ and geforce 6600le and the sound was horrible, so i had to resort to youtube.
whats that with bgb? shame its windows only. i tried running the rom with VisualBoyAdvance version 1.8.0 [SDL] --throttle=100 under ubuntu 10.04 with athlon 64 3000+ and geforce 6600le and the sound was horrible, so i had to resort to youtube.
As for scaling, wikipedia gives pointers on Pixel art scaling algorithms:
http://en.wikipedia.org/wiki/Image_scaling
http://en.wikipedia.org/wiki/Image_scaling
The question about scaling up the image was more of an aesthetic question. What do YOU like? 2xSai and similar algorithms are trying to scale up the image and get rid of the blockiness. I personally like blockiness to some degree, and trying to get rid of it completely is not desirable in my opinion.
As for BGB, you should try it under Wine. The author explicitly supports Wine as a way of running the emulator. His objection to porting it natively to another platform (which I also agree with) is that multi platform GUI applications typically sucks on at least one platform. Multi platform GUIs usually take one of the following forms:
1) Linux/*NIX native application. Uses Qt or wxWidgets or similar and looks horrible on Windows.
2) Uses SDL or similar and has no real GUI, only a framebuffer and a sound output, on at least one platform.
So yeah, try it with Wine, and report any bugs. Also feel free to drop by #gbdev on EFNet and I'm sure beware will let you test the latest beta version.
As for BGB, you should try it under Wine. The author explicitly supports Wine as a way of running the emulator. His objection to porting it natively to another platform (which I also agree with) is that multi platform GUI applications typically sucks on at least one platform. Multi platform GUIs usually take one of the following forms:
1) Linux/*NIX native application. Uses Qt or wxWidgets or similar and looks horrible on Windows.
2) Uses SDL or similar and has no real GUI, only a framebuffer and a sound output, on at least one platform.
So yeah, try it with Wine, and report any bugs. Also feel free to drop by #gbdev on EFNet and I'm sure beware will let you test the latest beta version.
I prefer blockiness. Filters are some times interesting, I switch them on emulators just to see how successful they are in different screens, though most of the time I play ScummVm or oldschool emuls I disable these filters.
And for demos, blockiness definitelly. It's better to preserve how the original looks (though it could still look quite different in blurry LCD screens)
I think that capture worked great, good idea for the scaling technique. Felt Gameboy-ish but worked on my 1400x1200 monitor.
Another accurate GBC emulator is Gambatte, which runs on Windows and Linux (although BGB has more accurate colours). BGB is probably the better option for making a video though.
http://img39.imageshack.us/img39/6169/screenshot020811085246.png Side-by-side comparison of BGB and Gambatte
FreeFull: Now I'm speaking on beware's behalf, but he takes pride in getting certain kinds of things right, and color calibration is one of them.
And I'm noticing you turned on the mix frame option. This reminds me once again of the 30 FPS issue. YouTube only allows a maximum of 30 FPS, which kind of sucks. What I should do is some form of frame blending. That demo, Dimension of Miracles exposes a flaw of the GBC LCD more clearly than usual, namely that it is interlaced in some crazy fashion. Something I ought to try, for the sake of realism is to emulate this effect. (Hell, why not. YouTube is going to drop every other frame anyway.)
And I'm noticing you turned on the mix frame option. This reminds me once again of the 30 FPS issue. YouTube only allows a maximum of 30 FPS, which kind of sucks. What I should do is some form of frame blending. That demo, Dimension of Miracles exposes a flaw of the GBC LCD more clearly than usual, namely that it is interlaced in some crazy fashion. Something I ought to try, for the sake of realism is to emulate this effect. (Hell, why not. YouTube is going to drop every other frame anyway.)
Definitely go with nearest neighbor all the way, with an integer ratio, and compress losslessly. Some softness will appear inevitably due to compression and reduced color resolution (no way around it atm due to how H.264 and YouTube work), and if you prefer crispness/blockiness you don't want to emphasize that.
See how the TASVideos guys do that: HD Encoding Guide.
See how the TASVideos guys do that: HD Encoding Guide.
Quote:
wait a minute.... ;)and compress losslessly. Some softness will appear inevitably due to compression
nitro2k01: Which is why in the end I said BGB is the better option for making videos. The reason I mentioned Gambatte was for vectory, in case he doesn't want to run BGB under Wine.
vectory: The reason why sound sounds horrible in VirtualBoyAdvance is that it doesn't use the audio for sync. Mednafen (which uses VBA's code to emulate GBA) does sync with audio and sounds very smooth. I use Mednafen for emulating GBA and Gambatte/BGB for emulating GBC and GB.
vectory: The reason why sound sounds horrible in VirtualBoyAdvance is that it doesn't use the audio for sync. Mednafen (which uses VBA's code to emulate GBA) does sync with audio and sounds very smooth. I use Mednafen for emulating GBA and Gambatte/BGB for emulating GBC and GB.