DemoSystem - A simple demo framework
category: code [glöplog]
These are some very good suggestions! Yeah, timeline and the whole pipeline process is pretty well described by Navis. There is a reset button for the timeline, adding cue time buttons can also solve many problems! Also, had never heard of imgui before and from what I've seen it's something really good that I am considering tying the framework to. :)
I can't help but come back to suggesting that if you would've made at least ONE demo with a framework, you'd know about all of this.
There used to be a lot of people who focused on making game engines instead of their first game as well. It's just one of those things.
If it helps, nowadays you can even load the entire demo in RAM at once.
Navis, just wanna share I really appreciate learning how you do the midicontroller + hardcoding + continuous reloading setup. It's so obvious and simple but I never saw it.
Same. :)
The best bit with the midi controller is that you don't need to move the mouse to take focus off your editor and onto your demo application. This saves a ton of time moving from window to window to capture mouse focus (that you need if you are pressing buttons/sliders ala imgui ).
Quote:
midicontroller + hardcoding + continuous reloading
how can I learn more about this?
There isn't that much to learn. It's quite simple:
+Midi: you need a library to communicate. I point you to search for "NanoKONTROL2.h c++". You can buy a nanocontroller (or any other, doesn't have to be from Korg) for around 50euros ?
+Hardcoding: it's not very specific, but it's the idea that you write a lot of the logic inside the shader with lots of if (time>xxx) etc. and interpolations. For example, you can do simple animations with the vertex shader and fades with your postprocessing fragment shader (same in D3D)
+Continuous reloading: just "reload" the shaders from the disk for this part you're actively playing with every 30 frames. You don't want to do it in every frame because it takes a bit of time to compile them, so would spoil your viewing experience while you experiment. 30 frames is a good compromise. Obviously you will remove the "continuous reloading" after you finish with the demo and release. So, while it's playing back and you can "scratch" in time, you can change the code of your shader on the fly and it will be updated in the next shader reload. I usually have a small number of shaders per "part" (even if the part is 200 secs long!) - 4 or 5 is enough. Yes, there may be a conflict while it's "reading" and you are "saving" to disk, but it happens very very rarely and worst case is that your editor will tell you that it cannot save to that filename. Then you try again.
My advice is don't use external files for configuring splines, positions, logic etc because it is just as good as hardcoding if you don't have the right tools to visualize the changes. The right tools for this is the demo itself I guess... so makes no difference. The "human readable" config is your own code. Nobody is going to change it outside you.
Unless... you are working with artists separately to the programmers, in which case my advice is not very sound.
+Midi: you need a library to communicate. I point you to search for "NanoKONTROL2.h c++". You can buy a nanocontroller (or any other, doesn't have to be from Korg) for around 50euros ?
+Hardcoding: it's not very specific, but it's the idea that you write a lot of the logic inside the shader with lots of if (time>xxx) etc. and interpolations. For example, you can do simple animations with the vertex shader and fades with your postprocessing fragment shader (same in D3D)
+Continuous reloading: just "reload" the shaders from the disk for this part you're actively playing with every 30 frames. You don't want to do it in every frame because it takes a bit of time to compile them, so would spoil your viewing experience while you experiment. 30 frames is a good compromise. Obviously you will remove the "continuous reloading" after you finish with the demo and release. So, while it's playing back and you can "scratch" in time, you can change the code of your shader on the fly and it will be updated in the next shader reload. I usually have a small number of shaders per "part" (even if the part is 200 secs long!) - 4 or 5 is enough. Yes, there may be a conflict while it's "reading" and you are "saving" to disk, but it happens very very rarely and worst case is that your editor will tell you that it cannot save to that filename. Then you try again.
My advice is don't use external files for configuring splines, positions, logic etc because it is just as good as hardcoding if you don't have the right tools to visualize the changes. The right tools for this is the demo itself I guess... so makes no difference. The "human readable" config is your own code. Nobody is going to change it outside you.
Unless... you are working with artists separately to the programmers, in which case my advice is not very sound.
Also worth noting that using sliders, dials and stuff is not a recent thing... For example, I remember seeing somewhere the IT guys from Lucas, on the set of Starwars (before the movie opening, so probably during/around 1976). They used dials to make rudimentary, wireframe 3D shapes turn in realtime on computers. This would later help them make the famous "Death Star Run" Animation shown as a simulation in the movie. Cf youtube video : "Making of the Computer Graphics for Starwars."
I checked quickly, and apparently, "Larry Cuba" is the main guy behind these early 3D. And of course, previously he had worked for... John Whitney Senior ! :D
Funny how things are connected.
I checked quickly, and apparently, "Larry Cuba" is the main guy behind these early 3D. And of course, previously he had worked for... John Whitney Senior ! :D
Funny how things are connected.
Btw, all operating systems have ways to detect changes to the filesystem, so you could even build it in such a way that shaders only get reloaded once they are saved (instead of reloading them every second).
Okay, maybe it was not exactly "on the set of Starwars", but in a computer lab... somewhere. Also check Larry Cuba's wikipedia page, there's a VERY cool anecdote about rendering crashes.
sagacity: He could simply check the modified file date and once this changed from the last shader reload he could reload them again. Truth be told when you code something more complex, stuff like these are kinda trivial, as you are trying to solve other bugs, etc.
or just reload every 30 frames without a worry in the world
MIDI is good when your parameters are in a fixed interval - most of the time they're not, though. Also, with shader compilation times like this, better not reload too often.
But then, you could just live reload those too.
My big problem with these choices of advice is that it takes a very specific mentality which I don't think is common.
Quote:
My advice is don't use external files for configuring splines, positions, logic etc because it is just as good as hardcoding if you don't have the right tools to visualize the changes.
But then, you could just live reload those too.
My big problem with these choices of advice is that it takes a very specific mentality which I don't think is common.
"or just reload every 30 frames without a worry in the world"
oh Navis, you sweet summer child <3
oh Navis, you sweet summer child <3
Quote:
MIDI is good when your parameters are in a fixed interval - most of the time they're not, though. Also, with shader compilation times like this, better not reload too often.
(If I understand correctly the problem) - the midi parameters are quantized to values between 0 and 127. This granularity is "good enough" for most problems, for example "The placement of the box on the plane" - 1 cm granularity for every meter. Good enough.
The shader compilation times are ok - I run 4-5 "reloading" of shaders (quite long ones of the order low k's of lines of code for the longest) and I don't "see" the spike in performance when reloading once every 30 frames. I would, if I did it in every frame. I haven't timed it, but I'm sure it's < 100 msec.
It also helps if any errors are displayed on the console window in red color (with line number).
Quote:
But then, you could just live reload those too.
You could, but you create a new problem without solving much else. This problem is called "The unique name of the variable you're changing", and when you potentially have thousands of these things, you will run of imagination for memorable names quite quickly. It's ok if the variable is just "Y Position of the box" but it may be "Random seed for acceleration initial state for splines growing in part 2a, second batch of flowers on the left". Lots of typing too..
In the config file, you have to have some reference. In code, you could just observe what's going on from the surrounding code + comments. So, why offset the problem in the first place... ?
- Yeah but, this way (with external configs) you avoid recompiling
- No, because you still need to code the reference for the value in the external file , for at least one cycle of recompilation. I think it works out the same.
(this is demos we're talking about, non-interactive, make once and forget etc. And, of course, that is my personal take; in no way I claim is the best, just that it seems to suit me)
Quote:
(If I understand correctly the problem) - the midi parameters are quantized to values between 0 and 127. This granularity is "good enough" for most problems, for example "The placement of the box on the plane" - 1 cm granularity for every meter. Good enough.
Maybe, but not for values that are, say, -inf..inf or 0..inf, like cameras or anything HDR. ImGui gets this right by distinguishing between "SliderScalar" (which has a min/max range) and "DragScalar" (which are infinite in range).
Quote:
You could, but you create a new problem without solving much else. This problem is called "The unique name of the variable you're changing", and when you potentially have thousands of these things, you will run of imagination for memorable names quite quickly. It's ok if the variable is just "Y Position of the box" but it may be "Random seed for acceleration initial state for splines growing in part 2a, second batch of flowers on the left". Lots of typing too..
How does this problem even occur though? Even if you don't have automatic per-effect scope for variable names (which you SHOULD), just renaming stuff from "particle_y" to "intropart.particle_y" makes everything more readable, and you can only blame yourself for not doing it. (And every single text editor has autocomplete nowadays.)
And again, this is the hacky method - the convenient method (which I use) would be to use shader reflection to automatically hook out e.g. all artist-driven shader variables to sliders / splines / whatever (again, ImGui can be your friend here), so adding a new animatable (or static, if you want) parameter is a matter of seconds.
Quote:
Maybe, but not for values that are, say, -inf..inf or 0..inf, like cameras or anything HDR. ImGui gets this right by distinguishing between "SliderScalar" (which has a min/max range) and "DragScalar" (which are infinite in range).
You can use the (fantastic) imgui for these values (or what I do is use mouse scroll wheel for one dimension of freedom. Although will take long to reach +infinity just by scrolling).
But I can't imagine what you mean about cameras and HDR - these usually have values that are "reasonable" (lets say meters) or require granularity that is "reasonable". Can you give an example?
Quote:
Even if you don't have automatic per-effect scope for variable names (which you SHOULD), just renaming stuff from "particle_y" to "intropart.particle_y" makes everything more readable, and you can only blame yourself for not doing it. (And every single text editor has autocomplete nowadays.)
This whole idea of mine doesn't work if you share the load with another person (artist/director). If I'm on my own, I find that it is best to use code to describe "on the fly" what I need to rather than reference (even with namespaces etc. I would still need a list of hundreds if not thousands of references). Both approaches work, but your approach is much better when you work with others. Rather, not just much better, mine is not functional at all ! :-)
Regarding the reflection method, that sounds interesting.. can you please give an example (lets say a vertex shader line that moves the object up) -
I would assume:
uniform float PosY;
..
glVertex.y+=PosY;
Can you fill in the gaps of the "reflection"?
Quote:
why does immediate mode get so much shit? I think it is brilliant!
Quote:
It's excellent for simple one-off things like post-processing rectangles, or some debug-rendering etc.
Quote:
we were unable to beat immediate mode + display lists. The NVIDIA driver just does an amazing amount of optimizations under the hood that you can't realistically match with more explicit API usage.
Quote:
"Reload" shaders
Excellent points!
Re display lists: One other interesting aspect is that you just need to execute one single API call (glCallList) for each draw call, cutting down on overhead compared to setting VBOs etc.
Also, NV_vertex_array_range is old but gold as it lets you directly access and manage GPU memory, just like fancy new APIs like Vulkan et al.
Except this extension is already available since 2001..
Since a while now I have this strange urge in the back of my head to just code a GL1 (+extensions) demo again and see how far things can get pushed without shaders :)
Shameless plug: My NVscene 2014 talk "Demo Engine Tricks of the Trade" mentions a lot of these things, plus some more stuff like GNU Rocket, instancing, etc.
youtube video
PDF slides
Fun fact: most OpenGL tutorials including the good old NeHe, introduce the newbie to immediate mode + display lists, and then some of them include shaders and VBOs as stuff for 'advanced coders'.
Then maybe take DemoSystem for a spin?! ;)
Quote:
Since a while now I have this strange urge in the back of my head to just code a GL1 (+extensions) demo again and see how far things can get pushed without shaders :)
Then maybe take DemoSystem for a spin?! ;)
no offence, but I hope after 20+ years of democoding I'm slightly past the NeHe tutorials level now :)
i feel like i spent the last 20 years trying to avoid this way of demomaking. :)
none taken! :) I know for a fact that most people who posted here have a well 20+ experience with the art of demomaking, and I couldn't feel more grateful for every post! Also, I understand most of you try to point me in the right direction on how to do stuff and I'm not offended in the slightest, it just means more work for me, I guess. :)
Quote:
Since a while now I have this strange urge in the back of my head to just code a GL1 (+extensions) demo again and see how far things can get pushed without shaders :)
taht's actually slowly falls down to the "midskool" territory nowadays :) looking down at voodoo2 SLI and glide api docs
