Judging code
category: general [glöplog]
Quote:
contrary to graphics and music, code is
something we don't have any physical experience of: we only have access
here to stimuli which are quite remote to what the code looks like in
reality:
The code can be fast, or slow. It can be bloated or tight. It
can be sophisticated or very simple in design. Those objective technical
qualities of code are not directly related to how enjoyable the
stimuli (music + visuals) that it produces are.
And due to that, there is no other sane way to me to judge that
hypothetical demo's code than by reframing the technical judgement into
something more aesthetical, for example by judging the resulting
"beauty" of the visuals and music.
I'll add that I believe painting "suffers" from this aspect of the
technical gesture being remote to the resulting stimuli. And that just
about any human creation is subject to that.
This is something very trivial to say, as we all know it, but in my mind
it serves as an example that the "technical" judgement is just an
illusion for what still amounts to an aesthetical judgement. It does
feel technical because you know the pleasing stimuli sufficiently well
to extrapolate from them into how skillful the original creator was when
creating the work.. The hint here is that you associate:
Code:skill <=> pleasing stimuli
The technicality being in how the creator produces those pleasing
stimuli, the illusion being in how universally pleasing or
interesting those stimulis are.
Thats true of non-size limited PC productions. But, for example, most Amiga sceners "know" what sort of effects you can get out of an A1200/060, and its pretty clear when someone comes along and pushes the boundary further than you've seen before. Most Amiga coders will know when an effect they're seeing is really hard to do on an Amiga. These are all judgements based on technical rather than aesthetic qualities.
Another example is the rendered tunnel in one of TBLs demos; it was pretty clear to coders that it was an animation mapped onto a skybox. So while aesthetically it looked amazing, you could judge that it wasn't actually as amazing as you might first think.
Size limited productions are also judged on both technical and aesthetic merits, since they are a compromise between size and aesthetic quality. Blueberry, Stingray and Loaderror have all really been pushing the boundaries of Amiga 4Ks, for example. Every year the number and quality of effects squeezed into that 4k get better and better. Sure you may use a purely aesthetic judgement "this years 4ks look better than last years", but you can also say "hey, the soft synth has feature x, y, z. that voxel engine is amazing for 4k, they've implimented speech synthesis..." etc. etc.
Another example is the rendered tunnel in one of TBLs demos; it was pretty clear to coders that it was an animation mapped onto a skybox. So while aesthetically it looked amazing, you could judge that it wasn't actually as amazing as you might first think.
Size limited productions are also judged on both technical and aesthetic merits, since they are a compromise between size and aesthetic quality. Blueberry, Stingray and Loaderror have all really been pushing the boundaries of Amiga 4Ks, for example. Every year the number and quality of effects squeezed into that 4k get better and better. Sure you may use a purely aesthetic judgement "this years 4ks look better than last years", but you can also say "hey, the soft synth has feature x, y, z. that voxel engine is amazing for 4k, they've implimented speech synthesis..." etc. etc.
!=, I agree.
xeron, it's true that you judge an effect on a basis like:
"that's a difficult effect to do" or "hm, i don't know how they do that" - and yes, that also goes for things like "well they sure managed to fit a lot of stuff inside those x kilobytes".
However, all those kind of things are still pretty much a drop in the ocean if you compare to all the things you can't know about the code from simply watching a demo.
Actually this is one of those annoying scene-related things - I mean, in the real world people tend to do programming also. I know a lot of sceners probably will be surprised by hearing this, but it's true. And what's even more surprising is that the kind of things valued in the scene isn't really
the same as in the rest of the world. A lot of sceners tries to judge code by characteristics like cleverness, flashyness, speed and size. Most of these things aren't normally what programmers tend to favor when judging code. Things like clean & general design, simplicity, extensibility are factors most programmers use
to judge other people's code. Note that this is pretty much impossible to know from simply watching a demo. Besides, cleverness and extreme optimization are usually characteristics most experienced programmers use to judge code as bad (eg "premature optimization is the root of all evil" - D. Knuth).
Also, a stable/unstable demo is a pretty clear indicator of how the code is written - but most sceners also tend to ignore this factor when trying to conclude whether the code is good or bad.
Ok, enough ranting I guess :)
On a side note I personally think that the whole effect/code/etc-obsession in the scene tend to strangle a lot of the potential artistic qualities in demos, which is sad sad sad.
xeron, it's true that you judge an effect on a basis like:
"that's a difficult effect to do" or "hm, i don't know how they do that" - and yes, that also goes for things like "well they sure managed to fit a lot of stuff inside those x kilobytes".
However, all those kind of things are still pretty much a drop in the ocean if you compare to all the things you can't know about the code from simply watching a demo.
Actually this is one of those annoying scene-related things - I mean, in the real world people tend to do programming also. I know a lot of sceners probably will be surprised by hearing this, but it's true. And what's even more surprising is that the kind of things valued in the scene isn't really
the same as in the rest of the world. A lot of sceners tries to judge code by characteristics like cleverness, flashyness, speed and size. Most of these things aren't normally what programmers tend to favor when judging code. Things like clean & general design, simplicity, extensibility are factors most programmers use
to judge other people's code. Note that this is pretty much impossible to know from simply watching a demo. Besides, cleverness and extreme optimization are usually characteristics most experienced programmers use to judge code as bad (eg "premature optimization is the root of all evil" - D. Knuth).
Also, a stable/unstable demo is a pretty clear indicator of how the code is written - but most sceners also tend to ignore this factor when trying to conclude whether the code is good or bad.
Ok, enough ranting I guess :)
On a side note I personally think that the whole effect/code/etc-obsession in the scene tend to strangle a lot of the potential artistic qualities in demos, which is sad sad sad.
It seems code in the scene is valued based on two things:
1. how small it is
2. how fast it is
In truth, I think these days #2 has much, much more value than #1 to the scene. The original definition I got of a scene demo was something that "did things you thought the hardware was incapable of doing." This kind of effect has real utility to a demo because it means the same machine can make a more beautiful picture.
I imagine that at some point in the past, #1 (code size) served the same purpose because disk/memory space was so limited. However, memory has increased at a much faster rate than speed, and on a modern system it almost always makes sense to make the space/time tradeoff in favor of time. (In other words, if you can write a much faster algorithm by using more data, you will.)
Tiny demos are a testament to this era of limited memory (not that anyone was ever limited to 32b of disk...) but I think these days they do more harm than good to the scene. A kid who doesn't yet know about the scene isn't going to realize why a 64k demo is cool - he's just going to see a crappy looking demo with lots of jaggies and decide to go play Half Life 2 instead of starting to code demos. :P
So, for the good of the scene (in my opinion), if you want criteria for judging code which will benefit the scene, ignore size and pay attention to efficiency. You may also want to ask the question, "Does this do something I couldn't do w/ a similarly sized video?" because otherwise it just makes sense to make megahuge demos...
My 2c.
1. how small it is
2. how fast it is
In truth, I think these days #2 has much, much more value than #1 to the scene. The original definition I got of a scene demo was something that "did things you thought the hardware was incapable of doing." This kind of effect has real utility to a demo because it means the same machine can make a more beautiful picture.
I imagine that at some point in the past, #1 (code size) served the same purpose because disk/memory space was so limited. However, memory has increased at a much faster rate than speed, and on a modern system it almost always makes sense to make the space/time tradeoff in favor of time. (In other words, if you can write a much faster algorithm by using more data, you will.)
Tiny demos are a testament to this era of limited memory (not that anyone was ever limited to 32b of disk...) but I think these days they do more harm than good to the scene. A kid who doesn't yet know about the scene isn't going to realize why a 64k demo is cool - he's just going to see a crappy looking demo with lots of jaggies and decide to go play Half Life 2 instead of starting to code demos. :P
So, for the good of the scene (in my opinion), if you want criteria for judging code which will benefit the scene, ignore size and pay attention to efficiency. You may also want to ask the question, "Does this do something I couldn't do w/ a similarly sized video?" because otherwise it just makes sense to make megahuge demos...
My 2c.
@mrtheplague
The point of tiny intros was never really to do with limited resources; its a purely artificial limitation, and one which many coders enjoy trying to work with.
Tiny code is pure c0d0rpr0n, and there are many of us who enjoy them because of it. I don't think they're harmful to the scene... i mean the scene is already dead, isn't it? ;-)
The point of tiny intros was never really to do with limited resources; its a purely artificial limitation, and one which many coders enjoy trying to work with.
Tiny code is pure c0d0rpr0n, and there are many of us who enjoy them because of it. I don't think they're harmful to the scene... i mean the scene is already dead, isn't it? ;-)
Quote:
A kid who doesn't yet know about the scene isn't going to realize why a 64k demo is cool - he's just going to see a crappy looking demo with lots of jaggies and decide to go play Half Life 2 instead of starting to code demos. :P
This is very, very wrong. Of all the people I've seen getting interested in the scene recently, the vast majority has been attracted to the small size and relative easyness of kkrieger and Farbrausch/CNS intros. The rest comes afterwards.
Quote:
So, for the good of the scene (in my opinion), if you want criteria for judging code which will benefit the scene, ignore size and pay attention to efficiency. You may also want to ask the question, "Does this do something I couldn't do w/ a similarly sized video?" because otherwise it just makes sense to make megahuge demos...
So, for the good of the scene we need to look at 64k intros and ask ourself the question: "could I do this with a 64k video?"
Quote:
So, for the good of the scene we need to look at 64k intros and ask ourself the question: "could I do this with a 64k video?"
Sorry, I meant that criterion as an upper bound, not a lower one. It's related to a different complaint I have about the scene which is
WHAT IS THE POINT OF A R-E-A-L T-I-M-E DEMO IF IT'S NOT INTERACTIVE, AND LOOKS IDENTICAL EVERY TIME YOU RUN IT?????????????????????
Ahem.
xeron: yeah, I think your point is well taken - small demos are just part of what the scene is... the dead, rotting scene. but it's culture and history and should be respected nonetheless.
On the other hand...
By my previous post you might infer that I'm an iconoclast. =)
On the other hand...
By my previous post you might infer that I'm an iconoclast. =)
not to forget, a lot of people judge code on what they believe to be impressive based on what they are told is impressive. (which is often a bit apart from reality.)
shouting about fx or your tools still works. :)
shouting about fx or your tools still works. :)
I guess most people simply judge coders depending on how they enjoyed their productions. Nothing else.
I believe it depends on what you think the scene could/should be. I would like to think of the scene as an electronic DIY art-community (a bit like the punk-scene, or the graffiti-scene).
Anyway, all of these scenes have had different hang-ups (like the whole code/effect-issue
in the demoscene or the respect-the-roots-thing with graffiti) and although they may have
positive effects after a while the whole thing usually get so inbred.
For example, I think artificial limits (like, size for instance) might have a point if it leads to a more intresting end-result, but not as an end itself.
Really stupid example: "I'm going to paint a painting just using one color" :) IF that would lead to a nice painting, great - if not, well - why do it?
In other words, I think something like "Well, this sure looked pretty ugly and was generally horrible but since it's only 4k I therefore think it's good" is utterly pointless.
In sports, the artifical limit is a part of the whole purpose - but I don't see the scene as an electronic equivalent of football.
Anyway, all of these scenes have had different hang-ups (like the whole code/effect-issue
in the demoscene or the respect-the-roots-thing with graffiti) and although they may have
positive effects after a while the whole thing usually get so inbred.
For example, I think artificial limits (like, size for instance) might have a point if it leads to a more intresting end-result, but not as an end itself.
Really stupid example: "I'm going to paint a painting just using one color" :) IF that would lead to a nice painting, great - if not, well - why do it?
In other words, I think something like "Well, this sure looked pretty ugly and was generally horrible but since it's only 4k I therefore think it's good" is utterly pointless.
In sports, the artifical limit is a part of the whole purpose - but I don't see the scene as an electronic equivalent of football.
!=, is this hidden opensource propaganda? :)
sofokles: i wholeheartedly agree.
Where did the original quote come from, out of interest?
sofokles got it right...
sofokles got it right...
Quote:
shouting about fx or your tools still works. :)
added on the 2006-07-20 by smash
We noticed, yeah ;D
But to stick to the subject, code is judged the same way as anything else really.
Let's assume you're not a musician. You hear an ASD soundtrack and simply decide if you want to hear it again or not - it's a subjective taste issue on a first impression.
If you're a musician, however, you are able to judge the technicality and effort behind the song and thus be able to say "I don't like it, but it's still awesome".
Same goes for graphics.
Hence, when you see a piece of code working, you can be a coder who has different aspects in evaluation, and you can be a non-coder who judges the end result.
Let's assume you're not a musician. You hear an ASD soundtrack and simply decide if you want to hear it again or not - it's a subjective taste issue on a first impression.
If you're a musician, however, you are able to judge the technicality and effort behind the song and thus be able to say "I don't like it, but it's still awesome".
Same goes for graphics.
Hence, when you see a piece of code working, you can be a coder who has different aspects in evaluation, and you can be a non-coder who judges the end result.
Finally an interesting thread at Pouet!
I second Optimus!
gargaj: ironic that you said that, because i had a personal policy of NOT shouting about tools or fx because it's bullshit and i want people to judge what they see, not what they're told.
In my opinion, the artificial size limits do have a point since they do direct the intros towards more algorithmic based approaches compared to demos which are biased towards (ab)using premade source data. And becuse of this there is a noticeable difference when comparing demos and intros. Besides, the size of code for demos doesn't relly have any relevance because in any case usually over 90% of the release size is taken by data and this leaves just intros to show code compression skillz.
As for judging code based on efficiency, I found it mostly irrelevant as long as the demo runs with a decent framerate on my machine. It's not like I would be checking what framerate the demo runs, so the only thing I usually notice is if it runs really slowly when it really shouldn't or vice versa.
The things that actually impress me about code are when the demo sets a new standard for something by doing something that I personally thought impossible, applies some new algorithms or manages to combine common techniques in a novel way. And the final result doesn't have be aesthetically great, but naturally that does help to cause a bigger impact.
As for judging code based on efficiency, I found it mostly irrelevant as long as the demo runs with a decent framerate on my machine. It's not like I would be checking what framerate the demo runs, so the only thing I usually notice is if it runs really slowly when it really shouldn't or vice versa.
The things that actually impress me about code are when the demo sets a new standard for something by doing something that I personally thought impossible, applies some new algorithms or manages to combine common techniques in a novel way. And the final result doesn't have be aesthetically great, but naturally that does help to cause a bigger impact.
As you may know, I've been hosting a size-optimizing competition series since 1998. This is a way how you can objectively determine a coder's skills - you simply compare the size of his program in bytes. However, of course this measures just a single skill, size-optimizing. Other things such as knowledge at graphics programming are probably more difficult to measure. Let alone the skill of arranging graphic effects in such a way that people will enjoy the show.
Moreover, it's actually impossible to say who's the best coder. I guess every coder has different strengths and weaknesses. E.g., I may be not that bad at size-optimizing, but I have next to no knowledge of graphics programming (it has simply not interested me so far), which is why I've never coded a demo and why I've never appeared in a coders chart of a demoscene-diskmag.
Moreover, it's actually impossible to say who's the best coder. I guess every coder has different strengths and weaknesses. E.g., I may be not that bad at size-optimizing, but I have next to no knowledge of graphics programming (it has simply not interested me so far), which is why I've never coded a demo and why I've never appeared in a coders chart of a demoscene-diskmag.
Well, if you check some demoscene coders charts or most praised ones (without trying to lower the qualities of these people), most of them are ranked as high as they are due to the work of at least one excellent designer / modeler or whatever.
Ie. take conspiracy & remove zoom and his contributions, what's left ? (or for that matter remove the work of gizmo & fiver2 from farbrausch productions...).
Extracting the code element from demos, intros or whatever to judge it independently is a rather pointless exercise as these productions are the results of collective effort where each part is made to fit with the other one (more or less), only the music may eventually be a stand alone crafting.
Ie. take conspiracy & remove zoom and his contributions, what's left ? (or for that matter remove the work of gizmo & fiver2 from farbrausch productions...).
Extracting the code element from demos, intros or whatever to judge it independently is a rather pointless exercise as these productions are the results of collective effort where each part is made to fit with the other one (more or less), only the music may eventually be a stand alone crafting.
hitch: and in some cases coders are highly ranked+respected in a big part because they are an excellent designer themselves.
True (but rare ;D).
hitch: no way. think of guys like curly, navis, unclex, memon.. :) excellent designers as well as coders.