Runtime loading of data
category: general [glöplog]
I'm not sure if this has been mentioned before but:
what is the situation with loading (even saving) data from the disk while the demo is running? is it allowed? I think that while ago it was not - once the demo starts it should not read/write anything to physical discs or the internet.
If this is the case, do you know the reason? what about streaming in video files?
what is the situation with loading (even saving) data from the disk while the demo is running? is it allowed? I think that while ago it was not - once the demo starts it should not read/write anything to physical discs or the internet.
If this is the case, do you know the reason? what about streaming in video files?
Streaming from the internet is obviously out for most compos, but why not streaming from disk? Don't remember seeing that rule, and it's been common since at least the amiga days.
Hmm.. then again.. streaming from the internet. We could just write a video player that streams a youtube HD video of the demo, which is how most people watch it anyway so what's the problem? :D
Hmm.. then again.. streaming from the internet. We could just write a video player that streams a youtube HD video of the demo, which is how most people watch it anyway so what's the problem? :D
I have never read a rule which said you could not stream from disk after the demo has started... writing is different though, but reading? Go ahead :-)
I think the reason it's not done much is because given the "usual" needs for a demo, streaming is just not needed (there's enough memory to hold all resources in memory), and streaming would make the whole thing a bit less performant (or much less, if it's not well done).
Streaming from the internet might be another thing though... there's still a bunch of computers which don't have internet (or hi-speed one, for the matter)... I guess streaming from internet and realtime don't yet mix up too well.
Plus, there are limited data plans around (even for landline DSL's), so not sure if people would want their bandwidth to be consumed for watching a demo many times :-)
I think the reason it's not done much is because given the "usual" needs for a demo, streaming is just not needed (there's enough memory to hold all resources in memory), and streaming would make the whole thing a bit less performant (or much less, if it's not well done).
Streaming from the internet might be another thing though... there's still a bunch of computers which don't have internet (or hi-speed one, for the matter)... I guess streaming from internet and realtime don't yet mix up too well.
Plus, there are limited data plans around (even for landline DSL's), so not sure if people would want their bandwidth to be consumed for watching a demo many times :-)
isnt streaming data from disk commonplace in demos on non-pc platforms already?
smash, think so... i remember many amiga demos which had the floppy disk almost constantly spinning (i "guess" streaming... but i have no clue on how amigas were programmed so that's just a wild guess)
As has already been said, streaming from disk is usually permitted. Writing to disk is often not permitted, but I'm not really sure why.
kusma: i guess that's a rule from the old times where people would make demo compilations on CD (as exe's), which has never been modified since :-)
... and from when there were fears of small demos writing datafiles for other demos to overcome the size limits in party compos :-)
ah ok. I think I got this rule in my head from the days of CDs: that the assumption of a very fast read from a disk was not always correct. That would kill your framerate.
Accessing the internet is a different thing altogether, and I'm sure some firewalls would start moaning halfway through watching, thus spoiling your experience.
Accessing the internet is a different thing altogether, and I'm sure some firewalls would start moaning halfway through watching, thus spoiling your experience.
kusma: i suppose it cant be (or usually isnt) enforced, so i suppose it's more a rule to discourage it from accidentally fucking up the compo machine.
Reading/streaming from disk is no problem and at least BP rules always allowed it (not sure about other parties).
Writing is slightly dicey in that you'd prefer the machine to be in the same state before and after running a demo (just to avoid potential problems when running it a second time), but unless you're stupid about it that's not a problem and BP generally allowed it.
Writing is slightly dicey in that you'd prefer the machine to be in the same state before and after running a demo (just to avoid potential problems when running it a second time), but unless you're stupid about it that's not a problem and BP generally allowed it.
Psionice: Other Nice idea.. presenting a realtime demo in a youtube HD video layout design ;-)
(something like Andromeda did in the early 90's presenting a realtime demo in an amiga dos layout)
(something like Andromeda did in the early 90's presenting a realtime demo in an amiga dos layout)
Quote:
It stems from the days of CD-ROM compilations.Writing to disk is often not permitted, but I'm not really sure why.
I don't see a reason for a demo to write anything down. Maybe selected gfx settings for future runs or performance details for benchmarking, but what else?
I kind of liked the idea Jcl presented, though - copy all the data you want from a large demo to some obscure location and win the next 4k compo by doing amazing amounts of stuff :) A pretty fragile plan, though, as the larger demo won't probably be run on the newer compo machine - and then there are the ethical downsides.
I kind of liked the idea Jcl presented, though - copy all the data you want from a large demo to some obscure location and win the next 4k compo by doing amazing amounts of stuff :) A pretty fragile plan, though, as the larger demo won't probably be run on the newer compo machine - and then there are the ethical downsides.
it was and is allowed at buenzli, no problem with streaming data from disk. it's just that most coders are too lazy to stream properly and waste tons of memory. but in the end, who cares? it is just the demo running, and 90% of the demos don't do magic things and eat up those neat 4gb/1.5gb vram or more in todays machines. oh wait, there are PCs with less memory? but hey, those PCs in the end normally don't have the necessary gfx power because everyone is wasting fillrate, so who cares? and yep, demos are watched on youtube today... (sad thing.)
writing on disk: i encountered that many 64ks 10yrs ago did unpack some files (like xms for minifmod) to disk... or there are demos which unpack to disk because of general lazyness. or write stupid logfiles. (sad thing aswell.)
oh yes, and i did demos that write to disk. and i watch demos on youtube. and i think i never did proper and good streaming, so blame me aswell :)
writing on disk: i encountered that many 64ks 10yrs ago did unpack some files (like xms for minifmod) to disk... or there are demos which unpack to disk because of general lazyness. or write stupid logfiles. (sad thing aswell.)
oh yes, and i did demos that write to disk. and i watch demos on youtube. and i think i never did proper and good streaming, so blame me aswell :)
Quote:
I kind of liked the idea Jcl presented, though - copy all the data you want from a large demo to some obscure location and win the next 4k compo by doing amazing amounts of stuff :) A pretty fragile plan, though, as the larger demo won't probably be run on the newer compo machine - and then there are the ethical downsides.
And that's why we used a separate (but identical) machine for compo preparation on BP, and show the smallest compos first. :)
You could still submit a bunch of junk 4ks that dump a few kilobytes of payload each to extend the size limit for 64ks. :) Preventing that involves wiping the compo machines from a clean image after every compo, which is possible but extremely inconvenient and unlikely to be done in practice :)
Cheating by reading from hard disk? Broncs of course :)
NFO file is additional data. Nice idea!
If you unzip, tons of empty directory junk, their names/structure being the data
NFO file is additional data. Nice idea!
If you unzip, tons of empty directory junk, their names/structure being the data
Ryg, thanks for the useful adv.. information :)
PC demos streaming from disk? Considering that PC demo file size limits were at about 1/16th of the compo machine's RAM for the last 10 years this didn't make any sense anyway. Just map your packfile into memory and you're set (especially on 64bit systems where there's absolutely no reason anymore NOT to use mmio for read only access). Also, in most cases this is faster than any fancy loader code you could hack up.
To top it off, the graphics APIs on Windows weren't exactly streaming friendly (as in: lots of overhead for resource loading and strictly single-tasked) before DirectX11, so I doubt anyone bothered. Perhaps people streamed sound data, but then again, loading a 10 megabytes blob doesn't hurt at all, and the abovementioned mmio gives you "streaming" for "free".
As for writing to disk: There are certain intros or demos that create a cache for all the generated stuff on disk because it's just so much faster to read it back than generate it again (because nomen est omen). That's the one thing I can think of in terms of writing to disk, and it's perfectly ok, even in demo compos (organizers hate long loading times :D).
To top it off, the graphics APIs on Windows weren't exactly streaming friendly (as in: lots of overhead for resource loading and strictly single-tasked) before DirectX11, so I doubt anyone bothered. Perhaps people streamed sound data, but then again, loading a 10 megabytes blob doesn't hurt at all, and the abovementioned mmio gives you "streaming" for "free".
As for writing to disk: There are certain intros or demos that create a cache for all the generated stuff on disk because it's just so much faster to read it back than generate it again (because nomen est omen). That's the one thing I can think of in terms of writing to disk, and it's perfectly ok, even in demo compos (organizers hate long loading times :D).
Quote:
copy all the data you want from a large demo to some obscure location and win the next 4k compo by doing amazing amounts of stuff
...and done
ryg: Couldn't something like Sandboxie be used in theory to prevent applications from writing to disk? I doubt it'd ever be worth the hassle though, I can't remember having seen a cheat like that myself before.
Hm, I guess I should have read Gargaj's post first :(
Yeah, writing isn't the problem, reading is. But there are tools you can use to easily detect both.
Quote:
(i "guess" streaming... but i have no clue on how amigas were programmed so that's just a wild guess)
"Streaming" is a good word for it, yes.
Ryg: at work some time back we used a package called 'deep freeze'. You build your basic install up, then freeze it. From then on, whenever it's rebooted it gets restored to the initial state, anything new or changed is removed.
Can't remember the details too well now, but I remember it being pretty fast, definitely nothing near re-imaging. Of course you'd still have a reboot between demos, but maybe with a fast system + ssd it's doable (and probably desirable incase one demo fucks up something in the system, spoiling the next demo).
Can't remember the details too well now, but I remember it being pretty fast, definitely nothing near re-imaging. Of course you'd still have a reboot between demos, but maybe with a fast system + ssd it's doable (and probably desirable incase one demo fucks up something in the system, spoiling the next demo).