pouët.net

Rust started to be somewhat popular in the demoscene.

It is of public knowledge that Rust is used by Ferris for most of Logicoma demos ( see his presentation of their demotool Demotivation ) and also by cpdt of Monad for ( at least ) their synth ( see Axiom ).

In the spirit of the Tiny Intro Toolbox Thread, I would like to start a thread where we could share our knowledge and resources about Rust, especially in the specific context of the Demoscene.

All contribution welcome :) !

I have a first question: For people making size limited demos with Rust, how are you dealing with #![no_std] ?

In my experience, it has been a big pain: Only available in nigthly (although that may have changes), recurrent breaking change, and things like #![feature(start)] not being supported on various platform.

But since some of you made it work, I am curious on how you did it.

I'm trying to do something, really really slowly, using wasm + rust, ideally I want to use most of the same code to target both the web and desktops (windows most likely).
.

It's a pretty good language. Better than C++, worse than C... Fucking awful learning curve, at least for me, but I'm having tons of fun with it, it kind of forces you to write better code, instead of crappy single use code.

First off, I believe blueberry and emoon are also using Rust extensively. I'm sure there are many others.

Second, The bulk of our code is in the tooling, which doesn't have any of these concerns (minus the engine/JIT compiler which is a crate that both the intro replayer and tool use, but that's actually somewhat small/straightforward). For the stuff the replayer uses, we build everything on nightly and use no_std and some core. Generally this hasn't caused _that_ much pain; we have our own Vec and fill in some iterator stuff, we use a custom allocator (both in the intro and in the tool), we do the typical lang_items shims, etc. We only target Windows with this code (we used to target mac as well for the parts that were used in the tool, but this required pretty minimal stuff and I dropped it when we moved to OpenGL 4.3 for compute). We have of course run into times where bootstrapping and interfaces have changed from beneath us; typically we've handled this by documenting the specific rust version we build with and stick with that for a bit (usually until an intro is finished) and then upgrade afterwards. I believe we did this 2-3 times; things seem to get more and more stable though, and our binaries get smaller and smaller (not accounting what we get from our packer, which is also rust [with some asm], but doesn't need to care about this stuff ofc).

I agree with this only insofar as FFI/bindings are concerned, i.e. interfacing with other C code and/or hardware directly. In my experience it's generally far better than C/C++ (our core tool/compiler code is really very straightforward, especially the threaded parts), but I'll admit having the replayer/engine parts in C (which interface OpenGL more than anything else) is a bit verbose in some places (though it's nice to have the same engine code in both places, rather than a separate replayer beyond just some bootstrapping).

It's difficult to say "we did it like this" and list everything exhaustively as our codebase has grown pretty organically and side-by-side with Rust over the past 3 years (or so), so beyond releasing source (which I'd like to do but won't happen any time soon) I can't really give a detailed list, but I'm happy to answer specific questions if you have them.

should read "having the replayer/engine parts in Rust instead of C"

Probably also worth mentioning we have our own Win32/GL bindings. These are also pretty straightforward (essentially just function lists and copied constants), albeit somewhat verbose. In some sense though I like to know _exactly_ what we're pulling in, so I can't really complain about that :)

I consider cleaner syntax to write more reusable code very subjective argument to fav Rust.

Learning curve may be confusing and that's the main reason why I dropped it so far to focus on clean and reusable, modern C++ code myself.

I see from above that it may be interesting for small exec/intro coding.

For this use-case is there any particular advantage provided by the platform or it's all driven by individual experience?

My thing with C has to do with the simplicity of the language, it's easy to know pretty well what all the language features do, while some things in Rust are not so transparent, like knowing that Option<Box<T>> actually translates to a single pointer. And it's hard to force Rust to do some things, when it assumes that whatever you're doing will cause an error, race conditions, pointer aliasing, or whatever, but that might have to do with my lack of experience in the language, considering that I first started compiling C code more than 15 years ago... I kind of gave up learning "modern C++" a while ago, since I'm not using it professionally for a while now, and I'm hoping to use Rust professionally at some point. Hopefully next year if we get a series A and don't go to startup heaven.

For now, I do agree that the simplicity of C make it attractive to do the "low-level" stuff (like calling win32 api). But for the rest of the code, I find that Rust free you from a lot of cognitive overhead. I especially prefer the Haskell-like trait system. Templating and OOP in C++ is a living nightmare for me.

Hey Ferris :) .



That's very nice, I am very curious about a lot of aspect of Demotivation, glancing over the source one day would be awesome !

By the way, how does your custom allocator works ? Is it similar to something like wee_alloc ?

I noticed you had a in your tiny-gl crates. Is it full Rust ? I did a crate to do this but it's basically a wrapper to the specific part of glad that do the pointer loading (gl_loader).

Doing a JIT compiler in Rust is not too complicated ? It glanced over a few implementation and it was quite scary x) .

Practically speaking, where do you need templating so much that you cannot avoid it today with a different pattern considered as a best practice?

I personally don't do templating too much these days.
Not needed beside some generic and totally not optimized cpu version of the core math with vectors and matrices in it..

But then I realize that my default precision for instanciating them is the only compile time implementation I ever use...

I think it's certainly subjective as well. Rust matches very well the way I like to think/simplify things; there's a reason why I picked it up in the first place and generally I've felt very little friction with the language. This is totally ymmv.



I think this is a bit of a mix. For the actual small parts of the intros we do, I'd argue Rust is generally slightly _worse_ than C, as what we're doing is fairly sane/small and it gets a bit tiring writing `unsafe` everywhere perhaps. But it's not _that_ bad either, and results in the ability to use the same engine code in the intro and in the tool. And I think we get a _huge_ benefit in terms of the tool code, especially with our compiler, threaded code (mainly for file watching/asset reloading), data model, and the ability to pull in/manage external libraries (which we use for font/model loading, project [de]serialization, clipboard interaction, parsing, ...). For UI, we expose a C API to the core data model code, and consume that from a minimal Qt/C++ layer, and this works extremely well, though it can be somewhat verbose. We never have crashes, memleaks, and rarely bugs (when we do it's typical OpenGL state leaks or user errors with the JIT, which doesn't protect us from unlimited recursion by design; nothing that breaks our projects or anything), everything is fast and snappy. Generally I'm very happy with this.

Keep in mind that I don't really care who uses Rust or anything, so if it sounds like I'm trying to sell something, please ignore that and insert appropriate grains of salt where necessary. I'm just very satisfied with our approach. :)

For some other questions...



Yes, this is very nice, although since our data model is relatively straightforward I actually don't think we use many of these features very often (and when we do it's in the tooling code).

The big exception is `Drop` ofc; we use RAII in a _lot_ of places to make GL state leaks a lot less common, and to eliminate a lot of manual memory management.



The custom allocator is _very_ simple. At the lowest level we have `malloc`/`free`/`calloc` implemented on top of `HeapAlloc` and `HeapFree`, and some aligned versions of these that we implement on top of those (mainly to please SIMD code). We use these internally to implement various collections in the engine directly. Of course in a 64k we're probably being a bit more conservative than we have to be here (though I'm starting to suspect we can just impl `free` with `unreachable_unchecked` and maybe the compiler works its magic and saves a ton of space for us... hehe).

Anyways, that covers code that needs those functions directly - to make Rust use the same allocator, we implement the `GlobalAlloc` trait as such:




..and in the intro and in the tool, we use the following code to instantiate it and make Rust use it:



That's literally the entire thing (minus the impl of `malloc_aligned` and `free_aligned` ofc, but I'm sure you can figure those out).



:) (this can surely be cleaned up a bit)



Nah, ours is not that bad. See this for some early tests (and even how we used to make it work for mac too). It's more or less the same still (which reminds me, need to optimize that some more too, but it seems quite small/packable already...)

Should probably mention that `tiny_alloc` referred to above is our internal crate which exposes `malloc`/`free` etc as I described above, and that's it.

Also bear in mind that a JIT is only as complicated as the opcodes you're JIT'ing; ours stays relatively simple because they only do a handful of different kinds of things. We only need to support a very basic "language" to manage the CPU-side GL stuff and glue together engine bits/assets/sync tracks. So it's not a huge leap from a simple interpreter (and I don't really have a great reason for making that leap either; it sounded fun at the time and totally was, so).

It's all right. I liked your comments. Thanks for sharing.

I wrote an 8k intro for linux in rust. I have the source available here if people want to see my no_std setup and the like: https://github.com/aleksanb/fourkay. It’s a real nice language to write in, although juggling raw pointers can be a tad difficult at first when you’re trying to use C APIs that expect pointers to uninitialized variables on your stack and rust really doesn’t want you to do that stuff. I’d never be able to write this stuff in C without segfault galore though, so that’s another advantage with rust for me.

The VST plugins for Oidos and Cinter are written in Rust, using the vst crate. An interesting feature of this crate (which I was also heavily involved in) is that it is structured in such a way that it is impossible, thanks to the Rust type system, to run into data races even when the VST host calls methods in your plugin from multiple threads concurrently.

For many years, I longed for a good systems programming language with modern syntax, properly well-defined semantics and a strong, expressive type system. Rust came along and was exactly that, and it was even better than I could have imagined.

I would say that Rust (or at least Safe Rust) has an honest learning curve. It has one extremely complex feature (the lifetime system / borrow checker), one moderately complex feature (the trait-based type system), and the rest are actually pretty straightforward. And after the compiler has accepted your code, there are usually very few surprises.

In contrast, C++ is deceptive in its apparent agreeableness. You can quickly learn to use it, and then you can continue learning the language for decades and still run into syntax you don't understand and runtime gotchas you didn't know you had to be aware of.

All of this <3

Holly molly thank you a lot Ferris <3 .

<3

Exactly what Blueberry said. For our Amiga stuff we don't use Rust for the run-time (Amiga side) but I use it quite a bit for various tools, prototypes and such.

RUST looks to me like great exotic niche with a heavy competition in the market, I'm curious about it - for a hobby project, yes - but I would not deprecate C/C++ for it.

Back in the historic times I considered C++ very much as compiler driven experience. At basic level it was all the same, but then all implementations were completely different dev platforms with different tactics to get most from it.

That was applied to tools/compiler itself, language features and most critical standard library, that was a fiction. Higher level framework like those for GUI were also numerous to prove immaturity of the market and limitation of the system libraries.

For demoscene/low level related use cases I ended up creating my own std libraries for most of the things in DOS to maintain control and I was wrapping like crazy in Windows to achieve the same.

But this is a story from 95-2005..

Then I forgot about C++ for 10-15 years in favour to various managed platforms, then to rediscover C++11/14/17/20.. modern approach, quite recently.

And I must say, it's very satisfactory and finally quite unified and consistent experience.
But I also agree it's for experienced developers that understand how to maintain simplicity in the code base.

C++1x != simplicity in code base

:wave:

I have been observing and even experimenting Rust, but for now C++ is still my default choice. Might or might not change in a few years.