The reason the enum is so nice is that it works as a terrific language-supplied abstraction that covers up those bit manipulations. It's very nice to get those abstractions for free like you do in Rust, but it can't be optimal for every specialized use case.
This new code also supplies similar abstractions. That actual specific code is much harder to reason about, but most users--and even the next person who works on the interpreter--simply won't care, or even know what is going on underneath the hood. The abstractions provided by the author do that work and apparently do it cleanly.
For most use-cases, that extra hand-written code isn't worth it. But in specific cases it can be, and the author has actually measured the value and determined that it is.
The computer's the one running the code, and it'll be running it a lot (or so its author hopes), so it's probably worth bearing its limitations in mind in the interests of making its life easier (so to speak) rather than prioritising the people who will modify the interpreter - a far less common occurrence.
The bit operations involved are pretty simple and won't take you long to figure out even if you've never done them before.
> As you can see, it has many convenience methods to make it easy to work with, compensating for the loss of the Rust enum.
Also, Rust does try to do some of these optimizations itself. These aren't exposed in the stable language to let you do some more advanced things, but it wouldn't be impossible for you to get the best of both worlds by letting you communicate this stuff more directly to the compiler. Right now those things are more like "this value is where you should put the tag" than the more advanced stuff here, though. Would be cool to see someday!
quibble: unsafe is stable. you can't do this in safe rust, but you can do it in unsafe rust. just isolate all the unsafe code in a single type, ideally a tiny crate.
Take a look at triomphe's ArcUnion and extrapolate from there. Basically make a crate for just your 64bit union type, do it unsafe there, test with miri, and now you have a safe 64bit type you can use with match. You're happy digging around assembly so this is well within your wheelhouse. The only challenge will be if you do use miri to verify then you need to use the 'provenance-preserving' pointer adjusting functions. Worth the learning experience in my opinion. I did one for my system and it was super fun and had the performance impact you describe.
This new code also supplies similar abstractions. That actual specific code is much harder to reason about, but most users--and even the next person who works on the interpreter--simply won't care, or even know what is going on underneath the hood. The abstractions provided by the author do that work and apparently do it cleanly.
For most use-cases, that extra hand-written code isn't worth it. But in specific cases it can be, and the author has actually measured the value and determined that it is.
The bit operations involved are pretty simple and won't take you long to figure out even if you've never done them before.
...unless it's supported by the language as a first class feature. See for example C++ and RVO.
Also, Rust does try to do some of these optimizations itself. These aren't exposed in the stable language to let you do some more advanced things, but it wouldn't be impossible for you to get the best of both worlds by letting you communicate this stuff more directly to the compiler. Right now those things are more like "this value is where you should put the tag" than the more advanced stuff here, though. Would be cool to see someday!