I think soon should be a time to look at performance. Performance should be close to or better than C++ on average. We know about "the root of all evil". But, if there is going to be any large uptake from those using C++ currently (and I imagine a lot of those developers are concerned about performance, memory usage etc), Rust simply has to compete on performance as well.
Go in a way started that path. The whole "systems language" re-interpretation and all. But it never quite became the systems systems language. It can't quite eat C's and C++'s lunch figuratively speaking.
Rust might be able to do it. But regardless of concurrency, type safety and other features if it cannot be fast enough it will have a hard time.
> I think soon should be a time to look at performance. Performance should be close to or better than C++ on average.
The language has been designed from the ground up to compete on this level. Also note that Rust potentially has far greater room for compile time optimizations than C and C++, because the compiler knows far more about lifetimes and ownerships. This can be seen in one of the last items in the release notes:
> Stack usage has been optimized with LLVM lifetime annotations.
We already do pay attention to performance. In general, Rust should already be in the same league as C or C++. Please open issues if you find something that isn't.
And even more important than current benchmark numbers is that the language is explicitly designed to be optimizable, and takes the idea of "zero-cost abstractions" seriously. I can count on two fingers the number of places where we impose dynamic checks by default in the name of safety, and one of those can be unsafely turned off and the other can be safely disabled with proper code structuring.
(Theoretically we can also leverage Rust's ownership rules to automatically generate aliasing information. When implemented this should be an across-the-board speed boost for all Rust code in existence, in a way that C and C++ compilers can only dream of.)
The underlying compiler backend would use the same alias optimization pass for both Rust and C. Rust's advantages here are that all Rust code is automatically eligible for such optimization without the programmer having to do any work at all, and whereas a programmer can get `restrict` usage wrong (or alternately fail to have the confidence that a given pointer is truly unaliasable), the Rust compiler has perfect information.
To be fair, there are also optimizations that C can do that Rust can't, often resulting from C's undefined behavior.
Rust's type system enforces a great deal more invariants in terms of lifetimes, ownership and mutability than C, which enforces virtually nothing. This should therefore give the compiler far more room make optimizations without the fear of changing the semantics of the program. So in the future safe, idiomatic Rust code should be able approach the performance of highly optimized C code, without having to drop resort to unsafe code (note of course that C is implicitly unsafe).
I noticed that in the last round of the language shootout benchmarks that rust was significantly behind C++(and I also noticed that this was being addressed on the rust reddit).
I'd like to see more benchmarking done between rust and C++ where there is a solution for each language that is idiomatic or safe while another solution optimizes for performance.
1. We haven't put in the time for the benchmarks game, because we're too busy working on the language.
2. IIRC, the C++ version uses GMP, and we don't, because licensing. You can install a package if you need GMP, but that doesn't work for the benchmarks game.
> I'd like to see more benchmarking done between rust and C++ where there is a solution for each language that is idiomatic or safe while another solution optimizes for performance.
Much of Rust's safety comes with no runtime overhead, but some of it does.
I thought it won't be more than two times slower. The basic design was supposed to alow the speed of C with the added benefit of safety. But it's much worse:
Most of the 'problems' there are Rust programs not micro-optimised to the ridiculous extent of the C ones, e.g. the C ones almost all use SIMD instrinsics or features like OpenMP.
Rust is a new, experimental language and the support for SIMD is still in development, and we still don't have much in the way of data-parallel libraries. The type system has been expressly adjusted to allow for safe data-parallel libraries though, so they will appear.
Also, a few other points:
- Rust allows for opting in to unsafe code (with rules and overhead similar to C), which can be used for performance optimisations. Last time I checked, none of the Rust benchmark programs used any unsafe. (On the other hand, all of the C programs are effectively using `unsafe` code always)
- C is using highly optimised libraries; Rust has zero overhead FFI, and so can easily call into them, but is not for those benchmark (e.g. the pidigits one is just a benchmark of "are you using GMP?", if the Rust was written to use it, there is no reason it would be slower than C)
- The worst offender (reverse-complement) is apparently written without any thought of performance, e.g. it is doing formatted IO for each and every character it emits, rather than writing to an in-memory buffer and dumping that in one go like the C.
In summary, those benchmarks do not reflect the fundamental speeds of the languages, other than the suboptimal SIMD support in Rust.
All it takes is someone to write a wrapper around gmp providing those parts of the API and it will be less verbose and not require the `unsafe` everywhere.
Timing:
$ /usr/bin/gcc -pipe -Wall -O3 -fomit-frame-pointer -march=native pidigits.c -o pidigits.gcc_run -lgmp
$ time ./pidigits.gcc_run 10000 > /dev/null
real 0m0.980s
user 0m0.976s
sys 0m0.000s
$ rustc --opt-level=3 pidigits.rs -o pidigits.rust
$ time ./pidigits.rust 10000 > /dev/null
real 0m0.987s
user 0m0.984s
sys 0m0.000s
Bravo! It's still a good and important demonstration of what the language features. Gmp is LGPL, so you should add the real safe wrappers (to avoid doing everything unsafe) and allow dynamic linking.
You see, I'm certainly not the only one who wouldn't know all this unless there are good benchmark examples (or have the luck of being able to ask and receive the answer from you. I believe Go couldn't do that once, I don't know the current state.
Thank you. But a lot of people wouldn't get so far. Therefore please do promote good examples of the fast code that do something real, like these benchmarks.
Yes I do prefer one good benchmark over ton of stale manuals. The good written benchmark code should show the best possible aspect of the language: how the code looks when it has to compete in speed too. If the code is demonstrably cleaner and almost as fast as C, then it can be a winner. If it's more than 2 times slower, I don't care, unless it has some fantastic libraries or framework which solves something much easier than other languages.
Fast and looking nice when solving something like a real problem. Benchmarks can demonstrate that. Manuals and references typically not.
Improve the benchmarks and submit them to the shutout. Use the benchmarks during the development too to remain focused on the performace. It's not helpful when somebody in the small circle knows that the benchmark can be implemented better if there aren't any reference examples.
I believe Servo is your reference, but you definitely need to have a bit wider horizons than that. The small benchmarks are good representation of the rest of the world. And the good advertisement, once they are fast. Please don't ignore them.
They are not being ignored. Better versions of those benchmarks exist in-tree: https://github.com/rust-lang/rust/tree/master/src/test/bench (the shootout-....rs). The only reason they're not on the website is licensing problems (which have recently been resolved) and the fact that the benchmarks game is using the 0.11 release (while the in-tree benchmarks are kept up-to-date with the master branch and so don't necessarily compile with 0.11); this new release represents a good chance to push them up.
Those small benchmarks are not good representations of the rest of the world; they are contrived and limited problems, with some fairly arbitrary rules about which languages/implementations are valid to include, e.g. PyPy is not allowed, and Java gets JIT warm-up time etc.
The site could be renamed to be "The Computer Language Implementation Benchmarks Game", since it's not testing the language speed (best approximated by the fastest known implementation), just certain implementations, some of which are designed with priorities above speed in mind (e.g. cPython).
> Java does not get JIT warm-up time! Please stop making up misinformation!
Oh, sorry! I must've been misremembering something someone told me. (Although, how does `Cold` differ from `Usual` there? It's not clear from the text what the difference is.)
> best approximated by the fastest known implementation
Best not to become so confused: " Measurement is highly specific -- the time taken for this benchmark task, by this toy program, with this programming language implementation, with these options, on this computer, with these workloads."
(Seems like a better way to test would be to allow anything, with three sets of numbers. One for "time from source to as far as you can go without input" (i.e. compilation time, loading the source into RAM, that sort of thing) "time from input to end of first run", one for "time for nth run", with n being high enough that the timing settles. So after any JITters are done, that sort of thing.)
I think soon should be a time to look at performance. Performance should be close to or better than C++ on average. We know about "the root of all evil". But, if there is going to be any large uptake from those using C++ currently (and I imagine a lot of those developers are concerned about performance, memory usage etc), Rust simply has to compete on performance as well.
Go in a way started that path. The whole "systems language" re-interpretation and all. But it never quite became the systems systems language. It can't quite eat C's and C++'s lunch figuratively speaking.
Rust might be able to do it. But regardless of concurrency, type safety and other features if it cannot be fast enough it will have a hard time.