C++ is a very fast language for numerical and scientific computing, however first compilers were not efficient. By mid-90s it caught up with fortran in speed, with some c++ libraries (valarray mentioned) performing still slow ([1], [2]).

More recent benchmarks show that - depending on the implementations (and libraries) you use - C++ can be faster than fortran ([3], [4] includes python). Of course, benchmarks are often skewed. A recurrent criticism is that the implementation for one language is sub-optimal. This is where the Computer Language Benchmark Game comes in. They define different problems from scientific computing and ask people to submit their code in about 30 different languages. They check the programs for correctness, benchmark the code, and publish the best results for each language. C++ is one of the best languages compared. Maybe not surprisingly java also fares pretty well. Bytecode compilers for java have been improving dramatically since 1995. While there have been claims that java code would outperform C++ it seems that C++ code is still faster [5].

Now, for C++, if there are big differences in speed between libraries, which one to use?

Many of them use wrappers for LAPACK, ATLAS, and other linear algebra packages or are otherwise optimized for speed.

You can define vectors and matrices in C++, however they do not provide a lot of comfort. You need to run loops and index the old way and do bounds checking. Of course, there is GNU Scientific Library (gsl), written like much GNU software in C, although there exist several wrappers for C++. The GSL syntax is very tedious and counter-intuitive for people used to algebra software such as Matlab/Octave or R. For example, instead of writing a(i)+=b you have to use this wordy statement to add scalar b to float vector a at position i:

gsl_vector_float_set(a,i,gsl_vector_float_get(a,i)+b)

Adding of vectors or matrices C=A+B is not even implemented. Instead there is A+=B [6]. In GSL matrix addition C=A+B is like this:

**void** gsl_matrix_Add( gsl_matrix* C,**const** gsl_matrix* A, **const** gsl_matrix* B ){

gsl_matrix_memcpy( C, A );

gsl_matrix_add( C, B);

}

I found two packages for numerical computing that look very neat and boasted with efficient implementations: Armadillo and IT++. Armadillo seems to be the project of mainly one Australian researcher and IT++ is a project of Chalmers University, Sweden. Both libraries are very easy to install on linux systems and both projects give example codes. The presentation of armadillo is excellent and IT++ gives a conversion table for matlab code. Armadillo is the newer project, so it's maybe not as mature (?), it doesn't have the many methods for signal processing that IT++ has, however the project claims it is much faster than IT++.

I installed both and made some attempts at benchmarking. I tried to compare IT++, Armadillo, and GSL for double matrix addition. I tested all three for addition of quadratic matrices with sides of 1 to 2000 (step size 100), doing 1000 repetitions. This gave nice plots of matrix size against seconds. First it seemed armadillo was fastest, however there was a lot of variation across trials for all 3 packages, so I think, results are not conclusive.

What I did however conclude was that all were surprisingly fast. I found especially Armadillo very easy to use and it also offers a very good online reference. Armadillo offers import/export to text files. IT++ uses its own binary format, but they offer a matlab script to read the data. You can actually use both packages together. Armadillo offers a library for conversions to and from IT++ C++ matrix and vector formats. It's my favorite of the three.

Hi,

ReplyDeleteVery happy to read your article and thank you for many interesting references. I'm also doing my research heavily relied on the speed of numerical libraries. I also wrote an article of an on-line survey of these libraries. (Please see this) "Eigen" may also be an interesting one for your purpose and it is also under active development. "Easy to install" is also one of my concern since the users may not be as geeky as the code writer and they may easily give up evaluating my code just because the installation is not as smooth as they expect. So IT++ is ruled out by this criterion for its many dependencies. Finally I go back to use Lapack directly for its ubiquitous acceptance and its ease to avoid license problems (another important issue) although the interface really stinks.

@Eggp: Thanks for your comment. I am happy this post is useful to somebody.

ReplyDeleteEigen looks good indeed from its tutorial. I think the ease of use is very similar to armadillo. Maybe I'll look more into it.

I saw in your post a pointer to a comparison of different C++ libraries (PDF), which I found especially useful for reference.

When I tried it, I thought IT++ had a little steeper learning curve than armadillo, but it was also quite good to use and very easy to install (at least in ubuntu, where it comes as a package).

Using Lapack directly I think doesn't make much sense. There are so many great wrappers out there, which are easy to install. I didn't look it up for Eigen, but for example, IT++ and Armadillo provide integration with Lapack and ATLAS and that's where they derive their speed from.

Very nice and impressive article you have posted. Its very helpful, i have read and bookmark this site and will recommend it to more other peoples.

ReplyDeleteNicely explained. It's indeed an art to stop new visitors with your attractive writing style. Truly impressive and nice information. Thanks for sharing.

ReplyDeleteI really appreciate your post and you explain each and every point very well.Thanks for sharing this information.And I’ll love to read your next post too.

ReplyDeleteRegards:

NABH

Thank you! Nice article.

ReplyDeleteVery nice and impressive article you have posted. Its very helpful..

ReplyDeleteKeep sharing this type article of article which related to health.

ReplyDeleteThanks for informative post.

Great info... C-libraries can be really confusing. Glad I found this article.

ReplyDeleteMany thanks for your quick review. This is very useful

ReplyDeleteAlejo Nevado-Holgado

Hello, i would like to ask that what is the scope of C language training, what all topics should be covered and it is kinda bothering me … and has anyone studies from this course http://www.wiziq.com/course/2118-learn-how-to-program-in-c-language of programming in C ?? or tell me any other guidance...

ReplyDeletewould really appreciate help… and Also i would like to thank for all the information you are providing on C concepts.

Very nice and impressive article you have posted.

ReplyDelete