Leveraging std::async for Your Applications

Asynchronous Tasks

The C++11 standard introduced several new types, functions, and methods for performing asynchronous tasks. For several years, C++ programmers, and developers in general, have winced at the though of dealing with multithreaded code, especially when it comes to protecting data and making sure no data corruption occurs. However, the C++11 standard has made it much easier and less stressful to write code that efficiently and intelligently incorporates the power of modern CPU's and the fact that most CPU's these days are at least dual core to leverage applications to their highest potential. We want to run fast, like Usain Bolt :)

In essence, a multithreaded application takes into consideration that systems these days can perform multiple operations at the same time. This has several benefits! Imagine being able to keep your application responsive to user input, while you send expensive and processor-intensive operations to a separate thread that will execute while you keep the UI responsive. This way, you do not jeopardize user experience or application responsiveness, and can leverage the CPU's capabilities. In other words, its all about being fast, efficient, and performing optimally! Traditionally, this has had its benefits and drawbacks. The main drawback being that maintaining and debugging mutlithreaded code is a problem on its own and usually creates more issues than it solves. I'm here to be the bearer of good news and tell you that in the new C++ standard, most of this stress has gone away.

As part of the new standard, C++ includes a variety of new std:: types such as std::thread, std::mutex, std::condition_variable, and the one we're mainly going to be talking about today, std::async. All these types have been standardized, however I do know that they have been around in Boost, TR1 and TR2, and even the POSIX pthread libraries  for quite some time now. My goal is to cover the C++11 recommended way of approaching a multithreaded application, which is with std::async. So let's begin!

Farewell to Threads!

Not quite, but its a good start! The std::async call, as the name implies, gives you asynchronous capabilities! It alleviates you, the programmer, of having to deal with std::thread's directly to achieve asynchronous code. Why is this good? It's good because now you don't have to worry about joining threads or any of the other fun stuff involved to get threads to work correctly, with the exception of ensuring corruption free code ( by means of external synchronization). So let's see std::async in action!

#include <iostream>
#include <future> //required for std::async


double getCrossProduct(matrix<int> m1, matrix<int> m2)
{
    // wild and CPU intensive code here
}
int main()
{
       matrix<int> m1;
       matrix<int> m2;
        /// code to get the matrix values ...

       auto f1 = std::async(std::launch::async,getCrossProduct,m1,m2);
         // more code
}

Woah, woah, wait a second! You told me this was supposed to be a relief? What are all those parameters! And what does std::async return!?  Yes, yes, I understand. It may look daunting at first, but give me a few minutes and all of this will make perfect sense to you.

In order to understand std::async a little bit better, I think its good if we look at the function declaration for it, so we can analyze it. Here it is (note, I'm omitting the return value for now, let's tackle it one piece at a time =) )

1.  async( std::launch policy, Function&& f, Args&&... args );
2.  async( Function&& f, Args&&... args );

So we see an overload for the function, one of which takes a std::launch . Focusing in on #1, we can see that as parameters we need a std::launch, a function, and arguments. Why do the function and arguments have a weird double ampersand (&&)? The && sign is the new r-value reference symbol! If you aren't familiar with rvalue references and move semantics, it would be a good idea to read up about that, but it isn't strictly necessary to do so to understand std::async. The "args" parameters also has "..." which signal something known as parameter packing. It is used in conjunction with variadic template arguments, which again is nice to know but not necessary. However, what this implies is that std::async can take a function with any number of arguments and any type of argument. So I can pass it a function that takes no parameters or even a function that takes 100 parameters, 1000 parameters, any amount of parameters you wish, and of any type. Looking at the example above with the made up matrix<> class, you see that I am trying to call the function getCrossProduct, and then passing in matrix m1 and matrix m2. async() happily accepts this and performs its job. But what if my getCrossProduct() was a member function, and it only needed to be passed a single argument? async() has no problem handling that case as well.The thing to realize is that both versions of async() take a variadic template argument list and forward it to the function you are calling. That topic, however, deserves a blog entry on its own. Hopefully its clear now that async() offers us a way to make any function, lambda, functor, or any callable object for that matter, asynchronous. So, going back to the example above, I have passed to std::async my function, getCrossProduct,  and its 2 matrix<int> parameters so it can perform its intended function.

The first parameter to std::async, a std::launch, is just a flag indicating something about the operation of this function call. There are 2 flags you need to worry about:

1. std::launch::async
2. std::launch::deferred

The former tells std::async that  you Really Really want your function to be ran on a separate thread. The latter tells std::async to run the code on the calling thread, in a synchronous manner. If you OR these flags together, you'll let the implementation decide whether to run this code on a separate thread or on the caller. For more information, please look at this link. 

As of now, the std::async routine seems pretty simple! I pass it a function, its arguments, and optionally a std::launch flag ( the default behavior if you do not explicitly pass a flag is std::launch::async | std::launch::deferred. It bitwise OR's the flags together, the behavior of which I explained above).

So, great, I can launch a function asynchronously, but what if I need a return value? What if I send std::async a function that actually returns useful data? Luckily for us, std::async provides us with a way to get a return value from our asynchronous functions! Let's stop and think for a moment before moving on. std::async cannot possibly return to us the value we expect by simply calling std::async(function,args). Why? It hasn't even had time to start spinning up a thread for you to run the code! How can it possibly know what the return type or value is if it hasn't even executed yet? So, we cannot say something like

double result = std::async(std::launch::async, getCrossProduct,m1,m2);

However, the C++11 standard comes with a data type known as a future. A future, is perfectly described by Herb Sutter as "A ticket redeemable for...". Basically, std::async will return to us a std::future that will later hold our expected return value, once the background thread has had enough time to finish executing all the work. To get our value from a std::future, we call get() on it. If the code has already executed, the call to get() will return the value immediately. However, if the code is still running in the background, the call to get() will block until the future has been filled with the data. As I mentioned earlier, the std::future is a ticket for our return value. It is very similar to preordering something. We preorder our return value with std::async. Since its a preorder, the product isn't available yet, however we have a ticket, A.K.A a proof of purchase. Whenever we want to, we can go to the store and ask the clerk for our item. If they have it in stock, we'll get our desired item and leave! However, if the store does not have our item, we will have to wait for it.

So by saying

future<int> my_future = std::async(std::launch::async,getCrossProduct,m1,m2);

int my_return_type = my_future.get();

// use my_return_type

We have asked std::async for our preorder of myFunction, and we get back our ticket. I call my_future.get() which will either return immediately with the value or wait until it is finished, and then use my value. Two quick things to mention before moving on. It is very likely that your function does not return anything at all. It could just be a void function that works on references, or writes to a log file, what have you. In which case, you're out of luck. ONLY KIDDING! async() works exactly the same with void functions as with returning functions. The return type from async() is then std::future<void>. Since you never need a return value, you don't need to call get() on it, unless you absolutely need the work to be finished before moving on to something else. Remember, get() will block if your task has not finished. For the case of non void functions, the return value happens to also be returned along with get(). The other thing, notice how I am typing std::future<whatever>... which is great for practicing and demonstration, but sometimes you don't know what will be returned, or are simply too lazy ;) So, you can also use the keyword auto to capture the return value as well.. So, let's move on :)

HOLD YOUR HORSES,...

You see, using std::async is a very simple, easy, and intuitive way to run asynchronous processes in your applications! However, be careful. There are some weird subtleties with std::async that have not yet been  addressed by the standardization committee. Here is one of them:

1. std::async(std::launch::async,some_function);

Will NOT be run asynchronously even though we told std::async, we really really wanted asynchronous code with the std::launch flag. Why won't this be ran asynchronously? The destructor for std::future will join() if it was launched from a std::async call. What does this mean? We called std::async, so there will be a std::future constructed, but it will be destroyed instantly since there is nothing to capture its value, thus ~future() will be called right away. Since it was spawned by std::async, it will join()  on the thread it had created for the call. So on the caller thread we actually have to wait/block, even though we had went through the pain of writing it all "asynchronously" with std::async and even the appropriate launch flag. In order to remedy this problem, we need to keep the std::future around for a bit longer so that it won't join() right away. To do this, just save it in an auto.

       auto f1 = std::async(std::launch::async,some_function);

Note, however, that ~future() will only join() if spawned by std::async, not by anything else! This remedy will still work even if your function returns void, so you're safe :) 

I hope this introduction to std::async was intuitive and helpful :) Drop a comment and tell me what you think! The next blog post will either be about r-value references/move semantics or my Code Synchronization Arsenal :) 

Thanks for reading!