Threading made Easy (1/2)

In my opinion it isn’t possible to have these two words in one sentence. I’ve been exposed to thread programming numerous times in the past. Despite the almost heroic nature of being a ‘thread programmer’, I’ve never had the guts to expose this in public. Brag over a few beers about success stories on how well my thread-aware and thread-safe software was running? It has never been part of my vocabulary. One simple reason: it’s way too complicated. Well, at least for me. Even with the aid of thorough design, good examples and extensive mock tests my code remained complicated and incomprehensible. And yes, I tried to refactor, redesign, recode, restructure, whatever, I never could get it right.

Well, up until .Net 4 and its component: System.Threading.Tasks. Included in the very core of the framework. Right out of the box, support for threading for the masses.

Why threading?

It is good to step back for one moment. The 1-million-dollar-question to always ask yourself is: do I really need it? The simple answer is: yes. Because why would you let anyone wait while your machine is sitting there doing nothing? Especially when it is not necessary to wait.

So: yes I need threading to make software as performant as can be to the best of my abilities as a software engineer.

Do I always have to use threading? Of course not. In its most essential form, current CPU’s can do multiple things at the same time (especially when having more cores) and are very fast and therefore often waiting for some other things (like database or web I/O stuff) to end before they can continue. So if you’re in such a situation, and I can assure you, this is most often, threads come to the rescue to really utilize the CPU to its full capacity.

The example

Let’s apply threading to a simple example to show how the new .Net threadinglayer can be utilized to increase performance 40-70% in a very easy manner.

Suppose you want to do a multiple query on Twitter (most popular demo environment) in which you need to return the first 100 hits for a couple of subjects. I have tried this with my webbrowser and you have to do some manual work to return such a list in one piece.

Beforehand I do know getting 100 tweets from Twitter isn’t that difficult, especially with their JSON based REST API (that’s a lot of cool keywords in one sentence!). So one task is easy to implement. I also know that getting something from the internet usually sets my CPU in a comfortable waiting state because network transfer is still way slower than my CPU. So the task is also suitable for multiplexing a couple of times.

Some preparation

First I have to define my testcases. I don’t want to get bogged down into testclasses (although I do agree it would have been better to use them) because hey, this is demo code anyway.

The thing I want to show you is:

In short, I want to show the performance differences between four separate scenarios
-      A non threaded one
-      A classic, pre 4.0, threadpool version
-      A 4.0 version using the new TaskFactory
-      And finally the version using the new Parallel class

These scenarios all do exactly the same thing; only the scheduling of the tasks differs. The thing they have to do is get some data from the web in the following pseudo manner:

foreach subject from the list
search for the last 100 tweets on this subject


The search API in Twitter is pretty straightword. For example, to perform the above for the subject ‘luminis’ the following url will suffice:

http://search.twitter.com/search.json?&q=luminis&rpp=100

To get data from the web I’ve defined a small routine which returns a byte array with the required data. Fortunately WCF made getting data from the web pretty straightforward in .Net, this resulted in the very simple basic routine:

private static byte[] Get(string url)
{
    byte[] data = null;
    WebClient wc = new WebClient();
    try
    {
        data = wc.DownloadData(url);
    }
    catch
    {
        // probable time out, for this demo just quit
    }
    finally
    {
        wc.Dispose();
    }
 
    return data;
}

This routine is part of a class called WebContent. This class also has a public property UrlList which can contain the list of urls for which the content must be read.

private static List _UrlList = new List();
public static List UrlList
{
    get
    {
        return _UrlList;
    }
}

What I have to do now is to iterate over this list and call my simple Get routine for each of the four scenarios.

#1: The non-threaded version

The simple version is the one without any threading at all. We do need this one to baseline our tests so we can do a comparison with the threaded counterparts. The only thing we need to do is read all the urls and the code pretty much resembles the pseudo code shown previously:

public static void GetSequential()
{
    foreach (string url in UrlList)
    {
        Get(url);
    }
}

#2: The 2.0 ThreadPool version

Actually I didn’t want to make the ThreadPool version because as said earlier I don’t get this ‘old’ thread programming very well. But as a reference I’ve hacked a version for which I can’t take any liabilities. It is just here for reference purposes. Note that I need two routines to achieve the same goal. Additionally I’ve changed the thread of the main to [MTAThreadAttribute] otherwise you will receive to message:

Anyway the code looks like:

public static void GetWithThreadPool()
{
    _ResetEvents = new ManualResetEvent[UrlList.Count];
 
    Random rand = new Random();
    for (int i = 0; i < UrlList.Count; i++)
    {
        _ResetEvents[i] = new ManualResetEvent(false);
        ThreadPool.QueueUserWorkItem(new WaitCallback(DoThreadPoolWork), (object)i);
    }
 
    WaitHandle.WaitAll(_ResetEvents);
}
 
private static void DoThreadPoolWork(object o)
{
    int index = (int)o;
 
    Get(UrlList[index]);
 
    _ResetEvents[index].Set();
}

Pretty impressive but incomprehensible, huh?
It doesn’t feel right that you have to add all these complex things just to get a bit of multi-threading. To me, the only easy part is the Get call somewhere near the end. All the other stuff is thread overhead. And if you would need more that 25 threads this code won’t work. But let’s not dwell on these passed issues and move over to the 4.0 versions.

#3: The  4.0 TaskFactory version

The task factory shows what the thread overhead could be if the framework is more powerful than the 2.0 version. Still it is some overhead but the code looks more like the pseudo code than the previous implementation:

public static void GetWithTaskFactory()
{
    Task[] tasks = new Task[UrlList.Count];
    int counter = 0;
 
    foreach (string url in UrlList)
    {
        var task = Task.Factory.StartNew(() =>
                                 Get(url), TaskCreationOptions.LongRunning);
        tasks[counter] = task;
        counter++;
    }
 
    Task.WaitAll(tasks);
}

The thing I need to do is to create a task for each subject and, with a lambda expression, call my good old private Get() on this task. Finally I just have to wait until all tasks are finished.

#4: The 4.0 Parallel version

The ultimate version is left to the end. The new Parallel class makes it possible to write threaded code without almost no (thread) overhead. It looks like:

public static void GetParallel()
{
    Parallel.ForEach(UrlList, url =>
        {
            Get(url);
        }
    );
}

Isn’t that just beautiful? It also resembles the pseudo code pretty well. Okay you have to understand the lambda expression with the anonymous delegate syntax and semantics but if you do it is just an ordinary for-loop. As far as I’m concerned: this is as it should be.

Performance results

So far the differences in the source code of the four implementations. To start each of the separate scenarios a little bit of timing code is needed to calculate the total of milliseconds the code runs.

private delegate void GetContentDelegate();
 
private static string DoTimedOperation(GetContentDelegate getForumContentDelegate)
{
    DateTime start = DateTime.Now;
 
    // perform the actual operation
    getForumContentDelegate();
 
    DateTime end = DateTime.Now;
    TimeSpan elapsed = end - start;
 
    return elapsed.TotalMilliseconds.ToString() + "ms";
}

And with the aid of a delegate the timing function is called four times as in:

private void StartSequential_Click(object sender, EventArgs e)
{
    this.lblSequentialTime.Text =
        DoTimedOperation(new GetContentDelegate(WebContent.GetSequential));
}
 
private void StartThreadPool_Click(object sender, EventArgs e)
{
    this.lblThreadPoolTime.Text =
        DoTimedOperation(new GetContentDelegate(WebContent.GetWithThreadPool));
}
 
private void StartTaskFactoryl_Click(object sender, EventArgs e)
{
    this.lblTaskFactoryTime.Text =
        DoTimedOperation(new GetContentDelegate(WebContent.GetWithTaskFactory));
}
 
private void StartParallel_Click(object sender, EventArgs e)
{
    this.lblParallelTime.Text =
        DoTimedOperation(new GetContentDelegate(WebContent.GetParallel));
}

I ran a few tests on my machine and one of them resulted in the following numbers: (By the way my machine is a Macbook Pro running Windows 7 in a Virtual Box with 1 processor and 2048 Mb memory.)

Of course you have to try this on your own machine but it is safe to say that the performance increase is worth the overhead in thread programming.

I have analyzed the thread behavior with the Performance Monitor (PerfMon). In the analysis I was interested in:
- # of processor threads (blue)
-      # of logical threads for the process (green)
-      # of physical threads for the process (red)
These are the results of one test I have run:

You can see the framework too needs some threads when the non threaded version is started. When the threadpool scenario starts you clearly see the increase in threads. The threadpool is managed by .Net so this amount doesn’t decrease when the scenario is finished. This does happen with the third scenario. Here the threads are in control of the routine and you can clearly see that the threads are released (at least to the system, the framework has a different view on it). In the fourth scenario again the threads are fully controlled by the framework.
Interesting to see is: in both the threadpool and the parallel scenario only a limited amount of threads are created (approximately 5 or 6) while in the Taskfactory 12 threads are created (for each subject one). It is hard to say what exactly happens behind the scenes. Do all logical threads have a physical thread associated? And do these threads run ’simultaneously’ or not? I have to say I don’t know.
Then there is one thing I don’t understand at all. How is it possible (near the end) that the total number of physical threads of the process is higher than the system amount of physical threads?! If somebody can tell me this I would be very hapy!

Conclusion

In a next blog I will extend this little example. So far I have concentrated on threads, starting and synchronizing them to show the performance differences. I have experimental proof it is worth taking an effort to learn more on the new .Net 4.0 threading stuff.
The part that is left out is the handling of the data and the presentation of the results.
This part is described in part 2 and can be found here.

have fun, florisz

Download the zip for the full source code here.

References

Twitter Search API

MSDN on System.Threading.Tasks

.Net 4.0 and System.Threading.Tasks

Threadpool example

Task Parallelism in C# 4.0 with System.Threading.Tasks