Presented by David Moskowitz to the Sarasota Java Users Group on June 11, 2009

Introduction

Java source code for this article

Java concurrency interests me because I see it as a somewhat untapped feature of the language. Most of the work I’ve done, mainly developing enterprise Java web applications, hasn’t dealt much with concurrency directly. I emphasis directly since Java Servlets are one of the most widely use forms of java functionality and are implemented in a multi threaded manner. While most experienced programmers know of the problems in using class variables in Servlets, studying multi threaded can give us a more complete understanding of this issue

Concurrency has been a core feature of the language since JDK 1.0. Indeed most of the concurrency functionality is contained in the java.lang package, so it is always there and available. If you’ve used code completion in an IDE, you will see methods like notify () and wait() on all objects. These are concurrency related methods contained in the Object class, the class from which all other classes inherit.

Java concurrency is also intriguing because of its two sided nature. Firstly, it is extremely easy to implement multiple threads in Java, as we shall see by example in this article. On the other hand, problems and bugs often occur in multi threaded application that don’t occur in their non threaded counterpart. Because of the non deterministic nature of the Java thread scheduler (especially on the Windows platform), two runs of the same program may (or more accurately, will) have different execution timings and therefore different interaction between threads.

Why use threads

Threads may be used for two reasons

1. Improved performance
2. Improved application design

Regarding improved performance, I am a firm believer in the idea, to paraphrase many others, “Premature optimization is evil”. Therefore, it would be unwise to initiate a threaded implementation based only on the assumption that performance can be improved by such an implementation. Performance improvements should only be made after application profiling indicates such a need, and more importantly where, a performance issue exists. It can then be determined whether added concurrency can improve the problem.

It is likely that threading will have more and more impact in the area of performance in the future. Processor trends have shifted from more and more powerful single CPU machines, to multiple CPU or multi core machines. These machines can inherently run more than one process simultaneously, as opposed to single CPU machines which only simulate multi tasking. While it is still likely that the JVM will take care of these issues, the knowledge of concurrency should still prove more important and potentially applicable than ever.

The main performance benefit, especially on a single CPU machine, is when an application is waiting on an outside resource. If all work is performed by the local CPU, it is questionable whether “simulating” multiple processes will have a performance benefit, due the fact that the single CPU is doing all the work anyway and the amount of work hasn’t changed. Indeed, there is minor overhead to creating new threads in the first place. But if the application involves much waiting an outside resource, like an external web service or database, concurrency may be able to provide improved performance.

Another common implementation of this idea is in the area of Ajax. While JavaScript is single threaded (or so I am told), remote calls can be done in an asynchronous manner, so the engine is not waiting for the results of that call by can perform other tasks while waiting. A true multi threaded application can do as many of these operations simultaneously as necessary and appropriate.

Regarding improved application design, this is one area that could indicate concurrent design from the start. Take the example of a real time strategy game. If is certainly conceptually more simple to break down the game into individual, autonomous entities (units, divisions, etc) that implement certain behavior with respect to the state of the game world. A non-concurrent implementation with some sort of master controller could be seen as more complicated in that it also has to maintain the appropriate view of the world provided to each entity and translate a non real time view into what should in fact appear to be real time activity.

That being said, this simulation example is somewhat non standard. It is debatable whether most typical business, database driven applications don’t immediately scream for concurrency in their design. We will see some examples where concurrency could be an appropriate for solution for a standard web application in a future article.

In addition, performance improvements are much easier to add to a logically design, object oriented application. In fact many improvements can come without any work whatsoever, in the form of faster machines, improved JVM and libraries, etc. Most JVM’s already optimize for performance, through inlining of methods or other techniques, and trying to implement these techniques, or other ways you think will improve things, could actually make it more difficult for the compiler to perform its own optimizations.

It also is much more difficult to refractor elegance into an application that might contain numerous low level or non object oriented approaches aimed at performance improvements.

On to the examples, beginning with the very simplest one.

Extending Thread

Our first example illustrates the simplest way to create and execute a Java thread, extending the      Java.lang.Thread class. This class implements the java.lang.Runnable interface. Runnable contains one method, run();

Code Link: Example 1a.java

In example 1a, we create three instances of the Example1a class. As Example1a extends Thread, the run () method is automatically inherited and available.

Threads are started or run by calling start(), which is actually a static method in the Thread class. This causes the Example1 instance’s run() method to be called in a new thread of execution.

Here is the output from example1a

Output Example1a

The only interesting thing about the output is that instance two is run before instance one, even through one.start() is called prior to two.start(). This happens because the JVM thread scheduler does not guarantee any particular order of execution when scheduling threads. The start() method causes the JVM to schedule the thread for execution at some point in the future.

The output doesn’t event verify that all three tasks are indeed running concurrently. Since not much happens within each task, each task completes before the next can even get started.

In the next example, we add some additional work in the run() method.

Code Link: Example1b.java

In this example, we add a sleep property and call Thread.sleep for that time period, in this case milliseconds. This added statement simulates additional “work” that a task may be performing, or perhaps simulates a wait on an outside resource.

Output Example1b

As shown in the output from running this program, all three classes start up, then all three complete. This sort of proves there is some simultaneous activity happening. As you may have noticed, we also added a sleep statement to the “Main” thread. The sleep method can be called in any Java program, even one that does not explicitly implement additional multithreading. There is always at least one thread of execution in a Java application and it can be treated in many respects like any of the other Threads we’ve created, although you won’t be controlling the starting of the main thread.

You may have also noticed an additional class imported, Sleeper. This class simply encapsulates the Thread.sleep method, which we will be using often on the examples.

Code Link: Sleeper.java

The next example shows how in even the simplest examples, threads might not behave as expected. This example uses the setName() method of the Thread class, and it does what you might expect: associates a name with a thread.

Code Link: Example1C.java

In this example, instead of setting a class property to store the name, we attempt to use the name property of Thread itself. A name is passed into the constructor, where the Thread’s name is set to this value.

The output, however, shows that this naming was unsuccessful, as the name printed are Thread-0, Thread-1, and thread-2, which are actually the default names in this case. Perhaps the name cannot actually be set in this manner?

Output Example1c

Actually, if you look at the last line of the output, where the name of the “Main” thread is printed, we see that its name was set to “three”. Either this is an extreme coincidence, or when we executed the line Example1c example3 = new Example1c(“three”, 900), it didn’t set the name of the thread associated with example example3, but instead set that associated with the Main thread. This is actually the case. Looking at the constructor of Example1,

Thread.currentThread().setName(name);

When this line is executed, the Example1c threads have not yet been started. The actual thread executing the constructor code is the calling thread (Main). So in fact, the name of the Main thread was set three times, with “three” begin the final value set.

The following example shows how to correctly set the thread names.

Code Link: Example1d.java

The output of this program is as follows. Note the name of the Main thread.

Output Example1d

Implementing Runnable

The more preferred way to create a thread is to implement java.lang.Runnable. When you extend Thread, you are implementing Runnable by inheritance. However, using the interface allows just about any class to act as a thread, since multiple interfaces can be implemented, but only one class can be inherited from. There is really no good reason to extend Thread, although it can sometimes be more convenience, as a later example shows.

Here is a modified version of the previous program that takes this approach.

Code Link: Example 2.java

Here is the program’s output, which is identical to that above (of course the order of thread execution may change).

Output Example2

More fun with Runnable

In the above example, we created three instances of a single Class, and then executed each in its own thread.

The next example creates two class instances, but created three threads of execution. How does these change things?

Code Link: Example 3.java

We create two instances of a single class

Example3 example1 = new Example3(1000);

Example3 example2 = new Example3(500);

Then we create three threads from those two instances; two threads are running on a single instance.

Thread thread1 = new Thread(example1);

Thread thread2 = new Thread(example2);

Thread thread3 = new Thread(example2);

From the output below, not much has changed, but the objects were created much differently. All three threads start, print their message, and stop.

Output Example3

Not much has changed because the threads are really not doing anything. As I’ve mentioned, creating and running a thread is not very difficult. Problems start to arise when the threads do some meaningful work, particular in how they can interact with each other.

Let’s add some actual calculation and properties to the above example. Here, the run method of each class performs a calculation, then prints out the result of that calculation when exiting run().

Code Link: Example 4a.java

In this example, we create three instances of a class, and start four threads from those instances, similar to what we did in the last example.

For each threads’s run() method, we perform a calculation and store the results as an class instance property, in this case simply generate a random integer. Then we sleep, to simulate performing some additional work. When the thread awakes, it prints out the result previously calculated.

The output from this program is the following

Output Example4a

If things perform as expected, the result printed with the starting method should match the result printed with the exiting method. In the output above, all looks fine except for thread “three (thread 1)”. It calculates and stores a value of 2, then sleeps and prints out the stored value, which is printed as 93.

What happened here? The answer illustrates the basic operation of threading.

When we create an instance of a class, a new object is allocated and stored on the heap. Any non static properties are also allocated memory. In our case, the new property “result” is allocated and stored with the instances we create. As above, we create several instances, but for one of those instances, we create two threads. These threads, therefore share the same instance variables. The change made from one can affect the values seen in another, as this example indicates. For this to happen, the thread scheduler needs to switch control to another thread after the result of the first thread has been calculated. The second thread then performs the calculation, storing the result in the single instance variable. Then both threads awake to print the result, which will be the last stored value.

Most threading problems occur from simultaneous (concurrent) access to shared resource.

One way around this problem is to avoid the use of instance variables in cases where multiple threads are involved.

Code Link: Example 4b.java

The only change made is in moving the result variable from the class level to a local method variable in run(). Code and variables in (non static) methods are executed on the JVM stack, and each thread has its own stack. Therefore, the output is what we want.

Output 4b

This situation has much in common with Java Servlet technology. Servlets are executed in a multi threaded manner. The doService(), or doGet() and doPost(), method is quite analogous to our run() method above. Any Servlet level variables run the risk of collision amongst multiple threads. This is called a Race Condition; the timing of the data access affects the data.

Another alternative solution to this problem is using the ThreadLocal object. ThreadLocal allows you to essentially create a single variable, but give each thread its own unique copy of this variable. In the example below, we store the result value in a ThreadLocal object. Note, we must switch the type from int to Integer, since we can’t define a generic object with a primitive type.

Code Link: Example 4c.java

Instead o f just accessing result directly, we must use the methods result.set () and result.get(), when using ThreadLocal. This will get and store the values specific to the current thread of execution.

The output below should yield the correct result.

Output example 4c

A caveat on using ThreadLocal is necessary. Using ThreadLocal in a managed environment (ex. an app server with a Thread Pool ) could potentially cause problems if the stored object is not cleaned up correctly when the thread’s processing is finished. Any ThreadLocal variables that existing when a thread is finished, could potentially be accessed by another thread if the thread pool simply keeps the thread active and allows another task to use it.

As a final solution to the problem of Race Condition we can use synchronized blocks of code. Synchronized blocks guarantee that only one thread of execution at a time can execute a give block of code. In our earlier example, issues arose when one thread set a value then went to sleep (or was swapped out by the thread scheduler). Another task came along, set the value again, and then the first task awoke only to read the incorrect value.

Synchronized blocks allow us to force the first thread to set the value, then finish whatever task is necessary, in this case print out the value, before any other thread can gain control over that block of code.

Code Link: Eample 4d.java

In the above example, we wrap the entire run method in a synchronized block. The statement

synchronized (this) {

..

}

First gets a lock on “this”. Note the lock can be gotten on a class instance, as in this example, or on the class itself. Locking an instance allows multiple instances of that class to process concurrently. In our case, since we are concerned about two threads running on the same instance conflicting, this approach works. The output below should again show the correct output/

Output example 4d

How did the output from this version differ from previous versions? Note that thread “three (thread1)” starts and completes before “three (thread2)” even starts, even though we put thread1 to sleep during that time period. This is the result of synchronization, as you might imagine, can lead to performance degradation, since no thread swapping can occur during the block. Therefore, even if we are waiting on an outside resource during this block, no other thread can gain control of that lock to execute the synchronized code.

Exceptions in Threads

Let’s try and implement a typical exception handler in our threaded program. Here, in our run () method we throw an exception to see what happens.

Code Link: Example5.java

The output is the following

Output example5

That is sort of expected, as we didn’t do anything to catch or handle the Exception. Let’s try to add error handling.

Code Link: Example5a.java

Here, we wrap everything in a try catch.
We even suspend the main thread to it doesn’t finish before the thread can throw it’s Exception.

Output example5a

That didn’t work. As you can see, the thread threw its exception almost immediately, certainly while the main thread was within its try catch loop. So as you might imagine, a try-catch like this will only work on the specific thread executing that code.

To catch an Exception thrown by a thread, we need to pass the Thread an instance of UncaughtExceptionHandler.

Code Link: Example5b.java

In the example above, we create things like before, but add the following lines.

thread1.setUncaughtExceptionHandler(new MyExceptionHandler());

Of course, we need to implement the actual UncaughtExceptionHandler, which is simply the following class

Code Link: MyExceptionHandler.java

When a thread throws an uncaught exception, and remember the thread class can catch exceptions the same as any Java program can, the uncaughtException() method will be called.

The output below demonstrates this.

Output example 5b

JDK 1.5 enhancements to concurrency

JDK 1.5 introduced numerous enhancements to Java Threading as part of the import java.util.concurrent package. Most of these changes add higher level controlling structures to performance and managed tasks that before needed to be programmed at a lower level. These changes save the programmer a great deal of work and ensure are generally quite easy to use and integrate. We will look at a few of the most significant additions in the next section.

JDK1.5 introduces Executors. Instead of controlling thread creation explicitly, as was done in the above examples through thread creation and thread start, we can create an Executor, tell the executor what to create, along with some parameters, and have the executor manager thread creation and control.

In the next example, we see how using an Executor can easily control thread pooling.

Code Link: Example6.java

In this example, we instantiate three types of ExecutorService. For each of these three, we spawn 10 thread instances of the same class.

The three types we use are

1. CachedThreadPool
2. FixedThreadPool
3. SingleThreadExecutor

Notice that two of these Executors are Thread Pools.

CachedThreadPool will create as many new Threads as necessary (in our case, up to 10), but reuse existing threads when they become available.

FixedThreadPool is similar to CachedThreadPool, but puts an upper limit on the number of Threads that will be created. New tasks submitted will wait until threads free up for use.

SingleThreadExecutor limits the tread pool size to one. This approach may be used for activity that must be serialized. Remember that, as the above example shows, different Executors can be used in the same application, so using a SingleThreadExecutor only affects a specific task without causing the entire application to become single threaded.

Also notice how easy it is to choose one approach or the new; simply a change in the object creation code.

Also note the shutdown() method. This doesn’t stop any of the threads that were created; it just stops new threads from being submitted to the corresponding thread pool.

Here is the (somewhat longer) output from the above program.

Output example6

With the CachedThreadPool, all 10 tasks start immediately, and then exit. With, SingleThreadExecutor each task is started then exits before the next task starts. With FixedThreadPool, three tasks start then exit, before the next three tasks start.

Returning Values

Another additional to JDK 1.5 is the ability for threads to return values. Prior to this capability, you needed to store method results in another object, or make the result available in a public get type method and handle all the issues around multi threaded access to shared data.

In JDK 1.5 we thread can return values by implementing the java.util.concurrent.Callable interface instead of java.lang.Runnable.

This approach is illustrated in the example below.

Code Link: Example7.java

In this example, we run () with call(). Call can take any return type, similar to any other java method.

The call() method in this example performs some (simulated) work, then returns an integer result.

Output example7

The output shows each class starting up, and then exiting along with the return value calculated.

Having a return value didn’t really do much for us in this example, as that value was lost when the thread exited.

To read this value, one approach might be to wait for the threads to finish, then interrogate the result value.

Let’s show how this could be done.

Code Link: Example7a.java

In this example, we create several instances of a class that inherits Thread and store these instances in an ArrayList. Since these are already first class Thread objects, we loop through the Array list, and start() each Thread.

We can then wait for the threads complete, but looping through the array list again

for (Example7a thread : threads) {

and calling

thread.join()

This method will cause the current Thread, in this case the Main thread, to wait until thread completes. The loop will therefore exit when all threads have completed.

The results show this performs as expected

Output example 7a

Since we are implementing (by inheritance) Runnable, there is no actual return value. However, the main thead can call the public getter on id, which is the return value we want. Remember that if we create multiple threads for a single class instance, this approach could lead to a race condition. But in our example, we create a new instance of the class in the line

threads.add(new Example7a(i));

Now let’s try this approach with what appears to be a preferred method for retrieve values , the Callable interface. We will also add a bit of randomness, but changing up the sleep time used during the call() method.

Code Link: Example7b.java

The above example does things similarly to the prior example. Here we use an ExecutorService to create the thread. Also, since in this example, the instantiated class, Example7b, does not extend Thread and is therefore not a Thread object so we cannot call thread.join() in the final check loop. Instead, we get the underlying Thread with

thread.getThread().join();

and attempt to wait using this approach.

Running the program will show that it hangs (or trust me, it hangs).

Output example 7b

The main thread starts all the threads, and then waits for the first to complete. Each thread completes its work, but Main is still waiting on thead 0.

The reason for this is that we told main to wait for itself to finish.

Looking at the getThread getter

protected Thread getThread() {

return Thread.currentThread();

}

It returns a thread, but actually returns the current thread executing that statement. This method is called by Main, so it returns the thread underlying Main. This causes an endless wait cycle, since Main can’t finish before the check loop finished, and the check loop won’t finish until main finished. Essentially we have deadlock on itself.

Before we discuss the correct way to retrieve return values from threads, lets continue down this path and look and other ways to ‘wait’ for threads to finish.

One way is by using the CyclicBarrier, introduced in JDK 1.5.

This object is useful when threads need to coordinate with other threads.

In this example, we create a CyclicBarrier and pass it to each of the threads created. The threads then perform their processing and then call cyclicBarrier.await(). This acts like a staging area, where all threads will wait until the barrier “opens”, which occurs after a specific number of threads call await.

The code illustrates this.

Code Link: Example7c.java

In the output below, we see each thread execute , then wait at the cyclicBarrier. Periodically, the main thread awakens, checks to see if the barrier is open, then sleeps. When all threads have completed and the barrier is open, Main completes and exits.

Output example 7c

The cyclic barrier is useful when threads coordinate the work they are doing with other threads. Our next option is a better choice in our situation, where one thread needs to wait for the completion of a group of other threads. We could have had Main call await(), but then it would be participating in the work, not just observing and waiting, and that just doesn’t seem right.

A CountDownLatch is initialized with a count value, then any task can call countdown() on the latch, decrementing the count by one. Calling countdown does not block the calling task, it can continue processing. Calling await() however, does block the calling task until the latch opens.

The following example illustrates this

Code Link: Example7d.java

This is a more natural approach to the problem of waiting for a set number of tasks to complete. No sleep/awake cycle was necessary in the checking class.

Output example 7d

And finally, let’s get back to the preferred approach to getting return from threads, the Future object.

Instead of storing our tasks in an ArrayList, we store them in a list of Future objects. The Future object represents the results of an asynchronous operation.

In the example below, we create a List of Future TaskResult objects

List<Future<TaskResult>> results = new ArrayList<Future<TaskResult>>();

Notice the use of Generic types.

We could create a Future<Integer>, but then we wouldn’t have access to name or other instance variables.

The example below shows this implementation

Code Link: Example7e.java

Here, instead of just calling the submit method to start each thread, we call submit and set store the result value in the Future list. Submit actually has a Future return type. The return type of call() must match the type of Future. No additional changes are necessary for the call() method, such as a countdown. The call() simply completes and returns a value. That value is stored in the future object.

The main thread then loops through the Future List similarly to how it looped through the ArrayList in the previous examples. For each future object, it can get the result of the operation by calling

taskResult.get().getResult()

The first part of this method chain, taskResult.get(), will cause the calling thread to block until future task completes. We could also call get() with a timeout parameter, which will only wait for a specific time interval. There are also additional methods to see if a task is complete.

Code Link: Cleanup.java

Here is the output from this example.

Output example 7e

Deadlock

We briefly mentioned deadlock in perhaps an unusual circumstance where a thread is deadlocked on itself. The more common occurrence is when two tasks are waiting on a resource the other task has.

This scenario is illustrated in the following example. Here, instead of simply sleeping, which can actually give up a lock that a task has, we load an external web page into an XML document (not caring whether the html is actual valid XML). This will certainly cause some processing delay, and is a good example of waiting on an outside resource, where multiple threads make sense.

Code Link: Example8.java

In this example, we create several threads as before. When started, each thread can do one of two things, depending on whether it has an odd or even id. In the case of even id’s, it gets a lock on Object1 , performs some processing, then attempts to get a lock on object 2. Locks are acquired implicitly, by attempting to enter a synchronized block of code (A Lock object is also introduced in JDK1.5). For the case of odd ids, it first gets a lock on object 2, performs some processing, and then attempts to get a lock on object 1. If we time things just right, one thread has a lock on 1 and is waiting for 2, while another has a lock on 2 and is waiting for 1, causing a deadlock.

In this example work1 and work 2 are created solely for their locks and serve no other purpose. This is perfectly legal.

Her e is the output illustrating this.

Output example 8

Deadlock can also occur when synchronized methods are used. Unlike the above example, where an explicit lock on an object is acquired, the next example acquires a lock on the class instance itself. Both are potential deadlock scenarios.

In the following example, instead of a local synchronized block, a synchronized method in an external class is called.

Code Link: Example8a.java

Code Link: SyncWorker.java

The output shows the predicted deadlock.

Output example 8a

If we remove the synchronization, as in the following example, the deadlock is avoided. In reality, we can assume the synchronization was there for a reason, and not simply to cause a deadlock. Therefore, simply removing synchronization is not the answer. The solution is dependent on the actual problem at hand. As a general heuristic, try to avoid multiple, nested lock acquisition and be aware of any external synchronous methods called.

Code Link: Example8b.java

Code Link: Worker.java

Output example 8b

Topics not covered

This article certainly does not intend to cover the entire domain of Java concurrency. Several excellent references are listed in the appendix. However, some topics are recommended for additional study. It is recommended that the reader become briefly acquainted with wait() and notify() methods and Blocking queue. Please research these on your own, as there was not enough neither time nor space to include these topics in the original presentation given at the Sarasota Java Users group.

The next article will discuss performance profiling of multi-threaded applications.

Books and links

1. Java Concurrency In Practice: The best, most current book on the subject, written by some of the people who wrote the Java concurrency implementation

http://jcip.net/

1. Thinking in Java has a nice, long section on concurrency, updated for JDK1.5. This book should be on every Java programmer’s bookshelf and can keep you learning for the rest of your Java career, or until the next edition is out.

http://www.mindview.net/Books/TIJ/

1. Java Tutorial: Concurrency

From Sun.com. This article serves as a good online reference when needed.

http://java.sun.com/docs/books/tutorial/essential/concurrency/

1. Any article by Brian Goetz, one of the authors of Java Concurrency In Practice. Search Google for some good, though dated, IBM publications.
2. http://tutorials.jenkov.com/java-concurrency/index.html