Java Threading
Filed in: Java
Java Threading Basics
This is a quick tour of the basics of Threading. I assume here you have a basic grasp of Java ( Imperative and Functional ) and Generics.
Thread creation
Thread myThread = new Thread(() -> {/*put runnable code here */} );
myThread.start();
This creates an on the fly runnable using the fact that runnable is functional and passes it to Thread and then the thread is started. You can only start a thread once. Once started the thread can not be stopped unless you either catch an InterruptedException and act on it or your pid is killed outside a java context. This implies that threads need a clear flow of what they will do and how they will end. This leads to the idea that Runnable and Callable’s are really tasks that can be given to a Thread or given to an Executor to be handled.
With the Thread you can call Thread.sleep(n) within the InterruptedException context as it might throw one. This sleeps the thread unless interrupted and keeps any locks held while asleep.
You can also wait for an Object to notify you that it is completed with its work by calling the Object.wait() which suspends the current Thread. By doing this you explicitly release your locks and wait for an object to call notify() and hope you are chosen or notifyAll() which notifies all the Objects that are waiting. You should have noticed here this is the Observer Observable pattern.
The one thing to watch for is that you can not wait on a specific thread ( you can join as below ). If you call Object.wait() the current thread will be suspended until notify()/NotifyAll() is issued somewhere. The notify might not be the specific notification you are looking for so you need to check that the notification is the one you want. From the API docs there is a clear example showing that you must check the notification as you wait ie:
while(!joy) {
try {
wait();
} catch (InterruptedException e) {}
}
You can also wait for another thread to to complete and attach yourself to continue after that thread’s runnable completes. To do this you call t.join() where t is the Thread you are waiting on.
Threads need to be careful of Synchronization ( Happens before ) and memory consistency / race errors ( Dirty reads / writes ). There are various strategies that can be used to avoid these problems. Some of these will be discussed here.
Synchronized methods
You could synchronize at the method level however that takes a monitor lock on the object. The lock you take is re-entrant meaning that once you have the lock you can re use it or reenter as much as you like before releasing
Synchronized blocks
You can synchronise a block of code only, synchronising on the this but recall this is just a monitor lock on the object.
Static Synchronization
As above but if you are in a static context you do not take the monitor lock from the object but from the class.
Volatile keyword
This keyword allows you to declare a variable this will always be involved in a check before act relationship. This means the variable will be checked to be the latest before acting on the variable.
Atomic and Accumulator utility classes
Java has a few Atomic classes to assist in the happens before relationship where the set() happens before the get() in a threaded context. These are based on most of the primitive values for example AtomicLong AtomicInteger plus we also have accumulators like LongAccumulator
Livelock and Starvation
When having thread interaction it is very important that we do not have a situation where there are too many threads running and the CPU is unable to meet all the compute request time for the threads. This can lead to starvation where the threads can compute but are not getting enough compute resource, this is the essence of starvation.
Another form of starvation and live-lock is when we have threads that spin waiting for a resource to become free. The thread is still good and holding locks and can run but it can not proceed as it is still waiting or spinning on something to occur before proceeding.
The best way to prevent situations like this is to not have too many threads such that the machine is constantly context switching, this can be observed via the CPU stats on the machine. You should also periodically log or have a way of indicating that you are waiting on a resource or timeout if you can not acquire what you need in a specified time period.
These strategies should generally be in a concurrency plan that all developers follow. This plan usually gives you the locking order so we always lock in the same order. How long we wait before doing a timeout and what to do in order to respond to an orderly shutdown of a long compute task. The order of locking becomes even more important when we talk about deadlock.
Deadlock (lock order strategy)
Dead lock is something you want to avoid at all costs because it implies that you are waiting on a resource and that same resource is waiting on you. Depending on the order that the locks were taken, if you keep waiting for each other then neither of you can do anything and the system generally comes to a halt. This can be prevented by always locking in the same order. If you always lock in a predefined order then you can never acquire the lock that causes you to get into a deadlock as it is locked and you would block.
Immutable Rules
Immutables are a useful strategy when it comes to threading. The real key here is that once you have an object that can not be changed you can freely pass it around between threads and the data never gets changed, saving you Memory and race conditions as nothing can change.
However you need to create an immutable carefully
- no setter methods
- all fields private and final
- Can not subclass so private constructor or better create via a factory method.
- No storing of References ( So they do not leak out and get modified )
- any method returns a deep copy of state rather than a ref.
Lock Interface from the Concurrent Package
We looked at the monitor locks above to actually lock an object so we can make changes, but there are some issues with just getting the lock. For example when you try to get the lock you will block until the lock becomes free. You also can not distinguish between a Read Lock and a Write lock. That is why there is the Lock interface.
This interface allows us to try a lock and if its not free we can then do something else rather than blocking.
Also dependant on the Implementation of the lock interface you can have Re-entrant Read and Write locks.
Perhaps an example will help
Lock myLock = new ReentrantLock();
if(myLock.tryLock()){
try{
//got lock
}finally{
myLock.unlock();
}
}else{
//No luck with lock
}
The lock implementations also include a readwrite lock where you can not acquire a read lock if there is a write lock open but you may have many readers. This is useful when looking at 1 writer many readers or when implementing a Disruptor Pattern.
ReentrantReadWriteLock rrw = new ReentrantReadWriteLock();
if(rrw.readLock().tryLock()){
;
}
Executors
Now we have looked at Threads which is a single execution of a Runnable. However Thread creation is expensive so if you were writing something like a Monte Carlo simulation it would be useful to have the ability to create pools of Threads and group them together and then submit a task to each of these threads for execution. Furthermore a Runnable interface does not allow a return value so it would also be useful to use Callable that returned a Future that would allow you to receive a return value. This is what Executors help you to do.
Firstly Executor is an Interface with an execute method that only supports a Runnable interface.
ExecutorService interface is a Super Interface of Executor and it supports a Callable interface via its submit method this Callable then returns a Future and Callable can throw a checked Exception.
Future submit(Callable task )
ScheduledExecutorService interface is a super interface of ExecutorService which allows you to schedule Runnable or Callable tasks at periodic intervals.
Class Executors /* Notice the Singular is the interface the plural is the Class */
There are factory methods on the Executors class that allow you to create different types of pools. Here are some examples - the API has many more.
ExecutorService cachedPool = Executors.newCachedThreadPool();
int nThreads = 1000;
ExecutorService fixedPool = Executors.newFixedThreadPool(nThreads );
They all return an ExecutorService from which you can execute or submit your tasks.
Fork/Join
The Fork Join paradigm is a work stealing Algorithm and is best suited for lots of little but identical tasks that can be recursively stitched back together later. For example imagine a 2d array of numbers representing an image. You could flatted this into one giant Array and then divide by 2 giving you 2 clear Arrays to work on in parallel.
Lets look at the classes involved.
ForkJoinPool class extends AbstractExecutorService class.
To use the ForkJoinPool your class must extend RecursiveTask which can return a result or use RecursiveAction
The API has a good bit of example code showing how to use this Paradigm.
Concurrent Collections
Interface BlockingQueue - no nulls blocks if full. Allows blocking calls put(e) and take() and is designed for Produce Consumer patterns
Classes implementing BlockingQueue
ArrayBlockingQueue - FIFO bounded Array backed
DelayQueue - unbounded only take once delay has expired
LinkedBlockingDeque - optionally bounded linked Deque (Double ended Q )
LinkedBlockingQueue - opt bounded liked FIFO
LinkedTransferQueue - unbounded linked FIFO
PriorityBlockingQueue - natural ordering Q must implement comparator.
SynchronousQueue - insert must wait for remove designed for handoff patterns
ConcurrentMap interface the implementation is ConcurrentHashMap - it allows striped access where put(e) is in a happens before relationship to retrieval.
ConcurrentNavigableMap sub interface of ConcurrentMap
The concrete class is ConcurrentSkipListMap which can be seen as a concurrent TreeMap. The ordering is natural ordering or driven by a Comparator.
ThreadLocalRandom
If you have ever to tried to write a Monte Carlo simulation in Java you have probably run into the problem of trying to create a random number that is truly random to each thread. Well there is now an easy way by using the ThreadLocalRandom that is random per thread so each thread can generate its own random numbers.
Naive Bayes classification AI algorithm
K-Means Clustering AI algorithm
Equity Derivatives tutorial
Fixed Income tutorial
Java
python
Scala
Investment Banking tutorials
HOME
This is a quick tour of the basics of Threading. I assume here you have a basic grasp of Java ( Imperative and Functional ) and Generics.
Thread creation
Thread myThread = new Thread(() -> {/*put runnable code here */} );
myThread.start();
This creates an on the fly runnable using the fact that runnable is functional and passes it to Thread and then the thread is started. You can only start a thread once. Once started the thread can not be stopped unless you either catch an InterruptedException and act on it or your pid is killed outside a java context. This implies that threads need a clear flow of what they will do and how they will end. This leads to the idea that Runnable and Callable’s are really tasks that can be given to a Thread or given to an Executor to be handled.
With the Thread you can call Thread.sleep(n) within the InterruptedException context as it might throw one. This sleeps the thread unless interrupted and keeps any locks held while asleep.
You can also wait for an Object to notify you that it is completed with its work by calling the Object.wait() which suspends the current Thread. By doing this you explicitly release your locks and wait for an object to call notify() and hope you are chosen or notifyAll() which notifies all the Objects that are waiting. You should have noticed here this is the Observer Observable pattern.
The one thing to watch for is that you can not wait on a specific thread ( you can join as below ). If you call Object.wait() the current thread will be suspended until notify()/NotifyAll() is issued somewhere. The notify might not be the specific notification you are looking for so you need to check that the notification is the one you want. From the API docs there is a clear example showing that you must check the notification as you wait ie:
while(!joy) {
try {
wait();
} catch (InterruptedException e) {}
}
You can also wait for another thread to to complete and attach yourself to continue after that thread’s runnable completes. To do this you call t.join() where t is the Thread you are waiting on.
Threads need to be careful of Synchronization ( Happens before ) and memory consistency / race errors ( Dirty reads / writes ). There are various strategies that can be used to avoid these problems. Some of these will be discussed here.
Synchronized methods
You could synchronize at the method level however that takes a monitor lock on the object. The lock you take is re-entrant meaning that once you have the lock you can re use it or reenter as much as you like before releasing
Synchronized blocks
You can synchronise a block of code only, synchronising on the this but recall this is just a monitor lock on the object.
Static Synchronization
As above but if you are in a static context you do not take the monitor lock from the object but from the class.
Volatile keyword
This keyword allows you to declare a variable this will always be involved in a check before act relationship. This means the variable will be checked to be the latest before acting on the variable.
Atomic and Accumulator utility classes
Java has a few Atomic classes to assist in the happens before relationship where the set() happens before the get() in a threaded context. These are based on most of the primitive values for example AtomicLong AtomicInteger plus we also have accumulators like LongAccumulator
Livelock and Starvation
When having thread interaction it is very important that we do not have a situation where there are too many threads running and the CPU is unable to meet all the compute request time for the threads. This can lead to starvation where the threads can compute but are not getting enough compute resource, this is the essence of starvation.
Another form of starvation and live-lock is when we have threads that spin waiting for a resource to become free. The thread is still good and holding locks and can run but it can not proceed as it is still waiting or spinning on something to occur before proceeding.
The best way to prevent situations like this is to not have too many threads such that the machine is constantly context switching, this can be observed via the CPU stats on the machine. You should also periodically log or have a way of indicating that you are waiting on a resource or timeout if you can not acquire what you need in a specified time period.
These strategies should generally be in a concurrency plan that all developers follow. This plan usually gives you the locking order so we always lock in the same order. How long we wait before doing a timeout and what to do in order to respond to an orderly shutdown of a long compute task. The order of locking becomes even more important when we talk about deadlock.
Deadlock (lock order strategy)
Dead lock is something you want to avoid at all costs because it implies that you are waiting on a resource and that same resource is waiting on you. Depending on the order that the locks were taken, if you keep waiting for each other then neither of you can do anything and the system generally comes to a halt. This can be prevented by always locking in the same order. If you always lock in a predefined order then you can never acquire the lock that causes you to get into a deadlock as it is locked and you would block.
Immutable Rules
Immutables are a useful strategy when it comes to threading. The real key here is that once you have an object that can not be changed you can freely pass it around between threads and the data never gets changed, saving you Memory and race conditions as nothing can change.
However you need to create an immutable carefully
- no setter methods
- all fields private and final
- Can not subclass so private constructor or better create via a factory method.
- No storing of References ( So they do not leak out and get modified )
- any method returns a deep copy of state rather than a ref.
Lock Interface from the Concurrent Package
We looked at the monitor locks above to actually lock an object so we can make changes, but there are some issues with just getting the lock. For example when you try to get the lock you will block until the lock becomes free. You also can not distinguish between a Read Lock and a Write lock. That is why there is the Lock interface.
This interface allows us to try a lock and if its not free we can then do something else rather than blocking.
Also dependant on the Implementation of the lock interface you can have Re-entrant Read and Write locks.
Perhaps an example will help
Lock myLock = new ReentrantLock();
if(myLock.tryLock()){
try{
//got lock
}finally{
myLock.unlock();
}
}else{
//No luck with lock
}
The lock implementations also include a readwrite lock where you can not acquire a read lock if there is a write lock open but you may have many readers. This is useful when looking at 1 writer many readers or when implementing a Disruptor Pattern.
ReentrantReadWriteLock rrw = new ReentrantReadWriteLock();
if(rrw.readLock().tryLock()){
;
}
Executors
Now we have looked at Threads which is a single execution of a Runnable. However Thread creation is expensive so if you were writing something like a Monte Carlo simulation it would be useful to have the ability to create pools of Threads and group them together and then submit a task to each of these threads for execution. Furthermore a Runnable interface does not allow a return value so it would also be useful to use Callable that returned a Future that would allow you to receive a return value. This is what Executors help you to do.
Firstly Executor is an Interface with an execute method that only supports a Runnable interface.
ExecutorService interface is a Super Interface of Executor and it supports a Callable interface via its submit method this Callable then returns a Future and Callable can throw a checked Exception.
Class Executors /* Notice the Singular is the interface the plural is the Class */
There are factory methods on the Executors class that allow you to create different types of pools. Here are some examples - the API has many more.
ExecutorService cachedPool = Executors.newCachedThreadPool();
int nThreads = 1000;
ExecutorService fixedPool = Executors.newFixedThreadPool(nThreads );
They all return an ExecutorService from which you can execute or submit your tasks.
Fork/Join
The Fork Join paradigm is a work stealing Algorithm and is best suited for lots of little but identical tasks that can be recursively stitched back together later. For example imagine a 2d array of numbers representing an image. You could flatted this into one giant Array and then divide by 2 giving you 2 clear Arrays to work on in parallel.
Lets look at the classes involved.
ForkJoinPool class extends AbstractExecutorService class.
To use the ForkJoinPool your class must extend RecursiveTask which can return a result or use RecursiveAction
The API has a good bit of example code showing how to use this Paradigm.
Concurrent Collections
Interface BlockingQueue - no nulls blocks if full. Allows blocking calls put(e) and take() and is designed for Produce Consumer patterns
Classes implementing BlockingQueue
ArrayBlockingQueue - FIFO bounded Array backed
DelayQueue - unbounded only take once delay has expired
LinkedBlockingDeque - optionally bounded linked Deque (Double ended Q )
LinkedBlockingQueue - opt bounded liked FIFO
LinkedTransferQueue - unbounded linked FIFO
PriorityBlockingQueue - natural ordering Q must implement comparator.
SynchronousQueue - insert must wait for remove designed for handoff patterns
ConcurrentMap interface the implementation is ConcurrentHashMap - it allows striped access where put(e) is in a happens before relationship to retrieval.
ConcurrentNavigableMap sub interface of ConcurrentMap
The concrete class is ConcurrentSkipListMap which can be seen as a concurrent TreeMap. The ordering is natural ordering or driven by a Comparator.
ThreadLocalRandom
If you have ever to tried to write a Monte Carlo simulation in Java you have probably run into the problem of trying to create a random number that is truly random to each thread. Well there is now an easy way by using the ThreadLocalRandom that is random per thread so each thread can generate its own random numbers.
People who enjoyed this article also enjoyed the following:
Naive Bayes classification AI algorithm
K-Means Clustering AI algorithm
Equity Derivatives tutorial
Fixed Income tutorial
And the following Trails:
C++Java
python
Scala
Investment Banking tutorials
HOME