Executor Service(Java Thread Pool Framework)

8 min readMar 26, 2020

1. Single Threaded Application

This is the simple java program for thread.

Every java programs has at least one thread. The Main thread in Java is the one that begins executing when the program starts.

According to the diagram, main thread and thread A run in parallely. Thread A is a user thread.

2. Created Threads for Every Tasks

Thread objects use a significant amount of memory, and in a large-scale application, allocating and de-allocating many thread objects creates a significant memory management overhead.
A system that creates a new thread for every request would spend more time and consume more system resources in creating and destroying threads than processing actual requests.

In here we created 100 threads in this java program. So we need to manage these 100 threads properly. Otherwise this java program may be crashed due to out of memory.

That is why, Thread Pool is introduced.

3. Thread Pool

This is collection of threads on which task can be scheduled. Instead of creating new threads for each task, select a thread from the thread pool and execute that task.

Thread pool is minimizing JVM overhead due to thread creation.
Since active threads consume system resources, a JVM creating too many threads at the same time can cause the system to run out of memory.
Executor framework in Java 1.5 uses a thread pool in this context.

4. What is Executor Framework?

While we are working small number of threads, it is easy to use and we don’t need to face much challenges. But we end up with lots of concurrent tasks, it is hard to manage threads manually. Moreover lots of boil plate code.

This works well for small applications, but in large-scale applications, it makes sense to separate thread management and creation from the rest of the application. Objects that encapsulate these functions are known as executors.

ThreadPoolExecutor is actual implementation of ThreadPool. You can create ThreadPoolExecutor from factory methods of Executor class.

There are four factory methods of Excutor class.

newFixedThreadPool
newCachedThreadPool
newSingleThreadedExecutor
newScheduledThreadPool

1. Fixed Thread Pool

When we create FixedThreadPool, it has fixed number of threads. In here we submit the tasks to exector. All the submitted tasks are stored in particular queue. This queue is thread safe(blocking queue). All the threads fetch tasks from the queue.

If one thread is finished task, it fetch next task from the queue and execute it.

According to the output, all the threads in the thread pool execute by selecting tasks from the blocking queue.

In here, 10 threads are running in parallel y. when one thread is completed one task, it get a task from the blocking queue and execute it.

2. Cached Thread Pool

This does not have fixed size of threads. This is an expandable thread pool.
This is suitable for application which have lot of short live tasks.
According to the below diagram,when coming new task to the cached thread pool, if all the threads are busy, cached thread pool is created new thread for that new task.
If thread is idle for 1 minute, it will tear down those threads.

In here, this thread pool created lots of threads. all the created threads are executed one of the submitted task.

3. Scheduled Thread Pool

If you need to schedule the task after certain delay, this is the thread pool for it.
If you need a check security check, login check within every 10 seconds, we can use scheduled thread pool.
There are three methods to schedule a task,

schedule
scheduleAtFixedRate
scheduleAtFixedDelay

i. schedule()

This method schedules the given Callable for execution after the given delay.

According to the program, this thread pool has three threads. First all the tasks stored in the delayed queue by submitting all the tasks.

We have defined a delay as 20 seconds. So after submitting task, those three threads started to execute three tasks from the delayed queue after 20 seconds.

submitting time was 20:58:36. But thread is executed one of the task at 20:58:56. Difference between those two time was 20.

ii. scheduleAtFixedRate(Task, initial delay, period, Time unit)

If we are using this, task is executed first time after initial delay. After that those task are executed periodically. next time those started tasks are executed after that period.

start of the next execution -  start of the previous execution = period

1 → task 01- now() + delay
2 → task 01-now() + delay + period
3 → task 03-now() + delay + period + period

If any execution of the given task throws an exception, the task is no longer executed.

If no exceptions are thrown, the task will continue to be executed until the executor service is shut down.

According to the output, all the five tasks submitted first.

we will consider task0. submitting time of task0 is 21:49:29. But it started after 20 seconds. it was 21:49:49.

after finishing task0 by therad1, again thread3 executed task0 after 10 seconds since that task0 previous executed time.

all the tasks are executing until executor is shutdown.

iii. scheduleAtFixedDelay(Task, initial delay, period, Time unit)

As scheduleAtFixedRate, task is executed first time after initial delay. After that those task are executed periodically. next time those started tasks are executed after that period since end of that task.

end of the previous execution + period = start of the next execution

1 → task 01- endtime

2 → task 01- endtime+ period

According to the output, all the five tasks submitted first.

we will consider task0. submitting time of task0 is 22:13:55. But it started after 20 seconds. it was 22:14:15.

after finishing task0 by therad1, again thread3 executed task0 after 10 seconds since that task0 previous end time.

next executing time of task0= 22:14:30
previous end time of task0 = 22:14:20

difference between above two time is 10 seconds. it was the period duration which we provided into that scheduleAtFixedDelay method.

all the tasks are executing until executor is shutdown.

4. Single Threaded Executor

Here, only one thread is using in the thread pool. All the tasks are keeping in blocking queue.

After finishing task, it fetch new task from the queue and execute it.

In here only one thread executed all the submitted tasks. in the output, there is only thread1.

5. Future Task

Future task is implemented using RunnableFuture interface. RunnableFuture is implemented by Future and Runnable interface.

We can use FutureTask as Runnable and execute by using executor service.

6. Future

Future object functions as a handle to the result of the asynchronous task.
Once the asynchronous task completes, the result can be accessed through Future.
When the task was started, Future returned. At that time Future does have a value.

There are most important methods of Future objects,

i. cancel()- You can cancel the asynchronous task(Future Object) by calling the Future cancel() method.

ii. isDone()- You can check if the asynchronous task is done or not by calling the Future isDone() method. When Future is done, we can get a result at that time.

iii. isCanceled()- to check whether the asynchronous task(Future object) is cancelled.

iv. get()- We can get the result from the Furure.

7. InvokeAny

we can submit collection of tasks using invokeAny(). We need to provide collection of callable.
But it does not return futures. But it return the result of one of the submitted callable object. So we do not know that result’s callable object.
If one of the task is completed, other submitted tasks are cancelled.
Moreover if one callable throws an exception, all other submitted tasks are cancelled.

According to the result, one result is returned. Actually, We don’t know that result is belong to which task.

8. InvokeAll

It waits for all the tasks to complete. In here also, we are sumbitted collection of callable tasks. It returns a list of futures with their status and results. We get the result only after all the tasks are completed.
When you need to wait for result of all tasks, use invokeAll. Otherwise you can use submit of task.
In here, when one task throw an exception, other task complete without any problem.