Python

ThreadPoolExecutor Python

ThreadPoolExecutor is a python class found in its package named “concurrent.futures”. Let’s say we needed to make a lot of threads into our highly parallelized jobs. It is the costliest, because there may be several efficiency concerns as a result of having so many more processes. A big concern might be that throughput becomes constrained. You can overcome this issue by constructing a thread pool. A thread pool is a collection of pre-instantiated or inactive processes that are waiting to be assigned work. Whenever a significant set of activities must be completed, establishing a thread pool is usually recommended to develop new processes for each job. A thread pool can do the following tasks:

  • Once a process inside a thread pool has completed its processing, it could be used again.
  • When a thread is ended, a new thread is formed to take its place.

The concurrent.futures package is a portion of the Python regular library. In Python 3.2, this package was created to give programmers an elevated interface for initiating concurrent processes. That is an intermediate level built atop of Python’s multitasking as well as multiprocessing packages that lets you perform tasks utilizing a pool of threads or procedures. Within this article, we will have a look at the thread pool executor class and its functions.

Example 01:

Let’s have a look at the thread pool executor class within our python code. Firstly, we have used the “concurrent.futures” package using the “from” keyword in the first line to import the “ThreadPoolExecutor” class in the code. After this, we have initialized a list of 4 numbers. The first function is a user-defined function named “square” and it will be calculating a square of a value passed in its arguments. The other method is the constructor function i.e., main() method as the execution starts from i. So, the user-defined function “square” is defined using the keyword “def” and takes the argument value to calculate its square.

The “{}” brackets are used to take the numerical value of a variable “z” instead of taking “z” as a string here. “f” character before the text and calculation statement shows that the values will be taken as floats and their square will be calculated. The print statement is utilizing this format to get a square and print it in the Spyder3 output shell. Now, here comes the main() function constructor. The constructor is initializing an empty list named “val”. It’s time to use the ThreadPoolExecutor class to make a pool of 5 threads to be used asynchronously.

For this, make use of the “with” statement with the name “ThreadPoolExecutor” taking an argument “max_workers” with value 5. You can see it as a loop taking 1 value each time. The submit function of ThreadPoolExecutor class is used to execute a function i.e., “square” and yields a square of a number passed to it. It is taking number 1 as starting value to the square method and after this, it will take the values from list “l”. The “map” function is used here to quickly translate the function and iterable collectively as one and raises an error in the “concurrent.futures” package. The exception will be stored to the variable “val” and the print statement will be used to display that exception used in the “for” loop on the output screen.

from concurrent.futures import ThreadPoolExecutor
l = [2,3,4,5]
def square(z):
    print(f'Square of {z}: {z*z}')
if __name__ == '__main__':
    val =[]
    with ThreadPoolExecutor(max_workers=5) as exe:
        exe.submit(square,1)
        val = exe.map(square,l)  
    for res in val:
      print(res)

The pool of 5 threads has been created to calculate a square of 5 values starting from 1 to 5 as displayed in the output.

Example 02:

Here comes the last example of using the ThreadPoolExecutor class in Python code. So, we have started these new examples with the import of a class using its package “concurrent.futures”. Then, initialize a variable “sum” with 0 value. The user-defined function named “Process” is declared taking an argument “n” in it. This function is printing the thread numbers via the print statement. The thread number is summed up with the number 5 added to the variable “sum”. This calculated sum for each thread will be shown on the output employing the print statement. The main method’s definition has been started here. It’s using the ThreadPoolExecutor for the creation of a pool of 3 threads. The “Process” function has been called 3 times using the “submit” function for 3 different values. After all the threads of the Process method get executed, it will display the completion message. The main() function declaration is used after that.

from concurrent.futures import ThreadPoolExecutor
Sum = 0
def Process(n):
 print("Processing the thread : {}".format(n))
 Sum = n + 5
 print("Sum of {} and 5 is: ".format(n) , Sum)
def main():
 print("Starting ThreadPoolExecutor")
 with ThreadPoolExecutor(max_workers=3) as ex:
   val = ex.submit(Process, (2))
   val = ex.submit(Process, (4))
   val = ex.submit(Process, (5))
 print("All the threads are processed completely")

if __name__ == '__main__':
 main()

The execution shows the Pool is started and each thread number with its sum with number 5 is displayed.

Conclusion:

We have covered and discussed the ThreadPoolExecutor class and all the possibly used functions of it in our examples. We ought to make it as unpretentious as likely for our users. We have taken a look at a map and submit function as well. We are expecting the best response.

About the author

Kalsoom Bibi

Hello, I am a freelance writer and usually write for Linux and other technology related content

Page was generated in 0.053946018218994