Can someone please explain what is thread per request and thread per connection? Which model do servlets work on? How threads are allocated to handle HTTP requests? Is it thread/request or connection?

And let&#39;s say if I want to perform a time consuming task in my `Servlet`&#39;s `doGet()` method asynchronously, I start a new thread using Java executors so that lengthy calculations are done in a separate thread and response is sent right away.

Now does that ensure that I have freed the thread which had been processing my `HttpServletRequest` or is it still being used because a child thread is still running?



How are Threads allocated to handle Servlet request?

In a Spring web application I have several DAO and service layer beans. One service layer bean has annotated @Async / @Scheduled methods. These methods depend on other (autowired) beans.
I have configured two thread pools in XML:

    &lt;bean id=&quot;taskExecutor&quot; class=&quot;org.springframework.scheduling.concurrent.ThreadPoolTaskExecutor&quot;&gt;
         &lt;property name=&quot;corePoolSize&quot; value=&quot;2&quot; /&gt;
         &lt;property name=&quot;maxPoolSize&quot; value=&quot;5&quot; /&gt;
         &lt;property name=&quot;queueCapacity&quot; value=&quot;5&quot; /&gt;
         &lt;property name=&quot;waitForTasksToCompleteOnShutdown&quot; value=&quot;true&quot; /&gt;
         &lt;property name=&quot;rejectedExecutionHandler&quot;&gt;
    			&lt;bean class=&quot;java.util.concurrent.ThreadPoolExecutor$CallerRunsPolicy&quot;/&gt;
    		&lt;/property&gt;
    	&lt;/bean&gt;
    	
    &lt;bean id=&quot;taskScheduler&quot; class=&quot;org.springframework.scheduling.concurrent.ThreadPoolTaskScheduler&quot;&gt;
         &lt;property name=&quot;poolSize&quot; value=&quot;10&quot; /&gt;
         &lt;property name=&quot;waitForTasksToCompleteOnShutdown&quot; value=&quot;true&quot; /&gt;
         &lt;property name=&quot;rejectedExecutionHandler&quot;&gt;
    			&lt;bean class=&quot;java.util.concurrent.ThreadPoolExecutor$CallerRunsPolicy&quot;/&gt;
    		&lt;/property&gt;
    	&lt;/bean&gt;
    
    	&lt;task:annotation-driven executor=&quot;taskExecutor&quot; scheduler=&quot;taskScheduler&quot;/&gt;

Everything works as expected. My problem is that I cannot get a clean shutdown of the task pools to work. The tasks operate on the database and on the file system. When I stop the web application it takes some time until it is stopped. This indicates that the `waitForTasksToCompleteOnShutdown` property works. However, I get IllegalStateExceptions in the log indicating that some beans are already destroyed but some worker task threads are still executing and they fail because their dependencies are destroyed.

There is a JIRA issue which might be relevant: [SPR-5387 ][1]

**My question is: Is there a way to tell Spring to initialize the task executor/scheduler beans last or is there a way to tell Spring to destroy them first?**

My understanding is that destruction takes place in reversed init order. Therefore the bean init&#39;ed last will be destroyed first. If the thread pool beans are destroyed first, all currently executing tasks would finish and could still access dependent beans.

I have also tried using the depends-on attribute on the thread pools referring to my service bean which has the @Async and @Scheduled annotations. Seems like they are never executed then and I do not get context initialization errors. I assume the annotated service bean somehow needs these thread pools initialized first and if I use depends-on I reverse the order and make them non-functional.

  [1]: https://jira.springsource.org/browse/SPR-5387 &quot;SPR-5387&quot;

How can I shutdown Spring task executor/scheduler pools before all other beans in the web app are destroyed?

My multithreading concepts are weak and trying to learn.

In Java what I know is, we can&#39;t call a thread more than once: 

    Thread t = new Thread; //Some Runnable
    t.start();
    
    t.start(); //Illegal and throw Exception at runtime.

As far as I know, it throws exception when you call `t.start()` again because the associated stack for the thread is destroyed once it goes out of `run()` method and you are trying to initialize things again.

In that case, what I know about thread pool is, it gives better performance &amp; saves time because there is no need to create new thread (I read in [this][1]).

If there is no need to create new thread in thread pool scenario, then how it works with same thread which just finished its run method, will that thread can be used again?


I read [this][2], and it says that &quot;**Most of the executor implementations in java.util.concurrent use thread pools, which consist of worker threads. This kind of thread exists separately from the Runnable and Callable tasks it executes and is often used to execute multiple tasks.**&quot;

So what is Worker thread here, is it something different then normal Java threads?

With [this][3] link, I got something but still confused on what kind of stuff can be eliminated when we use thread pool and why it gives better performance than using normal java threads.

So can we say like this,

Thread has three parts,

 1. Creation (Telling OS that it is new thread, create stack for it.)
 2. Execute Runnable with run() method.
 3. Destroying threads.

So, considering above 3 steps, With thread pool step 1 and step 3 can be eliminated after fixed number of thread creation. Only step 2 for each task will be executed that is why threadpool is faster? Can we say like this? Am I correct?


  [1]: http://www.javatpoint.com/thread-pooling-in-java
  [2]: http://docs.oracle.com/javase/tutorial/essential/concurrency/pools.html
  [3]: https://stackoverflow.com/questions/13699335/how-does-a-cached-thread-pool-reuse-existing-threads

How Threadpool re-use Threads and how it works

I wrote a couple of action methods in a controller to test the difference between **sync** and **async** controller actions in ASP.NET core:

    [Route(&quot;api/syncvasync&quot;)]
    public class SyncVAsyncController : Controller
    {
        [HttpGet(&quot;sync&quot;)]
        public IActionResult SyncGet()
        {
            Task.Delay(200).Wait();
    
            return Ok(new { });
        }
    
        [HttpGet(&quot;async&quot;)]
        public async Task&lt;IActionResult&gt; AsyncGet()
        {
            await Task.Delay(200);
    
            return Ok(new { });
        }
    }

I then load tested the sync end point:
[![enter image description here][1]][1]

... followed by the async end point:
[![enter image description here][2]][2]

Here&#39;s the results if I increase the delay to 1000 ms
[![enter image description here][3]][3]
[![enter image description here][4]][4]

As you can see there is not much difference in the **requests per second** - I expected the async end point to handle more requests per second. Am I missing something?


  [1]: https://i.stack.imgur.com/SFIiW.png
  [2]: https://i.stack.imgur.com/lCcsY.png
  [3]: https://i.stack.imgur.com/E9TSS.png
  [4]: https://i.stack.imgur.com/c4ISe.png

Not much difference between ASP.NET Core sync and async controller actions

I have been reading about the thread-pool pattern and I can&#39;t seem to find the usual solution for the following problem.

I sometimes want tasks to be executed serially. For example, I read chunks of text from a file and for some reason I need the chunks to be processed in that order. So basically I want to eliminate concurrency **for some of the tasks**.

Consider this scenario where the tasks with `*` need to be processed in the order they were pushed in. The other tasks can be processed in any order.

    push task1
    push task2
    push task3   *
    push task4   *
    push task5
    push task6   *
    ....
    and so on


In the context of a thread-pool, without this constraint, a single queue of pending tasks works fine but clearly here it doesn&#39;t.

I thought about having *some of the threads* operate on a thread-specific queue and the others on the &quot;global&quot; queue. Then, in order to execute some of the tasks serially, I simply have to push them onto a queue where a single thread looks. It *does* sounds a bit clumsy.

So, the real question in this long story: how would you solve this ? **How would you ensure those tasks are ordered**?

### EDIT

As a more general problem, suppose the scenario above becomes


    push task1
    push task2   **
    push task3   *
    push task4   *
    push task5
    push task6   *
    push task7   **
    push task8   *
    push task9
    ....
    and so on

What I mean is that the tasks within a group should be executed sequentially, but the groups themselves can mix. So you can have `3-2-5-4-7` for example.

One other thing to note is that I don&#39;t have access to all the tasks in a group upfront (and I can&#39;t wait for all of them to arrive before starting the group).

Ensuring task execution order in ThreadPool

As far as I had understood `ForkJoinPool`, that pool creates a fixed number of threads (default: number of cores) and will never create more threads (unless the application indicates a need for those by using `managedBlock`). 

However, using `ForkJoinPool.getPoolSize()` I discovered that in a program that creates 30,000 tasks (`RecursiveAction`), the `ForkJoinPool` executing those tasks uses 700 threads on average (threads counted each time a task is created). The tasks don&#39;t do I/O, but pure computation; the only inter-task synchronization is calling `ForkJoinTask.join()` and accessing `AtomicBoolean`s, i.e. there are no thread-blocking operations. 

Since `join()` does not block the calling thread as I understand it, there is no reason why any thread in the pool should ever block, and so (I had assumed) there should be no reason to create any further threads (which is obviously happening nevertheless).

So, why does `ForkJoinPool` create so many threads? What factors determine the number of threads created?

I had hoped that this question could be answered without posting code, but here it comes upon request. This code is an excerpt from a program of four times the size, reduced to the essential parts; it does not compile as it is. If desired, I can of course post the full program, too.

The program searches a maze for a path from a given start point to a given end point using depth-first search. A solution is guaranteed to exist. The main logic is in the `compute()` method of `SolverTask`: A `RecursiveAction` that starts at some given point and continues with all neighbor points reachable from the current point. Rather than creating a new `SolverTask` at each branching point (which would create far too many tasks), it pushes all neighbors except one onto a backtracking stack to be processed later and continues with only the one neighbor not pushed to the stack. Once it reaches a dead end that way, the point most recently pushed to the backtracking stack is popped, and the search continues from there (cutting back the path built from the taks&#39;s starting point accordingly). A new task is created once a task finds its backtracking stack larger than a certain threshold; from that time, the task, while continuing to pop from its backtracking stack until that is exhausted, will not push any further points to its stack when reaching a branching point, but create a new task for each such point. Thus, the size of the tasks can be adjusted using the stack limit threshold. 

The numbers I quoted above (&quot;30,000 tasks, 700 threads on average&quot;) are from searching a maze of 5000x5000 cells. So, here is the essential code:

    class SolverTask extends RecursiveTask&lt;ArrayDeque&lt;Point&gt;&gt; {
	// Once the backtrack stack has reached this size, the current task
	// will never add another cell to it, but create a new task for each
	// newly discovered branch:
	private static final int MAX_BACKTRACK_CELLS = 100*1000;

	/**
	 * @return Tries to compute a path through the maze from local start to end
	 * and returns that (or null if no such path found)
	 */
	@Override
	public ArrayDeque&lt;Point&gt;  compute() {
		// Is this task still accepting new branches for processing on its own,
		// or will it create new tasks to handle those?
		boolean stillAcceptingNewBranches = true;
		Point current = localStart;
		ArrayDeque&lt;Point&gt; pathFromLocalStart = new ArrayDeque&lt;Point&gt;();  // Path from localStart to (including) current
		ArrayDeque&lt;PointAndDirection&gt; backtrackStack = new ArrayDeque&lt;PointAndDirection&gt;();
		// Used as a stack: Branches not yet taken; solver will backtrack to these branching points later

		Direction[] allDirections = Direction.values();
		
		while (!current.equals(end)) {
			pathFromLocalStart.addLast(current);
			// Collect current&#39;s unvisited neighbors in random order: 
			ArrayDeque&lt;PointAndDirection&gt; neighborsToVisit = new ArrayDeque&lt;PointAndDirection&gt;(allDirections.length);  
			for (Direction directionToNeighbor: allDirections) {
				Point neighbor = current.getNeighbor(directionToNeighbor);
				
				// contains() and hasPassage() are read-only methods and thus need no synchronization
				if (maze.contains(neighbor) &amp;&amp; maze.hasPassage(current, neighbor) &amp;&amp; maze.visit(neighbor))
					neighborsToVisit.add(new PointAndDirection(neighbor, directionToNeighbor.opposite));
			}
			// Process unvisited neighbors
			if (neighborsToVisit.size() == 1) {
				// Current node is no branch: Continue with that neighbor
				current = neighborsToVisit.getFirst().getPoint();
				continue;
			}
			if (neighborsToVisit.size() &gt;= 2) {
				// Current node is a branch
				if (stillAcceptingNewBranches) {
					current = neighborsToVisit.removeLast().getPoint();
					// Push all neighbors except one on the backtrack stack for later processing
					for(PointAndDirection neighborAndDirection: neighborsToVisit) 
						backtrackStack.push(neighborAndDirection);
					if (backtrackStack.size() &gt; MAX_BACKTRACK_CELLS)
						stillAcceptingNewBranches = false;
					// Continue with the one neighbor that was not pushed onto the backtrack stack
					continue;
				} else {
					// Current node is a branch point, but this task does not accept new branches any more: 
					// Create new task for each neighbor to visit and wait for the end of those tasks
					SolverTask[] subTasks = new SolverTask[neighborsToVisit.size()];
					int t = 0;
					for(PointAndDirection neighborAndDirection: neighborsToVisit)  {
						SolverTask task = new SolverTask(neighborAndDirection.getPoint(), end, maze);
						task.fork();
						subTasks[t++] = task;
					}
					for (SolverTask task: subTasks) {
						ArrayDeque&lt;Point&gt; subTaskResult = null;
						try {
							subTaskResult = task.join();
						} catch (CancellationException e) {
							// Nothing to do here: Another task has found the solution and cancelled all other tasks
						}
						catch (Exception e) {
							e.printStackTrace();
						}
						if (subTaskResult != null) { // subtask found solution
							pathFromLocalStart.addAll(subTaskResult);
							// No need to wait for the other subtasks once a solution has been found
							return pathFromLocalStart;
						}
					} // for subTasks
				} // else (not accepting any more branches) 
			} // if (current node is a branch)
		    // Current node is dead end or all its neighbors lead to dead ends:
		    // Continue with a node from the backtracking stack, if any is left:
			if (backtrackStack.isEmpty()) {
				return null; // No more backtracking avaible: No solution exists =&gt; end of this task
			}
			// Backtrack: Continue with cell saved at latest branching point:
			PointAndDirection pd = backtrackStack.pop();
			current = pd.getPoint();
			Point branchingPoint = current.getNeighbor(pd.getDirectionToBranchingPoint());
			// DEBUG System.out.println(&quot;Backtracking to &quot; +  branchingPoint);
			// Remove the dead end from the top of pathSoFar, i.e. all cells after branchingPoint:
			while (!pathFromLocalStart.peekLast().equals(branchingPoint)) {
				// DEBUG System.out.println(&quot;    Going back before &quot; + pathSoFar.peekLast());
				pathFromLocalStart.removeLast();
			}
			// continue while loop with newly popped current
		} // while (current ...
		if (!current.equals(end)) {			
			// this task was interrupted by another one that already found the solution 
			// and should end now therefore:
			return null;
		} else {
			// Found the solution path:
			pathFromLocalStart.addLast(current);
			return pathFromLocalStart;
		}
	} // compute()
    } // class SolverTask

    @SuppressWarnings(&quot;serial&quot;)
    public class ParallelMaze  {
 	
	// for each cell in the maze: Has the solver visited it yet?
 	private final AtomicBoolean[][] visited;

	/**
	 * Atomically marks this point as visited unless visited before
	 * @return whether the point was visited for the first time, i.e. whether it could be marked
	 */
	boolean visit(Point p) {
		return  visited[p.getX()][p.getY()].compareAndSet(false, true);
	}

	public static void main(String[] args) {
		ForkJoinPool pool = new ForkJoinPool();
		ParallelMaze maze = new ParallelMaze(width, height, new Point(width-1, 0), new Point(0, height-1));
	    // Start initial task
		long startTime = System.currentTimeMillis();
		 // since SolverTask.compute() expects its starting point already visited, 
		// must do that explicitly for the global starting point:
		maze.visit(maze.start);
		maze.solution = pool.invoke(new SolverTask(maze.start, maze.end, maze));
		// One solution is enough: Stop all tasks that are still running
		pool.shutdownNow();
	    pool.awaitTermination(Integer.MAX_VALUE, TimeUnit.DAYS);
		long endTime = System.currentTimeMillis();
		System.out.println(&quot;Computed solution of length &quot; + maze.solution.size() + &quot; to maze of size &quot; + 
				width + &quot;x&quot; + height + &quot; in &quot; + ((float)(endTime - startTime))/1000 + &quot;s.&quot;);
    }



What determines the number of threads a Java ForkJoinPool creates?

Let&#39;s say I have an application that utilizes the `Executor` framework as such

    Executors.newSingleThreadExecutor().submit(new Runnable(){
        @Override
        public void run(){
            // do stuff
        }
    }

When I run this application in the debugger, a thread is created with the following (default) name: `Thread[pool-1-thread-1]`. As you can see, this isn&#39;t terribly useful and as far as I can tell, the `Executor` framework does not provide an easy way to name the created threads or thread-pools.

So, how does one go about providing names for the threads/thread-pools? For instance, `Thread[FooPool-FooThread]`.

Naming threads and thread-pools of ExecutorService

What is the simplest way to to wait for all tasks of `ExecutorService` to finish? My task is primarily computational, so I just want to run a large number of jobs - one on each core.  Right now my setup looks like this: 

    ExecutorService es = Executors.newFixedThreadPool(2);
    for (DataTable singleTable : uniquePhrases) {	
        es.execute(new ComputeDTask(singleTable));
    }
    try{
        es.wait();
    } 
    catch (InterruptedException e){
        e.printStackTrace();
    }

`ComputeDTask` implements runnable.  This appears to execute the tasks correctly, but the code crashes on `wait()` with `IllegalMonitorStateException`.  This is odd, because I played around with some toy examples and it appeared to work.  

`uniquePhrases` contains several tens of thousands of elements.  Should I be using another method?  I am looking for something as simple as possible

ExecutorService, how to wait for all tasks to finish

**Relevant questions**:

 About C++11:

 - https://stackoverflow.com/questions/12993451/c11-stdthread-pooled
 - https://stackoverflow.com/questions/14351352/will-asynclaunchasync-in-c11-make-thread-pools-obsolete-for-avoiding-expen

About Boost:

 - https://stackoverflow.com/questions/2659065/c-boost-thread-reusing-threads
 - https://stackoverflow.com/questions/3458397/boostthread-and-creating-a-pool-of-them

---

How do I get a **pool of threads** to **send tasks to**, without creating and deleting them over and over again? This means persistent threads to resynchronize without joining.

---

I have code that looks like this:

    namespace {
      std::vector&lt;std::thread&gt; workers;

      int total = 4;
      int arr[4] = {0};

      void each_thread_does(int i) {
        arr[i] += 2;
      }
    }

    int main(int argc, char *argv[]) {
      for (int i = 0; i &lt; 8; ++i) { // for 8 iterations,
        for (int j = 0; j &lt; 4; ++j) {
          workers.push_back(std::thread(each_thread_does, j));
        }
        for (std::thread &amp;t: workers) {
          if (t.joinable()) {
            t.join();
          }
        }
        arr[4] = std::min_element(arr, arr+4);
      }
      return 0;
    }

Instead of creating and joining threads each iteration, I&#39;d prefer to send tasks to my worker threads each iteration and only create them once.

Thread pooling in C++11

What is the difference between using a new thread and using a thread from the thread pool? What performance benefits are there and why should I consider using a thread from the pool rather than one I&#39;ve explicitly created? I&#39;m thinking specifically of .NET here, but general examples are fine.

Thread vs ThreadPool

I have a fixed thread pool that I submit tasks to (limited to **5** threads). How can I find out which one of those **5** threads executes my task (something like &quot;thread #3 of **5** is doing this task&quot;)?

    ExecutorService taskExecutor = Executors.newFixedThreadPool(5);
    
    //in infinite loop:
    taskExecutor.execute(new MyTask());
    ....
    
    private class MyTask implements Runnable {
    	public void run() {
    		logger.debug(&quot;Thread # XXX is doing this task&quot;);//how to get thread id?
    	}
    }

How to get thread id from a thread pool?

It is loosely related to this question: [Are std::thread pooled in C++11?][1]. Though the question differs, the intention is the same:

**Question 1: Does it still make sense to use your own (or 3rd-party library) thread pools to avoid expensive thread creation?**

The conclusion in the other question was that you cannot rely on `std::thread` to be pooled (it might or it might be not). However, `std::async(launch::async)` seems to have a much higher chance to be pooled.

It don&#39;t think that it is forced by the standard, but IMHO I would expect that all good C++11 implementations would use thread pooling if thread creation is slow. Only on platforms where it is inexpensive to create a new thread, I would expect that they always spawn a new thread.

**Question 2: This is just what I think, but I have no facts to prove it. I may very well be mistaken. Is it an educated guess?**

Finally, here I have provided some sample code that first shows how I think thread creation can be expressed by `async(launch::async)`:


Example 1:

     thread t([]{ f(); });
     // ...
     t.join();

becomes

     auto future = async(launch::async, []{ f(); });
     // ...
     future.wait();


Example 2: Fire and forget thread

     thread([]{ f(); }).detach();

becomes

     // a bit clumsy...
     auto dummy = async(launch::async, []{ f(); });

     // ... but I hope soon it can be simplified to
     async(launch::async, []{ f(); });

**Question 3: Would you prefer the `async` versions to the `thread` versions?**


----------

**The rest is no longer part of the question, but only for clarification:**

Why must the return value be assigned to a dummy variable?

Unfortunately, the current C++11 standard forces that you capture the return value of `std::async`, as otherwise the destructor is executed, which blocks until the action terminates. It is by some considered an error in the standard (e.g., by Herb Sutter).

This example from [cppreference.com][2] illustrates it nicely:

    {
      std::async(std::launch::async, []{ f(); });
      std::async(std::launch::async, []{ g(); });  // does not run until f() completes
    }

----------

Another clarification:

I know that **thread pools may have other legitimate uses but in this question I am only interested in the aspect of avoiding expensive thread creation costs**.

I think there are still situations where thread pools are very useful, especially if you need more control over resources.
For example, a server might decide to handle only a fixed number of requests simultaneously to guarantee fast response times and to increase the predictability of memory usage. Thread pools should be fine, here.

Thread-local variables may also be an argument for your own thread pools, but I&#39;m not sure whether it is relevant in practice:

  - Creating a new thread with `std::thread` starts without initialized thread-local variables. Maybe this is not what you want.
  - In threads spawned by `async`, it is somewhat unclear for me because the thread could have been reused. From my understanding, thread-local variables are not guaranteed to be resetted, but I may be mistaken.
  - Using your own (fixed-size) thread pools, on the other hand, gives you full control if you really need it.





  [1]: https://stackoverflow.com/questions/12993451/c11-stdthread-pooled
  [2]: http://en.cppreference.com/w/cpp/thread/async

Does async(launch::async) in C++11 make thread pools obsolete for avoiding expensive thread creation?

I have been trying to learn multi-threaded programming in C# and I am confused about when it is best to use a thread pool vs. create my own threads. One book recommends using a thread pool for small tasks only (whatever that means), but I can&#39;t seem to find any real guidelines.

What are some pros and cons of thread pools vs creating my own threads? And what are some example use cases for each?

When to use thread pool in C#?

I&#39;ve always been under the impression that using the ThreadPool for (let&#39;s say non-critical) short-lived background tasks was considered best practice, even in ASP.NET, but then I came across [this article][1] that seems to suggest otherwise - the argument being that you should leave the ThreadPool to deal with ASP.NET related requests.

So here&#39;s how I&#39;ve been doing small asynchronous tasks so far:

    ThreadPool.QueueUserWorkItem(s =&gt; PostLog(logEvent))

And [the article][1] is suggesting instead to create a thread explicitly, similar to:

    new Thread(() =&gt; PostLog(logEvent)){ IsBackground = true }.Start()

The first method has the advantage of being managed and bounded, but there&#39;s the potential (if the article is correct) that the background tasks are then vying for threads with ASP.NET request-handlers. The second method frees up the ThreadPool, but at the cost of being unbounded and thus potentially using up too many resources.

So my question is, is the advice in the article correct?

If your site was getting so much traffic that your ThreadPool was getting full, then is it better to go out-of-band, or would a full ThreadPool imply that you&#39;re getting to the limit of your resources anyway, in which case you shouldn&#39;t be trying to start your own threads?

Clarification: I&#39;m just asking in the scope of small non-critical asynchronous tasks (eg, remote logging), not expensive work items that would require a separate process (in these cases I agree you&#39;ll need a more robust solution).

  [1]: http://csharpfeeds.com/post/5415/Dont_use_the_ThreadPool_in_ASP.NET.aspx

Using ThreadPool.QueueUserWorkItem in ASP.NET in a high traffic scenario

In the code below, due to the interface, the class `LazyBar` must return a task from its method (and for argument&#39;s sake can&#39;t be changed). If `LazyBar`s implementation is unusual in that it happens to run quickly and synchronously - what is the best way to return a No-Operation task from the method?

I have gone with `Task.Delay(0)` below, however I would like to know if this has any performance side-effects if the function is called a *lot* (for argument&#39;s sake, say hundreds of times a second):

  - Does this syntactic sugar un-wind to something big?
  - Does it start clogging up my application&#39;s thread pool?
  - Is the compiler cleaver enough to deal with `Delay(0)` differently?
  - Would `return Task.Run(() =&gt; { });` be any different?
 
Is there a better way?

    using System.Threading.Tasks;

    namespace MyAsyncTest
    {
        internal interface IFooFace
        {
            Task WillBeLongRunningAsyncInTheMajorityOfImplementations();
        }

        /// &lt;summary&gt;
        /// An implementation, that unlike most cases, will not have a long-running
        /// operation in &#39;WillBeLongRunningAsyncInTheMajorityOfImplementations&#39;
        /// &lt;/summary&gt;
        internal class LazyBar : IFooFace
        {
            #region IFooFace Members

            public Task WillBeLongRunningAsyncInTheMajorityOfImplementations()
            {
                // First, do something really quick
                var x = 1;

                // Can&#39;t return &#39;null&#39; here! Does &#39;Task.Delay(0)&#39; have any performance considerations?
                // Is it a real no-op, or if I call this a lot, will it adversely affect the
                // underlying thread-pool? Better way?
                return Task.Delay(0);

                // Any different?
                // return Task.Run(() =&gt; { });

                // If my task returned something, I would do:
                // return Task.FromResult&lt;int&gt;(12345);
            }

            #endregion
        }

        internal class Program
        {
            private static void Main(string[] args)
            {
                Test();
            }

            private static async void Test()
            {
                IFooFace foo = FactoryCreate();
                await foo.WillBeLongRunningAsyncInTheMajorityOfImplementations();
                return;
            }

            private static IFooFace FactoryCreate()
            {
                return new LazyBar();
            }
        }
    }

If my interface must return Task what is the best way to have a no-operation implementation?

I have a program that spawns threads (~5-150) which perform a bunch of tasks. Originally, I used a `FixedThreadPool` because [this similar question][1] suggested they were better suited for longer lived tasks and with my very limited knowledge of multithreading, I considered the average life of the threads (several minutes) &quot;*long lived*&quot;.

However, I recently added the capability to spawn additional threads and doing so takes me above the thread limit I set. In this case, would it be better to guess and increase the number threads I can allow or to switch to a `CachedThreadPool` so I have no wasted threads?

Trying them both out preliminarily, there doesn&#39;t *seem* to be a difference so I&#39;m inclined to go with the `CachedThreadPool` just to avoid the waste. However, does the life span of the threads mean I should instead picked a `FixedThreadPool` and just deal with the unused threads? [This question][2] makes it seem like those extra threads aren&#39;t wasted but I would appreciate the clarification.

  [1]: https://stackoverflow.com/questions/949355/newcachedthreadpool-v-s-newfixedthreadpool
  [2]: https://stackoverflow.com/questions/13110313/java-fixedthreadpool-with-known-pool-size-but-unknown-workers


FixedThreadPool vs CachedThreadPool: the lesser of two evils

I am learning to use `ExectorService` to pool `threads` and send out tasks. I have a simple program below

```
import java.util.concurrent.ExecutorService;
import java.util.concurrent.Executors;
import java.util.concurrent.TimeUnit;


class Processor implements Runnable {

    private int id;

    public Processor(int id) {
        this.id = id;
    }

    public void run() {
        System.out.println(&quot;Starting: &quot; + id);

        try {
            Thread.sleep(5000);
        } catch (InterruptedException e) {
            System.out.println(&quot;sorry, being interupted, good bye!&quot;);
            System.out.println(&quot;Interrupted &quot; + Thread.currentThread().getName());
            e.printStackTrace();
        }

        System.out.println(&quot;Completed: &quot; + id);
    }
}


public class ExecutorExample {

    public static void main(String[] args) {
        Boolean isCompleted = false;

        ExecutorService executor = Executors.newFixedThreadPool(2);

        for (int i = 0; i &lt; 5; i++) {
            executor.execute(new Processor(i));
        }

        //executor does not accept any more tasks but the submitted tasks continue
        executor.shutdown();

        System.out.println(&quot;All tasks submitted.&quot;);

        try {
            //wait for the exectutor to terminate normally, which will return true
            //if timeout happens, returns false, but this does NOT interrupt the threads
            isCompleted = executor.awaitTermination(100, TimeUnit.SECONDS);
            //this will interrupt thread it manages. catch the interrupted exception in the threads
            //If not, threads will run forever and executor will never be able to shutdown.
            executor.shutdownNow();
        } catch (InterruptedException e) {
        }

        if (isCompleted) {
            System.out.println(&quot;All tasks completed.&quot;);
        } else {
            System.out.println(&quot;Timeout &quot; + Thread.currentThread().getName());
        }
    }
}
```

It does nothing fancy, but creates two `threads` and submits 5 tasks in total. After each `thread` completes its task, it takes the next one,
In the code above, I use `executor.submit`. I also changed to `executor.execute`. But I do not see any difference in the output. In what way are the `submit` and `execute` methods different?
This what the `API` says

&gt; Method submit extends base method Executor.execute(java.lang.Runnable) by creating and returning a Future that can be used to cancel execution and/or wait for completion. Methods invokeAny and invokeAll perform the most commonly useful forms of bulk execution, executing a collection of tasks and then waiting for at least one, or all, to complete. (Class ExecutorCompletionService can be used to write customized variants of these methods.)

But it&#39;s not clear to me as what it exactly means?

What is the difference between ExecutorService.submit and ExecutorService.execute in this code in Java?

I am trying to code a solution in which a single thread produces I/O-intensive tasks that can be performed in parallel. Each task have significant in-memory data. So I want to be able limit the number of tasks that are pending at a moment. 

If I create ThreadPoolExecutor like this:

        ThreadPoolExecutor executor = new ThreadPoolExecutor(numWorkerThreads, numWorkerThreads,
                                      0L, TimeUnit.MILLISECONDS,
                                      new LinkedBlockingQueue&lt;Runnable&gt;(maxQueue));



Then the `executor.submit(callable)` throws `RejectedExecutionException` when the queue fills up and all the threads are already busy. 

What can I do to make `executor.submit(callable)` block when the queue is full and all threads are busy?

**EDIT**:
I tried [this][1]:

    executor.setRejectedExecutionHandler(new ThreadPoolExecutor.CallerRunsPolicy());

And it somewhat achieves the effect that I want achieved but in an inelegant way (basically rejected threads are run in the calling thread, so this blocks the calling thread from submitting more).

**EDIT: (5 years after asking the question)**

To anyone reading this question and its answers, please don&#39;t take the accepted answer as one correct solution. Please read through all answers and comments.


  [1]: https://stackoverflow.com/a/5656329/32453

Java: ExecutorService that blocks on submission after a certain queue size

I am developing application server using spring boot app but now I want to know what is the default maximum (client request) thread pool size in spring and how can I customize that value?

Maximum (client request) thread pool size in spring

I am using an ExecutoreService in Java 1.6, started simply by 

	ExecutorService pool = Executors.newFixedThreadPool(THREADS). 

When my main thread is finished (along with all the tasks processed by the thread pool), this pool will prevent my program from shutting down until I explicitly call

	pool.shutdown();

Can I avoid having to call this by somehow turning the internal thread managing used by this pool into a deamon thread? Or am I missing something here.

Turning an ExecutorService to daemon in Java

Whats the difference between `ThreadPool` and `Pool` in `multiprocessing` module.  When I try my code out, this is the main difference I see:

    from multiprocessing import Pool
    import os, time
    
    print(&quot;hi outside of main()&quot;)
    
    def hello(x):
        print(&quot;inside hello()&quot;)
        print(&quot;Proccess id: &quot;, os.getpid())
        time.sleep(3)
        return x*x
    
    if __name__ == &quot;__main__&quot;:
        p = Pool(5)
        pool_output = p.map(hello, range(3))
    
        print(pool_output)

I see the following output:

    hi outside of main()
    hi outside of main()
    hi outside of main()
    hi outside of main()
    hi outside of main()
    hi outside of main()
    inside hello()
    Proccess id:  13268
    inside hello()
    Proccess id:  11104
    inside hello()
    Proccess id:  13064
    [0, 1, 4]

With &quot;ThreadPool&quot;:

    from multiprocessing.pool import ThreadPool
    import os, time
    
    print(&quot;hi outside of main()&quot;)
    
    def hello(x):
        print(&quot;inside hello()&quot;)
        print(&quot;Proccess id: &quot;, os.getpid())
        time.sleep(3)
        return x*x
    
    if __name__ == &quot;__main__&quot;:
        p = ThreadPool(5)
        pool_output = p.map(hello, range(3))
    
        print(pool_output)

I see the following output:

    hi outside of main()
    inside hello()
    inside hello()
    Proccess id:  15204
    Proccess id:  15204
    inside hello()
    Proccess id:  15204
    [0, 1, 4]

My questions are:

 - why is the “outside \__main\__()” run each time in the `Pool`?


 - `multiprocessing.pool.ThreadPool` doesn&#39;t spawn new processes?  It just creates new threads?  

 - If so whats the difference between using `multiprocessing.pool.ThreadPool` as opposed to just `threading` module?  

I don&#39;t see any official documentation for `ThreadPool` anywhere, can someone help me out where I can find it?

What&#39;s the difference between ThreadPool vs Pool in the multiprocessing module?

In one of my projects that&#39;s kinda an aggregator, I parse feeds, podcasts and so from the web.

If I use sequential approach, given that a large number of resources, it takes quite a time to process all of them (because of network issues and similar stuff);

    foreach(feed in feeds)
    {
       read_from_web(feed)
       parse(feed)
    }

So I want to implement concurrency and couldn&#39;t decide if I should basically use ThreadPools to process with worker threads or just rely on TPL to get it sorted.

ThreadPools for sure will handle the job for me with worker threads and I&#39;ll get what I expect (and in multi-core CPU environments, the other cores will be also utilized also).

![concurrency](https://upload.wikimedia.org/wikipedia/commons/thumb/0/0c/Thread_pool.svg/400px-Thread_pool.svg.png)

But I still want to consider TPL too as it&#39;s recommend method but I&#39;m a bit concerned about it. First of all I know that TPL uses ThreadPools but adds additional layer of decision making. I&#39;m mostly concerned of the condition that where a single-core environment is present. If I&#39;m not wrong TPL starts with a number worker-threads equal to number of available CPU-cores at the very beginning. I do fear of TPL producing similar results to sequential approach for my IO-bound case.

So for IO-bound operations (in my case reading resources from web), is it best to use ThreadPools and control the things, or better just rely on TPL? Can TPL also be used in IO-bound scenarios?

**Update**: My main concern is that -- **on a single-core CPU** environment will TPL just behave like sequential approach or will it still offer concurrency? I&#39;m already reading [Parallel Programming with Microsoft .NET](http://msdn.microsoft.com/en-us/library/ff963549.aspx) and so the [book](http://oreilly.com/catalog/0790145310262) but couldn&#39;t find an exact answer for this.

Note: this is a re-phrasing of my previous question [ https://stackoverflow.com/questions/5203871/is-it-possible-to-use-thread-concurrency-and-parallelism-together ] which was quite phrased wrong.

Should i use ThreadPools or Task Parallel Library for IO-bound operations

What is difference between the below

    ThreadPool.QueueUserWorkItem

vs

    Task.Factory.StartNew

If the above code is called 500 times for some long running task, does it mean all the thread pool threads will be taken up?

Or will TPL (2nd option) be smart enough to just take up threads less or equal to number of processors?

ThreadPool.QueueUserWorkItem vs Task.Factory.StartNew

How large is Scala&#39;s thread pool for futures?

My Scala application makes many millions of `future {}`s and I wonder if there is anything I can do to optimize them by configuring a thread pool.

Thank you.

How to configure a fine tuned thread pool for futures?

I am writing a server, and I send each action of into a separate thread when the request is received. I do this because almost every request makes a database query. I am using a threadpool library to cut down on construction/destruction of threads.

My question is: what is a good cutoff point for I/O threads like these? I know it would just be a rough estimate, but are we talking hundreds? Thousands?

How would I go about figuring out what this cutoff would be?

---
### EDIT:

Thank you all for your responses, it seems like I am just going to have to test it to find out my thread count ceiling. The question is though: how do I know I&#39;ve hit that ceiling? What exactly should I measure?


How many threads is too many?

As some may have seen in .NET 4.0, they&#39;ve added a new namespace `System.Threading.Tasks` which basically is what is means, a task. I&#39;ve only been using it for a few days, from using ThreadPool. 

Which one is more efficient and less resource consuming? (Or just better overall?)


C# - ThreadPool vs Tasks

How do I create a thread pool using boost in C++, and how do I assign tasks to the threadpool? 

How to create a thread pool using boost in C++?

Looking for some sample code (C#) for a simple thread pool implementation.

I found one on codeproject, but the codebase was just huge and I don&#39;t need all that functionality.

This is more for educational purposes anyways.



All Threadpool Solutions on Gang of Coders

Total of 28 Threadpool Solutions