In the [Python documentation][1]
it says:

&gt; A thread can be flagged as a &quot;daemon thread&quot;. The significance of this
&gt; flag is that the entire Python program exits when only daemon threads
&gt; are left. The initial value is inherited from the creating thread.

Does anyone have a clearer explanation of what that means or a practical example showing where you would set threads as `daemonic`?


Clarify it for me: so the only situation you wouldn&#39;t set threads as `daemonic`, is when you want them to continue running after the main thread exits?


  [1]: https://docs.python.org/2/library/threading.html#thread-objects

Daemon Threads Explanation

I was studying the python threading and came across [`join()`](http://docs.python.org/2/library/threading.html#threading.Thread.join).

The author told that if thread is in daemon mode then i need to use `join()` so that thread can finish itself before main thread terminates.

but I have also seen him using `t.join()` even though `t` was not `daemon`

example code is this


    import threading
    import time
    import logging
    
    logging.basicConfig(level=logging.DEBUG,
                        format=&#39;(%(threadName)-10s) %(message)s&#39;,
                        )
    
    def daemon():
        logging.debug(&#39;Starting&#39;)
        time.sleep(2)
        logging.debug(&#39;Exiting&#39;)
    
    d = threading.Thread(name=&#39;daemon&#39;, target=daemon)
    d.setDaemon(True)
    
    def non_daemon():
        logging.debug(&#39;Starting&#39;)
        logging.debug(&#39;Exiting&#39;)
    
    t = threading.Thread(name=&#39;non-daemon&#39;, target=non_daemon)
    
    d.start()
    t.start()
    
    d.join()
    t.join()


i don&#39;t know what is use of `t.join()` as it is not daemon and i can see no change even if i remove it




What is the use of join() in Python threading?

I have a multi-threading Python program, and a utility function, `writeLog(message)`, that writes out a timestamp followed by the message. Unfortunately, the resultant log file gives no indication of which thread is generating which message. 

I would like `writeLog()` to be able to add something to the message to identify which thread is calling it. Obviously I could just make the threads pass this information in, but that would be a lot more work. Is there some thread equivalent of `os.getpid()` that I could use?

How to obtain a Thread id in Python?

I need a queue which multiple threads can put stuff into, and multiple threads may read from.

Python has at least two queue classes, `Queue.Queue` and `collections.deque`, with the former seemingly using the latter internally. Both claim to be thread-safe in the documentation.

However, the Queue docs also state:

&gt; collections.deque is an alternative
&gt; implementation of unbounded queues
&gt; with fast atomic append() and
&gt; popleft() operations **that do not
&gt; require locking.**

Which I guess I don&#39;t quite unterstand: Does this mean deque isn&#39;t fully thread-safe after all?

If it is, I may not fully understand the difference between the two classes. I can see that Queue adds blocking functionality. On the other hand, it loses some deque features like support for the in-operator.

Accessing the internal deque object directly, is

&gt; x in Queue().deque

thread-safe?

Also, **why does Queue employ a mutex for it&#39;s operations when deque is thread-safe already?**

Queue.Queue vs. collections.deque

I notice that it is often suggested to use queues with multiple threads, instead of lists and `.pop()`. Is this because lists are not thread-safe, or for some other reason?

Are lists thread-safe?

I want to fire off a function every 0.5 seconds and be able to start and stop and reset the timer. I&#39;m not too knowledgeable of how Python threads work and am having difficulties with the python timer.

However, I keep getting `RuntimeError: threads can only be started once` when I execute `threading.timer.start()` twice. Is there a work around for this? I tried applying `threading.timer.cancel()` before each start.

Pseudo code:

    t=threading.timer(0.5,function)
    while True:
        t.cancel()
        t.start()



threading.Timer - repeat function every &#39;n&#39; seconds

I have a class MyThread. In that, I have a method sample. I am trying to run it from within the same object context. Please have a look at the code:

    class myThread (threading.Thread):
	    def __init__(self, threadID, name, counter, redisOpsObj):
			threading.Thread.__init__(self)
			self.threadID = threadID
			self.name = name
			self.counter = counter
			self.redisOpsObj = redisOpsObj
			
		def stop(self):
			self.kill_received = True
				
		def sample(self):
			print &quot;Hello&quot;
					
		def run(self):
			time.sleep(0.1)
			print &quot;\n Starting &quot; + self.name
			self.sample()



Looks very simple ain&#39;t it. But when I run it I get this error

`AttributeError: &#39;myThread&#39; object has no attribute &#39;sample&#39;` Now I have that method, right there. So what&#39;s wrong? Please help

Edit: This is the stack trace

    Starting Thread-0

    Starting Thread-1
    Exception in thread Thread-0:
    Traceback (most recent call last):
    File &quot;/usr/lib/python2.6/threading.py&quot;, line 525, in __bootstrap_inner
    self.run()
    File &quot;./redisQueueProcessor.py&quot;, line 51, in run
    self.sample()
    AttributeError: &#39;myThread&#39; object has no attribute &#39;sample&#39;

    Exception in thread Thread-1:
    Traceback (most recent call last):
    File &quot;/usr/lib/python2.6/threading.py&quot;, line 525, in __bootstrap_inner
    self.run()
    File &quot;./redisQueueProcessor.py&quot;, line 51, in run
    self.sample()
    AttributeError: &#39;myThread&#39; object has no attribute &#39;sample&#39;

I am calling it like this

    arThreads = []
    maxThreads = 2;

    for i in range( maxThreads ):
        redisOpsObj = redisOps()
        arThreads.append( myThread(i, &quot;Thread-&quot;+str(i), 10, redisOpsObj) )

Sorry, I can&#39;t post the redisOps class code. But I can assure you that it works just fine

Why am I getting AttributeError: Object has no attribute?

I was just very confused by some code that I wrote. I was surprised to discover that:


    with concurrent.futures.ThreadPoolExecutor(max_workers=4) as executor:
        results = list(executor.map(f, iterable))

and 


    with concurrent.futures.ThreadPoolExecutor(max_workers=4) as executor:
        results = list(map(lambda x: executor.submit(f, x), iterable))

produce different results. The first one produces a list of whatever type `f` returns, the second produces a list of `concurrent.futures.Future` objects that then need to be evaluated with their `result()` method in order to get the value that `f` returned. 

My main concern is that this means that `executor.map` can&#39;t take advantage of `concurrent.futures.as_completed`, which seems like an extremely convenient way to evaluate the results of some long-running calls to a database that I&#39;m making as they become available. 

I&#39;m not at all clear on how `concurrent.futures.ThreadPoolExecutor` objects work -- naively, I would prefer the (somewhat more verbose):

    with concurrent.futures.ThreadPoolExecutor(max_workers=4) as executor:
        result_futures = list(map(lambda x: executor.submit(f, x), iterable))
        results = [f.result() for f in futures.as_completed(result_futures)]

over the more concise `executor.map` in order to take advantage of a possible gain in performance. Am I wrong to do so?

How does ThreadPoolExecutor().map differ from ThreadPoolExecutor().submit?

If I have a thread in an infinite loop, is there a way to terminate it when the main program ends (for example, when I press &lt;kbd&gt;**Ctrl**&lt;/kbd&gt;+&lt;kbd&gt;**C**&lt;/kbd&gt;)?

How to terminate a thread when main program ends?

Say I have a very large list and I&#39;m performing an operation like so:

    for item in items:
        try:
            api.my_operation(item)
        except:
            print &#39;error with item&#39;

My issue is two fold:

 - There are a lot of items
 - api.my_operation takes forever to return

I&#39;d like to use multi-threading to spin up a bunch of api.my_operations at once so I can process maybe 5 or 10 or even 100 items at once. 

If my_operation() returns an exception (because maybe I already processed that item) - that&#39;s OK. It won&#39;t break anything. The loop can continue to the next item.

**Note**: this is for Python 2.7.3

How to Multi-thread an Operation Within a Loop in Python

I&#39;m calling a function in Python which I know may stall and force me to restart the script. 

How do I call the function or what do I wrap it in so that if it takes longer than 5 seconds the script cancels it and does something else?


Timeout on a function call

What&#39;s the difference between the `threading` and `thread` modules in Python?

Thread vs. Threading

I&#39;m a little confused about what setting a thread to be a daemon means. 

The documentation says this:

&gt; A thread can be flagged as a “daemon
&gt; thread”. The significance of this flag
&gt; is that the entire Python program
&gt; exits when only daemon threads are
&gt; left. The initial value is inherited
&gt; from the creating thread. The flag can
&gt; be set through the daemon property.

I&#39;m not sure what makes this different from a normal thread. 

Is this saying that this program won&#39;t ever finish?

    def threadfunc():
        while True:
            time.sleep(1)
    
    threading.Thread(target=threadfunc).start()

Even though the main thread finishes it&#39;s execution. While will finish immediately?


    def threadfunc():
        while True:
            time.sleep(1)
    
    th = threading.Thread(target=threadfunc)
    th.daemon = True
    th.start()


I ask because I have a situation where in my main thread I&#39;m calling sys.exit(), and the process just hangs and my other threads are running as I can see the log. 

Does this have anything to do with sys.exit() being called with threads alive?





Meaning of daemon property on Python Threads

I spent a whole day looking for the simplest possible multithreaded URL fetcher in Python, but most scripts I found are using queues or multiprocessing or complex libraries.

Finally I wrote one myself, which I am reporting as an answer. Please feel free to suggest any improvement.

I guess other people might have been looking for something similar.

A very simple multithreading parallel URL fetching (without queue)

How do I get a thread to return a tuple or any value of my choice back to the parent in Python?

Return value from thread

I&#39;d like to create a program that runs multiple light threads, but limits itself to a constant, predefined number of concurrent running tasks, like this (but with no risk of race condition):

    import threading
    
    def f(arg):
        global running
        running += 1
        print(&quot;Spawned a thread. running=%s, arg=%s&quot; % (running, arg))
        for i in range(100000):
            pass
        running -= 1
        print(&quot;Done&quot;)
    
    running = 0
    while True:
        if running &lt; 8:
            arg = get_task()
            threading.Thread(target=f, args=[arg]).start()

What&#39;s the safest/fastest way to implement this?


The right way to limit maximum number of threads running at once?

I want to pass named arguments to the target function, while creating a Thread object. 

Following is the code that I have written:

    import threading
    
    def f(x=None, y=None):
        print x,y
    
    t = threading.Thread(target=f, args=(x=1,y=2,))
    t.start()

I get a syntax error for &quot;x=1&quot;, in Line 6.
I want to know how I can pass keyword arguments to the target function.

Pass keyword arguments to target function in Python threading.Thread

I am trying to understand threading in Python. I&#39;ve looked at the documentation and examples, but quite frankly, many examples are overly sophisticated and I&#39;m having trouble understanding them.

How do you clearly show tasks being divided for multi-threading?



All Python Multithreading Solutions on Gang of Coders

Total of 18 Python Multithreading Solutions