You can pass the **length** to the argument **total** to make it work.

Example:

    from tqdm import tqdm
        
    length = 1000000
    generator = (3 * n for n in range(length))  # just doing something random
    for n in tqdm(generator, total=length):
        pass

I have a&#160;Linq query that fetches an entity and&#160;some of its navigation properties.

    context.MyEntity
        .AsNoTracking()
        .Include(i=&gt; i.Nav1)
        .Include(i=&gt; i.Nav2)
        .Where(x=&gt; x.Prop1==1)
        .FirstOrDefault();

my question is:

Is the above query enough to not track `MyEntity` nor the navigation properties `NAv1`&amp; `Nav2` or must I add `AsNoTracking` for each navigation&#160;property? 

like this: &#160;

    context.MyEntity
        .AsNoTracking()
        .Include(i=&gt; i.Nav1)
        .AsNoTracking()
        .Include(i=&gt; i.Nav2)
        .AsNoTracking()
        .Where(x=&gt; x.Prop1==1)
        .FirstOrDefault();



AsNoTracking() and Include

This might be a simple question but because of clear understanding between print() and debug() print in swift I am unable to understand where to use each one. 

print() vs debugPrint() in swift

I&#39;m looping over a large file that I know the length of, but am processing lazily since it&#39;s too large to fit in memory. I&#39;d like to be able to use tqdm to keep track of my progress through the file, but since it can&#39;t get the total number of examples out of the generator I&#39;m using, the only thing it shows is the estimated iterations/second. Is there any way to tell tqdm how many elements it&#39;s going to be looping over total so I can get some of the other statistics?

tqdm show progress for a generator I know the length of

<p>I'm looping over a large file that I know the length of, but am processing lazily since it's too large to fit in memory. I'd like to be able to use tqdm to keep track of my progress through the file, but since it can't get the total number of examples out of the generator I'm using, the only thing it shows is the estimated iterations/second. Is there any way to tell tqdm how many elements it's going to be looping over total so I can get some of the other statistics?</p>


I was trying to set default python version to `python3` in `Ubuntu 16.04`. By default it is `python2` (2.7). I followed below steps :  

    update-alternatives --remove python /usr/bin/python2
    update-alternatives --install /usr/bin/python python /usr/bin/python3

but I&#39;m getting the following error for the second statement,    

    rejeesh@rejeesh-Vostro-1015:~$ update-alternatives --install /usr/bin/python python /usr/bin/python3
    update-alternatives: --install needs &lt;link&gt; &lt;name&gt; &lt;path&gt; &lt;priority&gt;
    
    Use &#39;update-alternatives --help&#39; for program usage information.   

I&#39;m new to Ubuntu and Idon&#39;t know what I&#39;m doing wrong.



Unable to set default python version to python3 in ubuntu

I&#39;m using the `MinMaxScaler` model in sklearn to normalize the features of a model.

    training_set = np.random.rand(4,4)*10
    training_set

           [[ 6.01144787,  0.59753007,  2.0014852 ,  3.45433657],
           [ 6.03041646,  5.15589559,  6.64992437,  2.63440202],
           [ 2.27733136,  9.29927394,  0.03718093,  7.7679183 ],
           [ 9.86934288,  7.59003904,  6.02363739,  2.78294206]]


    scaler = MinMaxScaler()
    scaler.fit(training_set)    
    scaler.transform(training_set)


       [[ 0.49184811,  0.        ,  0.29704831,  0.15972182],
       [ 0.4943466 ,  0.52384506,  1.        ,  0.        ],
       [ 0.        ,  1.        ,  0.        ,  1.        ],
       [ 1.        ,  0.80357559,  0.9052909 ,  0.02893534]]



Now I want to use the same scaler to normalize the test set:

       [[ 8.31263467,  7.99782295,  0.02031658,  9.43249727],
       [ 1.03761228,  9.53173021,  5.99539478,  4.81456067],
       [ 0.19715961,  5.97702519,  0.53347403,  5.58747666],
       [ 9.67505429,  2.76225253,  7.39944931,  8.46746594]]

But I don&#39;t want so use the `scaler.fit()` with the training data all the time. Is there a way to save the scaler and load it later from a different file?    

Save MinMaxScaler model in sklearn

I am getting the error &quot;could not find or load the Qt platform plugin windows&quot; while using matplotlib in PyCharm.

How can I solve this?

![enter image description here][1]

  [1]: https://i.stack.imgur.com/XXw0M.png

How to fix &quot;could not find or load the Qt platform plugin windows&quot; while using Matplotlib in PyCharm

Based on [this answer][1] I want to build an async websoket client in a class which would be imported from another file:

    #!/usr/bin/env python3
    
    import sys, json
    import asyncio
    from websockets import connect
    
    class EchoWebsocket:
        def __await__(self):
            # see: https://stackoverflow.com/a/33420721/1113207
            return self._async_init().__await__()
    
        async def _async_init(self):
            self._conn = connect(&#39;wss://ws.binaryws.com/websockets/v3&#39;)
            self.websocket = await self._conn.__aenter__()
            return self
    
        async def close(self):
            await self._conn.__aexit__(*sys.exc_info())
    
        async def send(self, message):
            await self.websocket.send(message)
    
        async def receive(self):
            return await self.websocket.recv()
    
    class mtest:
        async def start(self):
            try:
                self.wws = await EchoWebsocket()
            finally:
                await self.wws.close()
    
        async def get_ticks(self):
            await self.wws.send(json.dumps({&#39;ticks_history&#39;: &#39;R_50&#39;, &#39;end&#39;: &#39;latest&#39;, &#39;count&#39;: 1}))
            return await self.wws.receive()
    
    if __name__ == &#39;__main__&#39;:
        a = mtest()
        loop = asyncio.get_event_loop()
        loop.run_until_complete(a.start())

And I import it in `main.py`, where I have the following:


    from testws import *
    
    a = mtest()
    print (a.get_ticks())
    print (&quot;this will be printed after the ticks&quot;)

But it retrieves me the following error:

    root@ubupc1:/home/dinocob# python3 test.py
    &lt;coroutine object hello.get_ticks at 0x7f13190a9200&gt;
    test.py:42: RuntimeWarning: coroutine &#39;mtest.get_ticks&#39; was never awaited
      print (a.get_ticks())
    this will be printed after the ticks

What&#39;s is going on here? Why I&#39;m not able to access mtest.get_ticks if it has the `async` word at the begining of `def`?

  [1]: https://stackoverflow.com/a/37370823/1709738


How to call a async function contained in a class?

I would like to use fixtures as arguments of `pytest.mark.parametrize` or something that would have the same results.

For example:

    import pytest
    import my_package
    
    @pytest.fixture
    def dir1_fixture():
        return &#39;/dir1&#39;
    
    @pytest.fixture
    def dir2_fixture():
        return &#39;/dir2&#39;

    @pytest.parametrize(&#39;dirname, expected&#39;, [(dir1_fixture, &#39;expected1&#39;), (dir2_fixture, &#39;expected2&#39;)]
    def test_directory_command(dirname, expected):
        result = my_package.directory_command(dirname)
        assert result == expected

The problem with fixture params is that every param of the fixture will get run every time it&#39;s used, but I don&#39;t want that. I want to be able to choose which fixtures will get used depending on the test.

pytest using fixtures as arguments in parametrize

I see lot of examples of generator functions, but I want to know how to write generators for classes. Lets say, I wanted to write Fibonacci series as a class. 
 

    class Fib:
        def __init__(self):
            self.a, self.b = 0, 1
            
        def __next__(self):
            yield self.a
            self.a, self.b = self.b, self.a+self.b
    
    f = Fib()

    for i in range(3):
        print(next(f))

Output:

&lt;!-- language: lang-none --&gt;
        
    &lt;generator object __next__ at 0x000000000A3E4F68&gt;
    &lt;generator object __next__ at 0x000000000A3E4F68&gt;
    &lt;generator object __next__ at 0x000000000A3E4F68&gt;


Why is the value `self.a` not getting printed?  Also, how do I write `unittest` for generators?   
 



How to write a generator class?

I&#39;m trying to write a `:rtype:` type hint for a generator function. What is the type it returns? 

For example, say I have this functions which yields strings:

    def read_text_file(fn):
        &quot;&quot;&quot;
        Yields the lines of the text file one by one.
        :param fn: Path of text file to read.
        :type fn: str
        :rtype: ???????????????? &lt;======================= what goes here?
        &quot;&quot;&quot;
        with open(fn, &#39;rt&#39;) as text_file:
            for line in text_file:
                yield line

The return type isn&#39;t just a string, it&#39;s some kind of iterable of strings? So I can&#39;t just write `:rtype: str`. What&#39;s the right hint?

What is the return type hint of a generator function?

From the perspective of someone who has written asyncio code but is looking to better understand the inner workings, what is `yield from`, `await` and how are those useful for allowing asynchronous code?

There is one [highly upvoted](https://stackoverflow.com/questions/9708902/in-practice-what-are-the-main-uses-for-the-new-yield-from-syntax-in-python-3) question asking about the uses of the `yield from` syntax and [one explaining async and await](https://stackoverflow.com/questions/31291129/pep-0492-python-3-5-async-keyword), but both go in depth about different topics and are not really a concise explanation of the underlying code and how it fits in with asyncio.


What does the &quot;yield from&quot; syntax do in asyncio and how is it different from &quot;await&quot;

I am using Keras with a Tensorflow backend in Python. To be more precise tensorflow **1.2.1** and its build-in contrib.keras lib.

I want to use the `fit_generator` method of a Sequential model object, but I am confused with what I should pass as the method-parameters.

From reading the doc [here][1] I got the following information:  

 - **generator** : a python training data batch generator; endlessly looping over its training data
 - **validation_data**:  -*in my case* - a python validation data batch generator; the doc doesn&#39;t mention endless looping over its validation data
 - **steps_per_epoch** : `number of training batches = uniqueTrainingData / batchSize`
 - **validation steps** : `???` ; = uniqueValidationData / batch size ???
 - **use_multiprocessing** : boolean; don&#39;t pass non picklable arguments ???
 - **workers** : max number of used processes 

As indicated above with ??? I don&#39;t really know what validation_steps means.
I know the definition of the above linked doc (`Number of steps to yield from validation generator at the end of every epoch`) but that only confuses my in the given context. From the doc i know that the validation_data generator has to yield data, label tuples in the form `(inputs, targets)`. In contrast to that the above statement indicates that there have to be multiple &quot;steps to yield from validation generator at the end of every epoch&quot; which in this context would mean, that multiple validation batches would be yielded after each training epoch.  

Questions about `validation_steps`:

 - Does it really work that way? If so: Why? I thought that after each epoch one validation batch, which ideally wasn&#39;t used before, is used for validation to ensure that the training gets validated without risking to &quot;train&quot; the model to perform better on already used validation sets.  
 - In context of the previous question: Why is the recommended amount of validation steps `uniqueValidationData / batches` and not `uniqueValidationData / epochs`? Isn&#39;t it better to have e.g. 100 validation batches for 100 epochs instead of x validation batches where x could be less or more than the specified number of epochs? Alternatively: If you have much less validation batches than number of epoches, is the model trained without validation for the rest of the epochs or do validation sets get reused / reshuffled+reused? 
 - Is it important that the training and validation batches have the same batch size (shared divisor of the dividends trainingDataCount and validationDataCount)?

Additional question about `use_multiprocessing`:

 - Are numpy arrays picklable or do I have to convert them to multidimensional lists?

  [1]: https://keras.io/models/sequential/#fit_generator

Meaning of validation_steps in Keras Sequential fit_generator parameter list

I was running a piece of code that unexpectedly gave a logic error at one part of the program. When investigating the section, I created a test file to test the set of statements being run and found out an unusual bug that seems very odd.

I tested this simple code:

    array = [1, 2, 2, 4, 5] # Original array
    f = (x for x in array if array.count(x) == 2) # Filters original
    array = [5, 6, 1, 2, 9] # Updates original to something else
    
    print(list(f)) # Outputs filtered

And the output was:

    &gt;&gt;&gt; []

Yes, nothing. I was expecting the filter comprehension to get items in the array with a count of 2 and output this, but I didn&#39;t get that:

    # Expected output
    &gt;&gt;&gt; [2, 2]

When I commented out the third line to test it once again:

    array = [1, 2, 2, 4, 5] # Original array
    f = (x for x in array if array.count(x) == 2) # Filters original
    ### array = [5, 6, 1, 2, 9] # Ignore line
    
    print(list(f)) # Outputs filtered

The output was correct (you can test it for yourself):

    &gt;&gt;&gt; [2, 2]

At one point I outputted the type of the variable `f`:

    array = [1, 2, 2, 4, 5] # Original array
    f = (x for x in array if array.count(x) == 2) # Filters original
    array = [5, 6, 1, 2, 9] # Updates original
    
    print(type(f))
    print(list(f)) # Outputs filtered

And I got:

    &gt;&gt;&gt; &lt;class &#39;generator&#39;&gt;
    &gt;&gt;&gt; []

Why is updating a list in Python changing the output of another generator variable? This seems very odd to me.

Unexpected behaviour with a conditional generator expression

I am using `tqdm` to print progress in a script I&#39;m running in a Jupyter notebook. I am printing all messages to the console via `tqdm.write()`. However, this still gives me a skewed output like so:

[![enter image description here][1]][1]

That is, each time a new line has to be printed, a new progress bar is printed on the next line. This does not happen when I run the script via terminal. How can I solve this?

  [1]: https://i.stack.imgur.com/xCfqE.jpg

tqdm in Jupyter Notebook prints new progress bars repeatedly

I am making a code that simulates a pawn going around a monopoly board a million times.
I would like to have a tqdm progress bar that is updated every time a turn around the board is achieved.

**Below is my current code.** I am using a while loop which stops when the number of turns around the board surpasses the desired number.

    import os
    from openpyxl import Workbook
    from monopolyfct import *


    def main(runs, fileOutput):

        ### EXCEL SETUP ###
        theWorkbook = Workbook()                              # Creates the workbook interface.
        defaultSheet = theWorkbook.active                     # Creates the used worksheet.
        currentData = [&quot;Current Table Turn&quot;, &quot;Current Tile&quot;]  # Makes EXCEL column titles.
        defaultSheet.append(currentData)                      # Appends column titles.

        ### CONTENT SETUP ###
        currentData = [1, 0]             # Sets starting position.
        defaultSheet.append(currentData) # Appends starting position.

        while currentData[0] &lt;= runs:

            ### ROLLING THE DICES PROCESS ###
            dices = twinDiceRoll()
            currentData[1] += dices[2]  # Updating the current tile

            ### SURPASSING THE NUMBER OF TILES ONBOARD ###
            if currentData[1] &gt; 37:   # If more than a table turn is achieved,
                currentData[0] += 1   # One more turn is registered
                currentData[1] -= 38  # Update the tile to one coresponding to a board tile.
            else:
                pass

            ### APPENDING AQUIRED DATA ###
            defaultSheet.append(currentData)

            ### MANAGIING SPECIAL TILES ###
            if currentData[1] == 2 or 15 or 31:   # Community chess
                pass                              #TODO: Make a mechanic simulating the community chest card draw and it&#39;s related action.
            elif currentData[1] == 5 or 20 or 34: # Chance
                pass                              #TODO: Make a mechanic simulating the chance card draw and it&#39;s related action.
            elif currentData[1] == 28:            # Go to Jail
                pass                              #TODO: Make a mechanic simulating the entire jail process

            ### TWIN DICE ROLL EXCEPTION ###
            if dices[3] is True:  # If the dices roll a double,
                pass              #TODO: Make a mechanic considering that three doubles sends one to Jail.


        ### STORING THE ACCUMULATED DATA ###
        theWorkbook.save(fileOutput)  # Compiles the data in a .xlxs file.


    if __name__ == &quot;__main__&quot;:
        terminalWidth = os.get_terminal_size().columns                                               # Gets current terminal width.
        space(3)
        print(&quot;Python Monopoly Statistics Renderer&quot;.upper().center(terminalWidth))                   # Prints the title.
        print(&quot;(PMSR)&quot;.center(terminalWidth))                                                        # Prints the acronym.
        space(2)
        runs = int(request(&quot;For how many table turns do you want the simulation to run?&quot;))           # Prompts for the desired run ammount
        #runs = 1000
        fileOutput = request(&quot;What should be the name of the file in which statistics are stored?&quot;)  # Prompts for the desired store filename
        #fileOutput = &quot;test&quot;
        fileOutput += &quot;.xlsx&quot;                                                                        # Adds file extension to filename
        main(runs, fileOutput)


Using tqdm progress bar in a while loop

Is it possible to use TQDM progress bar when importing and indexing large datasets using Pandas?

Here is an example of of some 5-minute data I am importing, indexing, and using to_datetime.  It takes a while and it would be nice to see a progress bar.

    #Import csv files into a Pandas dataframes and convert to Pandas datetime and set to index
    
    eurusd_ask = pd.read_csv(&#39;EURUSD_Candlestick_5_m_ASK_01.01.2012-05.08.2017.csv&#39;)
    eurusd_ask.index = pd.to_datetime(eurusd_ask.pop(&#39;Gmt time&#39;))



Use TQDM Progress Bar with Pandas

I can&#39;t see the tqdm progress bar when I use this code to iterate my opened file:

            with open(file_path, &#39;r&#39;) as f:
            for i, line in enumerate(tqdm(f)):
                if i &gt;= start and i &lt;= end:
                    print(&quot;line #: %s&quot; % i)
                    for i in tqdm(range(0, line_size, batch_size)):
                        # pause if find a file naed pause at the currend dir
                        re_batch = {}
                        for j in range(batch_size):
                            re_batch[j] = re.search(line, last_span)
what&#39;s the right way to use tqdm here?

Python enumerate() tqdm progress-bar when reading a file?

I have a multithreaded function that I would like a status bar for using `tqdm`. Is there an easy way to show a status bar with `ThreadPoolExecutor`? It is the parallelization part that is confusing me.

    import concurrent.futures

    def f(x):
        return f**2
    
    my_iter = range(1000000)
    
    def run(f,my_iter):
        with concurrent.futures.ThreadPoolExecutor() as executor:
            function = list(executor.map(f, my_iter))
        return results
    
    run(f, my_iter) # wrap tqdr around this function?

Content Type	Original Author	Original Content on Stackoverflow
Question	George	View Question on Stackoverflow
Solution 1 - Python	O. Edholm	View Answer on Stackoverflow

tqdm show progress for a generator I know the length of

Python Problem Overview

Python Solutions

Solution 1 - Python

print() vs debugPrint() in swift

AsNoTracking() and Include

Attributions