So I recently discovered this awesome tool, and it says 

&gt; Docker is an open-source project to easily create lightweight,
&gt; portable, self-sufficient containers from any application. The same
&gt; container that a developer builds and tests on a laptop can run at
&gt; scale, in production, on VMs, bare metal, OpenStack clusters, public
&gt; clouds and more.

Let&#39;s say I have a docker image which runs Nginx and a website connects to external database. How do I scale the container in production?

How to scale Docker containers in production

I was trying to setup an SSH connection with Github following this tutorial:

* [Testing your SSH connection](https://docs.github.com/en/github/authenticating-to-github/connecting-to-github-with-ssh/testing-your-ssh-connection)

I came across the following command: 

    $ ssh -T git@github.com 
    # Attempts to ssh to github

Curious, I looked at the ssh manual. It said the following:

     -T      Disable pseudo-tty allocation.

What is tty allocation? What does tty stand for? Why are we disabling it?&lt;br&gt;I earnestly tried to look it up but I was unable to find even a definition.

What is Pseudo TTY-Allocation? (SSH and Github)

I have a pandas data frame with multiple columns and I would like to construct a dict from two columns: one as the dict&#39;s keys and the other as the dict&#39;s values. How can I do that?

Dataframe:

               area  count
    co tp
    DE Lake      10      7
    Forest       20      5
    FR Lake      30      2
    Forest       40      3

I need to define area as key, count as value in dict. Thank you in advance.

python pandas dataframe columns convert to dict key and value

I have a pandas data frame with multiple columns and I would like to construct a dict from two columns: one as the dict's keys and the other as the dict's values. How can I do that?
Dataframe:
<pre><code class="hljs language-markdown"> area count
co tp
DE Lake 10 7
Forest 20 5
FR Lake 30 2
Forest 40 3
</code></pre>
I need to define area as key, count as value in dict. Thank you in advance.

What&#39;s the proper way to tell a looping thread to stop looping? 

I have a fairly simple program that pings a specified host in a separate `threading.Thread` class. In this class it sleeps 60 seconds, the runs again until the application quits.

I&#39;d like to implement a &#39;Stop&#39; button in my `wx.Frame` to ask the looping thread to stop. It doesn&#39;t need to end the thread right away, it can just stop looping once it wakes up. 

Here is my `threading` class (note: I haven&#39;t implemented looping yet, but it would likely fall under the run method in PingAssets)

    class PingAssets(threading.Thread):
        def __init__(self, threadNum, asset, window):
            threading.Thread.__init__(self)
            self.threadNum = threadNum
            self.window = window
            self.asset = asset
    
        def run(self):
            config = controller.getConfig()
            fmt = config[&#39;timefmt&#39;]
            start_time = datetime.now().strftime(fmt)
            try:
                if onlinecheck.check_status(self.asset):
                    status = &quot;online&quot;
                else:
                    status = &quot;offline&quot;
            except socket.gaierror:
                status = &quot;an invalid asset tag.&quot;
            msg =(&quot;{}: {} is {}.   \n&quot;.format(start_time, self.asset, status))
            wx.CallAfter(self.window.Logger, msg)

And in my wxPyhton Frame I have this function called from a Start button:

    def CheckAsset(self, asset):
            self.count += 1
            thread = PingAssets(self.count, asset, self)
            self.threads.append(thread)
            thread.start()

How to stop a looping thread in Python?

How do I get the index column name in python pandas?  Here&#39;s an example dataframe:

                 Column 1
    Index Title          
    Apples              1
    Oranges             2
    Puppies             3
    Ducks               4  

 
What I&#39;m trying to do is get/set the dataframe index title.  Here is what i tried:

    import pandas as pd
    data = {&#39;Column 1&#39;     : [1., 2., 3., 4.],
            &#39;Index Title&#39;  : [&quot;Apples&quot;, &quot;Oranges&quot;, &quot;Puppies&quot;, &quot;Ducks&quot;]}
    df = pd.DataFrame(data)
    df.index = df[&quot;Index Title&quot;]
    del df[&quot;Index Title&quot;]
    print df

Anyone know how to do this? 

Pandas index column title or name

I am creating a `setup.py` file for a project which depends on private GitHub repositories.  The relevant parts of the file look like this:

    from setuptools import setup
    setup(name=&#39;my_project&#39;,
        ...,
        install_requires=[
            &#39;public_package&#39;,
            &#39;other_public_package&#39;,
            &#39;private_repo_1&#39;,
            &#39;private_repo_2&#39;,
        ],
        dependency_links=[
            &#39;https://github.com/my_account/private_repo_1/master/tarball/&#39;,
            &#39;https://github.com/my_account/private_repo_2/master/tarball/&#39;,
        ],
        ...,
    )

I am using `setuptools` instead of `distutils` because the latter does not support the `install_requires` and `dependency_links` arguments per [this](https://stackoverflow.com/questions/9810603/adding-install-requires-to-setup-py-when-making-a-python-package) answer.

The above setup file fails to access the private repos with a 404 error - which is to be expected since GitHub returns a 404 to unauthorized requests for a private repository.  However, I can&#39;t figure out how to make `setuptools` authenticate.

Here are some things I&#39;ve tried:

1. Use `git+ssh://` instead of `https://` in `dependency_links` as I would if installing the repo with `pip`.  This fails because setuptools doesn&#39;t recognize this protocol (&quot;unknown url type: git+ssh&quot;), though the [distribute documentation](http://pythonhosted.org/distribute/setuptools.html#dependencies-that-aren-t-in-pypi) says it should.  Ditto `git+https` and `git+http`.

2. `https://&lt;username&gt;:&lt;password&gt;@github.com/...` - still get a 404.  (This method doesn&#39;t work with `curl` or `wget` from the command line either - though `curl -u &lt;username&gt; &lt;repo_url&gt; -O &lt;output_file_name&gt;` does work.)

3. Upgrading setuptools (0.9.7) and virtualenv (1.10) to the latest versions.  Also tried installing distribute though [this overview](https://stackoverflow.com/questions/6344076/differences-between-distribute-distutils-setuptools-and-distutils2) says it was merged back into setuptools.  Either way, no dice.

Currently I just have `setup.py` print out a warning that the private repos must be downloaded separately.  This is obviously less than ideal.  I feel like there&#39;s something obvious that I&#39;m missing, but can&#39;t think what it might be. :)

Duplicate-ish question with no answers [here](https://stackoverflow.com/questions/12956168/python-how-to-connect-to-a-protected-svn-repository-with-setuptools).

Python setuptools: How can I list a private repository under install_requires?

Working with Python 2.7, I&#39;m wondering what real advantage there is in using the type `unicode` instead of `str`, as both of them seem to be able to hold Unicode strings. Is there any special reason apart from being able to set Unicode codes in `unicode` strings using the escape char `\`?:

Executing a module with:

    # -*- coding: utf-8 -*-

    a = &#39;&#225;&#39;
    ua = u&#39;&#225;&#39;
    print a, ua

Results in: &#225;, &#225;


EDIT:

More testing using Python shell:

    &gt;&gt;&gt; a = &#39;&#225;&#39;
    &gt;&gt;&gt; a
    &#39;\xc3\xa1&#39;
    &gt;&gt;&gt; ua = u&#39;&#225;&#39;
    &gt;&gt;&gt; ua
    u&#39;\xe1&#39;
    &gt;&gt;&gt; ua.encode(&#39;utf8&#39;)
    &#39;\xc3\xa1&#39;
    &gt;&gt;&gt; ua.encode(&#39;latin1&#39;)
    &#39;\xe1&#39;
    &gt;&gt;&gt; ua
    u&#39;\xe1&#39;

So, the `unicode` string seems to be encoded using `latin1` instead of `utf-8` and the raw string is encoded using `utf-8`? I&#39;m even more confused now! :S

Python str vs unicode types

I&#39;m trying to use pandas to manipulate a .csv file but I get this error:

&gt;pandas.parser.CParserError: Error tokenizing data. C error: Expected 2 fields in line 3,  saw 12

I have tried to read the pandas docs, but found nothing.

My code is simple:


    path = &#39;GOOG Key Ratios.csv&#39;
    #print(open(path).read())
    data = pd.read_csv(path)

How can I resolve this? Should I use the `csv` module or another language ?

File is from [Morningstar][1]


  [1]: http://financials.morningstar.com/ratios/r.html?t=GOOG&amp;region=usa&amp;culture=en-US

Python Pandas Error tokenizing data

Can some please explain the difference between the asfreq and resample methods in pandas? When should one use what?

Difference between asfreq and resample

I have two Series `s1` and `s2` with the same (non-consecutive) indices. How do I combine `s1` and `s2` to being two columns in a DataFrame and keep one of the indices as a third column?


Combining two Series into a DataFrame in pandas

I have two series `s1` and `s2` in pandas and want to compute the intersection i.e. where all of the values of the series are common.

How would I use the `concat` function to do this? I have been trying to work it out but have been unable to (I don&#39;t want to compute the intersection on the indices of `s1` and `s2`, but on the values).

Finding the intersection between two series in Pandas

For example if I have this:

   

    n = c(2, 3, 5) 
    s = c(&quot;aa&quot;, &quot;bb&quot;, &quot;cc&quot;) 
    b = c(TRUE, FALSE, TRUE) 
    df = data.frame(n, s, b)

      n  s     b
    1 2 aa  TRUE
    2 3 bb FALSE
    3 5 cc  TRUE

Then how do I combine the two columns `n` and `s` into a new column named `x` such that it looks like this:

      n  s     b     x
    1 2 aa  TRUE  2 aa
    2 3 bb FALSE  3 bb
    3 5 cc  TRUE  5 cc



Combine two or more columns in a dataframe into a new column with a new name

I&#39;m running a program which is processing 30,000 similar files. A random number of them are stopping and producing this error...

    File &quot;C:\Importer\src\dfman\importer.py&quot;, line 26, in import_chr
         data = pd.read_csv(filepath, names=fields)
    File &quot;C:\Python33\lib\site-packages\pandas\io\parsers.py&quot;, line 400, in parser_f
         return _read(filepath_or_buffer, kwds)
    File &quot;C:\Python33\lib\site-packages\pandas\io\parsers.py&quot;, line 205, in _read
         return parser.read()
       File &quot;C:\Python33\lib\site-packages\pandas\io\parsers.py&quot;, line 608, in read
         ret = self._engine.read(nrows)
    File &quot;C:\Python33\lib\site-packages\pandas\io\parsers.py&quot;, line 1028, in read
         data = self._reader.read(nrows)
    File &quot;parser.pyx&quot;, line 706, in pandas.parser.TextReader.read (pandas\parser.c:6745)
    File &quot;parser.pyx&quot;, line 728, in pandas.parser.TextReader._read_low_memory (pandas\parser.c:6964)
    File &quot;parser.pyx&quot;, line 804, in pandas.parser.TextReader._read_rows (pandas\parser.c:7780)
    File &quot;parser.pyx&quot;, line 890, in pandas.parser.TextReader._convert_column_data (pandas\parser.c:8793)
    File &quot;parser.pyx&quot;, line 950, in pandas.parser.TextReader._convert_tokens (pandas\parser.c:9484)
    File &quot;parser.pyx&quot;, line 1026, in pandas.parser.TextReader._convert_with_dtype (pandas\parser.c:10642)
    File &quot;parser.pyx&quot;, line 1046, in pandas.parser.TextReader._string_convert (pandas\parser.c:10853)
    File &quot;parser.pyx&quot;, line 1278, in pandas.parser._string_box_utf8 (pandas\parser.c:15657)
    UnicodeDecodeError: &#39;utf-8&#39; codec can&#39;t decode byte 0xda in position 6: invalid    continuation byte

The source/creation of these files all come from the same place. What&#39;s the best way to correct this to proceed with the import?

UnicodeDecodeError when reading CSV file in Pandas with Python

A simple pandas question: 

Is there a `drop_duplicates()` functionality to drop every row involved in the duplication? 

An equivalent question is the following: Does pandas have a set difference for dataframes? 

For example:

    In [5]: df1 = pd.DataFrame({&#39;col1&#39;:[1,2,3], &#39;col2&#39;:[2,3,4]})
    
    In [6]: df2 = pd.DataFrame({&#39;col1&#39;:[4,2,5], &#39;col2&#39;:[6,3,5]})
    
    In [7]: df1
    Out[7]: 
       col1  col2
    0     1     2
    1     2     3
    2     3     4
    
    In [8]: df2
    Out[8]: 
       col1  col2
    0     4     6
    1     2     3
    2     5     5


so maybe something like `df2.set_diff(df1)` will produce this:

       col1  col2
    0     4     6
    2     5     5

However, I don&#39;t want to rely on indexes because in my case, I have to deal with dataframes that have distinct indexes.

By the way, I initially thought about an extension of the current `drop_duplicates()` method, but now I realize that the second approach using properties of set theory would be far more useful in general. Both approaches solve my current problem, though.

Thanks!

set difference for pandas

Is there a Pythonic way to assign the values of a dictionary to its keys, in order to convert the dictionary entries into variables?
I tried this out:

    &gt;&gt;&gt; d = {&#39;a&#39;:1, &#39;b&#39;:2}
    &gt;&gt;&gt; for key,val in d.items():
            exec(&#39;exec(key)=val&#39;)
        	    
            exec(key)=val
          	         ^ 
            SyntaxError: invalid syntax


I am certain that the key-value pairs are correct because they were previously defined as variables by me before. I then stored these variables in a dictionary (as key-value pairs) and would like to reuse them in a different function. I could just define them all over again in the new function, but because I may have a dictionary with about 20 entries, I thought there may be a more efficient way of doing this.

Convert dictionary entries into variables

I am trying to create a python dictionary which is to be used as a java script var inside a html file for visualization purposes. As a requisite, I am in need of creating the dictionary with all names inside double quotes instead of default single quotes which Python uses. Is there an easy and elegant way to achieve this. 

   

        couples = [
                   [&#39;jack&#39;, &#39;ilena&#39;], 
                   [&#39;arun&#39;, &#39;maya&#39;], 
                   [&#39;hari&#39;, &#39;aradhana&#39;], 
                   [&#39;bill&#39;, &#39;samantha&#39;]]
        pairs = dict(couples)
        print pairs

**Generated Output:**

    {&#39;arun&#39;: &#39;maya&#39;, &#39;bill&#39;: &#39;samantha&#39;, &#39;jack&#39;: &#39;ilena&#39;, &#39;hari&#39;: &#39;aradhana&#39;}

**Expected Output:**

    {&quot;arun&quot;: &quot;maya&quot;, &quot;bill&quot;: &quot;samantha&quot;, &quot;jack&quot;: &quot;ilena&quot;, &quot;hari&quot;: &quot;aradhana&quot;}

I know, `json.dumps(pairs)` does the job, but the dictionary as a whole is converted into a string which isn&#39;t what I am expecting.

**P.S.:** Is there an alternate way to do this with using json, since I am dealing with nested dictionaries.

How to create a Python dictionary with double quotes as default quote format?

Is there a built-in/quick way to use a list of keys to a dictionary to get a list of corresponding items?

For instance I have:

    &gt;&gt;&gt; mydict = {&#39;one&#39;: 1, &#39;two&#39;: 2, &#39;three&#39;: 3}
    &gt;&gt;&gt; mykeys = [&#39;three&#39;, &#39;one&#39;]

How can I use `mykeys` to get the corresponding values in the dictionary as a list?

    &gt;&gt;&gt; mydict.WHAT_GOES_HERE(mykeys)
    [3, 1]


Dictionary: Get list of values for list of keys

I just discovered [this feature][1]:

&gt; Map: Map objects are simple key/value maps.

That confused me. Regular JavaScript objects are dictionaries, so how is a [`Map`][2] different from a dictionary? Conceptually, they&#39;re identical (according to [another question on Stack Overflow][3])

The documentation doesn&#39;t help either:

&gt; Map objects are collections of key/value pairs where both the keys and values may be arbitrary ECMAScript language values. A distinct key value may only occur in one key/value pair within the Map’s collection. Distinct key values as discriminated using the a comparision algorithm that is selected when the Map is created.

&gt; A Map object can iterate its elements in insertion order. Map object must be implemented using either hash tables or other mechanisms that, on average, provide access times that are sublinear on the number of elements in the collection. The data structures used in this Map objects specification is only intended to describe the required observable semantics of Map objects. It is not intended to be a viable implementation model.

…still sounds like an object to me, so clearly I&#39;ve missed something.

Why is JavaScript gaining a (well-supported) [`Map`][2] object? What does it do?

  [1]: https://www.chromestatus.com/features/4818609708728320
  [2]: https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Global_Objects/Map
  [3]: https://stackoverflow.com/questions/2884068/what-is-the-difference-between-a-map-and-a-dictionary







Map vs Object in JavaScript

I&#39;m trying to access a dict_key&#39;s element by its index:

    test = {&#39;foo&#39;: &#39;bar&#39;, &#39;hello&#39;: &#39;world&#39;}
    keys = test.keys()  # dict_keys object

    keys.index(0)
    AttributeError: &#39;dict_keys&#39; object has no attribute &#39;index&#39;

I want to get `foo`.

same with:
    
    keys[0]
    TypeError: &#39;dict_keys&#39; object does not support indexing

How can I do this?

Accessing dict_keys element by index in Python3

How can I convert all numbers that are more than 3 digits down to a 4 digit or less number?

This is exactly what I mean:

    10345 = 10.3k
    10012 = 10k
    123546 = 123.5k
    4384324 = 4.3m

Rounding is not entirely important, but an added plus. 

I have looked into NSNumberFormatter but have not found the proper solution, and I have yet to find a proper solution here on SO. Any help is greatly appreciated, thanks! 

iOS convert large numbers to smaller format

In Python console:

    ~True

Gives me:

    -2

Why? Can someone explain this particular case to me in binary?



Why does ~True result in -2?

I&#39;d like to convert result table to JSON array in MySQL using preferably only plain MySQL commands. For example with query

    SELECT name, phone FROM person;

    | name | phone |
    | Jack | 12345 |
    | John | 23455 |

the expected JSON output would be

    [
      {
        &quot;name&quot;: &quot;Jack&quot;,
        &quot;phone&quot;: 12345
      },
      {
        &quot;name&quot;: &quot;John&quot;,
        &quot;phone&quot;: 23455
      }
    ]

Is there way to do that in plain MySQL?

EDIT:

There are some answers how to do this with e.g. [MySQL and PHP][1], but I couldn&#39;t find pure MySQL solution.


  [1]: https://stackoverflow.com/questions/383631/json-encode-mysql-results

python pandas dataframe columns convert to dict key and value

Python Problem Overview

Python Solutions

What is Pseudo TTY-Allocation? (SSH and Github)

How to scale Docker containers in production

Attributions