&gt; **Deprecation Notice**: `DataFrame.append` and `Series.append` were [deprecated in v1.4.0](https://pandas.pydata.org/pandas-docs/stable/whatsnew/v1.4.0.html#deprecated-dataframe-append-and-series-append)


I believe you can use the `append` method

    bigdata = data1.append(data2, ignore_index=True)

to keep their indexes just don&#39;t use the `ignore_index` keyword...

You can also use `pd.concat`, which is particularly helpful when you are joining more than two dataframes:

    bigdata = pd.concat([data1, data2], ignore_index=True, sort=False)

Thought to add this here in case someone finds it useful. @ostrokach already mentioned how you can merge the data frames across rows which is

    df_row_merged = pd.concat([df_a, df_b], ignore_index=True)

To merge across columns, you can use the following syntax:

    df_col_merged = pd.concat([df_a, df_b], axis=1)


If you&#39;re working with big data and need to concatenate multiple datasets calling `concat` many times can get performance-intensive.

If you don&#39;t want to create a new df each time, you can instead aggregate the changes and call `concat` only once:

```python
frames = [df_A, df_B]  # Or perform operations on the DFs
result = pd.concat(frames)
```

This is pointed out in the pandas docs under [concatenating objects](http://pandas.pydata.org/pandas-docs/stable/merging.html#concatenating-objects) at the bottom of the section):

&gt; **Note**: It is worth noting however, that `concat` (and therefore `append`)
&gt; makes a full copy of the data, and that constantly reusing this
&gt; function can create a significant performance hit. If you need to use
&gt; the operation over several datasets, use a list comprehension.

If you want to update/replace the values of first dataframe `df1` with the values of second dataframe `df2`. you can do it by following steps —

**Step 1:** Set index of the first dataframe (df1)

    df1.set_index(&#39;id&#39;)

**Step 2:** Set index of the second dataframe (df2)

    df2.set_index(&#39;id&#39;)

and finally update the dataframe using the following snippet —

    df1.update(df2)

I am using an img tag of HTML to show a photo in our application. I have set both its height and width attribute to 64. I need to show any image resolution (e.g. 256x256, 1024x768, 500x400, 205x246, etc.) as 64x64. But by setting the height and width attributes of an img tag to 64, it&#39;s not maintaining the aspect ratio, so the image looks distorted.

For your reference my exact code is:

    &lt;img src=&quot;Runtime Path to photo&quot; border=&quot;1&quot; height=&quot;64&quot; width=&quot;64&quot;&gt;

How to maintain aspect ratio using HTML IMG tag

I am trying to dump a Postgresql database using the **pg_dump** tool. 

    $ pg_dump books &gt; books.out

How ever i am getting this error. 

    pg_dump: server version: 9.2.1; pg_dump version: 9.1.6
    pg_dump: aborting because of server version mismatch

The `--ignore-version` option is now deprecated and really would not be a a solution to my issue even if it had worked.

How can I upgrade **pg_dump** to resolve this issue?


Postgresql 9.2 pg_dump version mismatch

I&#39;m using Pandas data frames. I have a initial data frame, say `D`. I extract two data frames from it like this:

    
    A = D[D.label == k]
    B = D[D.label != k]


I want to combine `A` and `B` so I can have them as one DataFrame, something like a union operation. The order of the data is not important. However, when we sample `A` and `B` from `D`, they retain their indexes from `D`.

How do I combine two dataframes?

I'm using Pandas data frames. I have a initial data frame, say <code>D</code>. I extract two data frames from it like this:
<pre><code class="hljs language-css">A = D[D.label == k]
B = D[D.label != k]
</code></pre>
I want to combine <code>A</code> and <code>B</code> so I can have them as one DataFrame, something like a union operation. The order of the data is not important. However, when we sample <code>A</code> and <code>B</code> from <code>D</code>, they retain their indexes from <code>D</code>.

Removing numbers from string

This code:

    df2 = (
        pd.DataFrame({
            &#39;X&#39; : [&#39;X1&#39;, &#39;X1&#39;, &#39;X1&#39;, &#39;X1&#39;], 
            &#39;Y&#39; : [&#39;Y2&#39;, &#39;Y1&#39;, &#39;Y1&#39;, &#39;Y1&#39;], 
            &#39;Z&#39; : [&#39;Z3&#39;, &#39;Z1&#39;, &#39;Z1&#39;, &#39;Z2&#39;]
        })
    )
    g = df2.groupby(&#39;X&#39;)
    pd.pivot_table(g, values=&#39;X&#39;, rows=&#39;Y&#39;, cols=&#39;Z&#39;, margins=False, aggfunc=&#39;count&#39;)

returns the following error:

    Traceback (most recent call last): ... 
    AttributeError: &#39;Index&#39; object has no attribute &#39;index&#39;

How do I get a Pivot Table with **counts of unique values** of one DataFrame column for two other columns?  
Is there `aggfunc` for count unique? Should I be using `np.bincount()`?

NB. I am aware of `pandas.Series.values_counts()` however I need a pivot table.

---
EDIT: The output should be:

    Z   Z1  Z2  Z3
    Y             
    Y1   1   1 NaN
    Y2 NaN NaN   1


Python Pandas : pivot table with aggfunc = count unique distinct

I would like to get the result of the fetchall operation in a list instead of tuple of tuple or tuple of dictionaries. 
For example, 

    cursor = connection.cursor() #Cursor could be a normal cursor or dict cursor
    query = &quot;Select id from bs&quot;
    cursor.execute(query)
    row = cursor.fetchall()

Now, the problem is the resultant row is either ((123,),(234,)) or ({&#39;id&#39;:123}, {&#39;id&#39;:234})
What I am looking for is (123,234) or [123,234]. Be best if I can save on parsing the resulset. Thanks in advance

Python MySQLDB: Get the result of fetchall in a list

I was looking at [PyPy](http://pypy.org/) and I was just wondering why it hasn&#39;t been adopted into the mainline Python distributions. Wouldn&#39;t things like JIT compilation and lower memory footprint greatly improve the speeds of all Python code?

In short, what are the main drawbacks of PyPy that cause it to remain a separate project?

Why wasn&#39;t PyPy included in standard Python?

This has to be easier than what I am running into. My problem is turning a string that looks like this:

    ABC12DEF3G56HIJ7

into

    12 * ABC
    3  * DEF
    56 * G
    7  * HIJ

And I can&#39;t, for the life of me, design a correct set of loops using REGEX matching. The crux of the issue is that the code has to be completely general because I cannot assume how long the `[A-Z]` fragments will be, nor how long the `[0-9]` fragments will be.

Thank you for any assistance!


Looping through python regex matches

I&#39;m reading a CSV with float numbers like this:

    Bob,0.085
    Alice,0.005

And import into a dataframe, and write this dataframe to a new place

    df = pd.read_csv(orig)
    df.to_csv(pandasfile)

Now this `pandasfile` has:

    Bob,0.085000000000000006
    Alice,0.0050000000000000001


What happen? maybe I have to cast to a different type like float32 or something? 

Im using __pandas 0.9.0__ and __numpy 1.6.2__.




float64 with pandas to_csv

I want to be able to set the major and minor xticks and their labels for a time series graph plotted from a Pandas time series object.  

The Pandas 0.9 &quot;what&#39;s new&quot; page says: 

&gt; &quot;you can either use to_pydatetime or register a converter for the
&gt; Timestamp type&quot;

 but I can&#39;t work out how to do that so that I can use the matplotlib `ax.xaxis.set_major_locator` and `ax.xaxis.set_major_formatter` (and minor) commands.

If I use them without converting the pandas times, the x-axis ticks and labels end up wrong.

By using the &#39;xticks&#39; parameter I can pass the major ticks to pandas.plot, and then set the major tick labels. I can&#39;t work out how to do the minor ticks using this approach. (I can set the labels on the default minor ticks set by pandas.plot)

Here is my test code:

&lt;!-- language: lang-py --&gt;
    
    import pandas
    print &#39;pandas.__version__ is &#39;, pandas.__version__
    print &#39;matplotlib.__version__ is &#39;, matplotlib.__version__    

    dStart = datetime.datetime(2011,5,1) # 1 May
    dEnd = datetime.datetime(2011,7,1) # 1 July    

    dateIndex = pandas.date_range(start=dStart, end=dEnd, freq=&#39;D&#39;)
    print &quot;1 May to 1 July 2011&quot;, dateIndex      

    testSeries = pandas.Series(data=np.random.randn(len(dateIndex)),
                               index=dateIndex)    

    ax = plt.figure(figsize=(7,4), dpi=300).add_subplot(111)
    testSeries.plot(ax=ax, style=&#39;v-&#39;, label=&#39;first line&#39;)    

    # using MatPlotLib date time locators and formatters doesn&#39;t work with new
    # pandas datetime index
    ax.xaxis.set_minor_locator(matplotlib.dates.WeekdayLocator(byweekday=(1),
                                                               interval=1))
    ax.xaxis.set_minor_formatter(matplotlib.dates.DateFormatter(&#39;%d\n%a&#39;))
    ax.xaxis.grid(True, which=&quot;minor&quot;)
    ax.xaxis.grid(False, which=&quot;major&quot;)
    ax.xaxis.set_major_formatter(matplotlib.dates.DateFormatter(&#39;\n\n\n%b%Y&#39;))
    plt.show()    

    # set the major xticks and labels through pandas
    ax2 = plt.figure(figsize=(7,4), dpi=300).add_subplot(111)
    xticks = pandas.date_range(start=dStart, end=dEnd, freq=&#39;W-Tue&#39;)
    print &quot;xticks: &quot;, xticks
    testSeries.plot(ax=ax2, style=&#39;-v&#39;, label=&#39;second line&#39;,
                    xticks=xticks.to_pydatetime())
    ax2.set_xticklabels([x.strftime(&#39;%a\n%d\n%h\n%Y&#39;) for x in xticks]);
    # set the text of the first few minor ticks created by pandas.plot
    #    ax2.set_xticklabels([&#39;a&#39;,&#39;b&#39;,&#39;c&#39;,&#39;d&#39;,&#39;e&#39;], minor=True)
    # remove the minor xtick labels set by pandas.plot 
    ax2.set_xticklabels([], minor=True)
    # turn the minor ticks created by pandas.plot off 
    # plt.minorticks_off()
    plt.show()
    print testSeries[&#39;6/4/2011&#39;:&#39;6/7/2011&#39;]

and its output:

    pandas.__version__ is  0.9.1.dev-3de54ae
    matplotlib.__version__ is  1.1.1
    1 May to 1 July 2011 &lt;class &#39;pandas.tseries.index.DatetimeIndex&#39;&gt;
    [2011-05-01 00:00:00, ..., 2011-07-01 00:00:00]
    Length: 62, Freq: D, Timezone: None

![Graph with strange dates on xaxis][1]

    xticks:  &lt;class &#39;pandas.tseries.index.DatetimeIndex&#39;&gt;
    [2011-05-03 00:00:00, ..., 2011-06-28 00:00:00]
    Length: 9, Freq: W-TUE, Timezone: None

![Graph with correct dates][2]

    2011-06-04   -0.199393
    2011-06-05   -0.043118
    2011-06-06    0.477771
    2011-06-07   -0.033207
    Freq: D

**Update:** I&#39;ve been able to get closer to the layout I wanted by using a loop to build the major xtick labels:

    # only show month for first label in month
    month = dStart.month - 1
    xticklabels = []
    for x in xticks:
        if  month != x.month :
            xticklabels.append(x.strftime(&#39;%d\n%a\n%h&#39;))
            month = x.month
        else:
            xticklabels.append(x.strftime(&#39;%d\n%a&#39;))

However, this is a bit like doing the x-axis using `ax.annotate`: possible but not ideal.

  [1]: http://i.stack.imgur.com/8ndjM.png
  [2]: http://i.stack.imgur.com/hQy8v.png

Pandas timeseries plot setting x-axis major and minor ticks and labels

I am going through the &#39;Python for Data Analysis&#39; book and having trouble in the &#39;Example: 2012 Federal Election Commision Database&#39; section reading the data to a DataFrame. The trouble is that one of the columns of data is always being set as the index column, even when the index_col argument is set to None. 

Here is the link to the data : &lt;http://www.fec.gov/disclosurep/PDownload.do&gt;.

Here is the loading code (to save time in the checking, I set the nrows=10):

    import pandas as pd
    fec = pd.read_csv(&#39;P00000001-ALL.csv&#39;,nrows=10,index_col=None)

To keep it short I am excluding the data column outputs, but here is my output (please not the Index values):

    In [20]: fec
    
    Out[20]:
    &lt;class &#39;pandas.core.frame.DataFrame&#39;&gt;
    Index: 10 entries, C00410118 to C00410118
    Data columns:
    ...
    dtypes: float64(4), int64(3), object(11)

And here is the book&#39;s output (again with data columns excluded):

    In [13]: fec = read_csv(&#39;P00000001-ALL.csv&#39;)
    In [14]: fec
    Out[14]:
    &lt;class &#39;pandas.core.frame.DataFrame&#39;&gt;
    Int64Index: 1001731 entries, 0 to 1001730
    ...
    dtypes: float64(1), int64(1), object(14)

The Index values in my output are actually the first column of data in the file, which is then moving all the rest of the data to the left by one. Would anyone know how to prevent this column of data to be listed as an index? I would like to have the index just +1 increasing integers.

I am fairly new to python and pandas, so I apologize for any inconvenience. Thanks.




pandas read_csv index_col=None not working with delimiters at the end of each line

In R when you need to retrieve a column index based on the name of the column you could do

    idx &lt;- which(names(my_data)==my_colum_name)

Is there a way to do the same with pandas dataframes?

Content Type	Original Author	Original Content on Stackoverflow
Question	MKoosej	View Question on Stackoverflow
Solution 1 - Python	Joran Beasley	View Answer on Stackoverflow
Solution 2 - Python	ostrokach	View Answer on Stackoverflow
Solution 3 - Python	pelumi	View Answer on Stackoverflow
Solution 4 - Python	martin-martin	View Answer on Stackoverflow
Solution 5 - Python	Mohsin Mahmood	View Answer on Stackoverflow

How do I combine two dataframes?

Python Problem Overview

Python Solutions

Solution 1 - Python

Solution 2 - Python

Solution 3 - Python

Solution 4 - Python

Solution 5 - Python

Postgresql 9.2 pg_dump version mismatch

How to maintain aspect ratio using HTML IMG tag

Attributions