Use [`pd.concat`][1] to merge a list of DataFrame into a single big DataFrame.

    appended_data = []
    for infile in glob.glob(&quot;*.xlsx&quot;):
        data = pandas.read_excel(infile)
        # store DataFrame in list
        appended_data.append(data)
    # see pd.concat documentation for more info
    appended_data = pd.concat(appended_data)
    # write DataFrame to an excel sheet 
    appended_data.to_excel(&#39;appended.xlsx&#39;)

  [1]: https://pandas.pydata.org/pandas-docs/stable/generated/pandas.concat.html

you can try this.

    data_you_need=pd.DataFrame()
    for infile in glob.glob(&quot;*.xlsx&quot;):
        data = pandas.read_excel(infile)
        data_you_need=data_you_need.append(data,ignore_index=True)

I hope it can help.

    

DataFrame.append() and Series.append() have been deprecated and will be removed in a future version. Use pandas.concat() instead (GH35407).

Can I specify a port range in a Dockerfile

    EXPOSE 7000-8000

and when running the container bind all these exposed ports to the same ports on the host machine?

    docker run -p 7000-8000:7000-8000

Docker expose all ports or range of ports from 7000 to 8000

I am using the AmazonS3Client in an Android app using a getObject request to download an image from my Amazon S3 bucket.

Currently, I am getting this exception:
 

    com.amazonaws.services.s3.model.AmazonS3Exception: 
    The specified key does not exist.
     (Service: Amazon S3; Status Code: 404; Error Code: NoSuchKey;

Even though I am able to see the object with the specified key in my S3 bucket.


Amazon S3 exception: &quot;The specified key does not exist&quot;

I am accessing a series of Excel files in a for loop. I then read the data in the excel file to a pandas dataframe. I cant figure out how to append these dataframes together to then save the dataframe (now containing the data from all the files) as a new Excel file.

Here&#39;s what I tried:

    for infile in glob.glob(&quot;*.xlsx&quot;):
        data = pandas.read_excel(infile)
        appended_data = pandas.DataFrame.append(data) # requires at least two arguments
    appended_data.to_excel(&quot;appended.xlsx&quot;)

Thanks!

Appending pandas dataframes generated in a for loop

I am accessing a series of Excel files in a for loop. I then read the data in the excel file to a pandas dataframe. I cant figure out how to append these dataframes together to then save the dataframe (now containing the data from all the files) as a new Excel file.
Here's what I tried:
<pre><code class="hljs language-ini">for infile in glob.glob("*.xlsx"):
 data = pandas.read_excel(infile)
 appended_data = pandas.DataFrame.append(data) # requires at least two arguments
appended_data.to_excel("appended.xlsx")
</code></pre>
Thanks!

I have a very large data frame in python and I want to drop all rows that have a particular string inside a particular column.

For example, I want to drop all rows which have the string &quot;XYZ&quot; as a substring in the column C of the data frame.

Can this be implemented in an efficient way using .drop() method?

How to drop rows from pandas data frame that contains a particular string in a particular column?

Is it possible to get a partial view of a `dict` in Python analogous of pandas `df.tail()/df.head()`. Say you have a very long `dict`, and you just want to check some of the elements (the beginning, the end, etc) of the `dict`. Something like:

    dict.head(3)  # To see the first 3 elements of the dictionary.

    {[1,2], [2, 3], [3, 4]}

Thanks


In Python how to obtain a partial view of a dict?

I have written a module (a file `my_mod.py` file residing in the folder `my_module`).
Currently, I am working in the file `cool_script.py` that resides in the folder `cur_proj`. I have opened the folder in PyCharm using File -- open (and I assume, hence, it is a PyCharm project).

In ProjectView (CMD-7), I can see my project `cur_proj` (in red) and under &quot;External Libraries&quot; I do see `my_module`. In cool_script.py, I can write 

    from my_module import my_mod as mm
 
and PyCharm even makes suggestion for my_mod. So far so good.

**However, when I try to run cool_script.py, PyCharm tells me 
*&quot;No module named my_module&quot;***

This seems strange to me, because

A) in the terminal (OS 10.10.2), in python, I can import the module no problem -- there is a corresponding entry in the PYTHONPATH in .bashrc

B) in PyCharm -- Settings -- Project cur_proj -- Project Interpreter -- CogWheel next to python interpreter -- more -- show paths for selected interpreter icon, the paths from PYTHONPATH do appear (as I think they should)

Hence, why do I get the error when I try to run cool_script.py?  -- **What am I missing?**

Notes:

- I am [not declaring a different / special python version](https://stackoverflow.com/questions/25150355/requests-module-works-in-console-but-not-script-all-other-modules-work-normall) at the top of cool_script.py
- I made sure that [the path to `my_module` is correct](https://stackoverflow.com/questions/19885821/how-do-i-import-modules-in-pycharm)
- I [put `__init__.py` files (empty files)](https://stackoverflow.com/questions/12284667/django-error-no-module-named) both in `my_module` and in `cur_proj`
- I am not using `virtualenv`


# Addendum 2015-Feb-25
When I go in PyCharm to Run -- Edit Configurations, for my current project, there are two options that are selected with a check mark: &quot;*Add content roots to PYTHONPATH*&quot; and &quot;*Add source roots to PYTHONPATH*&quot;. When I have both unchecked, I can load my module.

So it works now -- **but why**? 

Further questions emerged:

- What are &quot;content roots&quot; and what are &quot;source roots&quot;? And why does adding something to the PYTHONPATH make it somehow break?
- should I uncheck both of those options all the time (so also in the defaults, not only the project specific configurations (left panel of the Run/Debug Configurations dialog)?

PyCharm error: &#39;No Module&#39; when trying to import own module (python script)

I have a legacy db table which has composite primary key. I don&#39;t think I will be able to change the structure to include a surrogate key, as there is some code written that uses that table. And in django, I cannot use that table, as it doesn&#39;t have a primary key(non-composite). 

Do django models support composite primary keys? If not, is there any workaround without changing the structure of the table? 

P.S. I am using postgresql.

Composite primary key in django

We developed a Chrome extension, and I want to test our extension with Selenium. I created a test, but the problem is that our extension opens a new tab when it&#39;s installed, and I think I get an exception from the other tab. Is it possible to switch to the active tab I&#39;m testing? Or another option is to start with the extension disabled, then login to our website and only then enable the extension. Is it possible? Here is my code:

	def login_to_webapp(self):
		self.driver.get(url=&#39;http://example.com/logout&#39;)
		self.driver.maximize_window()
		self.assertEqual(first=&quot;Web Editor&quot;, second=self.driver.title)
		action = webdriver.ActionChains(driver=self.driver)
		action.move_to_element(to_element=self.driver.find_element_by_xpath(xpath=&quot;//div[@id=&#39;header_floater&#39;]/div[@class=&#39;header_menu&#39;]/button[@class=&#39;btn_header signature_menu&#39;][text()=&#39;My signature&#39;]&quot;))
		action.perform()
		self.driver.find_element_by_xpath(xpath=&quot;//ul[@id=&#39;signature_menu_downlist&#39;][@class=&#39;menu_downlist&#39;]/li[text()=&#39;Log In&#39;]&quot;).click()
		self.driver.find_element_by_xpath(xpath=&quot;//form[@id=&#39;atho-form&#39;]/div[@class=&#39;input&#39;]/input[@name=&#39;useremail&#39;]&quot;).send_keys(&quot;[email]&quot;)
		self.driver.find_element_by_xpath(xpath=&quot;//form[@id=&#39;atho-form&#39;]/div[@class=&#39;input&#39;]/input[@name=&#39;password&#39;]&quot;).send_keys(&quot;[password]&quot;)
		self.driver.find_element_by_xpath(xpath=&quot;//form[@id=&#39;atho-form&#39;]/button[@type=&#39;submit&#39;][@class=&#39;atho-button signin_button&#39;][text()=&#39;Sign in&#39;]&quot;).click()

The test fails with `ElementNotVisibleException: Message: element not visible`, because in the new tab (opened by the extension) &quot;Log In&quot; is not visible (I think the new tab is opened only after the command `self.driver.get(url=&#39;http://example.com/logout&#39;)`).

**Update**: I found out that the exception is not related to the extra tab, it&#39;s from our website. But I closed the extra tab with this code, according to @aberna&#39;s answer:

	def close_last_tab(self):
		if (len(self.driver.window_handles) == 2):
			self.driver.switch_to.window(window_name=self.driver.window_handles[-1])
			self.driver.close()
			self.driver.switch_to.window(window_name=self.driver.window_handles[0])

After closing the extra tab, I can see my tab in the video.

How do I switch to the active tab in Selenium?

Say `df` is a pandas dataframe.

- `df.loc[]` only accepts names 
- `df.iloc[]` only accepts integers (actual placements)
- `df.ix[]` accepts both names and integers:

When referencing rows, `df.ix[row_idx, ]` only wants to be given names. e.g. 

    df = pd.DataFrame({&#39;a&#39; : [&#39;one&#39;, &#39;two&#39;, &#39;three&#39;,&#39;four&#39;, &#39;five&#39;, &#39;six&#39;],
                       &#39;1&#39; : np.arange(6)})
    df = df.ix[2:6]
    print(df)

       1      a
    2  2  three
    3  3   four
    4  4   five
    5  5    six

    df.ix[0, &#39;a&#39;]

throws an error, it doesn&#39;t give return &#39;two&#39;. 


When referencing columns, iloc is prefers integers, not names. e.g.

    df.ix[2, 1]

returns &#39;three&#39;, not 2. (Although `df.idx[2, &#39;1&#39;]` does return `2`).

Oddly, I&#39;d like the exact opposite functionality. Usually my column names are very meaningful, so in my code I reference them directly. But due to a lot of observation cleaning, the row names in my pandas data frames don&#39;t usually correspond to `range(len(df))`. 

I realize I can use:

    df.iloc[0].loc[&#39;a&#39;] # returns three

But it seems ugly! Does anyone know of a better way to do this, so that the code would look like this?

    df.foo[0, &#39;a&#39;] # returns three

In fact, is it possible to add on my own new method to `pandas.core.frame.DataFrame`s, so e.g.
`df.idx(rows, cols)` is in fact `df.iloc[rows].loc[cols]`?



   

Indexing Pandas data frames: integer rows, named columns



Recently began branching out from my safe place (R) into Python and and am a bit confused by the cell localization/selection in `Pandas`. I&#39;ve read the documentation but I&#39;m struggling to understand the practical implications of the various localization/selection options. 

 Is there a reason why I should ever use `.loc` or `.iloc` over `at`, and `iat` or vice versa?  **In what situations should I use which method?**


-----

&gt; &lt;sub&gt;Note: future readers be aware that this question is old and was written before pandas v0.20 when there used to exist a function called `.ix`. This method was later split into two - `loc` and `iloc` - to make the explicit distinction between positional and label based indexing. Please beware that `ix` was discontinued due to inconsistent behavior and being hard to grok, and no longer exists in current versions of pandas (&gt;= 1.0).&lt;/sub&gt;

pandas loc vs. iloc vs. at vs. iat?

I have a `pandas.DataFrame` called `df` which has an automatically generated index, with a column `dt`:

    df[&#39;dt&#39;].dtype, df[&#39;dt&#39;][0]
    # (dtype(&#39;&lt;M8[ns]&#39;), Timestamp(&#39;2014-10-01 10:02:45&#39;))

What I&#39;d like to do is create a new column truncated to hour precision. I&#39;m currently using:

    df[&#39;dt2&#39;] = df[&#39;dt&#39;].apply(lambda L: datetime(L.year, L.month, L.day, L.hour))

This works, so that&#39;s fine. However, I&#39;ve an inkling there&#39;s some nice way using `pandas.tseries.offsets` or creating a `DatetimeIndex` or similar. 

**So if possible, is there some `pandas` wizardry to do this?** 


Truncate `TimeStamp` column to hour precision in pandas `DataFrame`

I have a line of code: 

    g = x.groupby(&#39;Color&#39;)

The colors are Red, Blue, Green, Yellow, Purple, Orange, and Black. How do I return this list? For similar attributes, I use x.Attribute and it works fine, but x.Color doesn&#39;t behave the same way.

Content Type	Original Author	Original Content on Stackoverflow
Question	El Confuso	View Question on Stackoverflow
Solution 1 - Python	biobirdman	View Answer on Stackoverflow
Solution 2 - Python	ye jiawei	View Answer on Stackoverflow
Solution 3 - Python	BO-CHRISTOPHER REDFEARN	View Answer on Stackoverflow

Appending pandas dataframes generated in a for loop

Python Problem Overview

Python Solutions

Solution 1 - Python

Solution 2 - Python

Solution 3 - Python

Amazon S3 exception: "The specified key does not exist"

Docker expose all ports or range of ports from 7000 to 8000

Attributions