Remove first x number of characters from each row in a column of a Python dataframe

PythonStringPandasDataframeReplace

Python Problem Overview


I have a Python dataframe with about 1,500 rows and 15 columns. With one specific column I would like to remove the first 3 characters of each row. As a simple example here is a dataframe:

import pandas as pd

d = {
    'Report Number':['8761234567', '8679876543','8994434555'],
    'Name'         :['George', 'Bill', 'Sally']
     }

d = pd.DataFrame(d)

I would like to remove the first three characters from each field in the Report Number column of dataframe d.

Python Solutions


Solution 1 - Python

Use vectorised str methods to slice each string entry

In [11]:
d['Report Number'] = d['Report Number'].str[3:]
d

Out[11]:
     Name Report Number
0  George       1234567
1    Bill       9876543
2   Sally       4434555

Solution 2 - Python

It is worth noting Pandas "vectorised" str methods are no more than Python-level loops.

Assuming clean data, you will often find a list comprehension more efficient:

# Python 3.6.0, Pandas 0.19.2

d = pd.concat([d]*10000, ignore_index=True)

%timeit d['Report Number'].str[3:]           # 12.1 ms per loop
%timeit [i[3:] for i in d['Report Number']]  # 5.78 ms per loop

Note these aren't equivalent, since the list comprehension does not deal with null data and other edge cases. For these situations, you may prefer the Pandas solution.

Attributions

All content for this solution is sourced from the original question on Stackoverflow.

The content on this page is licensed under the Attribution-ShareAlike 4.0 International (CC BY-SA 4.0) license.

Content TypeOriginal AuthorOriginal Content on Stackoverflow
Questiond84_n1nj4View Question on Stackoverflow
Solution 1 - PythonEdChumView Answer on Stackoverflow
Solution 2 - PythonjppView Answer on Stackoverflow