Strip all non-numeric characters (except for ".") from a string in Python


Python Problem Overview

I've got a pretty good working snippit of code, but I was wondering if anyone has any better suggestions on how to do this:

val = ''.join([c for c in val if c in '1234567890.'])

What would you do?

Python Solutions

Solution 1 - Python

You can use a regular expression (using the re module) to accomplish the same thing. The example below matches runs of [^\d.] (any character that's not a decimal digit or a period) and replaces them with the empty string. Note that if the pattern is compiled with the UNICODE flag the resulting string could still include non-ASCII numbers. Also, the result after removing "non-numeric" characters is not necessarily a valid number.

>>> import re
>>> non_decimal = re.compile(r'[^\d.]+')
>>> non_decimal.sub('', '12.34fe4e')

Solution 2 - Python

Another 'pythonic' approach

filter( lambda x: x in '0123456789.', s )

but regex is faster.

Solution 3 - Python

Here's some sample code:

$ cat
a = '27893jkasnf8u2qrtq2ntkjh8934yt8.298222rwagasjkijw'
for i in xrange(1000000):
    ''.join([c for c in a if c in '1234567890.'])

$ cat
import re

non_decimal = re.compile(r'[^\d.]+')

a = '27893jkasnf8u2qrtq2ntkjh8934yt8.298222rwagasjkijw'
for i in xrange(1000000):
    non_decimal.sub('', a)

$ cat
a = '27893jkasnf8u2qrtq2ntkjh8934yt8.298222rwagasjkijw'
for i in xrange(1000000):
    ''.join([c for c in a if c.isdigit() or c == '.'])

$ cat
a = '27893jkasnf8u2qrtq2ntkjh8934yt8.298222rwagasjkijw'
for i in xrange(1000000):
    b = []
    for c in a:
        if c.isdigit() or c == '.': continue


And the timing results:

$ time python
real    0m24.735s
user    0m21.049s
sys     0m0.456s

$ time python
real    0m10.775s
user    0m9.817s
sys     0m0.236s

$ time python
real    0m38.255s
user    0m32.718s
sys     0m0.724s

$ time python
real    0m46.040s
user    0m41.515s
sys     0m0.832s

Looks like the regex is the winner so far.

Personally, I find the regex just as readable as the list comprehension. If you're doing it just a few times then you'll probably take a bigger hit on compiling the regex. Do what jives with your code and coding style.

Solution 4 - Python

A simple solution is to use regular expessions

import re 
re.sub("[^0-9^.]", "", data)

Solution 5 - Python

import string
filter(lambda c: c in string.digits + '.', s)

Solution 6 - Python

If the set of characters were larger, using sets as below might be faster. As it is, this is a bit slower than

dec = set('1234567890.')

a = '27893jkasnf8u2qrtq2ntkjh8934yt8.298222rwagasjkijw'
for i in xrange(1000000):
''.join(ch for ch in a if ch in dec)

a = '27893jkasnf8u2qrtq2ntkjh8934yt8.298222rwagasjkijw' for i in xrange(1000000): ''.join(ch for ch in a if ch in dec)

At least on my system, you can save a tiny bit of time (and memory if your string were long enough to matter) by using a generator expression instead of a list comprehension in

a = '27893jkasnf8u2qrtq2ntkjh8934yt8.298222rwagasjkijw'
for i in xrange(1000000):
''.join(c for c in a if c in '1234567890.')

Oh, and here's the fastest way I've found by far on this test string (much faster than regex) if you are doing this many, many times and are willing to put up with the overhead of building a couple of character tables.

chrs = ''.join(chr(i) for i in xrange(256))
deletable = ''.join(ch for ch in chrs if ch not in '1234567890.')

a = '27893jkasnf8u2qrtq2ntkjh8934yt8.298222rwagasjkijw'
for i in xrange(1000000):
a.translate(chrs, deletable)

a = '27893jkasnf8u2qrtq2ntkjh8934yt8.298222rwagasjkijw' for i in xrange(1000000): a.translate(chrs, deletable)

On my system, that runs in ~1.0 seconds where the regex runs in ~4.3 seconds.


All content for this solution is sourced from the original question on Stackoverflow.

The content on this page is licensed under the Attribution-ShareAlike 4.0 International (CC BY-SA 4.0) license.

Content TypeOriginal AuthorOriginal Content on Stackoverflow
QuestionadamView Question on Stackoverflow
Solution 1 - PythonMilesView Answer on Stackoverflow
Solution 2 - PythonmaxpView Answer on Stackoverflow
Solution 3 - PythonColin BurnettView Answer on Stackoverflow
Solution 4 - PythonMidhun MohanView Answer on Stackoverflow
Solution 5 - PythonJosh BothunView Answer on Stackoverflow
Solution 6 - PythonView Answer on Stackoverflow