How to find overlapping matches with a regexp?

>>> match = re.findall(r'\w\w', 'hello')
>>> print match
['he', 'll']

Since \w\w means two characters, 'he' and 'll' are expected. But why do 'el' and 'lo' not match the regex?

>>> match1 = re.findall(r'el', 'hello')
>>> print match1
['el']
>>>

findall doesn't yield overlapping matches by default. This expression does however:

>>> re.findall(r'(?=(\w\w))', 'hello')
['he', 'el', 'll', 'lo']

Here (?=...) is a lookahead assertion:

> (?=...) matches if ... matches next, but doesn’t consume any of the > string. This is called a lookahead assertion. For example, > Isaac (?=Asimov) will match 'Isaac ' only if it’s followed by 'Asimov'.

You can use the new Python regex module, which supports overlapping matches.

>>> import regex as re
>>> match = re.findall(r'\w\w', 'hello', overlapped=True)
>>> print match
['he', 'el', 'll', 'lo']

Except for zero-length assertion, character in the input will always be consumed in the matching. If you are ever in the case where you want to capture certain character in the input string more the once, you will need zero-length assertion in the regex.

There are several zero-length assertion (e.g. ^ (start of input/line), $ (end of input/line), \b (word boundary)), but look-arounds ((?<=) positive look-behind and (?=) positive look-ahead) are the only way that you can capture overlapping text from the input. Negative look-arounds ((?<!) negative look-behind, (?!) negative look-ahead) are not very useful here: if they assert true, then the capture inside failed; if they assert false, then the match fails. These assertions are zero-length (as mentioned before), which means that they will assert without consuming the characters in the input string. They will actually match empty string if the assertion passes.

Applying the knowledge above, a regex that works for your case would be:

(?=(\w\w))

Am no regex expert but I would like to answer my similar question.

If you want to use a capture group with the lookahead:

example regex: (\d)(?=.\1)

string: 5252

this will match the first 5 as well as the first 2

The (\d) is to make a capture group, (?=\d\1) is to match any digit followed by the capture group 1 without consuming the string, thus allow overlapping