Python Regex Explained: Patterns, Groups, Lookaheads and Pitfalls, Tested

You have a log file and need every order number, every email address and every date in it. Splitting strings by hand quickly becomes a mess. Python regex, short for regular expressions, describes the shape of the text you want, such as “a hash followed by digits”, and lets the re module find it for you.

We ran every pattern below with Python 3.13 against this sample text, and the outputs shown are from that run:

text = ("Order #101 by asha@example.com on 2026-11-11 for Rs 1,180.50; "
        "order #102 by ben.k@shop.co.in on 2026-11-12 for Rs 300.")

What is a regular expression?

A regular expression is a small pattern language for text. Ordinary characters match themselves, and special characters describe classes, repetitions and positions. \d means a digit, + means “one or more”, so \d+ matches a run of digits. Regex is supported in nearly every language, editor and database, so the time you spend learning it pays off well beyond Python.

Always use raw strings

Regex uses backslashes heavily, and so do Python strings. In a normal string, "\b" is a backspace character, which we confirmed: Python showed it as '\x08'. In regex, \b means a word boundary. Writing the pattern as a raw string, r"\b", passes the backslash through untouched. Make r"..." a habit for every pattern, and many confusing bugs never happen.

search, match and fullmatch

import re

re.search(r"#(\d+)", text).group(1)            # '101'
re.match(r"#\d+", text)                         # None
bool(re.fullmatch(r"\d{4}-\d{2}-\d{2}", "2026-11-11"))   # True

search scans the whole string and returns the first match. match only checks the beginning, which is why it returned None: our text starts with “Order”, not “#”. fullmatch requires the pattern to cover the entire string, which is what you want for validation, such as checking that a field is a date and nothing else. All three return a match object or None, so check before calling .group().

findall and finditer

re.findall(r"#(\d+)", text)              # ['101', '102']
re.findall(r"\d{4}-\d{2}-\d{2}", text)   # ['2026-11-11', '2026-11-12']
re.findall(r"[\w.+-]+@[\w-]+(?:\.[\w-]+)+", text)
# ['asha@example.com', 'ben.k@shop.co.in']

findall returns all matches as a list. Watch how groups change the result: with one group it returns the group’s text, and with two groups it returns tuples:

re.findall(r"#(\d+) by (\S+)", text)
# [('101', 'asha@example.com'), ('102', 'ben.k@shop.co.in')]

For large texts, re.finditer yields match objects one at a time, which saves memory and gives access to positions and named groups.

Core syntax cheat sheet

Groups and named groups

m = re.search(r"(?P<year>\d{4})-(?P<month>\d{2})-(?P<day>\d{2})", text)
m.groupdict()   # {'year': '2026', 'month': '11', 'day': '11'}
m.group(0)      # '2026-11-11'

Named groups, (?P<name>...), make extraction code self-explanatory: m["year"] reads far better than m.group(1), and adding a group later does not shift the numbers. Group 0 is always the whole match. A non-capturing group, (?:...), groups parts of the pattern for repetition or alternation without adding to the results, which keeps findall output clean.

Greedy vs lazy quantifiers

re.findall(r"<.+>",  "<b>bold</b> and <i>it</i>")
# ['<b>bold</b> and <i>it</i>']

re.findall(r"<.+?>", "<b>bold</b> and <i>it</i>")
# ['<b>', '</b>', '<i>', '</i>']

Quantifiers are greedy by default: .+ grabbed everything from the first < to the very last >. Adding ? makes them lazy, so they stop at the first possible end. Even better, be specific: <[^>]+> says “anything except a closing bracket”, which is both clearer and faster. (For real HTML, use a proper parser; this is just a demonstration.)

sub and split

re.sub(r"(\d{4})-(\d{2})-(\d{2})", r"\3/\2/\1", "2026-11-11")   # '11/11/2026'
re.sub(r"\d+", lambda m: str(int(m.group()) * 2), "3 pens, 4 books")   # '6 pens, 8 books'
re.split(r"\s*[,;]\s*", "pen, book;ink ,  lamp")   # ['pen', 'book', 'ink', 'lamp']

sub replaces matches, and the replacement can refer to groups with \1 or \g<name>. Passing a function instead of a string lets you compute each replacement. split handles messy separators that str.split cannot, such as mixed commas and semicolons with uneven spaces.

Lookaheads and lookbehinds

Lookarounds check what comes before or after a position without including it in the match. A classic use is password rules, where several conditions must all be true:

pw = re.compile(r"^(?=.*[A-Z])(?=.*\d)(?=.*[^\w\s]).{8,}$")

'password'   False
'Passw0rd'   False   # no symbol
'Passw0rd!'  True
'P0!a'       False   # too short

Each (?=...) is a positive lookahead that checks one rule from the start of the string. A lookbehind, (?<=...), checks what came before:

re.findall(r"(?<=Rs )[\d,.]+\d", text)   # ['1,180.50', '300']

This extracted the amounts without the “Rs ” prefix. Negative versions, (?!...) and (?<!...), assert that something does not follow or precede. Python’s lookbehinds must have a fixed width, so patterns like (?<=\d+) are rejected. For complex password policies, separate checks in plain code are often easier to read and to give helpful error messages for.

Flags

verbose = re.compile(r"""
    (?P<user>[\w.+-]+)    # user part
    @
    (?P<domain>[\w.-]+)   # domain
""", re.X)
verbose.search("mail asha@example.com now").groupdict()
# {'user': 'asha', 'domain': 'example.com'}

Verbose mode turns a cryptic one-liner into something a teammate can review. Use it for any pattern longer than about twenty characters.

Unicode surprises

bool(re.fullmatch(r"\d", "५"))         # True   (Devanagari digit five)
bool(re.fullmatch(r"\d", "५", re.A))   # False

In Python 3, \d matches any Unicode decimal digit, not just 0 to 9. That is helpful for international text, but if a value will later go into int(), a database or an API that expects ASCII digits, a pattern that “validated” the input may let through characters that break the next step. Use [0-9] or the re.A flag when you mean ASCII digits. We met a related bug in our property-based testing article, where a superscript digit broke a parser.

Catastrophic backtracking

Some patterns take exponential time on inputs that almost match. We tested a classic example, ^(a+)+$, against strings of a’s ending in “b”:

n=16   0.002s
n=20   0.037s
n=22   0.154s
safe pattern ^a+$, n=22: 0.000004s

Each extra character roughly doubled the time, because the engine tries every way of splitting the a’s between the inner and outer groups before giving up. At that rate, 30 characters would take tens of seconds, and a few more would take minutes. If users control the input, this becomes a denial-of-service risk known as ReDoS. Avoid nested quantifiers over the same characters, prefer specific character classes, limit input length before matching, and test patterns with long near-miss inputs.

Testing your patterns

Regex is code, so test it. Write a small table of strings that should match and strings that should not, including empty strings, very long strings, Unicode and near misses, and run it with pytest parametrised tests. Edge cases are where regex bugs hide: an email pattern that rejects plus signs, a date pattern that accepts month 13, or a phone pattern that misses spaces.

When not to use regex

• Simple checks: "@" in s, s.startswith("INV-") and s.split(",") are clearer and faster.
• Structured formats: use real parsers for HTML, XML, JSON, CSV and URLs.
• Dates: regex can check a shape, but use datetime.strptime to check that a date actually exists.
• Emails: a simple pattern catches typos; only a confirmation email proves an address works.

Final thoughts

Python regex is a precise tool for messy text. Use raw strings, pick search, match or fullmatch deliberately, name your groups, prefer lazy or specific patterns over greedy ones, and write long patterns in verbose mode. Then test them with tricky inputs, including Unicode and long near-misses, so a pattern never surprises you in production.

Frequently asked questions

Keep learning