The unreadable regex: flags and compiling

11 p.m. Only one light is on in the office. The director sits at her old mechanical keyboard, typing out the last lines of the three letters. "A regex written on one line can only be read by the person who wrote it. Give it three months and even they can't."

Here's the file with the tails of the three letters copied in order, and the access log for the agency's fourth-floor doors, which the director pulled without a word. She did it because door, 4f and badge show up in the regex fragments.

<<(?mi)^(\d{4})([-./])(\d\d)\2(\d\d)[ T]
(0[0-4]):(\d\d)\s+(?:door|gate)=4f\s+
badge=(f-0[12])\b(?!.*denied)>>

2026-10-19 22:41 door=4F badge=F-01 result=ok
2026-10-20 02:13 door=4F badge=S-07 result=ok
2026-10-20 02:58 door=4F badge=F-02 result=denied
2026-10-20 03:02 door=4F badge=F-02 result=ok
2026.10.20 03:05 gate=4F badge=f-02 result=ok
2026-10-20 09:00 door=4F badge=F-01 result=ok

The (?mi) at the very front of the regex is the first clue. It's a flag: a switch, set from outside the pattern itself, that changes how the pattern is read. Today you'll untangle this one line by turning on the flags one at a time.

11-1. Case and lines: re.I, re.M (8 min)

In the access log, only the back-door (gate) machine writes the badge number in lowercase, as f-02. Pass re.I (re.IGNORECASE) and case stops mattering.

import re

re.findall(r"badge=F-02", door_log)          # ['badge=F-02', 'badge=F-02']
re.findall(r"badge=F-02", door_log, re.I)    # ['badge=F-02', 'badge=F-02', 'badge=f-02']

You got a taste of re.M (re.MULTILINE) in Lesson 4. Normally ^ matches only at the very start of the string and $ only at the very end. Turn on re.M and they match at the start and end of every line. Here door_log holds the six lines of the access log above.

re.findall(r"^\S+ 0[0-4]:\d\d", door_log)          # []
re.findall(r"^\S+ 0[0-4]:\d\d", door_log, re.M)
# ['2026-10-20 02:13', '2026-10-20 02:58', '2026-10-20 03:02', '2026.10.20 03:05']

re.findall(r"\w+$", door_log)          # ['ok']
re.findall(r"\w+$", door_log, re.M)    # ['ok', 'ok', 'denied', 'ok', 'ok', 'ok']

Without re.M, not a single early-morning entry shows up, because the first line is at 10 p.m. When you work with line-based logs, re.M is practically a default.

11-2. re.S opens the window (6 min)

Now the three tail lines need to become one. Grabbing everything from << to >> seems like it should work, but nothing comes out.

m = re.search(r"<<(.*)>>", tails)
print(m)    # None

Dot has been telling you since Lesson 1: . can't cross a line break. The .* after << stops at the end of the first line, and there's no >> there.

The director swivels her chair and opens the window. "Go on, Dot." Turn on re.S (re.DOTALL) and . becomes truly any character, line breaks included.

m = re.search(r"<<(.*)>>", tails, re.S)
one_line = m.group(1).replace("\n", "")
print(one_line)
# (?mi)^(\d{4})([-./])(\d\d)\2(\d\d)[ T](0[0-4]):(\d\d)\s+(?:door|gate)=4f\s+badge=(f-0[12])\b(?!.*denied)

Dot steps onto the windowsill, leaps outside, comes back in through the next window and settles on the desk. The three lines have become one.

re.M and re.S have similar names and get mixed up all the time. They do completely different jobs.

• re.M changes where ^ and $ match (on every line).
• re.S changes which characters . can eat (line breaks included).

re.findall(r"^.+$", "a\nb", re.M)    # ['a', 'b']
re.findall(r"^.+$", "a\nb", re.S)    # ['a\nb']

There's a reason the Backtracker's regex turns on only (?mi) and not s. The (?!.*denied) at the end means "there must be no denied further along this line". With s on, .* would run over into the following lines and throw out a match if denied appeared on any line below it.

11-3. Spelling it out: re.X and (?#...) (9 min)

It's one line now, but it still can't be read. Turn on re.X (re.VERBOSE) and spaces and line breaks inside the pattern are ignored, and everything from # to the end of a line becomes a comment. That lets you split a pattern over several lines and annotate it.

The director retypes it piece by piece. She gives the numbered groups names and replaces \2 with (?P=sep), which you learned in Lesson 10.

DOOR = re.compile(r'''
    ^
    (?P<year>\d{4}) (?P<sep>[-./]) (?P<month>\d\d) (?P=sep) (?P<day>\d\d)   # date, a single kind of separator
    [ T]                                   # between date and time: a space or T
    (?P<hour>0[0-4]) : (?P<minute>\d\d)    # from midnight through the 4 a.m. hour
    \s+ (?:door|gate) = 4f                 # 4th-floor front door or back door
    \s+ badge = (?P<badge>f-0[12]) \b      # badge F-01 or F-02
    (?! .* denied )                        # leave out denied entries
''', re.I | re.M | re.X)

for m in DOOR.finditer(door_log):
    print(m["hour"], m["minute"], m["badge"])
# 03 02 F-02
# 03 05 f-02

It's the same regex, but now you can read it line by line. It was looking for entries where badge F-01 or F-02 actually got in through a fourth-floor door between midnight and the 4 a.m. hour.

re.X has its exceptions. To match a real space or #, escape it or put it in a class. Spaces inside a class aren't removed; that's why the pattern above has [ T].

re.fullmatch(r"\d{4} - \d\d", "2026-10", re.X)   # <re.Match object; span=(0, 7), match='2026-10'>
re.fullmatch(r"a b", "a b", re.X)                # None
re.fullmatch(r"a\ b", "a b", re.X)               # <re.Match object; span=(0, 3), match='a b'>
re.fullmatch(r"a[ ]b", "a b", re.X)              # <re.Match object; span=(0, 3), match='a b'>
re.fullmatch(r"a\#b", "a#b", re.X)               # <re.Match object; span=(0, 3), match='a#b'>

When you just want a short comment in the middle of a pattern without re.X, use (?#...). Everything inside the parentheses is ignored. (The comment below is Korean for "founding members' badges".)

re.findall(r"F-0[12](?#창립 멤버 출입증)", door_log, re.I)
# ['F-01', 'F-02', 'F-02', 'f-02', 'F-01']

The director stares at the spelled-out pattern for a long time. "Not a single variable name, all on one line. That habit of writing comments… no, the habit of not writing comments. He always used to…" She stops. She puts the old photo frame, still lying face down on the desk, into the drawer and rests her hands on the keyboard again. "Let's keep going."

11-4. Which characters count: re.A, re.U, re.L (5 min)

In Lesson 2 you saw that in a str pattern, \w includes Hangul and \d includes the Arabic-Indic digit ٣. That's because Unicode is the default. Turn on re.A (re.ASCII) and \w \W \d \D \s \S \b \B look at ASCII characters only.

re.findall(r"\w+", "café 한글 F_02")          # ['café', '한글', 'F_02']
re.findall(r"\w+", "café 한글 F_02", re.A)    # ['caf', 'F_02']
re.findall(r"\d", "٣3")                       # ['٣', '3']
re.findall(r"\d", "٣3", re.A)                 # ['3']

• re.U (re.UNICODE) is already the default for str patterns, so you never need to write it. It's kept for compatibility with old code.
• re.L (re.LOCALE) works only with bytes patterns, and its results depend on the computer's locale settings, so it isn't recommended. Passing it to a str pattern is an error.

re.compile(r"\w", re.L)
# ValueError: cannot use LOCALE flag with a str pattern

11-5. Combining flags, turning them on and off (6 min)

Combine flags with |. Their values are an enum called re.RegexFlag, and you can treat them like integers. There's also re.NOFLAG (since 3.11, value 0), meaning "no flags at all", which makes a good default for function parameters.

flags = re.I | re.M
print(flags)          # re.IGNORECASE|re.MULTILINE
print(type(flags))    # <flag 'RegexFlag'>
print(int(flags))     # 10

def find_all(pattern, text, flags=re.NOFLAG):
    return re.findall(pattern, text, flags)

You can also do what the Backtracker did and write the letters you need from (?aiLmsux) at the very start of the pattern. (?im) is the same as re.I | re.M. Anywhere other than the start, it's an error.

re.findall(r"(?im)^\S+ 0\d:\d\d(?=.*f-02)", door_log)
# ['2026-10-20 02:58', '2026-10-20 03:02', '2026.10.20 03:05']

re.compile(r"badge=(?i)f-02")
# re.PatternError: global flags not at the start of the expression at position 6

To apply a flag to only part of a pattern, use a scoped group like (?i:...). (?-i:...) turns it off just within that scope, and (?i-s:...) turns i on and s off.

re.findall(r"badge=(?i:f)-02", door_log)        # ['badge=F-02', 'badge=F-02', 'badge=f-02']
re.findall(r"(?i)BADGE=(?-i:F)-02", door_log)   # ['badge=F-02', 'badge=F-02']
re.fullmatch(r"(?s)a.(?i-s:b.)", "a\nB\n")      # None
re.fullmatch(r"(?s)a.(?i-s:b.)", "a\nBc")       # <re.Match object; span=(0, 4), match='a\nBc'>

a, L and u decide which characters count, so you can turn them on within a scope but never off.

re.findall(r"(?a:\w+)", "café")    # ['caf']
re.compile(r"(?-a:\w+)")
# re.PatternError: bad inline flags: cannot turn off flags 'a', 'u' and 'L' at position 4

Finally, re.DEBUG prints exactly how re understood your pattern. Compare the Backtracker's signature with the same pattern minus the outer repeat. Below is the first part of the output; a list of execution instructions follows it.

re.compile(r"(a+)+$", re.DEBUG)
# MAX_REPEAT 1 MAXREPEAT
#   SUBPATTERN 1 0 0
#     MAX_REPEAT 1 MAXREPEAT
#       LITERAL 97
# AT AT_END

re.compile(r"a+$", re.DEBUG)
# MAX_REPEAT 1 MAXREPEAT
#   LITERAL 97
# AT AT_END

There's a repeat (MAX_REPEAT) inside another repeat. That's exactly the shape of the slow pattern from the challenge in Lesson 8.

11-6. re.compile and Pattern objects (6 min)

Just as the director stored her spelled-out pattern under the name DOOR, re.compile turns a pattern into a Pattern object. It has the same methods as the module functions, search match fullmatch findall finditer sub split, and also accepts a search range, pos and endpos.

p = re.compile(r"\d\d:\d\d")
line = "2026-10-20 03:02 door=4F"
print(p.search(line))              # <re.Match object; span=(11, 16), match='03:02'>
print(p.match(line))               # None
print(p.match(line, 11))           # <re.Match object; span=(11, 16), match='03:02'>
print(p.fullmatch(line, 11, 16))   # <re.Match object; span=(11, 16), match='03:02'>
print(p.findall(line, 0, 15))      # []
print(len(list(p.finditer(line, 11))))   # 1

pos isn't the same as slicing the string. ^ matches only at the string's real start (or the start of a line under re.M), not at pos.

q = re.compile(r"^\d\d")
print(q.match(line, 11))     # None
print(q.match(line[11:]))    # <re.Match object; span=(0, 2), match='03'>

A Pattern object also carries information about itself. For a str pattern, re.UNICODE (32) is automatically added to flags.

print(DOOR.groups)       # 7
print(DOOR.groupindex)
# {'year': 1, 'sep': 2, 'month': 3, 'day': 4, 'hour': 5, 'minute': 6, 'badge': 7}
print(DOOR.flags)        # 106
print(re.RegexFlag(DOOR.flags))   # re.IGNORECASE|re.UNICODE|re.MULTILINE|re.VERBOSE
print(re.compile("x").flags)      # 32
print(re.compile(one_line).pattern == one_line)   # True

• The module function re.search(pattern, ...) compiles internally too. Recently used patterns stay in an internal cache, so they aren't compiled again. re.purge() clears that cache (you'll rarely need it).
• So the real benefit of re.compile isn't speed but a name. A pattern named DOOR reads well in your code.
• For type hints, write re.Pattern[str] and re.Match[str] ([bytes] for bytes patterns).

def night_entries(p: re.Pattern[str], log: str) -> list[re.Match[str]]:
    return list(p.finditer(log))

print(len(night_entries(DOOR, door_log)))   # 2

Cracking the case (10 min)

Read the access log again with the spelled-out pattern.

for m in DOOR.finditer(door_log):
    print(f"{m['year']}-{m['month']}-{m['day']} {m['hour']}:{m['minute']} {m['badge'].upper()}")
# 2026-10-20 03:02 F-02
# 2026-10-20 03:05 F-02

The original one-liner pulled from the letters finds the same two lines. Same meaning; the only difference is whether you can read it.

[m.group(7) for m in re.finditer(one_line, door_log)]   # ['F-02', 'f-02']

In the early hours of October 20, badge F-02 was denied once at the front door at 02:58, was swiped in successfully four minutes later, and passed through the back door three minutes after that. The Backtracker sent this regex to say, "Read when I came, if you can."

The director opens the badge register. "F-01 is mine. F-02…" In that row, instead of a name, there are two lines struck through. "I should have cancelled it ten years ago. I never could."

Finally, the director puts the original one-liner up on the screen and says: "It's the same regex. One version only its author can read; the other, anyone can. We write the kind anyone can read."

Exercises

1. Spell out the username rule from Lesson 4, [a-z][a-z0-9_]{3,15}, with re.X, and add a comment to each piece.
2. Use pos with re.compile(r"\d+") to find only the numbers after at in "no.17 at 03:02".
3. Check the error from re.compile(r"badge=(?i)f-02"), then fix it with a scoped flag (?i:...) so it means the same thing.