Disclaimer & Intro

This post has been made as my notes, even though I attempt to explain what I have built and how, I do not owe anyone any explanation. Do NOT expect anything.

My blog is my garden.

WARNING: No AI was used to write se or any line of its codebase or any of the blog articles. Not one token. Not for the parser, not for the SIMD, not for the awk interpreter.

Before We Go Further

Part 1 covered the pipeline model and the then benchmark numbers. Part 2 covered the awk interpreter, fuzzy matching, semantic search, and tree navigation. Read those first if you haven't. This post assumes you have se in hand and shows you what it's actually for.

Two things to know before you go deep:

se is built and tuned for Apple Silicon. The NEON scanner is the whole reason the substitution and literal-search numbers in Part 1 look the way they do. Everything also compiles and runs on x86 and on Linux/*BSD through a SWAR fallback — it just doesn't get the same speedup there yet. If you're on one of those platforms and want to help close that gap, the repo is now public and the SIMD boundary is a clean seam to extend: github.com/lolt3under/se-rs.

Second: there's a libsers crate on the roadmap — a library extraction of the same engine (views, selectors, mutations, the SIMD backends) so you can embed structural text processing in your own Rust program instead of shelling out to the se binary. Not built yet. If you want it before I get to it, the repo takes PRs.

Install it and follow along:

cargo install se-rs

Part One: Three Cookbook Problems, Crushed

Every UNIX text-processing cookbook has the same handful of problems in it. Pull a value out of a nested structure. Find something despite a typo. Chain three operations into one without spilling intermediate results into a temp file. Below is each one run for real on this machine, se next to the alternative, output included.

1. The Nested Config Value

Given a config file, pull out the object that encloses a given key — not the key's own value, the block that contains it:

{
  "server": {
    "host": "0.0.0.0",
    "timeout": 30,
    "retry": {
      "attempts": 3,
      "backoff": { "base": 100, "max": 5000 }
    }
  },
  "client": { "timeout": 10, "retry": { "attempts": 1 } }
}

se 'x/backoff/ + p' config.json

{
      "attempts": 3,
      "backoff": {
        "base": 100,
        "max": 5000
      }
    }

One command. x/backoff/ finds the key, + widens the view to the smallest balanced bracket pair that encloses it — the retry object. sed and awk have no concept of a matched bracket at all; every line is independent to them. Here is the closest awk equivalent I could write, hand-rolled and tested, not a strawman:

{
  lines[NR] = $0
if ($0 ~ /backoff/ && !found) { found = 1; start = stack[depth] }
  o = gsub(/{/, "{"); c = gsub(/}/, "}")
  for (i = 0; i < o; i++) { depth++; stack[depth] = NR }
  depth -= c
}
END {
  if (!found) { print "no match"; exit 1 }
  d = 0
for (n = start; n <= NR; n++) {
    d += gsub(/{/, "{", lines[n]) - gsub(/}/, "}", lines[n])
    print lines[n]
    if (n > start && d == 0) break
  }
}

16 lines to hand-roll a bracket-depth stack, and it still only gets you line-level output — it grabs the whole "retry": { line, key label and all, because awk's finest unit is a line. se grabs exactly the byte span of the bracket pair, nothing more. That's Pike's entire argument from 1987, reproduced in miniature: the line is the wrong unit, and you can feel the seam the moment you try to work around it by hand.

2. The Typo That Grep Can't See

An incident log has one entry filed under the wrong spelling, and one that got mangled by a copy-paste from Slack:

service=auth  code=NullPointerException      note=null user session
service=cache code=OutOfMemoryError          note=heap exhausted
service=auth  code=IllegalStateException     note=double login
service=cache code=OutOfMemmoryError         note=heap exhausted, hand-typed ticket
service=io    code=OutOffMemoryError         note=copy-pasted from a Slack message
service=auth  code=NullPointerException      note=null user session again

grep 'OutOfMemoryError' incidents.log

service=cache code=OutOfMemoryError          note=heap exhausted

Neither grep nor rg has any notion of edit distance. They match exactly what you typed and nothing else — the two misspelled tickets simply don't exist as far as either tool is concerned. se has fuzzy matching built into the selector language:

se 'x/.*\n?/ ~2/OutOfMemoryError/ p' incidents.log

service=cache code=OutOfMemoryError          note=heap exhausted
service=cache code=OutOfMemmoryError         note=heap exhausted, hand-typed ticket
service=io    code=OutOffMemoryError         note=copy-pasted from a Slack message

~2/pattern/ keeps a view if it contains a substring within Levenshtein distance 2 of the literal. All three real tickets, nothing else. There's no -fuzzy flag on grep. This is a selector in the base language.

3. Filter, Extract, Join — One Breath

Take that same fuzzy match and turn it into a one-line incident report — filter the typo'd and correct spellings together, pull out just the ticket code, join them into one line:

se 'x/.*\n?/ ~2/OutOfMemoryError/ x/code=\S+/ r/,/ p' incidents.log

code=OutOfMemoryError,code=OutOfMemmoryError,code=OutOffMemoryError

Split into records, fuzzy-filter, extract the field, reduce to one joined value, print. Four structural operations, one pipeline, no intermediate file, no second process. The nearest classic-tools version can't even get the fuzzy part — grep under-matches by construction:

grep 'OutOfMemoryError' incidents.log \
  | awk -F'code=' '{split($2,a," "); printf "%s%s", (n++?",":""), a[1]} END{print ""}'

OutOfMemoryError

One result instead of three, because the pipeline was never given a chance to see the typos in the first place. This is the pattern from both prior posts, just with a fresh example: se composes selectors the way a Unix pipe composes processes, except the composition happens inside one expression instead of across fork() boundaries.

Part Two: Learn Regex, Then Learn to Stop Thinking in Lines

If you don't know regular expressions yet, learn the fundamentals first — literals, character classes ([a-z], \d, \s), quantifiers (*, +, ?, {m,n}), groups (...), alternation |, anchors ^ $. se's regex syntax comes from regex-automata — the same family Rust's regex crate is built on. Standard, modern, PCRE-adjacent, no surprises. Look-around and backreferences are the one thing it doesn't do; you won't miss them for anything in this post.

Once regex syntax is not the obstacle, read the paper that this entire tool is a response to: Rob Pike, Structural Regular Expressions (EUUG Spring 1987, also archived as a PDF). Six pages. Pike's argument, compressed: regex tells you where something is, but every classic tool then throws that location away and snaps back to "which line was that in." You searched with precision and got handed a blunt instrument back. Structural regular expressions keep the match itself as the unit you keep operating on — extract it, narrow it further, extract from that. se is that idea, implemented, with SIMD underneath.

With that context, here's the command language, basic to advanced. Every example below should run fine.

Selectors — replace the view stream:

# x/re/ — extract: each view becomes its matches
printf 'the cat sat on the mat\n' | se 'x/[a-z]+at/ p'
# → cat / sat / mat
# y/re/ — yank: each view becomes the gaps *between* matches
printf 'a=1,b=2,c=3\n' | se 'y/,/ p'
# → a=1 / b=2 / c=3
# g/re/ and v/re/ — keep or drop views containing a match
printf 'keep\nskip\nkeep too\n' | se 'x/.*\n?/ v/skip/ p'
# → keep / keep too

Line records, since almost everything starts here:

se '/error/i' file.log        # bare form: filter lines matching /error/, case-insensitive
se 'x/.*\n?/ g/error/i p'
# the same thing, spelled out explicitly

A program that begins with / is sugar for the explicit form — se '/error/i' desugars to x/.*\n?/ g/error/i p and gets fused into the same fast path Part 2 described. Use x/.*\n?/ for ordinary lines (the ? keeps a final unterminated line); use x/.*\n/ when you want to drop one.

Control flow, once you're past line-filtering:

# N — merge adjacent view pairs
printf 'a\nb\nc\nd\n' | se 'x/[a-z]\n?/ N ='
# → 0,4,4  then  4,8,4  — two merged views, not four
# m/re/ { ... } — extract every match, run a block on each in order
printf 'width:10 height:20 depth:5\n' | se 'm/[a-z]+:[0-9]+/ { x/[0-9]+/ p }'
# → 10 / 20 / 5
# t { ... } — run a block only where the previous s/// actually changed the view
printf 'ERROR: disk\nok: fine\nERROR: mem\n' | se 'x/.*\n?/ { s/ERROR/MATCHED/ t { = } }'
# → 0,12,12 / 21,32,11   (byte ranges of only the two lines that changed)
# → MATCHED: disk / ok: fine / MATCHED: mem   (the mutated document, written after)

That's t used as a dry run: ask which lines would change before you commit to -i.

Structural navigation and the two selectors nothing else has:

se 'x/needle/ + p' file      # + : widen to the enclosing balanced bracket pair
se -- '- p' file             # - : descend into the first balanced bracket pair
# (needs -- so the flag parser doesn't read - as an option)

printf 'f({a, b}, c)\n' | se 'x/b/ + - p'
# → a, b   (expand to enclose "b", then collapse into what that opened)

se 'x/.*\n?/ ~2/kubernetes/ p' words.txt   # ~k : fuzzy match within edit distance k
se 'x/.*\n?/ :sem:/network/ p' log.txt     # :sem: : lexicon-based concept match

Substitution, with capture groups, and in-place editing:

printf 'lastname=Vatsraj firstname=Achintya\n' | se 's/(\w+)=(\w+)/$1:$2/g'
# → lastname:Vatsraj firstname:Achintya

se -i=.bak 's/false/true/g' app.conf
# app.conf is rewritten; app.conf.bak holds the original, byte for byte

The awk action, for arithmetic and reports after structural selection — covered in full in Part 2, one fresh example here:

printf 'a 1\nb 2\nc 3\n' | se 'x/.*\n?/ @{ print $1, $2*10 }'
# → a 10 / b 20 / c 30

That's the whole language, basic to power-user, in the order you'd actually learn it: filter lines, extract fields, branch on mutation, walk brackets, forgive typos, do arithmetic. Every layer composes with the one below it.

Part Three: Chaining se Into the Rest of the Toolbox

se deliberately does not walk directories — that's find's job, or rg's. It doesn't do JSON-aware querying — that's jq's job. It doesn't do frequency tables — that's sort | uniq -c. se does the structural extraction step and hands off clean text to whatever's next. Composability was the whole design, so here it is composed.

Bulk-processing a tree with find:

find . -name '*.rs' -print0 | xargs -0 se 'x/TODO\([a-z]+\)/ p'

On a small test tree this pulled every TODO(owner) tag out of three files in one shot: TODO(carol), TODO(alice), TODO(bob), TODO(alice).

.gitignore-aware file selection with rg --files -0, exactly as the se manual page recommends, since se itself won't read an ignore file:

rg --files -0 . | xargs -0 se 'x/TODO\([a-z]+\)/ p'

Same tree, but with a lib/ directory .gitignore'd — Carol's TODO in lib/c.rs correctly never shows up. se extracts; rg decides what's in scope.

A frequency table with sort | uniq -c:

find . -name '*.rs' -print0 |
  xargs -0 se 'x/TODO\(([a-z]+)\)/ x/[a-z]+/ p' |
  sort | uniq -c | sort -rn

2 alice
   1 carol
   1 bob

Two nested x/re/ extractions pull the name out from inside the parens; sort | uniq -c turns that stream into a leaderboard. se never tries to be sort.

Handing clean JSON to jq:

A mixed log — plaintext boot messages next to structured JSON lines — filtered down to valid JSON before jq ever sees it:

[2026-08-20T10:00:00] booting worker pool
{"ts":"2026-08-20T10:00:01","lvl":"info","msg":"listening on :8080"}
[2026-08-20T10:00:02] gc pause 4ms
{"ts":"2026-08-20T10:00:03","lvl":"error","msg":"db timeout","retries":2}
-- health check ping --
{"ts":"2026-08-20T10:00:04","lvl":"info","msg":"request served","ms":12}

se 'x/.*\n?/ g/^\{/ p' mixed.log | jq -c 'select(.lvl=="error")'

{"ts":"2026-08-20T10:00:03","lvl":"error","msg":"db timeout","retries":2}

g/^\{/ throws away every line that isn't structurally JSON before jq has to parse a single byte of the noise. se does the shape-filtering; jq does what it's actually good at.

That's the pattern for all of this: se is not trying to replace find, rg, sort, uniq, or jq. It replaces the part of the pipeline where you used to reach for a fragile awk state machine or three chained sed invocations just to get text into a shape another tool could use.

Conclusion

Three cookbook problems, done in one line each, next to the honest classic-tools version instead of a strawman. A paper from 1987 that explains why the classic version needed that much more code. A command language that goes from grep-equivalent to bracket-aware tree navigation without changing tools. And a toolbox se was built to sit inside, not replace.

se is BSD-3-Clause and lives at github.com/lolt3under/se-rs, published on crates.io. If you build something with a piece of it, link back.

Engineering is measurable. Measure it.

Jai Hind.

Wake up. Dig deeper.

#rust #development #research #systems #linux #macos #regex

curl -fsSL https://xer0x.in/blog/se-03/content.txt | openssl dgst -sha256 | grep -qi 010e837fe94f9ab414d267381b1ae2797bf3c537778e05eb1f8ea2fc89414329 && echo "signature ok" || echo "TAMPERED"

← Back to blog