OpenBSD spamd: a multiline reply bug that turned into a heap overflow

This is a follow-up to my previous OpenBSD work on ftpd’s write() error-path UB (fixed upstream in openbsd/src@a904405). I took that blog post and fed it back into my agent as a “reference for good reasoning,” then changed the scope: this time, only hunt heap overflows.

Here’s the prompt I used:

Only focus on finding heap buffer overflows.
Target: network-facing daemons in the OpenBSD src tree.
Prioritize: long-lived code and config-driven formatting/parsing.
Output: exact file/function, why it’s reachable, and a safe ASan confirmation plan.

That prompt worked because it points the searchlight at a specific class of failure: hand-rolled string building where an “index” (i) and a “length” (len) must stay consistent. In the ftpd case, the invariant was “only advance on success.” For heap overflows, the invariant is “advance by what you appended, and always pass the remaining capacity to functions like snprintf().”

What spamd is doing here

spamd is the IP-based tarpit/greylisting daemon. When a connection matches a blacklist, it generates SMTP reply text using the list’s configured msg. The message supports escape sequences like \\n for newlines and %A for the connecting IP (see spamd(8) and spamd.conf(5)).

Multiline messages and escapes are part of the supported configuration (spamd.conf(5)). The code that renders the configured message into SMTP replies lives in append_error_string() in libexec/spamd/spamd.c. It writes into a growable output buffer and tracks the current end using an index (i).

How I found it

• Start with a simple invariant: if you append at c + i, you advance by the length of what you appended (often strlen(c + i)).
• Search for manual “format string” loops in daemons that keep their own i/len accounting.
• In append_error_string(), notice the newline-continuation path appends at c + i but advances with strlen(c).
• Confirm it’s not a dead feature: spamd.conf(5) documents \\n in msg.
• Confirm quickly with AddressSanitizer using a local harness (no remote exploitation needed).

The bug

Heap buffer overflow in append_error_string() when handling a newline continuation

The bug is small but nasty: when spamd sees a newline in the configured message, it starts a new SMTP reply line by appending a fresh reply prefix (like 450). That append is done at c + i. The mistake is how it updates i afterwards.

// When the current output ends with '\n', spamd starts a new reply line.
if (c[i - 1] == '\n') {
  snprintf(c + i, len, "%s ", nreply);
  i += strlen(c);   // BUG: should be strlen(c + i)
}

strlen(c) measures the string from the beginning of the buffer, not from the place you just appended (c + i). So i can jump forward too far. After that, the function continues copying characters with c[i++] = ..., and those writes can go past the end of the heap buffer.

Two extra details made this easier to miss in review:

• The function mixes int and size_t for indices/lengths (inviting underflow/overflow corner cases).
• It recomputed len as “remaining length” in one branch, but treated it as “total length” elsewhere.

Why it’s reachable

Multiline blacklist messages are explicitly supported. The spamd.conf(5) example includes a msg string with a newline escape (\\n) to wrap the text.

In practice, the message comes from the blacklist database generated by spamd-setup(8). When an SMTP session reaches the reject/tarpit path, doreply() calls append_error_string() to build the multi-line response.

How I confirmed it (ASan)

I confirmed it with a small local harness that calls the function with a long multiline format string. Compiled with AddressSanitizer (and UBSan), it crashes reliably with a precise file/line reference (AddressSanitizer: clang.llvm.org).

clang -O0 -g \
  -fsanitize=address,undefined -fno-omit-frame-pointer \
  spamd-asan-harness.c -o spamd-asan-harness

ASAN_SYMBOLIZER_PATH="$(command -v llvm-symbolizer || xcrun -f llvm-symbolizer 2>/dev/null)" \
ASAN_OPTIONS=symbolize=1 \
./spamd-asan-harness

The ASAN_SYMBOLIZER_PATH=… part is what makes the stack trace resolve to file names and line numbers. There’s no hardcoded path here: it first uses command -v llvm-symbolizer (if you have LLVM installed) and otherwise asks the system toolchain via xcrun.

Verified against the upstream commits

To make sure the harness matches reality, I verified the exact buggy lines in the parent of the fix commit, and the corrected lines in the fix itself. Run these from an OpenBSD src checkout:

# Vulnerable version (parent of the fix)
git show d6ec741d09d60a0d7dacf58ab76f838389bb1659^:libexec/spamd/spamd.c \
  | nl -ba | sed -n '470,515p'

# Fixed version (the fix commit)
git show d6ec741d09d60a0d7dacf58ab76f838389bb1659:libexec/spamd/spamd.c \
  | nl -ba | sed -n '470,515p'

==70730==ERROR: AddressSanitizer: heap-buffer-overflow
WRITE of size 1
    #0 append_error_string ... libexec/spamd/spamd.c
SUMMARY: AddressSanitizer: heap-buffer-overflow in append_error_string

With that trace in hand, I sent a short report to the OpenBSD security team including the analysis, the sanitizer trace, and a minimal patch (OpenBSD security: openbsd.org).

The upstream fix

Todd C. Miller agreed with the analysis and committed a fix with a small but important improvement: make the index and continuation marker size_t, and tighten the length checks to avoid int-vs-size_t mismatches.

I agree with your analysis. I made a slightly different change
that corrects some of the int vs. size_t mismatch to avoid casts.

- todd

The key changes:

• lastcont: int → size_t
• i: int → size_t
• Guard: if (len < 46 || i >= len - 46) (avoids len - 46 underflow)
• Append with remaining length: snprintf(c + i, len - i, ...)
• Fix the cursor advance logic in the newline continuation path

Impact

This is a memory corruption bug in a network daemon. It’s configuration-dependent (you need a multiline blacklist message), but when the path is active it can make spamd crash while generating an SMTP reply. In most real deployments the practical outcome is a denial-of-service (a crash), but it’s still the kind of bug you want fixed promptly.

Closing

The lesson for me wasn’t “AI found a bug.” It was that a narrow prompt plus a good reference (the ftpd post) steered the agent toward the right invariants — and sanitizers did the rest by turning a hunch into a concrete, line-numbered crash.

Takeaway

This bug was in everyday C string code. A narrow prompt helped us look in the right place, and ASan proved it with a clean crash.

OpenBSD and spamd are mentioned here for technical discussion; this site is not affiliated with the OpenBSD project.