ai × security

OpenBSD ftpd: a 29-year-old bug (almost 30)

April 19, 2026 · OpenBSD · ftpd · C · undefined behavior

I was curious about how Mythos — the frontier model behind Anthropic's Project Glasswing — was finding vulnerabilities. I was reading tweets and threads and trying to understand the workflow behind it. One thing that caught my eye was the mention of a 27-year-old vulnerability in OpenBSD.

That made me wonder: can we do something like that today with a current model? Why not try.

Setup

I used Codex 5.2 (medium reasoning) with a local RAG corpus. The RAG included Linux kernel vulnerability details from 1998 and above, and notes on C/C++ language rules and undefined behavior. I cloned the repo locally and worked from there.

Prompts

Prompt 1 — Scope + constraints

We're doing fully authorized offensive security research (lab-only) on the OpenBSD src tree.
Goal: find one novel, high-severity, true-positive vulnerability at a time, with a realistic PoC plan.
Focus on default settings; ignore intended/documented behavior and pure DoS-only issues.

Prompt 2 — History-driven review

Review the repository history (commit logs) specifically for security fixes.
Summarize what was fixed, why it was exploitable, and what patterns/themes show up across fixes.
Use this to choose high-risk areas to audit next.

Prompt 3 — Patch-adjacent hunting (old bugs)

For each security fix, "reverse" it to understand the original bug, then look for adjacent/incomplete fixes or alternate paths that still trigger the same class of issue.
Prioritize vulnerabilities likely to have existed for decades in privileged/network-facing code. Return a short, evidence-backed shortlist with file/func references and a PoC outline per candidate.

Result

Codex 5.2 ran for about 29 minutes and produced a concrete finding: undefined behavior in OpenBSD's ftpd in the mmap "fast path" in send_data(), reachable when the data connection fails mid-transfer.

cnt = write(netfd, bp, len);
if (recvurg) { munmap(buf, fsize); goto got_oob; }
len -= cnt;   /* wrong when cnt == -1 */
bp  += cnt;   /* wrong when cnt == -1 */
if (cnt > 0)
  byte_count += cnt;
} while (cnt > 0 && len > 0);

The issue is the unconditional state update after write(). When write() returns -1, the code still runs len -= cnt and bp += cnt, which produces undefined behavior (pointer arithmetic before the mapped region) and an incorrect length.

How the reasoning got there

This is the model's own walkthrough of how it reached the bug — preserved verbatim:

AI reasoning · Codex 5.2

I started from the FTP command surface, because that's the most reliable way to ensure I'm looking at attacker-reachable code. In libexec/ftpd/ftpcmd.y, every FTP verb is wired to a handler, and RETR is the canonical "send file contents to the client" operation, so I traced RETR into retrieve().

Once inside retrieve() (libexec/ftpd/ftpd.c:1111), the structure is straightforward: it opens the requested file, sets up the data channel with dataconn(), and then hands control to send_data() (libexec/ftpd/ftpd.c:1168) to actually push bytes over the data connection. That tells you exactly where the "bytes go out over a socket" logic lives, so send_data() is the next place to scrutinize.

In send_data() (libexec/ftpd/ftpd.c:1499), I was specifically looking for tight loops around read()/write(), because a huge fraction of low-level C bugs show up in the details of how code handles partial writes, errors, and the bookkeeping around pointers and remaining lengths. Mentally, I'm scanning for patterns like "call write(), then advance a pointer and subtract the number of bytes written" — the classic shape is cnt = write(...); len -= cnt; p += cnt;.

The mmap "fast path" immediately matched that shape: it mmap()s the file, sets bp = buf and len = fsize, then loops calling write(netfd, bp, len) (libexec/ftpd/ftpd.c:1556). At that point the key reasoning step is simply remembering the full contract of write(): it can return a positive byte count, it can return 0 in some edge cases, and it can return -1 on error. If a loop uses cnt to update both a pointer (bp) and a remaining-length counter (len), those updates must only happen when cnt > 0. Otherwise you risk moving the pointer backwards or producing a bogus remaining length.

Here, the updates happen unconditionally: len -= cnt; bp += cnt; at libexec/ftpd/ftpd.c:1561-1562. So if write() returns -1, the code computes bp += -1, which is pointer arithmetic that produces a pointer before the start of the mmap'd region — that's undefined behavior in C. In parallel, len is a size_t, so subtracting -1 effectively wraps and produces a nonsense large value, which is a correctness bug that reinforces that the error case wasn't handled properly.

Finally, I validated that I wasn't accidentally inventing the scenario by double-checking the local types (len is size_t, cnt is an int in this file) and using git blame on the exact lines to see that the logic really has been there a long time and isn't some transient diff artifact.

Fix

OpenBSD ftpd: Pointer Underflow UB in send_data() mmap Fast Path on write() Error

The fix was to only update len and bp when cnt > 0:

if (cnt > 0) {
  len -= (size_t)cnt;
  bp  += cnt;
  byte_count += cnt;
}

We reported this to OpenBSD and it was fixed.

The fix, on the record

openbsd/src a904405
Avoid potential undefined behavior on write error while sending data.
From Dhiraj Mishra · OK deraadt@ tb@ · libexec/ftpd/ftpd.c · +6 −5 View on GitHub →

How old was it?

The pattern dates back to 1996-07-28 (OpenBSD commit 3aba417a95d2) and the fix landed on 2026-04-17. That's 29 years, 8 months, 20 days in the tree — almost 30 years. There is no advisory since there is no crash unless ftpd is built with UBSan but a good start to dig more.

What I took away

Model quality matters, but context is king. The prompt, the references, and the time spent reasoning are what shaped the result.

Thanks

On a personal note, I want to thank my current employer, Deriv, for the enterprise access to the tooling that made an experiment like this possible on a weekend. I'm grateful to work somewhere that takes both security and AI seriously enough to invest in the gap between them.


OpenBSD and ftpd are mentioned here for technical discussion; this site is not affiliated with the OpenBSD project.