Link prefetching and security scanners

The problem

Something opens the links in your email before the recipient does. Corporate security products fetch every URL in an incoming message to scan it for malware, automatically, with no human action, often before the message even reaches the inbox. From your server's point of view that fetch is an ordinary GET request. It looks exactly like a real click.

For most links this is harmless. The scanner loads a page, finds nothing dangerous, and moves on. The trouble starts when a link is meant to be used exactly once: a magic sign-in link, an email verification link, a password reset, a one-click unsubscribe. The scanner's GET consumes the single-use token, and by the time the recipient clicks, the link reports itself expired or invalid.

It is one of the most reported and least understood failures in transactional email. The same symptom shows up identically across authentication libraries (next-auth, Better Auth, PocketBase, and Supabase all carry long threads on it), and it is usually blamed on the wrong thing first, a clock issue, a caching bug, a flaky user, before someone realizes a machine clicked the link before the human could.

Who is fetching your links

The clearest, best-documented culprit is Microsoft Defender for Office 365, through its Safe Links feature. Microsoft's own documentation states that URLs are "scanned prior to message delivery," and that URLs "that don't have a valid reputation are detonated asynchronously in the background." Detonation means Safe Links opens the URL in a sandbox to see what it does. On the strictest setting it will hold the message back until that scan completes.

The word to notice there is reputation. Safe Links does not necessarily fetch every link on every message; it detonates the ones it has no established reputation for. A one-time token URL is unique by construction. It has never been seen before and never will be again, so it has no reputation, which makes it precisely the kind of link that gets detonated. That is why this problem lands so squarely on auth and verification links and mostly spares ordinary marketing links to well-known domains.

Defender is not alone. The same GET-on-delivery behavior comes from other enterprise mail security products, including Mimecast, Proofpoint URL Defense, and Barracuda. Any recipient behind a corporate mail gateway may be behind one of these, so you cannot predict from your side whether a given send will be scanned.

Gmail is often named in the same breath, and here it pays to be precise. Gmail is documented to prefetch images under some conditions, which is one cause of inflated open rates, since it fires tracking pixels before the recipient opens the message. It is a much smaller cause than Apple's Mail Privacy Protection, which is covered in open and click tracking. Gmail fetching the links themselves is a separate claim again: developers have reported it since at least 2019, but Google does not document it as a feature, so treat it as observed behavior rather than a specified guarantee. The corporate scanners above are the part you can count on.

Why it breaks authentication

Walk the sequence for a magic sign-in link. Your user asks to log in. Your backend generates a one-time token, stores it, and emails a link like /verify?token=abc. The message passes through the recipient's mail security gateway, which GETs that link to scan it. If your verify endpoint consumes the token on GET, it is now spent. When the user finally clicks, the token is gone and they see an error.

The root cause is a GET request with a side effect. HTTP treats GET as safe: fetching a URL is not supposed to change anything. A verify or unsubscribe endpoint that mutates state on GET breaks that expectation, and automated fetchers are entirely within their rights to hit it.

// The vulnerable pattern: a GET that consumes the token on load.
// A scanner's fetch spends it before the recipient ever clicks.
app.get("/verify", async (req, res) => {
  const { token } = req.query;
  await consumeToken(token); // one-time: marks the token used
  await signIn(res, token);
  res.redirect("/dashboard");
});

How to fix it

There is no reliable way to tell a scanner's GET apart from a real click at the moment it arrives, so every workable fix does one of two things: it takes the side effect off the GET, or it lets the token survive being fetched. Pick based on how much friction you can accept.

• Put a confirmation page in front of the action (the strongest fix). The link in the email stays an ordinary GET, because a link is all an email can send. What changes is where it lands. Instead of pointing at an endpoint that spends the token, it opens a plain page with a button on it. The reader presses the button, that page POSTs, and only then is the token consumed. A scanner fetches the page, finds a form it will not fill in, and stops. The button belongs on the web page, not in the message: HTML forms in an email body are not a shortcut here, since support is patchy and several clients strip the form or quietly refuse to submit it.
  // GET only renders a confirmation page. The scanner lands here and stops.
  app.get("/verify", (req, res) => {
    res.send(confirmPage(req.query.token)); // a page with a form that POSTs back
  });
  
  // The token is consumed only on POST, which scanners don't send.
  app.post("/verify", async (req, res) => {
    await consumeToken(req.body.token);
    await signIn(res, req.body.token);
    res.redirect("/dashboard");
  });

• Keep the token out of the server's reach. Put the token in the URL fragment instead of the query string, as in /verify#token=abc. Browsers never send the fragment to the server, so no scanner GET can carry it. A small script on the landing page reads the fragment and submits it only after the user acts. This keeps a one-click feel while making the token structurally invisible to a plain fetch.
• Let the token survive the scan. Instead of deleting the token on first use, allow a small number of uses within a short window and make the action idempotent, so a scanner's fetch and the user's real click both succeed. Better Auth ships exactly this as its allowedAttempts option. The trade-off is a slightly weaker one-time guarantee, which you offset with a short expiry.
• Delay activation. Make the token invalid for the first few seconds after it is issued. Scanners usually hit within milliseconds of delivery, while a human takes longer to open the message and click, so the early automated fetch fails and the later real click works. This is a heuristic, not a guarantee: it adds latency to a genuinely fast click and can misfire when delivery is slow, so treat it as a supplement rather than your only defense.
• Detect the scanner. Some auth providers fingerprint the request to tell a scanner from a browser and simply decline to consume the token for a suspected scanner. Stytch offers this as a managed feature (its "protected" magic links). Doing it yourself reliably is hard, because scanners increasingly render links in a full browser precisely to look like real users.

The general rule

Any GET with a side effect is exposed to this, not just authentication. The clearest public example is from 2019, when the team behind Healthchecks.io found Microsoft's Safe Links crawler firing their GET-based unsubscribe links and unsubscribing real users who had clicked nothing. The fix was the same one RFC 8058 later standardized: do the real work on a POST. If a request can change state, do not let an unauthenticated GET be the thing that triggers it.

Sources

• Microsoft Learn: Complete Safe Links overview for Defender for Office 365 - URLs are scanned prior to delivery; no-reputation URLs are detonated
• next-auth: Antivirus software that scans emails eats the magic link token
• Better Auth: magic link token consumed by email scanners (allowedAttempts)
• PocketBase: OTP magic link being consumed by email link scanner
• Supabase: tokens consumed by email scanners, the URL-fragment fix
• Stytch: Protected Email Magic Links
• Litmus: how Gmail prefetching of images skews open data - documents image prefetch, not link fetching
• Healthchecks.io: preventing Office 365 ATP from clicking unsubscribe links
• RFC 8058: Signaling One-Click Functionality for List Email Headers
• Can I email: HTML form element - around half of clients; ProtonMail strips forms, macOS Outlook will not submit