Making Our DocsAgent-Readable

What an agent actually gets from a docs page

Someone points an agent at wraps.dev/docs/quickstart/email and asks it to send an email. The agent fetches the URL and gets back a Next.js document: a nav tree, a sidebar, a footer, a command palette, three hydration payloads, and—somewhere in there—four commands it actually needed. The signal is present. It is just buried in markup that exists for human eyes and a mouse.

We already had a partial answer. A llms.txt and a llms-full.txt have been live on wraps.dev since February, following the community convention of the same name. The first is a 185-line index of what Wraps is and where things live. The second is the whole corpus in one 920-line file, and llms.txt tells agents to fetch it once instead of crawling.

That is the right shape for one job: an agent that wants to learn the product from scratch. It is the wrong shape for the far more common one. An agent that landed on a single deep link does not want 920 lines to scan. It wants that page. Today we shipped the layer that gives it that page—plus the discovery documents that let a machine find any of it without parsing HTML at all.

Ask for markdown, get markdown

There is no new URL scheme here, no .md suffix to remember, no separate agent subdomain. The URL an agent already has is the URL that works. It just has to say what it wants in the Accept header, which is what content negotiation has been for since HTTP/1.1.

$ curl -H "Accept: text/markdown" https://wraps.dev/docs/quickstart/email

HTTP/2 200
content-type: text/markdown; charset=utf-8
vary: Accept
cache-control: public, max-age=3600

# Email Quickstart

Deploy production-ready email infrastructure to your AWS account in under 2 minutes.

## What You'll Build

- AWS SES with DKIM, SPF, and DMARC configured automatically
- A verified sending domain in your AWS account
- Your first email sent via the TypeScript SDK

The same URL with a browser's default Accept returns the full HTML page, unchanged. Nothing about the human experience moved.

The whole mechanism is a middleware that inspects one header and rewrites:

export async function middleware(request: NextRequest) {
  const accept = request.headers.get("accept") ?? "";
  if (!accept.includes("text/markdown")) {
    return NextResponse.next();
  }

  const { pathname } = request.nextUrl;
  // Route the request to /api/md/<path> so dynamic route params carry the page path
  const mdPath = pathname === "/" ? "/api/md/root" : `/api/md${pathname}`;
  const mdUrl = new URL(mdPath, request.nextUrl.origin);

  return NextResponse.rewrite(mdUrl);
}

A rewrite, not a redirect. The agent's URL bar—or whatever an agent has instead of one—never changes, and it gets a 200 on the URL it asked for. On the other end is a catch-all route handler that does the lookup:

const MD_HEADERS = {
  "Content-Type": "text/markdown; charset=utf-8",
  Vary: "Accept",
  "Cache-Control": "public, max-age=3600",
} as const;

export async function GET(
  _request: NextRequest,
  { params }: { params: Promise<{ path: string[] }> }
) {
  const { path } = await params;
  // "root" is a sentinel for "/" since Next.js dynamic routes can't match empty segments
  const pagePath =
    path[0] === "root" && path.length === 1 ? "/" : `/${path.join("/")}`;
  const content = AGENT_CONTENT[pagePath] ?? getLlmsFallback();

  return new NextResponse(content, { headers: MD_HEADERS });
}

Two details worth calling out. Vary: Accept is not optional—without it, any cache between us and the agent is free to hand the markdown body to the next browser that asks for the same URL. And root is a sentinel, because a Next.js catch-all segment cannot match the empty path that / produces. Unknown paths fall through to llms.txt read off disk, and if even that read fails, to the homepage entry. An agent asking for a page we have not written markdown for still gets something oriented rather than an error.

The first cut returned the site index

That is the second version. The first one shipped a few hours earlier the same morning and was wrong in an instructive way:

// Serve llms.txt as markdown for any page request from an AI agent
const llmsUrl = new URL("/llms.txt", request.nextUrl.origin);
const res = await fetch(llmsUrl, { next: { revalidate: 3600 } });

if (!res.ok) {
  return NextResponse.next();
}

const body = await res.text();

return new NextResponse(body, {
  headers: {
    "Content-Type": "text/markdown; charset=utf-8",
    Vary: "Accept",
    "Cache-Control": "public, max-age=3600",
  },
});

Content negotiation: correct. Headers: correct. Behavior: an agent that asked for /docs/guides/webhooks got the site index. Every page returned the same body. We had built the plumbing for per-page markdown and then piped every page to the one file we already had.

Eleven pages, written by hand

The corpus behind /api/md is deliberately unclever. It is one exported object:

export const AGENT_CONTENT: Record<string, string> = {
  "/": `# Wraps
  ...`,
  "/docs/quickstart/email": `# Email Quickstart
  ...`,
};

Route path in, markdown string out. No build step, no MDX, no renderer that walks the TSX pages and tries to reconstruct prose from JSX. Our docs are React components—there is no MDX pipeline on this site—so there is no source of truth to extract from. That means this is a hand-maintained parallel corpus, with everything that implies. It is the honest description and the obvious cost.

Eleven routes have hand-written markdown today, chosen because they are what an agent doing real work actually lands on:

Eleven, out of a site with more than a hundred and fifty page routes. Everything else still resolves—to llms.txt, via the fallback—but it is not page-scoped, and pretending otherwise would be the kind of claim that costs you a reader the first time they check.

How a machine finds any of this

Markdown that nobody knows about is not discoverable, it is just polite. The rest of this ship is three documents whose entire job is to be found by something that has never seen wraps.dev before and cannot read a nav bar.

A Link header on the homepage

RFC 8288 — Web Linking — lets a response advertise related resources in headers, no body parsing required. Fetch the homepage and the answer is in the response head:

$ curl -D- https://wraps.dev/

link: </docs>; rel="service-doc", </.well-known/api-catalog>; rel="api-catalog"

Two IANA-registered relation types. service-doc is documentation intended for humans; service-desc is its machine-readable sibling, and the distinction matters in a moment. This header is configured on source: "/" in next.config.ts, which means it is on the homepage and only the homepage. Not every page advertises. A crawler that starts at the root finds it; one that starts at a deep link does not.

An API catalog at a well-known URI

RFC 9727 defines /.well-known/api-catalog as the place to publish what APIs an organization has and where their descriptions live. It went to Proposed Standard in June 2025, and it requires the application/linkset+json media type from RFC 9264. Ours is a static route handler:

export function GET() {
  const catalog = {
    linkset: [
      {
        anchor: "https://api.wraps.dev",
        "service-desc": [
          {
            href: "https://api.wraps.dev/swagger/json",
            type: "application/openapi+json",
          },
        ],
        "service-doc": [{ href: "https://wraps.dev/docs" }],
        status: [{ href: "https://api.wraps.dev/health" }],
      },
    ],
  };

  return Response.json(catalog, {
    headers: { "Content-Type": "application/linkset+json" },
  });
}

Three links, three audiences. A machine that wants to call the API follows service-desc to the OpenAPI document and can generate a client from it. A model that wants to explain the API to a human follows service-doc to the docs. Anything that wants to know whether we are up follows status to the health endpoint. The anchor is api.wraps.dev while the document is served from the marketing origin, which is exactly the split RFC 9727 is designed for: the catalog describes the API, it does not have to live on it.

OAuth metadata, so a client can configure itself

A CLI has no browser. An agent has no safe place to hold a password. Both need to authenticate anyway, and the answer since 2019 has been the OAuth 2.0 Device Authorization Grant (RFC 8628): the client asks for a short user code, prints it, a human approves it in a browser on any device, and the client polls until a token comes back. No secret is ever typed into the client or stored by it.

What we added is the part that makes it self-configuring. Instead of a client hardcoding our endpoints, it fetches one document and learns them:

export function GET() {
  const metadata = {
    issuer: "https://api.wraps.dev",
    device_authorization_endpoint:
      "https://app.wraps.dev/api/auth/device/code",
    token_endpoint: "https://app.wraps.dev/api/auth/device/token",
    grant_types_supported: [
      "urn:ietf:params:oauth:grant-type:device_code",
    ],
    token_endpoint_auth_methods_supported: ["none"],
    service_documentation: "https://wraps.dev/docs",
  };

  return Response.json(metadata);
}

Four facts in one fetch: the device grant is supported at all, where to start it, where to exchange the code, and—from token_endpoint_auth_methods_supported: ["none"]—that this is a public client with no secret to leak. The endpoints it advertises are real, backed by better-auth's device authorization plugin behind app.wraps.dev. The same document is served from both wraps.dev and api.wraps.dev, where the API copy carries an OpenAPI description and is explicitly marked unauthenticated—discovery has to be public or it is not discovery.

Three tools for a browser API that does not exist yet

Everything above assumes the agent is fetching URLs. A different case is the agent that is already driving a browser tab, looking at our page the way a person would. WebMCP is a proposal for that case: a page calls navigator.modelContext.provideContext() and hands the browser a set of named tools, so the agent can call a function instead of guessing which button to click.

wraps.dev now registers three, site-wide, from a component mounted in the root layout:

export function WebMCP() {
  useEffect(() => {
    if (!navigator.modelContext) return;

    const cleanup = navigator.modelContext.provideContext({
      name: "Wraps",
      description:
        "Deploy email (AWS SES), SMS, and CDN infrastructure to your AWS account with one command. Full ownership, AWS pricing, no credentials stored.",
      tools: [
        {
          name: "get_pricing",
          description: "Get Wraps pricing plans and feature comparison",
          inputSchema: { type: "object", properties: {} },
          execute: async () => {
            const res = await fetch("/pricing.md");
            return res.ok ? res.text() : { error: "unavailable" };
          },
        },
        // get_quickstart, search_docs
      ],
    });

    return cleanup;
  }, []);

  return null;
}

The component renders null; it is a pure side effect. Line three is the whole compatibility story: modelContext is optional on the Navigator type and guarded at runtime, so in a browser without the API this code returns immediately and does nothing at all. The effect returns the cleanup function the API hands back, which unregisters the tools on unmount.

Two of the three names promise more than the implementations deliver, and we would rather say so here than have you find out by calling them. search_docs does not search—it takes no query and returns the entire llms-full.txt. get_quickstart declares a service enum of email, sms, and cdn, then ignores it and returns llms.txt for every value. Only get_pricing does exactly what its name says. Wiring the other two to the per-page markdown route is the obvious next move.

Saying out loud what crawlers may do

The last piece is the one that goes the other direction. If you are going to make your docs easy for models to read, you should be specific about which uses you want.

This cost us a file. Next.js generates robots.txt from a MetadataRoute.Robots export, and that API has no way to emit an arbitrary line. So the metadata export and the old static public/robots.txt both got deleted and replaced with a plain route handler that returns a string:

# Wraps - Email Infrastructure for Developers
# https://wraps.dev

# Content Signals (https://contentsignals.org/)
Content-Signal: ai-train=no, search=yes, ai-input=yes

User-agent: *
Allow: /
Disallow: /api/
Disallow: /ingest/
Disallow: /_next/

Sitemap: https://wraps.dev/sitemap.xml

Three declarations, and the stance is coherent: ai-train=no — do not train or fine-tune on this. search=yes — index it and link to it. ai-input=yes — read it at answer time and ground a response in it. Docs exist to be read when someone has a question. That is precisely the use we want, and it is a different use from becoming training weights.

Content Signals is a Cloudflare proposal from September 2025, released under CC0. It is not an IETF or W3C standard, there is no RFC, and the IETF working group on AI preferences has not published one either. It is also not enforced by anything: it is a request, and compliance is entirely voluntary. We publish it because stating the preference costs one line and not stating it means nobody can honor it even if they want to.

Which of these are actually standards

This corner of the web is full of things that look like specs. Some of them are. Sorted from most settled to least:

The bottom half of that list is why this whole ship is best described as cheap insurance rather than a strategy. Five ratified RFCs and four bets.

What this doesn't do

Being explicit about the surface area:

Try it

Everything here is public and unauthenticated. Nothing to install.

# a page as markdown instead of HTML
curl -H "Accept: text/markdown" https://wraps.dev/docs/quickstart/email

# the same for the agent quickstart
curl -H "Accept: text/markdown" https://wraps.dev/docs/quickstart/email/agents

# what links the homepage advertises
curl -sD- -o /dev/null https://wraps.dev/ | grep -i '^link:'

# the API catalog and the OAuth metadata
curl https://wraps.dev/.well-known/api-catalog
curl https://api.wraps.dev/.well-known/oauth-authorization-server

If you run a docs site, the middleware in this post is about fifteen lines and the hard part is the writing. That ratio is the whole point.