Repository navigation
feat(aeo): implement Cloudflare Agent Readiness and Answer Engine Optimization (AEO) - #687
Merged
Merged
Conversation
…d AEO - Add Content-Signal and bot access control rules to robots.txt - Add /llms.txt and /llms-full.txt following llmstxt.org specification - Add RFC 9727 /.well-known/api-catalog endpoint - Add MCP server cards at /.well-known/mcp/server-card.json and /.well-known/mcp.json - Add Cloudflare Agent Skills RFC catalog at /.well-known/agent-skills/index.json - Add Streamable HTTP MCP JSON-RPC 2.0 endpoint at /api/mcp for search_content - Add edge HTML-to-Markdown transformer with Accept: text/markdown content negotiation and *.md route rewrites - Add RFC 8288 Link discovery headers in middleware and link tags in BaseLayout - Add PodcastEpisode Schema.org structured data and hidden agent token-saving directives
|
The latest updates on your projects. Learn more about Vercel for GitHub. 1 Skipped Deployment
|
⚡ Cloudflare Worker Preview Ready!
|
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Implements full compliance with Cloudflare's Agent Readiness specifications (blog.cloudflare.com/agent-readiness / isitagentready.com) across all 4 core dimensions, alongside comprehensive Answer Engine Optimization (AEO) for AI search and agent consumers (GPTBot, ClaudeBot, PerplexityBot, Applebot-Extended, etc.).
1. Discoverability
robots.txt: Standardized format pointing to/sitemap.xmland/sitemap-index.xml.LinkHTTP Response Headers: Injected via edge middleware for all routes:rel="api-catalog"->/.well-known/api-catalogrel="mcp-server-card"->/.well-known/mcp/server-card.jsonrel="agent-skills"->/.well-known/agent-skills/index.jsonrel="llms-txt"->/llms.txt<link rel="alternate" type="text/markdown">,<link rel="api-catalog">, and<link rel="mcp-server-card">in<head>.2. Content & Token Efficiency
Accept: text/markdownor.md//index.mdURL suffixes and transforms HTML into clean, token-dense Markdown using a zero-dependency edge parser (htmlToMarkdown)./llms.txt: Implemented following llmstxt.org specification with core site architecture, API catalog, MCP server, search endpoint, RSS feeds, and markdown negotiation notes./llms-full.txt: Dynamic SSR endpoint fetching the latest 100 blog posts and podcasts directly from Sanity, linking to their markdown endpoints with summaries.<body>directing AI crawlers to useAccept: text/markdownor append.mdto avoid wasting LLM context window tokens on HTML layout boilerplate.3. Bot Access Control
Content-Signal: ai-train=no, search=yes, ai-input=yesinrobots.txtto permit search indexing and agent consumption while asserting copyright against model pre-training.GPTBot,ClaudeBot,PerplexityBot, andApplebot-Extended.4. Capabilities & MCP Integration
/.well-known/api-catalogdetailing search, MCP, RSS, and sponsorship endpoints./.well-known/mcp/server-card.jsonand/.well-known/mcp.json./.well-known/agent-skills/index.json./api/mcpsupportinginitialize,notifications/initialized,tools/list, andtools/callforsearch_contentwired directly to Sanity search.5. Enhanced AEO / Structured Data
PodcastEpisodestructured data withAudioObject,PodcastSeries, season, and episode numbers to supplement existingArticleandBreadcrumbListschemas.Verification
pnpm --filter @codingcatdev/site lint: 67 files checked, 0 errors.pnpm --filter @codingcatdev/site typecheck: 134 files checked, 0 errors.pnpm --filter @codingcatdev/site test:e2e: All 145 dual-track E2E tests passing.pnpm --filter @codingcatdev/site build: Clean edge server build in 1.03s.