From 11531afbcf6d200d825e156aa5d2cbaf13969d4c Mon Sep 17 00:00:00 2001 From: Bruce Hamilton Date: Mon, 8 Jun 2026 22:27:55 -0700 Subject: [PATCH 01/36] minor edit to add doc for commenting Signed-off-by: Bruce Hamilton --- docs/analysis/templates/analysis.md | 3 +++ 1 file changed, 3 insertions(+) diff --git a/docs/analysis/templates/analysis.md b/docs/analysis/templates/analysis.md index 2551255f..2da47531 100644 --- a/docs/analysis/templates/analysis.md +++ b/docs/analysis/templates/analysis.md @@ -23,6 +23,9 @@ TO USE THIS TEMPLATE, search and replace the named IDs: - `_PROJECT-DOC-REPO_`: repository where the project technical documentation is stored; this might be its own repo or a directory in the project main repo +Replace link placeholders in brackets, such as [_PROJECT_][project-website], +with the actual link surrounded by parenthesis. + For the analysis procedure, see [Analysis how-to](../howto.md). > Note: delete this "About this template" section after you have customized this From 30e402c237653529eecc8ff7ba104f7b923c9373 Mon Sep 17 00:00:00 2001 From: Bruce Hamilton Date: Wed, 10 Jun 2026 13:30:56 -0700 Subject: [PATCH 02/36] added AI section Signed-off-by: Bruce Hamilton --- docs/analysis/criteria.md | 21 +++++++++++++++++++++ 1 file changed, 21 insertions(+) diff --git a/docs/analysis/criteria.md b/docs/analysis/criteria.md index 9045bb01..ea2bc6bb 100644 --- a/docs/analysis/criteria.md +++ b/docs/analysis/criteria.md @@ -126,6 +126,27 @@ We evaluate on the following: [Inclusive Naming Initiative](https://inclusivenaming.org) website? - Does the project use language like "simple", "easy", etc.? +### AI Optimizaiton and Discoverability + +Configurations for AI agents to optimizie the understanding of content are +emerging and should be utilized. + +We evaluate on the following: + +- Is there an llms.txt file in the root of the website or documentation + repository? + + llms.txt is a proposed open standard for a Markdown file that websites can + host at the root (e.g., https://example.com/llms.txt). It helps large language + models (LLMs) and AI agents quickly understand a site's purpose, key content, + and where to find reliable information—similar to how robots.txt guides + crawlers or sitemap.xml lists pages, but optimized for AI inference (usage at + query time) rather than broad indexing. + + See https://llmstxt.org/ + + Task can be accomplished manually or by various tools and plugins. + ## Contributor documentation ### Communication methods documented From 2edb12ae2022409e5297d55e3fa69d3f84daaf0d Mon Sep 17 00:00:00 2001 From: Bruce Hamilton Date: Wed, 10 Jun 2026 13:35:10 -0700 Subject: [PATCH 03/36] spelling fix Signed-off-by: Bruce Hamilton --- docs/analysis/criteria.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/analysis/criteria.md b/docs/analysis/criteria.md index ea2bc6bb..eb6dc855 100644 --- a/docs/analysis/criteria.md +++ b/docs/analysis/criteria.md @@ -126,7 +126,7 @@ We evaluate on the following: [Inclusive Naming Initiative](https://inclusivenaming.org) website? - Does the project use language like "simple", "easy", etc.? -### AI Optimizaiton and Discoverability +### AI Optimization and Discoverability Configurations for AI agents to optimizie the understanding of content are emerging and should be utilized. From bc2b86dc950ab3e6af1bb52bcd5442cf301907e8 Mon Sep 17 00:00:00 2001 From: Bruce Hamilton Date: Wed, 10 Jun 2026 13:40:52 -0700 Subject: [PATCH 04/36] removed unneeded example link Signed-off-by: Bruce Hamilton --- docs/analysis/criteria.md | 10 +++++----- 1 file changed, 5 insertions(+), 5 deletions(-) diff --git a/docs/analysis/criteria.md b/docs/analysis/criteria.md index eb6dc855..502f4b33 100644 --- a/docs/analysis/criteria.md +++ b/docs/analysis/criteria.md @@ -137,11 +137,11 @@ We evaluate on the following: repository? llms.txt is a proposed open standard for a Markdown file that websites can - host at the root (e.g., https://example.com/llms.txt). It helps large language - models (LLMs) and AI agents quickly understand a site's purpose, key content, - and where to find reliable information—similar to how robots.txt guides - crawlers or sitemap.xml lists pages, but optimized for AI inference (usage at - query time) rather than broad indexing. + host at the root. It helps large language models (LLMs) and AI agents quickly + understand a site's purpose, key content, and where to find reliable + information—similar to how robots.txt guides crawlers or sitemap.xml lists + pages, but optimized for AI inference (usage at query time) rather than broad + indexing. See https://llmstxt.org/ From be4fbffe8f0c84d6e96ece7ece0085a1141b79ea Mon Sep 17 00:00:00 2001 From: Bruce Hamilton Date: Wed, 10 Jun 2026 13:47:42 -0700 Subject: [PATCH 05/36] spelling - fomratting fixes Signed-off-by: Bruce Hamilton --- docs/analysis/criteria.md | 6 ++---- 1 file changed, 2 insertions(+), 4 deletions(-) diff --git a/docs/analysis/criteria.md b/docs/analysis/criteria.md index 502f4b33..d4674aaa 100644 --- a/docs/analysis/criteria.md +++ b/docs/analysis/criteria.md @@ -128,7 +128,7 @@ We evaluate on the following: ### AI Optimization and Discoverability -Configurations for AI agents to optimizie the understanding of content are +Configurations for AI agents to optimize the understanding of content are emerging and should be utilized. We evaluate on the following: @@ -139,9 +139,7 @@ We evaluate on the following: llms.txt is a proposed open standard for a Markdown file that websites can host at the root. It helps large language models (LLMs) and AI agents quickly understand a site's purpose, key content, and where to find reliable - information—similar to how robots.txt guides crawlers or sitemap.xml lists - pages, but optimized for AI inference (usage at query time) rather than broad - indexing. + information. See https://llmstxt.org/ From c754d478f3beb406a3964dccd4a4d43b1685b644 Mon Sep 17 00:00:00 2001 From: Bruce Hamilton Date: Wed, 10 Jun 2026 13:53:37 -0700 Subject: [PATCH 06/36] put llms in code to fix spelling Signed-off-by: Bruce Hamilton --- docs/analysis/criteria.md | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/docs/analysis/criteria.md b/docs/analysis/criteria.md index d4674aaa..d5de7696 100644 --- a/docs/analysis/criteria.md +++ b/docs/analysis/criteria.md @@ -133,10 +133,10 @@ emerging and should be utilized. We evaluate on the following: -- Is there an llms.txt file in the root of the website or documentation +- Is there an `llms.txt` file in the root of the website or documentation repository? - llms.txt is a proposed open standard for a Markdown file that websites can + `llms.txt` is a proposed open standard for a Markdown file that websites can host at the root. It helps large language models (LLMs) and AI agents quickly understand a site's purpose, key content, and where to find reliable information. From dc567e2dd653f198919dfc75cafd08b4a7ddfbb8 Mon Sep 17 00:00:00 2001 From: Bruce Hamilton Date: Wed, 10 Jun 2026 14:02:39 -0700 Subject: [PATCH 07/36] trying quotes Signed-off-by: Bruce Hamilton --- docs/analysis/criteria.md | 14 +++++++------- 1 file changed, 7 insertions(+), 7 deletions(-) diff --git a/docs/analysis/criteria.md b/docs/analysis/criteria.md index d5de7696..00a6def8 100644 --- a/docs/analysis/criteria.md +++ b/docs/analysis/criteria.md @@ -133,17 +133,17 @@ emerging and should be utilized. We evaluate on the following: -- Is there an `llms.txt` file in the root of the website or documentation +- Is there an "llms.txt" file in the root of the website or documentation repository? - `llms.txt` is a proposed open standard for a Markdown file that websites can - host at the root. It helps large language models (LLMs) and AI agents quickly - understand a site's purpose, key content, and where to find reliable - information. + The "llms.txt" file is a proposed open standard for a Markdown file that + websites can host at the root. It helps large language models (LLMs) and AI + agents quickly understand a site's purpose, key content, and where to find + reliable information. - See https://llmstxt.org/ + For more informatino see https://llmstxt.org/ - Task can be accomplished manually or by various tools and plugins. + Task can be accomplished manually or by tools. ## Contributor documentation From ae6acf801a6bce2e844e895554973b4afb33163d Mon Sep 17 00:00:00 2001 From: Bruce Hamilton Date: Wed, 10 Jun 2026 14:12:34 -0700 Subject: [PATCH 08/36] changed llms.txt to _AGENT_ placedholder Signed-off-by: Bruce Hamilton --- docs/analysis/criteria.md | 10 +++++----- 1 file changed, 5 insertions(+), 5 deletions(-) diff --git a/docs/analysis/criteria.md b/docs/analysis/criteria.md index 00a6def8..af4ba113 100644 --- a/docs/analysis/criteria.md +++ b/docs/analysis/criteria.md @@ -133,13 +133,13 @@ emerging and should be utilized. We evaluate on the following: -- Is there an "llms.txt" file in the root of the website or documentation +- Is there an _AGENT_ file in the root of the website or documentation repository? - The "llms.txt" file is a proposed open standard for a Markdown file that - websites can host at the root. It helps large language models (LLMs) and AI - agents quickly understand a site's purpose, key content, and where to find - reliable information. + The _AGENT_ file is a proposed open standard for a Markdown file that websites + can host at the root. It helps large language models (LLMs) and AI agents + quickly understand a site's purpose, key content, and where to find reliable + information. For more informatino see https://llmstxt.org/ From 32219eb379a0844e1e0d7eb9becba0e0d885b63c Mon Sep 17 00:00:00 2001 From: Bruce Hamilton Date: Wed, 10 Jun 2026 14:18:03 -0700 Subject: [PATCH 09/36] spelling fix Signed-off-by: Bruce Hamilton --- docs/analysis/criteria.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/analysis/criteria.md b/docs/analysis/criteria.md index af4ba113..ad72704b 100644 --- a/docs/analysis/criteria.md +++ b/docs/analysis/criteria.md @@ -141,7 +141,7 @@ We evaluate on the following: quickly understand a site's purpose, key content, and where to find reliable information. - For more informatino see https://llmstxt.org/ + For more information, see https://llmstxt.org/ Task can be accomplished manually or by tools. From 3253ec1123d0f8c01c523dc472abbf0f085cf0bb Mon Sep 17 00:00:00 2001 From: Bruce Hamilton Date: Wed, 10 Jun 2026 17:02:39 -0700 Subject: [PATCH 10/36] added back llms.txt Signed-off-by: Bruce Hamilton --- docs/analysis/criteria.md | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/docs/analysis/criteria.md b/docs/analysis/criteria.md index ad72704b..58214c61 100644 --- a/docs/analysis/criteria.md +++ b/docs/analysis/criteria.md @@ -133,10 +133,10 @@ emerging and should be utilized. We evaluate on the following: -- Is there an _AGENT_ file in the root of the website or documentation +- Is there an llms.txt file in the root of the website or documentation repository? - The _AGENT_ file is a proposed open standard for a Markdown file that websites + The llms.txt file is a proposed open standard for a Markdown file that websites can host at the root. It helps large language models (LLMs) and AI agents quickly understand a site's purpose, key content, and where to find reliable information. From 0d1e328ac29594bbc3b3358b4a7d294b45f31070 Mon Sep 17 00:00:00 2001 From: Bruce Hamilton Date: Wed, 10 Jun 2026 17:20:33 -0700 Subject: [PATCH 11/36] ran prittier Signed-off-by: Bruce Hamilton --- docs/analysis/criteria.md | 8 ++++---- 1 file changed, 4 insertions(+), 4 deletions(-) diff --git a/docs/analysis/criteria.md b/docs/analysis/criteria.md index 58214c61..977e2dd3 100644 --- a/docs/analysis/criteria.md +++ b/docs/analysis/criteria.md @@ -136,10 +136,10 @@ We evaluate on the following: - Is there an llms.txt file in the root of the website or documentation repository? - The llms.txt file is a proposed open standard for a Markdown file that websites - can host at the root. It helps large language models (LLMs) and AI agents - quickly understand a site's purpose, key content, and where to find reliable - information. + The llms.txt file is a proposed open standard for a Markdown file that + websites can host at the root. It helps large language models (LLMs) and AI + agents quickly understand a site's purpose, key content, and where to find + reliable information. For more information, see https://llmstxt.org/ From 3dcf00a3bcc406c392b2e77389b1ab4bf8c1bfad Mon Sep 17 00:00:00 2001 From: Bruce Hamilton Date: Wed, 10 Jun 2026 18:21:45 -0700 Subject: [PATCH 12/36] added llms to word list Signed-off-by: Bruce Hamilton --- .cspell.yml | 1 + 1 file changed, 1 insertion(+) diff --git a/.cspell.yml b/.cspell.yml index bf753dbd..a58d2f93 100644 --- a/.cspell.yml +++ b/.cspell.yml @@ -45,3 +45,4 @@ words: - Uchechukwu - webdev - Welsch + - llms From b45666e8887afda16c539faab743cef62ff6a352 Mon Sep 17 00:00:00 2001 From: Bruce Hamilton Date: Tue, 16 Jun 2026 21:37:27 -0700 Subject: [PATCH 13/36] Applied Dave's suggestions Signed-off-by: Bruce Hamilton --- docs/analysis/criteria.md | 54 ++++++++++++++--------------- docs/analysis/templates/analysis.md | 21 +++++++---- 2 files changed, 42 insertions(+), 33 deletions(-) diff --git a/docs/analysis/criteria.md b/docs/analysis/criteria.md index 977e2dd3..b71d9610 100644 --- a/docs/analysis/criteria.md +++ b/docs/analysis/criteria.md @@ -15,7 +15,7 @@ referred to in the criteria. How your technical information is organized, laid out, and presented. This includes structure (pages/subpages/sections/subsections), information types -(conceptual/instructional/reference), and matching documentation content to user +(conceptual/instructional/reference), and content that matches user expectations. We evaluate on the following: @@ -23,7 +23,8 @@ We evaluate on the following: - Is there high level conceptual content? - Is every product feature documented? - Does the documentation define all user roles (personas) for the product? -- Are there instructions (tasks, tutorials) documented for features? +- Do instructions (tasks, tutorials) cover the intended purposes of all + user-facing features? - Are there instructions for all major use cases for each user role? - Are tasks organized by user role and use case? - Does instructional content demonstrate atomicity — are individual tasks @@ -31,8 +32,8 @@ We evaluate on the following: - Are tasks written as numbered step-by-step instructions? - Do task descriptions in headings and the TOC describe the task with a verb phrase? -- Is the documentation free of any key features which are documented but missing - task documentation? +- Are there any features that are documented (for example, described + conceptually) but are missing task documentation? - Is the “happy path” — the most common use case — documented? - If the documentation does not suffice, is there a clear escalation path for users needing more help? @@ -60,7 +61,8 @@ We evaluate on the following: - Are different types of installation documented (development, test, production) if necessary? - If needed, are multiple OSes documented? -- Do users know where to go after reading the getting started guide? +- Is there guidance for what to read next after reading the getting started + guide? - Is your new user content clearly signposted on your site’s homepage or at the top of your information architecture? - Is there easily copy-pastable sample code or other example content? @@ -128,20 +130,18 @@ We evaluate on the following: ### AI Optimization and Discoverability -Configurations for AI agents to optimize the understanding of content are -emerging and should be utilized. +Tools and techniques are emerging that enable AI agents to use documentation as +input. For example, many repositories now include "AI-friendly" Markdown files +that enable LLMs to ingest content. We encourage projects to optimize for +AI-based documentation use. We evaluate on the following: - Is there an llms.txt file in the root of the website or documentation repository? - The llms.txt file is a proposed open standard for a Markdown file that - websites can host at the root. It helps large language models (LLMs) and AI - agents quickly understand a site's purpose, key content, and where to find - reliable information. - - For more information, see https://llmstxt.org/ + The leading proposal for an AI-enabling file for websites is at + [llms-txt](https://llmstxt.org/ llms.txt). Task can be accomplished manually or by tools. @@ -169,10 +169,14 @@ Example: We evaluate on the following: -- Are docs issues well-triaged? +- Are docs issues triaged with respect to effort level and required expertise? - Is there a clearly marked way for new contributors to make code or documentation contributions (i.e. a “good first issue” label)? -- Are issues well-documented (i.e., more than just a title)? +- Are issues documented with: + - What is missing, incorrect, or improperly presented? + - What impact the issue has on documentation users? + - A suggested remedy? + - An estimate of the effort required to address the issue? - Are issues maintained for staleness? Example: @@ -191,7 +195,7 @@ We evaluate on the following: - Do you have a community repository or section on your website? - Is there a document specifically welcoming new contributors and documenting a first contribution process? -- Can new users find where to get help? +- Are there prominently displayed resources for new users to find help? Example: @@ -203,7 +207,7 @@ One of the CNCF’s core project values is open governance. We evaluate on the following: -- Is project governance clearly documented? +- Does the project governance align with CNCF guidelines and values? Example: @@ -258,8 +262,8 @@ requirements for sandbox projects._ Incubating status. - Is there rudimentary **project documentation**, or a placeholder or substitute? - - It is acceptable at this maturity level to link out to documentation that - hasn't yet been integrated into the website. + - It is acceptable at this maturity level to link to documentation that hasn't + yet been integrated into the website. - _Example_: website with a single homepage, without any documentation or, as was mentioned above, linking out to an external (preexisting) source for docs. @@ -285,7 +289,7 @@ requirements for sandbox projects._ #### Graduated - Are follow-through actions from the [Docs assessment][] complete? -- Does **project documentation** fully addresses the needs of key stakeholders? +- Does **project documentation** fully addresses the needs of stakeholders? #### Archived @@ -293,7 +297,7 @@ requirements for sandbox projects._ - Is the archived status of the project obvious to those visiting the website, such as through the use of a prominent banner? - If a successor project exists, are there links to its website and/or migration - documentation. + documentation? [archived state]: https://docs.github.com/en/repositories/archiving-a-github-repository/archiving-repositories @@ -314,12 +318,8 @@ We evaluate on the following: - Is the website usable from mobile? - Are doc pages readable? -- Are all or most website features accessible from mobile -- such as the - top-nav, site search, and in-page table of contents? - Might a [mobile-first][] design make sense for your project? -- Are color contrasts significant enough for color-impaired readers? - Are most website features usable using a keyboard only? -- Does text-to-speech offer listeners a good experience? [mobile-first]: https://developer.mozilla.org/en-US/docs/Web/Progressive_web_apps/Responsive/Mobile_first @@ -329,8 +329,8 @@ Plan for suitable [accessibility][] measures for your website. We evaluate on the following: - Are color contrasts significant enough for color-impaired readers? -- Are most website features usable using a keyboard only? -- Does text-to-speech offer listeners a good experience? +- Are all vital website features usable using a keyboard only? +- Does text-to-speech offer listeners a usable experience? It is up to each project to set their own guidelines. diff --git a/docs/analysis/templates/analysis.md b/docs/analysis/templates/analysis.md index 2da47531..8acb9fa2 100644 --- a/docs/analysis/templates/analysis.md +++ b/docs/analysis/templates/analysis.md @@ -24,7 +24,7 @@ TO USE THIS TEMPLATE, search and replace the named IDs: stored; this might be its own repo or a directory in the project main repo Replace link placeholders in brackets, such as [_PROJECT_][project-website], -with the actual link surrounded by parenthesis. +with a Markdown-style link. For the analysis procedure, see [Analysis how-to](../howto.md). @@ -89,11 +89,14 @@ This document is divided into three sections that represent three major areas of concern: - **Project documentation:** concerns documentation for users of the _PROJECT_ - software, aimed at people who intend to use the project software + software, aimed at people who intend to use the project software. - **Contributor documentation:** concerns documentation for new and existing - contributors to the _PROJECT_ OSS project + contributors to the _PROJECT_ OSS project. - **Website:** concerns the mechanics of publishing the documentation, and - includes branding, website structure, and maintainability + includes branding, website structure, and maintainability. + +A fourth area of concern, AI optimization and discovery, might be included in +this analysis. If so, it is discussed in a separate document, `ai-guidance.md`. Each section begins with summary ratings based on a rubric with appropriate [criteria] for the section, then proceeds to: @@ -103,6 +106,12 @@ Each section begins with summary ratings based on a rubric with appropriate - **Recommendations**: suggested changes that would improve the effectiveness of the documentation. +> Author's note: We recommend writing implementation notes only for large +> projects. An implementation plan can bridge the gap between the high-level +> recommendations in this document and individual issues entered in the doc +> repo. Remove the sentences about the implementation doc if you didn't write +> one. + The accompanying [implementation] document breaks the recommendations down into concrete actions that can be implemented by project contributors. Its focus is on drilling down to specific, achievable work that can be completed in @@ -178,8 +187,8 @@ documentation. We evaluate on the following: complete? (i.e., each product feature is documented) - Are there step-by-step instructions (tasks, tutorials) documented for features? -- Are there any key features which are documented but missing task - documentation? +- Are there any features that are documented (for example, described + conceptually) but are missing task documentation? - Is the “happy path”/most common use case documented? Does task and tutorial content demonstrate atomicity and isolation of concerns? (Are tasks clearly named according to user goals?) From daa92c300fe3a7d1fd1f331dc58e983bd5848dfd Mon Sep 17 00:00:00 2001 From: Bruce Hamilton Date: Tue, 7 Jul 2026 08:46:33 -0700 Subject: [PATCH 14/36] Added Appendix A Signed-off-by: Bruce Hamilton --- docs/analysis/howto.md | 388 +++++++++++++++++++++++++++++++++++++++++ 1 file changed, 388 insertions(+) diff --git a/docs/analysis/howto.md b/docs/analysis/howto.md index 19a71825..e8eec2c6 100644 --- a/docs/analysis/howto.md +++ b/docs/analysis/howto.md @@ -223,3 +223,391 @@ Create issues in the project documentation GitHub repository for: [project maturity level]: https://www.cncf.io/project-metrics [templates]: ./templates/index.md [issues list]: ./templates/issues-list.md + +## Appendix A - Build an LLM Wiki for Analysis + +This content for this appendix was itself generated by AI. Here is the prompt +originally used. + +> Prompt: My job is to do analysis on CNCF project documentation. For other +> analysts to use, we have a how-to. For its AI section, I need you to create a +> doc plan for me to describe how to create an llm wiki, set the Model level and +> settings, bring in repositories or files (pros and cons of each), sample +> prompts for ingest/create wiki pages/update index. Generally, the best +> practices you can think of. Output your outline to how-to-wiki.md in the +> current directory. + +> Audience: Analysts doing analysis on CNCF project documentation. Goal: Stand +> up and maintain an LLM-curated wiki (the "KubeMeow" pattern) that turns raw +> upstream docs into cross-linked, cited, synthesized knowledge pages. + +## Overview & mental model + +An Obsidian vault where raw source material is ingested and synthesized by an +LLM into curated, cross-linked, markdown pages. The core principal is that the +raw sources, `raw/`, are immutable and the wiki is derived. Every claim in a +wiki page traces back to a `raw/` path. There are three moving parts the analyst +drives: + +- The AI agent (Copilot CLI) +- The vault (Obsidian) +- The source corpus (`raw/`). + +```mermaid +flowchart LR + raw[raw/] + ingest[LLM ingest] + wiki[wiki/*.md] + index[index.md + log.md] + + raw --> ingest --> wiki --> index +``` + +## Setup + +To set up an LLM wiki, perform the following steps. (Alternatively, you can +install the +[Karpathy LLM Wiki - Obsidian Plugin](https://community.obsidian.md/plugins/karpathywiki)). + +1. Create the wiki structure as shown here. Replace `my-research` with the name + for wiki. For CNCF analysis, the wiki should be exclusive to one project so + use the project name in the wiki's name. + + ```text + my-research/ + ├── raw/ # Layer 1: Immutable source documents + │ ├── articles/ + │ ├── papers/ + │ ├── repos/ + │ ├── data/ + │ ├── images/ + │ └── assets/ + ├── wiki/ # Layer 2: LLM-generated markdown + │ ├── index.md # Content catalog (updated on every ingest) + │ ├── log.md # Append-only chronological record + │ ├── overview.md + │ ├── concepts/ # Concept pages + │ ├── entities/ # Entity pages + │ ├── sources/ # Source summaries + │ └── comparisons/ # Comparison pages + ├── outputs/ # Dated reports, presentations + ├── CLAUDE.md # Layer 3: Schema configuration + └── .gitignore + ``` + + | Path | Purpose | Editable by AI? | + | ------------------------------------------ | -------------------------------------------------- | ----------------- | + | `raw/topics//` | Immutable verbatim source checkouts | Never | + | `raw/{articles,data,images,papers,repos}/` | Placeholders for other source types | Never | + | `wiki/` | Synthesized concept/entity pages (the deliverable) | Yes | + | `wiki/sources/` | One source-summary page per ingested source | Yes | + | `wiki/{comparisons,concepts,entities}/` | Category placeholders | Yes | + | `wiki/index.md` | Master catalog + coverage map | Every op | + | `wiki/log.md` | Append-only operation history (newest on top) | Every op | + | `outputs/` | Generated reports (e.g. `lint-YYYY-MM-DD.md`) | Yes | + | `.obsidian/`, `*.base`, `*.canvas` | Obsidian app artifacts | Leave to Obsidian | + + Run the following bash commands create this structure: + + ```bash + mkdir -p ~/research/my-topic/{raw/{articles,papers,repos,data,images},wiki/{concepts,entities,sources,comparisons},outputs} + touch ~/research/my-topic/wiki/index.md + touch ~/research/my-topic/wiki/log.md + ``` + +1. Install tools: + - [Obsidian](https://obsidian.md/download) + - [Git](https://https://git-scm.com/install/) + - [GitHub CLI](https://cli.github.com/) and the LLM agent (GitHub Copilot + CLI). + +1. In Obsidian, open the command pallet and set the vault name to the name of + your wiki. + +1. Author the `.github/copilot-instructions.md`file, an important step for + achieving successful results. See + [Agent constitution](#agent-constitution---copilot-instructions-markdown-file) + for authoring the `copilot-instructions.md` file that resides at the root of + the wiki. + +## Choose model tier & refine settings + +Choose the model level desired, and fine tune with the settings. Heavier +reasoning models produce better synthesis but costs more and runs slower. + +## Model tier + +The Copilot CLI lets you switch models, including within a session. Run +`/model`and select. + +| Wiki task | Model tier | Why | +| ------------------------------------- | ---------------------------------------- | ----------------------------------------------------------- | +| Ingest + synthesize a large corpus | High-reasoning (e.g. Opus / GPT-5-class) | Cross-doc synthesis, contradiction detection, deduplication | +| Create/rewrite a concept page | High-reasoning | Quality of prose + citation accuracy matters most | +| Query / Q&A over existing wiki | Mid tier | Retrieval + short synthesis; cheaper | +| Lint / mechanical index & log updates | Fast/low tier | Deterministic, low-creativity edits | + +(TODO: confirm exact command/label in your CLI version and pick effort/verbosity +where offered.) + +## Critical settings + +For each model tier, you can control its access, scope, strength, time, and +other factors. + +| Wiki setting | Recommended configuration | +| ------------------------------------------------------------- | --------------------------------------------------------------------------------------------------------------------------------------- | +| Custom instructions (`.github/copilot-instructions.md`) | Refine for your project. | +| Permissions (`.claude/settings.local.json` or CLI equivalent) | Allow-list read paths, prose synthesis, `gh api`, and package installs. Keep write scope tight. | +| Temperature / determinism | Prefer low creativity for index/log/lint ops (TODO: note where this is exposed in your CLI.) | +| Context window / long-context tier | When a corpus is large, use a long-context mode or delegate sub-sections to sub-agents rather than stuffing everything into one prompt. | +| Session artifacts vs. committed files | Planning notes live in the agent's session state, not the vault. Only `wiki/`, `index.md`, `log.md`, `outputs/` get committed. | + +## Agent Constitution - copilot-instructions Markdown file + +The instructions are at the repository level, enabling results that are +repeatable across analysts and sessions. Without it, every analyst gets +different structure. Workflows: Ingest, Query, and Lint. Page conventions: +frontmatter schema, kebab-case filenames, `[[wikilinks]]` only, "synthesize +don't copy," date format. Watch out for cross-topic bleed, ensure `index.md` and +`log.md` are in sync. Things to watch": cross-topic bleed, keeping +`index.md`/`log.md` in sync. Best practice: treat the instructions file as +versioned policy — review it when conventions change; it is the contract the +agent follows. + +## Bringing in Source Material: Repositories vs. Files + +You can bring a documentation set for analysis as either a reference to a GitHub +repository in `raw/repos` or as a structure of files in `raw/articles` you can +rename `articles` to `topics` or define other types of sources in `raw/'. + +### How ingestion sources land in `raw/` + +All files in `/raw/topics/` should be the most recent sources for the project. + +- Everything goes under `raw/topics//` as a verbatim checkout/copy. Never + hand-edit. +- Record provenance (upstream URL + branch/commit) in the source-summary page + and `log.md`. + +### 4b. Option A — Bring in a whole repository + +To add a repo to the wiki, in the `raw/repos/` folder, run +`git clone https://github.com/.git`. You can also ask the agent to +do it in a prompt with the URL. + +(This repo's Flatcar topic is a checkout of `flatcar/flatcar-website`, +`expert-support-refactor` branch.) + +| Pros | Cons | +| ------------------------------------------------------------------ | ---------------------------------------------------------------------------- | +| Complete coverage — nothing missed | Pulls in non-content noise (Hugo templates, CI, `Makefile`, repo meta) | +| Exact provenance (branch/commit is pinnable & reproducible) | Larger footprint; slower first ingest | +| Easy to re-ingest on upstream changes (`git pull` → re-run ingest) | Must explicitly exclude meta files (see this repo's "Excluded Files" list) | +| Preserves upstream structure → clean section→page mapping | Risk of accidentally running upstream tooling (don't run `raw/`'s Makefiles) | +| Diffs show exactly what changed between ingests | Branch churn can silently move/rename sections (must audit) | + +### 4c. Option B — Bring in individual files + +- _How:_ copy specific docs/PDFs/articles into `raw/topics//` (or + `raw/articles/`, `raw/papers/`). + +| Pros | Cons | +| ------------------------------------------------------ | ----------------------------------------------------- | +| Minimal noise — only what you need | Provenance is manual and easy to lose | +| Fast to ingest; small footprint | No automatic re-ingest path; updates are manual | +| Good for one-off articles, papers, PDFs, mixed formats | Coverage gaps: easy to forget a related file | +| No upstream tooling comes along | No structural map to derive section→page mapping from | +| Easy when there is no single upstream repo | Harder to prove "this is the complete source set" | + +### 4d. Decision guidance + +- Whole repo when: source is a single upstream docs repo, you'll track it over + time, and reproducibility matters (the default for CNCF project docs). +- Individual files when: sources are scattered (blogs, papers, PDFs), one-off, + or you only need a slice. +- Either way: always create a `wiki/sources/.md` summary that records + what was ingested, counts, and what was deliberately excluded. + +## §5. Workflow — Ingest + +_What to write:_ step list + copyable prompts (the "sample prompts for ingest" +requirement). + +_Steps:_ read source under `raw/` → discuss key takeaways → create/update +`wiki/sources/.md` → create/update concept/entity pages → update +`wiki/index.md` → append to `wiki/log.md`. + +Sample prompt — full ingest: + +``` +Ingest the source at raw/topics//. Read it in full, then: +1. Summarize the key takeaways and propose a section→page mapping before writing. +2. Create wiki/sources/-docs.md capturing what's covered, file counts, and + any files deliberately excluded (repo meta, templates, boilerplate). +3. Create/update the concept & entity pages, one per major section. Every page must + have YAML frontmatter (title, type, sources, related, created, updated, confidence), + kebab-case -.md filenames, and [[wikilinks]] for all internal links. +4. Synthesize — condense sources into concise prose and comparison tables; do NOT copy + verbatim. Every claim must cite a raw/ path. +5. Update wiki/index.md (catalog + coverage map) and append a dated entry to the top of + wiki/log.md. +Do not edit anything under raw/. Do not run any tooling inside raw/. +``` + +Sample prompt — re-ingest after upstream changes: + +``` +The files under raw/topics// were updated. Re-ingest: +- Diff current sources against what the existing pages cover. +- Report source growth (new/renamed/removed sections) as a table. +- Re-synthesize affected pages, bump their `updated:` date, and expand `sources:`. +- Refresh index.md counts/coverage map and append a log.md entry describing the delta. +``` + +_Best practices to note:_ ask the agent to propose the mapping before writing; +ingest one topic at a time; for very large corpora, delegate sections to +sub-agents. + +## §6. Workflow — Create / Update Wiki Pages + +_What to write:_ the page contract + prompts (the "create wiki pages / update +index" requirement). + +- Frontmatter schema (show verbatim; every page starts with this): + +```yaml +--- +title: 'Human Readable Title' +type: concept # concept | entity | source-summary | comparison +sources: + - raw/topics//content/docs/latest/
/ +related: + - '[[-overview]]' +created: YYYY-MM-DD +updated: YYYY-MM-DD +confidence: high # high | medium | low +--- +``` + +- Rules: kebab-case filenames; `[[wikilinks]]` never relative paths; synthesize + don't copy; bump `updated:` on every change; `confidence:` reflects source + strength. + +Sample prompt — new concept page: + +``` +Create wiki/-.md as a `concept` page synthesizing the sources under +raw/topics//.../
/. Use the standard frontmatter, cite every claim to a +raw/ path, prefer comparison tables over prose where it aids scanning, and add [[wikilinks]] +to related pages. Then update index.md and append to log.md. +``` + +Sample prompt — update index: + +``` +Update wiki/index.md: ensure every page in wiki/ and wiki/sources/ appears in the catalog +tables with an accurate one-line summary, and refresh the coverage map (section → page, +with current file counts). Flag any orphan pages (no incoming [[links]]) or catalog entries +whose files no longer exist. +``` + +Sample prompt — append to log: + +``` +Append a new dated entry to the TOP of wiki/log.md describing this operation: operation type, +source, reason, pages created/updated, and index changes. Keep newest-on-top ordering. +``` + +## §7. Workflow — Lint (Content Consistency, not Code) + +_What to write:_ what "lint" means for a wiki + prompt. + +- Checks: contradictions across pages, orphan pages (no incoming links), + referenced-but-missing concepts (dangling `[[wikilinks]]`), and stale claims + superseded by newer sources. +- Output goes to `outputs/lint-YYYY-MM-DD.md` (not into the wiki pages + themselves). + +Sample prompt — lint: + +``` +Lint the wiki for content consistency (not code). Scan all wiki pages for: contradictions +between pages, orphan pages with no incoming [[links]], dangling wikilinks to pages that +don't exist, and stale claims superseded by newer sources. Write findings to +outputs/lint-.md with file/line references and suggested fixes. Do not modify wiki +pages in this pass. +``` + +## §8. Workflow — Query / Q&A + +_What to write:_ how analysts pull answers back out. + +- Steps: read `index.md` to find relevant pages → read them → synthesize → + answer with `[[wikilink]]` citations → offer to save a novel, valuable answer + as a new page. + +Sample prompt — query: + +``` +Using the wiki, answer: "". Start from index.md to locate relevant pages, read +them, synthesize a cited answer using [[wikilinks]], and if the answer is novel and reusable, +offer to save it as a new comparison/concept page. +``` + +## §9. Cross-Topic Hygiene & Auditing + +_What to write:_ + +- Cross-topic bleed: when a topic is dropped, audit remaining pages for stray + links to it — check both `related:` frontmatter and body content. (Reference + the KubeVirt bleed audit in `log.md` as the worked example.) +- Retain source-derived content that legitimately belongs to the kept topic even + if it _names_ the removed one (e.g. Flatcar-on-KubeVirt deploy docs stay). +- Keep `index.md` and `log.md` as the source of truth for what exists and what + happened. + +## §10. Best Practices (Cheat Sheet) + +_What to write — the "best practices" requirement, as a scannable list:_ + +- Raw is sacred. Never edit/move/delete `raw/`; never run tooling that lives + inside it. +- Cite everything. Every claim → a `raw/` path; every page → `sources:` + frontmatter. +- Synthesize, don't copy. Pages are condensed prose + tables, not reproductions. +- One source of truth. Update `index.md` and append `log.md` on _every_ + operation. +- Ask before writing. Have the agent propose a section→page mapping first. +- Pin provenance. Record upstream URL + branch/commit for reproducible + re-ingests. +- Prefer whole-repo ingest for trackable CNCF docs; files for one-offs (see §4). +- Right-size the model (§2): high-reasoning for synthesis, fast for mechanical + ops. +- Small, verifiable steps. Ingest one topic at a time; lint regularly. +- Keep planning out of the vault. Use session state, commit only deliverables. +- Version the constitution. Review `copilot-instructions.md` when conventions + change. +- Date discipline. `YYYY-MM-DD`; bump `updated:` whenever a page changes. + +## §11. Troubleshooting & FAQ + +_What to write:_ a short table. + +- Agent edited `raw/` → revert; strengthen instructions; re-run. +- Pages drifted from sources → re-ingest (§5) and diff. +- Broken `[[wikilinks]]` / orphans → run lint (§7). +- Duplicate/overlapping pages → merge, add cross-links, update `index.md`. +- Agent ran upstream Makefile/scripts → forbidden; reaffirm the "don't run + `raw/` tooling" rule. +- _TODO: add real incidents analysts hit in practice._ + +## §12. Appendix + +_What to write / attach:_ + +- Full copy of `.github/copilot-instructions.md` (or link). +- The exact model + settings your team standardized on. _TODO: fill in._ +- Glossary: ingest, synthesize, concept/entity/source-summary/comparison page, + coverage map, cross-topic bleed, orphan page. +- Links: Obsidian, Copilot CLI docs, the upstream source repos in `raw/`. From dc0a24dd204bf4312daddea8d18c434e2c538df5 Mon Sep 17 00:00:00 2001 From: Bruce Hamilton Date: Tue, 7 Jul 2026 12:35:57 -0700 Subject: [PATCH 15/36] remove and stop tracking howto Signed-off-by: Bruce Hamilton --- docs/analysis/howto.md | 613 ----------------------------------------- 1 file changed, 613 deletions(-) delete mode 100644 docs/analysis/howto.md diff --git a/docs/analysis/howto.md b/docs/analysis/howto.md deleted file mode 100644 index e8eec2c6..00000000 --- a/docs/analysis/howto.md +++ /dev/null @@ -1,613 +0,0 @@ ---- -title: CNCF TechDocs Analysis How-To ---- - -# CNCF TechDocs Analysis How-To - -## Audience - -This document is for members of the CNCF TechDocs team and others who wish to -conduct or assist with an analysis of a CNCF open-source project's technical -documentation. - -## Purpose - -The goals of a CNCF technical documentation analysis are to: - -- Examine the current project technical documentation and website against the - CNCF's analysis [criteria]. -- Compare the documentation against the current or proposed [project maturity - level]. -- Recommend a program of key documentation improvements with the largest return - on investment. These improvements are documented as _recommendations_ in the - analysis document, and expanded in a companion - [implementation plan](./templates/implementation.md) and - [issues list](./templates/issues-list.md). - -## Doing a Tech Docs Analysis - -The tech docs analysis consists of some repository bookkeeping (prerequisites), -then of three overall tasks: - -1. Write the analysis document: evaluate the existing project documentation with - respect to the [project maturity level] or proposed maturity level, if the - analysis is associated with a project upgrade request. Identify gaps with - CNCF [criteria]. Write general recommendations to close the largest and most - important identified issues. -2. Write a implementation plan: decompose recommendations in to specific - improvement suggestions. These can be additions or revisions to the docs; - reorganization; website infrastructure changes; or any other work that will - close address issues. Make suggestions specific enough (for example, if you - propose reorganizing a section then provide an outline) without being overly - constraining so that a contributor could solve the problem differently if - they have a different solution. For example, make it clear that your proposed - outline is only one possible reorganization. -3. Write an issue backlog. - -Finally, there are follow-up steps including: - -- Creating GitHub issues and a pull request -- Getting approval from project maintainers - -### Prerequisites - -This section assumes you are familiar with GitHub repositories and pull requests -(PRs). If you need a refresher, see -[Get started](https://docs.github.com/en/get-started) from the GitHub docs. - -Project analyses are kept in the -[CNCF tech docs repository](https://github.com/cncf/techdocs). Clone and prepare -for - -1. Clone the [CNCF tech docs repository](https://github.com/cncf/techdocs). -2. Create a branch for the analysis. -3. In the new branch, create a directory for the analysis in the CNCF tech docs - [analyses] directory. Name the directory `00NN-_PROJECT_`, where _NN_ is the - next index available in the directory (check for PRs as well, if someone else - is working on tech doc analyses), and where _PROJECT_ is a short but not - abbreviated project name. For example, for Kubernetes _PROJECT_ would be - _Kubernetes_, not _k8s_. -4. Copy all the doc analysis [templates]. - -### Writing the Analysis document - -Follow the steps outlined below as a part of writing the project's analysis -document. Record your findings in the project's -[analysis.md](./templates/analysis.md) file. - -1. Define the **scope** of the analysis. Edit "Scope of analysis" to reflect - URLs and repositories included and excluded from the analysis. -2. Review the in-scope URLs and repositories for compliance with the rubric - criteria. Note any gaps, as well as any areas that exceed criteria or are - exceptionally well executed. I find it easiest to do this separately for each - of the three areas of concern (project doc, contributor doc, website), making - a pass through the documentation once for each section (Information - architecture, New user content, Content maintainability, etc.). Don't worry - about a numerical score during this step; instead, note how the documentation - complies, or not, with each criterion with respect to the project maturity - level (or proposed maturity level, if the analysis is part of a petition for - upgrade). Write comments to note the most important gaps and best-executed - features of the documentation. -3. Assign ratings to each criterion based on your comments and compliance with - the maturity level expectations in the rubric. The ratings are - self-explanatory. Keep in mind that "needs improvement" or "meets standards" - is with respect to the current (or proposed) maturity level. -4. Write recommendations. The template implies that you'll do this for every - criterion; the "Recommendations" headings mirror the "Comments" headings. - However, if some alternative framework makes more sense, use that. For - example, it might be that two or three of the product documentation criteria - are improved by reorganizing the documentation. In this case, rather than - repeat the recommendation to reorganize in each section, write a single - recommendation and explain how it improves all the areas. This is the first - step in moving from the analysis to specific, actionable, time-bound backlog - items. - -#### General tips - -Things to keep in mind while doing the analysis: - -- Look for: - - Quick wins – low-effort changes that would have a major impact. - - Large, systemic issues that you can organize a number of issues around. - - The two or three most important issues that impede documentation - effectiveness. - - Anything the project does exceptionally well. We can call these out as - examples in other evaluations! -- Don't get bogged down in detail when writing comments. Include enough detail - that you can describe how to implement a suggested solution. A sentence or two - is sufficient for most issues. -- Keep in mind the overall goal of the technical documentation: to make its - users more effective. Focus on issues that get in the way of that goal. -- It is not necessary to come up with a recommendation in every case, especially - for elements that are satisfactory or if a recommendation would result in - minimal improvement. -- Don't worry about grammar and style if they don't affect documentation - effectiveness. If writing style impedes understanding, make a note; otherwise - move on. An exception: Insist that tasks and procedures be written as clear, - step-by-step instructions. -- Some of the criteria, especially around contributor documentation, project - governance, and website infrastructure, are essentially check-boxes. These you - can quickly investigate, note, and move on. Spend more time analyzing the - effectiveness of the project documentation. - -#### Common issues - -Many common issues seem to come about because open-source software documentation -is written by software developers while they are writing the software. This -results in documentation that: - -1. Is organized around features, not users and use cases. -2. Explains technical concepts well, including architecture and design - decisions. -3. Contains complete reference information for APIs, CLIs, and configuration - parameters. -4. Has missing or incomplete user-oriented "how-to" explanations and operational - procedures. - -You may or may not find the following issues in your analysis, but it's worth -keeping them in mind: - -- Ambiguity around user roles. -- Missing or unclear task-based documentation. -- Assumptions about the reader's level of knowledge. -- Organization that buries or scatters important information, especially tasks - and procedures. -- Missing or unclear new-user workflows. - -### Writing the implementation plan - -Write the implementation plan. Edit the `_PROJECT_-implementation.md` file. - -The gist of the implementation plan is to break down the recommendations in the -analysis document. This is an intermediate stage between general recommendations -and the issues backlog. For small projects and where the recommendations are -independent and time-bound, an implementation plan might not be necessary and -you can move straight to writing the backlog. - -If you do write an implementation plan, start with recommendations in the -analysis document. Rewrite the recommendations, making them more specific, with -a suggested (but not mandatory) implementation. For example, if you recommend -reorganizing the documentation, provide a suggested outline, along with an -explanation of the reason for the reorg. - -### Writing an issues backlog - -Write an issues backlog. Edit `_PROJECT_-issues.md`. - -The goal of writing an issues backlog is to offer project contributors the -opportunity to make the recommended changes. - -Rewrite each action in the implementation plan. If possible, break large actions -into smaller issues. Each issue should be: - -- Independent. As much as possible, no issue should have another issue as a - prerequisite. A contributor should be able to choose an issue, resolve it, and - close it without reference to any other issue. -- Time-bound. A contributor should be able to complete an issue in a reasonably - short time, say a few hours or a couple of days at most. - -Make the suggested solution even more specific. At this point, the issue should -almost be a recipe for making the doc improvement, with the caveat that a -contributor is not required to implement the solution as suggested in the issue. - -## Next Steps - -### Including supporting documentation - -If you have supporting material that might be helpful to a contributor working -on the documentation issues, include them in the directory with the other -documents. For example, you might inventory the existing tech doc pages in a -spreadsheet; in this case, include a CSV file of the inventory. - -### Creating a pull request - -If you have not created a pull request with the analysis documents, do so now. -Tag project maintainers and CNCF documentation staff, and ask for comments. - -### Getting contributor feedback - -If you haven't met with the project's maintainers yet, do so before you create -the issues in GitHub. Ideally you'd like to have a Zoom meeting with any -interested parties to get feedback on the analysis and implementation plan. - -### Creating GitHub issues - -Create issues in the project documentation GitHub repository for: - -- Each issues in the [issues list]. -- An umbrella issue that provides a context for the previously created - individual issues. - -[analyses]: ../../analyses/ -[criteria]: ./criteria.md -[project maturity level]: https://www.cncf.io/project-metrics -[templates]: ./templates/index.md -[issues list]: ./templates/issues-list.md - -## Appendix A - Build an LLM Wiki for Analysis - -This content for this appendix was itself generated by AI. Here is the prompt -originally used. - -> Prompt: My job is to do analysis on CNCF project documentation. For other -> analysts to use, we have a how-to. For its AI section, I need you to create a -> doc plan for me to describe how to create an llm wiki, set the Model level and -> settings, bring in repositories or files (pros and cons of each), sample -> prompts for ingest/create wiki pages/update index. Generally, the best -> practices you can think of. Output your outline to how-to-wiki.md in the -> current directory. - -> Audience: Analysts doing analysis on CNCF project documentation. Goal: Stand -> up and maintain an LLM-curated wiki (the "KubeMeow" pattern) that turns raw -> upstream docs into cross-linked, cited, synthesized knowledge pages. - -## Overview & mental model - -An Obsidian vault where raw source material is ingested and synthesized by an -LLM into curated, cross-linked, markdown pages. The core principal is that the -raw sources, `raw/`, are immutable and the wiki is derived. Every claim in a -wiki page traces back to a `raw/` path. There are three moving parts the analyst -drives: - -- The AI agent (Copilot CLI) -- The vault (Obsidian) -- The source corpus (`raw/`). - -```mermaid -flowchart LR - raw[raw/] - ingest[LLM ingest] - wiki[wiki/*.md] - index[index.md + log.md] - - raw --> ingest --> wiki --> index -``` - -## Setup - -To set up an LLM wiki, perform the following steps. (Alternatively, you can -install the -[Karpathy LLM Wiki - Obsidian Plugin](https://community.obsidian.md/plugins/karpathywiki)). - -1. Create the wiki structure as shown here. Replace `my-research` with the name - for wiki. For CNCF analysis, the wiki should be exclusive to one project so - use the project name in the wiki's name. - - ```text - my-research/ - ├── raw/ # Layer 1: Immutable source documents - │ ├── articles/ - │ ├── papers/ - │ ├── repos/ - │ ├── data/ - │ ├── images/ - │ └── assets/ - ├── wiki/ # Layer 2: LLM-generated markdown - │ ├── index.md # Content catalog (updated on every ingest) - │ ├── log.md # Append-only chronological record - │ ├── overview.md - │ ├── concepts/ # Concept pages - │ ├── entities/ # Entity pages - │ ├── sources/ # Source summaries - │ └── comparisons/ # Comparison pages - ├── outputs/ # Dated reports, presentations - ├── CLAUDE.md # Layer 3: Schema configuration - └── .gitignore - ``` - - | Path | Purpose | Editable by AI? | - | ------------------------------------------ | -------------------------------------------------- | ----------------- | - | `raw/topics//` | Immutable verbatim source checkouts | Never | - | `raw/{articles,data,images,papers,repos}/` | Placeholders for other source types | Never | - | `wiki/` | Synthesized concept/entity pages (the deliverable) | Yes | - | `wiki/sources/` | One source-summary page per ingested source | Yes | - | `wiki/{comparisons,concepts,entities}/` | Category placeholders | Yes | - | `wiki/index.md` | Master catalog + coverage map | Every op | - | `wiki/log.md` | Append-only operation history (newest on top) | Every op | - | `outputs/` | Generated reports (e.g. `lint-YYYY-MM-DD.md`) | Yes | - | `.obsidian/`, `*.base`, `*.canvas` | Obsidian app artifacts | Leave to Obsidian | - - Run the following bash commands create this structure: - - ```bash - mkdir -p ~/research/my-topic/{raw/{articles,papers,repos,data,images},wiki/{concepts,entities,sources,comparisons},outputs} - touch ~/research/my-topic/wiki/index.md - touch ~/research/my-topic/wiki/log.md - ``` - -1. Install tools: - - [Obsidian](https://obsidian.md/download) - - [Git](https://https://git-scm.com/install/) - - [GitHub CLI](https://cli.github.com/) and the LLM agent (GitHub Copilot - CLI). - -1. In Obsidian, open the command pallet and set the vault name to the name of - your wiki. - -1. Author the `.github/copilot-instructions.md`file, an important step for - achieving successful results. See - [Agent constitution](#agent-constitution---copilot-instructions-markdown-file) - for authoring the `copilot-instructions.md` file that resides at the root of - the wiki. - -## Choose model tier & refine settings - -Choose the model level desired, and fine tune with the settings. Heavier -reasoning models produce better synthesis but costs more and runs slower. - -## Model tier - -The Copilot CLI lets you switch models, including within a session. Run -`/model`and select. - -| Wiki task | Model tier | Why | -| ------------------------------------- | ---------------------------------------- | ----------------------------------------------------------- | -| Ingest + synthesize a large corpus | High-reasoning (e.g. Opus / GPT-5-class) | Cross-doc synthesis, contradiction detection, deduplication | -| Create/rewrite a concept page | High-reasoning | Quality of prose + citation accuracy matters most | -| Query / Q&A over existing wiki | Mid tier | Retrieval + short synthesis; cheaper | -| Lint / mechanical index & log updates | Fast/low tier | Deterministic, low-creativity edits | - -(TODO: confirm exact command/label in your CLI version and pick effort/verbosity -where offered.) - -## Critical settings - -For each model tier, you can control its access, scope, strength, time, and -other factors. - -| Wiki setting | Recommended configuration | -| ------------------------------------------------------------- | --------------------------------------------------------------------------------------------------------------------------------------- | -| Custom instructions (`.github/copilot-instructions.md`) | Refine for your project. | -| Permissions (`.claude/settings.local.json` or CLI equivalent) | Allow-list read paths, prose synthesis, `gh api`, and package installs. Keep write scope tight. | -| Temperature / determinism | Prefer low creativity for index/log/lint ops (TODO: note where this is exposed in your CLI.) | -| Context window / long-context tier | When a corpus is large, use a long-context mode or delegate sub-sections to sub-agents rather than stuffing everything into one prompt. | -| Session artifacts vs. committed files | Planning notes live in the agent's session state, not the vault. Only `wiki/`, `index.md`, `log.md`, `outputs/` get committed. | - -## Agent Constitution - copilot-instructions Markdown file - -The instructions are at the repository level, enabling results that are -repeatable across analysts and sessions. Without it, every analyst gets -different structure. Workflows: Ingest, Query, and Lint. Page conventions: -frontmatter schema, kebab-case filenames, `[[wikilinks]]` only, "synthesize -don't copy," date format. Watch out for cross-topic bleed, ensure `index.md` and -`log.md` are in sync. Things to watch": cross-topic bleed, keeping -`index.md`/`log.md` in sync. Best practice: treat the instructions file as -versioned policy — review it when conventions change; it is the contract the -agent follows. - -## Bringing in Source Material: Repositories vs. Files - -You can bring a documentation set for analysis as either a reference to a GitHub -repository in `raw/repos` or as a structure of files in `raw/articles` you can -rename `articles` to `topics` or define other types of sources in `raw/'. - -### How ingestion sources land in `raw/` - -All files in `/raw/topics/` should be the most recent sources for the project. - -- Everything goes under `raw/topics//` as a verbatim checkout/copy. Never - hand-edit. -- Record provenance (upstream URL + branch/commit) in the source-summary page - and `log.md`. - -### 4b. Option A — Bring in a whole repository - -To add a repo to the wiki, in the `raw/repos/` folder, run -`git clone https://github.com/.git`. You can also ask the agent to -do it in a prompt with the URL. - -(This repo's Flatcar topic is a checkout of `flatcar/flatcar-website`, -`expert-support-refactor` branch.) - -| Pros | Cons | -| ------------------------------------------------------------------ | ---------------------------------------------------------------------------- | -| Complete coverage — nothing missed | Pulls in non-content noise (Hugo templates, CI, `Makefile`, repo meta) | -| Exact provenance (branch/commit is pinnable & reproducible) | Larger footprint; slower first ingest | -| Easy to re-ingest on upstream changes (`git pull` → re-run ingest) | Must explicitly exclude meta files (see this repo's "Excluded Files" list) | -| Preserves upstream structure → clean section→page mapping | Risk of accidentally running upstream tooling (don't run `raw/`'s Makefiles) | -| Diffs show exactly what changed between ingests | Branch churn can silently move/rename sections (must audit) | - -### 4c. Option B — Bring in individual files - -- _How:_ copy specific docs/PDFs/articles into `raw/topics//` (or - `raw/articles/`, `raw/papers/`). - -| Pros | Cons | -| ------------------------------------------------------ | ----------------------------------------------------- | -| Minimal noise — only what you need | Provenance is manual and easy to lose | -| Fast to ingest; small footprint | No automatic re-ingest path; updates are manual | -| Good for one-off articles, papers, PDFs, mixed formats | Coverage gaps: easy to forget a related file | -| No upstream tooling comes along | No structural map to derive section→page mapping from | -| Easy when there is no single upstream repo | Harder to prove "this is the complete source set" | - -### 4d. Decision guidance - -- Whole repo when: source is a single upstream docs repo, you'll track it over - time, and reproducibility matters (the default for CNCF project docs). -- Individual files when: sources are scattered (blogs, papers, PDFs), one-off, - or you only need a slice. -- Either way: always create a `wiki/sources/.md` summary that records - what was ingested, counts, and what was deliberately excluded. - -## §5. Workflow — Ingest - -_What to write:_ step list + copyable prompts (the "sample prompts for ingest" -requirement). - -_Steps:_ read source under `raw/` → discuss key takeaways → create/update -`wiki/sources/.md` → create/update concept/entity pages → update -`wiki/index.md` → append to `wiki/log.md`. - -Sample prompt — full ingest: - -``` -Ingest the source at raw/topics//. Read it in full, then: -1. Summarize the key takeaways and propose a section→page mapping before writing. -2. Create wiki/sources/-docs.md capturing what's covered, file counts, and - any files deliberately excluded (repo meta, templates, boilerplate). -3. Create/update the concept & entity pages, one per major section. Every page must - have YAML frontmatter (title, type, sources, related, created, updated, confidence), - kebab-case -.md filenames, and [[wikilinks]] for all internal links. -4. Synthesize — condense sources into concise prose and comparison tables; do NOT copy - verbatim. Every claim must cite a raw/ path. -5. Update wiki/index.md (catalog + coverage map) and append a dated entry to the top of - wiki/log.md. -Do not edit anything under raw/. Do not run any tooling inside raw/. -``` - -Sample prompt — re-ingest after upstream changes: - -``` -The files under raw/topics// were updated. Re-ingest: -- Diff current sources against what the existing pages cover. -- Report source growth (new/renamed/removed sections) as a table. -- Re-synthesize affected pages, bump their `updated:` date, and expand `sources:`. -- Refresh index.md counts/coverage map and append a log.md entry describing the delta. -``` - -_Best practices to note:_ ask the agent to propose the mapping before writing; -ingest one topic at a time; for very large corpora, delegate sections to -sub-agents. - -## §6. Workflow — Create / Update Wiki Pages - -_What to write:_ the page contract + prompts (the "create wiki pages / update -index" requirement). - -- Frontmatter schema (show verbatim; every page starts with this): - -```yaml ---- -title: 'Human Readable Title' -type: concept # concept | entity | source-summary | comparison -sources: - - raw/topics//content/docs/latest/
/ -related: - - '[[-overview]]' -created: YYYY-MM-DD -updated: YYYY-MM-DD -confidence: high # high | medium | low ---- -``` - -- Rules: kebab-case filenames; `[[wikilinks]]` never relative paths; synthesize - don't copy; bump `updated:` on every change; `confidence:` reflects source - strength. - -Sample prompt — new concept page: - -``` -Create wiki/-.md as a `concept` page synthesizing the sources under -raw/topics//.../
/. Use the standard frontmatter, cite every claim to a -raw/ path, prefer comparison tables over prose where it aids scanning, and add [[wikilinks]] -to related pages. Then update index.md and append to log.md. -``` - -Sample prompt — update index: - -``` -Update wiki/index.md: ensure every page in wiki/ and wiki/sources/ appears in the catalog -tables with an accurate one-line summary, and refresh the coverage map (section → page, -with current file counts). Flag any orphan pages (no incoming [[links]]) or catalog entries -whose files no longer exist. -``` - -Sample prompt — append to log: - -``` -Append a new dated entry to the TOP of wiki/log.md describing this operation: operation type, -source, reason, pages created/updated, and index changes. Keep newest-on-top ordering. -``` - -## §7. Workflow — Lint (Content Consistency, not Code) - -_What to write:_ what "lint" means for a wiki + prompt. - -- Checks: contradictions across pages, orphan pages (no incoming links), - referenced-but-missing concepts (dangling `[[wikilinks]]`), and stale claims - superseded by newer sources. -- Output goes to `outputs/lint-YYYY-MM-DD.md` (not into the wiki pages - themselves). - -Sample prompt — lint: - -``` -Lint the wiki for content consistency (not code). Scan all wiki pages for: contradictions -between pages, orphan pages with no incoming [[links]], dangling wikilinks to pages that -don't exist, and stale claims superseded by newer sources. Write findings to -outputs/lint-.md with file/line references and suggested fixes. Do not modify wiki -pages in this pass. -``` - -## §8. Workflow — Query / Q&A - -_What to write:_ how analysts pull answers back out. - -- Steps: read `index.md` to find relevant pages → read them → synthesize → - answer with `[[wikilink]]` citations → offer to save a novel, valuable answer - as a new page. - -Sample prompt — query: - -``` -Using the wiki, answer: "". Start from index.md to locate relevant pages, read -them, synthesize a cited answer using [[wikilinks]], and if the answer is novel and reusable, -offer to save it as a new comparison/concept page. -``` - -## §9. Cross-Topic Hygiene & Auditing - -_What to write:_ - -- Cross-topic bleed: when a topic is dropped, audit remaining pages for stray - links to it — check both `related:` frontmatter and body content. (Reference - the KubeVirt bleed audit in `log.md` as the worked example.) -- Retain source-derived content that legitimately belongs to the kept topic even - if it _names_ the removed one (e.g. Flatcar-on-KubeVirt deploy docs stay). -- Keep `index.md` and `log.md` as the source of truth for what exists and what - happened. - -## §10. Best Practices (Cheat Sheet) - -_What to write — the "best practices" requirement, as a scannable list:_ - -- Raw is sacred. Never edit/move/delete `raw/`; never run tooling that lives - inside it. -- Cite everything. Every claim → a `raw/` path; every page → `sources:` - frontmatter. -- Synthesize, don't copy. Pages are condensed prose + tables, not reproductions. -- One source of truth. Update `index.md` and append `log.md` on _every_ - operation. -- Ask before writing. Have the agent propose a section→page mapping first. -- Pin provenance. Record upstream URL + branch/commit for reproducible - re-ingests. -- Prefer whole-repo ingest for trackable CNCF docs; files for one-offs (see §4). -- Right-size the model (§2): high-reasoning for synthesis, fast for mechanical - ops. -- Small, verifiable steps. Ingest one topic at a time; lint regularly. -- Keep planning out of the vault. Use session state, commit only deliverables. -- Version the constitution. Review `copilot-instructions.md` when conventions - change. -- Date discipline. `YYYY-MM-DD`; bump `updated:` whenever a page changes. - -## §11. Troubleshooting & FAQ - -_What to write:_ a short table. - -- Agent edited `raw/` → revert; strengthen instructions; re-run. -- Pages drifted from sources → re-ingest (§5) and diff. -- Broken `[[wikilinks]]` / orphans → run lint (§7). -- Duplicate/overlapping pages → merge, add cross-links, update `index.md`. -- Agent ran upstream Makefile/scripts → forbidden; reaffirm the "don't run - `raw/` tooling" rule. -- _TODO: add real incidents analysts hit in practice._ - -## §12. Appendix - -_What to write / attach:_ - -- Full copy of `.github/copilot-instructions.md` (or link). -- The exact model + settings your team standardized on. _TODO: fill in._ -- Glossary: ingest, synthesize, concept/entity/source-summary/comparison page, - coverage map, cross-topic bleed, orphan page. -- Links: Obsidian, Copilot CLI docs, the upstream source repos in `raw/`. From 6156c0def8c895e13e0e3299cf6815939a564b45 Mon Sep 17 00:00:00 2001 From: Bruce Hamilton Date: Tue, 7 Jul 2026 12:41:30 -0700 Subject: [PATCH 16/36] Removed Appendix A Signed-off-by: Bruce Hamilton --- docs/analysis/howto.md | 226 +++++++++++++++++++++++++++++++++++++++++ 1 file changed, 226 insertions(+) create mode 100644 docs/analysis/howto.md diff --git a/docs/analysis/howto.md b/docs/analysis/howto.md new file mode 100644 index 00000000..4b359616 --- /dev/null +++ b/docs/analysis/howto.md @@ -0,0 +1,226 @@ +--- +title: CNCF TechDocs Analysis How-To +--- + +# CNCF TechDocs Analysis How-To + +## Audience + +This document is for members of the CNCF TechDocs team and others who wish to +conduct or assist with an analysis of a CNCF open-source project's technical +documentation. + +## Purpose + +The goals of a CNCF technical documentation analysis are to: + +- Examine the current project technical documentation and website against the + CNCF's analysis [criteria]. +- Compare the documentation against the current or proposed [project maturity + level]. +- Recommend a program of key documentation improvements with the largest return + on investment. These improvements are documented as _recommendations_ in the + analysis document, and expanded in a companion + [implementation plan](./templates/implementation.md) and + [issues list](./templates/issues-list.md). + +## Doing a Tech Docs Analysis + +The tech docs analysis consists of some repository bookkeeping (prerequisites), +then of three overall tasks: + +1. Write the analysis document: evaluate the existing project documentation with + respect to the [project maturity level] or proposed maturity level, if the + analysis is associated with a project upgrade request. Identify gaps with + CNCF [criteria]. Write general recommendations to close the largest and most + important identified issues. +2. Write a implementation plan: decompose recommendations in to specific + improvement suggestions. These can be additions or revisions to the docs; + reorganization; website infrastructure changes; or any other work that will + close address issues. Make suggestions specific enough (for example, if you + propose reorganizing a section then provide an outline) without being overly + constraining so that a contributor could solve the problem differently if + they have a different solution. For example, make it clear that your proposed + outline is only one possible reorganization. +3. Write an issue backlog. + +Finally, there are follow-up steps including: + +- Creating GitHub issues and a pull request +- Getting approval from project maintainers + +### Prerequisites + +This section assumes you are familiar with GitHub repositories and pull requests +(PRs). If you need a refresher, see +[Get started](https://docs.github.com/en/get-started) from the GitHub docs. + +Project analyses are kept in the +[CNCF tech docs repository](https://github.com/cncf/techdocs). Clone and prepare +for + +1. Clone the [CNCF tech docs repository](https://github.com/cncf/techdocs). +2. Create a branch for the analysis. +3. In the new branch, create a directory for the analysis in the CNCF tech docs + [analyses] directory. Name the directory `00NN-_PROJECT_`, where _NN_ is the + next index available in the directory (check for PRs as well, if someone else + is working on tech doc analyses), and where _PROJECT_ is a short but not + abbreviated project name. For example, for Kubernetes _PROJECT_ would be + _Kubernetes_, not _k8s_. +4. Copy all the doc analysis [templates]. + +### Writing the Analysis document + +Follow the steps outlined below as a part of writing the project's analysis +document. Record your findings in the project's +[analysis.md](./templates/analysis.md) file. + +1. Define the **scope** of the analysis. Edit "Scope of analysis" to reflect + URLs and repositories included and excluded from the analysis. +2. Review the in-scope URLs and repositories for compliance with the rubric + criteria. Note any gaps, as well as any areas that exceed criteria or are + exceptionally well executed. I find it easiest to do this separately for each + of the three areas of concern (project doc, contributor doc, website), making + a pass through the documentation once for each section (Information + architecture, New user content, Content maintainability, etc.). Don't worry + about a numerical score during this step; instead, note how the documentation + complies, or not, with each criterion with respect to the project maturity + level (or proposed maturity level, if the analysis is part of a petition for + upgrade). Write comments to note the most important gaps and best-executed + features of the documentation. +3. Assign ratings to each criterion based on your comments and compliance with + the maturity level expectations in the rubric. The ratings are + self-explanatory. Keep in mind that "needs improvement" or "meets standards" + is with respect to the current (or proposed) maturity level. +4. Write recommendations. The template implies that you'll do this for every + criterion; the "Recommendations" headings mirror the "Comments" headings. + However, if some alternative framework makes more sense, use that. For + example, it might be that two or three of the product documentation criteria + are improved by reorganizing the documentation. In this case, rather than + repeat the recommendation to reorganize in each section, write a single + recommendation and explain how it improves all the areas. This is the first + step in moving from the analysis to specific, actionable, time-bound backlog + items. + +#### General tips + +Things to keep in mind while doing the analysis: + +- Look for: + - Quick wins – low-effort changes that would have a major impact. + - Large, systemic issues that you can organize a number of issues around. + - The two or three most important issues that impede documentation + effectiveness. + - Anything the project does exceptionally well. We can call these out as + examples in other evaluations! +- Don't get bogged down in detail when writing comments. Include enough detail + that you can describe how to implement a suggested solution. A sentence or two + is sufficient for most issues. +- Keep in mind the overall goal of the technical documentation: to make its + users more effective. Focus on issues that get in the way of that goal. +- It is not necessary to come up with a recommendation in every case, especially + for elements that are satisfactory or if a recommendation would result in + minimal improvement. +- Don't worry about grammar and style if they don't affect documentation + effectiveness. If writing style impedes understanding, make a note; otherwise + move on. An exception: Insist that tasks and procedures be written as clear, + step-by-step instructions. +- Some of the criteria, especially around contributor documentation, project + governance, and website infrastructure, are essentially check-boxes. These you + can quickly investigate, note, and move on. Spend more time analyzing the + effectiveness of the project documentation. + +#### Common issues + +Many common issues seem to come about because open-source software documentation +is written by software developers while they are writing the software. This +results in documentation that: + +1. Is organized around features, not users and use cases. +2. Explains technical concepts well, including architecture and design + decisions. +3. Contains complete reference information for APIs, CLIs, and configuration + parameters. +4. Has missing or incomplete user-oriented "how-to" explanations and operational + procedures. + +You may or may not find the following issues in your analysis, but it's worth +keeping them in mind: + +- Ambiguity around user roles. +- Missing or unclear task-based documentation. +- Assumptions about the reader's level of knowledge. +- Organization that buries or scatters important information, especially tasks + and procedures. +- Missing or unclear new-user workflows. + +### Writing the implementation plan + +Write the implementation plan. Edit the `_PROJECT_-implementation.md` file. + +The gist of the implementation plan is to break down the recommendations in the +analysis document. This is an intermediate stage between general recommendations +and the issues backlog. For small projects and where the recommendations are +independent and time-bound, an implementation plan might not be necessary and +you can move straight to writing the backlog. + +If you do write an implementation plan, start with recommendations in the +analysis document. Rewrite the recommendations, making them more specific, with +a suggested (but not mandatory) implementation. For example, if you recommend +reorganizing the documentation, provide a suggested outline, along with an +explanation of the reason for the reorg. + +### Writing an issues backlog + +Write an issues backlog. Edit `_PROJECT_-issues.md`. + +The goal of writing an issues backlog is to offer project contributors the +opportunity to make the recommended changes. + +Rewrite each action in the implementation plan. If possible, break large actions +into smaller issues. Each issue should be: + +- Independent. As much as possible, no issue should have another issue as a + prerequisite. A contributor should be able to choose an issue, resolve it, and + close it without reference to any other issue. +- Time-bound. A contributor should be able to complete an issue in a reasonably + short time, say a few hours or a couple of days at most. + +Make the suggested solution even more specific. At this point, the issue should +almost be a recipe for making the doc improvement, with the caveat that a +contributor is not required to implement the solution as suggested in the issue. + +## Next Steps + +### Including supporting documentation + +If you have supporting material that might be helpful to a contributor working +on the documentation issues, include them in the directory with the other +documents. For example, you might inventory the existing tech doc pages in a +spreadsheet; in this case, include a CSV file of the inventory. + +### Creating a pull request + +If you have not created a pull request with the analysis documents, do so now. +Tag project maintainers and CNCF documentation staff, and ask for comments. + +### Getting contributor feedback + +If you haven't met with the project's maintainers yet, do so before you create +the issues in GitHub. Ideally you'd like to have a Zoom meeting with any +interested parties to get feedback on the analysis and implementation plan. + +### Creating GitHub issues + +Create issues in the project documentation GitHub repository for: + +- Each issues in the [issues list]. +- An umbrella issue that provides a context for the previously created + individual issues. + +[analyses]: ../../analyses +[criteria]: ./criteria.md +[project maturity level]: https://www.cncf.io/project-metrics +[templates]: ./templates/index.md +[issues list]: ./templates/issues-list.md + From 59fd2c28e8cd83abec777e561ecab2edc0b2efc7 Mon Sep 17 00:00:00 2001 From: Bruce Hamilton Date: Tue, 7 Jul 2026 14:39:02 -0700 Subject: [PATCH 17/36] format edits Signed-off-by: Bruce Hamilton --- docs/analysis/howto.md | 1 - 1 file changed, 1 deletion(-) diff --git a/docs/analysis/howto.md b/docs/analysis/howto.md index 4b359616..1a00c5ec 100644 --- a/docs/analysis/howto.md +++ b/docs/analysis/howto.md @@ -223,4 +223,3 @@ Create issues in the project documentation GitHub repository for: [project maturity level]: https://www.cncf.io/project-metrics [templates]: ./templates/index.md [issues list]: ./templates/issues-list.md - From 999b8ca48204bb48d9491f82389ffda8392d0f93 Mon Sep 17 00:00:00 2001 From: Bruce Hamilton Date: Mon, 5 Oct 2026 13:37:20 -0700 Subject: [PATCH 18/36] added ai section, criteria questions in website section Signed-off-by: Bruce Hamilton --- docs/analysis/templates/analysis.md | 21 +++++++++++++++++++++ 1 file changed, 21 insertions(+) diff --git a/docs/analysis/templates/analysis.md b/docs/analysis/templates/analysis.md index 8acb9fa2..9b153e6a 100644 --- a/docs/analysis/templates/analysis.md +++ b/docs/analysis/templates/analysis.md @@ -43,6 +43,17 @@ prerequisite for program graduation. The documentation analysis is the first step of a CNCF process aimed at assisting projects with their documentation efforts. +### AI assisted analysis + +> Author's note: Remove if not applicable, otherwise edit as needed. + +The following parts of this analysis were generated by AI: + +- Overall comments on the areas in a section, that follow the section heading. +- Answers to the criteria questions for an area. +- Comments including strengths and weaknesss for an area. +- Recommendations for improvment for each area. + ### Purpose This document was written to analyze the current state of _PROJECT_ @@ -414,6 +425,11 @@ submodules][git-submodules]. If a project chooses to keep source files in multiple repos, they need a clearly documented strategy for managing mirrored files and new contributions. +We evaluate on the following: + +- Does the project have a single source for its documentation? If not, is there + a reason? + #### Minimal website requirements Listed here are the minimal website requirements for projects based on their @@ -437,6 +453,11 @@ only two levels for which a tech docs analysis can be requested.) https://github.com/cncf/toc/tree/main/process#ii-stages---definitions--expectations [cncf-servicedesk]: https://servicedesk.cncf.io +We evaluate on the following: + +- Are most of the applicable guidelines satisfied as described in the + [Webiste guidelines & checklist](https://github.com/cncf/techdocs/blob/main/docs/website-guidelines-checklist.md)? + #### Usability, accessibility and devices Most CNCF websites are accessed from mobile and other non-desktop devices at From 36636870dae4d2a0a5bbb246fcba0213c42faeff Mon Sep 17 00:00:00 2001 From: Bruce Hamilton Date: Mon, 5 Oct 2026 13:48:14 -0700 Subject: [PATCH 19/36] removed website checklist link for now Signed-off-by: Bruce Hamilton --- docs/analysis/templates/analysis.md | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/docs/analysis/templates/analysis.md b/docs/analysis/templates/analysis.md index 9b153e6a..5eba3b87 100644 --- a/docs/analysis/templates/analysis.md +++ b/docs/analysis/templates/analysis.md @@ -455,8 +455,8 @@ only two levels for which a tech docs analysis can be requested.) We evaluate on the following: -- Are most of the applicable guidelines satisfied as described in the - [Webiste guidelines & checklist](https://github.com/cncf/techdocs/blob/main/docs/website-guidelines-checklist.md)? +- Are most of the applicable guidelines satisfied as described in the Webiste + guidelines and checklist? #### Usability, accessibility and devices From f91ecb55190b5d6513fea9ec5b8b1fa1dc1085b4 Mon Sep 17 00:00:00 2001 From: Bruce Hamilton Date: Mon, 5 Oct 2026 16:54:56 -0700 Subject: [PATCH 20/36] changed 'your website' to 'the website' Signed-off-by: Bruce Hamilton --- docs/analysis/templates/analysis.md | 38 ++++++++++++++--------------- 1 file changed, 19 insertions(+), 19 deletions(-) diff --git a/docs/analysis/templates/analysis.md b/docs/analysis/templates/analysis.md index 5eba3b87..4ebc20e4 100644 --- a/docs/analysis/templates/analysis.md +++ b/docs/analysis/templates/analysis.md @@ -191,7 +191,7 @@ Documentation rubric. #### Information architecture -The overall structure (pages/subpages/sections/subsections) of your project +The overall structure (pages/subpages/sections/subsections) of the project documentation. We evaluate on the following: - Is there high level conceptual/“About” content? Is the documentation feature @@ -218,8 +218,8 @@ specifically for them. We evaluate on the following: - Is installation documented step-by-step? - If needed, are multiple OSes documented? - Do users know where to go after reading the getting started guide? -- Is your new user content clearly signposted on your site’s homepage or at the - top of your information architecture? +- Is the new user content clearly signposted on the site’s homepage or at the + top of the information architecture? - Is there sample code or other example content that can easily be copy-pasted? #### Content maintainability & site mechanics @@ -229,10 +229,10 @@ become large maintenance burdens, particularly if you don’t plan for them. We evaluate on the following: -- Is your documentation searchable? +- Is the documentation searchable? - Are you planning for localization/internationalization with regards to site directory structure? Is a localization framework present? -- Do you have a clearly documented method for versioning your content? +- Do you have a clearly documented method for versioning the content? #### Content creation processes @@ -243,7 +243,7 @@ We evaluate on the following: - Is there a clearly documented (ongoing) contribution process for documentation? -- Does your code release process account for documentation creation & updates? +- Does the code release process account for documentation creation & updates? - Who reviews and approves documentation pull requests? - Does the website have a clear owner/maintainer? @@ -314,8 +314,8 @@ to reach you. We evaluate on the following: - Is there a Slack/Discord/Discourse/etc. community and is it prominently linked - from your website? -- Is there a direct link to your GitHub organization/repository? + from the website? +- Is there a direct link to the GitHub organization/repository? - Are weekly/monthly project meetings documented? Is it clear how someone can join those meetings? - Are mailing lists documented? @@ -338,8 +338,8 @@ in easily? We evaluate on the following: -- Do you have a community repository or section on your website? -- Is there a document specifically for new contributors/your first contribution? +- Do you have a community repository or section on the website? +- Is there a document specifically for new contributors/the first contribution? - Do new users know where to get help? #### Project governance documentation @@ -461,19 +461,19 @@ We evaluate on the following: #### Usability, accessibility and devices Most CNCF websites are accessed from mobile and other non-desktop devices at -least 10-20% of the time. Planning for this early in your website's design will +least 10-20% of the time. Planning for this early in the website's design will be much less effort than retrofitting a desktop-first design. - Is the website usable from mobile? - Are doc pages readable? - Are all / most website features accessible from mobile -- such as the top-nav, site search and in-page table of contents? -- Might a [mobile-first] design make sense for your project? +- Might a [mobile-first] design make sense for the project? [mobile-first]: https://developer.mozilla.org/en-US/docs/Web/Progressive_web_apps/Responsive/Mobile_first -Plan for suitable [accessibility][] measures for your website. For example: +Plan for suitable [accessibility][] measures for the website. For example: - Are color contrasts significant enough for color-impaired readers? - Are most website features usable using a keyboard only? @@ -511,16 +511,16 @@ We evaluate on the following: #### SEO, Analytics and site-local search -SEO helps users find your project and it's documentation, and analytics helps -you monitor site traffic and diagnose issues like page 404s. Intra-site search, -while optional, can offer your readers a site-focused search results. +SEO helps users find the project and it's documentation, and analytics helps you +monitor site traffic and diagnose issues like page 404s. Intra-site search, +while optional, can offer the readers a site-focused search results. We evaluate on the following: - Analytics: - Is analytics enabled for the production server? - Is analytics disabled for all other deploys? - - If your project used Google Analytics, have you migrated to GA4? + - If the project used Google Analytics, have you migrated to GA4? - Can Page-not-found (404) reports easily be generated from you site analytics? Provide a sample of the site's current top-10 404s. - Is site indexing supported for the production server, while disabled for @@ -536,7 +536,7 @@ project maintainers aren’t web developers. We evaluate on the following: -- Is your website tooling well supported by the community (i.e., Hugo with the +- Is the website tooling well supported by the community (i.e., Hugo with the Docsy theme) or commonly used by CNCF projects (our recommended tech stack?) - Are you actively cultivating website maintainers from within the community? - Are site build times reasonable? @@ -544,7 +544,7 @@ We evaluate on the following: #### Other -- Is your website accessible via HTTPS? +- Is the website accessible via HTTPS? - Does HTTP access, if any, redirect to HTTPS? ### Recommendations From c28c6ea41807ccf0ac079bc133906a02fe601a76 Mon Sep 17 00:00:00 2001 From: Bruce Hamilton Date: Mon, 5 Oct 2026 17:16:53 -0700 Subject: [PATCH 21/36] spelling fixes Signed-off-by: Bruce Hamilton --- docs/analysis/templates/analysis.md | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/docs/analysis/templates/analysis.md b/docs/analysis/templates/analysis.md index 4ebc20e4..f4efc73a 100644 --- a/docs/analysis/templates/analysis.md +++ b/docs/analysis/templates/analysis.md @@ -51,8 +51,8 @@ The following parts of this analysis were generated by AI: - Overall comments on the areas in a section, that follow the section heading. - Answers to the criteria questions for an area. -- Comments including strengths and weaknesss for an area. -- Recommendations for improvment for each area. +- Comments including strengths and weaknesses for an area. +- Recommendations for improvement for each area. ### Purpose @@ -455,7 +455,7 @@ only two levels for which a tech docs analysis can be requested.) We evaluate on the following: -- Are most of the applicable guidelines satisfied as described in the Webiste +- Are most of the applicable guidelines satisfied as described in the Website guidelines and checklist? #### Usability, accessibility and devices From 906ee1e936b4fc712f45519c21c5417d0f225a52 Mon Sep 17 00:00:00 2001 From: Bruce Hamilton Date: Tue, 6 Oct 2026 18:42:27 -0700 Subject: [PATCH 22/36] minor edit to rebuild Signed-off-by: Bruce Hamilton --- docs/analysis/templates/analysis.md | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/docs/analysis/templates/analysis.md b/docs/analysis/templates/analysis.md index f4efc73a..53f6618d 100644 --- a/docs/analysis/templates/analysis.md +++ b/docs/analysis/templates/analysis.md @@ -49,10 +49,10 @@ efforts. The following parts of this analysis were generated by AI: -- Overall comments on the areas in a section, that follow the section heading. +- Overall comments, following the section heading, on the areas in that section. - Answers to the criteria questions for an area. -- Comments including strengths and weaknesses for an area. -- Recommendations for improvement for each area. +- Comments, including strengths and weaknesses, for an area. +- Recommendations for improvements in each area. ### Purpose From c076f7085faffb71ae27cc72cef18bc326adfdfd Mon Sep 17 00:00:00 2001 From: Patrice Chalin Date: Wed, 7 Oct 2026 21:11:17 -0400 Subject: [PATCH 23/36] Ignore Slack links in link check: bot wall yields 403 Signed-off-by: Patrice Chalin --- .htmltest.yml | 3 +++ 1 file changed, 3 insertions(+) diff --git a/.htmltest.yml b/.htmltest.yml index 128bc970..16284bcb 100644 --- a/.htmltest.yml +++ b/.htmltest.yml @@ -9,6 +9,9 @@ IgnoreInternalURLs: # list of paths IgnoreURLs: # list of regexes of URLs or path to be ignored - ^https?://localhost - \?no-link-check + # Bot wall: Slack returns 403 to htmltest's user agent while the workspace + # and channel pages load in a browser. + - ^https://cloud-native\.slack\.com # Old analysis pages are not updated. Ignore broken links: - ^https://(docs|www).tremor.rs/ - ^https://knative.dev/docs/(concepts|reference|relnotes)/ From 8ceb059b019e28b3b72b4cd34add8286362de706 Mon Sep 17 00:00:00 2001 From: Patrice Chalin Date: Wed, 7 Oct 2026 21:12:20 -0400 Subject: [PATCH 24/36] Align Slack ignore rationale with docsy.dev rule Signed-off-by: Patrice Chalin --- .htmltest.yml | 5 +++-- 1 file changed, 3 insertions(+), 2 deletions(-) diff --git a/.htmltest.yml b/.htmltest.yml index 16284bcb..8a3daa8d 100644 --- a/.htmltest.yml +++ b/.htmltest.yml @@ -9,8 +9,9 @@ IgnoreInternalURLs: # list of paths IgnoreURLs: # list of regexes of URLs or path to be ignored - ^https?://localhost - \?no-link-check - # Bot wall: Slack returns 403 to htmltest's user agent while the workspace - # and channel pages load in a browser. + # Bot wall (403): sign-in wall for members and strangers alike, so channel + # existence is unverifiable. Same rule as docsy.dev's lychee.toml, widened to + # the whole host since the workspace root 403s too. - ^https://cloud-native\.slack\.com # Old analysis pages are not updated. Ignore broken links: - ^https://(docs|www).tremor.rs/ From eb0d6d9c7bd78dcfc81d2f5d95c38bc7e1bce27b Mon Sep 17 00:00:00 2001 From: Bruce Hamilton Date: Fri, 9 Oct 2026 15:10:36 -0700 Subject: [PATCH 25/36] added max toc 4 to metadata Signed-off-by: Bruce Hamilton --- docs/analysis/templates/analysis.md | 1 + 1 file changed, 1 insertion(+) diff --git a/docs/analysis/templates/analysis.md b/docs/analysis/templates/analysis.md index 53f6618d..5df43f8a 100644 --- a/docs/analysis/templates/analysis.md +++ b/docs/analysis/templates/analysis.md @@ -3,6 +3,7 @@ title: _PROJECT_ Documentation Analysis tags: [_PROJECT_] created: YYYY-MM-DD modified: YYYY-MM-DD +toc_max_heading_level: 4 author: _NAME_ (@_HANDLE_) --- From aa210c1a87ba2da96609f7e376d5115066911fcf Mon Sep 17 00:00:00 2001 From: Bruce Hamilton Date: Sat, 10 Oct 2026 20:10:52 -0700 Subject: [PATCH 26/36] formatting and consistency edits Signed-off-by: Bruce Hamilton --- docs/analysis/templates/analysis.md | 70 ++++++++++++++--------------- 1 file changed, 35 insertions(+), 35 deletions(-) diff --git a/docs/analysis/templates/analysis.md b/docs/analysis/templates/analysis.md index 5df43f8a..1d813e53 100644 --- a/docs/analysis/templates/analysis.md +++ b/docs/analysis/templates/analysis.md @@ -11,26 +11,26 @@ author: _NAME_ (@_HANDLE_) ## About this template -TO USE THIS TEMPLATE, search and replace the named IDs: - -- `_PROJECT_`: project name -- `YYYY-MM-DD`: creation and modification dates of the analysis document -- `_NAME_`: name of the analysis author -- `@_HANDLE_`: GitHub handle of the analysis author -- `_PROJECT-WEBSITE_`: landing page of the project's information website -- `_PROJECT-DOC-URL_`: main page of the technical documentation for the current - project revision; this might be on the main website server, for example as - _PROJECT-WEBSITE_/doc -- `_PROJECT-DOC-REPO_`: repository where the project technical documentation is - stored; this might be its own repo or a directory in the project main repo +To use this template, use a copy of this file with the same `analysis.md` +filename and search and replace the named placeholder IDs: + +| ID | Description | +| -------------------- | --------------------------------------------------- | +| `_PROJECT_` | Project name | +| `YYYY-MM-DD` | Creation & modification dates of this document | +| `_NAME_` | Author's, technical writer, name | +| `@_HANDLE_` | Author's GitHub handle | +| `_PROJECT-WEBSITE_` | Main Landing page of the project's website | +| `_PROJECT-DOC-URL_` | Main page of the Project's documentation | +| `_PROJECT-DOC-REPO_` | GitHub repository where the project's documentation | Replace link placeholders in brackets, such as [_PROJECT_][project-website], with a Markdown-style link. For the analysis procedure, see [Analysis how-to](../howto.md). -> Note: delete this "About this template" section after you have customized this -> template for a specific project. +> Author's Note: Delete this "About this template" section after you have +> customized this template for a specific project. ## Introduction @@ -160,12 +160,12 @@ to legal requirements such as copyright and licensing issues. ## Project documentation -> AUTHOR NOTE: Pick the CNCF maturity level of the project: +> Author's Note: Pick the CNCF maturity level of the project: _PROJECT_ is a **graduated** project of CNCF. This means that the project should have [_very high_][criteria] standards for documentation. -> AUTHOR NOTE: or +> Author's Note: or _PROJECT_ is an **incubating** project of CNCF. This means that the project should be [_developing_][criteria] professional-quality documentation alongside @@ -181,13 +181,13 @@ the project code. ### Comments -> AUTHOR NOTE: make any overall comments about the Project Documentation here. +> Author's Note: make any overall comments about the Project Documentation here. The following sections contain brief assessments of each element of the Project Documentation rubric. -> AUTHOR NOTE: For each heading below, discuss how well the in-scope items, and -> especially the technical documentation, meet these criteria. (Criteria are +> Author's Note: For each heading below, discuss how well the in-scope items, +> and especially the technical documentation, meet these criteria. (Criteria are > copied from criteria.md) #### Information architecture @@ -261,7 +261,7 @@ We evaluate on the following: ### Recommendations -> AUTHOR NOTE: Write general recommendations based on the comments from the +> Author's Note: Write general recommendations based on the comments from the > previous section. #### Information architecture @@ -276,12 +276,12 @@ We evaluate on the following: ## Contributor documentation -> AUTHOR NOTE: Pick the CNCF maturity level of the project: +> Author's Note: Pick the CNCF maturity level of the project: _PROJECT_ is a **graduated** project of CNCF. This means that the project should have [_very high_][criteria] standards for documentation. -> AUTHOR NOTE: or +> Author's Note: or _PROJECT_ is an **incubating** project of CNCF. This means that the project should be [_developing_][criteria] professional-quality documentation alongside @@ -296,15 +296,15 @@ the project code. ### Comments -> AUTHOR NOTE: make any overall comments about the Contributor Documentation +> Author's Note: make any overall comments about the Contributor Documentation > here. The following sections contain brief assessments of each element of the Contributor Documentation rubric. -> AUTHOR NOTE: For each heading below, discuss how well the in-scope items meet -> these criteria. Keep in mind that much of the contributor documentation might -> be contained in the documentation repository. (Criteria are copied from +> Author's Note: For each heading below, discuss how well the in-scope items +> meet these criteria. Keep in mind that much of the contributor documentation +> might be contained in the documentation repository. (Criteria are copied from > criteria.md) #### Communication methods documented @@ -353,7 +353,7 @@ We evaluate on the following: ### Recommendations -> AUTHOR NOTE: Write general recommendations based on the comments from the +> Author's Note: Write general recommendations based on the comments from the > previous section. #### Communication methods documented @@ -366,12 +366,12 @@ We evaluate on the following: ## Website and infrastructure -> AUTHOR NOTE: Pick the CNCF maturity level of the project: +> Author's Note: Pick the CNCF maturity level of the project: _PROJECT_ is a **graduated** project of CNCF. This means that the project should have [_very high_][criteria] standards for documentation. -> AUTHOR NOTE: or +> Author's Note: or _PROJECT_ is an **incubating** project of CNCF. This means that the project should be [_developing_][criteria] professional-quality documentation alongside @@ -396,16 +396,16 @@ the project code. ### Comments -> AUTHOR NOTE: make any overall comments about the Website and documentation +> Author's Note: make any overall comments about the Website and documentation > infrastructure here. The following sections contain brief assessments of each element of the Website and documentation infrastructure rubric. -> AUTHOR NOTE: for each heading below, discuss how well the in-scope items meet -> these criteria. Keep in mind that much of the website infrastructure criteria -> depend on the tools (static site generator, website framework and hosting, -> analytics tools, etc.) and processes (project CI, release procedures, +> Author's Note: for each heading below, discuss how well the in-scope items +> meet these criteria. Keep in mind that much of the website infrastructure +> criteria depend on the tools (static site generator, website framework and +> hosting, analytics tools, etc.) and processes (project CI, release procedures, > governance, etc.) used to produce the documentation. (Criteria are copied from > criteria.md) @@ -550,7 +550,7 @@ We evaluate on the following: ### Recommendations -> AUTHOR NOTE: Write general recommendations based on the comments from the +> Author's Note: Write general recommendations based on the comments from the > previous section. #### Single-source requirement From 138d86adc86a6a569e3ccec8db988a873094614a Mon Sep 17 00:00:00 2001 From: Bruce Hamilton Date: Sat, 10 Oct 2026 20:23:49 -0700 Subject: [PATCH 27/36] Trigger build after timeout Signed-off-by: Bruce Hamilton From 182d8a8eacbd2382ffbd4d0e66c065e37bdafb21 Mon Sep 17 00:00:00 2001 From: Bruce Hamilton Date: Sat, 17 Oct 2026 17:47:43 -0700 Subject: [PATCH 28/36] processsed suggestions Signed-off-by: Bruce Hamilton --- docs/analysis/criteria.md | 43 ++++++++++++---------- docs/analysis/templates/analysis.md | 56 +++++++++++++++++++---------- 2 files changed, 62 insertions(+), 37 deletions(-) diff --git a/docs/analysis/criteria.md b/docs/analysis/criteria.md index b71d9610..5ccd71ce 100644 --- a/docs/analysis/criteria.md +++ b/docs/analysis/criteria.md @@ -42,6 +42,11 @@ We evaluate on the following: - Is content up to date and accurate? - Does the documentation separate conceptual, instructional, and reference information? +- If the product builds on or incorporates another product (such as Kubernetes), + are there external references where needed to install and configure those + products? +- Does the documentation organization and navigation need a high-level + restructuring? Example: @@ -128,23 +133,6 @@ We evaluate on the following: [Inclusive Naming Initiative](https://inclusivenaming.org) website? - Does the project use language like "simple", "easy", etc.? -### AI Optimization and Discoverability - -Tools and techniques are emerging that enable AI agents to use documentation as -input. For example, many repositories now include "AI-friendly" Markdown files -that enable LLMs to ingest content. We encourage projects to optimize for -AI-based documentation use. - -We evaluate on the following: - -- Is there an llms.txt file in the root of the website or documentation - repository? - - The leading proposal for an AI-enabling file for websites is at - [llms-txt](https://llmstxt.org/ llms.txt). - - Task can be accomplished manually or by tools. - ## Contributor documentation ### Communication methods documented @@ -352,7 +340,7 @@ Example: - https://helm.sh/ -### Case studies/social proof +### Case studies and social proof One of the best ways to advertise an open source project is to show other organizations using it. @@ -386,7 +374,7 @@ We evaluate on the following: - Is analytics disabled for all other deploys? - If your project used Google Analytics, have you migrated to GA4? - Can Page-not-found (404) reports easily be generated from you site - analytics? Provide a sample of the site's current top-10 404s. + analytics? - Is site indexing supported for the production server, while disabled for website previews and builds for non-default branches? - Is local intra-site search available from the website? @@ -410,6 +398,23 @@ Example: - https://kubernetes.io +### AI Optimization and Discoverability + +Tools and techniques are emerging that enable AI agents to use documentation as +input. For example, many repositories now include "AI-friendly" Markdown files +that enable LLMs to ingest content. We encourage projects to optimize for +AI-based documentation use. + +We evaluate on the following: + +- Is there an llms.txt file in the root of the website or documentation + repository? + + The leading proposal for an AI-enabling file for websites is at + [llms-txt](https://llmstxt.org/ llms.txt). + + Task can be accomplished manually or by tools. + ### Other - Is your website accessible via HTTPS? diff --git a/docs/analysis/templates/analysis.md b/docs/analysis/templates/analysis.md index 1d813e53..4639dfb3 100644 --- a/docs/analysis/templates/analysis.md +++ b/docs/analysis/templates/analysis.md @@ -24,9 +24,6 @@ filename and search and replace the named placeholder IDs: | `_PROJECT-DOC-URL_` | Main page of the Project's documentation | | `_PROJECT-DOC-REPO_` | GitHub repository where the project's documentation | -Replace link placeholders in brackets, such as [_PROJECT_][project-website], -with a Markdown-style link. - For the analysis procedure, see [Analysis how-to](../howto.md). > Author's Note: Delete this "About this template" section after you have @@ -59,8 +56,8 @@ The following parts of this analysis were generated by AI: This document was written to analyze the current state of _PROJECT_ documentation. It aims to provide project leaders with an informed understanding -of potential problems in current project documentation. A second -[implementation][] document, , outlines an actionable plan for improvement. A +of potential problems in current project documentation. Optionally, a second +[implementation][] document, outlines an actionable plan for improvement and a third document is an [issues list][] of issues to be added to the project documentation repository. These issues can be taken up by contributors to improve the documentation. @@ -107,8 +104,9 @@ concern: - **Website:** concerns the mechanics of publishing the documentation, and includes branding, website structure, and maintainability. -A fourth area of concern, AI optimization and discovery, might be included in -this analysis. If so, it is discussed in a separate document, `ai-guidance.md`. +A fourth area of concern, AI-assisted analysis, might be included in this +analysis. If so, it is discussed in a separate document, +`ai-assisted-analysis.md`. Each section begins with summary ratings based on a rubric with appropriate [criteria] for the section, then proceeds to: @@ -124,16 +122,16 @@ Each section begins with summary ratings based on a rubric with appropriate > repo. Remove the sentences about the implementation doc if you didn't write > one. -The accompanying [implementation] document breaks the recommendations down into -concrete actions that can be implemented by project contributors. Its focus is -on drilling down to specific, achievable work that can be completed in +An optional accompanying [implementation] document breaks the recommendations +down into concrete actions that can be implemented by project contributors. Its +focus is on drilling down to specific, achievable work that can be completed in constrained blocks of time. Ultimately, the implementation items are decomposed into a series of issues and entered on GitHub `PROJECT-DOC-WEBSITE/issues`. ### How to use this document -Readers interested only in actionable improvements should skip this document and -read the [implementation][] plan and [issues list][]. +If provided, readers interested only in actionable improvements should skip this +document and read the [implementation][] plan and [issues list][]. Readers interested in the current state of the documentation and the reasoning behind the recommendations should read the section of this document pertaining @@ -208,6 +206,11 @@ documentation. We evaluate on the following: users needing more help? (FAQ, Troubleshooting) - If the product exposes an API, is there a complete reference? - Is content up to date and accurate? +- If the product builds on or incorporates another product (such as Kubernetes), + are there external references where needed to install and configure those + products? +- Does the documentation organization and navigation need a high-level + restructuring? #### New user content @@ -242,8 +245,8 @@ requires the same kind of review and approval processes as code. We evaluate on the following: -- Is there a clearly documented (ongoing) contribution process for - documentation? +- Is there a clear path to the contributor documentation for new users who want + to contribute to the project? - Does the code release process account for documentation creation & updates? - Who reviews and approves documentation pull requests? - Does the website have a clear owner/maintainer? @@ -383,7 +386,7 @@ the project code. | Meets min website req. (for maturity level) | [rating (1-5)] | | Usability, accessibility, and design | [rating (1-5)] | | Branding and design | [rating (1-5)] | -| Case studies/social proof | [rating (1-5)] | +| Case studies & social proof | [rating (1-5)] | | SEO, Analytics, and site-local search | [rating (1-5)] | | Maintenance planning | [rating (1-5)] | | A11y plan & implementation | [rating (1-5)] | @@ -496,7 +499,7 @@ We evaluate on the following: - Is the brand used across the website consistently? - Is the website’s typography clean and well-suited for reading? -#### Case studies/social proof +#### Case studies and social proof One of the best ways to advertise an open source project is to show other organizations using it. @@ -523,7 +526,7 @@ We evaluate on the following: - Is analytics disabled for all other deploys? - If the project used Google Analytics, have you migrated to GA4? - Can Page-not-found (404) reports easily be generated from you site - analytics? Provide a sample of the site's current top-10 404s. + analytics? - Is site indexing supported for the production server, while disabled for website previews and builds for non-default branches? - Is local intra-site search available from the website? @@ -543,6 +546,23 @@ We evaluate on the following: - Are site build times reasonable? - Do site maintainers have adequate permissions? +#### AI Optimization and Discoverability + +Tools and techniques are emerging that enable AI agents to use documentation as +input. For example, many repositories now include "AI-friendly" Markdown files +that enable LLMs to ingest content. We encourage projects to optimize for +AI-based documentation use. + +We evaluate on the following: + +- Is there an llms.txt file in the root of the website or documentation + repository? + + The leading proposal for an AI-enabling file for websites is at + [llms-txt](https://llmstxt.org/ llms.txt). + + Task can be accomplished manually or by tools. + #### Other - Is the website accessible via HTTPS? @@ -561,7 +581,7 @@ We evaluate on the following: #### Branding and design -#### Case studies/social proof +#### Case studies and social proof #### SEO, Analytics and site-local search From 3ead2e861e5e782bf93950f27357184e08d37def Mon Sep 17 00:00:00 2001 From: Bruce Hamilton Date: Sun, 18 Oct 2026 10:56:28 -0700 Subject: [PATCH 29/36] processed copilot suggestions Signed-off-by: Bruce Hamilton --- docs/analysis/criteria.md | 4 ++-- docs/analysis/templates/analysis.md | 4 ++-- 2 files changed, 4 insertions(+), 4 deletions(-) diff --git a/docs/analysis/criteria.md b/docs/analysis/criteria.md index 5ccd71ce..117197bc 100644 --- a/docs/analysis/criteria.md +++ b/docs/analysis/criteria.md @@ -270,7 +270,7 @@ requirements for sandbox projects._ - Does **project documentation** meet these standards?: - Stakeholders / personas are identified and their core needs (in terms of docs) documented. - - Hosted directly on the website. + - Hosted, internally or externally, on the website. - The [single-source requirement][] met. - Doc is comprehensive, addressing most stakeholder needs. @@ -411,7 +411,7 @@ We evaluate on the following: repository? The leading proposal for an AI-enabling file for websites is at - [llms-txt](https://llmstxt.org/ llms.txt). + [llms-txt](https://llmstxt.org/). Task can be accomplished manually or by tools. diff --git a/docs/analysis/templates/analysis.md b/docs/analysis/templates/analysis.md index 4639dfb3..9b7a0ee8 100644 --- a/docs/analysis/templates/analysis.md +++ b/docs/analysis/templates/analysis.md @@ -459,7 +459,7 @@ only two levels for which a tech docs analysis can be requested.) We evaluate on the following: -- Are most of the applicable guidelines satisfied as described in the Website +- Are all of the applicable guidelines satisfied as described in the Website guidelines and checklist? #### Usability, accessibility and devices @@ -559,7 +559,7 @@ We evaluate on the following: repository? The leading proposal for an AI-enabling file for websites is at - [llms-txt](https://llmstxt.org/ llms.txt). + [llms-txt](https://llmstxt.org). Task can be accomplished manually or by tools. From 442333fac1d9755141b79434f928cfa6ce8ab701 Mon Sep 17 00:00:00 2001 From: Bruce Hamilton Date: Sun, 18 Oct 2026 11:28:10 -0700 Subject: [PATCH 30/36] removed unneeded H1 Signed-off-by: Bruce Hamilton --- docs/analysis/criteria.md | 2 -- 1 file changed, 2 deletions(-) diff --git a/docs/analysis/criteria.md b/docs/analysis/criteria.md index 117197bc..56e9871b 100644 --- a/docs/analysis/criteria.md +++ b/docs/analysis/criteria.md @@ -2,8 +2,6 @@ title: Assessment criteria and examples --- -# Assessment criteria and examples - We recommend reading the [CNCF sandbox project documentation primer](../sandbox-doc-primer.md) before reviewing these criteria. The primer contains key definitions concepts that are From 0601b0e5979dc5165c20ec1b2506055de429642d Mon Sep 17 00:00:00 2001 From: Bruce Hamilton Date: Sun, 18 Oct 2026 22:02:05 -0700 Subject: [PATCH 31/36] minor edits Signed-off-by: Bruce Hamilton --- docs/analysis/templates/analysis.md | 20 +++++++++++--------- 1 file changed, 11 insertions(+), 9 deletions(-) diff --git a/docs/analysis/templates/analysis.md b/docs/analysis/templates/analysis.md index 9b7a0ee8..d0c294d6 100644 --- a/docs/analysis/templates/analysis.md +++ b/docs/analysis/templates/analysis.md @@ -31,7 +31,7 @@ For the analysis procedure, see [Analysis how-to](../howto.md). ## Introduction -This document is an analyzes the effectiveness and completeness of the +This document is an analysis the effectiveness and completeness of the [_PROJECT_][project-website] open source software (OSS) project's documentation and website. It is funded by the CNCF Foundation as part of its overall effort to incubate, grow, and graduate open source cloud native software projects. @@ -43,7 +43,8 @@ efforts. ### AI assisted analysis -> Author's note: Remove if not applicable, otherwise edit as needed. +> Author's note: Remove this section if not applicable, otherwise edit as +> needed. The following parts of this analysis were generated by AI: @@ -64,9 +65,10 @@ improve the documentation. This document: -- Analyzes the current _PROJECT_ technical documentation and website -- Compares existing documentation against the CNCF’s standards -- Recommends a program of key improvements with the largest return on investment +- Analyzes the current _PROJECT_ technical documentation and website. +- Compares existing documentation against the CNCF’s standards. +- Recommends a program of key improvements with the largest return on + investment. ### Scope of analysis @@ -105,14 +107,14 @@ concern: includes branding, website structure, and maintainability. A fourth area of concern, AI-assisted analysis, might be included in this -analysis. If so, it is discussed in a separate document, -`ai-assisted-analysis.md`. +analysis. Each section begins with summary ratings based on a rubric with appropriate [criteria] for the section, then proceeds to: -- **Comments**: observations about the existing documentation, with a focus on - how it does or does not help _PROJECT_ users achieve their goals. +- **Comments**: observations about the existing documentation, including answers + to critera questions from the criteria specfication for the current area. Comments focus on how it does or does + not help _PROJECT_ users achieve their goals. - **Recommendations**: suggested changes that would improve the effectiveness of the documentation. From c5ca587491215ede64723d59deb13c8f9c8bf282 Mon Sep 17 00:00:00 2001 From: Bruce Hamilton Date: Sun, 18 Oct 2026 22:11:45 -0700 Subject: [PATCH 32/36] spelling fix Signed-off-by: Bruce Hamilton --- docs/analysis/templates/analysis.md | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/docs/analysis/templates/analysis.md b/docs/analysis/templates/analysis.md index d0c294d6..f041dd61 100644 --- a/docs/analysis/templates/analysis.md +++ b/docs/analysis/templates/analysis.md @@ -113,8 +113,8 @@ Each section begins with summary ratings based on a rubric with appropriate [criteria] for the section, then proceeds to: - **Comments**: observations about the existing documentation, including answers - to critera questions from the criteria specfication for the current area. Comments focus on how it does or does - not help _PROJECT_ users achieve their goals. + to questions from the criteria specfication. Comments focus on how it does or + does not help _PROJECT_ users achieve their goals. - **Recommendations**: suggested changes that would improve the effectiveness of the documentation. From f2d0109fe796a96e45e3c070f5e064c4ca012d9b Mon Sep 17 00:00:00 2001 From: Bruce Hamilton Date: Sun, 18 Oct 2026 22:21:43 -0700 Subject: [PATCH 33/36] spelling fix Signed-off-by: Bruce Hamilton --- docs/analysis/templates/analysis.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/analysis/templates/analysis.md b/docs/analysis/templates/analysis.md index f041dd61..ce6c5fb8 100644 --- a/docs/analysis/templates/analysis.md +++ b/docs/analysis/templates/analysis.md @@ -113,7 +113,7 @@ Each section begins with summary ratings based on a rubric with appropriate [criteria] for the section, then proceeds to: - **Comments**: observations about the existing documentation, including answers - to questions from the criteria specfication. Comments focus on how it does or + to questions from the criteria specification. Comments focus on how it does or does not help _PROJECT_ users achieve their goals. - **Recommendations**: suggested changes that would improve the effectiveness of the documentation. From f279fc70718b22a5873e05372ad21c162a50c63f Mon Sep 17 00:00:00 2001 From: Bruce Hamilton Date: Thu, 22 Oct 2026 14:01:25 -0700 Subject: [PATCH 34/36] Update docs/analysis/templates/analysis.md Co-authored-by: Dave Welsch <116022979+dwelsch-esi@users.noreply.github.com> Signed-off-by: Bruce Hamilton --- docs/analysis/templates/analysis.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/analysis/templates/analysis.md b/docs/analysis/templates/analysis.md index ce6c5fb8..0d942a69 100644 --- a/docs/analysis/templates/analysis.md +++ b/docs/analysis/templates/analysis.md @@ -31,7 +31,7 @@ For the analysis procedure, see [Analysis how-to](../howto.md). ## Introduction -This document is an analysis the effectiveness and completeness of the +This document is an analysis of the effectiveness and completeness of the [_PROJECT_][project-website] open source software (OSS) project's documentation and website. It is funded by the CNCF Foundation as part of its overall effort to incubate, grow, and graduate open source cloud native software projects. From 988cc0f0243fab19af4d86893bee3e6fa29dcabe Mon Sep 17 00:00:00 2001 From: Bruce Hamilton Date: Thu, 22 Oct 2026 14:01:39 -0700 Subject: [PATCH 35/36] Update docs/analysis/templates/analysis.md Co-authored-by: Dave Welsch <116022979+dwelsch-esi@users.noreply.github.com> Signed-off-by: Bruce Hamilton --- docs/analysis/templates/analysis.md | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/docs/analysis/templates/analysis.md b/docs/analysis/templates/analysis.md index 0d942a69..13944f38 100644 --- a/docs/analysis/templates/analysis.md +++ b/docs/analysis/templates/analysis.md @@ -132,8 +132,8 @@ into a series of issues and entered on GitHub `PROJECT-DOC-WEBSITE/issues`. ### How to use this document -If provided, readers interested only in actionable improvements should skip this -document and read the [implementation][] plan and [issues list][]. +Readers interested only in actionable improvements should skip this +document and read the [implementation][] plan (if provided) and [issues list][]. Readers interested in the current state of the documentation and the reasoning behind the recommendations should read the section of this document pertaining From 5dcba751c69057cffd8ca36c9ac5d17068efd80f Mon Sep 17 00:00:00 2001 From: Bruce Hamilton Date: Thu, 22 Oct 2026 14:08:25 -0700 Subject: [PATCH 36/36] ran prettier Signed-off-by: Bruce Hamilton --- docs/analysis/templates/analysis.md | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/docs/analysis/templates/analysis.md b/docs/analysis/templates/analysis.md index 13944f38..2847144a 100644 --- a/docs/analysis/templates/analysis.md +++ b/docs/analysis/templates/analysis.md @@ -132,8 +132,8 @@ into a series of issues and entered on GitHub `PROJECT-DOC-WEBSITE/issues`. ### How to use this document -Readers interested only in actionable improvements should skip this -document and read the [implementation][] plan (if provided) and [issues list][]. +Readers interested only in actionable improvements should skip this document and +read the [implementation][] plan (if provided) and [issues list][]. Readers interested in the current state of the documentation and the reasoning behind the recommendations should read the section of this document pertaining