diff --git a/.cspell.yml b/.cspell.yml index 9da23446..d73db896 100644 --- a/.cspell.yml +++ b/.cspell.yml @@ -20,6 +20,7 @@ ignorePaths: # ignoreRegExpList: # - CodeBlock words: + - afdocs - backstore - Chalin - CLOTributor @@ -30,6 +31,8 @@ words: - github - keda - knative + - llms + - llmstxt - kedacore - krator - krook diff --git a/.htmltest.yml b/.htmltest.yml index 74607e7a..6636761a 100644 --- a/.htmltest.yml +++ b/.htmltest.yml @@ -4,7 +4,7 @@ DirectoryPath: build TestFilesConcurrently: true IgnoreDirs: # We don't fix broken links on old analysis pages. - # - analyses/202[0-5]/ + - analyses/202[0-3]/ IgnoreInternalURLs: # list of paths IgnoreURLs: # list of regexes of URLs or path to be ignored - ^https?://localhost diff --git a/docs/analysis/ai-harness/_github/copilot-instructions.md b/docs/analysis/ai-harness/_github/copilot-instructions.md new file mode 100644 index 00000000..98b02ef2 --- /dev/null +++ b/docs/analysis/ai-harness/_github/copilot-instructions.md @@ -0,0 +1,549 @@ +--- +title: Copilot instructions for CNCF TechDocs analysis +applyTo: **/*.md +--- + +# CNCF TechDocs Analysis Copilot Instructions + +The CNCF TechDocs analysis focuses on evaluating the quality and completeness of +project documentation. Until now human reviewers have been performing this +analysis, and still will be. The goal of this project is to provide a harness +and workflow for writers to use for having Copilot to provide responses. As +such, predefined prompts are provided to guide Copilot in providing analysis +responses. Predefined prompts should also alleviate inconsistencies resulting +from having the writer craft the prompt. + +Given that the analysis writers are providing their own analysis, the Copilot +analysis must be written as to require minimal editing by the writers, such as +making fragments into complete sentences, or adding context to a comment. + +## Analysis Criteria + +The focus is to analyze documentation for CNCF projects and to provide comments, +recommendations, and issues. + +Review sources: + +- Analysis criteria for evaluating content: + https://github.com/cncf/techdocs/blob/main/docs/analysis/criteria.md. +- Template for the analysis report: + https://github.com/cncf/techdocs/blob/main/docs/analysis/templates/analysis.md + +## Analysis Guidelines + +Things to keep in mind while doing the analysis: + +Look for: + +- Quick wins – low-effort changes that would have a major impact. +- Large, systemic issues that you can organize a number of issues around. +- The two or three most important issues that impede documentation + effectiveness. +- Anything the project does exceptionally well. We can call these out as + examples in other evaluations! + +Don't get bogged down in detail when writing comments. Include enough detail +that you can describe how to implement a suggested solution. A sentence or two +is sufficient for most issues. + +Keep in mind the overall goal of the technical documentation: to make its users +more effective. Focus on issues that get in the way of that goal. + +It is not necessary to come up with a recommendation in every case, especially +for elements that are satisfactory or if a recommendation would result in +minimal improvement. + +Analysis criteria is further specified in the prompt files under +`.github/prompts/`. + +## Analysis responses style + +Readers of the analysis are subject-matter experts in the project's ecosystem +(named in the "Current Repository" section) who want to improve the +documentation quickly. Write for them: be concise and actionable, and skip +explanations they don't need. + +- Write in complete sentences, using active voice and the imperative mood. +- Fragments are fine inside a bulleted list that follows an introductory + sentence. +- Explain any term or acronym that a senior engineer in the project's ecosystem + might not know. +- Don't list technologies in parentheses; readers already know them. +- Use parentheses only when the content is needed to understand the + recommendation. +- When a sentence would list four or more items, end the paragraph with an + introductory sentence followed by a bulleted list of the items. +- You can refer to Markdown files instead of page titles; but unless the name is + obvious, also provide the page title. No need to include relative paths as + long as the page title is clear. +- Keep paragraphs under about 4 sentences; split longer ones. +- Treat any paragraph limit in a prompt as a maximum, not a target. +- If you're unsure whether something is technically wrong, phrase it as a + question for the SME rather than asserting it. +- Follow the project's documentation conventions as described in "Current + Repository". For Kubernetes-ecosystem projects that means UpperCamelCase for + API objects (Pod, Deployment) and code formatting for commands, fields, and + file names. + +## Analysis Criteria and Predefined Prompts + +The analysis has three sections (Project Documentation, Contributor +Documentation, and Website & Infrastructure). Each section contains multiple +areas, and each area has its own criteria and prompts for evaluation. + +The AI assisted analysis harness is organized into two types of files: criteria +and prompt Markdown files. These files are organized further by `areas` and +`sections` folders. These files live under `.github/criteria/areas`, +`.github/criteria/sections`, `.github/prompts/areas`, and +`.github/prompts/sections` respectively. Generated output lives in +`.github/output/analysis` and `.github/output/issues`. + + + +```text +.github/ + copilot-instructions.md + criteria/ + areas/.md # Per-area data: display name, file stem, questions, comment guidance + sections/
.md # Per-section data: display name, slug, member areas and comment-file stems + prompts/ + areas/ + answers.prompt.md # Shared engines + comment.prompt.md + recommendations.prompt.md + full.prompt.md + -.prompt.md # Per-area wrappers + sections/ + section-comment.prompt.md # Shared engine +
-comment.prompt.md # Per-section wrappers + issues/ + rank-recommendations.prompt.md # Ranks recommendations into input/ai-ranked-recommendations.csv + create-issues.prompt.md # Generates numbered issue files from ranked recommendations + create-umbrella-issues.prompt.md # Groups issue files into umbrella issues + templates/ + doc-issue.md # Documentation issue template (replace with the project's own) + umbrella-issue.md # Umbrella issue template + input/ + ai-ranked-recommendations.csv # Written by /rank-recommendations, read by /create-issues + output/ + analysis/--.md # Per-area outputs + analysis/--comment.md # Section-level comments + issues/-.md # Generated standalone issue files + issues/U--.md # Generated umbrella issue files + issues-coverage.md # Issue list and recommendation-to-issue mapping +``` + + + +Criteria files for an area define the per-area data: display name, output-file +stem, question set, and comment guidance. The area name is the criteria filename +without `.md` (for example, `.github/criteria/areas/information-architecture.md` +defines the `information-architecture` area). + +Criteria files for a section define the section's display name, slug, and member +areas (with the stem of each area's comment file). + +Prompt files for areas are organized into three layers so the per-area questions +(data) are separated from the task logic (verbs): + +- Per-area wrappers (`-.prompt.md`) — the entry points you invoke, + for example `/information-architecture-answers`. Each wrapper simply binds one + criteria area to one shared engine. +- Shared engines (`answers.prompt.md`, `comment.prompt.md`, + `recommendations.prompt.md`, all in `.github/prompts/areas/`) — the reusable + procedure for each output type. They are project-agnostic: the project name + and documentation label are read from the "Current Repository" section below, + so the same engines work in any repository. + +Prompt files for sections follow a similar structure, with section-level +wrappers and an engine (`section-comment.prompt.md`), all in +`.github/prompts/sections/`, to synthesize comments from the member areas. + +For areas only, there is also an optional composite engine `full.prompt.md`, +that runs the `answers`, `comment`, and `recommendations` engines for one area +in that order (recommendations depends on the answers and comment files existing +first). A per-area `full` wrapper (for example, +`/information-architecture-full`) invokes it to produce all three output files +in a single run. + +The `section-comment.prompt.md` engine writes a two-to-five paragraph overall +comment for one section by synthesizing that section's existing per-area comment +files (it does not re-analyze the documentation), and it prepends a summary +ratings table scraped from each area comment's `Rating:` line. Its data lives in +`.github/criteria/sections/
.md`, which lists the section's display +name, slug, and member areas with the stem of each area's comment file. A +per-section wrapper (for example, `/project-documentation-comment`) invokes it. +Because the per-area comment files are a precondition, run the relevant +`-comment` prompts first. By default the output is written to +`.github/output/analysis/--comment.md` (for example, +`kubevirt-project-documentation-comment.md`), overridable with a `title` input. + +Each criteria area needs three wrapper files (`answers`, `comment`, +`recommendations`), and the wrappers are mechanical: they reference the area +name and embed the display name in the description. A `full` wrapper is optional +per area. + +By default a prompt writes to `--.md` in the +`.github/output/analysis` directory (for example, +`kubevirt-information-architecture-answers.md`). Provide an optional `title` +input to override the filename. To add or change questions for an area, edit +only its `.github/criteria/areas/.md` file. + +Output types: `answers`, `comment`, `recommendations` (plus the optional `full` +composite). At the section level there is one additional output type, the +section-level `comment` produced by the `section-comment` engine and its +per-section wrappers. + +The analysis document has three sections: Project Documentation, Contributor +Documentation, and Website & Infrastructure. Each section contains multiple +criteria areas that are evaluated separately, and the results are synthesized +into section-level overall comments. + +Sections (for the section-level overall comment): + + + +| Section | Section slug | Member areas (comment-file stems) | +| ------------------------- | ------------------------- | --------------------------------------------------------------------------------------------------------------------------------------------------------------------- | +| Project Documentation | project-documentation | information-architecture, new-user, content-maintainability, content-creation-process, inclusive-language | +| Contributor Documentation | contributor-documentation | communication-methods-docd, beginner-issue-backlog, new-contributor, project-governance | +| Website & Infrastructure | website-infrastructure | single-source-requirement, website-requirements, usability-accessibility-devices, branding-design, case-studies, seo-analytics, maintenance-planning, agent-readiness | + + + +Areas (the Area column is the criteria filename under `.github/criteria/areas/` +and the prefix of the per-area wrapper prompts): + + + +| Area | Display Name | Output File Stem | Section in analysis.md | +| -------------------------------- | --------------------------------------- | ------------------------------- | ------------------------- | +| agent-readiness | AI Agent Readiness | agent-readiness | Website & Infrastructure | +| beginner-issue-backlog | Beginner Friendly Issue Backlog | beginner-issue-backlog | Contributor Documentation | +| branding-design | Branding and Design | branding-design | Website & Infrastructure | +| case-studies | Case Studies | case-studies | Website & Infrastructure | +| communication-methods-documented | Communication Methods Documented | communication-methods-docd | Contributor Documentation | +| content-creation-process | Content Creation Process | content-creation-process | Project Documentation | +| content-maintainability | Content Maintainability | content-maintainability | Project Documentation | +| inclusive-language | Inclusive Language | inclusive-language | Project Documentation | +| information-architecture | Information Architecture | information-architecture | Project Documentation | +| maintenance-planning | Maintenance Planning | maintenance-planning | Website & Infrastructure | +| new-contributor-content | New Contributor Getting Started Content | new-contributor | Contributor Documentation | +| new-user-content | New User Content | new-user | Project Documentation | +| project-governance-documentation | Project Governance Documentation | project-governance | Contributor Documentation | +| seo-analytics | SEO, Analytics, and Site Search | seo-analytics | Website & Infrastructure | +| single-source-requirement | Single-Source Requirement | single-source-requirement | Website & Infrastructure | +| usability-accessibility-devices | Usability, Accessibility and Devices | usability-accessibility-devices | Website & Infrastructure | +| website-requirements | Website Requirements | website-requirements | Website & Infrastructure | + + + +### Example: invoking a predefined prompt + +To have Copilot answer the information architecture questions, invoke the +per-area wrapper as a slash command in the current directory: + +```text +run /information-architecture-answers +``` + +This runs the `information-architecture` questions through the shared `answers` +engine and writes the result to the default file +`.github/output/analysis/kubevirt-information-architecture-answers.md`. + +To override the output filename, pass a `title` input: + +```text +run /information-architecture-answers title: my-analysis +``` + +That writes the answers to `.github/output/analysis/my-analysis.md` instead of +the default filename. The same pattern applies to every area and output type — +for example, `/inclusive-language-recommendations` or +`/new-user-content-comment`. + +## Analysis.md and Output Files Mapping + +Predefined prompts create Markdown files of their AI generated output. The +content from these files, without the temporary H1 headings at the top, is +copied into specific locations in the analysis.md file. + + + +| Output file | Location in analysis.md | +| ----------------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | +| `-project-documentation-comment.md` | **Start**: After the ratings table that follows the first paragraph that follows the "Project Documentation" H2.
**End**: Before the sentence "The following sections contain assessments of each element of the Project Documentation rubric." | +| `-contributor-documentation-comment.md` | **Start**: After the ratings table that follows the first paragraph that follows the "Contributor Documentation" H2.
**End**: Before the sentence "The following sections contain assessments of each element of the Contributor Documentation rubric." | +| `-website-infrastructure-comment.md` | **Start**: After the ratings table that follows the first paragraph that follows the "Website & Infrastructure" H2.
**End**: Before the sentence "The following sections contain assessments of each element of the Website & Infrastructure rubric." | +| `--answers.md` | **Start**: After the first paragraph that follows the H3 heading for an area, e.g. "### Information Architecture". That first paragraph should end with the sentence: "We evaluate on the following:".
**End**: Before the H4 "Comment" heading. | +| `--comment.md` | **Start**: After the H4 "Comment" heading.
**End**: Before the next H3 or H2 heading.
**Note**: Currently, only the Strengths and Weaknesses of the area comments are pasted into analysis.md.
**Note**: The rating for the area (last line in the output) is later manually copied by a writer into the area summary ratings table for the section. | +| `--recommendations.md` | **Start**: Under the H2 headings "Project documentation recommendations", "Contributor documentation recommendations", or "Website & infrastructure recommendations" and under the H3 heading for the area.
**End**: Before the next H3 or H2 heading. | + + + +## Recommendations and Issues + +The `/rank-recommendations` prompt collects the recommendations from the +per-area recommendations files in `.github/output/analysis/` and ranks them into +`.github/input/ai-ranked-recommendations.csv`, with the following columns: + +- `Section` - Either the "Project Documentation", "Contributor documentation", + or "Website & Infrastructure" section. +- `Area` - The specific area within the section that the recommendation pertains + to. The area names in the CSV are the display names used in the analysis, not + the criteria area filenames. +- `Recommendation` - The AI-written recommendation describing the action that + should be taken. +- `Suggested Priority` - The ranking priority: `1 - Urgent`, `2 - High`, or + `3 - Backlog`. These priorities were only for the recommendation ranking and + may be adjusted when creating issues. + +The priority labels used for issues are `urgent`, `high`, and `backlog`, +matching the CSV ranking without its number prefix. The upload process maps them +to the target repository's own labels using the "Priority label mapping" table +in the "Current Repository" section. Assume all three mapped labels exist when +writing the issue files; the pre-flight checklist in the harness `index.md` +covers creating them. + +Every issue also gets the `help wanted` label. It exists in the repository and +signals that maintainers are seeking community help; CNCF uses it to surface +documentation tasks on [CLOTributor](https://clotributor.dev). + +Issues are a subset of the recommendations that have been identified as problems +or areas needing attention. Not all recommendations are issues; some may be +suggestions for improvement that do not indicate a problem. Copilot is to create +the issues based on the ranked recommendations, and then group them into +umbrella issues. Copilot may change the priority of the issues as it sees fit. +As a reference, see the umbrella issues in +[CNCF TechDocs Assistance issues tracker #221](https://github.com/cncf/techdocs/issues/221), +the KEDA +[standalone issue example](https://github.com/kedacore/keda-docs/issues/1363), +and the KEDA +[umbrella issue example](https://github.com/kedacore/keda-docs/issues/1361). The +project's own assistance request is listed in the "Current Repository" section. + +Copilot will create Markdown files of the issues with updated priorities as +applicable. These files will be stored in the `.github/output/issues/` +directory. A process will then upload them to the project's repository, so every +file in that directory must be a complete, uploadable issue. Issue creation runs +in two passes, each with its own prompt: + +1. `/create-issues` writes one file per standalone issue, numbered sequentially. +2. `/create-umbrella-issues` reads those files and writes umbrella issues that + link to them. + +In writing the issues, Copilot should use the following guidelines: + +- Source the issues only from the ranked recommendations CSV and the per-area + recommendations files in `.github/output/analysis/`. Do not add issues from + general knowledge of the project's repositories; that over-generates issues + the assessment did not identify. +- Ensure that the issues are clearly described and actionable. Follow the + headings of the documentation issue template under `.github/templates/` + (`doc-issue.md` by default; the "Current Repository" section names the file). + Replace the default with the project's own documentation issue format when it + has one. +- Each issue should take about two to four hours for a technical writer, or + engineer, with project context to complete. +- Small tasks can be consolidated into one issue where appropriate. Avoid + flooding the project: aim for far fewer issues than recommendations. +- Umbrella issues group three to ten related issues. Follow the + `.github/templates/umbrella-issue.md` template: an `## Umbrella Issue` + overview and a `## Sub-issues` list that links to the sub-issue files. The + umbrella does not contain the sub-issue content. +- Copilot is at liberty to create subheadings in the issue files as needed to + organize the content effectively, but keeps the template's top-level headings. +- Everything outside the front matter is the issue description and must be + GitHub-flavored Markdown. +- Ensure the following front matter (metadata) is included at the top of each + issue file. + - Create a title and append the simulated issue number as `#{issue number}`. + It becomes the GitHub issue title. + - Set `labels` to the priority label (`urgent`, `high`, or `backlog`) plus + `help wanted`. Labels become GitHub issue labels. + - Set `repo` to the issues repository URL from the "Current Repository" + section. + + ```yaml + --- + title: Add a user-facing troubleshooting page #9007 + labels: [high, help wanted] + repo: https://github.com///issues + --- + ``` + +### Issue numbers + +Issue numbers simulate GitHub issue numbers so that files can reference each +other before upload. They are five digits, start at 9001, and increment by one +for each issue in the order written. Umbrella issues continue the sequence after +the last standalone issue. The `## Issues` table in +`.github/output/issues-coverage.md` maps each number to its title, label, and +file, and is how the umbrella prompt finds the next free number. + +In an umbrella issue, each sub-issue is one list item whose link text is the +sub-issue title and number and whose target is the sub-issue file, relative to +the umbrella file: + +```markdown +- [Reorganize Concepts #9938](9938-reorganize-concepts.md) +``` + +#### Issue file names + +Save the issue files to the `.github/output/issues/` directory. Name standalone +issue files `{issue-number}-{condensed-issue-title}.md` and umbrella issue files +`U-{issue-number}-{condensed-issue-title}.md`. The condensed title is a +lowercase, hyphenated version of the issue title, no more than about five words: + +- Keep the action verb and the main subject noun; drop articles, conjunctions, + and secondary clauses. +- Name the topic (for example, `navigation`, `footer`, `virtctl`), not the + analysis area it came from. +- When the title lists several changes, use a collective noun such as + `improvements` or `updates` instead of listing them. +- Strip punctuation from version numbers (`v1.9` becomes `v19`). + +Examples of a file name built from a full title: + + + +| Issue title | File name | +| ----------------------------------------------------------------- | --------------------------------------- | +| Add a user-facing troubleshooting page #9007 | `9007-add-troubleshooting-page.md` | +| Move Release Notes out of the main navigation #9004 | `9004-move-release-notes.md` | +| Document the v1.9 VirtualMachineBackup API #9012 | `9012-document-v19-backup-api.md` | +| Create a new-user getting started path #9061 | `U-9061-create-getting-started-path.md` | +| Fix navigation gaps and reorganize information architecture #9062 | `U-9062-fix-navigation-gaps.md` | + + + +To archive a previous run, move its files into a sub-folder of +`.github/output/issues/` (for example, `prev_1/`). The prompts ignore +sub-folders and the folder's `readme.md`. + +### Rank recommendations AI prompt + +The prompt lives in `.github/prompts/issues/rank-recommendations.prompt.md`. Run +it after all seventeen area recommendations files exist: + +```text +run /rank-recommendations +``` + +It reads every `-*-recommendations.md` file in +`.github/output/analysis/`, assigns each recommendation a `1 - Urgent`, +`2 - High`, or `3 - Backlog` priority by impact and effort, and writes +`.github/input/ai-ranked-recommendations.csv`. Review the ranking before running +`/create-issues`; to adjust it, re-run with the corrections in the same message. + +### Create issues AI prompt + +The prompt lives in `.github/prompts/issues/create-issues.prompt.md` and is the +single source of truth. Run it as a slash command from the repository root: + +```text +run /create-issues +``` + +It deletes issue files from a previous run, writes the numbered standalone issue +files, and writes `.github/output/issues-coverage.md` with the issue table, the +recommendation-to-issue mapping, priority adjustments, and counts. + +To re-run after coaching Copilot on scope, run the command again and add the +corrections in the same message, for example +`run /create-issues — split the "Getting started" issue into two`. + +### Create umbrella issues AI prompt + +The prompt lives in `.github/prompts/issues/create-umbrella-issues.prompt.md`. +Run it in a new session after `/create-issues` has finished and the writer has +reviewed the standalone issues: + +```text +run /create-umbrella-issues +``` + +It reads the standalone issue files and the `## Issues` table in +`issues-coverage.md`, groups the issues into themes of three to ten, writes the +`U-` umbrella files with the next free issue numbers, and appends the umbrellas +to the coverage report. It does not modify standalone issue files or re-read the +recommendations; if an issue needs re-scoping, re-run `/create-issues` first. + +To adjust the grouping, run the command again with the corrections in the same +message, for example +`run /create-umbrella-issues — merge the analytics and site-ownership umbrellas`. + +## Current Repository + +This section is the single place the engines and issue prompts read project +context from. Replace every `` placeholder before running any +prompt; the issue prompts stop if a placeholder is still present. See +`examples/kubevirt/current-repository.md` in the harness for a completed +example. + + + +- Project name: + +- Documentation label: + +- Ecosystem: + +- Main repository: +- Secondary repositories: + +- Published documentation URL: +- Project website URL: +- Issues repository URL: + +- Issue template file: + +- CNCF project page URL: + +- Assistance-request issue: + + + +Copilot shall output Markdown files from the AI-assisted analysis to the +`.github/output/analysis` directory and issue files to the +`.github/output/issues` directory in the current repository. + +### Priority label mapping + +The upload process maps the harness labels to the issue repository's labels. +Confirm each target label exists before uploading. + +| Harness label | Repository label | +| ------------- | ----------------------------------------------- | +| `urgent` | | +| `high` | | +| `backlog` | | +| `help wanted` | `help wanted` | + +### What This Repo Is + + + +### Content Architecture + + + +```text +docs/ + index.md +``` + +### Key Conventions + + + +#### No commits + +This work is solely for documentation analysis. Copilot makes no commits to the +documentation or application code in this repository; it writes only to +`.github/input` and `.github/output`. diff --git a/docs/analysis/ai-harness/_github/criteria/areas/agent-readiness.md b/docs/analysis/ai-harness/_github/criteria/areas/agent-readiness.md new file mode 100644 index 00000000..c1a6958e --- /dev/null +++ b/docs/analysis/ai-harness/_github/criteria/areas/agent-readiness.md @@ -0,0 +1,35 @@ +# AI agent readiness - Website and Infrastructure + +- Display name: AI Agent Readiness +- File stem: agent-readiness + +## Questions + +- Does the documentation site publish an `llms.txt` file at the site or docs + root that follows the llmstxt.org structure (H1 title, blockquote summary, H2 + sections of link lists)? +- Do all links in `llms.txt` resolve, point to Markdown versions of pages, and + stay in sync with the published site (for example, generated at build time)? +- Do HTML and Markdown pages include a directive near the top pointing agents to + `llms.txt`? +- Is a clean Markdown version of each page served at the same URL with `.md` + appended or substituted, with content matching the HTML version? +- Are pages rendered server-side or statically generated, with the main content + starting near the top of the response rather than behind large navigation or + script payloads? +- Are pages small enough to avoid truncation in an agent's context window, and + do tabbed components and code fences serialize cleanly to Markdown? +- Is the documentation publicly reachable without authentication gates or CDN + bot challenges, and do pages use correct HTTP status codes and standard HTTP + redirects? +- Does the project monitor its CLOMonitor "Agent readiness" checks or run + `afdocs check` against the site, ideally in CI? + +## Comment guidance + +- Additional instructions: Reference sources for this area are the llms.txt + specification (https://llmstxt.org/), the CLOMonitor agent readiness checks + (https://clomonitor.io/docs/topics/checks/#agent-readiness), and the AFDocs + Agent-Friendly Documentation Spec (https://afdocs.dev/). Where possible, + verify by fetching the live site (`/llms.txt`, a page URL with `.md` appended) + rather than relying on the repository source alone. diff --git a/docs/analysis/ai-harness/_github/criteria/areas/beginner-issue-backlog.md b/docs/analysis/ai-harness/_github/criteria/areas/beginner-issue-backlog.md new file mode 100644 index 00000000..c9441830 --- /dev/null +++ b/docs/analysis/ai-harness/_github/criteria/areas/beginner-issue-backlog.md @@ -0,0 +1,16 @@ +# Beginner friendly issue backlog - Contributor Documentation + +- Display name: Beginner Friendly Issue Backlog +- File stem: beginner-issue-backlog + +## Questions + +- Are docs issues well-triaged? +- Is there a clearly marked way for new contributors to make code or + documentation contributions (i.e. a “good first issue” label)? +- Are issues well-documented (i.e., more than just a title)? +- Are issues maintained for staleness? + +## Comment guidance + +- Additional instructions: None. diff --git a/docs/analysis/ai-harness/_github/criteria/areas/branding-design.md b/docs/analysis/ai-harness/_github/criteria/areas/branding-design.md new file mode 100644 index 00000000..35ee03c9 --- /dev/null +++ b/docs/analysis/ai-harness/_github/criteria/areas/branding-design.md @@ -0,0 +1,15 @@ +# Branding and design - Website and Infrastructure + +- Display name: Branding and Design +- File stem: branding-design + +## Questions + +- Is there an easily recognizable brand for the project (logo + color scheme) + clearly identifiable? +- Is the brand used across the website consistently? +- Is the website’s typography clean and well-suited for reading? + +## Comment guidance + +- Additional instructions: None. diff --git a/docs/analysis/ai-harness/_github/criteria/areas/case-studies.md b/docs/analysis/ai-harness/_github/criteria/areas/case-studies.md new file mode 100644 index 00000000..6aff04fe --- /dev/null +++ b/docs/analysis/ai-harness/_github/criteria/areas/case-studies.md @@ -0,0 +1,17 @@ +# Case studies - Website and Infrastructure + +- Display name: Case Studies +- File stem: case-studies + +## Questions + +- Are there case studies available for the project and are they documented on + the website? +- Are there user testimonials available? +- Is there an active project blog? +- Are there community talks for the project and are they present on the website? +- Is there a logo wall of users/participating organizations? + +## Comment guidance + +- Additional instructions: None. diff --git a/docs/analysis/ai-harness/_github/criteria/areas/communication-methods-documented.md b/docs/analysis/ai-harness/_github/criteria/areas/communication-methods-documented.md new file mode 100644 index 00000000..5b213a7a --- /dev/null +++ b/docs/analysis/ai-harness/_github/criteria/areas/communication-methods-documented.md @@ -0,0 +1,17 @@ +# Communication methods documented - Contributor Documentation + +- Display name: Communication Methods Documented +- File stem: communication-methods-docd + +## Questions + +- Is there a Slack/Discord/Discourse/etc. community and is it prominently linked + from your website? +- Is there a direct link to your GitHub organization/repository? +- Are weekly/monthly project meetings documented? Is it clear how someone can + join those meetings? +- Are mailing lists documented? + +## Comment guidance + +- Additional instructions: None. diff --git a/docs/analysis/ai-harness/_github/criteria/areas/content-creation-process.md b/docs/analysis/ai-harness/_github/criteria/areas/content-creation-process.md new file mode 100644 index 00000000..4d4f2754 --- /dev/null +++ b/docs/analysis/ai-harness/_github/criteria/areas/content-creation-process.md @@ -0,0 +1,19 @@ +# Content creation process - Project Documentation + +- Display name: Content Creation Process +- File stem: content-creation-process + +## Questions + +- Is there a clearly documented (ongoing) contribution process for + documentation? +- Does the code release process account for documentation creation & updates? +- Who reviews and approves documentation pull requests? +- Does the website have a clear owner/maintainer? + +## Comment guidance + +- Good examples: + - https://github.com/nats-io/nats-site/blob/master/MAINTAINERS.md + - https://thanos.io/tip/contributing/how-to-contribute-to-docs.md +- Additional instructions: None. diff --git a/docs/analysis/ai-harness/_github/criteria/areas/content-maintainability.md b/docs/analysis/ai-harness/_github/criteria/areas/content-maintainability.md new file mode 100644 index 00000000..85874b8c --- /dev/null +++ b/docs/analysis/ai-harness/_github/criteria/areas/content-maintainability.md @@ -0,0 +1,17 @@ +# Content maintainability - Project Documentation + +- Display name: Content Maintainability +- File stem: content-maintainability + +## Questions + +- Is the documentation searchable? +- Are there plans for localization/internationalization with regards to site + directory structure? Is a localization framework present? +- Is there a clearly documented method for versioning of content? + +## Comment guidance + +- Good examples: + - https://kubernetes.io/docs/ +- Additional instructions: None. diff --git a/docs/analysis/ai-harness/_github/criteria/areas/inclusive-language.md b/docs/analysis/ai-harness/_github/criteria/areas/inclusive-language.md new file mode 100644 index 00000000..83523dac --- /dev/null +++ b/docs/analysis/ai-harness/_github/criteria/areas/inclusive-language.md @@ -0,0 +1,17 @@ +# Inclusive language - Project Documentation + +- Display name: Inclusive Language +- File stem: inclusive-language + +## Questions + +- Are there any customer-facing utilities, endpoints, class names, or feature + names that use non-recommended words as documented by the + [Inclusive Naming Initiative](https://inclusivenaming.org) website? +- Does the project use language like "simple", "easy", etc.? + +## Comment guidance + +- Good examples: None. +- Additional instructions: Focus the comment on any inclusive language concerns + in the documentation. diff --git a/docs/analysis/ai-harness/_github/criteria/areas/information-architecture.md b/docs/analysis/ai-harness/_github/criteria/areas/information-architecture.md new file mode 100644 index 00000000..369feb5a --- /dev/null +++ b/docs/analysis/ai-harness/_github/criteria/areas/information-architecture.md @@ -0,0 +1,26 @@ +# Information architecture - Project Documentation + +- Display name: Information architecture +- File stem: information-architecture + +## Questions + +- Is there high level conceptual content? +- Is the documentation feature complete? +- Are there step-by-step instructions documented for features in tasks and + tutorials? +- Are there any key features that are documented but missing task documentation? +- Is the “happy path” (most common use case) documented? +- Are tasks clearly named according to user goals? +- If the documentation doesn't suffice, is there a clear escalation path for + users needing more help? (FAQ, Troubleshooting) +- If the product exposes an API, is there a complete reference that includes + documented CLIs as applicable? +- Is content up to date and accurate? +- Does the documentation need restructuring? + +## Comment guidance + +- Good examples: + - https://prometheus.io/docs +- Additional instructions: None. diff --git a/docs/analysis/ai-harness/_github/criteria/areas/maintenance-planning.md b/docs/analysis/ai-harness/_github/criteria/areas/maintenance-planning.md new file mode 100644 index 00000000..587309f7 --- /dev/null +++ b/docs/analysis/ai-harness/_github/criteria/areas/maintenance-planning.md @@ -0,0 +1,18 @@ +# Maintenance planning - Website and Infrastructure + +- Display name: Maintenance Planning +- File stem: maintenance-planning + +## Questions + +- Is the website tooling well supported by the community (i.e., Hugo with the + Docsy theme) or commonly used by CNCF projects? +- Is there active cultivating website maintainers from within the community? +- Are site build times reasonable? +- Do site maintainers have adequate permissions? +- Is the website accessible via HTTPS? +- Does HTTP access, if any, redirect to HTTPS? + +## Comment guidance + +- Additional instructions: None. diff --git a/docs/analysis/ai-harness/_github/criteria/areas/new-contributor-content.md b/docs/analysis/ai-harness/_github/criteria/areas/new-contributor-content.md new file mode 100644 index 00000000..ec32bb11 --- /dev/null +++ b/docs/analysis/ai-harness/_github/criteria/areas/new-contributor-content.md @@ -0,0 +1,14 @@ +# New contributor getting started content - Contributor Documentation + +- Display name: New Contributor Getting Started Content +- File stem: new-contributor + +## Questions + +- Do you have a community repository or section on your website? +- Is there a document specifically for new contributors/your first contribution? +- Do new users know where to get help? + +## Comment guidance + +- Additional instructions: None. diff --git a/docs/analysis/ai-harness/_github/criteria/areas/new-user-content.md b/docs/analysis/ai-harness/_github/criteria/areas/new-user-content.md new file mode 100644 index 00000000..c048353d --- /dev/null +++ b/docs/analysis/ai-harness/_github/criteria/areas/new-user-content.md @@ -0,0 +1,21 @@ +# New user content - Project Documentation + +- Display name: New User Content +- File stem: new-user + +## Questions + +- Is “Getting started” clearly labeled? (e.g “Getting started”, “Installation”, + “First steps”, etc.) +- Is installation documented step-by-step? +- If needed, is guidance provided for multiple operating systems and platforms? +- Do users know where to go after reading the getting started guide? +- Is your new user content clearly discoverable, such as on the documentation + home page? +- Is there sample code or content that can easily be copy-pasted? + +## Comment guidance + +- Good examples: + - https://falco.org/docs/getting-started/ +- Additional instructions: None. diff --git a/docs/analysis/ai-harness/_github/criteria/areas/project-governance-documentation.md b/docs/analysis/ai-harness/_github/criteria/areas/project-governance-documentation.md new file mode 100644 index 00000000..9efe6cba --- /dev/null +++ b/docs/analysis/ai-harness/_github/criteria/areas/project-governance-documentation.md @@ -0,0 +1,12 @@ +# Project Governance Documentation - Contributor Documentation + +- Display name: Project Governance Documentation +- File stem: project-governance + +## Questions + +- Is project governance clearly documented? + +## Comment guidance + +- Additional instructions: None. diff --git a/docs/analysis/ai-harness/_github/criteria/areas/seo-analytics.md b/docs/analysis/ai-harness/_github/criteria/areas/seo-analytics.md new file mode 100644 index 00000000..8192b967 --- /dev/null +++ b/docs/analysis/ai-harness/_github/criteria/areas/seo-analytics.md @@ -0,0 +1,20 @@ +# SEO, analytics, and local site search - Website and Infrastructure + +- Display name: SEO Analytics and Site Search +- File stem: seo-analytics + +## Questions + +- Is analytics enabled for the production server? +- Is analytics disabled for all other deploys? +- If project is using Google Analytics, has it migrated to GA4? +- Can Page-not-found (404) reports easily be generated from site analytics? +- Is site indexing supported for the production server, while disabled for + website previews and builds for non-default branches? +- Is local intra-site search available from the website? +- Are the current custodian(s) of the analytics accounts (such as Google CSE) + documented? + +## Comment guidance + +- Additional instructions: None. diff --git a/docs/analysis/ai-harness/_github/criteria/areas/single-source-requirement.md b/docs/analysis/ai-harness/_github/criteria/areas/single-source-requirement.md new file mode 100644 index 00000000..bb893a34 --- /dev/null +++ b/docs/analysis/ai-harness/_github/criteria/areas/single-source-requirement.md @@ -0,0 +1,13 @@ +# Single source requirement - Website and Infrastructure + +- Display name: Single-Source Requirement +- File stem: single-source-requirement + +## Questions + +- Does the project have a single source for its documentation? If not, is there + a reason? + +## Comment guidance + +- Additional instructions: None. diff --git a/docs/analysis/ai-harness/_github/criteria/areas/usability-accessibility-devices.md b/docs/analysis/ai-harness/_github/criteria/areas/usability-accessibility-devices.md new file mode 100644 index 00000000..aee3279c --- /dev/null +++ b/docs/analysis/ai-harness/_github/criteria/areas/usability-accessibility-devices.md @@ -0,0 +1,18 @@ +# Usability, accessibility and devices - Website and Infrastructure + +- Display name: Usability Accessibility and Devices +- File stem: usability-accessibility-devices + +## Questions + +- Is the website usable from mobile? +- Are doc pages readable? +- Are all / most website features accessible from mobile -- such as the top-nav, + site search and in-page table of contents? +- Are color contrasts significant enough for color-impaired readers? +- Are most website features usable using a keyboard only? +- Does text-to-speech offer listeners a good experience? + +## Comment guidance + +- Additional instructions: None. diff --git a/docs/analysis/ai-harness/_github/criteria/areas/website-requirements.md b/docs/analysis/ai-harness/_github/criteria/areas/website-requirements.md new file mode 100644 index 00000000..ca6c8668 --- /dev/null +++ b/docs/analysis/ai-harness/_github/criteria/areas/website-requirements.md @@ -0,0 +1,13 @@ +# Website requirements - Website and Infrastructure + +- Display name: Website Requirements +- File stem: website-requirements + +## Questions + +- Are most of the applicable CNCF Website Guidelines satisfied? See + https://github.com/cncf/techdocs/blob/main/docs/website-guidelines-checklist.md + +## Comment guidance + +- Additional instructions: None. diff --git a/docs/analysis/ai-harness/_github/criteria/sections/contributor-documentation.md b/docs/analysis/ai-harness/_github/criteria/sections/contributor-documentation.md new file mode 100644 index 00000000..51a50ec4 --- /dev/null +++ b/docs/analysis/ai-harness/_github/criteria/sections/contributor-documentation.md @@ -0,0 +1,17 @@ +# Contributor Documentation + +- Display name: Contributor Documentation +- Section slug: contributor-documentation + +## Areas + +Each row is an area in this section: its criterion label (used for the summary +ratings table) and the stem of its existing per-area comment file +(`--comment.md`). + +| Criterion | Comment file stem | +| ----------------------------------------- | -------------------------- | +| Communication methods documented | communication-methods-docd | +| Beginner friendly issue backlog | beginner-issue-backlog | +| "New contributor" getting started content | new-contributor | +| Project governance documentation | project-governance | diff --git a/docs/analysis/ai-harness/_github/criteria/sections/project-documentation.md b/docs/analysis/ai-harness/_github/criteria/sections/project-documentation.md new file mode 100644 index 00000000..ec44bd91 --- /dev/null +++ b/docs/analysis/ai-harness/_github/criteria/sections/project-documentation.md @@ -0,0 +1,18 @@ +# Project Documentation + +- Display name: Project Documentation +- Section slug: project-documentation + +## Areas + +Each row is an area in this section: its criterion label (used for the summary +ratings table) and the stem of its existing per-area comment file +(`--comment.md`). + +| Criterion | Comment file stem | +| -------------------------- | ------------------------ | +| Information architecture | information-architecture | +| New user content | new-user | +| Content maintainability | content-maintainability | +| Content creation processes | content-creation-process | +| Inclusive language | inclusive-language | diff --git a/docs/analysis/ai-harness/_github/criteria/sections/website-infrastructure.md b/docs/analysis/ai-harness/_github/criteria/sections/website-infrastructure.md new file mode 100644 index 00000000..5df4ad05 --- /dev/null +++ b/docs/analysis/ai-harness/_github/criteria/sections/website-infrastructure.md @@ -0,0 +1,25 @@ +# Website and Infrastructure + +- Display name: Website and Infrastructure +- Section slug: website-infrastructure + +## Areas + +Each row is an area in this section: its criterion label (used for the summary +ratings table) and the stem of its existing per-area comment file +(`--comment.md`). + + + +| Criterion | Comment file stem | +| ------------------------------------------- | ------------------------------- | +| Single-source for all files | single-source-requirement | +| Meets min website req. (for maturity level) | website-requirements | +| Usability, accessibility, and design | usability-accessibility-devices | +| Branding and design | branding-design | +| Case studies/social proof | case-studies | +| SEO, Analytics, and site-local search | seo-analytics | +| Maintenance planning | maintenance-planning | +| AI agent readiness | agent-readiness | + + diff --git a/docs/analysis/ai-harness/_github/examples/kubevirt/current-repository.md b/docs/analysis/ai-harness/_github/examples/kubevirt/current-repository.md new file mode 100644 index 00000000..456fb45c --- /dev/null +++ b/docs/analysis/ai-harness/_github/examples/kubevirt/current-repository.md @@ -0,0 +1,92 @@ +# KubeVirt example: Current Repository section + +This is the "Current Repository" section of `copilot-instructions.md` as filled +in for the KubeVirt user guide analysis +([cncf/techdocs#343](https://github.com/cncf/techdocs/issues/343)). Use it as a +model when completing the fill-in template in your own copy of +`.github/copilot-instructions.md`. The accompanying `kubevirt-doc-issue.md` is +the issue template the KubeVirt run used in place of the generic +`templates/doc-issue.md`. + +## Current Repository + +- Project name: KubeVirt +- Documentation label: the KubeVirt user guide +- Ecosystem: Kubernetes +- Main repository: [KubeVirt](https://github.com/kubevirt/kubevirt) +- Secondary repositories: + [KubeVirt.github.io](https://github.com/kubevirt/kubevirt.github.io) (website + and lab docs) +- Published documentation URL: https://kubevirt.io/user-guide/ +- Project website URL: https://kubevirt.io/ +- Issues repository URL: https://github.com/kubevirt/user-guide/issues +- Issue template file: kubevirt-doc-issue.md +- CNCF project page URL: https://www.cncf.io/projects/kubevirt/ +- Assistance-request issue: cncf/techdocs#343 + +Copilot shall output Markdown files from the AI-assisted analysis to the +`.github/output/analysis` directory and issue files to the +`.github/output/issues` directory in the current repository. + +### Priority label mapping + +| Harness label | Repository label | +| ------------- | -------------------------- | +| `urgent` | `priority/critical-urgent` | +| `high` | `priority/high` | +| `backlog` | `priority/backlog` | +| `help wanted` | `help wanted` | + +### What This Repo Is + +This is the [KubeVirt user guide](https://kubevirt.io/user-guide), a +documentation-only site built with [MkDocs](https://www.mkdocs.org/) using the +`mkdocs-material` theme and `mkdocs-awesome-nav` plugin. All content lives in +`./docs` as Markdown files. There is no application code. + +### Content Architecture + +```text +docs/ + .nav.yml # Top-level navigation order + index.md + architecture.md + cluster_admin/ # Installation, feature gates, RBAC, node ops + user_workloads/ # VM lifecycle, instancetypes, virtctl, startup scripts + compute/ # CPU/memory, live migration, hugepages, NUMA + network/ # Interfaces, hotplug, binding plugins, Istio + storage/ # CDI, volumes, snapshots, clone, export + debug_virt_stack/ # Debugging guides +``` + +### Key Conventions + +#### Navigation order via `.nav.yml` + +Every subdirectory under `docs/` has a `.nav.yml` that explicitly controls page +ordering. Alphabetical order is intentionally not used. When adding a new page, +add it to the relevant `.nav.yml`. + +#### Redirects in `mkdocs.yml` + +Old page paths (e.g. `operations/`, `virtual_machines/`) are redirected to their +new locations via the `redirects` plugin in `mkdocs.yml`. When moving or +renaming a page, add an entry there. + +#### No commits + +This work is solely for documentation analysis. There are no commits to the +application code in this repository. + +#### Spelling dictionary + +The yaspeller dictionary is sourced from +`kubevirt/project-infra/images/yaspeller/.yaspeller.json`. If a technical term +causes false positives, it needs to be added there (upstream), not locally — +unless a local `yaspeller.json` override is present. + +#### MkDocs extensions in use + +Pages can use: `admonition`, `footnotes`, `toc` with `permalink`, +`pymdownx.highlight`, `pymdownx.superfences`, `attr_list`, and `pymdownx.emoji`. +These are already configured in `mkdocs.yml` — no additional setup needed. diff --git a/docs/analysis/ai-harness/_github/examples/kubevirt/kubevirt-doc-issue.md b/docs/analysis/ai-harness/_github/examples/kubevirt/kubevirt-doc-issue.md new file mode 100644 index 00000000..8b6bb877 --- /dev/null +++ b/docs/analysis/ai-harness/_github/examples/kubevirt/kubevirt-doc-issue.md @@ -0,0 +1,24 @@ +--- +title: { title } #{Issue number} +labels: [high, help wanted] +repo: https://github.com/kubevirt/user-guide/issues +--- + + + +## Description + +A clear and concise description of what the issue is. + +## What you expected + +A clear and concise description of what you expected. + +## URL + +Provide a link to the documentation. + +## Additional context + +Add any other context about the issue here. diff --git a/docs/analysis/ai-harness/_github/input/readme.md b/docs/analysis/ai-harness/_github/input/readme.md new file mode 100644 index 00000000..76403e00 --- /dev/null +++ b/docs/analysis/ai-harness/_github/input/readme.md @@ -0,0 +1,16 @@ +# About this folder + +Use this folder to place temporary prompt files, spreadsheets, or anything else +you ask Copilot to read in a session. + +One file is required by the issue pipeline: + +- `ai-ranked-recommendations.csv` — the ranked recommendations that + `/create-issues` reads. The `/rank-recommendations` prompt produces it from + the per-area recommendations files in `../output/analysis/`. Columns: + - `Section` — `Project Documentation`, `Contributor Documentation`, or + `Website & Infrastructure`. + - `Area` — the area display name used in the analysis (for example + `Information Architecture`), not the criteria filename. + - `Recommendation` — the recommendation text, one per row. + - `Suggested Priority` — `1 - Urgent`, `2 - High`, or `3 - Backlog`. diff --git a/docs/analysis/ai-harness/_github/output/analysis/readme.md b/docs/analysis/ai-harness/_github/output/analysis/readme.md new file mode 100644 index 00000000..46e9a655 --- /dev/null +++ b/docs/analysis/ai-harness/_github/output/analysis/readme.md @@ -0,0 +1,8 @@ +# Analysis output + +The area prompts (`/-answers`, `/-comment`, +`/-recommendations`, `/-full`) and the section prompts +(`/
-comment`) write their Markdown output here. The +`/rank-recommendations` and `/create-issues` prompts read from this folder. + +This readme exists so Git keeps the folder; leave it in place. diff --git a/docs/analysis/ai-harness/_github/output/issues/readme.md b/docs/analysis/ai-harness/_github/output/issues/readme.md new file mode 100644 index 00000000..442a85f7 --- /dev/null +++ b/docs/analysis/ai-harness/_github/output/issues/readme.md @@ -0,0 +1,10 @@ +# Issue output + +`/create-issues` writes numbered standalone issue files here +(`{issue-number}-{condensed-title}.md`) and `/create-umbrella-issues` writes +umbrella issue files (`U-{issue-number}-{condensed-title}.md`). The upload +process treats every `*.md` file in this folder except this readme as an issue +to create. + +To keep a previous run, move its files into a sub-folder (for example +`prev_1/`); the prompts ignore sub-folders. diff --git a/docs/analysis/ai-harness/_github/output/readme.md b/docs/analysis/ai-harness/_github/output/readme.md new file mode 100644 index 00000000..82380b8d --- /dev/null +++ b/docs/analysis/ai-harness/_github/output/readme.md @@ -0,0 +1,11 @@ +# About this folder + +This folder contains two sub-folders for Copilot to place output: + +- `analysis/` — per-area and per-section output files written by the area and + section prompts (`--.md`). +- `issues/` — numbered standalone and umbrella issue files written by + `/create-issues` and `/create-umbrella-issues`. + +The `/create-issues` prompt also writes `issues-coverage.md` directly in this +folder, outside `issues/`, so it is not uploaded as an issue. diff --git a/docs/analysis/ai-harness/_github/prompts/areas/agent-readiness-answers.prompt.md b/docs/analysis/ai-harness/_github/prompts/areas/agent-readiness-answers.prompt.md new file mode 100644 index 00000000..e2492dd2 --- /dev/null +++ b/docs/analysis/ai-harness/_github/prompts/areas/agent-readiness-answers.prompt.md @@ -0,0 +1,20 @@ +--- +mode: agent +description: + Answer the CNCF TechDocs AI agent readiness questions for the documentation. +--- + +# AI Agent Readiness answers + +Run the shared **answers** analysis for the `agent-readiness` criteria area. + +1. Read the criteria definition in `.github/criteria/areas/agent-readiness.md`. +2. Read and follow the shared procedure in + `.github/prompts/areas/answers.prompt.md`, treating `agent-readiness` as the + selected area. + +Optional output filename override: `${input:title}` — if non-empty, use it as +the output filename; otherwise use the engine's default +(`--answers.md`). + +Output the file to the `.github/output/analysis` directory. diff --git a/docs/analysis/ai-harness/_github/prompts/areas/agent-readiness-comment.prompt.md b/docs/analysis/ai-harness/_github/prompts/areas/agent-readiness-comment.prompt.md new file mode 100644 index 00000000..c91214d1 --- /dev/null +++ b/docs/analysis/ai-harness/_github/prompts/areas/agent-readiness-comment.prompt.md @@ -0,0 +1,19 @@ +--- +mode: agent +description: Write a comment on the AI agent readiness of the documentation. +--- + +# AI Agent Readiness comment + +Run the shared **comment** analysis for the `agent-readiness` criteria area. + +1. Read the criteria definition in `.github/criteria/areas/agent-readiness.md`. +2. Read and follow the shared procedure in + `.github/prompts/areas/comment.prompt.md`, treating `agent-readiness` as the + selected area. + +Optional output filename override: `${input:title}` — if non-empty, use it as +the output filename; otherwise use the engine's default +(`--comment.md`). + +Output the file to the `.github/output/analysis` directory. diff --git a/docs/analysis/ai-harness/_github/prompts/areas/agent-readiness-full.prompt.md b/docs/analysis/ai-harness/_github/prompts/areas/agent-readiness-full.prompt.md new file mode 100644 index 00000000..0252c544 --- /dev/null +++ b/docs/analysis/ai-harness/_github/prompts/areas/agent-readiness-full.prompt.md @@ -0,0 +1,22 @@ +--- +mode: agent +description: + Run the full `agent-readiness` analysis (answers, comment, and + recommendations) for the documentation. +--- + +# AI Agent Readiness full analysis + +Run the shared **full** analysis for the `agent-readiness` criteria area. + +1. Read the criteria definition in `.github/criteria/areas/agent-readiness.md`. +2. Read and follow the shared procedure in + `.github/prompts/areas/full.prompt.md`, treating `agent-readiness` as the + selected area. + +Optional output filename override: `${input:title}` — if non-empty, use it as +the base name for all three output files (`-answers.md`, +`<title>-comment.md`, `<title>-recommendations.md`); otherwise use each engine's +default (`<project-slug>-<stem>-<type>.md`). + +Output the file to the `.github/output/analysis` directory. diff --git a/docs/analysis/ai-harness/_github/prompts/areas/agent-readiness-recommendations.prompt.md b/docs/analysis/ai-harness/_github/prompts/areas/agent-readiness-recommendations.prompt.md new file mode 100644 index 00000000..ef84718d --- /dev/null +++ b/docs/analysis/ai-harness/_github/prompts/areas/agent-readiness-recommendations.prompt.md @@ -0,0 +1,22 @@ +--- +mode: agent +description: + Provide recommendations for improving the AI agent readiness of the + documentation. +--- + +# AI Agent Readiness recommendations + +Run the shared **recommendations** analysis for the `agent-readiness` criteria +area. + +1. Read the criteria definition in `.github/criteria/areas/agent-readiness.md`. +2. Read and follow the shared procedure in + `.github/prompts/areas/recommendations.prompt.md`, treating `agent-readiness` + as the selected area. + +Optional output filename override: `${input:title}` — if non-empty, use it as +the output filename; otherwise use the engine's default +(`<project-slug>-<stem>-recommendations.md`). + +Output the file to the `.github/output/analysis` directory. diff --git a/docs/analysis/ai-harness/_github/prompts/areas/answers.prompt.md b/docs/analysis/ai-harness/_github/prompts/areas/answers.prompt.md new file mode 100644 index 00000000..472e162c --- /dev/null +++ b/docs/analysis/ai-harness/_github/prompts/areas/answers.prompt.md @@ -0,0 +1,45 @@ +--- +mode: agent +description: + Shared engine for producing analysis answers. Normally run via a per-area + wrapper such as /information-architecture-answers. +--- + +# Shared engine: answers + +This is the reusable procedure for answering an analysis area's questions. It is +normally invoked through a per-area wrapper prompt (for example, +`/information-architecture-answers`), which selects the criteria area for you. + +## Inputs + +- Area — the criteria area to analyze. When run from a wrapper, the wrapper + names it. If you reached this engine without an area, ask the user to choose + one of the areas defined in `.github/criteria/areas/` (the area name is the + criteria filename without `.md`). +- Title (optional) — overrides the default output filename. + +## Procedure + +1. Read the criteria definition at `.github/criteria/areas/<area>.md`. It + defines the area's Display name, File stem, and Questions. +2. Determine the project name from the "Current Repository" section of the + repository Copilot instructions. Derive a project slug by lowercasing the + name and replacing spaces with hyphens (for example, "KubeVirt" becomes + `kubevirt`). +3. Answer every question in the criteria file's "Questions" section, using the + sources described in the "Current Repository" section as context. + +## Output + +- Present the answers as indented paragraphs by two spaces under each question. + Format each question as a bullet, not as a heading. Do not bold the answers. +- If an answer is "yes" or "no", provide a brief explanation. +- Aim for two to three paragraphs. +- Write the result to a Markdown file in the `.github/output/analysis` + directory: + - If a title was provided, name the file `<title>.md`. + - Otherwise name it `<project-slug>-<stem>-answers.md`. + - If a file exists by the same name, overwrite it. + +Output the file to the `.github/output/analysis` directory. diff --git a/docs/analysis/ai-harness/_github/prompts/areas/beginner-issue-backlog-answers.prompt.md b/docs/analysis/ai-harness/_github/prompts/areas/beginner-issue-backlog-answers.prompt.md new file mode 100644 index 00000000..e7e84104 --- /dev/null +++ b/docs/analysis/ai-harness/_github/prompts/areas/beginner-issue-backlog-answers.prompt.md @@ -0,0 +1,23 @@ +--- +mode: agent +description: + Answer the CNCF TechDocs beginner friendly issue backlog questions for the + documentation. +--- + +# Beginner Friendly Issue Backlog answers + +Run the shared **answers** analysis for the `beginner-issue-backlog` criteria +area. + +1. Read the criteria definition in + `.github/criteria/areas/beginner-issue-backlog.md`. +2. Read and follow the shared procedure in + `.github/prompts/areas/answers.prompt.md`, treating `beginner-issue-backlog` + as the selected area. + +Optional output filename override: `${input:title}` — if non-empty, use it as +the output filename; otherwise use the engine's default +(`<project-slug>-<stem>-answers.md`). + +Output the file to the `.github/output/analysis` directory. diff --git a/docs/analysis/ai-harness/_github/prompts/areas/beginner-issue-backlog-comment.prompt.md b/docs/analysis/ai-harness/_github/prompts/areas/beginner-issue-backlog-comment.prompt.md new file mode 100644 index 00000000..db686c6d --- /dev/null +++ b/docs/analysis/ai-harness/_github/prompts/areas/beginner-issue-backlog-comment.prompt.md @@ -0,0 +1,22 @@ +--- +mode: agent +description: + Write a comment on the beginner friendly issue backlog of the documentation. +--- + +# Beginner Friendly Issue Backlog comment + +Run the shared **comment** analysis for the `beginner-issue-backlog` criteria +area. + +1. Read the criteria definition in + `.github/criteria/areas/beginner-issue-backlog.md`. +2. Read and follow the shared procedure in + `.github/prompts/areas/comment.prompt.md`, treating `beginner-issue-backlog` + as the selected area. + +Optional output filename override: `${input:title}` — if non-empty, use it as +the output filename; otherwise use the engine's default +(`<project-slug>-<stem>-comment.md`). + +Output the file to the `.github/output/analysis` directory. diff --git a/docs/analysis/ai-harness/_github/prompts/areas/beginner-issue-backlog-full.prompt.md b/docs/analysis/ai-harness/_github/prompts/areas/beginner-issue-backlog-full.prompt.md new file mode 100644 index 00000000..bf5b12c3 --- /dev/null +++ b/docs/analysis/ai-harness/_github/prompts/areas/beginner-issue-backlog-full.prompt.md @@ -0,0 +1,23 @@ +--- +mode: agent +description: + Run the full `beginner-issue-backlog` analysis (answers, comment, and + recommendations) for the documentation. +--- + +# Beginner Friendly Issue Backlog full analysis + +Run the shared **full** analysis for the `beginner-issue-backlog` criteria area. + +1. Read the criteria definition in + `.github/criteria/areas/beginner-issue-backlog.md`. +2. Read and follow the shared procedure in + `.github/prompts/areas/full.prompt.md`, treating `beginner-issue-backlog` as + the selected area. + +Optional output filename override: `${input:title}` — if non-empty, use it as +the base name for all three output files (`<title>-answers.md`, +`<title>-comment.md`, `<title>-recommendations.md`); otherwise use each engine's +default (`<project-slug>-<stem>-<type>.md`). + +Output the file to the `.github/output/analysis` directory. diff --git a/docs/analysis/ai-harness/_github/prompts/areas/beginner-issue-backlog-recommendations.prompt.md b/docs/analysis/ai-harness/_github/prompts/areas/beginner-issue-backlog-recommendations.prompt.md new file mode 100644 index 00000000..ae95b46d --- /dev/null +++ b/docs/analysis/ai-harness/_github/prompts/areas/beginner-issue-backlog-recommendations.prompt.md @@ -0,0 +1,23 @@ +--- +mode: agent +description: + Provide recommendations for improving the beginner friendly issue backlog of + the documentation. +--- + +# Beginner Friendly Issue Backlog recommendations + +Run the shared **recommendations** analysis for the `beginner-issue-backlog` +criteria area. + +1. Read the criteria definition in + `.github/criteria/areas/beginner-issue-backlog.md`. +2. Read and follow the shared procedure in + `.github/prompts/areas/recommendations.prompt.md`, treating + `beginner-issue-backlog` as the selected area. + +Optional output filename override: `${input:title}` — if non-empty, use it as +the output filename; otherwise use the engine's default +(`<project-slug>-<stem>-recommendations.md`). + +Output the file to the `.github/output/analysis` directory. diff --git a/docs/analysis/ai-harness/_github/prompts/areas/branding-design-answers.prompt.md b/docs/analysis/ai-harness/_github/prompts/areas/branding-design-answers.prompt.md new file mode 100644 index 00000000..c20c7bee --- /dev/null +++ b/docs/analysis/ai-harness/_github/prompts/areas/branding-design-answers.prompt.md @@ -0,0 +1,20 @@ +--- +mode: agent +description: + Answer the CNCF TechDocs branding and design questions for the documentation. +--- + +# Branding and Design answers + +Run the shared **answers** analysis for the `branding-design` criteria area. + +1. Read the criteria definition in `.github/criteria/areas/branding-design.md`. +2. Read and follow the shared procedure in + `.github/prompts/areas/answers.prompt.md`, treating `branding-design` as the + selected area. + +Optional output filename override: `${input:title}` — if non-empty, use it as +the output filename; otherwise use the engine's default +(`<project-slug>-<stem>-answers.md`). + +Output the file to the `.github/output/analysis` directory. diff --git a/docs/analysis/ai-harness/_github/prompts/areas/branding-design-comment.prompt.md b/docs/analysis/ai-harness/_github/prompts/areas/branding-design-comment.prompt.md new file mode 100644 index 00000000..c0ed58d5 --- /dev/null +++ b/docs/analysis/ai-harness/_github/prompts/areas/branding-design-comment.prompt.md @@ -0,0 +1,19 @@ +--- +mode: agent +description: Write a comment on the branding and design of the documentation. +--- + +# Branding and Design comment + +Run the shared **comment** analysis for the `branding-design` criteria area. + +1. Read the criteria definition in `.github/criteria/areas/branding-design.md`. +2. Read and follow the shared procedure in + `.github/prompts/areas/comment.prompt.md`, treating `branding-design` as the + selected area. + +Optional output filename override: `${input:title}` — if non-empty, use it as +the output filename; otherwise use the engine's default +(`<project-slug>-<stem>-comment.md`). + +Output the file to the `.github/output/analysis` directory. diff --git a/docs/analysis/ai-harness/_github/prompts/areas/branding-design-full.prompt.md b/docs/analysis/ai-harness/_github/prompts/areas/branding-design-full.prompt.md new file mode 100644 index 00000000..b19dc37d --- /dev/null +++ b/docs/analysis/ai-harness/_github/prompts/areas/branding-design-full.prompt.md @@ -0,0 +1,22 @@ +--- +mode: agent +description: + Run the full `branding-design` analysis (answers, comment, and + recommendations) for the documentation. +--- + +# Branding and Design full analysis + +Run the shared **full** analysis for the `branding-design` criteria area. + +1. Read the criteria definition in `.github/criteria/areas/branding-design.md`. +2. Read and follow the shared procedure in + `.github/prompts/areas/full.prompt.md`, treating `branding-design` as the + selected area. + +Optional output filename override: `${input:title}` — if non-empty, use it as +the base name for all three output files (`<title>-answers.md`, +`<title>-comment.md`, `<title>-recommendations.md`); otherwise use each engine's +default (`<project-slug>-<stem>-<type>.md`). + +Output the file to the `.github/output/analysis` directory. diff --git a/docs/analysis/ai-harness/_github/prompts/areas/branding-design-recommendations.prompt.md b/docs/analysis/ai-harness/_github/prompts/areas/branding-design-recommendations.prompt.md new file mode 100644 index 00000000..f09f2866 --- /dev/null +++ b/docs/analysis/ai-harness/_github/prompts/areas/branding-design-recommendations.prompt.md @@ -0,0 +1,22 @@ +--- +mode: agent +description: + Provide recommendations for improving the branding and design of the + documentation. +--- + +# Branding and Design recommendations + +Run the shared **recommendations** analysis for the `branding-design` criteria +area. + +1. Read the criteria definition in `.github/criteria/areas/branding-design.md`. +2. Read and follow the shared procedure in + `.github/prompts/areas/recommendations.prompt.md`, treating `branding-design` + as the selected area. + +Optional output filename override: `${input:title}` — if non-empty, use it as +the output filename; otherwise use the engine's default +(`<project-slug>-<stem>-recommendations.md`). + +Output the file to the `.github/output/analysis` directory. diff --git a/docs/analysis/ai-harness/_github/prompts/areas/case-studies-answers.prompt.md b/docs/analysis/ai-harness/_github/prompts/areas/case-studies-answers.prompt.md new file mode 100644 index 00000000..ca43c9b0 --- /dev/null +++ b/docs/analysis/ai-harness/_github/prompts/areas/case-studies-answers.prompt.md @@ -0,0 +1,20 @@ +--- +mode: agent +description: + Answer the CNCF TechDocs case studies questions for the documentation. +--- + +# Case Studies answers + +Run the shared **answers** analysis for the `case-studies` criteria area. + +1. Read the criteria definition in `.github/criteria/areas/case-studies.md`. +2. Read and follow the shared procedure in + `.github/prompts/areas/answers.prompt.md`, treating `case-studies` as the + selected area. + +Optional output filename override: `${input:title}` — if non-empty, use it as +the output filename; otherwise use the engine's default +(`<project-slug>-<stem>-answers.md`). + +Output the file to the `.github/output/analysis` directory. diff --git a/docs/analysis/ai-harness/_github/prompts/areas/case-studies-comment.prompt.md b/docs/analysis/ai-harness/_github/prompts/areas/case-studies-comment.prompt.md new file mode 100644 index 00000000..5007f16d --- /dev/null +++ b/docs/analysis/ai-harness/_github/prompts/areas/case-studies-comment.prompt.md @@ -0,0 +1,19 @@ +--- +mode: agent +description: Write a comment on the case studies of the documentation. +--- + +# Case Studies comment + +Run the shared **comment** analysis for the `case-studies` criteria area. + +1. Read the criteria definition in `.github/criteria/areas/case-studies.md`. +2. Read and follow the shared procedure in + `.github/prompts/areas/comment.prompt.md`, treating `case-studies` as the + selected area. + +Optional output filename override: `${input:title}` — if non-empty, use it as +the output filename; otherwise use the engine's default +(`<project-slug>-<stem>-comment.md`). + +Output the file to the `.github/output/analysis` directory. diff --git a/docs/analysis/ai-harness/_github/prompts/areas/case-studies-full.prompt.md b/docs/analysis/ai-harness/_github/prompts/areas/case-studies-full.prompt.md new file mode 100644 index 00000000..a93c5b83 --- /dev/null +++ b/docs/analysis/ai-harness/_github/prompts/areas/case-studies-full.prompt.md @@ -0,0 +1,22 @@ +--- +mode: agent +description: + Run the full `case-studies` analysis (answers, comment, and recommendations) + for the documentation. +--- + +# Case Studies full analysis + +Run the shared **full** analysis for the `case-studies` criteria area. + +1. Read the criteria definition in `.github/criteria/areas/case-studies.md`. +2. Read and follow the shared procedure in + `.github/prompts/areas/full.prompt.md`, treating `case-studies` as the + selected area. + +Optional output filename override: `${input:title}` — if non-empty, use it as +the base name for all three output files (`<title>-answers.md`, +`<title>-comment.md`, `<title>-recommendations.md`); otherwise use each engine's +default (`<project-slug>-<stem>-<type>.md`). + +Output the file to the `.github/output/analysis` directory. diff --git a/docs/analysis/ai-harness/_github/prompts/areas/case-studies-recommendations.prompt.md b/docs/analysis/ai-harness/_github/prompts/areas/case-studies-recommendations.prompt.md new file mode 100644 index 00000000..34c84c9e --- /dev/null +++ b/docs/analysis/ai-harness/_github/prompts/areas/case-studies-recommendations.prompt.md @@ -0,0 +1,21 @@ +--- +mode: agent +description: + Provide recommendations for improving the case studies of the documentation. +--- + +# Case Studies recommendations + +Run the shared **recommendations** analysis for the `case-studies` criteria +area. + +1. Read the criteria definition in `.github/criteria/areas/case-studies.md`. +2. Read and follow the shared procedure in + `.github/prompts/areas/recommendations.prompt.md`, treating `case-studies` as + the selected area. + +Optional output filename override: `${input:title}` — if non-empty, use it as +the output filename; otherwise use the engine's default +(`<project-slug>-<stem>-recommendations.md`). + +Output the file to the `.github/output/analysis` directory. diff --git a/docs/analysis/ai-harness/_github/prompts/areas/comment.prompt.md b/docs/analysis/ai-harness/_github/prompts/areas/comment.prompt.md new file mode 100644 index 00000000..df3de2bb --- /dev/null +++ b/docs/analysis/ai-harness/_github/prompts/areas/comment.prompt.md @@ -0,0 +1,54 @@ +--- +mode: agent +description: + Shared engine for producing an analysis comment. Normally run via a per-area + wrapper such as /information-architecture-comment. +--- + +# Shared engine: comment + +Reusable procedure for writing an analysis comment for one area. Normally +invoked through a per-area wrapper prompt (for example, +`/information-architecture-comment`), which selects the criteria area for you. + +## Inputs + +- Area — the criteria area to analyze. When run from a wrapper, the wrapper + names it. +- Title (optional) — overrides the default output filename. + +## Procedure + +1. Read the criteria definition at `.github/criteria/areas/<area>.md`. Use its + Display name, File stem, and Comment guidance (good examples and any + additional instructions). +2. Determine the project name and documentation label from the "Current + Repository" section of the repository Copilot instructions (for example, "the + KubeVirt user guide"). Derive the project slug by lowercasing the name and + replacing spaces with hyphens. +3. Write one to three paragraphs for a high-level comment. No need to replicate + points from the answers output. Follow with two bulleted lists: one of + strengths and one of weaknesses. Preceded the lists with labels (not bold) + "Strengths:" and "Weaknesses:". +4. You can follow with any "Additional instructions" from the criteria file. You + may reference the listed "good examples" as models of quality. +5. Add a rating for this area, using the CNCF 1–5 scale: + - 1 - Not present + - 2 - Needs improvement + - 3 - Meets standards + - 4 - Meets or exceeds standards + - 5 - Exemplary + + State the rating on its own line at the end of the comment in the form + `Rating: <number> - <label>` (for example, `Rating: 3 - Meets standards`). Do + not include any other text on the rating line. + +## Output + +- Write the result to a Markdown file in the `.github/output/analysis` + directory: + - If a title was provided, name the file `<title>.md`. + - Otherwise name it `<project-slug>-<stem>-comment.md`. + - If a file exists by the same name, overwrite it. + +Output the file to the `.github/output/analysis` directory. diff --git a/docs/analysis/ai-harness/_github/prompts/areas/communication-methods-documented-answers.prompt.md b/docs/analysis/ai-harness/_github/prompts/areas/communication-methods-documented-answers.prompt.md new file mode 100644 index 00000000..21cadd8a --- /dev/null +++ b/docs/analysis/ai-harness/_github/prompts/areas/communication-methods-documented-answers.prompt.md @@ -0,0 +1,23 @@ +--- +mode: agent +description: + Answer the CNCF TechDocs communication methods documented questions for the + documentation. +--- + +# Communication Methods Documented answers + +Run the shared **answers** analysis for the `communication-methods-documented` +criteria area. + +1. Read the criteria definition in + `.github/criteria/areas/communication-methods-documented.md`. +2. Read and follow the shared procedure in + `.github/prompts/areas/answers.prompt.md`, treating + `communication-methods-documented` as the selected area. + +Optional output filename override: `${input:title}` — if non-empty, use it as +the output filename; otherwise use the engine's default +(`<project-slug>-<stem>-answers.md`). + +Output the file to the `.github/output/analysis` directory. diff --git a/docs/analysis/ai-harness/_github/prompts/areas/communication-methods-documented-comment.prompt.md b/docs/analysis/ai-harness/_github/prompts/areas/communication-methods-documented-comment.prompt.md new file mode 100644 index 00000000..63ea60e4 --- /dev/null +++ b/docs/analysis/ai-harness/_github/prompts/areas/communication-methods-documented-comment.prompt.md @@ -0,0 +1,22 @@ +--- +mode: agent +description: + Write a comment on the communication methods documented of the documentation. +--- + +# Communication Methods Documented comment + +Run the shared **comment** analysis for the `communication-methods-documented` +criteria area. + +1. Read the criteria definition in + `.github/criteria/areas/communication-methods-documented.md`. +2. Read and follow the shared procedure in + `.github/prompts/areas/comment.prompt.md`, treating + `communication-methods-documented` as the selected area. + +Optional output filename override: `${input:title}` — if non-empty, use it as +the output filename; otherwise use the engine's default +(`<project-slug>-<stem>-comment.md`). + +Output the file to the `.github/output/analysis` directory. diff --git a/docs/analysis/ai-harness/_github/prompts/areas/communication-methods-documented-full.prompt.md b/docs/analysis/ai-harness/_github/prompts/areas/communication-methods-documented-full.prompt.md new file mode 100644 index 00000000..209181fe --- /dev/null +++ b/docs/analysis/ai-harness/_github/prompts/areas/communication-methods-documented-full.prompt.md @@ -0,0 +1,24 @@ +--- +mode: agent +description: + Run the full `communication-methods-documented` analysis (answers, comment, + and recommendations) for the documentation. +--- + +# Communication Methods Documented full analysis + +Run the shared **full** analysis for the `communication-methods-documented` +criteria area. + +1. Read the criteria definition in + `.github/criteria/areas/communication-methods-documented.md`. +2. Read and follow the shared procedure in + `.github/prompts/areas/full.prompt.md`, treating + `communication-methods-documented` as the selected area. + +Optional output filename override: `${input:title}` — if non-empty, use it as +the base name for all three output files (`<title>-answers.md`, +`<title>-comment.md`, `<title>-recommendations.md`); otherwise use each engine's +default (`<project-slug>-<stem>-<type>.md`). + +Output the file to the `.github/output/analysis` directory. diff --git a/docs/analysis/ai-harness/_github/prompts/areas/communication-methods-documented-recommendations.prompt.md b/docs/analysis/ai-harness/_github/prompts/areas/communication-methods-documented-recommendations.prompt.md new file mode 100644 index 00000000..62bf92ed --- /dev/null +++ b/docs/analysis/ai-harness/_github/prompts/areas/communication-methods-documented-recommendations.prompt.md @@ -0,0 +1,23 @@ +--- +mode: agent +description: + Provide recommendations for improving the communication methods documented of + the documentation. +--- + +# Communication Methods Documented recommendations + +Run the shared **recommendations** analysis for the +`communication-methods-documented` criteria area. + +1. Read the criteria definition in + `.github/criteria/areas/communication-methods-documented.md`. +2. Read and follow the shared procedure in + `.github/prompts/areas/recommendations.prompt.md`, treating + `communication-methods-documented` as the selected area. + +Optional output filename override: `${input:title}` — if non-empty, use it as +the output filename; otherwise use the engine's default +(`<project-slug>-<stem>-recommendations.md`). + +Output the file to the `.github/output/analysis` directory. diff --git a/docs/analysis/ai-harness/_github/prompts/areas/content-creation-process-answers.prompt.md b/docs/analysis/ai-harness/_github/prompts/areas/content-creation-process-answers.prompt.md new file mode 100644 index 00000000..a44f81cb --- /dev/null +++ b/docs/analysis/ai-harness/_github/prompts/areas/content-creation-process-answers.prompt.md @@ -0,0 +1,23 @@ +--- +mode: agent +description: + Answer the CNCF TechDocs content creation process questions for the + documentation. +--- + +# Content Creation Process answers + +Run the shared **answers** analysis for the `content-creation-process` criteria +area. + +1. Read the criteria definition in + `.github/criteria/areas/content-creation-process.md`. +2. Read and follow the shared procedure in + `.github/prompts/areas/answers.prompt.md`, treating + `content-creation-process` as the selected area. + +Optional output filename override: `${input:title}` — if non-empty, use it as +the output filename; otherwise use the engine's default +(`<project-slug>-<stem>-answers.md`). + +Output the file to the `.github/output/analysis` directory. diff --git a/docs/analysis/ai-harness/_github/prompts/areas/content-creation-process-comment.prompt.md b/docs/analysis/ai-harness/_github/prompts/areas/content-creation-process-comment.prompt.md new file mode 100644 index 00000000..267bf5e6 --- /dev/null +++ b/docs/analysis/ai-harness/_github/prompts/areas/content-creation-process-comment.prompt.md @@ -0,0 +1,22 @@ +--- +mode: agent +description: + Write a comment on the content creation process of the documentation. +--- + +# Content Creation Process comment + +Run the shared **comment** analysis for the `content-creation-process` criteria +area. + +1. Read the criteria definition in + `.github/criteria/areas/content-creation-process.md`. +2. Read and follow the shared procedure in + `.github/prompts/areas/comment.prompt.md`, treating + `content-creation-process` as the selected area. + +Optional output filename override: `${input:title}` — if non-empty, use it as +the output filename; otherwise use the engine's default +(`<project-slug>-<stem>-comment.md`). + +Output the file to the `.github/output/analysis` directory. diff --git a/docs/analysis/ai-harness/_github/prompts/areas/content-creation-process-full.prompt.md b/docs/analysis/ai-harness/_github/prompts/areas/content-creation-process-full.prompt.md new file mode 100644 index 00000000..2afc94db --- /dev/null +++ b/docs/analysis/ai-harness/_github/prompts/areas/content-creation-process-full.prompt.md @@ -0,0 +1,24 @@ +--- +mode: agent +description: + Run the full content creation process analysis (answers, comment, and + recommendations) for the documentation. +--- + +# Content Creation Process full analysis + +Run the shared **full** analysis for the `content-creation-process` criteria +area. + +1. Read the criteria definition in + `.github/criteria/areas/content-creation-process.md`. +2. Read and follow the shared procedure in + `.github/prompts/areas/full.prompt.md`, treating `content-creation-process` + as the selected area. + +Optional output filename override: `${input:title}` — if non-empty, use it as +the base name for all three output files (`<title>-answers.md`, +`<title>-comment.md`, `<title>-recommendations.md`); otherwise use each engine's +default (`<project-slug>-<stem>-<type>.md`). + +Output the file to the `.github/output/analysis` directory. diff --git a/docs/analysis/ai-harness/_github/prompts/areas/content-creation-process-recommendations.prompt.md b/docs/analysis/ai-harness/_github/prompts/areas/content-creation-process-recommendations.prompt.md new file mode 100644 index 00000000..2254b7c4 --- /dev/null +++ b/docs/analysis/ai-harness/_github/prompts/areas/content-creation-process-recommendations.prompt.md @@ -0,0 +1,23 @@ +--- +mode: agent +description: + Provide recommendations for improving the content creation process of the + documentation. +--- + +# Content Creation Process recommendations + +Run the shared **recommendations** analysis for the `content-creation-process` +criteria area. + +1. Read the criteria definition in + `.github/criteria/areas/content-creation-process.md`. +2. Read and follow the shared procedure in + `.github/prompts/areas/recommendations.prompt.md`, treating + `content-creation-process` as the selected area. + +Optional output filename override: `${input:title}` — if non-empty, use it as +the output filename; otherwise use the engine's default +(`<project-slug>-<stem>-recommendations.md`). + +Output the file to the `.github/output/analysis` directory. diff --git a/docs/analysis/ai-harness/_github/prompts/areas/content-maintainability-answers.prompt.md b/docs/analysis/ai-harness/_github/prompts/areas/content-maintainability-answers.prompt.md new file mode 100644 index 00000000..120c4fa0 --- /dev/null +++ b/docs/analysis/ai-harness/_github/prompts/areas/content-maintainability-answers.prompt.md @@ -0,0 +1,23 @@ +--- +mode: agent +description: + Answer the CNCF TechDocs content maintainability questions for the + documentation. +--- + +# Content Maintainability answers + +Run the shared **answers** analysis for the `content-maintainability` criteria +area. + +1. Read the criteria definition in + `.github/criteria/areas/content-maintainability.md`. +2. Read and follow the shared procedure in + `.github/prompts/areas/answers.prompt.md`, treating `content-maintainability` + as the selected area. + +Optional output filename override: `${input:title}` — if non-empty, use it as +the output filename; otherwise use the engine's default +(`<project-slug>-<stem>-answers.md`). + +Output the file to the `.github/output/analysis` directory. diff --git a/docs/analysis/ai-harness/_github/prompts/areas/content-maintainability-comment.prompt.md b/docs/analysis/ai-harness/_github/prompts/areas/content-maintainability-comment.prompt.md new file mode 100644 index 00000000..e2e0f1b4 --- /dev/null +++ b/docs/analysis/ai-harness/_github/prompts/areas/content-maintainability-comment.prompt.md @@ -0,0 +1,22 @@ +--- +mode: agent +description: + Write a comment on the content maintainability of the documentation. +--- + +# Content Maintainability comment + +Run the shared **comment** analysis for the `content-maintainability` criteria +area. + +1. Read the criteria definition in + `.github/criteria/areas/content-maintainability.md`. +2. Read and follow the shared procedure in + `.github/prompts/areas/comment.prompt.md`, treating `content-maintainability` + as the selected area. + +Optional output filename override: `${input:title}` — if non-empty, use it as +the output filename; otherwise use the engine's default +(`<project-slug>-<stem>-comment.md`). + +Output the file to the `.github/output/analysis` directory. diff --git a/docs/analysis/ai-harness/_github/prompts/areas/content-maintainability-full.prompt.md b/docs/analysis/ai-harness/_github/prompts/areas/content-maintainability-full.prompt.md new file mode 100644 index 00000000..8719e3a0 --- /dev/null +++ b/docs/analysis/ai-harness/_github/prompts/areas/content-maintainability-full.prompt.md @@ -0,0 +1,24 @@ +--- +mode: agent +description: + Run the full content maintainability analysis (answers, comment, and + recommendations) for the documentation. +--- + +# Content Maintainability full analysis + +Run the shared **full** analysis for the `content-maintainability` criteria +area. + +1. Read the criteria definition in + `.github/criteria/areas/content-maintainability.md`. +2. Read and follow the shared procedure in + `.github/prompts/areas/full.prompt.md`, treating `content-maintainability` as + the selected area. + +Optional output filename override: `${input:title}` — if non-empty, use it as +the base name for all three output files (`<title>-answers.md`, +`<title>-comment.md`, `<title>-recommendations.md`); otherwise use each engine's +default (`<project-slug>-<stem>-<type>.md`). + +Output the file to the `.github/output/analysis` directory. diff --git a/docs/analysis/ai-harness/_github/prompts/areas/content-maintainability-recommendations.prompt.md b/docs/analysis/ai-harness/_github/prompts/areas/content-maintainability-recommendations.prompt.md new file mode 100644 index 00000000..b13ee6b7 --- /dev/null +++ b/docs/analysis/ai-harness/_github/prompts/areas/content-maintainability-recommendations.prompt.md @@ -0,0 +1,23 @@ +--- +mode: agent +description: + Provide recommendations for improving the content maintainability of the + documentation. +--- + +# Content Maintainability recommendations + +Run the shared **recommendations** analysis for the `content-maintainability` +criteria area. + +1. Read the criteria definition in + `.github/criteria/areas/content-maintainability.md`. +2. Read and follow the shared procedure in + `.github/prompts/areas/recommendations.prompt.md`, treating + `content-maintainability` as the selected area. + +Optional output filename override: `${input:title}` — if non-empty, use it as +the output filename; otherwise use the engine's default +(`<project-slug>-<stem>-recommendations.md`). + +Output the file to the `.github/output/analysis` directory. diff --git a/docs/analysis/ai-harness/_github/prompts/areas/full.prompt.md b/docs/analysis/ai-harness/_github/prompts/areas/full.prompt.md new file mode 100644 index 00000000..cd9d87b8 --- /dev/null +++ b/docs/analysis/ai-harness/_github/prompts/areas/full.prompt.md @@ -0,0 +1,103 @@ +--- +mode: agent +description: + Shared engine for running all three analyses (answers, comment, + recommendations) for one area. Normally run via a per-area wrapper such as + /information-architecture-full. +--- + +# Shared engine: full + +Reusable procedure for running the complete analysis of one criteria area: it +gathers the material for the **answers**, **comment**, and **recommendations** +engines in a single pass, sorts it by output type, and then writes the three +files in sequence. Normally invoked through a per-area wrapper prompt (for +example, `/information-architecture-full`), which selects the criteria area for +you. + +## Inputs + +- Area — the criteria area to analyze. When run from a wrapper, the wrapper + names it. If you reached this engine without an area, ask the user to choose + one of the areas defined in `.github/criteria/areas/` (the area name is the + criteria filename without `.md`). +- Title (optional) — a base name that overrides the default output filenames. + When provided, the outputs are `<title>-answers.md`, `<title>-comment.md`, and + `<title>-recommendations.md`. + +## Procedure + +Verify output style as Claude default, and review the "Analysis responses style" +guidelines in copilot-instructions.md. + +Analyze first, write last. While answering the questions and forming the comment +you will inevitably notice things to fix; those observations belong in the +recommendations file, not in the answers or comment. Collect everything before +writing so each file contains only its own kind of content. + +### Phase 1: Read + +1. Read the criteria definition at `.github/criteria/areas/<area>.md` once, so + its Display name, File stem, Questions, and Comment guidance are available to + every step below. +2. Read all three engines — `.github/prompts/areas/answers.prompt.md`, + `.github/prompts/areas/comment.prompt.md`, and + `.github/prompts/areas/recommendations.prompt.md` — so you know the required + structure of each output before you start analyzing. + +### Phase 2: Analyze and sort + +1. Investigate the documentation as the answers engine directs. As you work, + keep three running lists: + - Findings: factual observations that answer a question (what exists, where + it is, what is missing). + - Assessments: high-level judgments about strengths and weaknesses, and the + evidence for the rating. + - Fixes: anything you would tell the project to change or add. +2. Whenever an observation suggests a fix, record the fix in the Fixes list and + keep only the underlying finding in the Findings or Assessments list. A + finding may state that something is missing or unclear; it must not say what + to do about it. +3. Draft all three documents in memory from these lists: + - Answers, from the Findings list, following the answers engine's Output + section. + - Comment, from the Assessments list, following the comment engine's + Procedure and Output sections. + - Recommendations, from the Fixes list, following the recommendations + engine's Output section. Base each recommendation on a finding or + assessment that appears in the drafted answers or comment. +4. Review the drafts against each other before writing anything to disk: + - Remove any imperative or "should" phrasing from the answers and comment; + move it to the recommendations. + - Confirm every recommendation traces back to something stated in the answers + or comment. If it does not, add the supporting finding or drop the + recommendation. + - Confirm the answers and comment do not duplicate each other; the comment + synthesizes, it does not restate. + +### Phase 3: Write + +1. Write the three files in this order, since the recommendations engine expects + the first two to exist: + 1. Answers: `.github/prompts/areas/answers.prompt.md` output. + 2. Comment: `.github/prompts/areas/comment.prompt.md` output. + 3. Recommendations: `.github/prompts/areas/recommendations.prompt.md` output. + Because the sorting in Phase 2 already kept recommendations out of the + comment, its instruction to move stray recommendations out of the comment + file should find nothing to move. + +When a Title is provided, pass it through to each engine so the three files +share the `<title>` base name described under "Inputs". + +## Output + +- Produce all three Markdown files in the `.github/output/analysis` directory: + - Answers: `<title>-answers.md`, or `<project-slug>-<stem>-answers.md` by + default. + - Comment: `<title>-comment.md`, or `<project-slug>-<stem>-comment.md` by + default. + - Recommendations: `<title>-recommendations.md`, or + `<project-slug>-<stem>-recommendations.md` by default. +- After writing them, list the three filenames you created. + +Output the file to the `.github/output/analysis` directory. diff --git a/docs/analysis/ai-harness/_github/prompts/areas/inclusive-language-answers.prompt.md b/docs/analysis/ai-harness/_github/prompts/areas/inclusive-language-answers.prompt.md new file mode 100644 index 00000000..46369126 --- /dev/null +++ b/docs/analysis/ai-harness/_github/prompts/areas/inclusive-language-answers.prompt.md @@ -0,0 +1,21 @@ +--- +mode: agent +description: + Answer the CNCF TechDocs inclusive language questions for the documentation. +--- + +# Inclusive Language answers + +Run the shared **answers** analysis for the `inclusive-language` criteria area. + +1. Read the criteria definition in + `.github/criteria/areas/inclusive-language.md`. +2. Read and follow the shared procedure in + `.github/prompts/areas/answers.prompt.md`, treating `inclusive-language` as + the selected area. + +Optional output filename override: `${input:title}` — if non-empty, use it as +the output filename; otherwise use the engine's default +(`<project-slug>-<stem>-answers.md`). + +Output the file to the `.github/output/analysis` directory. diff --git a/docs/analysis/ai-harness/_github/prompts/areas/inclusive-language-comment.prompt.md b/docs/analysis/ai-harness/_github/prompts/areas/inclusive-language-comment.prompt.md new file mode 100644 index 00000000..349037f8 --- /dev/null +++ b/docs/analysis/ai-harness/_github/prompts/areas/inclusive-language-comment.prompt.md @@ -0,0 +1,20 @@ +--- +mode: agent +description: Write a comment on the inclusive language of the documentation. +--- + +# Inclusive Language comment + +Run the shared **comment** analysis for the `inclusive-language` criteria area. + +1. Read the criteria definition in + `.github/criteria/areas/inclusive-language.md`. +2. Read and follow the shared procedure in + `.github/prompts/areas/comment.prompt.md`, treating `inclusive-language` as + the selected area. + +Optional output filename override: `${input:title}` — if non-empty, use it as +the output filename; otherwise use the engine's default +(`<project-slug>-<stem>-comment.md`). + +Output the file to the `.github/output/analysis` directory. diff --git a/docs/analysis/ai-harness/_github/prompts/areas/inclusive-language-full.prompt.md b/docs/analysis/ai-harness/_github/prompts/areas/inclusive-language-full.prompt.md new file mode 100644 index 00000000..0b189c30 --- /dev/null +++ b/docs/analysis/ai-harness/_github/prompts/areas/inclusive-language-full.prompt.md @@ -0,0 +1,23 @@ +--- +mode: agent +description: + Run the full inclusive language analysis (answers, comment, and + recommendations) for the documentation. +--- + +# Inclusive Language full analysis + +Run the shared **full** analysis for the `inclusive-language` criteria area. + +1. Read the criteria definition in + `.github/criteria/areas/inclusive-language.md`. +2. Read and follow the shared procedure in + `.github/prompts/areas/full.prompt.md`, treating `inclusive-language` as the + selected area. + +Optional output filename override: `${input:title}` — if non-empty, use it as +the base name for all three output files (`<title>-answers.md`, +`<title>-comment.md`, `<title>-recommendations.md`); otherwise use each engine's +default (`<project-slug>-<stem>-<type>.md`). + +Output the file to the `.github/output/analysis` directory. diff --git a/docs/analysis/ai-harness/_github/prompts/areas/inclusive-language-recommendations.prompt.md b/docs/analysis/ai-harness/_github/prompts/areas/inclusive-language-recommendations.prompt.md new file mode 100644 index 00000000..421bcc23 --- /dev/null +++ b/docs/analysis/ai-harness/_github/prompts/areas/inclusive-language-recommendations.prompt.md @@ -0,0 +1,23 @@ +--- +mode: agent +description: + Provide recommendations for improving the inclusive language of the + documentation. +--- + +# Inclusive Language recommendations + +Run the shared **recommendations** analysis for the `inclusive-language` +criteria area. + +1. Read the criteria definition in + `.github/criteria/areas/inclusive-language.md`. +2. Read and follow the shared procedure in + `.github/prompts/areas/recommendations.prompt.md`, treating + `inclusive-language` as the selected area. + +Optional output filename override: `${input:title}` — if non-empty, use it as +the output filename; otherwise use the engine's default +(`<project-slug>-<stem>-recommendations.md`). + +Output the file to the `.github/output/analysis` directory. diff --git a/docs/analysis/ai-harness/_github/prompts/areas/information-architecture-answers.prompt.md b/docs/analysis/ai-harness/_github/prompts/areas/information-architecture-answers.prompt.md new file mode 100644 index 00000000..33253b18 --- /dev/null +++ b/docs/analysis/ai-harness/_github/prompts/areas/information-architecture-answers.prompt.md @@ -0,0 +1,23 @@ +--- +mode: agent +description: + Answer the CNCF TechDocs information architecture questions for the + documentation. +--- + +# Information Architecture answers + +Run the shared **answers** analysis for the `information-architecture` criteria +area. + +1. Read the criteria definition in + `.github/criteria/areas/information-architecture.md`. +2. Read and follow the shared procedure in + `.github/prompts/areas/answers.prompt.md`, treating + `information-architecture` as the selected area. + +Optional output filename override: `${input:title}` — if non-empty, use it as +the output filename; otherwise use the engine's default +(`<project-slug>-<stem>-answers.md`). + +Output the file to the `.github/output/analysis` directory. diff --git a/docs/analysis/ai-harness/_github/prompts/areas/information-architecture-comment.prompt.md b/docs/analysis/ai-harness/_github/prompts/areas/information-architecture-comment.prompt.md new file mode 100644 index 00000000..37f5792d --- /dev/null +++ b/docs/analysis/ai-harness/_github/prompts/areas/information-architecture-comment.prompt.md @@ -0,0 +1,22 @@ +--- +mode: agent +description: + Write a comment on the information architecture of the documentation. +--- + +# Information Architecture comment + +Run the shared **comment** analysis for the `information-architecture` criteria +area. + +1. Read the criteria definition in + `.github/criteria/areas/information-architecture.md`. +2. Read and follow the shared procedure in + `.github/prompts/areas/comment.prompt.md`, treating + `information-architecture` as the selected area. + +Optional output filename override: `${input:title}` — if non-empty, use it as +the output filename; otherwise use the engine's default +(`<project-slug>-<stem>-comment.md`). + +Output the file to the `.github/output/analysis` directory. diff --git a/docs/analysis/ai-harness/_github/prompts/areas/information-architecture-full.prompt.md b/docs/analysis/ai-harness/_github/prompts/areas/information-architecture-full.prompt.md new file mode 100644 index 00000000..272297a6 --- /dev/null +++ b/docs/analysis/ai-harness/_github/prompts/areas/information-architecture-full.prompt.md @@ -0,0 +1,24 @@ +--- +mode: agent +description: + Run the full information architecture analysis (answers, comment, and + recommendations) for the documentation. +--- + +# Information Architecture full analysis + +Run the shared **full** analysis for the `information-architecture` criteria +area. + +1. Read the criteria definition in + `.github/criteria/areas/information-architecture.md`. +2. Read and follow the shared procedure in + `.github/prompts/areas/full.prompt.md`, treating `information-architecture` + as the selected area. + +Optional output filename override: `${input:title}` — if non-empty, use it as +the base name for all three output files (`<title>-answers.md`, +`<title>-comment.md`, `<title>-recommendations.md`); otherwise use each engine's +default (`<project-slug>-<stem>-<type>.md`). + +Output the file to the `.github/output/analysis` directory. diff --git a/docs/analysis/ai-harness/_github/prompts/areas/information-architecture-recommendations.prompt.md b/docs/analysis/ai-harness/_github/prompts/areas/information-architecture-recommendations.prompt.md new file mode 100644 index 00000000..e461c2bd --- /dev/null +++ b/docs/analysis/ai-harness/_github/prompts/areas/information-architecture-recommendations.prompt.md @@ -0,0 +1,23 @@ +--- +mode: agent +description: + Provide recommendations for improving the information architecture of the + documentation. +--- + +# Information Architecture recommendations + +Run the shared **recommendations** analysis for the `information-architecture` +criteria area. + +1. Read the criteria definition in + `.github/criteria/areas/information-architecture.md`. +2. Read and follow the shared procedure in + `.github/prompts/areas/recommendations.prompt.md`, treating + `information-architecture` as the selected area. + +Optional output filename override: `${input:title}` — if non-empty, use it as +the output filename; otherwise use the engine's default +(`<project-slug>-<stem>-recommendations.md`). + +Output the file to the `.github/output/analysis` directory. diff --git a/docs/analysis/ai-harness/_github/prompts/areas/maintenance-planning-answers.prompt.md b/docs/analysis/ai-harness/_github/prompts/areas/maintenance-planning-answers.prompt.md new file mode 100644 index 00000000..d4bf0aba --- /dev/null +++ b/docs/analysis/ai-harness/_github/prompts/areas/maintenance-planning-answers.prompt.md @@ -0,0 +1,22 @@ +--- +mode: agent +description: + Answer the CNCF TechDocs maintenance planning questions for the documentation. +--- + +# Maintenance Planning answers + +Run the shared **answers** analysis for the `maintenance-planning` criteria +area. + +1. Read the criteria definition in + `.github/criteria/areas/maintenance-planning.md`. +2. Read and follow the shared procedure in + `.github/prompts/areas/answers.prompt.md`, treating `maintenance-planning` as + the selected area. + +Optional output filename override: `${input:title}` — if non-empty, use it as +the output filename; otherwise use the engine's default +(`<project-slug>-<stem>-answers.md`). + +Output the file to the `.github/output/analysis` directory. diff --git a/docs/analysis/ai-harness/_github/prompts/areas/maintenance-planning-comment.prompt.md b/docs/analysis/ai-harness/_github/prompts/areas/maintenance-planning-comment.prompt.md new file mode 100644 index 00000000..52c11d2a --- /dev/null +++ b/docs/analysis/ai-harness/_github/prompts/areas/maintenance-planning-comment.prompt.md @@ -0,0 +1,21 @@ +--- +mode: agent +description: Write a comment on the maintenance planning of the documentation. +--- + +# Maintenance Planning comment + +Run the shared **comment** analysis for the `maintenance-planning` criteria +area. + +1. Read the criteria definition in + `.github/criteria/areas/maintenance-planning.md`. +2. Read and follow the shared procedure in + `.github/prompts/areas/comment.prompt.md`, treating `maintenance-planning` as + the selected area. + +Optional output filename override: `${input:title}` — if non-empty, use it as +the output filename; otherwise use the engine's default +(`<project-slug>-<stem>-comment.md`). + +Output the file to the `.github/output/analysis` directory. diff --git a/docs/analysis/ai-harness/_github/prompts/areas/maintenance-planning-full.prompt.md b/docs/analysis/ai-harness/_github/prompts/areas/maintenance-planning-full.prompt.md new file mode 100644 index 00000000..a7b7d0e1 --- /dev/null +++ b/docs/analysis/ai-harness/_github/prompts/areas/maintenance-planning-full.prompt.md @@ -0,0 +1,23 @@ +--- +mode: agent +description: + Run the full `maintenance-planning` analysis (answers, comment, and + recommendations) for the documentation. +--- + +# Maintenance Planning full analysis + +Run the shared **full** analysis for the `maintenance-planning` criteria area. + +1. Read the criteria definition in + `.github/criteria/areas/maintenance-planning.md`. +2. Read and follow the shared procedure in + `.github/prompts/areas/full.prompt.md`, treating `maintenance-planning` as + the selected area. + +Optional output filename override: `${input:title}` — if non-empty, use it as +the base name for all three output files (`<title>-answers.md`, +`<title>-comment.md`, `<title>-recommendations.md`); otherwise use each engine's +default (`<project-slug>-<stem>-<type>.md`). + +Output the file to the `.github/output/analysis` directory. diff --git a/docs/analysis/ai-harness/_github/prompts/areas/maintenance-planning-recommendations.prompt.md b/docs/analysis/ai-harness/_github/prompts/areas/maintenance-planning-recommendations.prompt.md new file mode 100644 index 00000000..c25f697d --- /dev/null +++ b/docs/analysis/ai-harness/_github/prompts/areas/maintenance-planning-recommendations.prompt.md @@ -0,0 +1,23 @@ +--- +mode: agent +description: + Provide recommendations for improving the maintenance planning of the + documentation. +--- + +# Maintenance Planning recommendations + +Run the shared **recommendations** analysis for the `maintenance-planning` +criteria area. + +1. Read the criteria definition in + `.github/criteria/areas/maintenance-planning.md`. +2. Read and follow the shared procedure in + `.github/prompts/areas/recommendations.prompt.md`, treating + `maintenance-planning` as the selected area. + +Optional output filename override: `${input:title}` — if non-empty, use it as +the output filename; otherwise use the engine's default +(`<project-slug>-<stem>-recommendations.md`). + +Output the file to the `.github/output/analysis` directory. diff --git a/docs/analysis/ai-harness/_github/prompts/areas/new-contributor-content-answers.prompt.md b/docs/analysis/ai-harness/_github/prompts/areas/new-contributor-content-answers.prompt.md new file mode 100644 index 00000000..646325c4 --- /dev/null +++ b/docs/analysis/ai-harness/_github/prompts/areas/new-contributor-content-answers.prompt.md @@ -0,0 +1,23 @@ +--- +mode: agent +description: + Answer the CNCF TechDocs new contributor getting started content questions for + the documentation. +--- + +# New Contributor Getting Started Content answers + +Run the shared **answers** analysis for the `new-contributor-content` criteria +area. + +1. Read the criteria definition in + `.github/criteria/areas/new-contributor-content.md`. +2. Read and follow the shared procedure in + `.github/prompts/areas/answers.prompt.md`, treating `new-contributor-content` + as the selected area. + +Optional output filename override: `${input:title}` — if non-empty, use it as +the output filename; otherwise use the engine's default +(`<project-slug>-<stem>-answers.md`). + +Output the file to the `.github/output/analysis` directory. diff --git a/docs/analysis/ai-harness/_github/prompts/areas/new-contributor-content-comment.prompt.md b/docs/analysis/ai-harness/_github/prompts/areas/new-contributor-content-comment.prompt.md new file mode 100644 index 00000000..bec0d8ab --- /dev/null +++ b/docs/analysis/ai-harness/_github/prompts/areas/new-contributor-content-comment.prompt.md @@ -0,0 +1,23 @@ +--- +mode: agent +description: + Write a comment on the new contributor getting started content of the + documentation. +--- + +# New Contributor Getting Started Content comment + +Run the shared **comment** analysis for the `new-contributor-content` criteria +area. + +1. Read the criteria definition in + `.github/criteria/areas/new-contributor-content.md`. +2. Read and follow the shared procedure in + `.github/prompts/areas/comment.prompt.md`, treating `new-contributor-content` + as the selected area. + +Optional output filename override: `${input:title}` — if non-empty, use it as +the output filename; otherwise use the engine's default +(`<project-slug>-<stem>-comment.md`). + +Output the file to the `.github/output/analysis` directory. diff --git a/docs/analysis/ai-harness/_github/prompts/areas/new-contributor-content-full.prompt.md b/docs/analysis/ai-harness/_github/prompts/areas/new-contributor-content-full.prompt.md new file mode 100644 index 00000000..af6253cc --- /dev/null +++ b/docs/analysis/ai-harness/_github/prompts/areas/new-contributor-content-full.prompt.md @@ -0,0 +1,24 @@ +--- +mode: agent +description: + Run the full `new-contributor-content` analysis (answers, comment, and + recommendations) for the documentation. +--- + +# New Contributor Getting Started Content full analysis + +Run the shared **full** analysis for the `new-contributor-content` criteria +area. + +1. Read the criteria definition in + `.github/criteria/areas/new-contributor-content.md`. +2. Read and follow the shared procedure in + `.github/prompts/areas/full.prompt.md`, treating `new-contributor-content` as + the selected area. + +Optional output filename override: `${input:title}` — if non-empty, use it as +the base name for all three output files (`<title>-answers.md`, +`<title>-comment.md`, `<title>-recommendations.md`); otherwise use each engine's +default (`<project-slug>-<stem>-<type>.md`). + +Output the file to the `.github/output/analysis` directory. diff --git a/docs/analysis/ai-harness/_github/prompts/areas/new-contributor-content-recommendations.prompt.md b/docs/analysis/ai-harness/_github/prompts/areas/new-contributor-content-recommendations.prompt.md new file mode 100644 index 00000000..bde072dd --- /dev/null +++ b/docs/analysis/ai-harness/_github/prompts/areas/new-contributor-content-recommendations.prompt.md @@ -0,0 +1,23 @@ +--- +mode: agent +description: + Provide recommendations for improving the new contributor getting started + content of the documentation. +--- + +# New Contributor Getting Started Content recommendations + +Run the shared **recommendations** analysis for the `new-contributor-content` +criteria area. + +1. Read the criteria definition in + `.github/criteria/areas/new-contributor-content.md`. +2. Read and follow the shared procedure in + `.github/prompts/areas/recommendations.prompt.md`, treating + `new-contributor-content` as the selected area. + +Optional output filename override: `${input:title}` — if non-empty, use it as +the output filename; otherwise use the engine's default +(`<project-slug>-<stem>-recommendations.md`). + +Output the file to the `.github/output/analysis` directory. diff --git a/docs/analysis/ai-harness/_github/prompts/areas/new-user-content-answers.prompt.md b/docs/analysis/ai-harness/_github/prompts/areas/new-user-content-answers.prompt.md new file mode 100644 index 00000000..49ae543d --- /dev/null +++ b/docs/analysis/ai-harness/_github/prompts/areas/new-user-content-answers.prompt.md @@ -0,0 +1,20 @@ +--- +mode: agent +description: + Answer the CNCF TechDocs new user content questions for the documentation. +--- + +# New User Content answers + +Run the shared **answers** analysis for the `new-user-content` criteria area. + +1. Read the criteria definition in `.github/criteria/areas/new-user-content.md`. +2. Read and follow the shared procedure in + `.github/prompts/areas/answers.prompt.md`, treating `new-user-content` as the + selected area. + +Optional output filename override: `${input:title}` — if non-empty, use it as +the output filename; otherwise use the engine's default +(`<project-slug>-<stem>-answers.md`). + +Output the file to the `.github/output/analysis` directory. diff --git a/docs/analysis/ai-harness/_github/prompts/areas/new-user-content-comment.prompt.md b/docs/analysis/ai-harness/_github/prompts/areas/new-user-content-comment.prompt.md new file mode 100644 index 00000000..e96a5834 --- /dev/null +++ b/docs/analysis/ai-harness/_github/prompts/areas/new-user-content-comment.prompt.md @@ -0,0 +1,19 @@ +--- +mode: agent +description: Write a comment on the new user content of the documentation. +--- + +# New User Content comment + +Run the shared **comment** analysis for the `new-user-content` criteria area. + +1. Read the criteria definition in `.github/criteria/areas/new-user-content.md`. +2. Read and follow the shared procedure in + `.github/prompts/areas/comment.prompt.md`, treating `new-user-content` as the + selected area. + +Optional output filename override: `${input:title}` — if non-empty, use it as +the output filename; otherwise use the engine's default +(`<project-slug>-<stem>-comment.md`). + +Output the file to the `.github/output/analysis` directory. diff --git a/docs/analysis/ai-harness/_github/prompts/areas/new-user-content-full.prompt.md b/docs/analysis/ai-harness/_github/prompts/areas/new-user-content-full.prompt.md new file mode 100644 index 00000000..9488d0a2 --- /dev/null +++ b/docs/analysis/ai-harness/_github/prompts/areas/new-user-content-full.prompt.md @@ -0,0 +1,22 @@ +--- +mode: agent +description: + Run the full new user content analysis (answers, comment, and recommendations) + for the documentation. +--- + +# New User Content full analysis + +Run the shared **full** analysis for the `new-user-content` criteria area. + +1. Read the criteria definition in `.github/criteria/areas/new-user-content.md`. +2. Read and follow the shared procedure in + `.github/prompts/areas/full.prompt.md`, treating `new-user-content` as the + selected area. + +Optional output filename override: `${input:title}` — if non-empty, use it as +the base name for all three output files (`<title>-answers.md`, +`<title>-comment.md`, `<title>-recommendations.md`); otherwise use each engine's +default (`<project-slug>-<stem>-<type>.md`). + +Output the file to the `.github/output/analysis` directory. diff --git a/docs/analysis/ai-harness/_github/prompts/areas/new-user-content-recommendations.prompt.md b/docs/analysis/ai-harness/_github/prompts/areas/new-user-content-recommendations.prompt.md new file mode 100644 index 00000000..3c8cdfe0 --- /dev/null +++ b/docs/analysis/ai-harness/_github/prompts/areas/new-user-content-recommendations.prompt.md @@ -0,0 +1,22 @@ +--- +mode: agent +description: + Provide recommendations for improving the new user content of the + documentation. +--- + +# New User Content recommendations + +Run the shared **recommendations** analysis for the `new-user-content` criteria +area. + +1. Read the criteria definition in `.github/criteria/areas/new-user-content.md`. +2. Read and follow the shared procedure in + `.github/prompts/areas/recommendations.prompt.md`, treating + `new-user-content` as the selected area. + +Optional output filename override: `${input:title}` — if non-empty, use it as +the output filename; otherwise use the engine's default +(`<project-slug>-<stem>-recommendations.md`). + +Output the file to the `.github/output/analysis` directory. diff --git a/docs/analysis/ai-harness/_github/prompts/areas/project-governance-documentation-answers.prompt.md b/docs/analysis/ai-harness/_github/prompts/areas/project-governance-documentation-answers.prompt.md new file mode 100644 index 00000000..57c9e7c1 --- /dev/null +++ b/docs/analysis/ai-harness/_github/prompts/areas/project-governance-documentation-answers.prompt.md @@ -0,0 +1,22 @@ +--- +mode: agent +description: + Answer the CNCF TechDocs project governance questions for the documentation. +--- + +# Project Governance Documentation answers + +Run the shared **answers** analysis for the `project-governance-documentation` +criteria area. + +1. Read the criteria definition in + `.github/criteria/areas/project-governance-documentation.md`. +2. Read and follow the shared procedure in + `.github/prompts/areas/answers.prompt.md`, treating + `project-governance-documentation` as the selected area. + +Optional output filename override: `${input:title}` — if non-empty, use it as +the output filename; otherwise use the engine's default +(`<project-slug>-<stem>-answers.md`). + +Output the file to the `.github/output/analysis` directory. diff --git a/docs/analysis/ai-harness/_github/prompts/areas/project-governance-documentation-comment.prompt.md b/docs/analysis/ai-harness/_github/prompts/areas/project-governance-documentation-comment.prompt.md new file mode 100644 index 00000000..7bec2446 --- /dev/null +++ b/docs/analysis/ai-harness/_github/prompts/areas/project-governance-documentation-comment.prompt.md @@ -0,0 +1,21 @@ +--- +mode: agent +description: Write a comment on the project governance of the documentation. +--- + +# Project Governance Documentation comment + +Run the shared **comment** analysis for the `project-governance-documentation` +criteria area. + +1. Read the criteria definition in + `.github/criteria/areas/project-governance-documentation.md`. +2. Read and follow the shared procedure in + `.github/prompts/areas/comment.prompt.md`, treating + `project-governance-documentation` as the selected area. + +Optional output filename override: `${input:title}` — if non-empty, use it as +the output filename; otherwise use the engine's default +(`<project-slug>-<stem>-comment.md`). + +Output the file to the `.github/output/analysis` directory. diff --git a/docs/analysis/ai-harness/_github/prompts/areas/project-governance-documentation-full.prompt.md b/docs/analysis/ai-harness/_github/prompts/areas/project-governance-documentation-full.prompt.md new file mode 100644 index 00000000..62bbbc31 --- /dev/null +++ b/docs/analysis/ai-harness/_github/prompts/areas/project-governance-documentation-full.prompt.md @@ -0,0 +1,24 @@ +--- +mode: agent +description: + Run the full `project-governance-documentation` analysis (answers, comment, + and recommendations) for the documentation. +--- + +# Project Governance Documentation full analysis + +Run the shared **full** analysis for the `project-governance-documentation` +criteria area. + +1. Read the criteria definition in + `.github/criteria/areas/project-governance-documentation.md`. +2. Read and follow the shared procedure in + `.github/prompts/areas/full.prompt.md`, treating + `project-governance-documentation` as the selected area. + +Optional output filename override: `${input:title}` — if non-empty, use it as +the base name for all three output files (`<title>-answers.md`, +`<title>-comment.md`, `<title>-recommendations.md`); otherwise use each engine's +default (`<project-slug>-<stem>-<type>.md`). + +Output the file to the `.github/output/analysis` directory. diff --git a/docs/analysis/ai-harness/_github/prompts/areas/project-governance-documentation-recommendations.prompt.md b/docs/analysis/ai-harness/_github/prompts/areas/project-governance-documentation-recommendations.prompt.md new file mode 100644 index 00000000..bb9770f9 --- /dev/null +++ b/docs/analysis/ai-harness/_github/prompts/areas/project-governance-documentation-recommendations.prompt.md @@ -0,0 +1,23 @@ +--- +mode: agent +description: + Provide recommendations for improving the project governance of the + documentation. +--- + +# Project Governance Documentation recommendations + +Run the shared **recommendations** analysis for the +`project-governance-documentation` criteria area. + +1. Read the criteria definition in + `.github/criteria/areas/project-governance-documentation.md`. +2. Read and follow the shared procedure in + `.github/prompts/areas/recommendations.prompt.md`, treating + `project-governance-documentation` as the selected area. + +Optional output filename override: `${input:title}` — if non-empty, use it as +the output filename; otherwise use the engine's default +(`<project-slug>-<stem>-recommendations.md`). + +Output the file to the `.github/output/analysis` directory. diff --git a/docs/analysis/ai-harness/_github/prompts/areas/recommendations.prompt.md b/docs/analysis/ai-harness/_github/prompts/areas/recommendations.prompt.md new file mode 100644 index 00000000..060121ae --- /dev/null +++ b/docs/analysis/ai-harness/_github/prompts/areas/recommendations.prompt.md @@ -0,0 +1,51 @@ +--- +mode: agent +description: + Shared engine for producing analysis recommendations. Normally run via a + per-area wrapper such as /information-architecture-recommendations. +--- + +# Shared engine: recommendations + +Reusable procedure for producing recommendations for one area, based on the +previously generated answers and comment. Normally invoked through a per-area +wrapper prompt (for example, `/information-architecture-recommendations`), which +selects the criteria area for you. + +## Inputs + +- Area — the criteria area to analyze. When run from a wrapper, the wrapper + names it. If you reached this engine without an area, ask the user to choose + one of the areas defined in `.github/criteria/areas/` (the area name is the + criteria filename without `.md`). +- Title (optional) — overrides the default output filename. + +## Procedure + +1. Read the criteria definition at `.github/criteria/areas/<area>.md` for its + Display name and File stem. +2. Determine the project name and documentation label from the "Current + Repository" section of the repository Copilot instructions. Derive the + project slug by lowercasing the name and replacing spaces with hyphens. +3. Read the answers and comment produced for this area from the + `.github/output/analysis` directory: `<project-slug>-<stem>-answers.md` and + `<project-slug>-<stem>-comment.md`. If either file is missing, ask the user + to run the corresponding answers or comment prompt first (or to point you at + the files). If the comment file contains a recommendation, move it to the + recommendations file. +4. Base the recommendations on those two documents. + +## Output + +- Output the recommendations as a bulleted list, introduced by this exact + sentence: "The following recommendations address the <display name> of + the <project documentation label>." For example: "The following + recommendations address the information architecture of the KubeVirt user + guide." +- Write the result to a Markdown file in the `.github/output/analysis` + directory: + - If a title was provided, name the file `<title>.md`. + - Otherwise name it `<project-slug>-<stem>-recommendations.md`. + - If a file exists by the same name, overwrite it. + +Output the file to the `.github/output/analysis` directory. diff --git a/docs/analysis/ai-harness/_github/prompts/areas/seo-analytics-answers.prompt.md b/docs/analysis/ai-harness/_github/prompts/areas/seo-analytics-answers.prompt.md new file mode 100644 index 00000000..e19dc436 --- /dev/null +++ b/docs/analysis/ai-harness/_github/prompts/areas/seo-analytics-answers.prompt.md @@ -0,0 +1,21 @@ +--- +mode: agent +description: + Answer the CNCF TechDocs seo, analytics & site search questions for the + documentation. +--- + +# SEO, Analytics, and Site Search answers + +Run the shared **answers** analysis for the `seo-analytics` criteria area. + +1. Read the criteria definition in `.github/criteria/areas/seo-analytics.md`. +2. Read and follow the shared procedure in + `.github/prompts/areas/answers.prompt.md`, treating `seo-analytics` as the + selected area. + +Optional output filename override: `${input:title}` — if non-empty, use it as +the output filename; otherwise use the engine's default +(`<project-slug>-<stem>-answers.md`). + +Output the file to the `.github/output/analysis` directory. diff --git a/docs/analysis/ai-harness/_github/prompts/areas/seo-analytics-comment.prompt.md b/docs/analysis/ai-harness/_github/prompts/areas/seo-analytics-comment.prompt.md new file mode 100644 index 00000000..99a193f2 --- /dev/null +++ b/docs/analysis/ai-harness/_github/prompts/areas/seo-analytics-comment.prompt.md @@ -0,0 +1,20 @@ +--- +mode: agent +description: + Write a comment on the seo, analytics & site search of the documentation. +--- + +# SEO, Analytics, and Site Search comment + +Run the shared **comment** analysis for the `seo-analytics` criteria area. + +1. Read the criteria definition in `.github/criteria/areas/seo-analytics.md`. +2. Read and follow the shared procedure in + `.github/prompts/areas/comment.prompt.md`, treating `seo-analytics` as the + selected area. + +Optional output filename override: `${input:title}` — if non-empty, use it as +the output filename; otherwise use the engine's default +(`<project-slug>-<stem>-comment.md`). + +Output the file to the `.github/output/analysis` directory. diff --git a/docs/analysis/ai-harness/_github/prompts/areas/seo-analytics-full.prompt.md b/docs/analysis/ai-harness/_github/prompts/areas/seo-analytics-full.prompt.md new file mode 100644 index 00000000..c006a560 --- /dev/null +++ b/docs/analysis/ai-harness/_github/prompts/areas/seo-analytics-full.prompt.md @@ -0,0 +1,22 @@ +--- +mode: agent +description: + Run the full `seo-analytics` analysis (answers, comment, and recommendations) + for the documentation. +--- + +# SEO, Analytics, and Site Search full analysis + +Run the shared **full** analysis for the `seo-analytics` criteria area. + +1. Read the criteria definition in `.github/criteria/areas/seo-analytics.md`. +2. Read and follow the shared procedure in + `.github/prompts/areas/full.prompt.md`, treating `seo-analytics` as the + selected area. + +Optional output filename override: `${input:title}` — if non-empty, use it as +the base name for all three output files (`<title>-answers.md`, +`<title>-comment.md`, `<title>-recommendations.md`); otherwise use each engine's +default (`<project-slug>-<stem>-<type>.md`). + +Output the file to the `.github/output/analysis` directory. diff --git a/docs/analysis/ai-harness/_github/prompts/areas/seo-analytics-recommendations.prompt.md b/docs/analysis/ai-harness/_github/prompts/areas/seo-analytics-recommendations.prompt.md new file mode 100644 index 00000000..24fad9ce --- /dev/null +++ b/docs/analysis/ai-harness/_github/prompts/areas/seo-analytics-recommendations.prompt.md @@ -0,0 +1,22 @@ +--- +mode: agent +description: + Provide recommendations for improving the seo, analytics & site search of the + documentation. +--- + +# SEO, Analytics, and Site Search recommendations + +Run the shared **recommendations** analysis for the `seo-analytics` criteria +area. + +1. Read the criteria definition in `.github/criteria/areas/seo-analytics.md`. +2. Read and follow the shared procedure in + `.github/prompts/areas/recommendations.prompt.md`, treating `seo-analytics` + as the selected area. + +Optional output filename override: `${input:title}` — if non-empty, use it as +the output filename; otherwise use the engine's default +(`<project-slug>-<stem>-recommendations.md`). + +Output the file to the `.github/output/analysis` directory. diff --git a/docs/analysis/ai-harness/_github/prompts/areas/single-source-requirement-answers.prompt.md b/docs/analysis/ai-harness/_github/prompts/areas/single-source-requirement-answers.prompt.md new file mode 100644 index 00000000..c827def7 --- /dev/null +++ b/docs/analysis/ai-harness/_github/prompts/areas/single-source-requirement-answers.prompt.md @@ -0,0 +1,23 @@ +--- +mode: agent +description: + Answer the CNCF TechDocs single-source requirement questions for the + documentation. +--- + +# Single-Source Requirement answers + +Run the shared **answers** analysis for the `single-source-requirement` criteria +area. + +1. Read the criteria definition in + `.github/criteria/areas/single-source-requirement.md`. +2. Read and follow the shared procedure in + `.github/prompts/areas/answers.prompt.md`, treating + `single-source-requirement` as the selected area. + +Optional output filename override: `${input:title}` — if non-empty, use it as +the output filename; otherwise use the engine's default +(`<project-slug>-<stem>-answers.md`). + +Output the file to the `.github/output/analysis` directory. diff --git a/docs/analysis/ai-harness/_github/prompts/areas/single-source-requirement-comment.prompt.md b/docs/analysis/ai-harness/_github/prompts/areas/single-source-requirement-comment.prompt.md new file mode 100644 index 00000000..e12c4150 --- /dev/null +++ b/docs/analysis/ai-harness/_github/prompts/areas/single-source-requirement-comment.prompt.md @@ -0,0 +1,22 @@ +--- +mode: agent +description: + Write a comment on the single-source requirement of the documentation. +--- + +# Single-Source Requirement comment + +Run the shared **comment** analysis for the `single-source-requirement` criteria +area. + +1. Read the criteria definition in + `.github/criteria/areas/single-source-requirement.md`. +2. Read and follow the shared procedure in + `.github/prompts/areas/comment.prompt.md`, treating + `single-source-requirement` as the selected area. + +Optional output filename override: `${input:title}` — if non-empty, use it as +the output filename; otherwise use the engine's default +(`<project-slug>-<stem>-comment.md`). + +Output the file to the `.github/output/analysis` directory. diff --git a/docs/analysis/ai-harness/_github/prompts/areas/single-source-requirement-full.prompt.md b/docs/analysis/ai-harness/_github/prompts/areas/single-source-requirement-full.prompt.md new file mode 100644 index 00000000..286f0e9a --- /dev/null +++ b/docs/analysis/ai-harness/_github/prompts/areas/single-source-requirement-full.prompt.md @@ -0,0 +1,24 @@ +--- +mode: agent +description: + Run the full `single-source-requirement` analysis (answers, comment, and + recommendations) for the documentation. +--- + +# Single-Source Requirement full analysis + +Run the shared **full** analysis for the `single-source-requirement` criteria +area. + +1. Read the criteria definition in + `.github/criteria/areas/single-source-requirement.md`. +2. Read and follow the shared procedure in + `.github/prompts/areas/full.prompt.md`, treating `single-source-requirement` + as the selected area. + +Optional output filename override: `${input:title}` — if non-empty, use it as +the base name for all three output files (`<title>-answers.md`, +`<title>-comment.md`, `<title>-recommendations.md`); otherwise use each engine's +default (`<project-slug>-<stem>-<type>.md`). + +Output the file to the `.github/output/analysis` directory. diff --git a/docs/analysis/ai-harness/_github/prompts/areas/single-source-requirement-recommendations.prompt.md b/docs/analysis/ai-harness/_github/prompts/areas/single-source-requirement-recommendations.prompt.md new file mode 100644 index 00000000..734dfd4d --- /dev/null +++ b/docs/analysis/ai-harness/_github/prompts/areas/single-source-requirement-recommendations.prompt.md @@ -0,0 +1,23 @@ +--- +mode: agent +description: + Provide recommendations for improving the single-source requirement of the + documentation. +--- + +# Single-Source Requirement recommendations + +Run the shared **recommendations** analysis for the `single-source-requirement` +criteria area. + +1. Read the criteria definition in + `.github/criteria/areas/single-source-requirement.md`. +2. Read and follow the shared procedure in + `.github/prompts/areas/recommendations.prompt.md`, treating + `single-source-requirement` as the selected area. + +Optional output filename override: `${input:title}` — if non-empty, use it as +the output filename; otherwise use the engine's default +(`<project-slug>-<stem>-recommendations.md`). + +Output the file to the `.github/output/analysis` directory. diff --git a/docs/analysis/ai-harness/_github/prompts/areas/usability-accessibility-devices-answers.prompt.md b/docs/analysis/ai-harness/_github/prompts/areas/usability-accessibility-devices-answers.prompt.md new file mode 100644 index 00000000..276543c0 --- /dev/null +++ b/docs/analysis/ai-harness/_github/prompts/areas/usability-accessibility-devices-answers.prompt.md @@ -0,0 +1,23 @@ +--- +mode: agent +description: + Answer the CNCF TechDocs usability accessibility and devices questions for the + documentation. +--- + +# Usability, Accessibility and Devices answers + +Run the shared **answers** analysis for the `usability-accessibility-devices` +criteria area. + +1. Read the criteria definition in + `.github/criteria/areas/usability-accessibility-devices.md`. +2. Read and follow the shared procedure in + `.github/prompts/areas/answers.prompt.md`, treating + `usability-accessibility-devices` as the selected area. + +Optional output filename override: `${input:title}` — if non-empty, use it as +the output filename; otherwise use the engine's default +(`<project-slug>-<stem>-answers.md`). + +Output the file to the `.github/output/analysis` directory. diff --git a/docs/analysis/ai-harness/_github/prompts/areas/usability-accessibility-devices-comment.prompt.md b/docs/analysis/ai-harness/_github/prompts/areas/usability-accessibility-devices-comment.prompt.md new file mode 100644 index 00000000..4c896479 --- /dev/null +++ b/docs/analysis/ai-harness/_github/prompts/areas/usability-accessibility-devices-comment.prompt.md @@ -0,0 +1,23 @@ +--- +mode: agent +description: + Write a comment on the usability accessibility and devices of the + documentation. +--- + +# Usability, Accessibility and Devices comment + +Run the shared **comment** analysis for the `usability-accessibility-devices` +criteria area. + +1. Read the criteria definition in + `.github/criteria/areas/usability-accessibility-devices.md`. +2. Read and follow the shared procedure in + `.github/prompts/areas/comment.prompt.md`, treating + `usability-accessibility-devices` as the selected area. + +Optional output filename override: `${input:title}` — if non-empty, use it as +the output filename; otherwise use the engine's default +(`<project-slug>-<stem>-comment.md`). + +Output the file to the `.github/output/analysis` directory. diff --git a/docs/analysis/ai-harness/_github/prompts/areas/usability-accessibility-devices-full.prompt.md b/docs/analysis/ai-harness/_github/prompts/areas/usability-accessibility-devices-full.prompt.md new file mode 100644 index 00000000..56897390 --- /dev/null +++ b/docs/analysis/ai-harness/_github/prompts/areas/usability-accessibility-devices-full.prompt.md @@ -0,0 +1,24 @@ +--- +mode: agent +description: + Run the full `usability-accessibility-devices` analysis (answers, comment, and + recommendations) for the documentation. +--- + +# Usability, Accessibility and Devices full analysis + +Run the shared **full** analysis for the `usability-accessibility-devices` +criteria area. + +1. Read the criteria definition in + `.github/criteria/areas/usability-accessibility-devices.md`. +2. Read and follow the shared procedure in + `.github/prompts/areas/full.prompt.md`, treating + `usability-accessibility-devices` as the selected area. + +Optional output filename override: `${input:title}` — if non-empty, use it as +the base name for all three output files (`<title>-answers.md`, +`<title>-comment.md`, `<title>-recommendations.md`); otherwise use each engine's +default (`<project-slug>-<stem>-<type>.md`). + +Output the file to the `.github/output/analysis` directory. diff --git a/docs/analysis/ai-harness/_github/prompts/areas/usability-accessibility-devices-recommendations.prompt.md b/docs/analysis/ai-harness/_github/prompts/areas/usability-accessibility-devices-recommendations.prompt.md new file mode 100644 index 00000000..f5252dc9 --- /dev/null +++ b/docs/analysis/ai-harness/_github/prompts/areas/usability-accessibility-devices-recommendations.prompt.md @@ -0,0 +1,23 @@ +--- +mode: agent +description: + Provide recommendations for improving the usability accessibility and devices + of the documentation. +--- + +# Usability, Accessibility and Devices recommendations + +Run the shared **recommendations** analysis for the +`usability-accessibility-devices` criteria area. + +1. Read the criteria definition in + `.github/criteria/areas/usability-accessibility-devices.md`. +2. Read and follow the shared procedure in + `.github/prompts/areas/recommendations.prompt.md`, treating + `usability-accessibility-devices` as the selected area. + +Optional output filename override: `${input:title}` — if non-empty, use it as +the output filename; otherwise use the engine's default +(`<project-slug>-<stem>-recommendations.md`). + +Output the file to the `.github/output/analysis` directory. diff --git a/docs/analysis/ai-harness/_github/prompts/areas/website-requirements-answers.prompt.md b/docs/analysis/ai-harness/_github/prompts/areas/website-requirements-answers.prompt.md new file mode 100644 index 00000000..7fa69b61 --- /dev/null +++ b/docs/analysis/ai-harness/_github/prompts/areas/website-requirements-answers.prompt.md @@ -0,0 +1,22 @@ +--- +mode: agent +description: + Answer the CNCF TechDocs website requirements questions for the documentation. +--- + +# Website Requirements answers + +Run the shared **answers** analysis for the `website-requirements` criteria +area. + +1. Read the criteria definition in + `.github/criteria/areas/website-requirements.md`. +2. Read and follow the shared procedure in + `.github/prompts/areas/answers.prompt.md`, treating `website-requirements` as + the selected area. + +Optional output filename override: `${input:title}` — if non-empty, use it as +the output filename; otherwise use the engine's default +(`<project-slug>-<stem>-answers.md`). + +Output the file to the `.github/output/analysis` directory. diff --git a/docs/analysis/ai-harness/_github/prompts/areas/website-requirements-comment.prompt.md b/docs/analysis/ai-harness/_github/prompts/areas/website-requirements-comment.prompt.md new file mode 100644 index 00000000..3fb59cb7 --- /dev/null +++ b/docs/analysis/ai-harness/_github/prompts/areas/website-requirements-comment.prompt.md @@ -0,0 +1,21 @@ +--- +mode: agent +description: Write a comment on the website requirements of the documentation. +--- + +# Website Requirements comment + +Run the shared **comment** analysis for the `website-requirements` criteria +area. + +1. Read the criteria definition in + `.github/criteria/areas/website-requirements.md`. +2. Read and follow the shared procedure in + `.github/prompts/areas/comment.prompt.md`, treating `website-requirements` as + the selected area. + +Optional output filename override: `${input:title}` — if non-empty, use it as +the output filename; otherwise use the engine's default +(`<project-slug>-<stem>-comment.md`). + +Output the file to the `.github/output/analysis` directory. diff --git a/docs/analysis/ai-harness/_github/prompts/areas/website-requirements-full.prompt.md b/docs/analysis/ai-harness/_github/prompts/areas/website-requirements-full.prompt.md new file mode 100644 index 00000000..deb29bb0 --- /dev/null +++ b/docs/analysis/ai-harness/_github/prompts/areas/website-requirements-full.prompt.md @@ -0,0 +1,23 @@ +--- +mode: agent +description: + Run the full `website-requirements` analysis (answers, comment, and + recommendations) for the documentation. +--- + +# Website Requirements full analysis + +Run the shared **full** analysis for the `website-requirements` criteria area. + +1. Read the criteria definition in + `.github/criteria/areas/website-requirements.md`. +2. Read and follow the shared procedure in + `.github/prompts/areas/full.prompt.md`, treating `website-requirements` as + the selected area. + +Optional output filename override: `${input:title}` — if non-empty, use it as +the base name for all three output files (`<title>-answers.md`, +`<title>-comment.md`, `<title>-recommendations.md`); otherwise use each engine's +default (`<project-slug>-<stem>-<type>.md`). + +Output the file to the `.github/output/analysis` directory. diff --git a/docs/analysis/ai-harness/_github/prompts/areas/website-requirements-recommendations.prompt.md b/docs/analysis/ai-harness/_github/prompts/areas/website-requirements-recommendations.prompt.md new file mode 100644 index 00000000..59d49c2d --- /dev/null +++ b/docs/analysis/ai-harness/_github/prompts/areas/website-requirements-recommendations.prompt.md @@ -0,0 +1,23 @@ +--- +mode: agent +description: + Provide recommendations for improving the website requirements of the + documentation. +--- + +# Website Requirements recommendations + +Run the shared **recommendations** analysis for the `website-requirements` +criteria area. + +1. Read the criteria definition in + `.github/criteria/areas/website-requirements.md`. +2. Read and follow the shared procedure in + `.github/prompts/areas/recommendations.prompt.md`, treating + `website-requirements` as the selected area. + +Optional output filename override: `${input:title}` — if non-empty, use it as +the output filename; otherwise use the engine's default +(`<project-slug>-<stem>-recommendations.md`). + +Output the file to the `.github/output/analysis` directory. diff --git a/docs/analysis/ai-harness/_github/prompts/issues/create-issues.prompt.md b/docs/analysis/ai-harness/_github/prompts/issues/create-issues.prompt.md new file mode 100644 index 00000000..56b360eb --- /dev/null +++ b/docs/analysis/ai-harness/_github/prompts/issues/create-issues.prompt.md @@ -0,0 +1,195 @@ +--- +mode: agent +description: + Create numbered GitHub issue files in .github/output/issues/ from the ranked + analysis recommendations, one file per issue, each scoped to 2-4 hours. Run + /create-umbrella-issues afterwards to group them. +--- + +# Create documentation issues from the analysis recommendations + +## Role and goal + +You are a senior technical writer on the CNCF TechDocs team. Turn the ranked +recommendations from the project's documentation analysis into GitHub issue +files that a technical writer or engineer with project context can pick up and +complete. A process will later upload every file in the output directory to +GitHub, so each file must be a complete, uploadable issue. + +## Project context + +Read the "Current Repository" section of `.github/copilot-instructions.md` +before you start. It supplies every project-specific value used below: + +- Project name, from which you derive `<project-slug>` (lowercase, spaces + replaced with hyphens). +- Published documentation URL (`<docs URL>`). +- Issues repository URL (`<issues repo URL>`), used for `repo:` in the front + matter. +- Issue template file name under `.github/templates/` (`<issue template>`). +- CNCF project page URL (`<CNCF project URL>`). +- Assistance-request issue (`cncf/techdocs#<NNN>`). + +If any of these fields is still a placeholder, stop and ask the writer to fill +it in. + +This prompt creates standalone issues only. Do not create umbrella issues; a +separate prompt, `/create-umbrella-issues`, groups the issues you create here +after the writer has reviewed them. + +## Sources + +Use only these sources. Do not add issues from your general knowledge of the +project's repositories or documentation; the assessment defines the scope. + +- `.github/input/ai-ranked-recommendations.csv` — the ranked recommendations. + Columns: `Section`, `Area`, `Recommendation`, `Suggested Priority` + (`1 - Urgent`, `2 - High`, `3 - Backlog`). +- `.github/output/analysis/<project-slug>-*-recommendations.md` — the per-area + recommendations files, for context on why each recommendation was made. +- `.github/output/analysis/<project-slug>-*-comment.md` — the per-area comments, + only when you need the problem statement behind a recommendation. +- `.github/templates/<issue template>` — the project's documentation issue + template (`doc-issue.md` by default). Every issue follows its headings. +- https://github.com/kedacore/keda-docs/issues/1363 — an example of a CNCF + TechDocs issue from a previous analysis. Match its scope and tone: a specific + problem, a concrete proposed change, and links to the affected pages. + +Ignore any sub-folders of `.github/output/issues/`; they are archives of +previous runs. + +## Procedure + +1. Read the CSV and the recommendations files. Group the recommendations by + section and area. +2. Decide which recommendations are issues. A recommendation is an issue when it + describes a concrete problem or gap in the documentation that a contributor + can fix. Skip recommendations that are optional improvements with minimal + impact, that depend on decisions the maintainers have not made, or that + cannot be acted on without information the assessment does not provide. + Record every skipped recommendation and the reason in the coverage report + (see Outputs). +3. Scope every issue to about two to four hours of work for a technical writer + or engineer with project context. Split a recommendation that is larger than + that into several issues; merge recommendations that are smaller into one + issue. Merged recommendations should address the same pages or the same kind + of change so the issue stays coherent. +4. Give each issue a title that is specific and imperative, for example "Add a + Getting Started page that walks through the first deployment". A reader + should understand the task from the title alone, because umbrella issues will + list titles only. +5. Number the issues. Issue numbers are five digits and start at 9001. Increment + by one for each issue, in the order you write them. Numbers simulate GitHub + issue numbers so the files can cross-reference each other before upload. +6. Assign a priority label to every issue. Start from the CSV ranking without + its number prefix (`1 - Urgent` → `urgent`, `2 - High` → `high`, + `3 - Backlog` → `backlog`). When an issue merges recommendations with + different rankings, use the highest. You may adjust a priority when merging + or splitting changes the impact; note any adjustment in the coverage report. +7. Keep the total number of issues well below the number of recommendations. A + good target is 40 to 60 issues. If you exceed that, merge further. + +## Issue file format + +Every file in `.github/output/issues/` starts with this front matter, followed +by the issue body: + +```yaml +--- +title: <GitHub issue title> #<issue number> +labels: [<urgent | high | backlog>, help wanted] +repo: <issues repo URL> +--- +``` + +- `title` becomes the GitHub issue title. Append the simulated issue number + after a space and `#`, for example + `title: Add a user-facing troubleshooting page #9007`. +- `labels` becomes the GitHub issue labels. Include exactly one of `urgent`, + `high`, or `backlog`, and always include `help wanted`. Do not add any other + labels. +- `repo` is always the issues repository URL from "Current Repository". +- Everything after the front matter is the issue description. Write it in + GitHub-flavored Markdown. + +Follow the issue template and keep its top-level headings in this order. Omit +any HTML comment from the template. The default template has these four: + +- `## Description` — state the problem or gap, where it is, and why it matters + to users. Then describe the proposed change concretely enough that a + contributor can start without further research: which pages or files to add, + edit, or move, and what the result should contain. Use a bulleted list or a + proposed outline when the change has several parts. +- `## What you expected` — describe what done looks like from the reader's point + of view, in one to three sentences. This is the acceptance criterion. +- `## URL` — link to the affected pages on `<docs URL>` and name the source + Markdown files. For issues that affect the project website or a repository + file rather than a documentation page, link to the affected page or file in + its repository. +- `## Additional context` — end with this sentence, adapted as needed: "This + issue tracks recommended changes resulting from an analysis of the + <project name> documentation commissioned by [CNCF](<CNCF project URL>). The + assistance request is tracked in + [cncf/techdocs#<NNN>](https://github.com/cncf/techdocs/issues/<NNN>)." Before + it, add anything else a contributor needs: related issues by simulated number + (for example "Depends on #9003"), decisions the maintainers must make first, + or questions for the maintainers. + +Add subheadings within a section when they help organize the content, but do not +add or remove top-level headings. + +In all issue bodies: + +- Write in complete sentences, active voice, and the imperative mood. +- Explain any term a senior engineer in the project's ecosystem might not know; + do not explain the ecosystem's basics. +- Follow the project's documentation conventions as described in "Current + Repository" (for Kubernetes-ecosystem projects, UpperCamelCase for API objects + and code formatting for commands, fields, and file names). +- If you are unsure whether something is technically wrong, phrase it as a + question for the maintainers rather than asserting it. +- Do not mention the CSV, the priority ranking, or this prompt in the issue + body. The body is for the project's contributors. + +## Outputs + +1. Issue files in `.github/output/issues/`. Name each file + `<issue number>-<slug>.md`, for example `9007-add-troubleshooting-page.md`. + Build the slug from the title with these rules: + - Use lowercase words separated by hyphens, no punctuation, and no more than + about five words. + - Keep the action verb and the main subject noun; drop articles, + conjunctions, and secondary clauses. Name the topic (`navigation`, + `footer`, `virtctl`), not the analysis area. + - When the title lists several changes, use a collective noun such as + `improvements` or `updates` instead of listing them. + - Strip punctuation from version numbers (`v1.9` becomes `v19`). + + Examples: "Add a Getting Started page that walks through the first + deployment" becomes `9001-add-getting-started-page.md`; "Move Release Notes + out of the main navigation" becomes `9004-move-release-notes.md`; "Document + the v1.9 VirtualMachineBackup API" becomes `9012-document-v19-backup-api.md`. + + Before writing, delete any `<number>-*.md` and `U-*.md` files already in + `.github/output/issues/` from a previous run so the numbering restarts + cleanly at 9001. Leave sub-folders and `readme.md` alone. + +2. A coverage report at `.github/output/issues-coverage.md` (outside the issues + directory so it is not uploaded as an issue). Overwrite any existing report. + Include, in this order: + - An `## Issues` table with the columns `Issue` (the simulated number, for + example `#9001`), `Title`, `Label`, and `File`. The + `/create-umbrella-issues` prompt reads this table to continue the numbering + and to look up titles, so list every issue file here. + - A `## Recommendation mapping` table mapping every CSV recommendation + (numbered by its row order) to the issue number that covers it, or "Not an + issue" with a one-sentence reason. A recommendation may map to several + issues; an issue may cover several recommendations. + - A `## Priority adjustments` table with the recommendation, its CSV ranking, + the label you assigned, and the reason. + - A `## Counts` list with the number of recommendations, recommendations + turned into issues, recommendations skipped, and issue files written. + +When finished, summarize the issues you created (number, title, label) and ask +the writer to review the scope of each one before running +`/create-umbrella-issues`. diff --git a/docs/analysis/ai-harness/_github/prompts/issues/create-umbrella-issues.prompt.md b/docs/analysis/ai-harness/_github/prompts/issues/create-umbrella-issues.prompt.md new file mode 100644 index 00000000..c703978a --- /dev/null +++ b/docs/analysis/ai-harness/_github/prompts/issues/create-umbrella-issues.prompt.md @@ -0,0 +1,136 @@ +--- +mode: agent +description: + Group the numbered issue files in .github/output/issues/ into umbrella issues + of 3-10 sub-issues each. Run after /create-issues and after the writer has + reviewed the issues. +--- + +# Create umbrella issues from the documentation issues + +## Role and goal + +You are a senior technical writer on the CNCF TechDocs team. The +`/create-issues` prompt has already turned the project's documentation analysis +into numbered, standalone issue files. Group those issues into umbrella issues +so maintainers can track related work as a unit and so CNCF can report progress +on the analysis. Each umbrella issue is itself an uploadable issue file that +links to its sub-issues; it does not repeat their content. + +## Project context + +Read the "Current Repository" section of `.github/copilot-instructions.md` for +the project name, the issues repository URL (`<issues repo URL>`), the CNCF +project page URL (`<CNCF project URL>`), and the assistance-request issue +(`cncf/techdocs#<NNN>`). If any of these is still a placeholder, stop and ask +the writer to fill it in. + +## Sources + +- `.github/output/issues/<number>-*.md` — the standalone issue files. Read every + one. Ignore `readme.md` and any sub-folders of `.github/output/issues/`; the + sub-folders are archives of previous runs. +- `.github/output/issues-coverage.md` — the coverage report. Its `## Issues` + table lists every issue number, title, label, and file; use it to look up + titles and to find the next free issue number. +- `.github/templates/umbrella-issue.md` — the umbrella issue template. +- https://github.com/kedacore/keda-docs/issues/1361 — an example umbrella issue + from a previous CNCF TechDocs analysis. Match its purpose: a short statement + of what the group achieves, followed by a checklist of sub-issues. Do not copy + its "Use-Case:" and "Specification:" heading prefixes. + +Do not read the recommendations CSV or the analysis files. The issue files are +the only input; if an issue is missing or wrongly scoped, tell the writer to fix +it with `/create-issues` instead of working around it here. + +## Procedure + +1. Read the `## Issues` table in the coverage report and every standalone issue + file. Note each issue's number, title, label, and the pages or repository it + touches. +2. Group the issues into themes. Group by the outcome a reader or maintainer + gets, not by the analysis section or area an issue came from; a theme may + span sections. Typical themes are a new-user path, navigation and information + architecture, a feature-coverage gap, site configuration, accessibility, + contributor onboarding, community channels, and site ownership. +3. Size each umbrella at three to ten sub-issues. Split a theme that has more + than ten; merge or drop a theme that has fewer than three. An issue belongs + to at most one umbrella. An issue that fits no theme stays standalone; that + is acceptable, and you should list such issues in the coverage report rather + than force them into an umbrella. +4. Order the sub-issues within each umbrella in the sequence a contributor + should work them: prerequisites first, then by priority (`urgent`, `high`, + `backlog`). +5. Assign each umbrella the highest priority label among its sub-issues. +6. Number the umbrellas. Continue the sequence from the coverage report: the + first umbrella takes the number after the highest number in the `## Issues` + table, and each further umbrella increments by one. +7. Give each umbrella a title that names the outcome, for example "Create a + new-user getting started path" or "Fix navigation gaps in the user guide". Do + not use "Umbrella" in the title; the body identifies it as an umbrella issue. + +## Umbrella file format + +Every umbrella file starts with this front matter, followed by the body: + +```yaml +--- +title: <umbrella issue title> #<issue number> +labels: [<urgent | high | backlog>, help wanted] +repo: <issues repo URL> +--- +``` + +Follow `umbrella-issue.md`, replacing its `{issues repo URL}` placeholder with +the issues repository URL, and keep its two headings: + +- `## Umbrella Issue` — in one or two paragraphs, state the problem the theme + addresses, how the sub-issues together resolve it, and the order in which to + work them if it matters. End with this sentence, adapted as needed: "This + umbrella issue tracks recommended changes resulting from an analysis of the + <project name> documentation commissioned by [CNCF](<CNCF project URL>). The + assistance request is tracked in + [cncf/techdocs#<NNN>](https://github.com/cncf/techdocs/issues/<NNN>)." +- `## Sub-issues` — a bulleted list with one line per sub-issue, in work order, + formatted as a link whose text is the sub-issue title followed by its number + and whose target is the sub-issue file name relative to the umbrella file: + + <!-- markdownlint-disable MD013 --> + + ```markdown + - [Add a Getting Started page that walks through the first deployment #9001](9001-add-getting-started-page.md) + - [Rewrite Basic Use as "Your first deployment" #9002](9002-rewrite-basic-use.md) + ``` + + <!-- markdownlint-enable MD013 --> + + The upload process replaces the `#<number>` references with real issue numbers + and links, so every title and number must match the sub-issue's front matter + exactly. Do not describe the sub-issues here; the link is enough. + +Write in complete sentences, active voice, and the imperative mood. Do not +mention the CSV, the priority ranking, or this prompt in the body. + +## Outputs + +1. Umbrella files in `.github/output/issues/`. Name each file + `U-<issue number>-<slug>.md`, for example + `U-9061-create-getting-started-path.md`. Build the slug from the title with + the same rules as standalone issues: lowercase hyphenated words, no more than + about five, keeping the action verb and main subject noun and naming the + topic rather than the analysis area. Before writing, delete any `U-*.md` + files already in `.github/output/issues/` from a previous run. Do not modify + the standalone issue files. +2. Update `.github/output/issues-coverage.md`: + - Append the umbrella issues to the `## Issues` table with the same columns + (`Issue`, `Title`, `Label`, `File`). + - Add a `## Umbrella issues` section after the `## Issues` table with one + subsection per umbrella listing its number, title, label, and sub-issue + numbers, followed by a list of standalone issues that belong to no + umbrella. + - Update the `## Counts` list with the number of umbrella issues and the + number of standalone issues not in an umbrella. + +When finished, summarize the umbrella issues you created (number, title, label, +sub-issue count) and list any issues left outside an umbrella. Ask the writer to +review the grouping. diff --git a/docs/analysis/ai-harness/_github/prompts/issues/rank-recommendations.prompt.md b/docs/analysis/ai-harness/_github/prompts/issues/rank-recommendations.prompt.md new file mode 100644 index 00000000..114710a8 --- /dev/null +++ b/docs/analysis/ai-harness/_github/prompts/issues/rank-recommendations.prompt.md @@ -0,0 +1,83 @@ +--- +mode: agent +description: + Collect every recommendation from the per-area recommendations files in + .github/output/analysis/ and rank them into + .github/input/ai-ranked-recommendations.csv for /create-issues. +--- + +# Rank the analysis recommendations + +## Role and goal + +You are a senior technical writer on the CNCF TechDocs team. The area prompts +have written one recommendations file per analysis area. Gather every +recommendation into a single CSV and assign each a priority so that +`/create-issues` can turn the ranked list into GitHub issues. Rank by the impact +on documentation users relative to the effort to implement; do not rewrite, +merge, or drop recommendations here. + +## Sources + +- The "Current Repository" section of `.github/copilot-instructions.md` — gives + the project name, from which you derive the project slug (lowercase, spaces + replaced with hyphens). +- `.github/output/analysis/<project-slug>-*-recommendations.md` — the per-area + recommendations files. Every bullet in these files is one recommendation. +- `.github/output/analysis/<project-slug>-*-comment.md` — the per-area comments, + for the `Rating:` line and the weaknesses that motivate each recommendation. +- `.github/criteria/sections/*.md` — map each area's comment-file stem to its + section display name and criterion label (the area display name). + +If any area listed in the section files has no recommendations file, stop and +tell the user which `/<area>-recommendations` or `/<area>-full` prompt to run +first. + +## Procedure + +1. Read every recommendations file. Record each bullet as one recommendation, + with its section display name and area display name taken from the section + criteria files. +2. Rank each recommendation on two factors: + - Impact: how much it improves users' ability to succeed with the project. + Gaps in the new-user path, missing task documentation for core features, + broken or misleading content, and accessibility blockers rank highest. The + area's rating is a signal: areas rated 1 or 2 usually hold the + highest-impact recommendations. + - Effort: a recommendation that a contributor can complete in a few hours + ranks above one of equal impact that needs maintainer decisions, new + tooling, or a large restructuring. +3. Assign a priority: + - `1 - Urgent` — high impact; blocks or badly misleads users. Aim for roughly + the top 15 to 20 percent. + - `2 - High` — clear improvement to effectiveness; should be done this cycle. + - `3 - Backlog` — worthwhile but low impact, cosmetic, or dependent on + decisions not yet made. +4. Keep the recommendation text as written in the source file, minus the leading + bullet marker. Do not paraphrase; `/create-issues` maps CSV rows back to the + source files. + +## Output + +Write `.github/input/ai-ranked-recommendations.csv`, overwriting any existing +file. Use these columns, in this order, with a header row: + +```csv +Section,Area,Recommendation,Suggested Priority +``` + +- `Section` — `Project Documentation`, `Contributor Documentation`, or + `Website & Infrastructure`. +- `Area` — the area display name (for example `Information Architecture`), not + the criteria filename or file stem. +- `Recommendation` — the recommendation text. Quote the field and double any + embedded quotes so the file is valid RFC 4180 CSV. +- `Suggested Priority` — `1 - Urgent`, `2 - High`, or `3 - Backlog`. + +Order rows by section (Project Documentation, Contributor Documentation, Website +& Infrastructure), then by area in the order the section file lists them, then +by priority. + +When finished, report the number of recommendations per priority and per +section, and ask the writer to review the ranking before running +`/create-issues`. diff --git a/docs/analysis/ai-harness/_github/prompts/sections/contributor-documentation-comment.prompt.md b/docs/analysis/ai-harness/_github/prompts/sections/contributor-documentation-comment.prompt.md new file mode 100644 index 00000000..63d1ba47 --- /dev/null +++ b/docs/analysis/ai-harness/_github/prompts/sections/contributor-documentation-comment.prompt.md @@ -0,0 +1,23 @@ +--- +mode: agent +description: + Write an overall comment on the Contributor Documentation analysis section by + synthesizing its per-area comment files. +--- + +# Contributor Documentation section comment + +Run the shared **section-comment** analysis for the `contributor-documentation` +section. + +1. Read the section definition in + `.github/criteria/sections/contributor-documentation.md`. +2. Read and follow the shared procedure in + `.github/prompts/sections/section-comment.prompt.md`, treating + `contributor-documentation` as the selected section. + +Optional output filename override: `${input:title}` — if non-empty, use it as +the output filename; otherwise use the engine's default +(`<project-slug>-<section-slug>-comment.md`). + +Output the file to the `.github/output/analysis` directory. diff --git a/docs/analysis/ai-harness/_github/prompts/sections/project-documentation-comment.prompt.md b/docs/analysis/ai-harness/_github/prompts/sections/project-documentation-comment.prompt.md new file mode 100644 index 00000000..197340d6 --- /dev/null +++ b/docs/analysis/ai-harness/_github/prompts/sections/project-documentation-comment.prompt.md @@ -0,0 +1,23 @@ +--- +mode: agent +description: + Write an overall comment on the Project Documentation analysis section by + synthesizing its per-area comment files. +--- + +# Project Documentation section comment + +Run the shared **section-comment** analysis for the `project-documentation` +section. + +1. Read the section definition in + `.github/criteria/sections/project-documentation.md`. +2. Read and follow the shared procedure in + `.github/prompts/sections/section-comment.prompt.md`, treating + `project-documentation` as the selected section. + +Optional output filename override: `${input:title}` — if non-empty, use it as +the output filename; otherwise use the engine's default +(`<project-slug>-<section-slug>-comment.md`). + +Output the file to the `.github/output/analysis` directory. diff --git a/docs/analysis/ai-harness/_github/prompts/sections/section-comment.prompt.md b/docs/analysis/ai-harness/_github/prompts/sections/section-comment.prompt.md new file mode 100644 index 00000000..a2f4e32f --- /dev/null +++ b/docs/analysis/ai-harness/_github/prompts/sections/section-comment.prompt.md @@ -0,0 +1,95 @@ +--- +mode: agent +description: + Shared engine for producing an overall comment on an analysis section by + synthesizing its per-area comment files. Normally run via a per-section + wrapper such as /project-documentation-comment. +--- + +# Shared engine: section-comment + +Reusable procedure for writing an overall comment on one analysis **section** +(for example, "Project Documentation"). Unlike the per-area `comment` engine, +this engine does not analyze the documentation directly — it synthesizes the +section's existing per-area comment files into a single overall comment. +Normally invoked through a per-section wrapper prompt (for example, +`/project-documentation-comment`), which selects the section for you. + +## Inputs + +- Section — the analysis section to comment on. When run from a wrapper, the + wrapper names it. If you reached this engine without a section, ask the user + to choose one of: `project-documentation`, `contributor-documentation`, + `website-infrastructure`. +- Title (optional) — overrides the default output filename. + +## Precondition + +This engine reads the per-area comment files that already exist in the +`.github/output/analysis` directory; it does not re-analyze the documentation. +Every area listed in the section definition must already have its per-area +comment file (produced by that area's `comment` wrapper, for example +`/information-architecture-comment`). If one or more are missing, stop and tell +the user exactly which per-area comment prompts to run first. Do not write a +partial section comment. + +## Procedure + +1. Read the section definition at `.github/criteria/sections/<section>.md`. Use + its Display name, Section slug, and the Areas table (each row gives an area's + criterion label and the stem of its per-area comment file). +2. Determine the project name and documentation label from the "Current + Repository" section of the repository Copilot instructions (for example, "the + KubeVirt user guide"). Derive the project slug by lowercasing the name and + replacing spaces with hyphens. +3. For each area row, read the existing per-area comment file + `<project-slug>-<stem>-comment.md` from the `.github/output/analysis` + directory. If any are missing, follow the Precondition above and stop. +4. From each area comment, extract its overall standing, its rating (the final + `Rating: <n> - <label>` line), and its key strengths and highest-impact gaps + or recommendations. +5. Build a summary ratings table with one row per area, in the order listed in + the section definition: + + | Criterion | Rating (1-5) | + | ----------------- | ------------- | + | <criterion label> | <n> - <label> | + + If an area comment has no `Rating:` line, use `N/A` for that row. + +6. If `website-infrastructure` is the current section, add this table and its + label for these other metrics: + + Other Metrics: + + | Criterion | [Rating (1-5)] | + | ------------------------------------------- | -------------- | + | A11y plan & implementation | [rating (1-5)] | + | Mobile-first plan & implementation | [rating (1-5)] | + | HTTPS access & HTTP redirect | [rating (1-5)] | + | Google Analytics 4 for production only | [rating (1-5)] | + | Indexing allowed for production server only | [rating (1-5)] | + | Intra-site / local search | [rating (1-5)] | + | Account custodians are documented | [rating (1-5)] | + +7. Write a two to five paragraph overall comment on the section beneath the + table(s). Synthesize rather than concatenate: lead with the section's overall + standing, describe only the most important cross-cutting themes that recur + across two or more areas. Do not describe detailed suggestions, as they + should be covered in the per-area comments and recommendations. Avoid large + paragraphs with listed details, so use bulleted lists where appropriate. Keep + it concise, actionable, and in complete sentences, following the repository's + analysis response style. + +## Output + +- Begin the file with a level-1 heading + `# <Section display name> - Overall Comment`. +- Follow the heading with the summary ratings table, then the two-to-five + paragraph comment. +- Write the result to a Markdown file in the `.github/output/analysis` + directory: + - If a title was provided, name the file `<title>.md`. + - Otherwise name it `<project-slug>-<section-slug>-comment.md`. + +Output the file to the `.github/output/analysis` directory. diff --git a/docs/analysis/ai-harness/_github/prompts/sections/website-infrastructure-comment.prompt.md b/docs/analysis/ai-harness/_github/prompts/sections/website-infrastructure-comment.prompt.md new file mode 100644 index 00000000..8dcc0ae2 --- /dev/null +++ b/docs/analysis/ai-harness/_github/prompts/sections/website-infrastructure-comment.prompt.md @@ -0,0 +1,23 @@ +--- +mode: agent +description: + Write an overall comment on the Website and Infrastructure analysis section by + synthesizing its per-area comment files. +--- + +# Website & Infrastructure section comment + +Run the shared **section-comment** analysis for the `website-infrastructure` +section. + +1. Read the section definition in + `.github/criteria/sections/website-infrastructure.md`. +2. Read and follow the shared procedure in + `.github/prompts/sections/section-comment.prompt.md`, treating + `website-infrastructure` as the selected section. + +Optional output filename override: `${input:title}` — if non-empty, use it as +the output filename; otherwise use the engine's default +(`<project-slug>-<section-slug>-comment.md`). + +Output the file to the `.github/output/analysis` directory. diff --git a/docs/analysis/ai-harness/_github/templates/doc-issue.md b/docs/analysis/ai-harness/_github/templates/doc-issue.md new file mode 100644 index 00000000..38288d46 --- /dev/null +++ b/docs/analysis/ai-harness/_github/templates/doc-issue.md @@ -0,0 +1,28 @@ +--- +title: { title } #{Issue number} +labels: [high, help wanted] +repo: { issues repo URL } +--- + +<!-- Generic documentation issue template. Replace this file with the target +project's own documentation issue template if it has one, keeping the four +top-level headings below (or update create-issues.prompt.md to match), and +name the file in the "Current Repository" section of copilot-instructions.md. --> + +## Description + +A clear and concise description of the problem or gap, where it is, and the +proposed change. + +## What you expected + +A clear and concise description of what done looks like for the reader. + +## URL + +Links to the affected documentation pages and source files. + +## Additional context + +Related issues, decisions the maintainers must make, and the sentence that links +the issue back to the CNCF TechDocs analysis. diff --git a/docs/analysis/ai-harness/_github/templates/umbrella-issue.md b/docs/analysis/ai-harness/_github/templates/umbrella-issue.md new file mode 100644 index 00000000..258d6d89 --- /dev/null +++ b/docs/analysis/ai-harness/_github/templates/umbrella-issue.md @@ -0,0 +1,18 @@ +--- +title: { title } #{Issue number} +labels: [high, help wanted] +repo: { issues repo URL } +--- + +## Umbrella Issue + +Describe the umbrella issue and how the sub-issues work to resolve it. + +## Sub-issues + +<!-- One line per sub-issue: the title and simulated number as link text, +the sub-issue file name (sibling of this file in .github/output/issues/) +as the target. --> + +- [{issue title} #{issue number}]({issue-number}-{condensed-issue-title}.md) +- [Reorganize Concepts #9938](9938-reorganize-concepts.md) diff --git a/docs/analysis/ai-harness/index.md b/docs/analysis/ai-harness/index.md new file mode 100644 index 00000000..fd42a1f2 --- /dev/null +++ b/docs/analysis/ai-harness/index.md @@ -0,0 +1,151 @@ +--- +title: AI Assisted Analysis Harness +--- + +# AI Assisted Analysis Harness + +This folder holds a reusable harness for running a CNCF TechDocs +[documentation analysis](../howto.md) with GitHub Copilot. The `_github/` folder +is a template for the target project's `.github/` folder: it contains Copilot +instructions, the analysis criteria broken into areas and sections, predefined +prompts for each step, issue templates, and input and output folders. The +harness is project-agnostic; everything project-specific lives in one section of +`copilot-instructions.md`. + +The harness was built for the KubeVirt analysis +([cncf/techdocs#343](https://github.com/cncf/techdocs/issues/343)). A worked +example of its project-specific section is in `_github/examples/kubevirt/`. + +## How it fits the analysis + +A writer still owns the analysis. The harness has Copilot answer the +[criteria](../criteria.md) questions, draft comments and ratings, propose +recommendations, rank them, and write issue files, all in a consistent format +that maps onto the [analysis template](../templates/analysis.md). The writer +reviews and edits every output before it goes into `analysis.md` or the +project's issue tracker. For how Copilot should write, see "Analysis responses +style" in `_github/copilot-instructions.md`. + +## Validated toolchain + +The prompts were validated with: + +- GitHub Copilot CLI 1.0.x (`copilot --version`; 1.0.103 at the time of + writing), run from the root of the target project's documentation repository + so that `.github/copilot-instructions.md` is picked up automatically. +- A Claude model (`/model` in the CLI shows the current one). The `full` engine + asks for "Claude default" output style; other models work but may need the + style section tuned. <!-- FILL IN: the model used for the KubeVirt run --> +- Node.js and Prettier (`npm run check:format` in `cncf/techdocs`) for + formatting the generated Markdown before pasting it into `analysis.md`. +- Prompt files that rely on two Copilot prompt-file features: `mode: agent` in + the front matter and the `${input:title}` variable for the optional output + filename override. The prompts are invoked as slash commands + (`/information-architecture-full`), which requires the files to end in + `.prompt.md` and live under `.github/prompts/`. + +Record the version and model you used in the analysis document's front matter or +introduction so later runs can be compared. + +## Pre-flight checklist + +Complete these before the first prompt run: + +- [ ] The project has an assistance request issue in `cncf/techdocs`, and a + branch in `cncf/techdocs` with `docs/analysis/<NNNN>-<project>/` + scaffolded from the [templates](../templates/) (`analysis.md`, + `implementation.md`, `issues-list.md`). +- [ ] The target repository's `.github/` has no existing `prompts/` or + `copilot-instructions.md` that would conflict. If it does, merge rather + than overwrite, and keep the harness instructions at the top. +- [ ] The issues repository has the labels the "Priority label mapping" table + maps to (for KubeVirt: `priority/critical-urgent`, `priority/high`, + `priority/backlog`) plus `help wanted`. Ask the maintainers to create any + that are missing before uploading issues. +- [ ] You can run `copilot` in the repository and it reports the expected + version and model. +- [ ] You have maintainer agreement that the analysis work is not committed to + the project's repository. The harness writes only under `.github/input` + and `.github/output`; keep those changes on a local branch. + +## Workflow + +1. **Install the harness.** Copy `_github/` into the root of the target + project's documentation repository and rename it to `.github/`. Every prompt + references `.github/...` paths, so the rename is required. Do not copy + `examples/`. +2. **Fill in the project context.** Complete every `<!-- FILL IN -->` + placeholder in the "Current Repository" section of + `.github/copilot-instructions.md`, using `examples/kubevirt/` as a model. If + the project has its own documentation issue template, copy it to + `.github/templates/` with the four headings `Description`, + `What you expected`, `URL`, and `Additional context`, and name it in the + "Issue template file" field. +3. **Run the area prompts.** For each of the seventeen areas, run `/<area>-full` + (for example `/information-architecture-full`). Each run writes three files + to `.github/output/analysis/`: answers, comment, and recommendations. Start a + new session for each area so context from one area does not leak into the + next. Review each comment's `Rating:` line; it feeds the section tables. +4. **Run the section prompts.** Run `/project-documentation-comment`, + `/contributor-documentation-comment`, and `/website-infrastructure-comment`. + Each synthesizes its member areas' comment files into an overall comment with + a ratings table. +5. **Assemble `analysis.md`.** Paste the outputs into the analysis document + following the "Analysis.md and Output Files Mapping" table in + `copilot-instructions.md`. Drop the temporary H1 headings. Edit for accuracy + and voice; the writer's judgment is the final word on ratings. +6. **Rank the recommendations.** Run `/rank-recommendations`. It writes + `.github/input/ai-ranked-recommendations.csv`. Review the priorities and + re-run with corrections if needed. +7. **Create issues.** Run `/create-issues` and review the scope of each file in + `.github/output/issues/`. Then, in a new session, run + `/create-umbrella-issues`. Both prompts update + `.github/output/issues-coverage.md`, which maps every recommendation to an + issue and lists the issue numbers. +8. **Upload the issues.** Use the upload process described below. Record the + real issue numbers in `issues-list.md` and link the umbrella issues from the + assistance request issue. +9. **Finish the analysis documents.** Complete `implementation.md` and + `issues-list.md` from the coverage report, then open the pull request in + `cncf/techdocs`. + +## Upload process + +The issue prompts write one Markdown file per issue with YAML front matter +(`title`, `labels`, `repo`) followed by the GitHub-flavored Markdown body. The +upload step reads every `*.md` file in `.github/output/issues/` except +`readme.md` and sub-folders, and for each file: + +1. Creates a GitHub issue in `repo` with the front matter `title` (minus the + trailing `#<number>`) and the body. +2. Maps the harness labels to repository labels using the "Priority label + mapping" table and applies them. +3. Records the simulated number (`#9001`) against the real issue number. +4. After all standalone issues exist, creates the `U-*` umbrella issues, + rewriting each `[title #9001](9001-file.md)` link to the real issue URL so + the umbrella's sub-issue list resolves. + +For KubeVirt this was done with a `gh issue create` script run by the writer; +the script is not part of the harness yet. Until it is added here, write the +script per project following the steps above, and upload standalone issues first +so the umbrella links can be resolved. Keep the resulting mapping of simulated +to real numbers in `.github/input/` for the `issues-list.md` step. + +## Folder layout + +```text +ai-harness/ + index.md # This file + _github/ # Copy to <project repo>/.github/ + copilot-instructions.md # Run-time instructions; fill in "Current Repository" + criteria/ # Per-area questions and per-section membership + prompts/ # Area, section, and issue prompts + templates/ # doc-issue.md and umbrella-issue.md + input/ # ai-ranked-recommendations.csv and ad hoc inputs + output/analysis/ # Area and section outputs + output/issues/ # Standalone and umbrella issue files + examples/kubevirt/ # Worked example of the project-specific section +``` + +For the structure of the criteria and prompt files, the output file naming, and +the `analysis.md` mapping, see `_github/copilot-instructions.md`.