Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 4 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -10,6 +10,10 @@ Every entry says what changed as a caller sees it, why, and what to watch for

### Added

- **`performance/projects/snapshot/check.mjs`** checks the public read-only snapshot SDK with invented
data, preserving proposed state and alias provenance while omitting source bodies. It does not reset
or restore a project.

- **`performance/projects/large-document/result-read-once.json`**: the same run on Memo 0.80.0 / Core 0.369.0 charges
95,260 bytes, the document once (476,300 on 0.79.0). `after-fix.mjs` now pins those versions.

Expand Down
4 changes: 4 additions & 0 deletions performance/projects/PROCESS.md
Original file line number Diff line number Diff line change
Expand Up @@ -12,6 +12,10 @@ retained bytes; it does not run the applications or establish that a historical

## Existing evidence and its limits

The [published snapshot check](snapshot/README.md) exercises a selected annotation read with invented
data. It is functional release acceptance, separate from retrieval and extraction scores. It does
not establish a complete project backup or a safe reset/restore path.

The [installed post-fix observation](installed-observation/README.md) is a separate mutable-archive
measurement. It publishes totals through an allowlisted projection; cases, source references and
per-item rows remain private. Do not combine it with the frozen synthetic precision/recall scores.
Expand Down
2 changes: 2 additions & 0 deletions performance/projects/README.md
Original file line number Diff line number Diff line change
@@ -1,5 +1,7 @@
# Projects

- [Published snapshots](snapshot/README.md): invented-data checks of the released read-only annotation SDK; selected scope, source-body omission and proposed-state preservation.

- [Installed observation](installed-observation/README.md): allowlisted totals from a private fixed-window
gathering rerun after the document pagination fix; finished pagination remains separate from archive
completeness, relevance and interpretation.
Expand Down
43 changes: 43 additions & 0 deletions performance/projects/snapshot/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,43 @@
# Published project annotation snapshot check

Project gathering needs a safe way to retain selected annotations before an isolated reconstruction.
This check asks whether the published SDK can read that selection without copying external source
bodies, accepting proposed facts or creating an implicit store. It informs adoption of the read-only
snapshot API; it does not establish that a project can be cleared or restored.

## Setup and reproduction

Linux x64, Node24.19.0, exact published `@wirecat/cli-core@0.373.0`. Install independently with
lifecycle scripts disabled, then run the driver from this checkout:

```sh
T=$(mktemp -d -p /var/tmp) && HOME=$T XDG_CONFIG_HOME=$T/c XDG_DATA_HOME=$T/d XDG_CACHE_HOME=$T/k MESSAGING_STORE=$T/store WIRECAT_STATE_DIR=$T/s TMPDIR=$T NPM_CONFIG_USERCONFIG=/dev/null npm install --prefix "$T/package" --ignore-scripts --no-audit --no-fund @wirecat/cli-core@0.373.0 && HOME=$T XDG_CONFIG_HOME=$T/c XDG_DATA_HOME=$T/d XDG_CACHE_HOME=$T/k MESSAGING_STORE=$T/store WIRECAT_STATE_DIR=$T/s TMPDIR=$T node performance/projects/snapshot/check.mjs "$T/package/node_modules/@wirecat/cli-core/dist/store/index.js"
```

The driver seeds its own invented project, alias, imported document and agent-authored proposed
statement through the package's public store API. It closes the writer before using the dedicated
read-only snapshot API. It never accepts an existing input store or starts a provider connector.
Its only argument selects the built package entry; it never reads the owner's archive or config.

## Interpretation

The retained result records assertions, not performance or semantic scores:

- Version2 preserves alias provenance and project description.
- Repeating the same snapshot returns the same fingerprint.
- Proposed facts remain proposed, without confirmation provenance.
- External document bodies and local source paths are absent from the snapshot.
- Unknown fields are refused; a missing store is not created.
- Importing the SDK and calling the explicit APIs do not create the implicit `MESSAGING_STORE` path.

This selected format explicitly omits notes, specialized meeting memberships, tasks, memories,
labels and findings. A document hash checks content equality, not its meaning. This check does not
exercise destination preview, atomic restore, alias namespace collisions, OAuth, a live source,
extraction quality or the version2 planner introduced after this snapshot release.

The public fixture is independent of the private gathering request and any retained real-project
references. [Result and release provenance](result.json) record all assertions passing against the published
package. The registry tarball matches the release CI artifact byte-for-byte; its SHA-256 is
`d7fcf5d3f355095e723bfd2d80cca0d3c6435a6d4ad6f69dfb0bd037cc07ada3`.
The initial registry version lookups returned404 while npm propagated the upload; the check ran only
after the version became visible and the downloaded bytes matched validation.
81 changes: 81 additions & 0 deletions performance/projects/snapshot/check.mjs
Original file line number Diff line number Diff line change
@@ -0,0 +1,81 @@
import assert from "node:assert/strict"
import { existsSync, mkdtempSync } from "node:fs"
import { tmpdir } from "node:os"
import { join, resolve } from "node:path"
import { pathToFileURL } from "node:url"

// Run with temporary HOME/MESSAGING_STORE/WIRECAT_STATE_DIR; never accepts an existing store.
const entry = process.argv[2]
assert(entry, "pass the built published package store entry")
assert(process.env.MESSAGING_STORE && process.env.HOME && process.env.WIRECAT_STATE_DIR)
assert.equal(existsSync(process.env.MESSAGING_STORE), false)
const sdk = await import(pathToFileURL(resolve(entry)).href)
assert.equal(existsSync(process.env.MESSAGING_STORE), false)
const dir = mkdtempSync(join(tmpdir(), "published-project-snapshot-"))
const source = join(dir, "source.db")
assert.equal(existsSync(source), false)
const store = await sdk.openStore({ path: source, now: () => 1700000000000 })
let project
try {
project = await store.knowledge.addProject({
name: "Research Example",
key: "REX",
description: "Invented SDK check",
})
await store.knowledge.addVocabularyAlias({ kind: "project", id: project.id }, "Example study")
const folder = await store.notes.addFolder({ name: "Invented sources" })
const { note: doc } = await store.notes.saveFileNote({
folderId: folder.id,
path: "source.md",
title: "Example evidence",
text: "INVENTED_SOURCE_BODY_SENTINEL",
})
await store.facts.add({
type: "statement",
statement: "An invented report awaits review",
project: project.ref,
by: { bot: "synthetic-sdk-reviewer" },
evidence: [doc.ref],
})
} finally {
await store.close()
}
const first = await sdk.snapshotProjectAnnotations({ source, project: project.ref })
const second = await sdk.snapshotProjectAnnotations({ source, project: project.ref })
assert.equal(first.snapshot.version, 2)
assert.equal(first.snapshot.project.description, "Invented SDK check")
assert.equal(first.snapshot.aliases[0].name, "Example study")
assert.equal(first.snapshot.aliases[0].source, "owner")
assert.equal(first.snapshot.facts.length, 1)
assert.equal(first.snapshot.facts[0].status, "proposed")
assert.equal(first.snapshot.facts[0].confirmed_by, null)
assert.equal(first.fingerprint, second.fingerprint)
assert.equal(first.snapshot.coverage.notes, "omitted")
assert.equal(first.snapshot.coverage.meeting_memberships, "omitted")
const json = JSON.stringify(first.snapshot)
assert(!json.includes("INVENTED_SOURCE_BODY_SENTINEL"))
assert(!json.includes(dir))
assert.deepEqual(sdk.decodeProjectAnnotationSnapshot(json), first.snapshot)
assert.equal(sdk.projectAnnotationSnapshotFingerprint(json), first.fingerprint)
assert.throws(() => sdk.decodeProjectAnnotationSnapshot(JSON.stringify({ ...first.snapshot, unexpected: true })), {
code: "validation_error",
})
await assert.rejects(() => sdk.snapshotProjectAnnotations({ source: join(dir, "absent.db"), project: project.ref }), {
code: "not_found",
})
assert.equal(existsSync(join(dir, "absent.db")), false)
assert.equal(existsSync(process.env.MESSAGING_STORE), false)
process.stdout.write(
`${JSON.stringify({
public_sdk: true,
snapshot_version: 2,
stable_repeat: true,
proposed_state_preserved: true,
alias_provenance_preserved: true,
external_bodies_omitted: true,
unknown_field_refused: true,
missing_store_not_created: true,
implicit_store_created: false,
owner_data_used: false,
})}\n`,
)
30 changes: 30 additions & 0 deletions performance/projects/snapshot/result.json
Original file line number Diff line number Diff line change
@@ -0,0 +1,30 @@
{
"purpose": "Published selected-annotation snapshot acceptance, not a retrieval or restore score",
"package": {
"name": "@wirecat/cli-core",
"version": "0.373.0",
"source_commit": "73d84265481aa3662f91337986b8194ef883646b",
"release_run": 39432002556,
"tarball_sha256": "d7fcf5d3f355095e723bfd2d80cca0d3c6435a6d4ad6f69dfb0bd037cc07ada3",
"integrity": "sha512-+2Hn6NgUYqMaL0dnrlnTx6ACtKIMBRZ36MlSK0WvDtKQjp1/C1HidMPCoukN6+mPD4XqshiDel+0DtqpWKm0QQ==",
"registry_matches_validated": true,
"entry_sha256": "929011e1be283c8b15d7b1fc627fb0c33d809a76710517cc65eb300e438b2e61"
},
"runtime": {
"node": "v24.19.0",
"platform": "linux",
"arch": "x64"
},
"checks": {
"public_sdk": true,
"snapshot_version": 2,
"stable_repeat": true,
"proposed_state_preserved": true,
"alias_provenance_preserved": true,
"external_bodies_omitted": true,
"unknown_field_refused": true,
"missing_store_not_created": true,
"implicit_store_created": false,
"owner_data_used": false
}
}
Loading