Repository navigation
feat(amazonq): ITGen test HITL handlers, any-depth diff apply, completeHitlWithTestResults (behind ATX_ITGEN_ENABLED) - #2976
Open
xueningl-az wants to merge 3 commits into
Conversation
…GEN_ENABLED With integration tests on, the agent nests LBV under local-verification (CR-315347405), so LBV error-fix rounds move to depth 3 and are never applied, while the nested LBV step (depth 2, no diff) is reported as a failed apply on every poll. Behind the new ATX_ITGEN_ENABLED flag (stand-in for the row 18 feature flag), completed steps are selected by having a step-tagged CUSTOMER_OUTPUT artifact rather than by depth: one paginated job-wide ListArtifacts call, only when a SUCCEEDED step still needs a download. Steps without an artifact are skipped, not failed; steps apply in tree post-order. With the flag off, the depth-2 behavior is unchanged.
…esults behind ATX_ITGEN_ENABLED Language-server side of the ITGen-IDE contract v0 (sections 3-6), all gated on ATX_ITGEN_ENABLED; with the flag off every path is unchanged. - HITL selection: integ-app-launch, then integ-test-run, after LBV and missing-packages and before *-checkpoint, at all three selection sites. integ-* HITLs are returned with their StepId on the EXECUTING, PLANNING and AWAITING_HUMAN_INPUT paths, never treated as a plan-review ZIP. - getTransformInfo returns OpenHitls[] (TaskId, Tag, StepId, expiry), built from the ListHitls the poll already made. For integ-* HITLs the agent's request is saved once per task to artifactWorkspace/<job>/hitl/<taskId>/request.json and rejected (RequestError) when the schemaVersion major isn't 1. For integ-test-run the repo artifact with the generated tests is downloaded again for every task into itgen/<runId>/tests (TestsPath), since the agent rewrites it under the same id. Agent-supplied ids used as folder names are checked. - New aws/atxTransform/completeHitlWithTestResults: checks the result ZIP holds summary.json (test run) or launch-result.json (app launch) with a supported schemaVersion, uploads it as HITL_FROM_USER, completes the upload and submits the task by id (SubmitCriticalHitlTask).
|
Codecov Report❌ Patch coverage is 📢 Thoughts on this report? Let us know! |
pranav-firake
approved these changes
Oct 26, 2026
Follows the 10/26 contract update (interactive app launch):
- completeHitlWithTestResults adds the customer's changed files to every
transformed-side result ZIP (launch and test run) as
workspace/metadata.json + workspace/<relative path>, with the same scan
and metadata as updateWorkspace (tracked files modified since job
start). The IDE's ZIP is left as it is; a copy is uploaded. Original
side results are uploaded unchanged.
- AtxAppLaunchResult gains optional launchedBy ('ide' | 'customer').
- NOT_LOCALHOST is removed from AtxTestRunStatus (published URLs are
allowed on the original side).
- ATX_TEST_RUN_SCHEMA_VERSION is 1.1 (1.0 payloads are still accepted).
- The side comment now says the original runs from artifactWorkspace/before/.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Problem
Row 14 of the ITGen plan: the language-server side of the ITGen–IDE contract v0 (§2–§6), so the IDE can run integration tests for the Transform agent.
local-verification(CR-315347405). The LBV error-fix rounds move to depth 3, and the ITGen fix steps (app-start fixes, code fixes) sit at depth 4. Today the LS applies diffs only for depth-2 steps, and it reports the nested LBV step, which has no diff, as a failed apply on every poll.integ-app-launch/integ-test-runHITLs have to be handed to the IDE with their request, and the IDE needs a command to send results back.Solution
Everything is gated on the env var
ATX_ITGEN_ENABLED=true(isItGenEnabled(), a stand-in for the row 18 feature flag). With the flag off, every path behaves exactly as before, and there are explicit regression tests for EXECUTING, PLANNING and AWAITING_HUMAN_INPUT.1. Apply step diffs at any depth (
d18caf4c0)ListArtifactscall finds them, and only when a SUCCEEDED step still needs a download.DiffApplyFailed.2. Test HITL handlers (
dde3be967)integ-app-launch→integ-test-run→*-checkpoint, at all three selection sites.integ-*HITLs are returned with theirStepIdand are never treated as a plan-review ZIP.OpenHitls[]ongetTransformInfo(TaskId, Tag, StepId, ExpiryTimestampUTC), built from theListHitlscall the poll already makes.integ-*HITLs, the request is saved once per task toartifactWorkspace/<job>/hitl/<taskId>/request.json. AschemaVersionmajor other than 1 givesRequestErrorinstead.integ-test-run, the repo artifact with the generated tests is downloaded again for every task intoitgen/<runId>/tests(TestsPath), because the agent rewrites it under the same id.aws/atxTransform/completeHitlWithTestResults{WorkspaceId, TransformationJobId, TaskId, ResultZipPath, SolutionRootPath}:summary.json(test run) orlaunch-result.json(app launch) with a supportedschemaVersion;HITL_FROM_USER→CompleteArtifactUpload→SubmitCriticalHitlTaskby task id.Testing
tscis clean, and lint has no errors.getTransformInfoagainst a fake FES through a full scripted job: LBV rounds → original app launch → test run → ITGen test fix → rerun → transformed launch fails → app-start fix (depth 4) → relaunch → test run → code fix (depth 4) → rerun. The harness isn't part of this PR.Not in this PR: the beam diff path (beamed jobs still skip completed-step diffs), and the IDE handlers that consume this (rows 5/6, toolkit). The matching C# wrapper is in the toolkit PR from the same branch name.
Update (10/27): contract v0 schema 1.1
Follows the 10/26 contract update with Mark (interactive app launch), commit
c5289a7c1:completeHitlWithTestResultsadds the customer's changed files to every transformed-side result ZIP (launch and test run) asworkspace/metadata.json+workspace/<relative path>. It uses the same scan and metadata asupdateWorkspace(tracked files modified since job start). The IDE's ZIP is left as it is, and a copy is uploaded. Original-side results are uploaded unchanged.AtxAppLaunchResultgains optionallaunchedBy('ide' | 'customer').NOT_LOCALHOSTis removed fromAtxTestRunStatus(published URLs are allowed on the original side).ATX_TEST_RUN_SCHEMA_VERSIONis1.1(1.0 payloads are still accepted).sidecomment now says the original runs fromartifactWorkspace/before/.Tests: 548 netTransform tests pass (5 new for
workspace/).License
By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.