Skip to content

{Tests} Add validation error unit tests for Resources - #1469

Open
MaddyMicrosoft wants to merge 6 commits into
mainfrom
test/resources-error-unit-tests
Open

MaddyMicrosoft wants to merge 6 commits into
mainfrom
test/resources-error-unit-tests

Conversation

@MaddyMicrosoft

Copy link
Copy Markdown
Member

Related command
az deployment group create|validate|what-if, az deployment mg create|validate|what-if, az deployment tenant wait, az deployment operation group show, az role assignment create|delete|list, az group update, az rest (and the matching PowerShell cmdlets)

Description
Adds unit tests that lock in the exact user-facing error messages produced by custom input validation in the Resources extension. If the core argument parser, the engine, or an orchestrator changes in a way that alters or skips one of these errors (for example during the upcoming parameter-set work), a test will now fail.

New test project:

  • Adds tests/Extensions.Resources.Tests (registered in azclips.slnx and listed in the testing wiki), matching the other extension test projects.
  • Adds InternalsVisibleTo for Extensions.Resources.Tests in Extensions.Resources.csproj so the internal validators can be tested directly. No product behaviour change.

What is covered:

  • A test class per shared validator: DeploymentRequestBuilder, DeploymentIdResolver, ResourceGroupIdResolver, RoleAssignmentScope, JmesPathCondition and ManagementGroupDeploymentOrchestratorBase. Each covers every rejection reason, including blank and null values.
  • One test class per orchestrator, with a test proving it calls the shared checks before any request, plus tests for rules specific to that orchestrator (for example the what-if ResultFormat).
  • Error tests for the existing Resources tests in Cli.Tests (resource group deployment create/validate/what-if, role assignment create/delete/list, group update and rest). Only the tests added by this PR follow the new style; existing tests in those files are unchanged.

How the tests work:

  • They are unit tests only: no CLI parsing, no scenario host, no network.
  • Each test has one assert, on the exact message and exception type, and follows arrange-act-assert.
  • The HTTP pipeline is a strict Moq mock, so any request throws. A passing test therefore also proves the validation ran before any network call.
  • Some tests pin current behaviour that looks unintended, with a Current behaviour: comment. For example, deployment validate throws ArgumentException where create throws InvalidInputException for the same rollback rule, and malformed deployment ids surface as InvalidOperationException. These are left for separate fixes.

  • The PR title starts with [Component Name] for customer-facing changes or {Component Name} for non-customer-facing changes, and does not end with trailing punctuation or spaces.
  • Customer-facing changes include a useful release note in the PR title or the History Notes section.
  • I added or updated tests for behavior changes.
  • I updated documentation when behavior, setup, support, or security guidance changed.

MaddyMicrosoft and others added 6 commits November 16, 2026 01:31
Add a dedicated unit test project for the Resources extension, with a helper that initializes processors against a pipeline that throws on any request. Register it in azclips.slnx so CI runs it, point the Resources InternalsVisibleTo at it, and list it in the testing wiki.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Cover every input case of RoleAssignmentScope, DeploymentRequestBuilder, ResourceGroupIdResolver, DeploymentIdResolver and JmesPathCondition with exact messages: each rejection reason, blank and null values, both sides of each boundary, and accepted inputs.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Add single-assert tests for the management group, tenant wait and deployment operation orchestrators, and tighten the resource group deployment create, validate and what-if tests: one test per shared validator call plus each orchestrator's own rules (rollback options, ResultFormat, required inputs, wait conditions).

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Assert exact messages for role assignment create, list and delete input rules (assignee, role, scope and resource group, --all combinations, GUID checks, blank values). The shared AssertThrows helpers now require the exact exception type and message while keeping their existing checks.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Assert exact messages for az rest input rules (method, URL vs piece-wise parameters, ARM host and https, URI parameters, headers) and resource group set/update rules (name vs id, tags, --set paths).

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
…class under test

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
@MaddyMicrosoft
MaddyMicrosoft marked this pull request as ready for review November 16, 2026 06:03
Copilot AI balanced review requested due to automatic review settings November 16, 2026 06:03

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟢 Approval recommended

No active review issues block the Resources input-validation test changes.

No open issues 🎉

What changed in this PR

Adds regression tests for Resources input validation, covering exception types, user-facing messages, and validation before HTTP requests without changing product behavior.

Changes:

  • Adds a Resources unit-test project covering shared validators and deployment orchestrators.
  • Adds validation cases and strengthens message assertions in existing CLI test suites.
  • Registers the project, grants internal access for testing, and updates the testing wiki.
File Description
wiki/​02-deep-dive/​testing-benchmarking.md Lists the new Resources test project.
tests/​Extensions.Resources.Tests/​WhatIfManagementGroupDeploymentProcessor_OrchestratorTests.cs Tests management-group what-if validation and error precedence.
tests/​Extensions.Resources.Tests/​WaitTenantDeploymentProcessor_OrchestratorTests.cs Tests missing wait conditions and invalid custom expressions.
tests/​Extensions.Resources.Tests/​ValidateManagementGroupDeploymentProcessor_OrchestratorTests.cs Tests management-group validation before requests.
tests/​Extensions.Resources.Tests/​RoleAssignmentScopeTests.cs Tests scope validation and normalization.
tests/​Extensions.Resources.Tests/​ResourcesProcessorTestHelpers.cs Provides shared processor initialization for mocked tests.
tests/​Extensions.Resources.Tests/​ResourceGroupIdResolverTests.cs Tests resource-group ID parsing and errors.
tests/​Extensions.Resources.Tests/​ManagementGroupDeploymentOrchestratorBaseTests.cs Tests shared management-group deployment checks.
tests/​Extensions.Resources.Tests/​JmesPathConditionTests.cs Tests expression validation and error prefixes.
tests/​Extensions.Resources.Tests/​GetResourceGroupDeploymentOperationProcessor_OrchestratorTests.cs Tests rejection of empty operation IDs.
tests/​Extensions.Resources.Tests/​Extensions.Resources.Tests.csproj Defines test dependencies, references, and Release signing.
tests/​Extensions.Resources.Tests/​DeploymentRequestBuilderTests.cs Tests deployment inputs, parameter files, and required parameters.
tests/​Extensions.Resources.Tests/​DeploymentIdResolverTests.cs Tests deployment ID parsing at resource-group and management-group scopes.
tests/​Extensions.Resources.Tests/​CreateManagementGroupDeploymentProcessor_OrchestratorTests.cs Tests management-group creation validation before requests.
tests/​Cli.Tests/​RoleAssignmentDeleteTests.cs Pins delete validation messages and adds blank-name cases.
tests/​Cli.Tests/​RoleAssignmentCreateTests.cs Pins create validation messages and adds invalid principal and role cases.
tests/​Cli.Tests/​RoleAssignmentCliTests.cs Adds role-assignment list validation cases.
tests/​Cli.Tests/​RestProcessorTests.cs Expands REST validation coverage and exact-message assertions.
tests/​Cli.Tests/​ResourcesProcessorMockHelpers.cs Adds shared mock initialization with optional profiles.
tests/​Cli.Tests/​ResourceGroupDeploymentWhatIfTests.cs Tests invalid result format, resource group, and mode.
tests/​Cli.Tests/​ResourceGroupDeploymentValidateTests.cs Tests rollback conflicts and strengthens deployment validation assertions.
tests/​Cli.Tests/​ResourceGroupDeploymentCreateTests.cs Tests rollback conflicts, missing resource groups, and invalid modes.
tests/​Cli.Tests/​GroupUpdateTests.cs Expands address, tag, and update-expression validation tests.
src/​Extensions/​Resources/​Extensions.Resources.csproj Grants the new test assembly access to internal validators.
azclips.slnx Registers the Resources test project.

Review effort: Balanced

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants