Skip to content

fixed: pass namespace files to cloudquery commands - #190

Merged
fdelbrayelle merged 1 commit into
kestra-io:mainfrom
Pavani3001:fix-cloudquery-namespace-files
Oct 27, 2026
Merged

fdelbrayelle merged 1 commit into
kestra-io:mainfrom
Pavani3001:fix-cloudquery-namespace-files

Conversation

@Pavani3001

Copy link
Copy Markdown
Contributor

What changes are being made and why?

This PR fixes the handling of namespaceFiles in the CloudQueryCLI task.

The namespaceFiles were rendered but were not being passed to the underlying CommandsWrapper, so the files were unavailable when executing the CloudQuery command.

This change passes namespaceFiles to CommandsWrapper, ensuring they are correctly available during execution.

Closes #188


How the changes have been QAed?

The CloudQueryCLI implementation was updated to pass namespaceFiles to CommandsWrapper.

The relevant CloudQuery CLI tests were executed to verify the change.


Setup Instructions

No additional setup is required.


Contributor Checklist ✅

  • I have read and followed the plugin contribution guidelines.

@github-project-automation github-project-automation Bot moved this to To review in Pull Requests Oct 24, 2026
@kestrabot kestrabot Bot added area/plugin Plugin-related issue or feature request kind/external Pull requests raised by community contributors labels Oct 24, 2026
@kestrabot
kestrabot Bot requested review from a team and fdelbrayelle October 24, 2026 07:29

@fdelbrayelle fdelbrayelle left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Kestra Plugin Code Review

Business Requirements — met

Issue #188 asks for CloudQueryCLI to pass namespaceFiles to the CommandsWrapper, the same way Sync does. The one-line change does exactly that, with no scope creep: no build, dependency, or CI changes. The PR head compiles (compileJava, compileTestJava).

Kestra Guidelines — 2 findings

  • 🟠 CloudQueryCLI.java:109 [Guidelines] No regression test for namespace files
  • 🟢 PR title / commit message [Guidelines] Not in Conventional Commits form: fixed: pass namespace files ... should be fix(cloudquery): pass namespace files to CloudQueryCLI commands (Pull Request & Commit Descriptions). This matters because the title becomes the squash-merge commit.

Security (OWASP Top 10:2025 + KPS) — 0 findings

✅ No issues found

Performance — 0 findings

✅ No issues found

Confidence: 59% (100 - 15 Request Changes - 1 Nit = 84, capped at 59 for REQUEST CHANGES)

Verdict: REQUEST CHANGES; recommendation only, posted as COMMENT, a maintainer decides.

Comment thread src/main/java/io/kestra/plugin/cloudquery/CloudQueryCLI.java
@Pavani3001
Pavani3001 force-pushed the fix-cloudquery-namespace-files branch from df26dd9 to 63aac81 Compare October 26, 2026 23:55

@fdelbrayelle fdelbrayelle left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Kestra Plugin Code Review

Business Requirements: met

Issue #188 asks CloudQueryCLI to pass namespaceFiles to the CommandsWrapper. The fix does that, and the new commit adds a regression test. No build, dependency or CI changes.

Previous Review Follow-up: 1 of 2 addressed

Baseline: 70679df8 (force-pushed; new head 63aac81d is a single commit on the same base 29e37f90, compared against the old head content directly).

  • ✅ CloudQueryCLI.java:109 [Guidelines] No regression test for namespace files: CloudQueryCLITest#namespaceFiles (lines 61-85) stores config.yml in the namespace and runs test -f config.yml. Ran locally on 63aac81d: 2/2 tests pass. With .withNamespaceFiles(namespaceFiles) removed, the test fails with Command failed with exit code 1, so it guards #188. The thread was already resolved.
  • 🟡 PR title / commit message [Guidelines] Conventional Commits: the commit is now fix(cloudquery): pass namespace files to CloudQueryCLI commands, but the PR title is still fixed: pass namespace files to cloudquery commands. On squash-merge the title becomes the commit, so please rename it too (or the maintainer can edit it when merging).

Kestra Guidelines: 1 finding

  • 🟢 CloudQueryCLITest.java:64 [Guidelines] Use var for locals in the new test

Security (OWASP Top 10:2025 + KPS): 0 findings

✅ No issues found

Performance: 0 findings

✅ No issues found

Verdict: APPROVE; recommendation only, posted as COMMENT, a maintainer decides.

Confidence: 98% (100 - 1 Nit - 1 Nit still open from the previous review)


@Test
void namespaceFiles() throws Exception {
CloudQueryCLI execute = CloudQueryCLI.builder()

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟢 [Guidelines] Use var for locals in the new test
Problem: The new namespaceFiles test declares CloudQueryCLI execute, RunContext runContext and ScriptOutput runOutput with explicit types, while the Java Style & Local Variables section asks for var on all local variables where it is legal.
Fix: Write var execute = CloudQueryCLI.builder()..., var runContext = TestsUtils.mockRunContext(...) and var runOutput = execute.run(runContext). Optional.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ok will cross check and fix it

@fdelbrayelle

Copy link
Copy Markdown
Member

QA report

Commit: 63aac81
Edition: OSS | Docker tag: v1.3.44 (plugin kestraVersion 1.3.39 line, highest patch)
Instance: http://localhost:18190 (container qa-cloudquery190-kestra, left running; docker rm -f qa-cloudquery190-kestra to stop; login admin@kestra.io / Kestra42!)
Screenshots: https://claude.ai/artifact/TFR1rVTPrDpjSg5CQT2boD
Confidence: 100% (3 PASS of 3 scenarios = 100%, no deductions)

Result: PASS. The one-line fix works: CloudQueryCLI now makes namespace files available to the command container. The same flow FAILS on the main JAR and PASSES on the PR JAR.

Setup: PR head JAR built with shadowJar (build-file gate: PR only touches CloudQueryCLI.java and its test, no build files). Local namespace file config.yml in namespace qa.cloudquery, default Docker task runner with the real ghcr.io/cloudquery/cloudquery image (cloudquery 6.49.2). No cloud, no secrets. Gradle tests were not run (CI already covers CloudQueryCLITest#namespaceFiles).

Scenario Type PR JAR main JAR (baseline)
cq_namespace_files new behavior SUCCESS FAILED (cat: config.yml: No such file or directory)
cq_input_files namespace files + inputFiles together SUCCESS FAILED
cq_no_namespace_files non-regression (no namespaceFiles, file must be absent) SUCCESS SUCCESS

Flow 1: cq_namespace_files (SUCCESS)

Flow YAML
id: cq_namespace_files
namespace: qa.cloudquery
tasks:
  - id: cli
    type: io.kestra.plugin.cloudquery.CloudQueryCLI
    namespaceFiles:
      enabled: true
    commands:
      - cloudquery --version
      - cat config.yml

Gantt (screenshot)

Task Status Duration
cli SUCCESS 1.1s
Total SUCCESS 1.6s

Logs synthesis (screenshot)
"Loaded 1 namespace files from 'qa.cloudquery'", then cat config.yml prints the file content (kind: source, name: sqlite_local) and cloudquery version 6.49.2. Exit code 0.

Outputs synthesis: exitCode: 0, vars: {}, no output files.

Main baseline (screenshot): same flow on the main JAR fails with cat: config.yml: No such file or directory, exit code 1. This reproduces the bug from #188.

Flow 2: cq_input_files (SUCCESS)

Flow YAML
id: cq_input_files
namespace: qa.cloudquery
tasks:
  - id: cli
    type: io.kestra.plugin.cloudquery.CloudQueryCLI
    inputFiles:
      extra.yml: "kind: destination"
    namespaceFiles:
      enabled: true
    commands:
      - cat config.yml extra.yml

Gantt

Task Status Duration
cli SUCCESS 0.8s
Total SUCCESS 0.8s

Logs synthesis (screenshot)
Both the namespace file and the input file are present; cat prints both contents. Exit code 0. The inputFiles option is not broken by the new withNamespaceFiles call. On main this flow fails.

Flow 3: cq_no_namespace_files (SUCCESS, non-regression)

Flow YAML
id: cq_no_namespace_files
namespace: qa.cloudquery
tasks:
  - id: cli
    type: io.kestra.plugin.cloudquery.CloudQueryCLI
    commands:
      - cloudquery --version
      - test ! -f config.yml

Gantt (screenshot)

Task Status Duration
cli SUCCESS 0.8s
Total SUCCESS 0.8s

Logs synthesis: cloudquery version 6.49.2; with namespace files not enabled, config.yml is correctly absent (no namespace file leak). Exit code 0.

Notes

  • Early drafts of the flows ended with a && cat chain that masked a failing first command (script has no set -e); flows were rewritten so the failing step is the last command, which is what the table above reflects.
  • No timeouts. Topology section not applicable (no UI/artifact change).

@fdelbrayelle
fdelbrayelle merged commit 73132b3 into kestra-io:main Oct 27, 2026
4 checks passed
@github-project-automation github-project-automation Bot moved this from To review to Done in Pull Requests Oct 27, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

area/plugin Plugin-related issue or feature request kind/external Pull requests raised by community contributors

Projects

Status: Done

Development

Successfully merging this pull request may close these issues.

CloudQueryCLI ignores namespaceFiles

2 participants