Skip to content

Example CLI built on gfw-common, without Beam - #19

Merged
tomaslink merged 2 commits into
mainfrom
PIPELINE-4673-example-cli-gfw-common
Oct 16, 2026
Merged

tomaslink merged 2 commits into
mainfrom
PIPELINE-4673-example-cli-gfw-common

Conversation

@tomaslink

@tomaslink tomaslink commented Oct 16, 2026 •

Copy link
Copy Markdown
Collaborator

PIPELINE-4673 · Stacked on #18 (merge that first).

Summary

Replaces the template's plain sys.argv entry point with an example CLI built on gfw-common, so new projects start from the standard pattern. No Beam: gfw-common is installed without extras, so the template also suits services.

Layout (as in pipe-anchorages)

src/python_app_template/
  cli/main.py                 # gfw-common CLI; registers commands; --labels shared by all
  cli/commands/hello.py       # Command: name, description, options → builds the config, calls its pipeline
  pipelines/hello/config.py   # HelloConfig(PipelineConfig): name, times (+ labels)
  pipelines/hello/main.py     # run(config: HelloConfig): the logic
config/hello/sample.yaml      # example configuration file
  • python-app-template hello --name GFW --times 2 --labels environment=development, or -c config/hello/sample.yaml. Command-line arguments override the config file, and the config file overrides the defaults.
  • HelloConfig inherits PipelineConfig (labels, from_namespace) and extends its validation (times >= 1).
  • run() receives the typed config, not the CLI's SimpleNamespace: the command converts the CLI input at the edge (HelloConfig.from_namespace), so run has a typed signature that mypy checks, and other callers (DAGs, scripts, tests) build a validated HelloConfig directly. This differs from pipe-anchorages, where run converts the namespace itself; the template is meant to be the reference going forward.
  • Tests: the command with run mocked (defaults, options, config file precedence), the configuration and the logic as plain functions.
  • README: how to use the CLI, a config file example, and the new modules and config/ directory.

init_project.py: rename every Python file of the package and the tests

It only renamed a fixed list of files, so the new modules that import the package by name would have broken after renaming (reproduced: 6 files kept python_app_template). It now renames every .py file under src/<package>/ and tests/ (except its own tests, which use the template's name on purpose), and requirements-test.txt, whose compiled comments name the package.

Note

gfw-common doesn't ship a py.typed marker, so mypy treats its classes as Any; subclassing them under strict mypy needs # type: ignore[misc]. Follow-up: add py.typed to gfw-common.

Verification

  • pytest: 18 passed, 100% coverage; flake8, ruff, black, mypy, codespell: clean.
  • make docker-build && make docker-test: 11 passed (the image excludes init_project's tests). The production image runs python-app-template hello, and doesn't include apache_beam (148 MB).
  • python init_project.py my-service on a copy: its 18 tests pass, my-service hello -c config/hello/sample.yaml greets twice, and the only remaining python-app-template strings are the links to the shared contributing guide.

🤖 Generated with Claude Code

@tomaslink tomaslink self-assigned this Oct 16, 2026
@tomaslink
tomaslink added this pull request to stack #20 October 16, 2026 19:26
Base automatically changed from PIPELINE-4672-shared-contributing-guide to main October 16, 2026 19:29
init_project.py only renamed a fixed list of files, so any new module that
imports the package by name (e.g. cli/main.py importing its commands) kept
the template's name and broke after renaming. It now renames every .py file
under src/<package>/ and tests/, except init_project's own tests, and also
requirements-test.txt, whose compiled comments name the package.
Replaces the plain sys.argv entry point with gfw-common's CLI, laid out as
in pipe-anchorages:

- cli/main.py builds the CLI and registers the commands, with the shared
  --labels option.
- cli/commands/hello.py: an example command that only defines its name,
  description and options, and delegates to its pipeline.
- pipelines/hello/: its configuration (HelloConfig, a PipelineConfig) and
  its logic (run()).
- config/hello/sample.yaml: an example configuration file. Command-line
  arguments take precedence over it, and it over the defaults.
- Tests for the command (with run mocked), the configuration and the logic.
- README: how to use the CLI and the new modules.

gfw-common is installed without extras, so no Beam: the production image
doesn't include it. gfw-common doesn't ship a py.typed marker, so mypy sees
its classes as Any; subclassing them needs a type: ignore[misc].
@tomaslink
tomaslink force-pushed the PIPELINE-4673-example-cli-gfw-common branch from b45f370 to f030070 Compare October 16, 2026 19:29
@tomaslink
tomaslink merged commit e7cc4e8 into main Oct 16, 2026
3 checks passed
@tomaslink
tomaslink deleted the PIPELINE-4673-example-cli-gfw-common branch October 16, 2026 19:33
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant