Skip to content

Keep non-ASCII letters and punctuation in ids derived from names - #1605

Open
midigofrank wants to merge 2 commits into
mainfrom
fix-slugify-unicode
Open

midigofrank wants to merge 2 commits into
mainfrom
fix-slugify-unicode

Conversation

@midigofrank

@midigofrank midigofrank commented Oct 29, 2026 •

Copy link
Copy Markdown
Contributor

Short Description

Workflow, step and channel ids now keep non-ASCII letters and punctuation from their names, so they match the ids Lightning uses.

Fixes #1602
Fixes #1603

Background

Until OpenFn/lightning#5106 (September 2026), Lightning only allowed ASCII letters, digits, spaces, _ and - in job and workflow names, and slugify was written for that. Lightning now accepts any character except control characters, so names like 患者確認 or Event-based workflow 1 (rename) reach the CLI. Lightning's own v2 export builds ids by lowercasing the name and replacing spaces with -, and nothing else. This PR follows the same rule.

Implementation Details

slugify used to turn every non-word character into a hyphen. Names in other scripts lost their letters (患者確認 became an empty id), and names with punctuation got ids that didn't match Lightning's (Event-based workflow 1 rename and Event-based workflow 1 (rename) both became event-based-workflow-1-rename). It now trims, NFC-normalizes and lowercases the name, and turns spaces into - as Lightning does. Characters that aren't safe in file names (/ \ : * ? " < > |) and a leading . are also replaced with -. Those are the only cases where our ids differ from Lightning's.

The same slugify is used for workflow, step, channel and project ids. Project ids for names with punctuation change too. That's harmless, because projects are matched by uuid first.

Ids are no longer URL-safe. Nothing puts a local id in a URL: they're only used as file and folder names and as YAML keys.

Some different names still resolve to the same id:

  • names that differ only by case or by space vs hyphen (Foo Bar and foo-bar). Lightning's export rejects these too.
  • names that differ only by a replaced character (Q3/Q4 and Q3:Q4) or by a leading dot (.foo and -foo). Lightning keeps these characters in its ids, so its export allows these pairs.

A pull now fails with an error listing the clashing workflow, step or channel names. Clashing channel keys used to get a -2 suffix in toResourceChannels. They are now rejected like the others, so a project with channels named My Channel and my-channel that used to pull will now fail until one of them is renamed.

The new rules change some ids, which changes some resources.yaml channel keys. Merge matched channels by key only, so a channel whose key changed lost its id and was flagged as removed. To fix that, the checkout file now records channel_uuids by key, as it already does for workflow_uuids. from-fs reads them back, and mergeChannels matches by id first, then key, then name.

Matching by name covers checkouts written by an older CLI, which have no channel_uuids. One case can still recreate a channel: the checkout has no channel_uuids, the channel's name has punctuation, and the name was changed locally before deploying. The first pull with this version writes channel_uuids and closes that gap.

Known limits: Windows reserved names (CON, NUL) and trailing dots aren't handled.

QA Notes

  • Pull a project with workflow, step or channel names containing non-ASCII characters or punctuation, e.g. Flujo de atención (v2.0). The folder and step ids should keep those characters.
  • On a project pulled with an older CLI, pull again. Workflow folders and resources.yaml keys for those names will change. Channels should keep their ids and should not show up as removed on deploy.
  • Pull with an older CLI. Edit a step in a workflow whose step names have accents or punctuation, then deploy with this CLI without pulling first. The step should keep its uuid and should not be recreated.
  • Give two workflows (or two steps in one workflow, or two channels) names like My Flow and my-flow, then pull. It should fail with an error naming both.

AI Usage

Please disclose whether you've used AI anywhere in this PR (it's cool, we just
want to know!):

  • I have used Claude Code
  • I have used another model
  • I have not used AI

You can read more details in our
Responsible AI Policy

slugify replaced every non-word character with a hyphen, so names in
other scripts lost their letters and their ids didn't match Lightning's.
It now only replaces characters that aren't safe in file names.

Names that differ only by case or by space vs hyphen still give the same
id. A pull now rejects them, as Lightning's export does, so the
resources.yaml writer no longer adds -2 suffixes to channel keys.
The new slug rules change some resources.yaml keys, and merging only
matched channels by key, so a channel whose key changed lost its id and
was treated as removed. The checkout file now records channel_uuids by
key, as it already does for workflows. A merge matches channels by id
first, then by key, then by name.
@github-project-automation github-project-automation Bot moved this to New Issues in Core Oct 29, 2026
@midigofrank midigofrank changed the title Fix slugify unicode Keep non-ASCII letters and punctuation in project ids Oct 29, 2026
@midigofrank midigofrank changed the title Keep non-ASCII letters and punctuation in project ids Keep non-ASCII letters and punctuation in ids derived from names Oct 29, 2026
@midigofrank
midigofrank marked this pull request as ready for review October 29, 2026 15:30
@midigofrank midigofrank self-assigned this Oct 29, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

Status: New Issues

1 participant