Skip to content

Allow Extra NR catalogues to be included from the TeamType - #8946

Open
hardillb wants to merge 17 commits into
mainfrom
team-type-catalogues
Open

hardillb wants to merge 17 commits into
mainfrom
team-type-catalogues

Conversation

@hardillb

Copy link
Copy Markdown
Contributor

fixes FlowFuse/engineering#534

Description

Allow catalogue URLS to be added to team types and only added to devices and instance is they are not already present

Related Issue(s)

FlowFuse/engineering#534

Checklist

  • I have read the contribution guidelines
  • Suitable unit/system level tests have been added and they pass
  • Documentation has been updated
    • Upgrade instructions
    • Configuration details
    • Concepts
  • Changes flowforge.yml?
    • Issue/PR raised on FlowFuse/helm to update ConfigMap Template
    • Issue/PR raised on FlowFuse/CloudProject to update values for Staging/Production
  • Link to Changelog Entry PR, or note why one is not needed.

Labels

  • Includes a DB migration? -> add the area:migration label

fixes FlowFuse/engineering#534

Allow catalogue URLS to be added to team types and only added to
devices and instance is they are not already present
These didn't flag locally
@codecov

codecov Bot commented Oct 16, 2026 •

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 80.71%. Comparing base (c8e2fba) to head (ce6402c).
⚠️ Report is 45 commits behind head on main.

Additional details and impacted files
@@            Coverage Diff             @@
##             main    #8946      +/-   ##
==========================================
+ Coverage   80.01%   80.71%   +0.70%     
==========================================
  Files         476      477       +1     
  Lines       26191    26390     +199     
  Branches     6982     7029      +47     
==========================================
+ Hits        20958    21302     +344     
+ Misses       5233     5088     -145     
Flag Coverage Δ
backend 80.71% <100.00%> (+0.70%) ⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

@hardillb

Copy link
Copy Markdown
Contributor Author

The failing UI tests appear to pass locally

@knolleary knolleary left a comment •

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Whilst this works, had some feedback for bits that might need tidying up.

In both deviceLive.js and project.js

  • Accesses team.TeamType?.properties?.extraCatalogues directory. Should be team.getProperty("extraCatalogues", null) so that it could be overridden at the team level
  • The check for that property assumes response.palette.catalogues exists. Earlier in the function is a check that initialises that property based on other properties. Not sure if there's a combination there that will trip us up
  • We only apply this setting if cert nodes is enabled - but at the TeamType dialog this is presented as a general option for additional catalogues that don't need to be certified. Either we want to make this option truly about adding catalogues without it being specific to cert nodes (move the work to append the catalogues out of the containing if block), or we update the dialog to be explicit this is about cert nodes.
  • URL.parse(cat) was flagged as a new API that cuts off Safari 17 which isn't that old. We don't use it in the code base already, so potentially breaking.

@hardillb

hardillb commented Nov 5, 2026

Copy link
Copy Markdown
Contributor Author

I've fixed 1, 3 and 4

As for 2, I think we are safe as cert nodes or ffnodes has to be enabled, so we will have injected at least one catalogue.

@knolleary knolleary left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Need to try/catch url constructor to handle invalid entries.

Co-authored-by: Nick O'Leary <nick.oleary@gmail.com>

@Steve-Mcl Steve-Mcl left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Works as described - requesting changes mainly on validation:

  • extraCatalogues is not validated by the API (server side). The create/update team type routes (forge/routes/api/teamType.js, properties: { type: 'object' } on the POST and PUT schemas) accept any shape, and only the dialog checks for URLs. I can't comment inline cos those lines arent in the diff, but I recomend we add add server side validation (array of http(s) URL strings, 400 otherwise) and make the runtime loops (for (const cat of xxx) defensive (cos a string value is iterated per character and a non iterable would throw in to the catch block).

Thinks of note

  • The dialog doesn't say one URL per line, and invalid lines are dropped silently.
  • Dedupe compares bare strings, and the test dont assert form diffs.
  • Tests mutate shared default team type (I think! eye scan only).
  • The team type GET returns extraCatalogues to non-admin users (the view only strips price/product IDs). Catalogue URLs might carry tokens?

updateSettingsForCatalogue('@flowfuse-certified-nodes', cat)
}
}
if (team.getProperty('extraCatalogues', null)) {

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

  1. getProperty('extraCatalogues') is the guard, but the loop reads team.TeamType.properties.extraCatalogues directly. If a team-level override ever exists, the check and the value come from different places. Use one source, e.g. const extra = team.getProperty('extraCatalogues', null) and loop over extra.
  2. Add Array.isArray(extra) and skip entries that are not valid URL strings, so bad stored data can't break the block (or add a character per iteration if it is a string).

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Also this only runs inside the platformNPMEnabled && (certifiedNodes || ffNodes) block, but the UI disables the field based on certifiedNodes only.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Also this only runs inside the platformNPMEnabled && (certifiedNodes || ffNodes) block, but the UI disables the field based on certifiedNodes only.

Yes, because ffNodes adds a URL to the list, but the override box is to ONLY add more certified nodes catalogues

Comment thread forge/routes/api/deviceLive.js Resolved
Comment thread forge/routes/api/project.js Resolved
Comment thread forge/routes/api/project.js Resolved
Comment on lines +1911 to +1916
async function setTeamTypeCatalogues (cats) {
const defaultTeamTypeProperties = app.defaultTeamType.properties
defaultTeamTypeProperties.extraCatalogues = cats
app.defaultTeamType.properties = defaultTeamTypeProperties
await app.defaultTeamType.save()
}

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think this mutates deaullt team type. Maybe a before (to store original) and an afterEach (to reset it) is needed?

async function setTeamTypeCatalogues (cats) {
const defaultTeamTypeProperties = app.defaultTeamType.properties
defaultTeamTypeProperties.extraCatalogues = cats
app.defaultTeamType.properties = defaultTeamTypeProperties

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

same mutation as device tests - should we have before (to capture starting value) and afterEach (to restoire)?

settings.palette.should.have.property('catalogue')
settings.palette.catalogue.should.containEql('https://example.com/team-type-cat.json')
})
it('Should add extra catalogue to project only if not present', async function () {

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This sends a duplicate but never asserts it appears. It splices the two entries out and then asserts cat2 is gone, which passes regardless of dedupe. I think we should assert the count of team-type-cat.json is exactly 1.

Noted the device spec has no dedupe test either.

Additionally, I think a test that uses URLs which differ only in form but resolve to the same catalogue (e.g. https://example.com/cat.json and https://EXAMPLE.com/cat.json, or https://example.com and https://example.com/) would be worthwhile. Existing entries are normalised with new URL(...).toString(), but the extra catalogues are compared and pushed as raw strings, so indexOf treats these as different and both end up in the list. Asserting that only one entry remains should fail as it stands.

@hardillb

Copy link
Copy Markdown
Contributor Author

Catalogue URLs might carry tokens?

Nope, catalogue URLs can not hold tokens

This branch was successfully deployed

1 active deployment
staging — ce6402c3 Deployed Nov 16, 2026 by hardillb via Deploy application #13132
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants