Skip to content

IBX-13142: Ran the language bitmask upgrade test on MySQL and PostgreSQL too - #960

Draft
Steveb-p wants to merge 54 commits into
feature/language-bitmask-6-remove-6.0from
feature/language-bitmask-upgrade-test-engines-6.0
Draft

Steveb-p wants to merge 54 commits into
feature/language-bitmask-6-remove-6.0from
feature/language-bitmask-upgrade-test-engines-6.0

Conversation

@Steveb-p

@Steveb-p Steveb-p commented Nov 15, 2026 •

Copy link
Copy Markdown
Contributor
🎫 Issue IBX-13142

Related PRs:

Description:

LanguageBitmaskUpgradeSequenceTest (#946) runs the whole language bitmask upgrade, from the always-available columns to the mask drop, on a seeded pre-6.0 database. It lives in the unit suite though, so CI only ever runs it on SQLite, and the MySQL and PostgreSQL SQL of those migrations (the strip's correlated NOT EXISTS, the per-platform object state DELETEs, the drop's guards) never runs anywhere but on our machines. Konrad pointed that out in #801 (comment), back when the test lived in #801.

This moves the class to tests/bundle/RepositoryInstaller/Migration/Upgrade/ with its own phpunit-migrations.xml, which composer integration runs after the two integration suites. Every integration job already calls that script, so CI runs the test on SQLite, PostgreSQL and MySQL, on both install paths, without touching backend-ci.yaml. It keeps its raw-connection base class, so the database comes from DATABASE, which those jobs already set, and it runs last, so rebuilding a pre-6.0 schema in testdb can't affect the other suites. The unit suite excludes the folder.

The setup needed a few changes to work outside SQLite: DEFAULT FALSE instead of DEFAULT 0 for booleans, dropping the translation tables before importing the pre-6.0 schema (MySQL won't drop the tables they reference, and a pre-6.0 database doesn't have them anyway), and dropIndex() instead of a bare DROP INDEX.

I'm not sure about this form yet, so opinions welcome :) What we considered:

  • A step in the PostgreSQL and MySQL jobs of backend-ci.yaml, running just this class with DATABASE set. Same coverage, but the workflow would name a test file, and decide what runs on which engine next to the phpunit*.xml files.
  • Moving it into the integration-legacy suite, which already runs on every engine. That's the setup we want to migrate away from, so I'd rather not add to it.
  • A test entry in phpunit-integration.xml. The bootstrapper installs the 6.0 schema once, into the database every kernel test after it uses, so rebuilding a pre-6.0 schema there would break them, and a separate database isn't possible with CI's MySQL user.
  • A kernel test running the real Doctrine Migrations runner. The most realistic, but DAMA's per-test transaction doesn't survive MySQL's autocommitting DDL, and every test would have to tear the bootstrapped 6.0 schema back down.
  • This one, its own config. Also a home for later upgrade tests that need real engines. The price is a third config, and a few more seconds of composer integration locally.

For QA:

No QA needed. To run it against a database:

DATABASE="pgsql://postgres:postgres@localhost:5432/testdb" vendor/bin/phpunit -c phpunit-migrations.xml

…in tables

Adds ibexa_content_translation and ibexa_content_version_translation
(and ibexa_url_alias_ml_translation, which the same migration creates)
and an always_available column on ibexa_content and
ibexa_content_version, backfills them from the language masks, and
switches the content and location gateways, the content mapper and the
Filter to them. The content gateway keeps writing the masks and the
always-available bit of the field and name language ids, which the
remaining readers still use until the bitmask is removed.

Adds the ibexa:languages:backfill-translations and
ibexa:languages:verify-translations commands, and the new columns and
rows to cleandata.sql. FixtureImporter fills the join tables for
fixtures that still carry masks, and creates its SchemaAssetsFilterBypass
itself when it isn't given one.
…join table code

IBX-13070 removed SchemaAssetsFilterBypass together with the filter it
lifted, so the language translation commands and FixtureImporter
introspect the connection directly again.
ibexa_search_object_word_link gets language_id and
is_main_and_always_available columns, filled from its mask. The field
criteria and sort clause, the full-text and language code criteria and
the language filter of content and location search use the join tables
and those columns instead of arithmetic on the masks. The indexer keeps
writing the word link mask until the bitmask is removed, and field
language ids are compared without their always-available bit until it is
stripped.
ReindexCommand passes the --content-ids values as strings, and the
untyped remove() hands them on to SearchIndex's int parameters, which
strict types turned into a TypeError for deleted or unpublished content.
ibexa_url_alias_ml gets an is_always_available column, and an alias's
languages come from ibexa_url_alias_ml_translation instead of lang_mask,
in cleandata.sql too. The gateway keeps lang_mask in step with them
until the bitmask is removed.
Content types read their languages from ibexa_content_type_name instead
of decoding the type's language_mask, and resolve language ids through
the language handler instead of MaskGenerator. They keep writing the
mask and the always-available bit of their name rows until the bitmask
is removed.
Object states resolve language ids through the language handler instead
of MaskGenerator. They keep writing the masks and the always-available
bit until the bitmask is removed.
Stops writing the masks and drops them, together with MaskGenerator. New
languages get the id after the highest one instead of the next power of
two, which lifts the limit of 62 languages, and the always-available bit
is stripped from the language_id columns, after removing the stale
object state translations eZ Publish left behind.
…en't a language, and from core's test data itself
…bitmask upgrade test

IBX-13070 removed SchemaAssetsFilterBypass, and LegacySchemaImporter no
longer takes one.
@Steveb-p
Steveb-p force-pushed the feature/language-bitmask-upgrade-test-engines-6.0 branch from e38fec2 to abe16fc Compare November 17, 2026 11:19
@Steveb-p
Steveb-p force-pushed the feature/language-bitmask-6-remove-6.0 branch from 3cbeb68 to 153157f Compare November 17, 2026 11:19
The names and fields of a version only matched its languages with the
always-available bit added too, which the language ids don't carry
anymore.
…SQL too

Moved it to its own PHPUnit config, run by "composer integration" after the two integration suites, and made its setup portable.
@Steveb-p
Steveb-p force-pushed the feature/language-bitmask-upgrade-test-engines-6.0 branch from abe16fc to 8682b71 Compare November 17, 2026 11:50
@sonarqubecloud

Copy link
Copy Markdown

Quality Gate Failed Quality Gate failed

Failed conditions
8.6% Duplication on New Code (required ≤ 3%)

See analysis details on SonarQube Cloud

@Steveb-p
Steveb-p force-pushed the feature/language-bitmask-6-remove-6.0 branch 5 times, most recently from bcd1688 to 7f0bb8d Compare November 21, 2026 16:16
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant