Repository navigation
[OPENJPA-3014] Back a named sequence generator by a table, and use real sequences on MariaDB - #217
Merged
Merged
Conversation
…atabase has none A named @SequenceGenerator failed outright on every database whose dictionary defines no NextSequenceQuery. AnnotationPersistenceMetaDataParser sets the plugin of such a generator to SequenceMetaData.IMPL_NATIVE unless the sequence name is itself a plugin string, JDBCSeqValue maps that alias to NativeJDBCSeq, and NativeJDBCSeq.endConfiguration() refuses to configure without a NextSequenceQuery: ArgumentException: Error instantiating named sequence "SEQGENERATOR": Your database dictionary does not support native sequences openjpa.Sequence cannot help, because SequenceMetaData.instantiate() consults the configured system sequence only for the system sequence name, so a named generator always takes the plugin path. There was no property that avoided the failure. The report names MySQL, found while running the TCK, but the dictionaries without a NextSequenceQuery include Apache Derby, so the failure also reproduces on the database the default test profile uses; the new test fails on Derby, H2 excepted, before this change. They also include AbstractSQLServerDictionary, SQLServerDictionary, SybaseDictionary, MaxDBDictionary, SolidDBDictionary, HerdDBDictionary, AccessDictionary and InterbaseDictionary, and SQL Server has had sequence objects since 2012, so there the fallback stands in for a sequence the database could have provided. The migration notes name those databases and say that configuring NextSequenceQuery is how a generator keeps drawing from a sequence that already exists, with a pointer from the SQL Server section. Jakarta Persistence 3.1 section 11.2.4 and the GeneratedValue strategies in section 11.1.21 define how a generator declaration is mapped, and GenerationType.SEQUENCE says "the persistence provider must assign primary keys for the entity using a database sequence", but the specification elsewhere leaves the realisation of a generator to the provider, which is why providers emulate sequences where a database has none. The observable contract is that identifiers are unique and that initialValue and allocationSize are honoured. Resolving a named generator to a table backed sequence therefore raises no conformance obstacle; it is a design decision with a compatibility dimension, not something the specification demands. SequenceMapping.instantiate() now substitutes a value table sequence when the plugin is a plain request for a native sequence and dict.nextSequenceQuery is null, gated on the dictionary's own property rather than on a dictionary class so that a user supplied NextSequenceQuery keeps the native path. The substitution sits where the sequence is resolved, so the runtime and the mapping tool agree - a patch proposed on the dev list changed only MappingTool.addSequenceComponents, which leaves a plain EntityManagerFactory failing and works by mutating shared metadata. Instead of mutating this SequenceMapping, which is shared, the fallback builds a separate metadata instance and instantiates that; the substitution is logged as a warning, once per generator, because the sequence instance is cached on the metadata. ValueTableJDBCSeq rather than TableJDBCSeq: ValueTableJDBCSeq keys its counter row by a name, so the row is keyed by the sequence name the generator asked for and entities sharing one generator share one counter, as they would with a real sequence. TableJDBCSeq keys the row per class and would give each entity an independent counter, which is neither the annotation's semantics nor what the native path does. The sequence name is qualified with the catalog and schema of the sequence, so that two sequences of the same name in different schemas stay distinct without the internal table having to live in a schema the generator meant for a sequence object. That schema is the one a sequence object would have been created in: the generator's own, or, when it declares none, the persistence unit default that SequenceMetaData.setSchema substitutes. The emulated counter is therefore scoped exactly as the sequence it stands in for, which the Javadoc, the migration notes and a test with a qualified generator now say. initialValue and allocationSize travel on the SequenceMetaData rather than in the plugin string and are copied onto the substitute, which the test asserts both on the resulting Seq and through the identifiers it hands out, by counting the statements that advance the generator's counter: one allocation per block of allocationSize, not one per identifier. The increment is deliberately not copied: TableJDBCSeq.setIncrement is a deprecated alias for setAllocate and, appended after Allocate, would overwrite the allocation size. openjpa.Compatibility=EmulateNativeSequences=false restores the hard failure. The default is the fallback, because the behaviour it replaces is an unconditional failure with no workaround, so nothing that works today changes; a schema built by an earlier release will simply gain the sequence table. A dictionary identified from the connection URL alone has not seen a connection when the generator is resolved, and some dictionaries only learn from the server version whether the database has sequences, so where the dictionary itself declares no NextSequenceQuery the gate asks the configuration for its connection factory, which installs the dictionary's connected configuration. Otherwise which sequence backs a generator would depend on whether anything else had touched the database first. The cheap check comes first, so a dictionary that sets NextSequenceQuery in its constructor - Oracle, DB2, PostgreSQL, H2, HSQL, Informix, Firebird, Ingres - is still resolved without a connection and an offline mapping tool run against such a configuration does not stall on a connect attempt. DBDictionary.isConnected() exposes the connected state. If that connect attempt fails, the generator fails to resolve, with the connect failure as the cause of a MetaDataException, rather than falling back. Guessing cannot help there - the table fallback advances its counter over a connection of its own, so nothing hands out an identifier without one either way - and it can harm: SequenceMetaData.getInstance caches the Seq, so the guess holds for the life of the factory, and a node of a clustered deployment whose JNDI lookup happened to fail would then hand out identifiers from a counter the other nodes do not use. That is the same divergence the sibling commit avoids by parsing the version prefix, and a log warning is not a sufficient signal for it. A missing driver class arriving as a LinkageError is wrapped alongside the runtime exceptions. The test asks the dictionary which kind of database it is on rather than assuming, because both kinds exist and the same observable contract has to hold on both. Whether a database has sequence objects is DBDictionary.nextSequenceQuery, which a dictionary may only set once it has seen the server, so the test connects it exactly as the fallback above does - DBDictionary.isConnected(), then JDBCConfiguration .getConnectionFactory() - before reading it; reading it off an unconnected dictionary would expect the fallback on a server that has sequences. The unique identifiers, the initialValue and the allocation count are asserted on both paths. Only which Seq implementation backs the generator, and that EmulateNativeSequences=false brings the old failure back, are specific to the fallback; the latter is skipped with a message naming the platform where the database has sequences. The allocation count is measured as the number of statements that advance the counter, which is the UPDATE of the counter row on the fallback and the dictionary's NextSequenceQuery - for instance SELECT NEXTVAL(OJPA3014_SEQ) - on the native path, with the CREATE SEQUENCE and the ALTER SEQUENCE ... INCREMENT BY excluded, since neither hands out a value. On a database that has sequence objects that test only re-asserts the native path: of the five cases, four hold on master there as well, and only the Derby half of the CI matrix sees the substitution end to end. The gate itself is therefore covered server free by the new TestSequenceMappingFallback in openjpa-jdbc, which drives SequenceMapping.instantiate() with a stub dictionary and a stub DataSource on every platform: a dictionary that learns of sequence support only in connectedConfiguration() reaches NativeJDBCSeq, one that never declares any reaches ValueTableJDBCSeq with the sequence name, initialValue and allocationSize carried over, one that declares NextSequenceQuery in its constructor is resolved without the data source being touched at all, a data source that cannot connect makes the resolution fail with its own SQLException reachable in the cause chain, and EmulateNativeSequences=false brings the historical failure back.
MariaDB 10.3 introduced sequence objects - CREATE SEQUENCE, NEXTVAL(), NEXT VALUE FOR - but MariaDBDictionary still declared no NextSequenceQuery, so every @SequenceGenerator was treated as unsupported. Set it in connectedConfiguration() once the server reports 10.3 or later, the way PostgresDictionary version gates storeCharsAsNumbers. Older servers keep the table sequence fallback. MariaDBDictionary.java:57 reads "public class MariaDBDictionary extends DBDictionary": the dictionary is a copy of the MySQL one, not a subclass of it, down to a private getMajorMinorVersions of its own at line 244. Nothing here is inherited, and MySQLDictionary is untouched. There is no setter for NextSequenceQuery and hence no flag recording that the user configured one, and this change does not invent one: this dictionary leaves the field null in its constructor and DBDictionary.endConfiguration() nulls an empty value out, so a value seen in connectedConfiguration() was configured explicitly and is left alone. A MariaDB 10.x server reports its version to a client that is not MariaDB aware with a replication compatibility prefix - "5.5.5-10.3.39-MariaDB" rather than "10.3.39-MariaDB". The prefix was dropped in 11.0, so an 11.x server reports "11.8.9-MariaDB-ubu2404" to any client; both forms were read off the handshake greeting of mariadb:10.3 and mariadb:11.8 to confirm it. Both forms reach this dictionary, but not by the same route: DBDictionaryFactory .newDBDictionary selects on getDatabaseProductName(), falls back to the URL protocol, and looks at getDatabaseProductVersion() only when the name maps to MySQL (lines 93-102). With mariadb-java-client the product name is already "MariaDB" and the version string plays no part in the selection; the prefixed form matters because MySQL Connector/J reports the product name as "MySQL", so for that client the version string is both what selects this dictionary and what carries the prefix. What the version parsing has to cope with is the prefix itself, because it stops at the first character that is neither a digit nor a dot: on the prefixed form it would read 5.5 and withhold sequence support from an 11.8 server reached through a driver that passes the handshake version through, while the same server reached through mariadb-java-client used a sequence object - two deployments of one application handing out identifiers from two independent counters. Strip the prefix before parsing. The strip moves every feature this dictionary gates on the version, not only sequence support. The pre-existing gate for 10.2 and later (maj > 10 || (maj == 10 && min > 1)) sets timestampTypeName="DATETIME{0}", moves it into fractionalTypeNameSet and sets dateFractionDigits=6, so a server reported as "5.5.5-10.2.44-MariaDB" now gets microsecond DATETIME columns where it was parsed as 5.5 and got whole-second DATETIME before - the same DDL the same server already got through mariadb-java-client. TestMariaDBDictionary pins both forms of a 10.2 version to DATETIME{0}/6 and a genuine 10.1 version to DATETIME/0, and the MariaDB section of the migration notes says that such a deployment now generates microsecond columns, that a schema from an earlier release keeps its whole-second ones, and that the gate assigns both fields outright, so only a subclass restores the previous types. Declaring sequence support means the schema tool now issues CREATE SEQUENCE and DROP SEQUENCE for these generators, which brings two more things with it: - MariaDB caches 1000 increments of a sequence per server by default and discards what is left of that range whenever the cache is flushed, which an ALTER SEQUENCE or a restart does. OpenJPA caches values itself, in blocks of the generator's allocation size, and NativeJDBCSeq issues ALTER SEQUENCE ... INCREMENT BY once per sequence instance to do so, which is itself such a flush, so the server side cache adds nothing but those gaps. getCreateSequenceSQL() therefore creates sequences with NOCACHE. Measured on MariaDB 11.8 against a sequence created START WITH 100 INCREMENT BY 7: SELECT NEXTVAL returns 100 and leaves NEXT_NOT_CACHED_VALUE at 7100, and after ALTER SEQUENCE ... INCREMENT BY 7 the next SELECT NEXTVAL returns 7100 - 1000 increments skipped, not 1000 values. With NOCACHE the same statements return 100 and then 107. The statements OpenJPA emits for a generator with initialValue 100 and allocationSize 7, as the SQL that TestNamedSequenceGenerator collects shows them on MariaDB 11.8, are CREATE SEQUENCE OJPA3014_SEQ START WITH 100 INCREMENT BY 7 NOCACHE ALTER SEQUENCE OJPA3014_SEQ INCREMENT BY 7 SELECT NEXTVAL(OJPA3014_SEQ) - unquoted, and with no CYCLE clause. The 7 in INCREMENT BY is increment * allocate, which DBDictionary.commonCreateAlterSequenceSQL multiplies out; NativeJDBCSeq leaves the increment at its default of 1, so for this generator it is the allocation size. In that order no jump occurs, the ALTER arriving before the first NEXTVAL has cached anything, and 15 identifiers cost three NEXTVALs; the jump is what a later EntityManagerFactory would see, issuing the ALTER against a sequence whose values have been cached, and that is the case NOCACHE covers. - Reflecting on the schema to find existing sequences needs a query, since DatabaseMetaData does not report them. MariaDB has no INFORMATION_SCHEMA.SEQUENCES before 10.6, but a sequence appears in INFORMATION_SCHEMA.TABLES with a TABLE_TYPE of SEQUENCE in every version that has sequences, so getSequencesSQL() reads it from there. The schema name is matched case insensitively because DBDictionary.getSequences() upper cases the schema it passes, while MariaDB keeps the case a schema was created with. INFORMATION_SCHEMA spans every database on the server, though, and every caller asks for a null schema - SchemaGenerator .generateSequences() passes DBIdentifier.NULL deliberately, and an unqualified @SequenceGenerator has none - so the unqualified case is scoped to DATABASE(). Unrestricted it would report the sequences of other applications' databases, which makes SchemaTool.buildSchema() skip a CREATE SEQUENCE it owes, because SchemaGroup.findSequence() matches any schema for an unqualified sequence, and makes a retain, refresh or dropDB action issue DROP SEQUENCE outside the configured database. DATABASE() takes no bind parameter, so the parameter indices getSequences() binds stay as they were. TestMariaDBDictionary drives connectedConfiguration() with a reflection proxy DatabaseMetaData, as TestPostgresDictionary does for OPENJPA-2971, so the version gate, the survival of an explicitly configured NextSequenceQuery, the generated temporal types, the NOCACHE suffix and the predicates and binding order of the sequence query are covered without a server: 18 tests, all passing. TestNamedSequenceGenerator passes on Derby and H2, the two profiles CI runs, and passed on MariaDB 10.2 and 11.8 and MySQL 8.0 when it was measured there, which is where the SQL quoted above was collected.
cristof
approved these changes
Oct 18, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
3 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Reported by @solomax from the MySQL TCK run: a named
@SequenceGeneratorfails with "Error instantiating named sequence … Your database dictionary does not support native sequences" wherever the dictionary leavesnextSequenceQuerynull. That is not MySQL-specific — it includes Derby, the default CI profile.openjpa.Sequencecannot help, becauseSequenceMetaData.instantiate()consults the configured system sequence only for the system sequence name, so a named generator always takes the plugin path.Two commits, deliberately separate (the first changes identity generation on databases people use, the second is a dictionary change):
SequenceMapping.instantiate()substitutes a table-backed sequence when the plugin resolves toNativeJDBCSeqwith no properties and the dictionary has nonextSequenceQuery, gated on the dictionary property rather than a dictionary class, building a separateSequenceMappingrather than mutating shared metadata, and warning once per generator. It resolves where the sequence is resolved, so the runtime and the mapping tool agree — theMappingToolpatch from the dev list fixes the tool while a plain EMF still fails.IMPL_VALUE_TABLE, notIMPL_TABLE:ValueTableJDBCSeqkeys its counter row by name, so entities sharing a generator share a counter as they would with a real sequence, whereTableJDBCSeqkeys per class and would give each entity its own.initialValue/allocationSizetravel on theSequenceMetaDataand are asserted to survive. NewEmulateNativeSequencescompatibility option, default true — what it replaces is an unconditional failure, so nothing that works today changes;falserestores the hard failure.MariaDBDictionary.connectedConfiguration()setsnextSequenceQueryfor server 10.3 and later, in the version-gated mannerPostgresDictionaryuses. There is no "explicitly set" flag for this property and I did not invent one; the gate isnextSequenceQuery == null, which is sound because the constructor leaves it null. Declaring sequence support pulled in two more things, both checked against a real server:NOCACHEonCREATE SEQUENCE, becauseALTER SEQUENCE … INCREMENT BYflushes MariaDB default 1000-value cache, and agetSequencesSQL()that readsINFORMATION_SCHEMA.TABLES WHERE TABLE_TYPE = SEQUENCE, sinceINFORMATION_SCHEMA.SEQUENCESdoes not exist before 10.6.Verified per server, sequence package and
TestNamedSequenceGenerator: Derby, H2, MySQL 8.0, MariaDB 10.2 and MariaDB 11.8 all green — 10.2 takes the fallback and 11.8 the native path, so the version gate is exercised both ways.TestMariaDBDictionary18/18. The query, kernel and jpql packages show identical failing class lists to pristine master on both CI profiles.Two rounds of review correction worth noting, since the evidence in the messages matters: the first version of the test asserted "expected a table backed sequence on MariaDB" and so failed 3 of 5 on 11.8 once commit 2 gave MariaDB sequences — the tests now ask the connected dictionary instead of assuming a platform; and the commit messages had quoted DDL OpenJPA never emits and claimed
MariaDBDictionary extends MySQLDictionary(it extendsDBDictionary) and that 11.x servers carry the5.5.5-prefix (dropped in 11.0 — read off the handshake of both servers). Those are corrected, and the one remaining synthetic test input is now labelled as such.