Skip to content

refactor(server): upgrade TinkerPop to 3.8.1 with compatibility gates (3/3) - #3263

Merged
imbajin merged 14 commits into
apache:masterfrom
hugegraph:task/tp381-3-upgrade-validation
Oct 6, 2026
Merged

imbajin merged 14 commits into
apache:masterfrom
hugegraph:task/tp381-3-upgrade-validation

Conversation

@contrueCT

@contrueCT contrueCT commented Oct 3, 2026 •

Copy link
Copy Markdown
Contributor

Purpose of the PR

Upgrade Server and distributed modules from TinkerPop 3.5.1 / Groovy 2.5.23 to TinkerPop 3.8.1 / Groovy 4.0.25, with executable compatibility gates. Parts #3261 and #3262 are merged; this PR contains the final runtime-upgrade stage.

The source branch also updates hugegraph/hugegraph#233. This split series advances #3117 and #3069. The shared-type consolidation in #3270 follows this PR and must carry its DATE normalization into Struct.

Stage 3: TinkerPop 3.8.1, typed transport and safe YAML

Main Changes

  • Keep Gremlin Groovy and the existing HTTP JSON path. Default Driver/Console GraphBinary preserves Vertex/Edge/Path/UUID types and returns Schema as standard maps and Blob as binary values, including inside real elements and nested results. Default HTTP/message V1 Tree retains its legacy key/value arrays and accepts scalar keys through the scoped message registry; direct GraphSON IO and typed GraphSON keep their standard version-specific behavior.
  • Adapt predicate/step APIs and Cypher results. Local String ID not/and/or filters now match their separate-filter equivalents. Provider conversion preserves caller-owned named GValue bindings across reuse and pre-execution clones; updating a caller predicate applies to newly built traversals.
  • Honor explicit named limit/range bounds during count optimization. For example, inject(1,2,3).limit(GValue.of("size",3L)).count().is(P.gt(0L)) returns [3], matching limit(3). DATE schema accepts OffsetDateTime writes and indexed equality/range inputs by normalizing the instant to the existing millisecond Date representation.
  • Coordinate SnakeYAML 2.2, Boot 2.5.15, Spring 5.3.27 and Jackson YAML 2.15.2. Safe data-only readers remain; Fabric8 can read a real kubeconfig with the upgraded YAML runtime. Align Store's Log4j binding and CLI logging to 2.18.0. Align Netty through the 4.1.137.Final BOM.
  • Gate actual Structure/Process/provider execution for Memory, RocksDB and HStore. Keep release LICENSE, NOTICE, component license files and dependency inventory together in one dedicated commit.

Review fixes (2026-10-06)

  • Preserve full counts for without(-1) and without(-2, -1) by declining non-positive count bounds. Regression results cover empty/nonempty graphs, subsequent gt(0) and nested adjacency filters; positive exclusion bounds retain the existing +1 behavior.
  • Adapt the four Cypher extension predicate factories through the existing Translator extension point and PBiPredicate, delegating to the original type/regex predicates. Real HTTP tests execute computed-expression regex filters and verify whole-string matching.
  • Serialize Groovy GString values as standard strings over GraphBinary, including nested lists/maps. Both bundled Driver configurations and a standard GraphBinary reader are covered; untyped GraphSON retains the legacy bean shape. The migration guide documents this output contract.

Java 17 formatting, all-module clean compile and Server distribution packaging passed. All 30 focused unit tests and all 86 count tests on each of Memory and RocksDB passed with no skips. Against the assembled RocksDB Server with Basic authentication, all 11 Cypher API tests and the actual Console/Driver smoke passed, including top-level and nested interpolated script results. The temporary Server was stopped after validation. Full current-head CI remains separate acceptance evidence.

Verifying these changes

Local repair validation covers an all-module clean compile, the Server unit suite, real Memory/RocksDB query regressions, a real Gremlin service using both bundled GraphBinary client configurations and default HTTP JSON, production DATE schema/index behavior, and real Fabric8 ConfigBuilder/YAML calls. The final assembled package passes the same network checks using only packaged jars, and its resolved runtime dependencies match the release legal inventory.

Full API, Structure/Process, PD/Store/HStore and dependency/security CI must pass on the current PR head before merging. A skipped or empty TinkerPop report is insufficient. The no-network kubeconfig regression establishes the dependency call path; it does not claim a complete Kubernetes cluster deployment or an external client/authentication matrix.

Does this PR potentially affect the following parts?

  • Dependencies
  • Modify configurations
  • The public API
  • Other affects: client serialization and script compatibility
  • Nope

Documentation Status

@codecov

codecov Bot commented Oct 3, 2026 •

Copy link
Copy Markdown

Codecov Report

❌ Patch coverage is 26.72176% with 266 lines in your changes missing coverage. Please review.
✅ Project coverage is 41.03%. Comparing base (662a97d) to head (2ca1f25).

Files with missing lines Patch % Lines
...he/hugegraph/traversal/optimize/TraversalUtil.java 23.85% 73 Missing and 10 partials ⚠️
...aph/io/HugeGraphTypeSerializerRegistryBuilder.java 20.48% 56 Missing and 10 partials ⚠️
...ugegraph/traversal/optimize/HugeCountStrategy.java 11.36% 39 Missing ⚠️
.../org/apache/hugegraph/api/cypher/CypherClient.java 34.61% 25 Missing and 9 partials ⚠️
...va/org/apache/hugegraph/io/HugeGraphSONModule.java 28.94% 27 Missing ⚠️
.../hugegraph/opencypher/CypherGremlinPredicates.java 45.45% 6 Missing ⚠️
.../org/apache/hugegraph/opencypher/CypherPlugin.java 0.00% 4 Missing ⚠️
...aph/traversal/optimize/HugePrimaryKeyStrategy.java 62.50% 1 Missing and 2 partials ⚠️
...a/org/apache/hugegraph/structure/HugeFeatures.java 0.00% 2 Missing ⚠️
.../org/apache/hugegraph/backend/query/Condition.java 50.00% 1 Missing ⚠️
... and 1 more
Additional details and impacted files
@@             Coverage Diff              @@
##             master    #3263      +/-   ##
============================================
+ Coverage     36.41%   41.03%   +4.61%     
- Complexity     6262     7239     +977     
============================================
  Files           774      788      +14     
  Lines         67843    68451     +608     
  Branches       9064     9256     +192     
============================================
+ Hits          24706    28087    +3381     
+ Misses        40312    37090    -3222     
- Partials       2825     3274     +449     

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

contrueCT and others added 12 commits October 6, 2026 21:29
- Upgrade TinkerPop and Groovy with aligned runtime dependencies
- Configure the default and Raft Gremlin servers for the upgraded serializers
- Update version checks and retain Groovy script compatibility coverage

Co-authored-by: imbajin <jin@apache.org>
- Adapt graph steps and count strategies to the upgraded traversal APIs
- Align predicate handling across core and shared condition types
- Update feature declarations and related graph and condition tests
- Adapt Cypher client and processor integration to the upgraded Driver APIs
- Update GraphSON modules and type serializer registration
- Retain Cypher and JSON serialization compatibility checks
- Run feature scenarios with isolated graph provider context and lifecycle
- Validate compatibility reports and retain API metrics coverage
- Integrate suite selection into Server and Store CI and cancel superseded runs
- Keep lifecycle RocksDB data and WAL within the test temporary directory

Co-authored-by: imbajin <jin@apache.org>
- Keep unsupported type predicates in local traversal filters
- Preserve native and UUID ID comparisons during local evaluation
- Add core and optimizer coverage for ID and count predicate behavior

Co-authored-by: contrueCT <contrue_ct@outlook.com>
- Align PD and Store YAML dependencies with the secured runtime
- Adapt Kubernetes and Store configuration readers to safe YAML construction
- Cover resource quotas and PD listener configuration and register unit tests

Co-authored-by: contrueCT <contrue_ct@outlook.com>
- Align bundled Driver and remote serializers with the upgraded Gremlin runtime
- Preserve typed result objects in Driver configurations
- Cover serializer configuration and secured YAML loading

Co-authored-by: contrueCT <contrue_ct@outlook.com>
- Explain supported TinkerPop behavior and concrete upgrade checks
- Clarify list processing and typed GraphSON configuration guidance
- Align build instructions and project version references

Co-authored-by: contrueCT <contrue_ct@outlook.com>
- Update release LICENSE and NOTICE for the upgraded runtime
- Consolidate third-party license additions, replacements and removals
- Align the packaged dependency inventory with the resolved runtime

Co-authored-by: imbajin <jin@apache.org>
- Restore finite GraphBinary results and legacy HTTP Tree output
- Preserve named bindings and local ID/count filter semantics
- Normalize DATE values and align YAML/logging dependencies
- Cover real regressions and paired upgrade documentation
- Guard current-head retries and retain HStore failure evidence
- remove unsafe typed GraphSON V1 from shipped configs
- preserve mixed edge ID membership and count semantics
- encode public enums and SchemaManager with standard wire values
- share compatible Cypher factories across imports and replay
- align Jackson modules and release dependency notices
- encode Shard results as standard GraphBinary maps
- mirror OffsetDateTime conversion in the shared graph model
- regenerate the exact runtime dependency inventory
- retain TP report gates in the new reusable CI layout
@imbajin
imbajin force-pushed the task/tp381-3-upgrade-validation branch from 0d495f8 to a3bb1d4 Compare October 6, 2026 13:48
- limit the exception to ASF PR 3263 and its source branch
- keep unit, core and API tests with explicit TP skip summaries
- mark the temporary branch patch with a removal TODO
@imbajin
imbajin force-pushed the task/tp381-3-upgrade-validation branch from a3bb1d4 to 2ca1f25 Compare October 6, 2026 13:51
@imbajin
imbajin merged commit d9abcd4 into apache:master Oct 6, 2026
38 of 45 checks passed
@imbajin
imbajin deleted the task/tp381-3-upgrade-validation branch October 8, 2026 16:21
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants