Skip to content

BREAKING CHANGE: prepare the basic Java17 env (1/3) - #3261

Merged
imbajin merged 26 commits into
apache:masterfrom
hugegraph:task/tp381-1-java17-foundation
Oct 5, 2026
Merged

imbajin merged 26 commits into
apache:masterfrom
hugegraph:task/tp381-1-java17-foundation

Conversation

@contrueCT

@contrueCT contrueCT commented Oct 3, 2026 •

Copy link
Copy Markdown
Contributor

Purpose of the PR

Part 1 of a three-PR upgrade series targeting apache/hugegraph:master. Establish a Java 17 build and runtime baseline while retaining TinkerPop 3.5.1.

Merge order: apache#3261 (Java 17 foundation) → apache#3262 (query semantics) → apache#3263 (TinkerPop upgrade). All three PRs target master directly. This PR is the prerequisite for the query-semantics and TinkerPop-upgrade PRs.

Source: hugegraph/hugegraph#231, submitted directly from hugegraph:task/tp381-1-java17-foundation. The local validation statements below are carried over from that source PR; CI on this ASF PR head remains the merge gate.

Stage overview

Build and run Server, PD and Store on Java 17 while publishing Commons/RPC as Java 11 bytecode. Retain TinkerPop 3.5.1, use the Groovy 2.5.23 compatibility pin, and preserve valid old graph-ID reservations.

Stage 1: Java 17 foundation and Java 11 library bytecode

Relationship to other work

  • This three-part series is the split delivery path for the modernization tracked in apache#3117 and issue #3069. BREAKING CHANGE(server): upgrade Java17 + TP3.7 + Groovy4 #3117's current code also targets TinkerPop 3.8.1; the series is intended to replace that monolithic delivery, rather than add another independent upgrade.

  • Generic label/ID/SEARCH condition resolution and selective predicate pushdown remain with apache#2994, issue #3201 and candidate-index coverage in apache#3243. This series retains whole-step local filtering and transaction/count fixes; it adds no generalized partial extraction or optimizer-time index-coverage heuristic. Existing special handling around match() and connective label filters is inherited from master.

  • community #260 relocates shared query/model types, and community #261 changes transaction lifecycle. Their overlapping engine files need integration coordination; those migrations are not folded into this series.

  • Single-ID query fast paths (apache#3175, apache#2859) and adjacency-query optimization (apache#2864) remain separate performance work. Cypher parameter binding (community #238) and error mapping (apache#3259, apache#3241) remain separate behavior changes; part 3 only adapts upgraded transport APIs and result normalization.

  • The Java 17 launcher checks also cover the older Java 11 startup-check goal in apache#2846. Configurable distribution paths (apache#3253) and restart diagnostics (apache#3258) touch the same scripts and remain separate integration work.

Main Changes

  • Publish hugegraph-common and hugegraph-rpc as Java 11 bytecode (previously Java 8), preserving Java 11 consumer compatibility. Build and service runtimes use Java 17.
  • Replace the inherited Ivy versions with 2.6.0 and synchronize release licensing.
  • Align compiler, test plugins, JVM options, launchers, Docker images and cluster-test process handling on Java 17. Use Groovy 2.5.23 as a temporary compatibility pin: 2.5.14 cannot read Java 17 classes. Part 3 replaces it with Groovy 4.
  • Carry the GraphId allocation/repair work already reviewed in #163, together with the Java 17 work from #183 and #194.
  • Declare Commons Configuration 2.10.1 and Commons Text 1.11.0 in the release LICENSE, matching the dependency inventory.
  • The current community master already contains upstream apache#3157 and apache#3220; these are baseline changes and are excluded from this PR diff.

Verifying these changes

  • Fresh-cache Grape resolution and class loading passed through Gremlin Groovy on both Groovy API lines; the assembled foundation distribution contains Ivy 2.6.0.
  • Commons unit validation on this host is limited by existing MAC-address tests: the network interface has no hardware address. A clean Java 17 bytecode baseline reproduces the same errors; the library compatibility smoke and RPC suite pass on Java 11.
  • Already covered by existing tests: Java 17 launcher contracts, process-wait helpers, backend/serializer registration, auth reflection, and ordered scans under the Gremlin sandbox.
  • Local validation: Java 17 formatting and all-module clean compile passed. A local HTTP fixture verified that the actual PD startup script waits through 503/401 responses, accepts readiness HTTP 200, and fails on persistent unauthorized responses.
  • CI results remain the merge gate. The Groovy compatibility pin is the new integration adjustment requiring attention beyond the previously reviewed inputs.

Does this PR potentially affect the following parts?

  • Dependencies
  • Modify configurations
  • The public API
  • Other affects: minimum build/runtime JDK and cluster test lifecycle
  • Nope

Documentation Status

Repository documentation is included in docs/BUILDING.md, the README and PD/Store deployment guides. License declarations and the dependency inventory are updated for the Groovy compatibility pin.

Review follow-up

  • Graph ID safety keeps valid old IDs reserved when mappings change. Performance optimization and physical-prefix protection for explicit repair of legacy out-of-range IDs remain follow-ups; normal access rejects invalid mappings.

  • Quote complete JVM argument-file paths and the process test build directory, so checkouts containing spaces can launch test JVMs. RPC test resources use decoded file URIs; the local Commons build script explicitly runs its tests.

  • Synchronized the existing organization branch with the latest master, preserving the reviewed RocksDB/JRaft runtime changes.

  • Kept GraphId controller tests in the Node module with Jupiter; both controller tests executed and passed locally.

  • Isolated temporary RPC test listeners on random ports and connected their clients to the actual bound ports. The RPC server/client regression suite passed with isolated listeners.

  • Java 17 formatting and all-module clean compile passed. Commons/RPC JARs contain Java 11 bytecode and were loaded successfully on Java 11. The branch inherits the independently merged upstream CI runtime update; this follow-up makes no additional gate/workflow activation changes.

  • Synchronized the concurrent Store scan test so the slow source starts before the failing source can cancel it. The ordered-iterator regression suite passed with deterministic failure ordering.

GraphId allocation sample

Measured the shared GraphIdManager implementation shared by all three layers on Java 17, Apple M5 / 24 GiB, and RocksDB 8.10.2. Each sample used a fresh real RocksDB database and one partition, with logging disabled. Allocation timing includes the existing synchronous metadata flush; these timings do not isolate the additional mapping scan cost. One sample was run per case.

Graphs Workers Mapping rows read/decoded Total time Allocation p95 Lock wait p95 / max
1,000 1 499,500 13.735 s 23.886 ms <0.001 / 0.001 ms
10,000 1 49,995,000 157.315 s 23.223 ms <0.001 / 0.006 ms
10,000 4 49,995,000 116.328 s 99.000 ms 86.040 / 6,628.927 ms

The measured mapping-row totals match N(N-1)/2. The concurrent sample confirms global-lock contention during graph allocation. The authoritative mapping check is retained; production-scale performance acceptance and any migration/allocator redesign remain separate work.

CI on the updated PR head remains a separate verification step.

@codecov

codecov Bot commented Oct 3, 2026 •

Copy link
Copy Markdown

Codecov Report

❌ Patch coverage is 74.82014% with 35 lines in your changes missing coverage. Please review.
✅ Project coverage is 41.70%. Comparing base (0a3e4ae) to head (804a0e7).

Files with missing lines Patch % Lines
...ain/java/org/apache/hugegraph/util/Reflection.java 20.00% 16 Missing ⚠️
...rg/apache/hugegraph/store/meta/GraphIdManager.java 85.96% 8 Missing and 8 partials ⚠️
...ugegraph/backend/serializer/SerializerFactory.java 33.33% 1 Missing and 1 partial ⚠️
...ugegraph/backend/store/BackendProviderFactory.java 50.00% 0 Missing and 1 partial ⚠️
Additional details and impacted files
@@             Coverage Diff              @@
##             master    #3261      +/-   ##
============================================
+ Coverage     41.60%   41.70%   +0.10%     
- Complexity     7320     7355      +35     
============================================
  Files           794      794              
  Lines         69127    69229     +102     
  Branches       9258     9268      +10     
============================================
+ Hits          28761    28873     +112     
+ Misses        37091    37077      -14     
- Partials       3275     3279       +4     

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

@bitflicker64 bitflicker64 left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Blocking: yes. Summary: The Java 17 move also changes the published hugegraph-common and hugegraph-rpc jars from Java 8 bytecode to Java 17 bytecode, which downstream projects that still target Java 8 and 11 cannot load. The cluster-test changes in this PR are also not exercised by any CI run on this head, because the Cluster Test CI workflow is disabled in the repository. Evidence: Built hugegraph-common at bdce410 with JDK 17 and ran javap on a compiled class (major version 61; master compiles this module with source/target 1.8). gh api actions/workflows shows Cluster Test CI as disabled_manually and no run exists for this head. The three new minicluster unit tests pass locally on JDK 17.

Comment thread hugegraph-commons/pom.xml
- name: Run simple cluster test
run: |
mvn test -pl hugegraph-cluster-test/hugegraph-clustertest-test -am -P simple-cluster-test
timeout 45m mvn test -pl hugegraph-cluster-test/hugegraph-clustertest-test \

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Important: this PR changes the cluster-test lifecycle (AbstractEnv start timeouts, ServerNodeWrapper port readiness, Gremlin port rewriting, pd.peers in the template) and adds EnvUtilTest, GremlinServerConfigTest and AbstractNodeWrapperTest, but none of it ran on this head. gh api repos/apache/hugegraph/actions/workflows reports Cluster Test CI as disabled_manually, and its last run was in October 2025. No other active workflow builds hugegraph-cluster-test with tests. The PR body says CI on this head is the merge gate, so the lane that covers the "cluster test foundation" part of the title is missing from that gate. I ran mvn test -pl hugegraph-cluster-test/hugegraph-clustertest-minicluster locally on JDK 17 and the three new unit tests pass. The simple and multi cluster suites, which depend on the new readiness logic, are still unverified. Please get Cluster Test CI re-enabled and run against this head, or link a passing run of both cluster profiles on this exact SHA, before merge.

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Rechecked the upstream workflow state: Cluster Test CI is still disabled_manually. The workflow file has simple/multi cluster commands, but that does not establish that either profile ran on this PR head. Keeping this open for the separate CI work; local compilation/unit tests and package validation are not substitutes for those cluster runs.

Comment thread hugegraph-server/hugegraph-test/conf/jvm-test-module.options Outdated
- remove Commons from required checks
- require the Java 17 memory Server check
- retain analysis and license checks
- avoid claiming TinkerPop 3.8.1 in the foundation
- describe shared TinkerPop and Kryo access needs
- retain the test and runtime permission boundary
- update both Store dependencies to Fastjson 1.2.84
- align the release license and dependency inventory
- retain the Fastjson 1.x dependency contract
- override the legacy client transitive dependency
- retain the existing client and production versions
- remove obsolete release inventory entries

@bitflicker64 bitflicker64 left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Blocking: yes. Summary: The new FixGraphIdControllerTest never runs in CI, so the controller-level validation that this PR adds to /update_graph_id has no test that runs. Separately, the .asf.yaml change drops the Commons build from the required status checks instead of moving it to Java 17. The two open findings from bitflicker64 at this head (Java 17 bytecode in the published hugegraph-common and hugegraph-rpc jars, and the disabled Cluster Test CI workflow) still apply and are not repeated here. Evidence: store job log for run 37172218875 at 7e6da14 (every surefire:test (default-test) @ hg-store-node reports Tests run: 0 under the auto-detected JUnitPlatformProvider), hugegraph-store/hg-store-test/pom.xml store-core-test includes, and git diff 176fb56d..7e6da142 -- .asf.yaml.

Comment thread .asf.yaml Outdated
imbajin and others added 5 commits October 4, 2026 16:11
Move existing controller cases into the Store test module.
Select the controller class in the core test profile.
Retain the existing validation assertions.
Use the existing Node JUnit Platform provider.
Keep controller tests alongside the Node implementation.
Remove the cross-module core test include.
@imbajin imbajin added this to the 1.8.0 milestone Oct 4, 2026
imbajin and others added 6 commits October 4, 2026 23:01
Adopt the Java-independent memory test gate.
Use the shared runtime definition with the Java 17 baseline.
Preserve foundation validation and platform contracts.
Set the Commons and RPC compiler release to 11.
Keep Java 17 as the build and service runtime baseline.
Document the published library compatibility contract.
Manage Ivy consistently across the upgrade series.
Update bundled dependency and licensing metadata.
Verify Gremlin and fresh Grape class loading.
- Keep one active run per workflow and PR
- Preserve push and manual workflow executions
- Cover CI workflows on this PR target branch
- incorporate the upstream Helm deployment chart
- retain the foundation build and dependency changes
- preserve existing Java sources and POM files
- merge the remote PR concurrency updates
- keep the upstream master synchronization
- preserve the validated Java and POM content
- Cover the newly added Helm workflow
- Keep the latest Helm checks per PR
- Preserve concurrent branch changes
Quote JVM option files and the process build directory.
Load RPC test configuration through a decoded file URI.
Enable Commons tests in the local build script.
Preserve the independently added Helm concurrency rule.
Keep validated test path and resource fixes.
Retain a normal fast-forward push history.
@imbajin imbajin changed the title refactor(build): prepare Java 17 and cluster test foundation (1/3) BREAKING CHANGE: prepare the basic Java17 env (1/3) Oct 5, 2026
@imbajin
imbajin merged commit 39a3a8d into apache:master Oct 5, 2026
32 of 34 checks passed
byteayan added a commit to byteayan/hugegraph that referenced this pull request Oct 5, 2026
Resolve the hugegraph-server.sh conflict with the Java 17 baseline (apache#3261): keep the crash-file defaults in JAVA_TOOL_OPTIONS, drop the removed JAVA_VERSION > 9 --add-exports block (now in jvm-module.options), and report the new missing module options error through report_error.
@imbajin
imbajin deleted the task/tp381-1-java17-foundation branch October 8, 2026 16:21
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants