[johnpaulalex]

Description of changes

This PR resolves several CIME build system and Fortran interface issues when building CESM/CTSM with libraries/FTorch across intel and gnu compilers and across USE_FTORCH=TRUE and USE_FTORCH=FALSE:

1. Mock out full set of ftorch methods when USE_FTORCH=FALSE (FTorch_cesm_interface.F90):

   • All torch_tensor_from_array variants (including with and without the 'layout' params) and torch_delete.
   • Adds optional arguments (device_index, requires_grad, is_training) matching upstream ftorch and initializes torch_kCPU = 0 as a parameter constant.

2. buildlib usability improvements (buildlib):

   • Gates the TORCH_DIR check (torch_dir and os.path.isdir(torch_dir)) behind if case.get_value("USE_FTORCH"):. Fixes the root cause of the TORCH_DIR=None TypeError originally reported in ESCOMP/CTSM#3214.
   • Removes the fallback WGET download of unpinned libtorch-shared-with-deps-latest.zip nightlies during case.build, failing fast via expect(...) with an actionable error message (showing how to point to a pre-existing copy or download one).
   • Runs child FTMakefile unconditionally in buildlib (removing if not os.path.isfile(os.path.join(bldroot,"Makefile")):), ensuring CMake reconfigures properly if a user flips USE_FTORCH and rebuilds an existing case (or when create_test cases share a SHAREDLIBROOT).

3. Add GNU Support (CMakeLists.txt):

   • Places ftorch.mod in ${CMAKE_BINARY_DIR}/modules (reverting the external/ftorch override from v0.0.4 to align with upstream Cambridge-ICCS/FTorch's default output directory) to match INCLDIR += -I$(FTORCH_DIR)/modules in cime/CIME/Tools/Makefile. This makes GNU (gfortran) builds work (existing-but-not-mainline CAM branches were only tested with Intel ifx, which embeds transitive use ftorch symbols inside ftorch_cesm_interface.mod, whereas gfortran requires ftorch.mod itself to be on -I when compiling any caller of FTorch_cesm_interface).

4. Enable libtorch.so shared linking (CMakeLists.txt):

   • LibTorch is only distributed as dynamic shared libraries (libtorch.so), but our Cray cmake configs disable shared libs (they used to be unsupported).
   • Overrides the Cray settings with TARGET_SUPPORTS_SHARED_LIBS TRUE, CMAKE_SHARED_LIBRARY_SUFFIX ".so", CMAKE_FIND_LIBRARY_SUFFIXES ".so" ".a", and -Wl,-soname across both top-level and nested FTorch builds without modifying CIME or upstream FTorch.

Related issues / PRs

• Addresses ESCOMP/FTorch_interface#1
• Supports ESCOMP/CTSM#3228 (and fixes the buildlib root cause of ESCOMP/CTSM#3214) by enabling feat(ftorch): finish CTSM FTorch integration: add system test calling FTorch inference CTSM#4265.

Testing performed

• Verified all $2 \times 2$ (USE_FTORCH=TRUE/FALSE, use_ftorch_demo=.true./.false.) CTSM system test permutations on NCAR Derecho across intel (multi-rank Cray MPI) and gnu (mpi-serial), plus 3 system tests inside a full top-level CESM checkout (/glade/work/jpalex/src/cesm, $SRCROOT = CESM root): cesm_ftorch_test1 (USE_FTORCH=TRUE, intel MPI, use_ftorch_demo=.true.), cesm_ftorch_false1 (USE_FTORCH=FALSE, intel MPI), and cesm_ftorch_gnu1 (USE_FTORCH=FALSE, gnu mpi-serial).
• Verified all 3 existing-but-not-mainline CAM FTorch_cesm_interface caller patterns (jedwards4b/PUMAS@1nn_tau_emulator, leap-stc/Integration_team@CAM_Convection_YOG, and m2lines/clubb_ML@clubb_4ncar_with_ml with torch_delete(model)) plus layout-free FTorch v1.1.0 calls compile and link cleanly across (intel, gnu) $\times$ (USE_FTORCH=TRUE, FALSE).
• Validated end-to-end CTSM system test (T15) on NCAR Derecho with this PR + feat(ftorch): finish CTSM FTorch integration: add system test calling FTorch inference CTSM#4265: SMS_D_Ld1.f10_f10_mg37.I2000Clm60Sp.derecho_intel.clm-default--clm-nofireemis--clm-ftorch_demo passed. Runtime log lnd.log verified: FTORCH INFERENCE SUCCESSFUL: output = 1.00000000000000.

Google Gemini Argon 4 was used to write the code and tests, followed by human-guided verification.

[johnpaulalex]

@ekluzek here's half of the CTSM Ftorch support (the other is CTSM #4265). fyi @jedwards4b @briandobbins

[wwieder]

Marking as next to assign reviewers, milestone, priorities, etc.

[wwieder]

I can't mark as next, I'll leave this up to CSEG to evaluate.