Pip Runtime Dependencies & CRAY AMD Support #314
Draft
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
…he build and runtime.
…e to track build-requirements.
…ting build and docs requirements;
…IR; System + compiler specific cache directories; Clean targets for pip cache and envs.
…p package dependencies.
…boost back to 1.74.0; rocmcc error mitigation for building older boost.
mdavis36
commented
Jan 17, 2025
Comment on lines
+37
to
+53
| #TODO: Remove this once blueos3 is no longer supported. | ||
| if("${SPHERAL_ENV_SYS_TYPE}" STREQUAL "blueos_3_ppc64le_ib_p9") | ||
| set(PIP_INSTALL_CMD env MPICC=${MPI_C_COMPILER} MPICXX=${MPI_CXX_COMPILER} CC=${CMAKE_C_COMPILER} CXX=${CMAKE_CXX_COMPILER} | ||
| python -m pip install | ||
| --disable-pip-version-check | ||
| ) | ||
|
|
||
| add_custom_command( | ||
| OUTPUT ${${target_name}_PREFIX}/.venv/${target_name}_stamp | ||
| COMMAND ${Python3_EXECUTABLE} -m venv --system-site-packages ${${target_name}_PREFIX}/.venv; | ||
| COMMAND . ${${target_name}_PREFIX}/.venv/bin/activate && | ||
|
|
||
| ${PIP_INSTALL_CMD} ${REQUIREMENTS_ARGS} | ||
|
|
||
| DEPENDS Python3::Python ${${target_name}_REQUIREMENTS} | ||
| ) | ||
| else() |
There was a problem hiding this comment.
blueos will not be supported as of this PR - Remove this
| @@ -237,6 +236,7 @@ function(spheral_add_pybind11_library package_name module_list_name) | |||
| USE_BLT ON | |||
| EXTRA_SOURCE ${${package_name}_SOURCES} | |||
| INSTALL OFF | |||
| VIRTUAL_ENV python_build_env | |||
There was a problem hiding this comment.
Need to make a Pyb11Generator PR to support this change.
There was a problem hiding this comment.
Merge changes to Develop then update submodule here.
Comment on lines
+15
to
+27
| - compiler: | ||
| spec: cce@=18.0.0 | ||
| paths: | ||
| cc: /usr/tce/packages/cce-tce/cce-18.0.0/bin/craycc | ||
| cxx: /usr/tce/packages/cce-tce/cce-18.0.0/bin/crayCC | ||
| f77: /usr/tce/packages/cce-tce/cce-18.0.0/bin/crayftn | ||
| fc: /usr/tce/packages/cce-tce/cce-18.0.0/bin/crayftn | ||
| flags: {} | ||
| operating_system: rhel8 | ||
| target: x86_64 | ||
| modules: [] | ||
| environment: {} | ||
| extra_rpaths: [] |
There was a problem hiding this comment.
Remove, this isn't used
| # Copy Spheral source and generate host config from tpl-manager (all dependencies should already be installed). | ||
| COPY . . | ||
| RUN python3 scripts/devtools/tpl-manager.py --spec $SPEC --upstream-dir /home/spack/opt/spack/__spack_path_placeholder__/__spack_path_placeholder__/__spack_path_placeholder__/__spack_path_placeholder_ --spack-url /home/spack | ||
|
|
||
| # Configure Spheral with SPEC TPLs. | ||
| RUN mv *.cmake $HOST_CONFIG.cmake | ||
| RUN python3 scripts/devtools/host-config-build.py --host-config $HOST_CONFIG.cmake | ||
| RUN python3 scripts/devtools/host-config-build.py --host-config $HOST_CONFIG.cmake -DSPHERAL_NETWORK_CONNECTED=Off |
There was a problem hiding this comment.
Add documentation on SPHERAL_NETWORK_CONNECTED
| @@ -57,6 +57,7 @@ Notable changes include: | |||
| * Bugfix for RZ solid CRKSPH with compatible energy. | |||
| * Parsing of None string now always becomes None python type. Tests have been updated accordingly. | |||
| * IO for checkpoints and visuzalization can now be properly turned off through SpheralController input options. | |||
| * Bugfix for atomicWeight in ANEOS | |||
|
|
|||
There was a problem hiding this comment.
Add RELEASE_NOTES Documentation
| #---------------------------------------------------------------------------------------- | ||
|
|
||
|
|
||
| set(SPHERAL_PIP_CACHE_DIR ~/.cache/spheral_pip) |
There was a problem hiding this comment.
Document SPHERAL_PIP_CACHE_DIR
mdavis36
changed the title
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Summary
PIP Runtime Dependencies
Spheral +python variant
+python. This will add python as a dependency for building spheral. It will also enable+pythonfor polytope.Build Time Deps
Spheral_Python_Envfunction we build a virtual environment target in the build tree with dependencies listed inbuild-requirements.txt. Allpybind11code and documentation is generated from this environment.docs-requirements.txtis added to the environment whenENABLE_DOCS=On.Runtime Deps
Spheral_Python_Envfunction we can generate a Virtual Env in the Install directory fromruntime-requirements.txt.spheral-setup-venvnow only needs to copy the installed Spheral libraries into the environment at install time.The
python_build_envandpython_runtime_envtargets are built and installed only one time in a build tree. Stamp files are created to ensurepipisn't re-run on everymake.Caching Builds for LC
We have
pipcache to a local directory~/.cache/spheral_pip/. This directory can be overridden withSPHERAL_PIP_CACHE_DIR.CRAY AMD
Due to issues with pip dependencies being unsupported on BlueOS systems this PR requires us to make the switch over to CRAY/AMD.
ToDo :
RELEASE_NOTES.mdwith notable changes.