Spack
We use the Spack package manager to provide a collection of common HPC software packages. This page explains how to use the central Spack installation to build your own modulefiles.
Table of Contents
- Quick Setup
- Guide to Using Spack
- Central Spack Installation
- Overriding Package Definitions
Quick Setup with rub-deploy-spack-configs
You can directly copy the configuration files described in Central Spack Installation (upstreams.yaml
, config.yaml
, modules.yaml
, compilers.yaml
) to your home directory using the rub-deploy-spack-configs
command:
Add these lines to your ~/.bashrc
to activate spack
with every login:
export MODULEPATH=$MODULEPATH:$HOME/spack/share/spack/lmod/linux-almalinux9-x86_64/Core
. /cluster/spack/0.23.0/share/spack/setup-env.sh
Guide to Using Spack
Below is a detailed guide on how to effectively use Spack.
- Searching for Packages
- Viewing Package Variants
- Enabling/Disabling Variants
- Specifying Compilers
- Specifying Dependencies
- Putting It All Together
- Building and Adding a New Compiler
- Comparing Installed Package Variants
- Removing Packages
Searching for Packages
To find available packages, use:
spack list <keyword> # Search for packages by name
# Example:
spack list openfoam
openfoam openfoam-org
==> 2 packages
For detailed information about a package:
spack info <package> # Show versions, variants, and dependencies
# Example:
spack info hdf5
For a quick search for all available packages in spack
, visit https://packages.spack.io/.
Viewing Package Variants
Variants are build options that enable or disable features. List them with spack info <package>
:
Output includes:
Preferred version:
1.14.3 https://support.hdfgroup.org/ftp/HDF5/releases/hdf5-1.14/hdf5-1.14.3/src/hdf5-1.14.3.tar.gz
Safe versions:
1.14.3 https://support.hdfgroup.org/ftp/HDF5/releases/hdf5-1.14/hdf5-1.14.3/src/hdf5-1.14.3.tar.gz
1.14.2 https://support.hdfgroup.org/ftp/HDF5/releases/hdf5-1.14/hdf5-1.14.2/src/hdf5-1.14.2.tar.gz
1.14.1-2 https://support.hdfgroup.org/ftp/HDF5/releases/hdf5-1.14/hdf5-1.14.1-2/src/hdf5-1.14.1-2.tar.gz
1.14.0 https://support.hdfgroup.org/ftp/HDF5/releases/hdf5-1.14/hdf5-1.14.0/src/hdf5-1.14.0.tar.gz
1.12.3 https://support.hdfgroup.org/ftp/HDF5/releases/hdf5-1.12/hdf5-1.12.3/src/hdf5-1.12.3.tar.gz
1.12.2 https://support.hdfgroup.org/ftp/HDF5/releases/hdf5-1.12/hdf5-1.12.2/src/hdf5-1.12.2.tar.gz
1.12.1 https://support.hdfgroup.org/ftp/HDF5/releases/hdf5-1.12/hdf5-1.12.1/src/hdf5-1.12.1.tar.gz
1.12.0 https://support.hdfgroup.org/ftp/HDF5/releases/hdf5-1.12/hdf5-1.12.0/src/hdf5-1.12.0.tar.gz
1.10.11 https://support.hdfgroup.org/ftp/HDF5/releases/hdf5-1.10/hdf5-1.10.11/src/hdf5-1.10.11.tar.gz
1.10.10 https://support.hdfgroup.org/ftp/HDF5/releases/hdf5-1.10/hdf5-1.10.10/src/hdf5-1.10.10.tar.gz
Variants:
api [default] default, v110, v112, v114, v116, v16, v18
Choose api compatibility for earlier version
cxx [false] false, true
Enable C++ support
fortran [false] false, true
Enable Fortran support
hl [false] false, true
Enable the high-level library
mpi [true] false, true
Enable MPI support
Defaults are shown in square brackets, possible values to the right.
Checking the Installation
To see which dependencies will be installed, use:
Output includes:
Input spec
--------------------------------
- hdf5
Concretized
--------------------------------
[+] hdf5@1.14.3%gcc@11.4.1~cxx~fortran+hl~ipo~java~map+mpi+shared~subfiling~szip~threadsafe+tools api=default build_system=cmake build_type=Release generator=make patches=82088c8 arch=linux-almalinux9-zen4
[+] ^cmake@3.27.9%gcc@11.4.1~doc+ncurses+ownlibs build_system=generic build_type=Release arch=linux-almalinux9-zen4
[+] ^curl@8.7.1%gcc@11.4.1~gssapi~ldap~libidn2~librtmp~libssh+libssh2+nghttp2 build_system=autotools libs=shared,static tls=mbedtls,openssl arch=linux-almalinux9-zen4
[+] ^libssh2@1.11.0%gcc@11.4.1+shared build_system=autotools crypto=mbedtls patches=011d926 arch=linux-almalinux9-zen4
[+] ^xz@5.4.6%gcc@11.4.1~pic build_system=autotools libs=shared,static arch=linux-almalinux9-zen4
[+] ^mbedtls@2.28.2%gcc@11.4.1+pic build_system=makefile build_type=Release libs=shared,static arch=linux-almalinux9-zen4
[+] ^nghttp2@1.52.0%gcc@11.4.1 build_system=autotools arch=linux-almalinux9-zen4
[+] ^diffutils@3.10%gcc@11.4.1 build_system=autotools arch=linux-almalinux9-zen4
[+] ^openssl@3.3.0%gcc@11.4.1~docs+shared build_system=generic certs=mozilla arch=linux-almalinux9-zen4
[+] ^ca-certificates-mozilla@2023-05-30%gcc@11.4.1 build_system=generic arch=linux-almalinux9-zen4
[+] ^ncurses@6.5%gcc@11.4.1~symlinks+termlib abi=none build_system=autotools patches=7a351bc arch=linux-almalinux9-zen4
[+] ^gcc-runtime@11.4.1%gcc@11.4.1 build_system=generic arch=linux-almalinux9-zen4
[e] ^glibc@2.34%gcc@11.4.1 build_system=autotools arch=linux-almalinux9-zen4
[+] ^gmake@4.4.1%gcc@11.4.1~guile build_system=generic arch=linux-almalinux9-zen4
[+] ^openmpi@5.0.3%gcc@11.4.1~atomics~cuda~gpfs~internal-hwloc~internal-libevent~internal-pmix~java+legacylaunchers~lustre~memchecker~openshmem~orterunprefix~romio+rsh~static+vt+wrapper-rpath build_system=autotools fabrics=ofi romio-filesystem=none schedulers=slurm arch=linux-almalinux9-zen4
It’s always a good idea to check the specs before installing.
Enabling/Disabling Variants
Control variants with +
(enable) or ~
(disable):
spack install hdf5 +mpi +cxx ~hl # Enable MPI and C++, disable high-level API
For packages with CUDA, use compute capabilities 8.0 (for GPU nodes) and 9.0 (for FatGPU nodes):
spack install openmpi +cuda cuda_arch=80,90
Specifying Compilers
Use %
to specify a compiler. Check available compilers with:
Example:
spack install hdf5 %gcc@11.4.1
When using compilers other than GCC 11.4.1, dependencies must also be built with that compiler:
spack install --fresh hdf5 %gcc@13.2.0
Specifying Dependencies
Use ^
to specify dependencies with versions or variants:
spack install hdf5 ^openmpi@4.1.5
Dependencies can also have variants:
spack install hdf5 +mpi ^openmpi@4.1.5 +threads_multiple
Make sure to set the variants for the package and the dependencies on the right position or installation will fail.
Putting It All Together
Combine options for customized installations:
spack install hdf5@1.14.3 +mpi ~hl ^openmpi@4.1.5 +cuda cuda_arch=80,90 %gcc@11.4.1
- Compiles hdf 1.14.3 with GCC 11.4.1.
- Enables MPI support, disable high-level API.
- Uses OpenMPI 4.1.5 with cuda support as a dependency.
Building and Adding a New Compiler
Install a new compiler (e.g., GCC 13.2.0) with:
Add it to Spack’s compiler list:
spack compiler add $(spack location -i gcc@13.2.0)
Verify it’s recognized:
Use it to build packages:
spack install --fresh hdf5 %gcc@13.2.0
Comparing installed package variants
If you have multiple installations of the same package with different variants, you can inspect their configurations using Spack’s spec
command or the find
tool.
List installed packages with variants
Use spack find -vl
to show all installed variants and their hashes:
spack find -vl hdf5
-- linux-almalinux9-zen4 / gcc@11.4.1 ---------------------------
amrsck6 hdf5@1.14.3~cxx~fortran+hl~ipo~java~map+mpi+shared~subfiling~szip~threadsafe+tools api=default build_system=cmake build_type=Release generator=make patches=82088c8
2dsgtoe hdf5@1.14.3+cxx+fortran~hl~ipo~java~map+mpi+shared~subfiling~szip~threadsafe+tools api=default build_system=cmake build_type=Release generator=make patches=82088c8
==> 2 installed packages
amrsck6
and 2dsgtoe
are the unique hashes for each installation.
- You can see one package uses
+hl
while the other does not.
Inspect specific installations
Use spack spec /<hash>
to view details of a specific installation:
spack spec /amrsck6
spack spec /2dsgtoe
Compare two installations
To compare variants between two installations, use spack diff
with both hashes:
spack diff /amrsck6 /2dsgtoe
You will see a diff in the style of git:
--- hdf5@1.14.3/amrsck6mml43sfv4bhvvniwdydaxfgne
+++ hdf5@1.14.3/2dsgtoevoypx7dr45l5ke2dlb56agvz4
@@ virtual_on_incoming_edges @@
- openmpi mpi
+ mpich mpi
So one version depends on OpenMPI while the other depends on MPICH.
Removing Packages
For multiple variants of a package, specify the hash:
Central Spack Installation
Activate the central Spack installation with:
source /cluster/spack/0.23.0/share/spack/setup-env.sh
Use it as a starting point for your own builds without rebuilding everything from scratch.
Add these files to ~/.spack
:
~/.spack/upstreams.yaml
:
upstreams:
central-spack:
install_tree: /cluster/spack/opt
~/.spack/config.yaml
:
config:
install_tree:
root: $HOME/spack/opt/spack
source_cache: $HOME/spack/cache
license_dir: $HOME/spack/etc/spack/licenses
~/.spack/modules.yaml
:
modules:
default:
roots:
lmod: $HOME/spack/share/spack/lmod
enable: [lmod]
lmod:
all:
autoload: direct
hide_implicits: true
hierarchy: []
Add these lines to your ~/.bashrc
:
export MODULEPATH=$MODULEPATH:$HOME/spack/share/spack/lmod/linux-almalinux9-x86_64/Core
. /cluster/spack/0.23.0/share/spack/setup-env.sh
Then run:
Overriding Package Definitions
Create ~/.spack/repos.yaml
:
repos:
- $HOME/spack/var/spack/repos
And a local repo description in ~/spack/var/spack/repos/repo.yaml
:
repo:
namespace: overrides
Copy and edit a package definition, e.g., for ffmpeg
:
cd ~/spack/var/spack/repos/
mkdir -p packages/ffmpeg
cp /cluster/spack/0.23.0/var/spack/repos/builtin/packages/ffmpeg/package.py packages/ffmpeg
vim packages/ffmpeg/package.py
Alternatively, you can use a fully independent Spack installation in your home directory or opt for EasyBuild.