Help for building MOOSE in a offline HPC

[shrituntunroy]

Check these boxes if you have followed the posting rules.

• I have consulted the Posting Guidelines.
• I have searched the Discussions Forum and MOOSE Framework Troubleshooting and have not found what I was looking for
• Q&A Getting Started is the most appropriate category for my question (trouble installing, beginner user, ...)

Issue or question about MOOSE

Hi Moose Team,

I am a new MOOSE user and currently learning to use MOOSE. I am installing moose in a offline HPC (on which I dont have sudo access) following this method - https://mooseframework.inl.gov/getting_started/installation/offline_installation.html

I have already installed these offline packages in my system and they are working correctly.

But in the HPC, I am getting following error related to clang compiler:

/home/lus03/vikramr/projects/moose/scripts
INFO: Checking for HDF5...
INFO: HDF5 library not detected, opting to download via PETSc...
=============================================================================================
                         Configuring PETSc to compile on your system
=============================================================================================
TESTING: checkCCompiler from config.setCompilers(config/BuildSystem/config/setCompilers.py:1427)
*********************************************************************************************
           UNABLE to CONFIGURE with GIVEN OPTIONS (see configure.log for details):
---------------------------------------------------------------------------------------------
             C compiler you provided with -CC= cannot be found or does not work.
*********************************************************************************************

There was an error. Exiting...
[vikramr@atulya335 scripts]$ module load openmpi-gcc/4.0.1
[vikramr@atulya335 scripts]$ ./update_and_rebuild_petsc.sh --CC=$CC --CXX=$CXX --FC=$FC --with-packages-download-dir=~/projects/downloads
/home/lus03/vikramr/projects/moose/scripts
INFO: Checking for HDF5...
INFO: HDF5 library not detected, opting to download via PETSc...
=============================================================================================
                         Configuring PETSc to compile on your system
=============================================================================================
TESTING: checkCCompiler from config.setCompilers(config/BuildSystem/config/setCompilers.py:1427)
*********************************************************************************************
           UNABLE to CONFIGURE with GIVEN OPTIONS (see configure.log for details):
---------------------------------------------------------------------------------------------
             C compiler you provided with -CC= cannot be found or does not work.
*********************************************************************************************

There was an error. Exiting...
[vikramr@atulya335 scripts]$ export CC=mpicc CXX=mpicxx FC=mpif90 F90=mpif90 F77=mpif77
[vikramr@atulya335 scripts]$ ./update_and_rebuild_petsc.sh --CC=$CC --CXX=$CXX --FC=$FC --with-packages-download-dir=~/projects/downloads
/home/lus03/vikramr/projects/moose/scripts
INFO: Checking for HDF5...
INFO: HDF5 library not detected, opting to download via PETSc...
=============================================================================================
                         Configuring PETSc to compile on your system
=============================================================================================
=============================================================================================
                                     ***** WARNING *****
    Found environment variable: CC=mpicc. Ignoring it, since its also set on command line
=============================================================================================
=============================================================================================
                                     ***** WARNING *****
   Found environment variable: CXX=mpicxx. Ignoring it, since its also set on command line
=============================================================================================
=============================================================================================
                                     ***** WARNING *****
    Found environment variable: FC=mpif90. Ignoring it, since its also set on command line
=============================================================================================
=============================================================================================
                                     ***** WARNING *****
  Found environment variable: F77=mpif77. Ignoring it! Use "./configure F77=$F77" if you
  really want to use this value
=============================================================================================
=============================================================================================
                                     ***** WARNING *****
  Found environment variable: F90=mpif90. Ignoring it! Use "./configure F90=$F90" if you
  really want to use this value
=============================================================================================
TESTING: checkCxxDialect from config.setCompilers(config/BuildSystem/config/setCompilers.py:773)
*********************************************************************************************
           UNABLE to CONFIGURE with GIVEN OPTIONS (see configure.log for details):
---------------------------------------------------------------------------------------------
  Using CUDA dialect C++11 as lower bound due to package(s):
  - hypre
  - strumpack
  - SuperLU_DIST
  But CUDA compiler (clang) appears non-compliant with C++11 or didn't accept:
  - -std=gnu++20
  - -std=c++20
  - -std=gnu++17
  - -std=c++17
  - -std=gnu++14
  - -std=c++14
  - -std=gnu++11
  - -std=c++11
*********************************************************************************************

There was an error. Exiting...

I have tried hiding the clang compiler by exporting the following line - export clang=0 in bashrc but still I m getting the same error.

In my pc I didnt had clang but still moose installed correctly

Kindly help me for resolving this error.

(Optional) code in question / simulation log / errors

No response

Encountering Errors? Please include diagnostic output

##################################################################################################
Influential Environment Variables

CC=mpicc
CONDA_CHANNEL=https://conda.software.inl.gov/public
CONDA_SHLVL=0
CURL_CA_BUNDLE=
CXX=mpicxx
F77=mpif77
F90=mpif90
FC=mpif90
LD_LIBRARY_PATH=/dlocal/sysapps/openmpi-gcc-4.0.1/lib64:/dlocal/sysapps/gcc-9.2.0/lib64:/dlocal/sysapps/gcc-9.2.0/lib:/home/lus03/vikramr/app/gcc/gcc1230/lib64:/home/lus03/vikramr/app/gcc/gcc1230/lib:/home/lus03/vikramr/app/gcc/mpc121/lib:/home/lus03/vikramr/app/gcc/mpfr410/lib:/home/lus03/vikramr/app/gcc/gmp621/lib:/home/lus03/vikramr/app/gcc/gcc1230/lib:/home/lus03/vikramr/app/gcc/mpc121/lib:/home/lus03/vikramr/app/gcc/mpfr410/lib:/home/lus03/vikramr/app/gcc/gmp621/lib:/home/lus03/vikramr/app/gcc/gcc1230/lib64:/home/lus03/vikramr/app/gcc/gcc1230/lib:/home/lus03/vikramr/app/gcc/mpc121/lib:/home/lus03/vikramr/app/gcc/mpfr410/lib:/home/lus03/vikramr/app/gcc/gmp621/lib:/home/lus03/vikramr/app/gcc/gcc1230/lib:/home/lus03/vikramr/app/gcc/mpc121/lib:/home/lus03/vikramr/app/gcc/mpfr410/lib:/home/lus03/vikramr/app/gcc/gmp621/lib:
MODULESHOME=/usr/share/Modules
MOOSE_JOBS=6
PATH=/home/lus03/vikramr/.python3/3.7.4/bin:/dlocal/sysapps/anaconda3/2019.10/bin/:/dlocal/sysapps/openmpi-gcc-4.0.1/bin/:/dlocal/sysapps/gcc-9.2.0/bin:/home/lus03/vikramr/app/python/Python-3.8.0:/home/lus03/vikramr/app/cmake/cmake-3.28.3/bin:/home/lus03/vikramr/app/python/Python-3.8.0:/home/lus03/vikramr/app/gcc/gcc1230/bin:/home/lus03/vikramr/app/cmake/cmake-3.28.3/bin:/usr/bin:/usr/sbin:/usr/lib64/qt-3.3/bin:/usr/condabin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/usr/local/anupam:/opt/VirtualGL/bin:/opt/ibutils/bin:.:/home/lus03/vikramr/app/lammps2023/lammps-2Aug2023/src:/home/lus03/vikramr/.local/bin:/home/lus03/vikramr/bin:/home/lus03/vikramr/app/lammps2023/lammps-2Aug2023/src
REQUESTS_CA_BUNDLE=
SSL_CERT_FILE=

##################################################################################################
Compiler(s) (CC CXX FC F77 F90):

CC=/dlocal/sysapps/openmpi-gcc-4.0.1/bin/mpicc
CC -show:
gcc -I/dlocal/sysapps/openmpi-gcc-4.0.1/include/openmpi -I/dlocal/sysapps/openmpi-gcc-4.0.1/include/openmpi/opal/mca/hwloc/hwloc201/hwloc/include -I/dlocal/sysapps/openmpi-gcc-4.0.1/include/openmpi/opal/mca/event/libevent2022/libevent -I/dlocal/sysapps/openmpi-gcc-4.0.1/include/openmpi/opal/mca/event/libevent2022/libevent/include -I/dlocal/sysapps/openmpi-gcc-4.0.1/include -pthread -Wl,-rpath -Wl,/dlocal/sysapps/openmpi-gcc-4.0.1/lib64 -Wl,--enable-new-dtags -L/dlocal/sysapps/openmpi-gcc-4.0.1/lib64 -lmpi
CC version:     gcc (GCC) 9.2.0

CXX=/dlocal/sysapps/openmpi-gcc-4.0.1/bin/mpicxx
CXX -show:
g++ -I/dlocal/sysapps/openmpi-gcc-4.0.1/include/openmpi -I/dlocal/sysapps/openmpi-gcc-4.0.1/include/openmpi/opal/mca/hwloc/hwloc201/hwloc/include -I/dlocal/sysapps/openmpi-gcc-4.0.1/include/openmpi/opal/mca/event/libevent2022/libevent -I/dlocal/sysapps/openmpi-gcc-4.0.1/include/openmpi/opal/mca/event/libevent2022/libevent/include -I/dlocal/sysapps/openmpi-gcc-4.0.1/include -pthread -Wl,-rpath -Wl,/dlocal/sysapps/openmpi-gcc-4.0.1/lib64 -Wl,--enable-new-dtags -L/dlocal/sysapps/openmpi-gcc-4.0.1/lib64 -lmpi
CXX version:    g++ (GCC) 9.2.0

FC=/dlocal/sysapps/openmpi-gcc-4.0.1/bin/mpif90
FC -show:
gfortran -I/dlocal/sysapps/openmpi-gcc-4.0.1/include -pthread -I/dlocal/sysapps/openmpi-gcc-4.0.1/lib64 -Wl,-rpath -Wl,/dlocal/sysapps/openmpi-gcc-4.0.1/lib64 -Wl,--enable-new-dtags -L/dlocal/sysapps/openmpi-gcc-4.0.1/lib64 -lmpi_usempi -lmpi_mpifh -lmpi
FC version:     GNU Fortran (GCC) 9.2.0

F77=/dlocal/sysapps/openmpi-gcc-4.0.1/bin/mpif77
F77 -show:
gfortran -I/dlocal/sysapps/openmpi-gcc-4.0.1/include -pthread -I/dlocal/sysapps/openmpi-gcc-4.0.1/lib64 -Wl,-rpath -Wl,/dlocal/sysapps/openmpi-gcc-4.0.1/lib64 -Wl,--enable-new-dtags -L/dlocal/sysapps/openmpi-gcc-4.0.1/lib64 -lmpi_usempi -lmpi_mpifh -lmpi
F77 version:    GNU Fortran (GCC) 9.2.0

F90=/dlocal/sysapps/openmpi-gcc-4.0.1/bin/mpif90
F90 -show:
gfortran -I/dlocal/sysapps/openmpi-gcc-4.0.1/include -pthread -I/dlocal/sysapps/openmpi-gcc-4.0.1/lib64 -Wl,-rpath -Wl,/dlocal/sysapps/openmpi-gcc-4.0.1/lib64 -Wl,--enable-new-dtags -L/dlocal/sysapps/openmpi-gcc-4.0.1/lib64 -lmpi_usempi -lmpi_mpifh -lmpi
F90 version:    GNU Fortran (GCC) 9.2.0

OK

##################################################################################################
Python Sanity Checks

Verify `/usr/bin/env python3 --version` (reporting as: Python 3.7.4),
matches versions for: `which python3 && which python`

OK

##################################################################################################
Python Modules (TestHarness, run-ability)

OK

[milljm]

With the mention of Cuda, I wouldn't be surprised if GCC 9.x is too old. I hope I am wrong. Tagging @grmnptr for possible confirmation...

Also, would it be possible for you to attach the configure.log it mentions? It should reside:

/home/lus03/vikramr/projects/moose/petsc/configure.log

[shrituntunroy]

Hi Jason,

Thankyou for your reply.

Please find the configure.log file.

I have also tried gcc 12.3.0 but still I am getting the same error. Is it possible to build moose without cuda so as to avoid this error

configure.log -

configure.log

[milljm]

Looks like its in /usr/bin according to the configure.log. Not a PATH we can easily remove.

[shrituntunroy]

Hi Giudicelli ,

Thanks for your reply

Which clang returns the following -

[vikramr@atulya335 ~]$ which clang
/usr/bin/clang

I have tried removing the usr/bin/ from the path but I am unable to run the script as shown below

[vikramr@atulya335 scripts]$ which clang
/usr/bin/clang
[vikramr@atulya335 scripts]$ export PATH=$(echo $PATH | sed -e 's|:/usr/bin||g' -e 's|/usr/bin:||g')
[vikramr@atulya335 scripts]$ which clang
/usr/bin/which: no clang in (/home/lus03/vikramr/.python3/3.7.4/bin:/dlocal/sysapps/anaconda3/2019.10/bin/:/dlocal/sysapps/openmpi-gcc-4.0.1/bin/:/home/lus03/vikramr/app/python/Python-3.8.0:/home/lus03/vikramr/app/gcc/gcc1230/bin:/home/lus03/vikramr/app/cmake/cmake-3.28.3/bin:/usr/mpi/gcc/openmpi-4.0.2rc3/bin/:/usr/sbin:/usr/lib64/qt-3.3/bin:/usr/condabin:/usr/local/bin:/usr/local/sbin:/usr/sbin:/usr/local/anupam:/opt/VirtualGL/bin:/opt/ibutils/bin:.:/home/lus03/vikramr/app/lammps2023/lammps-2Aug2023/src:/home/lus03/vikramr/.local/bin:/home/lus03/vikramr/bin)
[vikramr@atulya335 scripts]$ ./update_and_rebuild_petsc.sh --skip-submodule-update --with-packages-download-dir=~/projects/downloads
/usr/bin/env: bash: No such file or directory

I also tried an alias

alias clang='echo "clang is disabled"'

but still it results in same error as in configure.log.

Can you suggest some method so as to compile petsc without using clang or cuda

[milljm]

yeah, you can't remove /usr/bin from PATH. Nothing will work afterwards. I am surprised that which worked to be honest.

[GiudGiud]

Jason is right we wont be able to do that.

I'm not seeing --with-cuda in the configure command.
I am seeing this at the top of the configure log though

Python version:
3.7.4 (default, Aug 13 2019, 20:35:49) 
[GCC 7.3.0]

[GiudGiud]

so let's clean this PATH still. Make sure the conda python is found first, and the newer GCC too.
PATH is read from beginning (left) to the end

[shrituntunroy]

Thank you Jason and Guillaume,

For your replies and support.

I successfully compiled and installed moose in the HPC system by installing and compiling a newer version of llvm and clang.