Unsuccessful install with mpi

Discussion about building and installing Elmer
spacedout
Posts: 177
Joined: 30 Mar 2020, 23:27
Antispam: Yes

Unsuccessful install with mpi

Post by spacedout »

Hi

I think Canada's computer cluster run on Redhat. Therefore I cannot use the Ubuntu binaries from Elmer's Github. So I downloaded the source files (version of April 2020) and since I require mpi, I generated the make files using

cmake -DWITH_OpenMP:BOOL=TRUE -DWITH_MPI:BOOL=TRUE ../install ../elmerfem

which showed no errors and so I proceeded with

make install

This is the output ( I use ...... to mean mpif-sizeof.h:nn:mm: Error: Array specification at (1) has more than 7 dimensions is repeated many times)

[ 1%] Built target matc
[ 1%] Built target Matc_bin
[ 1%] Built target umfpack_srcs
[ 41%] Built target umfpack
[ 41%] Built target amdf77
[ 44%] Built target amd
[ 44%] Built target fhuti
[ 45%] Built target Mesh2D
[ 50%] Built target arpack
[ 50%] Building Fortran object mathlibs/src/parpack/CMakeFiles/parpack.dir/pcgetv0.f.o
mpif-sizeof.h:75:48: Error: Array specification at (1) has more than 7 dimensions
............
mpif-sizeof.h:2256:44: Error: Array specification at (1) has more than 7 dimensions
mpif-sizeof.h:1135:72: Error: Ambiguous interfaces ‘mpi_sizeof_real64_r15’ and ‘mpi_sizeof_real64_r14’ in generic interface 'mpi_sizeof' at (1)
mpif-sizeof.h:2259:72: Error: Ambiguous interfaces ‘pmpi_sizeof_real64_r15’ and ‘pmpi_sizeof_real64_r14’ in generic interface 'pmpi_sizeof' at (1)
make[2]: *** [mathlibs/src/parpack/CMakeFiles/parpack.dir/build.make:63: mathlibs/src/parpack/CMakeFiles/parpack.dir/pcgetv0.f.o] Error 1
make[1]: *** [CMakeFiles/Makefile2:25166: mathlibs/src/parpack/CMakeFiles/parpack.dir/all] Error 2
make: *** [Makefile:163: all] Error 2

Is it possible that the error is due to a wrong mpi package ? They have several versions but I happen to be using

openmpi/2.1.1

Any feedback welcome

Thanks
kevinarden
Posts: 2237
Joined: 25 Jan 2019, 01:28
Antispam: Yes

Re: Unsuccessful install with mpi

Post by kevinarden »

You might try testing the compile without the mpi to see if it works. Resolve any other issues, first.
spacedout
Posts: 177
Joined: 30 Mar 2020, 23:27
Antispam: Yes

Re: Unsuccessful install with mpi

Post by spacedout »

Thank you but I had successfully succeeded in installing Elmer without openmpi back in April 2020 on Canada's computer cluster running on Redhat or CentOS. Since then I used elmerf90 to modify solvers and I execute the .sif files without any problems. However this is all serial programming and I need to upgrade to a parallel version for speed.
I suppose I could try clearing the build folder before generating the makefiles from command cmake. And if that does not help, then it is still a mystery to me.

Best, Marc
kevinarden
Posts: 2237
Joined: 25 Jan 2019, 01:28
Antispam: Yes

Re: Unsuccessful install with mpi

Post by kevinarden »

Perhaps openmpi 2.1.1 is too old it is retired. openmpi is up to version 4 now.
spacedout
Posts: 177
Joined: 30 Mar 2020, 23:27
Antispam: Yes

Re: Unsuccessful install with mpi

Post by spacedout »

Thanks for the hint but after I loaded openmpi/4.0.5 available on our system
and executed make install, I now got two different error messages:

[ 1%] Built target matc
[ 1%] Built target Matc_bin
[ 1%] Built target umfpack_srcs
[ 41%] Built target umfpack
[ 41%] Built target amdf77
[ 44%] Built target amd
[ 44%] Built target fhuti
[ 45%] Built target Mesh2D
[ 50%] Built target arpack
[ 50%] Building Fortran object mathlibs/src/parpack/CMakeFiles/parpack.dir/pcgetv0.f.o
/home/elmer/elmerfem/mathlibs/src/parpack/pcgetv0.f:366:38:
335 | call MPI_ALLREDUCE( cnorm_buf, cnorm, 1,
| 2
......
366 | call MPI_ALLREDUCE( workl(j+1), workl, j-1,
| 1
Error: Rank mismatch between actual argument at (1) and actual argument at (2) (scalar and rank-1)
/home/elmer/elmerfem/mathlibs/src/parpack/pcgetv0.f:446:39:
442 | call pcvout (comm, logfil, 1, cnorm2, ndigit,
| 2
......
446 | call pcvout (comm, logfil, n, resid, ndigit,
| 1
Error: Rank mismatch between actual argument at (1) and actual argument at (2) (scalar and rank-1)
make[2]: *** [mathlibs/src/parpack/CMakeFiles/parpack.dir/build.make:63: mathlibs/src/parpack/CMakeFiles/parpack.dir/pcgetv0.f.o] Error 1
make[1]: *** [CMakeFiles/Makefile2:25166: mathlibs/src/parpack/CMakeFiles/parpack.dir/all] Error 2
make: *** [Makefile:163: all] Error 2

Hopefully, this rank mismatch will ring a bell to someone
Kindest regards
Marc

P.S.
I could add that the cmake had always yielded

-- Mesh adaptation 2D/3D looking for [Mmg] tools
-- Library not found: >MMG_FOUND<
-- Missing: >MMG_INCLUDE_DIR< , >MMG_LIBRARY<, to compile MMG3DSolver

even in the serial case, but I never bothered to solve that problem since for the time being,I dont need mesh adaptation in my research.
raback
Site Admin
Posts: 4812
Joined: 22 Aug 2009, 11:57
Antispam: Yes
Location: Espoo, Finland
Contact:

Re: Unsuccessful install with mpi

Post by raback »

Hi

Looks similar to this one:
https://github.com/ElmerCSC/elmerfem/issues/250

Maybe the remedy could be the same too?

As it says there we tried to resolve the Arpack mismatch issue for new compilers. Arpack is a legacy library that always compiled ok but now with GCC>=10 the compiler refuses to compile it. What I hear from people who have been around long enough is that the Arpack mismatch is intentional and allows same routine to be used for different ranks.

So it seems that the new versions are becoming picky which makes life more difficult for the legacy libraries. ElmerSolver library itself is of course contemporary and there are no problems...

-Peter
spacedout
Posts: 177
Joined: 30 Mar 2020, 23:27
Antispam: Yes

Re: Unsuccessful install with mpi

Post by spacedout »

That last post was for sure very helpful: I downloaded the latest source files which includes the very latest change to cmakefiles.txt and now the problem with rank mismatch indeed vanishes.

Unfortunately make install still fails:
........
Install the project...
-- Install configuration: "RelWithDebInfo"
CMake Error at matc/src/cmake_install.cmake:47 (file):
file cannot create directory: /lib/elmersolver. Maybe need administrative
privileges.
Call Stack (most recent call first):
matc/cmake_install.cmake:42 (include)
cmake_install.cmake:42 (include)
make: *** [Makefile:86: install] Error 1

It is as if the option

-DCMAKE_INSTALL_PREFIX=../install

was completely ignored by the cmake command so that all of install's sub folders /bin, /lib, /shared are then created under some Linux folder like /usr/local which I course I have no administrative rights to access with sudo since I am not in charge of Canada's computer clusters.

So, i am at a loss as to how I should go about this.

Thanks anyhow
kevinarden
Posts: 2237
Joined: 25 Jan 2019, 01:28
Antispam: Yes

Re: Unsuccessful install with mpi

Post by kevinarden »

I think if it was completely ignore it you would have hit an error before getting to matc. Did you look in the install directory to see if lib/elmersolver was there?
spacedout
Posts: 177
Joined: 30 Mar 2020, 23:27
Antispam: Yes

Re: Unsuccessful install with mpi

Post by spacedout »

I cannot look into the contents of the install directory since the install directory itself was never created. So, I think it would make sense for the make install to fail in creating an elmersolver folder under Linux's /usr/lib/ folder, since I dont have administrative rights to write to /usr/lib/.
Post Reply