elmersolver_mpi doesn't exit on completion

Numerical methods and mathematical models of Elmer
mark smith
Posts: 215
Joined: 26 Aug 2009, 18:20
Location: Peterborough, England

elmersolver_mpi doesn't exit on completion

Post by mark smith »

I'm running elmer solver_mpi version 8 (Rev: f0c1adb) on a windows 7 professional OS system and have noticed that when it finishes it mostly hangs and doesn't exit to the cmd window i.e.
the echoes output just ends like so:

ElmerSolver: *** Elmer Solver: ALL DONE ***
ElmerSolver: The end
SOLVER TOTAL TIME(CPU,REAL): 3712.91 3712.91
ELMER SOLVER FINISHED AT: 2016/06/07 12:23:51

I need to hit <ctrl> C to exit back to the cmd line, everything about the run seems to be OK with all the result being written out, so in the most part it is an inconvenience but not a problem.
I'm now wanting to run a series of simulations launched from a batch file but as the programme doesn't exit correctly the batch job will just hang, is this a known problem & if so is there a way (say in the sif file) to get it to finish properly or can someone suggest a work around?

Thanks in advance
Regards
Mark
Romuald
Posts: 16
Joined: 28 Sep 2020, 17:13
Antispam: Yes

Re: elmersolver_mpi doesn't exit on completion

Post by Romuald »

Hi

did you finally manage to solve this issue?

I am meeting the same problem.

Thanks,

Romuald
Romuald
Posts: 16
Joined: 28 Sep 2020, 17:13
Antispam: Yes

Re: elmersolver_mpi doesn't exit on completion

Post by Romuald »

Apparently, the problem appears when I have more than 2 instances running.

I see that one instance does not terminate.

Romuald
acdichong
Posts: 1
Joined: 06 Oct 2020, 04:23
Antispam: Yes

Re: elmersolver_mpi doesn't exit on completion

Post by acdichong »

I have the same problem. Did you fix it? It is not convenient if the solver cannot be terminated.
Romuald
Posts: 16
Joined: 28 Sep 2020, 17:13
Antispam: Yes

Re: elmersolver_mpi doesn't exit on completion

Post by Romuald »

I did not solve the problem yet unfortunately and will definitely post a solution if I find one.

I am surprised that such behaviour apparently relatively common did not get any echo on the forum.

Let us see.

Best
raback
Site Admin
Posts: 4812
Joined: 22 Aug 2009, 11:57
Antispam: Yes
Location: Espoo, Finland
Contact:

Re: elmersolver_mpi doesn't exit on completion

Post by raback »

Hi

Can you confirm that you're all using Windows. This is also the likely reason why this hasn't been noticed. I would think that most MPI jobs are run under Linux.

The line with "ELMER SOLVER FINISHED" is definately the last one in the code. It is only printed for partition zero so there is of course the slim chance that some partition would be hanging there but I doubt it.

Does it hand always or does there seem to be some critical feature that triggers this? Linear solver, module, ...

-Peter
sslone
Posts: 5
Joined: 06 Feb 2020, 19:35
Antispam: Yes

Re: elmersolver_mpi doesn't exit on completion

Post by sslone »

I'm having a similar hanging on windows. I have tried it with several sif scripts, some trivial inert systems, and it always hangs regardless of the module used.

I've not yet found myself if this issue is Elmersolver_mpi or Mpiexec specific.

-Scott
Romuald
Posts: 16
Joined: 28 Sep 2020, 17:13
Antispam: Yes

Re: elmersolver_mpi doesn't exit on completion

Post by Romuald »

Hi

I confirm I run the simulations on Windows 10. It seems to me to be an ElmerSolver_mpi issue. When I run with 4 nodes for example, I can see in the windows task manager 4 instances of ElmerSolver_mpi poping up as expected. Once the message "ELMER SOLVER FINISHED" is printed, 3 nodes dissapeared and then one remains. Mpiexec in the console is still hanging waiting for the last node to end. When I kill this late node from the task manager, mpiexec ends and the console prompt is restored.

I hope this report helps.

Best,

Romuald
raback
Site Admin
Posts: 4812
Joined: 22 Aug 2009, 11:57
Antispam: Yes
Location: Espoo, Finland
Contact:

Re: elmersolver_mpi doesn't exit on completion

Post by raback »

Hi

What if you set to simulation section,

Code: Select all

Max Output Level = 10
Max Output Partition = 4  ! all partitions printed
Do all partitions make the same last print commands?

-Peter
Romuald
Posts: 16
Joined: 28 Sep 2020, 17:13
Antispam: Yes

Re: elmersolver_mpi doesn't exit on completion

Post by Romuald »

here are the last outputs with such option:

Code: Select all

Part0: The end
Part3: The end
ElmerSolver:
Part2: The end
SOLVER TOTAL TIME(CPU,REAL):        47.94       47.94
Part1: The end
ELMER SOLVER FINISHED AT: 2020/10/06 17:32:08
Post Reply