Page 1 of 1

Parallel Operation

Posted: 23 Jun 2022, 19:33
by willsprocket
I have a Windows PC with dual Xeons (each with 14 cores/28 logical cores). It seems if I specify more than 16 processors, ElmerSolver fails. Is there another setting I need to change?

Re: Parallel Operation

Posted: 24 Jun 2022, 13:25
by kevinarden
By fail what is the specific reason reported?

Re: Parallel Operation

Posted: 24 Jun 2022, 15:00
by raback
Hi,

Threading is supported partly. The main method for parallel performance is mpi. There are Elmer packages with mpi also for Windows.

OpenMP only works with shared memory and therefore is limited in scalabillity.

-Peter

Re: Parallel Operation

Posted: 24 Jun 2022, 18:03
by kevinarden
Cores and threads are 2 different things in hardware and programing. However if you have 2 cpu with 14 core each then you should have 28 cores or logical processors that MPI can use. To verify this go to task manager, performance tab, got to the bottom and it should say how cores and how many logical processors are available.

Re: Parallel Operation

Posted: 24 Jun 2022, 18:38
by willsprocket
For steady-state
The errors start after 23 cores. It gives an error in allocating 24 bytes for a matrix in a piece of Fortran code.

For transient
The errors start after 20 cores. Error: AllocateElementVector: Unable to Allocate structured 699624 element array

The PC has 28 cores and 56 logical processors.

Re: Parallel Operation

Posted: 24 Jun 2022, 19:10
by kevinarden
Sounds like RAM allocation issues. If it is one machine with shared memory than each processor has to have so much RAM to use. I believe parallel increases the memory demand because each core has a certain amount basic usage. How much RAM does the machine have? If you use the task manager, performance tab, there is a plot available to monitor memory usage.

Re: Parallel Operation

Posted: 29 Jul 2022, 23:08
by Dmitrij
Hi,
I also encountered parallel calculation problem by using MS MPI when threads number exceed a certain value. In example project AcousticWaves it is 12 threads, below this value Elmer calculates normally, but above this value MS MPI/Elmer gives error. While the same project with Intel MPI runs without errors at any threads number, but runtime is considerably slower that that in the case of MS MPI. See attached logs.


Hardware Cisco server with 4 socket CPU 4xE7-8880v3 with 512GB RAM(OS Win Server 2019)

Re: Parallel Operation

Posted: 29 Jul 2022, 23:08
by Dmitrij
intel MPI 64 thread log

Re: Parallel Operation

Posted: 29 Jul 2022, 23:35
by kevinarden
There is a difference between cores and threads. We seem to be using the terms interchangeable here. I believe Elmer is programmed for cores, not threads.

https://www.temok.com/blog/cores-vs-threads/

Re: Parallel Operation

Posted: 30 Jul 2022, 00:05
by Dmitrij
Yes, I mean threads, not cores. The system have 64 active physical cores (HT off). Number of threads in Elmer was changed only.