Parallel Operation

[willsprocket]

I have a Windows PC with dual Xeons (each with 14 cores/28 logical cores). It seems if I specify more than 16 processors, ElmerSolver fails. Is there another setting I need to change?

[kevinarden]

By fail what is the specific reason reported?

[raback]

Hi,

Threading is supported partly. The main method for parallel performance is mpi. There are Elmer packages with mpi also for Windows.

OpenMP only works with shared memory and therefore is limited in scalabillity.

-Peter

[kevinarden]

Cores and threads are 2 different things in hardware and programing. However if you have 2 cpu with 14 core each then you should have 28 cores or logical processors that MPI can use. To verify this go to task manager, performance tab, got to the bottom and it should say how cores and how many logical processors are available.

[willsprocket]

For steady-state
The errors start after 23 cores. It gives an error in allocating 24 bytes for a matrix in a piece of Fortran code.

For transient
The errors start after 20 cores. Error: AllocateElementVector: Unable to Allocate structured 699624 element array

The PC has 28 cores and 56 logical processors.

[kevinarden]

Sounds like RAM allocation issues. If it is one machine with shared memory than each processor has to have so much RAM to use. I believe parallel increases the memory demand because each core has a certain amount basic usage. How much RAM does the machine have? If you use the task manager, performance tab, there is a plot available to monitor memory usage.

[Dmitrij]

Hi,
I also encountered parallel calculation problem by using MS MPI when threads number exceed a certain value. In example project AcousticWaves it is 12 threads, below this value Elmer calculates normally, but above this value MS MPI/Elmer gives error. While the same project with Intel MPI runs without errors at any threads number, but runtime is considerably slower that that in the case of MS MPI. See attached logs.


Hardware Cisco server with 4 socket CPU 4xE7-8880v3 with 512GB RAM(OS Win Server 2019)

[Dmitrij]

intel MPI 64 thread log

[kevinarden]

There is a difference between cores and threads. We seem to be using the terms interchangeable here. I believe Elmer is programmed for cores, not threads.

https://www.temok.com/blog/cores-vs-threads/

[Dmitrij]

Yes, I mean threads, not cores. The system have 64 active physical cores (HT off). Number of threads in Elmer was changed only.