Parallel Whitney, MGDynamicsCalc: distorted results

[Takala]

Hi,

has anyone tried A-V Whitney solver in parallel mode? Am I the only one who gets distorted results with it: The potential variable P seems fine but the Magnetic Flux Density does not seem correct to me.

Please have a look at the attached sif file. It is basically the "mgdyn_transient" -case which you can find from the tests, but with parallel solver. I used 6 processes.
My Elmer version is "MAIN: Library version: 7.0 (Rev: 5875)".

[Juha]

Hi Eelis,

i'm afraid that at least the direct solver (using tree gauge) won't work in parallel. When
using the ungauged system (and iterative solver, e.g. BiCGStabL; the transient system
is not symmetric) the safest preconditioning is 'none', as the over determined system
might give bad ILUs. This also has the benefit that the iteration count should not
depend on the number of partitions.

Can you try using an iterative solver (with 'none' as a preconditioner &
delete the 'Linear System Use HYPRE=True' flag)?

Regards, Juha

[Takala]

Hi Juha,

thanks for the reply! It seems to work with the corrections you proposed.

Is there any way one could use parallel direct solver?

Cheers,

Eelis

[raback]

Hi Eelis

Only direct solver available in parallel is MUMPS. The license does not allow us to distribute it, and it is notoriously difficult to compile, but if you overcome those obstacles it could give good performance with ~10 partitions or so.

-Peter

[Takala]

Hi Peter,

now I'm a bit confused:
- In the original post my configuration was the following

Code: Select all

Solver 1
  Equation = "MGDynamics"

  Variable = P
  Procedure = "MagnetoDynamics" "WhitneyAVSolver"
  Linear System Symmetric = True

  Linear System Solver = "Direct"
  Linear System Direct Method = MUMPS

  Constant System = Logical True
  Linear System refactorize = False

!  Linear System Solver = Iterative
!  Linear System Iterative Method = BiCGStab
  Linear System Max Iterations = 500
  Linear System Convergence Tolerance = 1.0e-8
  Linear System Preconditioning = ILU0
  Linear System ILUT Tolerance = 1.0e-3
  Linear System Abort Not Converged = False
  Linear System Residual Output = 1
  Linear System Precondition Recompute = 1
  Linear System Use HYPRE = True


! USE Tree Gauge = Logical True
! Before Linsolve = "EliminateDirichlet" "EliminateDirichlet"

  Steady State Convergence Tolerance = 1e-09
End

So I was using MUMPS and did not use the "tree gauge" (commented out). Now when I re-read your and Juha's post I think it should work. Why it was not working?

Cheers,

Eelis

EDIT:
Maybe Juha means that direct solver needs to use the "Tree Gauge" in any case to remove the nullspace? Otherwise there is a singular matrix? But is this relevant for well constrained transient cases as well?

[raback]

Sorry Eelis, I responded out of context. The creation of the tree gauge is such a messy task that I guess it is not implemented in parallel. Generally MUMPS is the only parallel direct MPI solver. -Peter

[Takala]

Ok. Now I understand.
Thanks,

Eelis

[pavel]

Hei,

I have observed also distorted results in parallel with MagnetoDynamics2D. The potential was calculated correctly, but flux density was distorted in the air gap region of simulated electrical machine.

This resulted in completely wrong estimation of torque.
In serial simulation with UMFPACK solver there were no problems.
I used direct MUMPS solvers with discontinuous Galerkin. The mesh was partitioned with

Code: Select all

ElmerGrid 14 2 PMSM.msh -metis 4 3 -connect 2 -partdual -halo 

This -connect 2 is required for sliding boundary in the center of the air gap.The last option is said to be required for discontinuous Galerkin methods.
However, when the mesh was repartitioned without -halo option, the parallel solution was not distorted (Discontinuous Galerkin was set to True in the parallel solver). The flux density in the air gap became realistic. The torque estimation became the same as in serial.

BR, Pavel