Hi,
has anyone tried A-V Whitney solver in parallel mode? Am I the only one who gets distorted results with it: The potential variable P seems fine but the Magnetic Flux Density does not seem correct to me.
Please have a look at the attached sif file. It is basically the "mgdyn_transient" -case which you can find from the tests, but with parallel solver. I used 6 processes.
My Elmer version is "MAIN: Library version: 7.0 (Rev: 5875)".
Parallel Whitney, MGDynamicsCalc: distorted results
Parallel Whitney, MGDynamicsCalc: distorted results
- Attachments
-
- MGDynamicsTransient.sif
- (2.51 KiB) Downloaded 554 times
Re: Parallel Whitney, MGDynamicsCalc: distorted results
Hi Eelis,
i'm afraid that at least the direct solver (using tree gauge) won't work in parallel. When
using the ungauged system (and iterative solver, e.g. BiCGStabL; the transient system
is not symmetric) the safest preconditioning is 'none', as the over determined system
might give bad ILUs. This also has the benefit that the iteration count should not
depend on the number of partitions.
Can you try using an iterative solver (with 'none' as a preconditioner &
delete the 'Linear System Use HYPRE=True' flag)?
Regards, Juha
i'm afraid that at least the direct solver (using tree gauge) won't work in parallel. When
using the ungauged system (and iterative solver, e.g. BiCGStabL; the transient system
is not symmetric) the safest preconditioning is 'none', as the over determined system
might give bad ILUs. This also has the benefit that the iteration count should not
depend on the number of partitions.
Can you try using an iterative solver (with 'none' as a preconditioner &
delete the 'Linear System Use HYPRE=True' flag)?
Regards, Juha
Re: Parallel Whitney, MGDynamicsCalc: distorted results
Hi Juha,
thanks for the reply! It seems to work with the corrections you proposed.
Is there any way one could use parallel direct solver?
Cheers,
Eelis
thanks for the reply! It seems to work with the corrections you proposed.
Is there any way one could use parallel direct solver?
Cheers,
Eelis
-
- Site Admin
- Posts: 4832
- Joined: 22 Aug 2009, 11:57
- Antispam: Yes
- Location: Espoo, Finland
- Contact:
Re: Parallel Whitney, MGDynamicsCalc: distorted results
Hi Eelis
Only direct solver available in parallel is MUMPS. The license does not allow us to distribute it, and it is notoriously difficult to compile, but if you overcome those obstacles it could give good performance with ~10 partitions or so.
-Peter
Only direct solver available in parallel is MUMPS. The license does not allow us to distribute it, and it is notoriously difficult to compile, but if you overcome those obstacles it could give good performance with ~10 partitions or so.
-Peter
Re: Parallel Whitney, MGDynamicsCalc: distorted results
Hi Peter,
now I'm a bit confused:
- In the original post my configuration was the following
So I was using MUMPS and did not use the "tree gauge" (commented out). Now when I re-read your and Juha's post I think it should work. Why it was not working?
Cheers,
Eelis
EDIT:
Maybe Juha means that direct solver needs to use the "Tree Gauge" in any case to remove the nullspace? Otherwise there is a singular matrix? But is this relevant for well constrained transient cases as well?
now I'm a bit confused:
- In the original post my configuration was the following
Code: Select all
Solver 1
Equation = "MGDynamics"
Variable = P
Procedure = "MagnetoDynamics" "WhitneyAVSolver"
Linear System Symmetric = True
Linear System Solver = "Direct"
Linear System Direct Method = MUMPS
Constant System = Logical True
Linear System refactorize = False
! Linear System Solver = Iterative
! Linear System Iterative Method = BiCGStab
Linear System Max Iterations = 500
Linear System Convergence Tolerance = 1.0e-8
Linear System Preconditioning = ILU0
Linear System ILUT Tolerance = 1.0e-3
Linear System Abort Not Converged = False
Linear System Residual Output = 1
Linear System Precondition Recompute = 1
Linear System Use HYPRE = True
! USE Tree Gauge = Logical True
! Before Linsolve = "EliminateDirichlet" "EliminateDirichlet"
Steady State Convergence Tolerance = 1e-09
End
Cheers,
Eelis
EDIT:
Maybe Juha means that direct solver needs to use the "Tree Gauge" in any case to remove the nullspace? Otherwise there is a singular matrix? But is this relevant for well constrained transient cases as well?
-
- Site Admin
- Posts: 4832
- Joined: 22 Aug 2009, 11:57
- Antispam: Yes
- Location: Espoo, Finland
- Contact:
Re: Parallel Whitney, MGDynamicsCalc: distorted results
Sorry Eelis, I responded out of context. The creation of the tree gauge is such a messy task that I guess it is not implemented in parallel. Generally MUMPS is the only parallel direct MPI solver. -Peter
Re: Parallel Whitney, MGDynamicsCalc: distorted results
Ok. Now I understand.
Thanks,
Eelis
Thanks,
Eelis
Re: Parallel Whitney, MGDynamicsCalc: distorted results
Hei,
I have observed also distorted results in parallel with MagnetoDynamics2D. The potential was calculated correctly, but flux density was distorted in the air gap region of simulated electrical machine.
This resulted in completely wrong estimation of torque.
In serial simulation with UMFPACK solver there were no problems.
I used direct MUMPS solvers with discontinuous Galerkin. The mesh was partitioned with
This -connect 2 is required for sliding boundary in the center of the air gap.The last option is said to be required for discontinuous Galerkin methods.
However, when the mesh was repartitioned without -halo option, the parallel solution was not distorted (Discontinuous Galerkin was set to True in the parallel solver). The flux density in the air gap became realistic. The torque estimation became the same as in serial.
BR, Pavel
I have observed also distorted results in parallel with MagnetoDynamics2D. The potential was calculated correctly, but flux density was distorted in the air gap region of simulated electrical machine.
This resulted in completely wrong estimation of torque.
In serial simulation with UMFPACK solver there were no problems.
I used direct MUMPS solvers with discontinuous Galerkin. The mesh was partitioned with
Code: Select all
ElmerGrid 14 2 PMSM.msh -metis 4 3 -connect 2 -partdual -halo
However, when the mesh was repartitioned without -halo option, the parallel solution was not distorted (Discontinuous Galerkin was set to True in the parallel solver). The flux density in the air gap became realistic. The torque estimation became the same as in serial.
BR, Pavel