Parallel Eliminate Dirichlet

[Franz Pichler]

Hello,

I just tried the test StokesPFEM in parallel.
So first i partitioned the mesh with Elmergrid using metis into 2 partitions. then i runned the mpirun -np 2 ElmerSolver_mpi command.

I tried that specific test because i want to use a modification of hte EliminateDirichlet function, which is applied before solving.
And the StokesPFEM is using that. So i had a look into EliminateDirichlet. In there the Paralel enviroment is handled so i guess it should work in parallel.

The problem is somewhere around the call to ParallelInitMatrix( Solver, BMatrix ). This Segfaults.

Could someone have alook at that? THat would be very nice.

best regards
Franz Pichler

[Franz Pichler]

Hello once again,

i would like to warm up this thread because i am still very interested in a parallel EliminateDirichlet function.

I digged into the source code a bit more and tracked down the segfault to the following:


The actual segfault happens in the "SplitMatrix" function called by "ParInitMatrix" called by "ParallelInitMatrix" called by "EliminateDircihlet"
There the command

RowOwner = ParallelInfo % NeighbourList(RowInd) % Neighbours(1)

segfaults because this variable is not allocated.

The reason for this,as i believe, is that "paralllinimatrix" assumes that every node has all its DOFS represented in the matrix that is inputted by "eliminateDirichletmatrix".
A hint for that is allready the beginning lines of parallelinitmatrix

Code: Select all

 
Perm => Solver % Variable % Perm
      n = SIZE(Perm)
       k = n*DOFs + Matrix % ExtraDOFs
       ALLOCATE( Matrix % Perm(k), Matrix % InvPerm(k))

And also later on it uses the same idea for looping through the entries

Code: Select all

       Matrix % Perm = 0
       DO i=1,n
         IF ( Perm(i) /= 0 )  THEN
            DO j=1,DOFs
               Matrix % Perm((i-1)*DOFs+j) = DOFs * (Perm(i)-1) + j
            END DO
         END IF
       END DO

In many cases that does not apply. For example if only one dof of a vector field is set by a dirichlet value then EliminateDirichlet takes out only the row for one dof and not for all the dofs.

So if anybody has a working solution for this, a good idea, a comment or whatever please write back. I tried to fix it myself but whatever exactly happens in splitmatrix is a bit too rough for me right now.

best Regards

Franz Pichler