Page 1 of 1
Semi-lagrangian with particule tracking
Posted: 28 Jun 2022, 23:55
by AdrienG
Hello,
I am trying to use the semi-Lagrangian solver to advect a variable while modeling ice-flow. My setup works fine in serial but in parallel the solver got stuck (no error or crash but infinite loop). It seems that the issue come from the ChangeParticulePartition routine or the LocateParticule routine. My understanding is that when a particle changes of partition, it is not correctly deleted from the original partition ?
Can I get some help on this ?
Attached my setup on 4 partitions.
Thanks a lot,
Adrien
Re: Semi-lagrangian with particule tracking
Posted: 03 Jul 2022, 17:07
by Rich_B
Hello,
Would you please upload your serial version, the working version?
If you could also state the command you used to create the partitions, that would help.
Thanks, Rich.
Re: Semi-lagrangian with particule tracking
Posted: 04 Jul 2022, 15:18
by AdrienG
Hello Rich,
Thanks a lot for your answer. Attached the serial version which is working. To create the partition i simply used the command :
Code: Select all
ElmerGrid 2 2 taco3D -autoclean -metis 4
where taco3D is an existing serial mesh. I tried different things to makes the partition but the problem remains the same.
Thanks again for your help,
Adrien
Re: Semi-lagrangian with particule tracking
Posted: 04 Jul 2022, 18:43
by raback
Hi
The advector has several modes. By default we have nodal variables used for advecting. However, any of the following keywords (just one) overwrites it:
Code: Select all
Advect Elemental = Logical True
Advect Ip = Logical True
Advect Dg = Logical True
You could try any of these. The elemental is just one point in each element but the others have several dofs for each element thereby maintaining the accuracy.
The problem in parallel is that the particles that are initially at the nodes may be exactly at the partition interface and then the algorhitms that try to locate them end up to some kind of loop. This is not observed as easily if the particles are not at the interface to start with. Of course this issue should be resolved as well...
-Peter
Re: Semi-lagrangian with particule tracking
Posted: 04 Jul 2022, 20:19
by AdrienG
Hi,
I tried the different modes but this does not seem to solve the issue...
It seems that the solver is looping because some particles should be deleted from the partition but in fact are not deleted. The ChangeParticlePartition is looping on this :
Code: Select all
ChangeParticlePartition: Sending particles among partitions
ChangeParticlePartition: Number of active partitions: 2
ChangeParticlePartition: Number of particles to send: 17
ChangeParticlePartition: Global number of particles to sent: 46
ChangeParticlePartition: Number of particles to receive: 4
ChangeParticlePartition: Global number of particles to receive: 46
ChangeParticlePartition: Total number of particles to sent: 17
ChangeParticlePartition: Collected particles from partitions: 17
ChangeParticlePartition: Transferring real entries between particles: 15
ChangeParticlePartition: Transferring integer entries between particles: 2
ChangeParticlePartition: Size of data buffer: 1118
ChangeParticlePartition: Now sending particle data
ChangeParticlePartition: Now receiving particle data
DeleteLostParticles: Number of active particles: 5701
DeleteLostParticles: No particles need to be deleted
ChangeParticlePartition: Information exchange done
ChangeParticlePartition: Sending particles among partitions
ChangeParticlePartition: Number of active partitions: 2
ChangeParticlePartition: Number of particles to send: 4
ChangeParticlePartition: Global number of particles to sent: 46
ChangeParticlePartition: Number of particles to receive: 17
ChangeParticlePartition: Global number of particles to receive: 46
ChangeParticlePartition: Total number of particles to sent: 4
ChangeParticlePartition: Collected particles from partitions: 4
ChangeParticlePartition: Transferring real entries between particles: 15
ChangeParticlePartition: Transferring integer entries between particles: 2
ChangeParticlePartition: Size of data buffer: 1092
ChangeParticlePartition: Now sending particle data
ChangeParticlePartition: Now receiving particle data
DeleteLostParticles: Number of active particles: 5701
DeleteLostParticles: No particles need to be deleted
ChangeParticlePartition: Information exchange done
It seems to me that there is an issue with the DeleteLostParticles routine ?
Adrien
Re: Semi-lagrangian with particule tracking
Posted: 13 Jul 2022, 11:47
by AdrienG
Hi,
So I could not properly fix the issue. For now, I just did a little hack for limiting the number of iteration in the LocateParticle subroutine since the function ChangeParticlePartition never return zero in my case. See line 4288 in ParticleUtils.F90.
This allows the solver to run but some little bugs are present when the advected field cross a partition.
I will keep working like this for now but that would be helpful to find out what's wrong ? Otherwise the semi-lagrangian solver is a great improvement for transport problem. Thanks for the work done!
Cheers,
Adrien
Re: Semi-lagrangian with particule tracking
Posted: 04 Aug 2022, 17:22
by raback
Hi,
I had a look at the case. It seems that some particles that were not even moving resulted to an eternal loop trying to settle the owner partition. I tried to fix that. I hope the results are now better, or at least the wrong results are obtained faster as we are not in an eternal loop
The changes are in “devel” branch since a few minutes.
-Peter