Hello,
I am trying to solve a mesh with over 1 billion 3d elements (partitioned over 120 parts). In ElmerSolver_mpi I am getting load mesh errors consistent with integer types perhaps being overrun. Is there a simple way to compile with LONG INT support?
Regards,
Joe
Trouble with mesh of over 1 billion elements

 Site Admin
 Posts: 3464
 Joined: 22 Aug 2009, 11:57
 Antispam: Yes
 Location: Espoo, Finland
 Contact:
Re: Trouble with mesh of over 1 billion elements
Hi Joe,
Nice!
Unfortunately current the integer type is 32 bits for all integers which means max value of 2,147,483,647. So I could see indexing hitting a wall here.
We are aware of the problem but so far there has been no reason to use long integers. It would probably make sense to only alter the global indexing since it will be some time until the local indexing will hit this limit. The global indexing is more localized in the code. Altering all INTEGERs would loose some memory/efficiency in vain.
Are you really solving this large problems. For use the limit has been hit only in benchmarking of Poisson type of equations.
Peter
Nice!
Unfortunately current the integer type is 32 bits for all integers which means max value of 2,147,483,647. So I could see indexing hitting a wall here.
We are aware of the problem but so far there has been no reason to use long integers. It would probably make sense to only alter the global indexing since it will be some time until the local indexing will hit this limit. The global indexing is more localized in the code. Altering all INTEGERs would loose some memory/efficiency in vain.
Are you really solving this large problems. For use the limit has been hit only in benchmarking of Poisson type of equations.
Peter
Re: Trouble with mesh of over 1 billion elements
Thanks for getting back to me, Peter. I know gfortran has a commandline option to use 8byte integers instead of 4byte. Would this make my troubles go away? If so, where can I tell elmer to compile with this option? would anything be needed for mpi?
Are fortran integers the only culprets? Or is there C code as well (I hope not)!
Joe
Are fortran integers the only culprets? Or is there C code as well (I hope not)!
Joe

 Site Admin
 Posts: 3464
 Joined: 22 Aug 2009, 11:57
 Antispam: Yes
 Location: Espoo, Finland
 Contact:
Re: Trouble with mesh of over 1 billion elements
Hi Joe,
How do you create your mesh. I hope internally with "Mesh Levels"? Then the preprocessing with ElmerGrid should not lead to any problems.
There are a number of linear solver libraries and I don't know which of them would work outofbox. Probably it would be easiest to start with the internal iterative solvers of Elmer.
May I ask what is such a huge case that you're hitting the integer length?
Peter
How do you create your mesh. I hope internally with "Mesh Levels"? Then the preprocessing with ElmerGrid should not lead to any problems.
There are a number of linear solver libraries and I don't know which of them would work outofbox. Probably it would be easiest to start with the internal iterative solvers of Elmer.
May I ask what is such a huge case that you're hitting the integer length?
Peter
Re: Trouble with mesh of over 1 billion elements
Peter,
Thank you for your suggestions! Sorry it took so long to get back to you. I am running a coupled cfd and thermal model on two magnetics structures in a large cabinet being cooled by both water and air.
Fortunately (and unfortunately) I have given up at the present at getting such a large mesh to run. After reading your suggestions above along with that you have suggested to other people in other posts, I have been able to pursue a much better path to solution as well as learn much more about Elmer! Thank you again for such valuable help!
Regards,
Joe
Thank you for your suggestions! Sorry it took so long to get back to you. I am running a coupled cfd and thermal model on two magnetics structures in a large cabinet being cooled by both water and air.
Fortunately (and unfortunately) I have given up at the present at getting such a large mesh to run. After reading your suggestions above along with that you have suggested to other people in other posts, I have been able to pursue a much better path to solution as well as learn much more about Elmer! Thank you again for such valuable help!
Regards,
Joe