Trouble with Mesh Interpolation

Numerical methods and mathematical models of Elmer
Post Reply
edge025
Posts: 18
Joined: 27 Apr 2019, 08:48
Antispam: Yes

Trouble with Mesh Interpolation

Post by edge025 » 07 Oct 2019, 22:44

Hello,

I have a model that contains 6 water pipes and air, along with a host of other non-CFD components. I have run Naviers-Stokes solvers on each of the CFD components separately... when I say separately I mean COMPLETELY separately (even meshes that JUST contain the CFD components). I have saved the results in .dat files for each. So I have

Water-componet1 (partitioned mesh and result files) - I ran in parallel on 64 partitions
Water-componet2 (partitioned mesh and result files) - I ran in parallel on 64 partitions
Water-componet3 (partitioned mesh and result files) - I ran in parallel on 64 partitions
Water-componet4 (partitioned mesh and result files) - I ran in parallel on 64 partitions
Water-componet5 (partitioned mesh and result files) - I ran in parallel on 64 partitions
Water-componet6 (partitioned mesh and result files) - I ran in parallel on 64 partitions
Air-component (partitioned mesh and result files) - I ran in parallel on 64 partitions

Ok. Then I am trying to "merge" or interpolate the results above into the big model with the non-CFD components (global mesh) as well (partitioned into 64 as well) using a Body Force like this

Body Force 5
Name = "NS Water Variable Imports"
Velocity 1 = Equals Velocity-water-vol8 1
Velocity 2 = Equals Velocity-water-vol8 2
Velocity 3 = Equals Velocity-water-vol8 3
Pressure = Equals Pressure-water-vol8
End

However, when it comes time to do anything I get errors after BuildQuadrantTree Ready. Is it because the elements in the global mesh and the component mesh are not in the same partition (they were all partitioned individually using ElmerGrid)

Regards,
Joe
Attachments
master.sif
(3.79 KiB) Downloaded 4 times
master-Body Forces.sif
(1.65 KiB) Not downloaded yet
master-watervars-import.sif
(7.6 KiB) Downloaded 2 times

edge025
Posts: 18
Joined: 27 Apr 2019, 08:48
Antispam: Yes

Re: Trouble with Mesh Interpolation

Post by edge025 » 07 Oct 2019, 22:45

more attachments
Attachments
master-airvars-import.sif
(1.2 KiB) Downloaded 1 time
master-BCs.sif
(263 Bytes) Downloaded 1 time
master-Bodies.sif
(9.76 KiB) Downloaded 1 time

raback
Site Admin
Posts: 3461
Joined: 22 Aug 2009, 11:57
Antispam: Yes
Location: Espoo, Finland
Contact:

Re: Trouble with Mesh Interpolation

Post by raback » 08 Oct 2019, 01:21

Hi Joe,

I really respect your effort to make such a complex simulation. It may be stretching the limits of the code...

If I understand correctly your pipes are not joined so you can save some resources by computing them separately when you peform a hierarchical thermal simulation (no feedback). I have no objections to that. So the challenge here is how to gather all the 7 Navier-Stokes results to the global mesh.

Basically when you request a field that is not defined in the mesh the code find the mesh that owns these results and then does the interpolation. Octree data structure is used since it allows for n*LOG(n) scaling. In principle the interpolation is also implemented in parallel even though it is much more labirious then.

Personally I've not been as ambitious as you. I have only interpolated between two meshes. I could see several reasons why this could fail
  • There is maybe some structures that are not allocated/deallocated when doing new interpolation. Does this fail for the 1st interpolation already.
  • It would be prettier if all the 7 NS meshes would have the standard field name. However, you correctly realized that the interpolation would be then done only for the 1st piece. For this kind of simulation it would be good to allow interpolation over all meshes.
  • If you initialize the solver sections with the NS equation you will use excess memory since partitions will allocate some structures. You could perhaps use just a dummy AllocateSolver() module to reserve the space and do the restart.
Sorry, this is probably not very helpfull. The case is just quite ambitious for a 10-minute analysis. An output log would be helpful.

-Peter

edge025
Posts: 18
Joined: 27 Apr 2019, 08:48
Antispam: Yes

Re: Trouble with Mesh Interpolation

Post by edge025 » 08 Oct 2019, 06:00

Thanks for your answer, Peter. The rational is that I am only interested in the end result of temperature. I wanted to be able to refine the temperature mesh and still be able to import in the naviers-stokes results (since their accuracy is not paramount, neither are they dependent on temperature) so I would not have to rerun the navier-stokes solvers (due to time).

I am still trying to get this to work. In my current tests, I have cut out the temperature solver and am only trying to import one single variable from a NV solution. I do not seem to get a MPI communication error anymore, but after "BuildQuandrantTree: Ready" there doesn't seem to be any further progress. I've let Elmersolver run for probably close to 30min now with no movement. Any guidance appreciated.

Would a Variable Export and then assigning variables using expressions in Body Forces section work any different?

Regards,
Joe
Attachments
master.sif
(3.91 KiB) Downloaded 1 time
master-Bodies.sif
(9.76 KiB) Downloaded 1 time
master-Body Forces.sif
(1.68 KiB) Downloaded 1 time
Last edited by edge025 on 08 Oct 2019, 07:28, edited 1 time in total.

edge025
Posts: 18
Joined: 27 Apr 2019, 08:48
Antispam: Yes

Re: Trouble with Mesh Interpolation

Post by edge025 » 08 Oct 2019, 06:00

solver log
Attachments
solver.log
(108.34 KiB) Downloaded 3 times

edge025
Posts: 18
Joined: 27 Apr 2019, 08:48
Antispam: Yes

Re: Trouble with Mesh Interpolation

Post by edge025 » 08 Oct 2019, 06:35

Weird. Even though the log file shows that all variables were reloaded or "restart" for all solvers, when I try to restart one of the solvers that has good navier-stokes data, the solver acts like it doesn't have the variables loaded (it does not pick up where it left off). Any ideas please how to fix this?

raback
Site Admin
Posts: 3461
Joined: 22 Aug 2009, 11:57
Antispam: Yes
Location: Espoo, Finland
Contact:

Re: Trouble with Mesh Interpolation

Post by raback » 08 Oct 2019, 11:28

Hi

Difficult to say. I would make a dummy solver that would do the interpolation. In principle you could do something like this

Code: Select all

  VarSum => VariableGet( Mesh % Variables,'Velocity 1')
  Var1 => VariableGet( Mesh % Variables,'Velocity-water-vol1')
  Var2 => VariableGet( Mesh % Variables,'Velocity-water-vol2')
  ...
  VarSum % Values = Var1 % Values + Var2 % Values  ! actually this probably must consider permutations 
The interpolation from mesh vol1 is done automatically to your current mesh when VariableGet is called. The interpolated values will be zero where not found. Hence you need to sum up the pieces to get the correct sum. This kind of small solver would also enable to have better control on where and when the interpolation takes place.

-Peter

edge025
Posts: 18
Joined: 27 Apr 2019, 08:48
Antispam: Yes

Re: Trouble with Mesh Interpolation

Post by edge025 » 08 Oct 2019, 19:06

Thanks for your help, Peter.

Oy...I don't know how to write solvers yet even though it may be "simple." I will have to do this another way for now and then hopefully return to this concept on another project.

Regards,
Joe

raback
Site Admin
Posts: 3461
Joined: 22 Aug 2009, 11:57
Antispam: Yes
Location: Espoo, Finland
Contact:

Re: Trouble with Mesh Interpolation

Post by raback » 09 Oct 2019, 09:57

Hi Joe,

Here's a solver close to what you need:
https://github.com/ElmerCSC/elmerfem/bl ... Solver.F90

Then just elmerf90 -o MySolver.so MySolver.dll

The main difference is that this solver interpolates many variables from one nonconforming mesh whereas you should interpolate the same variable from many noncoforming meshes.

This solver is even unnecesssary complex as there are some checks built in.

-Peter

Post Reply