optimization of Scanning Helmholtz Solver

[Sergejs]

Hello I'm a guy who tends to make complex audio tasks more accessible - https://www.youtube.com/@Sergejs_on_sound

And right now I am learning from scratch Elmer acoustic simulations based on https://computational-acoustics.gitlab. ... dio-part-4 tutorials . Starting with just re-creating the same simulation from scratch, learning the tools and fixing a-Lot of problems in the process

Me being me, I am learning, writing a future tutorial and optimizing the process simultaneously. So would like to ask for help with some optimizations:

q 1.
I do not manage to correctly use "**Exported Variable**". To analyze results in ParaView I need to pass on Frequency of the scanning step into ParaView. I read about Exported Variable in the manual page 62 in https://www.nic.funet.fi/pub/sci/physic ... Manual.pdf and wrote a couple of commands to try to export simulation frequency of a current step into .vtu result file. Not surprisingly it does not work Note, the author of the original tutorial uses extra Julia code to externally generate frequency data files for ParaView - there must be faster way to export this data straight from Elmer.

Code: Select all

  Exported Variable 1 = Frequency
  Exported Variable 1 DOFs = 1

q 2.
Is there any impact on the result quality if mpiexec.exe is used with standard "ElmerGrid 2 2 %msh -partdual -metiskway %n" for parallel processing? The reason I am asking is:

• I saw many recommendations to avoid using MPI and that it can cause visible artifacts in ParaView result visualizations - is it true?

• Theoretically "Scanning" simulations over frequency should be parallelized by simply providing one frequency from the list to simulate for each of the solver instances. In fact I once coded such job-distribution and resource monitoring for Mesh2HRTF https://sourceforge.net/p/mesh2hrtf-too ... c_manager/ . In case current splitting of mesh MPI approach has no major downsides, then there is no need to add parallelization based on frequencies.

More question to come

[kevinarden]

I believe Exported Variable means that the solver exports the variable back to the general Elmer solution routine, what that variable is used for is up to the user. It does not mean export to the results file.

when you use
Post File = case.vtu

in the simulation section you are getting the default output for each solver in the vtu file.

If you want something different in the vtu file, for example the exported variable from solver 1 than you have to specify how the output is to be written
there is a results output solver on page 296
https://www.nic.funet.fi/pub/sci/physic ... Manual.pdf

where you can take control of what is output.

[kevinarden]

q2
You could set up a script to create 5 different sif files with 5 different output names and submit all five jobs. This would be equivalent to not splitting the mesh and running frequencies in parallel.

[Sergejs]

I believe Exported Variable means that the solver exports the variable back to the general Elmer solution routine, what that variable is used for is up to the user. It does not mean export to the results file.

when you use
Post File = case.vtu

in the simulation section you are getting the default output for each solver in the vtu file.

If you want something different in the vtu file, for example the exported variable from solver 1 than you have to specify how the output is to be written
there is a results output solver on page 296
https://www.nic.funet.fi/pub/sci/physic ... Manual.pdf

where you can take control of what is output.

Thanks, I need to experiment with this - perhaps "Saving Scalar Values to a File" page 284
https://www.nic.funet.fi/pub/sci/physic ... Manual.pdf is even more suitable?

Right now I made a primitive workaround - I just simulate from 0.1, 1 , 2, 3... [Hz] so that time step numbering matches actual frequency value - primitive but I can use this for the moment.

[Sergejs]

q2
You could set up a script to create 5 different sif files with 5 different output names and submit all five jobs. This would be equivalent to not splitting the mesh and running frequencies in parallel.

Not sure this would be convenient. How would ParaView merge the results from 5 simulations into one "time line"? Also that is not easy to run for every simulation. All of this reminds me too much of the state of Mesh2HRTF simulation execution in 2021:
- there was no way to monitor available resources to maximize number of running instances
- there was no way to re-start an interrupted simulation without loosing all the already-simulated frequency steps.
- scanning simulation started from the simple, low-frequencies and you found out about convergence problems only at the end of the simulation - effectively no early warning about upcoming problems.

So either i don't know how to use more advanced features, or there is a bit performance improvement potential for Elmer frequency scanning simulations.


Question3 - is it possible to ask Elmer to compute just a specific time step?
for example if I have:

Code: Select all

Timestep intervals = 151

then How do I ask solver to ONLY compute timestep "150"? (and get result file called "case_t0150.vtu")

(if it is possible, then I could write Python script to selectively simulate only selected frequencies - to resume interrupted simulations, to optimize resource usage, to start simulation from higher frequencies, to include more/less frequencies into the simulation - a lot of flexibility).

[kevinarden]

There is a restart capability built in to support resuming simulations
Chapter 3
https://www.nic.funet.fi/pub/sci/physic ... Manual.pdf

ParaView would merge the results from 5 simulations into one "time line" if you controlled the name of the output files such as paraview would think they were 5 different steps. I have also used scripts to rename vtu files so that paraview would think they were a time line.

If you had a table of frequencies you could have it run 150 by doing
Timestep size = 150
timestep interval = 1
Post File = "case_150.vtu"

[raback]

Hi

To split the analysis try this:

Code: Select all

$npart=5
$dn=10
$n0=(npart-1)*dn
Simulation 
...
  Simulation type = "scanning"
  timestep intervals = $dn
  Post File = "case.vtu"
  vtu: fileindex offset = $n0
...
Frequency = Variable "time"
  Real MATC "f0*1.1^(n0+tx-1)"

Solver n
  Procedure = "SaveData" "SaveScalars"
  Filename = f$npart$.dat
  ...

Alternating value of $npart you can run different part of the frequency sweep and catenate results
cat f*.dat > f.dat

-Peter

[raback]

Generally it is not straight forward to answer what is the best use of resources when making the sweep. I have not heard that VTU output problem is any of these.

For Helmholtz equation the preconditioning may sometimes be tricky in parallel leading to more linear system iterations. Hence if you have enough memory it may be faster to have several serial jobs run at the same time. On the other hand, if the parallel performance remains good there may be no reason for this. When you use one simulation, the linear system often has a decent initial guess as you're starting from the previous frequency nearby so using iterative solvers may keep the performance quite good over the frequency sweep.

-Peter

[Sergejs]

Sorry, I will read the comments tomorrow, but here is a plot from a room simulation where you can clearly count 10 cells from the "ElmerGrid 2 2 %msh -partdual -metiskway %n" command:


So now it is quite obvious that MPI partitioning of mesh is not harmless.

[raback]

To me that seems just paraview artefact of volume rendering. I seldom use that myself. I guess cuts are ok?