ElmerSolver test case how-to

Numerical methods and mathematical models of Elmer
Post Reply
raback
Site Admin
Posts: 3291
Joined: 22 Aug 2009, 11:57
Antispam: Yes
Location: Espoo, Finland
Contact:

ElmerSolver test case how-to

Post by raback » 25 Oct 2013, 22:44

[This is a cross posting from the tests directory of Elmer repository. Some people never even end up there so I thought that it could be more easily found here. I think that credits of writing the instructions go to Eelis but I might remember incorrectly...]


ElmerSolver test case how-to
========================

This How-to describes how to create and use test cases in software
development and case definition.

The test cases are used to check for possible bugs, and to ensure
consistancy and backward compability. For that purpose they are
currently around 160 (Feb 2011) minimalistic cases the run-time of
each is typically measured in seconds. The tests should be located in
directory

$ELMERHOME/tests

To run the cases in Unix/mingw you can go the the directory tests and say

Make check



A. How to create a test case
---------------------------------

When a new feature has been verified it is often a good idea to
create a mimalistic test case that ensures the feature will be
maintained also in the future versions of Elmer. To do this:

1) create new directory in the "tests" directory under fem/. The folders
in this directory will be automatically scanned and tests run

2) define your analysis write your .sif file, mesh files
etc. necessary to run the test analysis ELMERSOLVER_STARTINFO

3) define how to run the test write a Makefile (named Makefile), that
details two targets: run: how to run the test clean: how to
clean up if successfull Note, that ElmerSolver should be
referred to by variable $(ELMER_SOLVER) to allow the test
runner to select the binary to test

4) Define expected results Add the following lines to your .sif file

$fprintf( stderr, "TEST CASE 1\n"); RUN $fprintf( stderr, "END
TEST CASE 1: Target NRM=<target>,EPS=<eps>\n" );

where you replace <target> with the target NRM value and <eps>
with the desired tolerance. These lines will add lines to
output and the test runner find these and compare actual
values emitted by the solver with the specified target. The
NRM value is the norm of the result vector. Normal way to
determine desired value is to examine the analysis results in
detail the first time and if the results are OK, record the
NRM value and use it as target value in the future.

5 ) If you have access right to the svn you may also

svn add mytestcase
svn commit mytestcase

If not, contact somebody who has access rights.


B. How to use the test case as starting point
-----------------------------------------------------

One can take these examples as a starting point or copy-paste
appropriate solver sections from there. These examples do not include
any ElmerGUI project file and therefore the graphical user interface
cannot be used to view these cases. You can look at the Makefile in
each test to see what sequence of command is usually needed. Typically
some simple ascii definition for the mesh is used and the running of
the case could be something like

ElmerGrid 1 2 square
ElmerSolver

Now to get more out of the test the user may add verbosity of
ElmerSolver and define an output file in the Simulation section

Max Output Level = 10
Post File = case.ep

Also, to increase the problem size one can increase the mesh density,
for example, by

ElmerGrid 1 2 square -relh 0.5

rgladstone
Posts: 61
Joined: 15 Apr 2013, 16:23
Antispam: Yes

Re: ElmerSolver test case how-to

Post by rgladstone » 26 Nov 2014, 04:09

Hi Peter and co, should we expect all tests to complete successfully? I am running the tests now because I plan some minor commits soon and I want to check I haven't broken anything. When I run the tests I get a number of individual fails but at the end it says the test has passed successfully... is this how it should look?

Thanks,
Rupert

Code: Select all

...
test 261 :                   Step_ke 		[PASSED], CPU time= 371.7    
test 262 :                   Step_ns 		[PASSED], CPU time= 372.0    
test 263 :                   Step_sa 		[PASSED], CPU time= 377.7    
test 264 :            Step_sst-kw-wf 		[PASSED], CPU time= 388.1    
test 265 :               Step_stokes 		[PASSED], CPU time= 388.2    
test 266 :         Step_stokes_block 		[PASSED], CPU time= 388.3    
test 267 :                  Step_v2f 		[PASSED], CPU time= 402.3    
test 268 :                StokesPFEM 		[PASSED], CPU time= 402.4    
test 269 :                StokesProj 		[PASSED], CPU time= 402.7    
test 270 :       StrainCalculation01 		[PASSED], CPU time= 402.9    
test 271 :       StrainCalculation02 		[PASSED], CPU time= 406.5    
test 272 :       StrainCalculation03 		[PASSED], CPU time= 410.4    
test 273 :               streamlines 		[PASSED], CPU time= 410.6    
test 274 :                    stress 		[PASSED], CPU time= 410.9    
test 275 :               StressEigen [FAILED] Computed NRM=.994029E+007 Target NRM=121834., look at [StressEigen/test.log] for details
test 276 :             StressEigen3D [FAILED] Computed NRM=-.271698E-006 Target NRM=163190., look at [StressEigen3D/test.log] for details
test 277 :                 structmap 		[PASSED], CPU time= 425.0    
test 278 :                structmap2 		[PASSED], CPU time= 425.9    
test 279 :                structmap3 		[PASSED], CPU time= 434.9    
test 280 :     StructuredHeightDepth 		[PASSED], CPU time= 437.0    
test 281 :           ThermalActuator [FAILED] Computed NRM=.614332E-006 Target NRM=.615048E-006, look at [ThermalActuator/test.log] for details
test 282 :            ThermalBiMetal 		[PASSED], CPU time= 477.9    
test 283 :           ThermalBiMetal2 		[PASSED], CPU time= 478.0    
test 284 :           ThermalCompress 		[PASSED], CPU time= 478.6    
test 285 :            ThermoElectric 		[PASSED], CPU time= 478.7    
test 286 :                 TimeAdapt [FAILED] Computed NRM=.809879E-002 Target NRM=.787856E-002, look at [TimeAdapt/test.log] for details
test 287 :                  TimeFunc 		[PASSED], CPU time= 480.6    
test 288 :                    tresca 		[PASSED], CPU time= 480.7    
test 289 :                  vortex2d 		[PASSED], CPU time= 481.4    
test 290 :                  vortex3d 		[PASSED], CPU time= 482.3    
test 291 :                   WaveEqu 		[PASSED], CPU time= 482.4    
test 292 :         WeightComputation 		[PASSED], CPU time= 482.5    
Tests completed, passed: 221 out of total 292 tests
Cumulative CPU time used in test:  482.5     s
PASS: runtests
==================
All 1 tests passed
==================
make[1]: Leaving directory `/home/svali-user/Work/Source/trunk/fem/tests'

rgladstone
Posts: 61
Joined: 15 Apr 2013, 16:23
Antispam: Yes

Re: ElmerSolver test case how-to

Post by rgladstone » 26 Nov 2014, 05:08

Hang on, I had a link that pointed to an older version of ElmerSolver. I am re-running now with svn version 7029. I get fewer fails, but still not all tests pass. I'll post again when it finishes...

Cheers,
Rupert

rgladstone
Posts: 61
Joined: 15 Apr 2013, 16:23
Antispam: Yes

Re: ElmerSolver test case how-to

Post by rgladstone » 26 Nov 2014, 05:41

So, much better than before, nearly all pass. Should I be concerned about the few that fail? Or does this look acceptable?

Code: Select all

...
test 151 :                 levelset3 [FAILED] Computed NRM=.373585E-001 Target NRM=.503468E-001, look at [levelset3/test.log] for details
...
test 171 : mgdyn_lamstack_lowfreq_harmonic [FAILED] Computed NRM=.527528E-001 Target NRM=.526985E-001, look at [mgdyn_lamstack_lowfreq_harmonic/test.log] for details
...
test 175 :     mgdyn_steady_periodic [FAILED] Computed NRM=.221406E-002 Target NRM=.205527E-002, look at [mgdyn_steady_periodic/test.log] for details
...
test 282 :            ThermalBiMetal 		[PASSED], CPU time= 1504.    
test 283 :           ThermalBiMetal2 		[PASSED], CPU time= 1504.    
test 284 :           ThermalCompress 		[PASSED], CPU time= 1506.    
test 285 :            ThermoElectric 		[PASSED], CPU time= 1506.    
test 286 :                 TimeAdapt [FAILED] Computed NRM=.790666E-002 Target NRM=.787856E-002, look at [TimeAdapt/test.log] for details
test 287 :                  TimeFunc 		[PASSED], CPU time= 1515.    
test 288 :                    tresca 		[PASSED], CPU time= 1515.    
test 289 :                  vortex2d 		[PASSED], CPU time= 1517.    
test 290 :                  vortex3d 		[PASSED], CPU time= 1519.    
test 291 :                   WaveEqu 		[PASSED], CPU time= 1519.    
test 292 :         WeightComputation 		[PASSED], CPU time= 1519.    
Tests completed, passed: 288 out of total 292 tests
Cumulative CPU time used in test:  1519.     s
PASS: runtests
==================
All 1 tests passed
==================
make[1]: Leaving directory `/home/svali-user/Work/Source/trunk/fem/tests'

raback
Site Admin
Posts: 3291
Joined: 22 Aug 2009, 11:57
Antispam: Yes
Location: Espoo, Finland
Contact:

Re: ElmerSolver test case how-to

Post by raback » 26 Nov 2014, 10:36

Hi

That's ok. There are currently three tests that fail on me also. The norms may be too tight if the case is sensitive to the tolerances etc. Still of course we should have all pass, as they are called consisntency tests.

-Peter

Post Reply