A Discrete-Event Network Simulator
Home
Tutorials ▼
English
Documentation ▼
Installation
Manual
Models
Contributing
Wiki
Development ▼
API Docs
Issue Tracker
Merge Requests
API
Loading...
Searching...
No Matches
example-as-test.h
Go to the documentation of this file.
1
/*
2
* Copyright (c) 2020 Lawrence Livermore National Laboratory
3
*
4
* SPDX-License-Identifier: GPL-2.0-only
5
*
6
* Author: Peter D. Barnes, Jr. <pdbarnes@llnl.gov>
7
*/
8
9
#ifndef NS3_EXAMPLE_AS_TEST_SUITE_H
10
#define NS3_EXAMPLE_AS_TEST_SUITE_H
11
12
#include "
test.h
"
13
14
#include <string>
15
16
/**
17
* \file
18
* \ingroup testing
19
* Enable examples to be run as meaningful tests.
20
* Declaration of classes ns3::ExampleAsTestSuite and ns3::ExampleAsTestCase.
21
*/
22
23
namespace
ns3
24
{
25
26
/**
27
* \ingroup testing
28
* Execute an example program as a test, by comparing the output
29
* to a reference file.
30
*
31
* User can subclass and override the GetCommandTemplate() and
32
* GetPostProcessingCommand() methods if more complex
33
* example invocation patterns are required.
34
*
35
* \see examples-as-tests-test-suite.cc
36
*/
37
class
ExampleAsTestCase
:
public
TestCase
38
{
39
public
:
40
/**
41
* Constructor.
42
* \param [in] name The test case name, typically the program name
43
* and summary of the arguments, such as `my-example-foo`
44
* \param [in] program The actual example program names, such as `my-example`
45
* \param [in] dataDir The location of the reference file.
46
* This is normally provided by the symbol
47
* `NS_TEST_SOURCEDIR` in the `module-examples-test-suite.cc`
48
* file.
49
* The reference file should be named after
50
* the test case name,
51
* for example `my-example-foo.log`. If you use
52
* the `--update` argument to `test.py` or
53
* `test-runner` the reference file will be created
54
* with the correct name.
55
* \param [in] args Any additional arguments to the program.
56
* \param [in] shouldNotErr Whether an error return status should be
57
* considered a test failure. This is useful when testing
58
* error detection which might return a non-zero status.
59
* The output (on `std::cout` and `std::cerr`) will
60
* be compared to the reference logs as normal.
61
*/
62
ExampleAsTestCase
(
const
std::string name,
63
const
std::string program,
64
const
std::string dataDir,
65
const
std::string args =
""
,
66
const
bool
shouldNotErr =
true
);
67
68
/** Destructor. */
69
~ExampleAsTestCase
()
override
;
70
71
/**
72
* Customization point for more complicated patterns
73
* to invoke the example program.
74
*
75
* \returns The string to be given to the `ns3 --command-template=` argument.
76
*/
77
virtual
std::string
GetCommandTemplate
()
const
;
78
79
/**
80
* Customization point for tests requiring post-processing of stdout.
81
*
82
* For example to sort return `"| sort"`
83
*
84
* One common case is to mask memory addresses, which can change
85
* when things are built on different platforms, recompiled locally,
86
* or even from run to run. A simple post-processing filter could be
87
*
88
* `"| sed -E 's/0x[0-9a-fA-F]{8,}/0x-address/g'"`
89
*
90
* Default is `""`, no additional processing.
91
*
92
* \returns The string of post-processing commands
93
*/
94
virtual
std::string
GetPostProcessingCommand
()
const
;
95
96
// Inherited
97
void
DoRun
()
override
;
98
99
protected
:
100
std::string
m_program
;
/**< The program to run. */
101
std::string
m_dataDir
;
/**< The source directory for the test. */
102
std::string
m_args
;
/**< Any additional arguments to the program. */
103
bool
m_shouldNotErr
;
/**< Whether error return status is a test failure. */
104
105
};
// class ExampleAsTestCase
106
107
/**
108
* \ingroup testing
109
* Execute an example program as a test suite.
110
*
111
* You can use this TestSuite to add an example to the test suite with
112
* checking of the example output (`std::out` and `std::err`). This
113
* is an alternative to adding an example using the
114
* `examples-to-run.py` file. The key difference between the two
115
* methods is what criteria is used to for success. Examples added to
116
* `examples-to-run.py` will be run and the exit status checked
117
* (non-zero indicates failure). ExampleAsTestSuite adds checking of
118
* output against a specified known "good" reference file.
119
*
120
* \warning If you are thinking about using this class, strongly
121
* consider using a standard test instead. The TestSuite class has
122
* better checking using the NS_TEST_* macros and in almost all cases
123
* is the better approach. If your test can be done with a TestSuite
124
* class you will be asked by the reviewers to rewrite the test when
125
* you do a pull request.
126
*
127
* \par Test Addition
128
*
129
* To use an example program as a test you need to create a test suite
130
* file and add it to the appropriate list in your module CMakeLists.txt
131
* file. The "good" output reference file needs to be generated for
132
* detecting regressions.
133
*
134
* Let's assume your module is called `mymodule`, and the example
135
* program is `mymodule/examples/mod-example.cc`. First you should
136
* create a test file `mymodule/test/mymodule-examples-test-suite.cc`
137
* which looks like this:
138
*
139
* \code{.cpp}
140
* #include "ns3/example-as-test.h"
141
* static ns3::ExampleAsTestSuite g_modExampleOne("mymodule-example-mod-example-one",
142
* "mod-example", NS_TEST_SOURCEDIR, "--arg-one");
143
* static ns3::ExampleAsTestSuite g_modExampleTwo("mymodule-example-mod-example-two",
144
* "mod-example", NS_TEST_SOURCEDIR, "--arg-two"); \endcode
145
*
146
* The arguments to the constructor are the name of the test suite, the
147
* example to run, the directory that contains the "good" reference file
148
* (the macro `NS_TEST_SOURCEDIR` is normally the correct directory),
149
* and command line arguments for the example. In the preceding code
150
* the same example is run twice with different arguments.
151
*
152
* You then need to add that newly created test suite file to the list
153
* of test sources in `mymodule/CMakeLists.txt`. Building of examples
154
* is an option so you need to guard the inclusion of the test suite:
155
*
156
* \code{.py}
157
* if (bld.env['ENABLE_EXAMPLES']):
158
* module.source.append('model/mymodule-examples-test-suite.cc')
159
* \endcode
160
*
161
* Since you modified a CMakeLists.txt file you need to reconfigure and
162
* rebuild everything.
163
*
164
* You just added new tests so you will need to generate the "good"
165
* output reference files that will be used to verify the example:
166
*
167
* `./test.py --suite="mymodule-example-*" --update`
168
*
169
* This will run all tests starting with "mymodule-example-" and save
170
* new "good" reference files. Updating the reference file should be
171
* done when you create the test and whenever output changes. When
172
* updating the reference output you should inspect it to ensure that
173
* it is valid. The reference files should be committed with the new
174
* test.
175
*
176
* \par Test Verification
177
*
178
* You can run the test with the standard `test.py` script. For
179
* example to run the suites you just added:
180
*
181
* `./test.py --suite="mymodule-example-*"`
182
*
183
* This will run all `mymodule-example-...` tests and report whether they
184
* produce output matching the reference files.
185
*
186
* \par Writing good examples for testing
187
*
188
* When setting up an example for use by this class you should be very
189
* careful about what output the example generates. For example,
190
* writing output which includes simulation time (especially high
191
* resolution time) makes the test sensitive to potentially minor
192
* changes in event times. This makes the reference output hard to
193
* verify and hard to keep up-to-date. Output as little as needed for
194
* the example and include only behavioral state that is important for
195
* determining if the example has run correctly.
196
*
197
*/
198
class
ExampleAsTestSuite
:
public
TestSuite
199
{
200
public
:
201
/**
202
* \copydoc ExampleAsTestCase::ExampleAsTestCase
203
* \param [in] duration Amount of time this test takes to execute
204
* (defaults to QUICK).
205
*/
206
ExampleAsTestSuite
(
const
std::string name,
207
const
std::string program,
208
const
std::string dataDir,
209
const
std::string args =
""
,
210
const
Duration
duration =
Duration::QUICK
,
211
const
bool
shouldNotErr =
true
);
212
};
// class ExampleAsTestSuite
213
214
}
// namespace ns3
215
216
#endif
/* NS3_EXAMPLE_TEST_SUITE_H */
ns3::ExampleAsTestCase
Execute an example program as a test, by comparing the output to a reference file.
Definition
example-as-test.h:38
ns3::ExampleAsTestCase::GetCommandTemplate
virtual std::string GetCommandTemplate() const
Customization point for more complicated patterns to invoke the example program.
Definition
example-as-test.cc:55
ns3::ExampleAsTestCase::m_shouldNotErr
bool m_shouldNotErr
Whether error return status is a test failure.
Definition
example-as-test.h:103
ns3::ExampleAsTestCase::GetPostProcessingCommand
virtual std::string GetPostProcessingCommand() const
Customization point for tests requiring post-processing of stdout.
Definition
example-as-test.cc:64
ns3::ExampleAsTestCase::DoRun
void DoRun() override
Implementation to actually run this TestCase.
Definition
example-as-test.cc:72
ns3::ExampleAsTestCase::ExampleAsTestCase
ExampleAsTestCase(const std::string name, const std::string program, const std::string dataDir, const std::string args="", const bool shouldNotErr=true)
Constructor.
Definition
example-as-test.cc:35
ns3::ExampleAsTestCase::m_args
std::string m_args
Any additional arguments to the program.
Definition
example-as-test.h:102
ns3::ExampleAsTestCase::m_dataDir
std::string m_dataDir
The source directory for the test.
Definition
example-as-test.h:101
ns3::ExampleAsTestCase::m_program
std::string m_program
The program to run.
Definition
example-as-test.h:100
ns3::ExampleAsTestCase::~ExampleAsTestCase
~ExampleAsTestCase() override
Destructor.
Definition
example-as-test.cc:49
ns3::ExampleAsTestSuite
Execute an example program as a test suite.
Definition
example-as-test.h:199
ns3::ExampleAsTestSuite::ExampleAsTestSuite
ExampleAsTestSuite(const std::string name, const std::string program, const std::string dataDir, const std::string args="", const Duration duration=Duration::QUICK, const bool shouldNotErr=true)
Constructor.
Definition
example-as-test.cc:140
ns3::TestCase
encapsulates test code
Definition
test.h:1050
ns3::TestCase::Duration
Duration
How long the test takes to execute.
Definition
test.h:1054
ns3::TestCase::Duration::QUICK
@ QUICK
Fast test.
ns3::TestSuite
A suite of tests to run.
Definition
test.h:1267
ns3
Every class exported by the ns3 library is enclosed in the ns3 namespace.
test.h
ns3::TestCase, ns3::TestSuite, ns3::TestRunner declarations, and NS_TEST_ASSERT macro definitions.
src
core
model
example-as-test.h
Generated on Fri Nov 8 2024 13:58:59 for ns-3 by
1.11.0