htf.main() — Testscript utility¶
htf.main()
is used to start tests if not using htf
(the command-line-utility) within a test-script.
-
class
htf.
main
(title=u'Testreport', html_report=None, junit_xml_report=None, json_report=None, extra_report=None, tests=u'__main__', pattern=u'test*.py', exit=True, verbosity=1, failfast=None, catchbreak=None, argv=None, parse_args=False)¶ The htf.main() and
htf
command-line utility are used to run a set of tests.Parameters: - title (str) – the test’s title
- html_report=None (str or list of str) – HTML-report filename(s)
- junit_xml_report=None (str or list of str) – JUnit-XML-report filename(s)
- json_report=None (str or list of str) – JSON-report filename(s)
- extra_report=None (instance or list of instances) – Extra reports
that support instance.render(data). This is useful for
htf.DOORSTestReport
for example that needs additional parameters. - tests=None (test-specifier or list of test-specifiers) – a test-specifier ora list of test-specifiers (folder, file, test-case, test-case-method, module)
- pattern="test*.py" (str) – the pattern to be used when searching for python files in folders
- exit=True (bool) – if set to
True
the exit() is called at the end of the tests - verbosity=1 (int) – the verbosity level (the higher the level the more verbose is the output)
- failfast=None (bool) – if set to
True
the test run ends after the first failing test - catchbreak=None (bool) – if set to
True
CTRL-C is catched wile tests run. If CTRL-C is pressed the test run is stopped after the current test. - argv=None – the arguments to be parsed (useful for testing).
If set to
None
sys.argv
is used instead. - parse_args=False – if set to
True
argv
orsys.argv
is parsed using argparse (useful as a command-line utility)
Running tests¶
This section show some usage examples.
The first example covers the basic usage for a simple test-case running on Python 2.7 and 3.5+.
from __future__ import \
absolute_import, division, print_function, unicode_literals
import htf
class ExampleTestCase(htf.TestCase):
def test_example(self):
self.assertTrue(True)
if __name__ == "__main__":
htf.main()
This runs all tests within ExampleTestCase
and generates an HTML-report
names testreport.html
.
Specifying tests¶
htf.main()
supports the tests
parameter. It may be None
,
a test-specifier or a list of test-specifiers.
A test-specifier may be a python module, a folder or an import-string (python style).
In comparison to the htf
command-line-utility it may also be a module-instance,
a test-case or a test-case-method.
Test-specifiers can be mixed.
htf.main(tests=["<test-specifier1>", "<test-specifier2>", .. "<test-specifierN>"])
If only one test-specifier is supplied, it is not necessary to use a list
htf.main(tests="<test-specifier>")
If a test-specifier is a python module it is imported and all testable objects are collected into a test-suite that is used for the test run.
htf.main(tests="test_1.py")
If a test-specifier is a folder this folder is searched recursively and all
python modules found within (see File pattern, pattern=
)
are imported and all testable
objects are collected into a test suite that is used for the test run.
Consider the folder tests
, which contains python modules with unit tests.
To execute all test cases in the folder tests
, simply use
htf.main(tests="tests")
In case tests
is an importable package or module, located in the python search path, the names might collide.
In this case, simply use
htf.main(tests="tests/")
If a test-specifier is an import string, this may be a package, a module, a class (inheriting from htf.TestCase
)
or a method.
If a test-specifier is a package or a module, it is imported and containing tests are executed.
htf.main(tests=["tests", "tests.test_example"])
If a class is used it should be a testcase inheriting from htf.TestCase
.
This testcase is used in a test suite and all tests are run.
htf.main(tests=tests.test_example.ExampleTestCase)
If a method is used it should be a method within a test case.
htf.main(tests="tests.test_example.ExampleTestCase.test_one")
The examples above are analog to the command-line-utility htf
.
In contrast to the command-line-utility htf, htf.main()
supports even more test-specifiers.
To test a module-instance use
from __future__ import \
absolute_import, division, print_function, unicode_literals
import htf
import htf_examples.htf_main_1
if __name__ == "__main__":
htf.main(tests=htf_examples.htf_main_1)
To test a test-case use
from __future__ import \
absolute_import, division, print_function, unicode_literals
import htf
class ExampleTestCase(htf.TestCase):
def test_example(self):
self.assertTrue(True)
if __name__ == "__main__":
htf.main(tests=ExampleTestCase)
And to test a test-case-method use
from __future__ import \
absolute_import, division, print_function, unicode_literals
import htf
class ExampleTestCase(htf.TestCase):
def test_example1(self):
self.assertTrue(True)
def test_example2(self):
self.assertTrue(True)
if __name__ == "__main__":
htf.main(tests=ExampleTestCase.test_example1)
Test-title¶
To specify the test-title use the title=
argument.
htf.main(title="This is my first test")
HTML-report¶
By default htf.main()
generates an HTML-report called testreport.html
.
To specify an HTML-report use the html_report=
argument.
htf.main(html_report="tests.html")
Multiple reports can be generated by supplying a list of filenames
htf.main(html_report=["tests1.html", "tests2.html"])
JUnit-XML-report¶
htf.main()
is also able to generate JUnit-XML-compatible XML-reports. The reports can be evaluated by different tools, eg.
Jenkins.
To specify an XML-report use the junit_xml_report=
argument.
htf.main(junit_xml_report="tests.xml")
or with multiple reports
htf.main(junit_xml_report=["tests1.xml", "tests2.xml"])
JSON-report¶
htf.main()
can also generate JSON-reports.
To specify a JSON-report use the json_report=
argument.
htf.main(json_report="tests.json")
or with multiple reports
htf.main(json_report=["tests1.json", "tests2.json"])
Other reports¶
There are other reports that need more parameters so that
they need to be instantiated outside of htf.main()
.
For example htf.DOORSTestReport
needs more
information about report-module, requirements-modules and links-modules.
To use such a report use the extra_report=
argument
from __future__ import \
absolute_import, division, print_function, unicode_literals
import htf
class Tests(htf.TestCase):
def test_assertionError(self):
o = None
print(o.no_there)
if __name__ == "__main__":
doors_report = htf.DOORSTestReport(
filename="doors.dxl",
reportModule="/path/to/report/module",
requirementsModules={
"REQ_.*": "/path/to/requirements_module",
"TEST_.*": "/path/to/test_specification",
},
linksModules={
"REQ_.*": "/path/to/links/to/requirements_module",
"TEST_.*": "/path/to/links/to/test_specification",
}
)
htf.main(extra_report=doors_report)
You can also supply a list of report instances
htf.main(extra_report=[report1, report2, ..., reportN])
Filename-templates¶
Filenames for reports can include prepopulated templates.
Available templates:
{{title}}
- the title as a filename{{date}}
- the current date"%Y-%m-%d_%H-%M-%S"
{{hostname}}
,{{host}}
or{{node}}
- the hostname{{htf_version}}
or{{version}}
- the htf version, eg.htf_1.0.0
{{python_version}}
- the python version, eg.Python_3.5.1
To generate an HTML-report containing the current node and the current date
htf.main(html_report="test_{{host}}_{{date}}.html")
Verbose output¶
To enable verbose output use the verbosity=2
argument.
Valid values are None
, 1
or 2
.
Catch CTRL-C¶
Long-running test suites can be terminated using a keyboard interrupt (CTRL-C
). Normally,
the execution is terminated immediately and no report is generated.
If catchbreak=True
is used and CTRL-C
is pressed the
current test is run to completion and the report is generated.
No further tests are run afterwards.
Fail fast¶
If the failfast=True
argument is used the test run ends after the first
failing test.
File pattern¶
If a test-specifier is a folder the default pattern used for file discovery is test*.py
.
The pattern can be changed using the pattern=
argument.
htf.main(tests="tests/", pattern="*.py")