htf.main() — Testscript utility¶

htf.main() is used to start tests if not using htf (the command-line-utility) within a test-script.

htf.main(title=u'Testreport', tests=u'__main__', tags=None, html_report=None, junit_xml_report=None, json_report=None, extra_report=None, pattern=u'test*.py', metadata=None, exit=True, verbosity=1, failfast=None, catchbreak=None)¶

Test runner.

Parameters:	title (str) – the test’s title tests=None (test-specifier or list of test-specifiers) – a test-specifier ora list of test-specifiers (folder, file, test-case, test-case-method, module) tags=None (str) – tags filter expression html_report=None (str or list of str) – HTML-report filename(s) junit_xml_report=None (str or list of str) – JUnit-XML-report filename(s) json_xml_report=None (str or list of str) – JSON-report filename(s) extra_report=None (instance or list of instances) – Extra reports that support instance.render(data). This is useful for `htf.DOORSTestReport` for example that needs additional parameters. *pattern="test.py"** (str) – the pattern to be used when searching for python files in folders metadata=None (dict) – a dictionary that contains metadata about the test run exit=True (bool) – if set to `True` the exit() is called at the end of the tests verbosity=1 (int) – the verbosity level (the higher the level the more verbose is the output) failfast=None (bool) – if set to `True` the test run ends after the first failing test catchbreak=None (bool) – if set to `True` CTRL-C is catched wile tests run. If CTRL-C is pressed the test run is stopped after the current test.
Returns:	`0` if the run was successful or the number of failed tests in case or errors and failures.
Return type:	int

Running tests¶

This section show some usage examples.

The first example covers the basic usage for a simple test-case running on Python 2.7 and 3.5+.

from __future__ import \
    absolute_import, division, print_function, unicode_literals

import htf


class ExampleTestCase(htf.TestCase):
    def test_example(self):
        self.assertTrue(True)

if __name__ == "__main__":
    htf.main()

This runs all tests within ExampleTestCase and generates an HTML-report names testreport.html.

Specifying tests¶

htf.main() supports the tests parameter. It may be None, a test-specifier or a list of test-specifiers.

A test-specifier may be a python module, a globbing-expression for python modules (e.g. test_*.py), a folder or an import-string (python style). In comparison to the htf command-line-utility it may also be a module-instance, a test-case or a test-case-method.

Test-specifiers can be mixed.

htf.main(tests=["<test-specifier1>", "<test-specifier2>", .. "<test-specifierN>"])

If only one test-specifier is supplied, it is not necessary to use a list

htf.main(tests="<test-specifier>")

If a test-specifier is a python module it is imported and all testable objects are collected into a test-suite that is used for the test run.

htf.main(tests="test_1.py")

If a test-specifier ist a globbing-expression the expression is evaluated and all matching files are imported.

htf.main(tests="test_*.py")

If a test-specifier is a folder this folder is searched recursively and all python modules found within (see File pattern, pattern=) are imported and all testable objects are collected into a test suite that is used for the test run.

Consider the folder tests, which contains python modules with unit tests. To execute all test cases in the folder tests, simply use

htf.main(tests="tests")

In case tests is an importable package or module, located in the python search path, the names might collide. In this case, simply use

htf.main(tests="tests/")

If a test-specifier is an import string, this may be a package, a module, a class (inheriting from htf.TestCase) or a method.

If a test-specifier is a package or a module, it is imported and containing tests are executed.

htf.main(tests=["tests", "tests.test_example"])

If a class is used it should be a testcase inheriting from htf.TestCase. This testcase is used in a test suite and all tests are run.

htf.main(tests=tests.test_example.ExampleTestCase)

If a method is used it should be a method within a test case.

htf.main(tests="tests.test_example.ExampleTestCase.test_one")

The examples above are analog to the command-line-utility htf.

In contrast to the command-line-utility htf, htf.main() supports even more test-specifiers.

To test a module-instance use

from __future__ import \
    absolute_import, division, print_function, unicode_literals

import htf
import htf_examples.htf_main_1

if __name__ == "__main__":
    htf.main(tests=htf_examples.htf_main_1)

To test a test-case use

from __future__ import \
    absolute_import, division, print_function, unicode_literals

import htf


class ExampleTestCase(htf.TestCase):
    def test_example(self):
        self.assertTrue(True)

if __name__ == "__main__":
    htf.main(tests=ExampleTestCase)

And to test a test-case-method use

from __future__ import \
    absolute_import, division, print_function, unicode_literals

import htf


class ExampleTestCase(htf.TestCase):
    def test_example1(self):
        self.assertTrue(True)

    def test_example2(self):
        self.assertTrue(True)

if __name__ == "__main__":
    htf.main(tests=ExampleTestCase.test_example1)

Test-title¶

To specify the test-title use the title= argument.

htf.main(title="This is my first test")

HTML-report¶

By default htf.main() generates an HTML-report called testreport.html. To specify an HTML-report use the html_report= argument.

htf.main(html_report="tests.html")

Multiple reports can be generated by supplying a list of filenames

htf.main(html_report=["tests1.html", "tests2.html"])

JUnit-XML-report¶

htf.main() is also able to generate JUnit-XML-compatible XML-reports. The reports can be evaluated by different tools, eg. Jenkins.

To specify an XML-report use the junit_xml_report= argument.

htf.main(junit_xml_report="tests.xml")

or with multiple reports

htf.main(junit_xml_report=["tests1.xml", "tests2.xml"])

JSON-report¶

htf.main() can also generate JSON-reports.

To specify a JSON-report use the json_report= argument.

htf.main(json_report="tests.json")

or with multiple reports

htf.main(json_report=["tests1.json", "tests2.json"])

Other reports¶

There are other reports that need more parameters so that they need to be instantiated outside of htf.main().

For example htf.DOORSTestReport needs more information about report-module, requirements-modules and links-modules.

To use such a report use the extra_report= argument

from __future__ import \
    absolute_import, division, print_function, unicode_literals

import htf


class Tests(htf.TestCase):

    def test_assertionError(self):
        o = None
        print(o.no_there)

if __name__ == "__main__":
    doors_report = htf.DOORSTestReport(
                       filename="doors.dxl",
                       reportModule="/path/to/report/module",
                       requirementsModules={
                           "REQ_.*": "/path/to/requirements_module",
                           "TEST_.*": "/path/to/test_specification",
                       },
                       linksModules={
                           "REQ_.*": "/path/to/links/to/requirements_module",
                           "TEST_.*": "/path/to/links/to/test_specification",
                           }
                       )
    htf.main(extra_report=doors_report)

You can also supply a list of report instances

htf.main(extra_report=[report1, report2, ..., reportN])

Filename-templates¶

Filenames for reports can include prepopulated templates.

Available templates:

{{title}} - the title as a filename

{{date}} - the current date "%Y-%m-%d_%H-%M-%S"

{{hostname}}, {{host}} or {{node}} - the hostname

{{htf_version}} or {{version}} - the htf version, eg. htf_1.0.0

{{python_version}} - the python version, eg. Python_3.5.1

To generate an HTML-report containing the current node and the current date

htf.main(html_report="test_{{host}}_{{date}}.html")

Verbose output¶

To enable verbose output use the verbosity=2 argument. Valid values are None, 1 or 2.

Catch CTRL-C¶

Long-running test suites can be terminated using a keyboard interrupt (CTRL-C). Normally, the execution is terminated immediately and no report is generated.

If catchbreak=True is used and CTRL-C is pressed the current test is run to completion and the report is generated. No further tests are run afterwards.

Fail fast¶

If the failfast=True argument is used the test run ends after the first failing test.

File pattern¶

If a test-specifier is a folder the default pattern used for file discovery is test*.py. The pattern can be changed using the pattern= argument.

htf.main(tests="tests/", pattern="*.py")

Tagging¶

To select tests using a logical expression use the tags option set to an expression.

htf.main(tags="foo|bar")

To list all available tags use the htf.get_tags() function.

htf.get_tags(tests="/path/to/test")

htf.get_tags(tests=u'__main__', pattern=u'test*.py')¶

Get a list of tags found in the specified tests.

Parameters:	tests=None (test-specifier or list of test-specifiers) – a test-specifier ora list of test-specifiers (folder, file, test-case, test-case-method, module) *pattern="test.py"** (str) – the pattern to be used when searching for python files in folders
Returns:	a list of tags found in the specified tests.
Return type:	list of str

Metadata¶

To add metadata about the test run use the metadata parameter set to a dictionary containing metadata.

This feature can be used to, for example, add information about a tested firmware version, etc.

metadata_dict = dict(firmware_version="1.2.3", hardware_version="4.5.6")
htf.main(metadata=metadata_dict)

The metadata will appear in the test reports.

htf.run()¶

htf.run() is an alias for htf.main().

htf.run(title=u'Testreport', tests=u'__main__', tags=None, html_report=None, junit_xml_report=None, json_report=None, extra_report=None, pattern=u'test*.py', metadata=None, exit=True, verbosity=1, failfast=None, catchbreak=None)¶