Testing Suite

Exhale uses pytest for its testing suite. To run the tests, Exhale uses tox.

$ cd /path/to/exhale
$ pip install tox
$ tox

Note

See tox.ini in the root of the repository. You only need to install tox to be able to run the tests, which will internally download the additional dependencies such as pytest. The tests will be run in a virtual environment.

Running Specific Tests

By default, tox will run the python tests and linting checks with flake8. More formally, the default environment list is defined as:

[testenv]
envlist = py, flake8

This means that the version of python the tests are run with are the interpreter that you installed tox for. To run a specific test:

tox -e py: Run the python unit tests.
tox -e flake8: Run the lint tests.
tox -e docs: Build the sphinx documentation using the html builder (in nitpicky mode). You can view the generated html website with open .tox/docs/tmp/html/index.html.
tox -e linkcheck: Build the sphinx documentation using the linkcheck builder.

Tip

If you need to debug a test case to discern why it is failing, the vibrantly colorful ipdb debugger is already installed in the environment running the tests. It is nearly identical to pdb, and although it is designed for IPython, it works just as well in “regular” code. Just like with pdb, set a trace in the test case you are debugging:

import ipdb
ipdb.set_trace()

In order to be able to achieve the break-point, you would launch the tests with

$ tox -e py -- -s

The -- signals that the command line arguments for tox are complete, and everything afterward should be forwarded to the specified environment (in this case to pytest since we launched -e py). The -s switch for pytest enables the debugger to stop the test, without this flag the test will fail at the break-point.

The same goes for other test cases, if you need to specify a flag to flake8 for some reason you would run tox -e flake8 -- --ignore XYYY where XYYY is the linter check you want to ignore.

Writing Project Test Cases

exhale provides tools to build tests that need to generate output files (= *.rst files) from actual test projects. To create such tests:

Create a new test-case folder in testing/projects/{new project name}.
Inherit from testing.base.ExhaleTestCase in a module you create in the testing/tests folder, setting the test_project class-level variable to {new project name}
Add some test_{something} methods. testing.base.ExhaleTestCase will work its magic to apply the necessary pytest fixtures to all these methods so that the configuration is applied, sphinx is bootstrapped and all *.rst files generated by exhale are in the specified output directory at the start of your test. Note that the sphinx application is made available as self.app.
In case you need to run tests with a configuration differing from the default configuration, you can use the testing.decorators.confoverrides() decorator to tweak the configuration for a test method or the whole test class:
```
@confoverrides(conf_var1=conv_val1, conf_var2=conf_val2)
class MyTestCase(ExhaleTestCase):
    test_project = 'my_project'
```
or:
```
@confoverrides(conf_var1=conv_val1, conf_var2=conf_val2)
def test_something(self):
    ...
```
Note that @confoverrides applied to a test method are combined with @confoverrides applied to test classes and have precedence.
In case you don’t want exhale to generate any files and just need to test the configuration, you can use the testing.decorators.no_run() on a test method or test class:
```
@no_run
class MyTestCase(ExhaleTestCase):
    test_project = 'my_project'
```
or:
```
@no_run
def test_something_else(self):
    ...
```

For more examples, just have a look at the existing tests in the testing/tests folder.

Full Testing Suite Documentation

A HUGE thank you to Thomas Khyn for his thorough help figuring out how to coordinate pytest, sphinx, metaclasses, and more – this testing framework would never have come to fruition without his epic pull request.

The testing package for Exhale.

testing.TEST_PROJECTS_ROOT = '/home/docs/checkouts/readthedocs.org/user_builds/exhale/checkouts/latest/testing/projects'

The global projects folder that individual test cases use.

Each test case indexes into this folder with their specified test_project. See also testing.base.ExhaleTestCase.test_project.

testing.get_exhale_root(test)[source]: Return the finalized exhale.graph.ExhaleRoot object for the specified test.

Todo

document better / this gets updated in v1.0.0