Testing Suite
Exhale uses pytest for its testing suite. To run the tests, Exhale uses tox.
$ cd /path/to/exhale
$ pip install tox
$ tox
Note
See tox.ini
in the root of the repository. You only need to install tox
to
be able to run the tests, which will internally download the additional dependencies
such as pytest
. The tests will be run in a virtual environment.
Running Specific Tests
By default, tox
will run the python tests and linting checks with
flake8. More formally, the default environment
list is defined as:
[testenv]
envlist = py, flake8
This means that the version of python the tests are run with are the interpreter that
you installed tox
for. To run a specific test:
tox -e py
Run the python unit tests.
tox -e flake8
Run the lint tests.
tox -e docs
Build the sphinx documentation using the html builder (in nitpicky mode). You can view the generated html website with
open .tox/docs/tmp/html/index.html
.tox -e linkcheck
Build the sphinx documentation using the linkcheck builder.
Tip
If you need to debug a test case to discern why it is failing, the vibrantly colorful
ipdb debugger is already installed in the
environment running the tests. It is nearly identical to pdb
, and although it
is designed for IPython, it works just as well in “regular” code. Just like with
pdb
, set a trace in the test case you are debugging:
import ipdb
ipdb.set_trace()
In order to be able to achieve the break-point, you would launch the tests with
$ tox -e py -- -s
The --
signals that the command line arguments for tox
are complete, and
everything afterward should be forwarded to the specified environment (in this case
to pytest
since we launched -e py
). The -s
switch for pytest
enables
the debugger to stop the test, without this flag the test will fail at the
break-point.
The same goes for other test cases, if you need to specify a flag to flake8
for
some reason you would run tox -e flake8 -- --ignore XYYY
where XYYY
is the
linter check you want to ignore.
Writing Project Test Cases
exhale
provides tools to build tests that need to generate output files (= *.rst
files) from actual test projects. To create such tests:
Create a new test-case folder in
testing/projects/{new project name}
.Inherit from
testing.base.ExhaleTestCase
in a module you create in thetesting/tests
folder, setting thetest_project
class-level variable to{new project name}
Add some
test_{something}
methods.testing.base.ExhaleTestCase
will work its magic to apply the necessary pytest fixtures to all these methods so that the configuration is applied,sphinx
is bootstrapped and all*.rst
files generated byexhale
are in the specified output directory at the start of your test. Note that thesphinx
application is made available asself.app
.In case you need to run tests with a configuration differing from the default configuration, you can use the
testing.decorators.confoverrides()
decorator to tweak the configuration for a test method or the whole test class:@confoverrides(conf_var1=conv_val1, conf_var2=conf_val2) class MyTestCase(ExhaleTestCase): test_project = 'my_project'
or:
@confoverrides(conf_var1=conv_val1, conf_var2=conf_val2) def test_something(self): ...
Note that
@confoverrides
applied to a test method are combined with@confoverrides
applied to test classes and have precedence.In case you don’t want
exhale
to generate any files and just need to test the configuration, you can use thetesting.decorators.no_run()
on a test method or test class:@no_run class MyTestCase(ExhaleTestCase): test_project = 'my_project'
or:
@no_run def test_something_else(self): ...
For more examples, just have a look at the existing tests in the testing/tests
folder.
Full Testing Suite Documentation
A HUGE thank you to Thomas Khyn for his thorough help figuring out how to coordinate
pytest
,sphinx
, metaclasses, and more – this testing framework would never have come to fruition without his epic pull request.
The testing package for Exhale.
- testing.TEST_PROJECTS_ROOT = '/home/docs/checkouts/readthedocs.org/user_builds/exhale/checkouts/stable/testing/projects'
The global
projects
folder that individual test cases use.Each test case indexes into this folder with their specified
test_project
. See alsotesting.base.ExhaleTestCase.test_project
.
- testing.get_exhale_root(test)[source]
Return the finalized
exhale.graph.ExhaleRoot
object for the specified test.Todo
document better / this gets updated in v1.0.0
- Testing Base Module
- Testing Conftest Module
- Testing Decorators Module
- Testing Fixtures Module
- Testing Hierarchies Module
- Testing Projects Module
testing.projects.c_maths
Projecttesting.projects.cpp with spaces
Projecttesting.projects.cpp_fortran_mixed
Projecttesting.projects.cpp_func_overloads
Projecttesting.projects.cpp_long_names
Projecttesting.projects.cpp_dir_underscores
Projecttesting.projects.cpp_nesting
Projecttesting.projects.cpp_pimpl
Project
- Testing Utils Module
- All Tests
- Exhale Core Tests
- Project Tests