pytest_flakefighters.flakefighters.coverage_independence

This module implements the CoverageIndependence FlakeFighter.

Module Contents

Classes

CoverageIndependence

Classify tests as flaky if they fail independently of passing test cases that exercise overlapping code.
class pytest_flakefighters.flakefighters.coverage_independence.CoverageIndependence(threshold: float = 0, metric: str = 'jaccard', linkage_method='single')

Bases: pytest_flakefighters.flakefighters.abstract_flakefighter.FlakeFighter

Classify tests as flaky if they fail independently of passing test cases that exercise overlapping code.

Note

To use this flakefighter, you will need to install scipy. We do not include this by default as it can be problematic to install on Windows. You can do this by running pip install pytest-flakefighers[scipy] or pip install scipy from within your virtual environment.

Variables:

  • run_live – Run detection “live” after each test. Otherwise run as a postprocessing step after the test suite. This is always False as live classification is not supported.

  • threshold – The minimum distance to consider as “similar”, expressed as a proportion 0 <= threshold < 1 where 0 represents no difference and 1 represents complete difference.

  • metric – The distance metric to use. For a full list of valid values,see the SciPy documentation for spatial distances.

  • linkage_method – From scipy.cluster.hierarchy.linkage: [‘single’, ‘complete’, ‘average’, ‘weighted’, ‘centroid’, ‘median’, ‘ward’]

classmethod from_config(config: dict)

Factory method to create a new instance from a pytest configuration.

params()

Convert the key parameters into a dictionary so that the object can be replicated. :return A dictionary of the parameters used to create the object.

abstract flaky_test_live(execution: pytest_flakefighters.database_management.TestExecution)

NOT SUPPORTED. Detect whether a given test execution is flaky and append the result to its flakefighter_results attribute. :param execution: The test execution to classify.

flaky_tests_post(run: pytest_flakefighters.database_management.Run)

Go through each test in the test suite and append the result to its flakefighter_results attribute. :param run: Run object representing the pytest run, with tests accessible through run.tests.