pytest_flakefighters.plugin

This module implements the DeFlaker algorithm [Bell et al. 10.1145/3180155.3180164] as a pytest plugin.

Module Contents

Classes

RerunStrategy	Enum for supported test rerunning strategies.
FlakeFighterPlugin	The main plugin to manage the various FlakeFighter tools.

class pytest_flakefighters.plugin.RerunStrategy

Bases: enum.Enum

Enum for supported test rerunning strategies. :cvar ALL: Rerun all tests, regardless of outcome. :cvar FLAKY_FAILURE: Rerun failing tests marked as flaky. :cvar PREVIOUS_FLAKY: Rerun tests that have previously been marked as flaky as well as newly failing flaky tests.

ALL = 'ALL'

FLAKY_FAILURE = 'FLAKY_FAILURE'

PREVIOUSLY_FLAKY = 'PREVIOUSLY_FLAKY'

class pytest_flakefighters.plugin.FlakeFighterPlugin(root: str, database: pytest_flakefighters.database_management.Database, cov: coverage.Coverage | pytest_flakefighters.function_coverage.Profiler, flakefighters: list[pytest_flakefighters.flakefighters.abstract_flakefighter.FlakeFighter], save_run: bool = True, rerun_strategy: RerunStrategy = RerunStrategy.FLAKY_FAILURE, display_outcomes: int = 0, display_verdicts: bool = False)

The main plugin to manage the various FlakeFighter tools.

pytest_sessionstart(session: pytest.Session)

Start the coverage measurement before tests are collected so we measure class and method definitions as covered. :param session: The session.

pytest_collection_finish(session: pytest.Session)

Stop the coverage measurement after tests are collected. :param session: The session.

pytest_runtest_call(item: pytest.Item)

Start the coverage measurement and label the coverage for the current test, run the test, then stop coverage measurement.

Parameters:

item – The item.

pytest_runtest_makereport(item: pytest.Item, call: pytest.CallInfo)

Called after a test execution call (setup, call, teardown) to create a TestReport.

Parameters:

• item – The item.

• call – The call info.

pytest_runtest_protocol(item: pytest.Item, nextitem: pytest.Item) → bool

Rerun flaky tests. Follows a similar control logic to the pytest-rerunfailures plugin.

Parameters:

• item – The item.

• nextitem – The next item.

Returns:

The return value is not used, but only stops further processing.

pytest_report_teststatus(report: pytest.TestReport, config: pytest.Config) → tuple[str, str, str]

Report flaky failures as such. :param report: The report object whose status is to be returned. :param config: The pytest config object. :returns: The test status.

pytest_runtestloop(session: pytest.Session)

Run postprocessing flakefighters. :param session: The pytest session object.

pytest_json_modifyreport(json_report: dict)

Add flakefighter results to the pytest-json-report report.

Parameters:

json_report – The report dict.

build_outcome_string(test: pytest_flakefighters.database_management.Test) → str

Construct a string to represent previous flakefighter outcomes for a given test and its associated executions.

Parameters:

test – The test case.

modify_xml(xml_path: str)

Modify the JUnitXML file to add the flakefighter results for each test.

Parameters:

xml_path – The path of the saved XML file.

pytest_html_results_summary(prefix: list, summary: list, postfix: list)

Add the test-level flakefighter results. :param prefix: The prefix content. UNUSED. :param summary: The summary content. UNUSED. :param postfix: The postfix content.

pytest_sessionfinish(session: pytest.Session, exitstatus: pytest.ExitCode) → None

Called after whole test run finished, right before returning the exit status to the system. :param session: The pytest session object. :param exitstatus: The status which pytest will return to the system.