:py:mod:`pytest_flakefighters.flakefighters.traceback_matching` =============================================================== .. py:module:: pytest_flakefighters.flakefighters.traceback_matching .. autoapi-nested-parse:: This module implements three FlakeFighters based on failure de-duplication from Alshammari et. al. [https://arxiv.org/pdf/2401.15788]. Module Contents --------------- Classes ~~~~~~~ .. autoapisummary:: pytest_flakefighters.flakefighters.traceback_matching.TracebackMatching pytest_flakefighters.flakefighters.traceback_matching.CosineSimilarity .. py:class:: TracebackMatching(run_live: bool, previous_runs: list[pytest_flakefighters.database_management.Run], root: str = '.') Bases: :py:obj:`pytest_flakefighters.flakefighters.abstract_flakefighter.FlakeFighter` Simple text-based matching classifier from Section II.A of `Alshammari et al. (2024) `_. We implement text-based matching on the failure logs for each test. Each failure log is represented by its failure exception and stacktrace. :ivar run_live: Run detection "live" after each test. Otherwise run as a postprocessing step after the test suite. .. py:method:: from_config(config: dict) :classmethod: Factory method to create a new instance from a pytest configuration. .. py:method:: params() Convert the key parameters into a dictionary so that the object can be replicated. :return A dictionary of the parameters used to create the object. .. py:method:: _flaky_execution(execution, previous_executions) -> bool Classify an execution as flaky if any of its failing executions has a traceback that matches a test previously classed as flaky. :return: Boolean True if the test is classed as flaky and False otherwise. .. py:method:: previous_flaky_executions(runs: list[pytest_flakefighters.database_management.Run]) -> list Extract the relevant information from previous flaky executions and collapse into a single list. :param runs: The runs to consider. Defaults to self.previous_runs. :return: List containing the relative path, line number, column number, and code statement of all previous test executions. .. py:method:: flaky_test_live(execution: pytest_flakefighters.database_management.TestExecution, previous_runs: list[pytest_flakefighters.database_management.Run] = None) Classify executions as flaky if they have the same failure logs as a flaky execution. :param execution: Test execution to consider. :param previous_runs: The previous runs to which the execution will be compared. .. py:method:: flaky_tests_post(run: pytest_flakefighters.database_management.Run) -> list[bool | None] Classify failing executions as flaky if any if their executions are flaky. :param run: Run object representing the pytest run, with tests accessible through run.tests. .. py:class:: CosineSimilarity(run_live: bool, previous_runs: list[pytest_flakefighters.database_management.Run], root: str = '.', threshold: float = 1) Bases: :py:obj:`TracebackMatching` TF-IDF cosine similarity matching classifier from Section II.C of `Alshammari et al. (2024) `_. Test executions are classified as flaky if the stack trace is sufficiently similar to a previous flaky execution. :ivar run_live: Run detection "live" after each test. Otherwise run as a postprocessing step after the test suite. :ivar previous_runs: List of previous FlakeFighters runs. :ivar root: The root directory of the code repository. :ivar threshold: The minimum distance to consider as "similar", expressed as a proportion 0 <= threshold < 1 where 0 represents no difference and 1 represents complete difference. .. py:method:: from_config(config: dict) :classmethod: Factory method to create a new instance from a pytest configuration. .. py:method:: params() Convert the key parameters into a dictionary so that the object can be replicated. :return A dictionary of the parameters used to create the object. .. py:method:: _tf_idf_matrix(executions) .. py:method:: _flaky_execution(execution, previous_executions) -> bool Classify an execution as flaky if the test execution is sufficiently cosine-similar to any of the previous executions. :return: Boolean True if the test is classed as flaky and False otherwise.