Skip to content

Experiment: use test runner error/failure to distinguish direct and transitive defects #71

Description

@redcatbear

Goal

Test-drive a variant where the IntelliJ test runner presentation uses error and failure to distinguish transitive and direct defects. This is explicitly an experiment, not a committed semantic change yet.

Background

We already mark transitive defects visually in the test runner. The open question is whether the status split would give a better practical result in real usage, especially because OFT traces do not raise exceptions the way unit tests do. In that context, the direct/transitive distinction may be closer to severity than to exception handling.

Hypothesis

If transitive defects are shown as error and direct defects as failure, the test runner may gain:

  • clearer visual separation
  • simpler filtering
  • a more natural severity model, where transitive defects are "nice to know" and direct defects need fixing

What to test

  • Map transitive defects to error and direct defects to failure in the test runner variant.
  • Run the variant against realistic OFT traces.
  • Evaluate whether the new status split improves readability and filtering without making the semantics harder to understand.

Success criteria

  • The experiment is implemented on a feature branch and can be exercised in real traces.
  • We can judge whether the status split is a useful model or whether it blurs the meaning of transitive defects too much.
  • The result is documented clearly enough to decide whether to keep or discard the approach.

Notes

This issue is intentionally framed as an experiment so we can compare it against the current direct/transitive model without prematurely locking in the semantics.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions