Skip to content

wpt: handle missing/unresolvable ref files gracefully; exclude tools/resources dirs - #660

Open
nicoburns wants to merge 1 commit into
mainfrom
devin/1786204383-wpt-missing-refs
Open

wpt: handle missing/unresolvable ref files gracefully; exclude tools/resources dirs#660
nicoburns wants to merge 1 commit into
mainfrom
devin/1786204383-wpt-missing-refs

Conversation

@nicoburns

@nicoburns nicoburns commented Aug 8, 2026

Copy link
Copy Markdown
Member

Summary

17 tests crashed in process_ref_test when the <link rel=match> reference couldn't be resolved or read:

  • href="about:blank" panicked on ref_url.path().strip_prefix('/').unwrap() (the URL has no leading-slash path). An about:blank ref means "matches a blank page", so it is now rendered as an empty document and compared normally (several of these tests now PASS).
  • Missing ref files (e.g. css-transforms/css-transform-inherit-rotate.html pointing at a deleted ttwf-reftest-tutorial-ref.html) panicked on fs::read_to_string(...).expect("Ref file not found."). Unresolvable hrefs, unsupported schemes, and unreadable ref files now log a warning and return SubtestCounts::ZERO_OF_ZERO (reported as SKIP).
  • Template files under css/css-images/tools/ (with literal REPLACEME_REFERENCE_FILENAME refs) and helpers under resources/ are not tests at all: filter_path now excludes tools and resources directories from collection, matching the upstream WPT manifest rules.

Crash count for the affected dirs (CSS2/generated-content, css-break, css-images, css-masking, css-overflow, css-transforms, css-view-transitions): 16 of the 17 group crashes eliminated; the remaining one (overflow-body-propagation-013.html) then crashes in the engine (Document node queued as inline root when :root is display:none) and is fixed separately in #661-to-be (layout classification PR).

Link to Devin session: https://dioxus.staging.devinenterprise.com/sessions/f1a0d6696eaa4076a48f366a24bfd26f
Requested by: @nicoburns

WPT results

4 newly passing, 0 newly failing (net +4), 4 other status changes, 73 removed tests.

Full diff (81 changed tests)
+ Crash => Pass css/CSS2/generated-content/content-counter-001.xht
- REM css/css-backgrounds/box-shadow/tools/box-shadow-blur-definition-001-image-generator.html
- REM css/css-borders/corner-shape/resources/green.html
+ Crash => Pass css/css-break/ink-overflow-001-print.html
- REM css/css-contain/content-visibility/resources/frame.html
- REM css/css-contain/content-visibility/resources/slot-content-visibility.html
- REM css/css-contain/content-visibility/resources/text-fragment-target-auto.html
- REM css/css-contain/content-visibility/resources/text-fragment-target-matchable.html
- REM css/css-highlight-api/painting/resources/iframe-code.html
- REM css/css-highlight-api/painting/resources/iframe-container.html
- REM css/css-highlight-api/painting/resources/iframe-viewport-first-line.html
- REM css/css-highlight-api/painting/resources/iframe-viewport.html
- REM css/css-images/tools/object-view-box-fit-contain-ref-template.html
- REM css/css-images/tools/object-view-box-fit-contain-template.html
- REM css/css-images/tools/object-view-box-fit-cover-ref-template.html
- REM css/css-images/tools/object-view-box-fit-cover-template.html
- REM css/css-images/tools/object-view-box-fit-fill-ref-template.html
- REM css/css-images/tools/object-view-box-fit-fill-template.html
- REM css/css-images/tools/object-view-box-fit-none-ref-template.html
- REM css/css-images/tools/object-view-box-fit-none-template.html
- REM css/css-images/tools/object-view-box-writing-mode-ref-template.html
- REM css/css-images/tools/object-view-box-writing-mode-template.html
- REM css/css-images/tools/template-object-fit-test.html
- REM css/css-images/tools/template-object-position-test.html
+ Crash => Pass css/css-masking/mask-image/backdrop-filter-bad-mask-image.html
! Crash => Fail css/css-masking/mask-image/backdrop-filter-mask-image-while-loading.html
! Crash => Fail css/css-masking/mask-image/bad-mask-image-svg.html
+ Crash => Pass css/css-masking/mask-image/bad-mask-image.html
- REM css/css-overflow/scroll-markers/resources/root-scroll-marker-activation-iframe.html
- REM css/css-overscroll-behavior/resources/keyboard-scroll-child-frame-iframe.html
- REM css/css-page/resources/iframe-with-abspos.html
- REM css/css-page/resources/mq-frame-100px.html
- REM css/css-position/resources/position-absolute-iframe-child-002.sub.html
- REM css/css-position/resources/position-absolute-iframe-child.html
- REM css/css-position/resources/position-sticky-fixed-ancestor-iframe-child.html
- REM css/css-sizing/responsive-iframe/resources/iframe-contents-400x200root.html
- REM css/css-sizing/responsive-iframe/resources/iframe-contents-dom-content-loaded.html
- REM css/css-sizing/responsive-iframe/resources/iframe-contents-icb.html
- REM css/css-sizing/responsive-iframe/resources/iframe-contents-meta-after-head.html
- REM css/css-sizing/responsive-iframe/resources/iframe-contents-meta-dynamic-append-after-body.html
- REM css/css-sizing/responsive-iframe/resources/iframe-contents-meta-dynamic-append-to-doc.html
- REM css/css-sizing/responsive-iframe/resources/iframe-contents-meta-dynamic-append.html
- REM css/css-sizing/responsive-iframe/resources/iframe-contents-meta-dynamic-name-change-after-body.html
- REM css/css-sizing/responsive-iframe/resources/iframe-contents-meta-dynamic-name-change.html
- REM css/css-sizing/responsive-iframe/resources/iframe-contents-meta-dynamic-remove.html
- REM css/css-sizing/responsive-iframe/resources/iframe-contents-meta-dynamic-write.html
- REM css/css-sizing/responsive-iframe/resources/iframe-contents-meta-in-body.html
- REM css/css-sizing/responsive-iframe/resources/iframe-contents-navigation.html
- REM css/css-sizing/responsive-iframe/resources/iframe-contents-request-resize-error.html
- REM css/css-sizing/responsive-iframe/resources/iframe-contents-request-resize.html
- REM css/css-sizing/responsive-iframe/resources/iframe-contents-slow.html
- REM css/css-sizing/responsive-iframe/resources/iframe-contents-unsized.html
- REM css/css-sizing/responsive-iframe/resources/iframe-contents.html
- REM css/css-tables/tools/markup-generator.html
! Crash => Skip css/css-transforms/css-transform-inherit-rotate.html
- REM css/css-variables/resources/variable-reference-refresh-iframe.html
- REM css/css-view-transitions/navigation/resources/abort-before-render.html
- REM css/css-view-transitions/navigation/resources/at-rule-opt-in-auto.html
- REM css/css-view-transitions/navigation/resources/at-rule-opt-in-none.html
- REM css/css-view-transitions/navigation/resources/chromium-paint-holding-timeout.html
- REM css/css-view-transitions/navigation/resources/outbound-before-render.html
- REM css/css-view-transitions/navigation/resources/root-element-transition-iframe-inner-result.html
- REM css/css-view-transitions/navigation/resources/root-element-transition-iframe.html
- REM css/css-view-transitions/navigation/resources/transition-to-prerender.html
! Crash => Skip css/css-view-transitions/navigation/transition-to-prerender-manual.html
- REM css/css-view-transitions/navigation/two-phase/resources/instant-transition.https.html
- REM css/css-viewport/zoom/resources/iframe_content.html
- REM css/css-viewport/zoom/resources/leaf.html
- REM css/css-viewport/zoom/resources/nested-iframe-no-zoom.html
- REM css/css-viewport/zoom/resources/nested-iframe-with-zoom.html
- REM css/css-viewport/zoom/resources/nested-iframe.html
- REM css/css-writing-modes/tools/generators/template.html
- REM css/cssom-view/resources/iframe1.html
- REM css/cssom-view/resources/iframe2.html
- REM css/cssom-view/resources/scrollIntoView-frame.html
- REM css/selectors/attribute-selectors/attribute-case/resources/semantics-quirks.html
- REM css/selectors/attribute-selectors/attribute-case/resources/semantics-xml.xhtml
- REM css/selectors/attribute-selectors/attribute-case/resources/syntax-quirks.html
- REM css/selectors/attribute-selectors/attribute-case/resources/syntax-xml.xhtml
- REM css/selectors/resources/partitioned-visited-cross-site-iframe.html
- REM css/selectors/resources/partitioned-visited-same-site-iframe.html

Generated by the WPT workflow.

@nicoburns nicoburns self-assigned this Aug 8, 2026
@staging-devin-ai-integration

Copy link
Copy Markdown
Contributor

🤖 Devin AI Engineer

I'll be helping with this pull request! Here's what you should know:

✅ I will automatically:

  • Address comments on this PR that start with 'DevinAI' or '@devin'.
  • Look at CI failures and help fix them

Note: I can only respond to comments from users who have write access to this repository.

⚙️ Control Options:

  • Disable automatic comment, CI, and merge conflict monitoring

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant