Servo and the Web Platform Tests

2023-07-20 How we use the Web Platform Tests to guide Servo’s development.

With over 52000 tests and nearly two million subtests, the Web Platform Tests are one of the most important parts of the web’s interoperability and compatibility story. Servo has long used the WPT to catch regressions and flakiness, but the suite has also played a role in our layout engine migration and guiding our way towards CSS2 conformance.

You can find Servo on

wpt.fyi

, a dashboard for all browsers integrated with the Web Platform Tests.

To see the pass rates for Servo, follow this direct link, or click EDIT, ADD PRODUCT, choose Servo, then click SUBMIT.

While fyi is not so great for tracking progress over time (though definitely possible), it has been very helpful for drilling down from specs to sections, tests, and subtests. This has made it easier for us to decide what to work on next.

But the key to this was our

Servo WPT dashboard

.

The full dashboard can be found at wpt.servo.org, and there’s a simplified version on our about page.

Built in April, the dashboard has since tracked Servo’s pass rate over time, not just over the suite as a whole but also over several focus areas:

  • CSS overall
    • our CSS2 focus areas →
    • CSS position module
    • CSS flexbox module
    • CSSOM
  • abspos
  • box-display
  • floats
  • floats-clear
  • linebox
  • margin-padding-clear
  • normal-flow
  • positioning

The dashboard has data for both our legacy and new layout engines, which made it easy to see where the new engine most needed catching up.

Over the last three months, we’ve halved the gap with legacy or better in several of those areas:

  • CSS overall (7.7pp → 3.7pp)
  • our CSS2 focus areas (19.0pp → 8.2pp)
  • margin-padding-clear (33.0pp → 10.3pp)
  • CSS flexbox module (10.5pp → 3.1pp)
  • CSS position module (2.5pp → 0.2pp(!))

We’re two thirds of the way to legacy in floats (31.5pp → 10.8pp) and floats-clear (48.3pp → 3.3pp(!)):

floats: 18 (legacy 50) on 1 april, 23 (legacy 52) on 15 june, 33 on 17 june, 37 (legacy 53) on 18 july, 42 on 19 julyfloats-clear: 18 (legacy 66) on 1 april, 20 (legacy 67) on 15 june, 51 on 16 june, 57 on 22 june, 63 on 1 july, 64 on 20 july

And in the CSS2 positioning tests, we’ve surpassed legacy and built a healthy 5.5pp lead:

positioning: 77.7 (legacy 80.9) on 1 april, 81.7 (legacy 81.1) on 13 april, 85.7 on 1 july, 86.6 on 20 july

Over time, as our pass rates improve or people start working on new things, we’ll continue adding new focus areas, and with the web platform continuously evolving, we probably won’t run out of areas any time soon.