login bonus william hill casino club_royal panda promotions_online casino bonus

Mon, 03 Dec 2018 16:07:32 -0800

Hello everyone,

I have a proposal to make writing layout tests for specific types of devices 
more straight-forward.

Currently, we name directories with the beginning of a device name, such as 
¡®iphone7¡¯ or ¡®ipad¡¯ and hope that those directory names match the name of a 
device type in the associated port¡¯s CUSTOM_DEVICE_CLASS. This causes some 
problems, however, in a few circumstances. The first problem arises when two 
ports have associated devices (such as watchOS and iOS), in this case, a 
directory which maps to a device type on one port might not on another. The 
second problem is that our current code for managing devices assumes that 
devices can be booted on-demand, which is not the case in on-device testing, 
where we need to run different sets of tests depending on the type of device 
attached. The last (and most important) problem is that we cannot run the same 
test twice on two different devices in a single test run (ie, run test.html 
once on an iPhone and once on an iPad).

To resolve this, I propose 3 changes to how we organize layout tests.

1) Allow *-expected.txt files to be device-specific 
<https://bugs.webkit.org/show_bug.cgi?id=192162>

The idea here would be to optionally allow *-expected.txt files to be a 
directory looking something like this:

dir/
    test.html
    test-expected/
        iPhone-7.txt

Under this scheme, the name of the .txt file would map directly to a device 
type. Multiple device-specific expectations would be permitted, but making 
sense of these results requires change #2.

2) Support multiple results for a specific test 
<https://bugs.webkit.org/show_bug.cgi?id=192163>

Similar to #1, we need to allow test results to be attributed to a device type. 
This would be done with a similar scheme, where results for a test would 
optionally be saved as a directory looking like this:

dir/
    test/
        iPhone-7-actual.txt
        iPhone-7-diff.txt
        iPhone-7-expected.txt
        iPhone-7-diff.html
        iPhone-8-actual.txt
        iPhone-8-diff.txt
        iPhone-8-expected.txt
        iPhone-8-diff.html

When reporting, results would be reported to as a new ¡®queue¡¯ of the form 
'<queue>-<device>¡¯.

3) Provide device specific tags in test expectations 
<https://bugs.webkit.org/show_bug.cgi?id=192164>

We already have Debug/Release as well as version tags in our test expectations. 
The addition here would be device-type tags, those tags would look like this:

[ iPhone 7 ] dir/test.html [ Pass ]
[ iPad ] dir/other-test.html [ Pass ]

These tags would allow for 2 features. The first is similar to existing tags, 
allowing a certain test (or group of tests) to have different expectations or 
to be skipped entirely on certain device types. The second feature allows for 
tests and groups of tests to be run on multiple different devices in a single 
instantiation of run-webkit-tests. For example,

[ iPhone 7, iPad ] dir/some-dir/ [ Pass ]

would indicate that every test in dir/some-dir/ will be run on BOTH iPhone 7 
and iPad.

I would welcome anyone else¡¯s thoughts on this.

Jonathan
_______________________________________________
webkit-dev mailing list
webkit-dev@lists.webkit.org
https://lists.webkit.org/mailman/listinfo/webkit-dev

Reply via email to