Clarify behavior when AsyncTestComplete/Failure are both present

[ivankra]

Common sense and this test dictate that failure should be preferred. This is apparently not obvious to LLM agents, see this issue / commit.

[ivankra]

Another example: Samsung/escargot#1553

[ptomato]

Not sure about this? I always thought that print should only be invoked exactly once, and invoking it twice means a buggy test.

@jugglinmike As someone who may have helped write the original spec, do you remember what was intended here?

[ivankra]

A good quarter of ~5k async tests seem to exhibit this behavior - had an LLM look at a sample of 100 tests. I think conclusion is clear:

The typical buggy patterns are .then(..., $DONE).then($DONE, $DONE) and .then($DONE, ...).then($DONE, $DONE): an early $DONE(error) prints failure, but because $DONE returns undefined, the
promise chain can continue and a later terminal $DONE() can still print success. In a small number of parallel-chain cases one async branch can print failure while another still reaches success. In a 100-test sample of $DONE-using tests, 27% had a real same-run mixed-marker risk, 1% were duplicate-only, and 72% were fine. Crucially, in all 27 mixed-marker cases, the author intent is clearly “fail if both markers appear”; we found 0 cases where mixed output was plausibly intended to succeed.

[ivankra]

test262-harness behavior is to fail the test if Test262:AsyncTestFailure marker is present, it doesn't even check for Test262:AsyncTestComplete:

https://github.com/tc39/test262-harness/blob/fdd44af1361b123c2c6406a5856b25a12f0c6733/lib/agent-pool.js#L64