Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix flaky domain tests #2220

Merged
merged 1 commit into from
Jan 20, 2024
Merged

Fix flaky domain tests #2220

merged 1 commit into from
Jan 20, 2024

Conversation

NingLin-P
Copy link
Member

@NingLin-P NingLin-P commented Nov 10, 2023

This PR tries to fix the flaky domain tests, from the log of the previous failed CI test there are 2 cases that cause the test to fail:

  1. the mock consensus node can't get the already submitted bundle from its tx pool, which is supposed to be included in block #37, this may be due to there is a 500 microsecond timeout limit to get tx from the pool, and the timeout may be passed due to kernel interrupt before getting the tx, this PR fix it by removing the timeout limit in the test.
2023-11-07 12:08:24+00:00 [//Alice (DomainId(0))] 📦 Claimed bundle at slot 37
2023-11-07 12:08:24+00:00 [//Alice (DomainId(0))] 🔖 Producing bundle at slot Slot(37)
2023-11-07 12:08:24+00:00 [//Alice (DomainId(0))] Submitted bundle from slot 37, extrinsics: 0    
2023-11-07 12:08:24+00:00 [Ferdie] Timeout fired waiting for transaction pool at #36, proceeding with production.
2023-11-07 12:08:24+00:00 [Ferdie] 🎁 Prepared block for proposing at 37 (22 ms) [hash: 0x8854f7970246cb69c5683c6f5d361f05504514484952cc3887c0a9adfb7c09d6; parent_hash: 0x96ec…0ffc; extrinsics (1): [0xe985…013d]]
2023-11-07 12:08:24+00:00 [//Ferdie (Consensus)] ✨ Imported #37 (0x8854…09d6)    
2023-11-07 12:08:24+00:00 [Ferdie] 🎁 Prepared block for proposing at 38 (1 ms) [hash: 0xff223c00f5d50994a630af93c948b5ebf967bf63c3bbad6dde264a3b0b88dc72; parent_hash: 0x8854…09d6; extrinsics (1): [0xeb31…7fd7]]
2023-11-07 12:08:24+00:00 [//Ferdie (Consensus)] ✨ Imported #38 (0xff22…dc72)
  1. the test tries to get the bundle from the tx pool before the bundle is submitted, this seems to be due to the confirm_acknowledgement returning early before the slot notification is finished processed:
thread 'tests::test_domain_block_production' panicked at domains\client\domain-operator\src\tests.rs:189:39:
called `Option::unwrap()` on a `None` value
note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace
2023-11-03 22:28:15+00:00 [//Alice (DomainId(0))] 📦 Claimed bundle at slot 74
2023-11-03 22:28:15+00:00 [//Alice (DomainId(0))] 🔖 Producing bundle at slot Slot(74)
thread 'tokio-runtime-worker' panicked at C:\actions-runner\_work\subspace\subspace\domains\client\domain-operator\src\domain_worker_starter.rs:170:13:
all branches are disabled and there is no else branch
2023-11-03 22:28:15+00:00 [//Alice (DomainId(0))] Submitted bundle from slot 74, extrinsics: 0

It is suspected cause by smol-rs/async-channel#54, this PR try to use the future::channel workaround the issue.

Code contributor checklist:

Signed-off-by: linning <[email protected]>
@NingLin-P
Copy link
Member Author

PTAL, let's get this in to see how it goes.

@nazar-pc nazar-pc added this pull request to the merge queue Jan 20, 2024
Merged via the queue into main with commit b735521 Jan 20, 2024
10 checks passed
@nazar-pc nazar-pc deleted the fix-domain-test branch January 20, 2024 01:20
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants