Skip to content

Commit

Permalink
E2E: Disable Temp Sensor Test on Windows (#5770)
Browse files Browse the repository at this point in the history
I thought we could mitigate this SDK issue in our tests, but it seems worse than previously thought:
Azure/azure-iot-sdk-csharp#2223

We thought ModuleClient would recover on subsequent reconnects, but it never recovers. I have tried fiddling with the delays, removing the explicit `OpenAsync`, allowing more reconnect attempts, etc. Nothing seems to stabilize this flakiness. I didn't resort to changing the protocol as I believe our tests should use the default (and I believe most stable) Amqp_Tcp_Only. 

I feel like the loose thread here for further investigation is the fact that our Longhaul Tests don't suffer from this issue. So it seems like something specific to the TempSensor module might be at play.

For this PR, I have disabled the TempSensor test on windows with a comment to re-enable when the SDK issue is fixed. For the windows 10 minimal suite, I switched to using EdgeAgent ping test. 

Waiting on this test run before merge:
https://dev.azure.com/msazure/One/_build/results?buildId=48491714&view=results

5 E2E runs completed successfully


## Azure IoT Edge PR checklist:

This checklist is used to make sure that common guidelines for a pull request are followed.

### General Guidelines and Best Practices
- [x] I have read the [contribution guidelines](https://github.com/azure/iotedge#contributing).
- [x] Title of the pull request is clear and informative.
- [x] Description of the pull request includes a concise summary of the enhancement or bug fix.

### Testing Guidelines
- [x] Pull request includes test coverage for the included changes.
- Description of the pull request includes 
	- [x] concise summary of tests added/modified
	- [x] local testing done.  

### Draft PRs
- Open the PR in `Draft` mode if it is:
	- Work in progress or not intended to be merged.
	- Encountering multiple pipeline failures and working on fixes.

_Note: We use the kodiakhq bot to merge PRs once the necessary checks and approvals are in place. When it merges a PR, kodiakhq converts the PR title to the commit title, PR description to the commit description, and squashes all the commits in the PR to a single commit. The net effect is that entire PR becomes a single commit. Please follow the best practices mentioned [here](https://chris.beams.io/posts/git-commit/#:~:text=The%20seven%20rules%20of%20a%20great%20Git%20commit,what%20and%20why%20vs.%20how%20For%20example%3A%20) for the PR title and description_
  • Loading branch information
and-rewsmith authored Nov 3, 2021
1 parent 54b3180 commit 3f21dce
Show file tree
Hide file tree
Showing 2 changed files with 8 additions and 1 deletion.
6 changes: 5 additions & 1 deletion builds/e2e/templates/e2e-run.yaml
Original file line number Diff line number Diff line change
@@ -1,6 +1,7 @@
# This E2E test pipeline uses the following filters in order to skip certain tests:
# Flaky: Flaky on multiple platforms
# FlakyOnArm: Flaky only on arm
# FlakyOnWindows: Flaky only on windows
# CentOsSafe: Can be run on CentOs
steps:
- pwsh: |
Expand All @@ -26,7 +27,10 @@ steps:
$(Join-Path $env:ProgramFiles 'iotedge-moby') -join ';'
if ('$(minimal)' -eq 'true')
{
$filter += '&Name~TempSensor'
# The minimal test suite on windows cannot use TempSensor test.
# Using another test for now until this issue is resolved:
# https://github.com/Azure/azure-iot-sdk-csharp/issues/2223
$filter += '&Name~TestPing'
}
dotnet test $testFile --no-build --logger 'trx' --filter "$filter"
Expand Down
3 changes: 3 additions & 0 deletions test/Microsoft.Azure.Devices.Edge.Test/Module.cs
Original file line number Diff line number Diff line change
Expand Up @@ -19,6 +19,9 @@ public class Module : SasManualProvisioningFixture

[Test]
[Category("CentOsSafe")]
// This test should be disabled on windows until the following is resolved:
// https://github.com/Azure/azure-iot-sdk-csharp/issues/2223
[Category("FlakyOnWindows")]
public async Task TempSensor()
{
string sensorImage = Context.Current.TempSensorImage.GetOrElse(DefaultSensorImage);
Expand Down

0 comments on commit 3f21dce

Please sign in to comment.