Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

The active test run was aborted. Reason: Test host process crashed #451

Open
3 tasks
fanyang-mono opened this issue Jul 25, 2023 · 8 comments
Open
3 tasks

Comments

@fanyang-mono
Copy link
Member

fanyang-mono commented Jul 25, 2023

Build

https://dev.azure.com/dnceng-public/cbb18261-c48f-4abb-8651-8cdcb5474649/_build/results?buildId=351450

Build leg reported

chrome-DebuggerTests.EvaluateOnCallFrameTests.WorkItemExecution

Pull Request

dotnet/runtime#89217

Known issue core information

Fill out the known issue JSON section by following the step by step documentation on how to create a known issue

 {
    "ErrorMessage" : "The active test run was aborted. Reason: Test host process crashed",
    "BuildRetry": false,
    "ErrorPattern": "",
    "ExcludeConsoleLog": false
 }

@dotnet/dnceng

Release Note Category

  • Feature changes/additions
  • Bug fixes
  • Internal Infrastructure Improvements

Release Note Description

Additional information about the issue reported

No response

Report

Build Definition Step Name Console log Pull Request
928955 dotnet/roslyn Run Integration Tests Log
928927 dotnet/roslyn Run Integration Tests Log dotnet/roslyn#76918
928738 dotnet/roslyn Run Integration Tests Log
927094 dotnet/roslyn Run Integration Tests Log
2625793 dotnet-interactive Test / Blame Log
924837 dotnet/roslyn Run Integration Tests Log dotnet/roslyn#76852
2625489 dotnet-interactive Test / Blame Log
2625493 dotnet-interactive Test / Blame Log
923681 dotnet/roslyn Run Integration Tests Log
923659 dotnet/roslyn Run Integration Tests Log
921574 dotnet/roslyn Run Integration Tests Log dotnet/roslyn#75581
2620937 dotnet-interactive Test / Blame Log
2620694 dotnet-interactive Test / Blame Log
917698 dotnet/roslyn Run Integration Tests Log dotnet/roslyn#76752
2619106 dotnet-interactive Test / Blame Log
914993 dotnet/roslyn Run Integration Tests Log dotnet/roslyn#76725
914308 dotnet/roslyn Run Integration Tests Log
914177 dotnet/roslyn Run Integration Tests Log dotnet/roslyn#76692
913987 dotnet/roslyn Run Integration Tests Log
912714 dotnet/roslyn Run Integration Tests Log dotnet/roslyn#76695
2616802 dotnet-interactive Test / Blame Log
2616717 dotnet-interactive Test / Blame Log
2615694 dotnet-fsharp CmdLine@2 Log
2615469 dotnet-interactive Test / Blame Log
2614768 dotnet-interactive Test / Blame Log
909122 dotnet/roslyn Run Integration Tests Log dotnet/roslyn#75970
2614588 dotnet-fsharp CmdLine@2 Log
908618 dotnet/roslyn Run Integration Tests Log dotnet/roslyn#76649
908555 dotnet/roslyn Run Integration Tests Log
908425 dotnet/roslyn Run Integration Tests Log dotnet/roslyn#76646
2614116 dotnet-interactive Test / Blame Log
2614101 dotnet-interactive Test / Blame Log
2613990 dotnet-interactive Test / Blame Log
2613544 dotnet-interactive Test / Blame Log
2612493 dotnet-interactive Test / Blame Log
906308 dotnet/roslyn Run Integration Tests Log
2612433 dotnet-interactive Test / Blame Log
2612333 dotnet-interactive Test / Blame Log
905800 dotnet/roslyn Run Integration Tests Log
2611980 dotnet-interactive Test / Blame Log
2611872 dotnet-interactive Test / Blame Log
2611807 dotnet-interactive Test / Blame Log
905108 dotnet/roslyn Run Integration Tests Log dotnet/roslyn#70455
905102 dotnet/roslyn Run Integration Tests Log
903012 dotnet/roslyn Run Integration Tests Log
Build Definition Test Pull Request
929297 dotnet/sdk Microsoft.NET.Publish.Tests.dll.5.WorkItemExecution dotnet/sdk#46297
929276 dotnet/aspnetcore Microsoft.AspNetCore.OutputCaching.Tests--net8.0.WorkItemExecution
929143 dotnet/sdk dotnet.Tests.dll.16.WorkItemExecution dotnet/sdk#46298
929217 dotnet/sdk Microsoft.NET.Sdk.Razor.Tests.dll.11.WorkItemExecution dotnet/sdk#46261
929122 dotnet/sdk Microsoft.NET.Publish.Tests.dll.5.WorkItemExecution dotnet/sdk#46297
929118 dotnet/sdk Microsoft.NET.Sdk.BlazorWebAssembly.Tests.dll.4.WorkItemExecution dotnet/sdk#46296
929062 dotnet/sdk Microsoft.NET.Sdk.Razor.Tests.dll.11.WorkItemExecution dotnet/sdk#46295
2627837 dotnet-aspnetcore Microsoft.AspNetCore.Mvc.FunctionalTests--net10.0.WorkItemExecution
928709 dotnet/sdk dotnet-watch.Tests.dll.1.WorkItemExecution dotnet/sdk#46183
928756 dotnet/sdk dotnet-watch.Tests.dll.3.WorkItemExecution dotnet/sdk#46285
928809 dotnet/sdk dotnet-watch.Tests.dll.1.WorkItemExecution dotnet/sdk#46286
928587 dotnet/sdk Microsoft.NET.Sdk.Razor.Tests.dll.11.WorkItemExecution dotnet/sdk#46280
928598 dotnet/sdk dotnet-watch.Tests.dll.1.WorkItemExecution dotnet/sdk#46275
927610 dotnet/sdk Microsoft.NET.Sdk.Razor.Tests.dll.11.WorkItemExecution dotnet/sdk#46257
928195 dotnet/sdk dotnet-watch.Tests.dll.1.WorkItemExecution dotnet/sdk#46244
928112 dotnet/sdk dotnet-watch.Tests.dll.1.WorkItemExecution dotnet/sdk#46261
928278 dotnet/sdk msbuild.Integration.Tests.dll.WorkItemExecution dotnet/sdk#46275
928130 dotnet/sdk Microsoft.NET.Build.Tests.dll.23.WorkItemExecution dotnet/sdk#46269
927929 dotnet/sdk Microsoft.NET.Sdk.Razor.Tests.dll.11.WorkItemExecution dotnet/sdk#46045
927969 dotnet/aspnetcore Microsoft.AspNetCore.Mvc.FunctionalTests--net10.0.WorkItemExecution
927842 dotnet/sdk Microsoft.NET.Build.Tests.dll.23.WorkItemExecution dotnet/sdk#46021
927641 dotnet/sdk dotnet-watch.Tests.dll.1.WorkItemExecution dotnet/sdk#46184
927711 dotnet/sdk Microsoft.NET.Publish.Tests.dll.8.WorkItemExecution dotnet/sdk#46262
927626 dotnet/sdk dotnet-watch.Tests.dll.1.WorkItemExecution dotnet/sdk#46260
927509 dotnet/sdk Microsoft.NET.Publish.Tests.dll.11.WorkItemExecution dotnet/sdk#46183
919402 dotnet/sdk Microsoft.NET.Sdk.Razor.Tests.dll.10.WorkItemExecution dotnet/sdk#46033
927121 dotnet/sdk Microsoft.NET.Publish.Tests.dll.11.WorkItemExecution dotnet/sdk#46184
2627211 dotnet-dotnet-monitor Microsoft.Diagnostics.Monitoring.Tool.UnitTests-net9.0-x64.WorkItemExecution
926310 dotnet/sdk Microsoft.NET.Publish.Tests.dll.8.WorkItemExecution dotnet/sdk#46232
927208 dotnet/sdk Microsoft.NET.Build.Tests.dll.23.WorkItemExecution dotnet/sdk#46238
927315 dotnet/sdk Microsoft.NET.Sdk.Razor.Tests.dll.14.WorkItemExecution dotnet/sdk#46253
927428 dotnet/aspire Aspire.Hosting.Elasticsearch.Tests.WorkItemExecution dotnet/aspire#7219
927403 dotnet/aspire Aspire.Hosting.Elasticsearch.Tests.WorkItemExecution dotnet/aspire#7135
927072 dotnet/aspire Aspire.Hosting.Elasticsearch.Tests.WorkItemExecution dotnet/aspire#7224
927140 dotnet/aspire Aspire.Hosting.Elasticsearch.Tests.WorkItemExecution dotnet/aspire#7127
927075 dotnet/sdk dotnet-watch.Tests.dll.1.WorkItemExecution dotnet/sdk#46021
926765 dotnet/sdk Microsoft.NET.Build.Tests.dll.23.WorkItemExecution dotnet/sdk#46101
926900 dotnet/sdk Microsoft.NET.Sdk.Razor.Tests.dll.10.WorkItemExecution dotnet/sdk#46180
926867 dotnet/sdk Microsoft.NET.Sdk.Razor.Tests.dll.10.WorkItemExecution dotnet/sdk#46216
926917 dotnet/sdk dotnet-watch.Tests.dll.1.WorkItemExecution dotnet/sdk#46248
926938 dotnet/aspire Aspire.Hosting.Nats.Tests.WorkItemExecution dotnet/aspire#6972
926664 dotnet/sdk Microsoft.NET.Sdk.Razor.Tests.dll.14.WorkItemExecution dotnet/sdk#46238
926743 dotnet/sdk dotnet-watch.Tests.dll.1.WorkItemExecution dotnet/sdk#46021
926618 dotnet/sdk Microsoft.NET.Publish.Tests.dll.10.WorkItemExecution dotnet/sdk#46237
2626659 dotnet-aspnetcore Microsoft.AspNetCore.Mvc.FunctionalTests--net10.0.WorkItemExecution
926376 dotnet/sdk dotnet-watch.Tests.dll.3.WorkItemExecution dotnet/sdk#46234
926330 dotnet/aspire Aspire.Hosting.Elasticsearch.Tests.WorkItemExecution dotnet/aspire#7214
926113 dotnet/sdk Microsoft.NET.Sdk.Razor.Tests.dll.11.WorkItemExecution dotnet/sdk#46222
926284 dotnet/aspnetcore Templates.Blazor.WebAssembly.Tests--net10.0.WorkItemExecution
926303 dotnet/sdk dotnet-watch.Tests.dll.2.WorkItemExecution dotnet/sdk#46021
926156 dotnet/sdk Microsoft.NET.Sdk.Razor.Tests.dll.14.WorkItemExecution dotnet/sdk#45552
925430 dotnet/sdk Microsoft.NET.Sdk.Razor.Tests.dll.11.WorkItemExecution dotnet/sdk#46213
925972 dotnet/sdk Microsoft.NET.Sdk.Razor.Tests.dll.11.WorkItemExecution dotnet/sdk#46220
924839 dotnet/sdk Microsoft.NET.Build.Tests.dll.23.WorkItemExecution dotnet/sdk#46198
926022 dotnet/sdk Microsoft.NET.Sdk.Razor.Tests.dll.14.WorkItemExecution dotnet/sdk#45552
926028 dotnet/sdk dotnet-watch.Tests.dll.1.WorkItemExecution dotnet/sdk#45969
926000 dotnet/sdk Microsoft.NET.Sdk.Razor.Tests.dll.11.WorkItemExecution dotnet/sdk#46222
925915 dotnet/sdk Microsoft.NET.Sdk.Razor.Tests.MvcBuildIntegrationTest22.Microsoft.NET.Sdk.Razor.Tests.MvcBuildIntegrationTest22.Build_ProducesDepsFileWithCompilationContext_ButNoReferences dotnet/sdk#46218
925669 dotnet/sdk Microsoft.NET.Build.Tests.dll.23.WorkItemExecution dotnet/sdk#46171
925579 dotnet/sdk Microsoft.NET.Sdk.Razor.Tests.dll.11.WorkItemExecution dotnet/sdk#46186
925697 dotnet/sdk msbuild.Integration.Tests.dll.WorkItemExecution dotnet/sdk#46164
925672 dotnet/sdk Microsoft.NET.Sdk.Razor.Tests.dll.11.WorkItemExecution dotnet/sdk#45932
925679 dotnet/sdk dotnet-watch.Tests.dll.1.WorkItemExecution dotnet/sdk#46063
925655 dotnet/sdk Microsoft.NET.Sdk.Razor.Tests.dll.11.WorkItemExecution dotnet/sdk#46219
925467 dotnet/sdk Microsoft.NET.Build.Tests.dll.23.WorkItemExecution dotnet/sdk#46215
925445 dotnet/sdk Microsoft.NET.Sdk.Razor.Tests.dll.14.WorkItemExecution dotnet/sdk#46214
925439 dotnet/sdk Microsoft.NET.Sdk.Razor.Tests.dll.14.WorkItemExecution dotnet/sdk#46108
925397 dotnet/sdk Microsoft.NET.Build.Tests.dll.23.WorkItemExecution dotnet/sdk#45932
925341 dotnet/sdk Microsoft.NET.Sdk.BlazorWebAssembly.Tests.dll.4.WorkItemExecution dotnet/sdk#46211
925350 dotnet/sdk Microsoft.NET.Build.Tests.dll.23.WorkItemExecution dotnet/sdk#45932
925270 dotnet/sdk Microsoft.NET.Sdk.Razor.Tests.dll.11.WorkItemExecution dotnet/sdk#46209
925259 dotnet/sdk Microsoft.NET.Sdk.Razor.Tests.dll.14.WorkItemExecution dotnet/sdk#45362
925289 dotnet/sdk dotnet-watch.Tests.dll.1.WorkItemExecution dotnet/sdk#46164
925295 dotnet/sdk Microsoft.NET.Sdk.Razor.Tests.dll.11.WorkItemExecution dotnet/sdk#46045
925308 dotnet/aspire Aspire.Hosting.Elasticsearch.Tests.WorkItemExecution dotnet/aspire#7206
925257 dotnet/aspnetcore Microsoft.AspNetCore.Mvc.FunctionalTests--net10.0.WorkItemExecution
925292 dotnet/sdk Microsoft.NET.Sdk.Razor.Tests.dll.11.WorkItemExecution dotnet/sdk#46210
925223 dotnet/aspnetcore Microsoft.AspNetCore.Mvc.FunctionalTests--net10.0.WorkItemExecution
925046 dotnet/sdk Microsoft.NET.Sdk.Razor.Tests.dll.20.WorkItemExecution dotnet/sdk#46183
925058 dotnet/sdk Microsoft.NET.Publish.Tests.dll.8.WorkItemExecution dotnet/sdk#45362
924808 dotnet/sdk Microsoft.NET.Publish.Tests.dll.8.WorkItemExecution dotnet/sdk#46193
925026 dotnet/sdk Microsoft.NET.Sdk.Razor.Tests.dll.11.WorkItemExecution dotnet/sdk#46184
924803 dotnet/sdk Microsoft.NET.Sdk.Razor.Tests.dll.14.WorkItemExecution dotnet/sdk#46191
924794 dotnet/sdk Microsoft.NET.Publish.Tests.dll.10.WorkItemExecution dotnet/sdk#46190
924881 dotnet/aspire Aspire.Hosting.Azure.Tests.AzureCosmosDBEmulatorFunctionalTests.Aspire.Hosting.Azure.Tests.AzureCosmosDBEmulatorFunctionalTests.VerifyWaitForOnCosmosDBEmulatorBlocksDependentResources(usePreview: False) dotnet/aspire#7163
924843 dotnet/sdk dotnet-watch.Tests.dll.1.WorkItemExecution dotnet/sdk#46200
924723 dotnet/sdk Microsoft.NET.Sdk.Razor.Tests.dll.11.WorkItemExecution dotnet/sdk#46164
924733 dotnet/aspnetcore Microsoft.AspNetCore.Http.Extensions.Tests--net10.0.WorkItemExecution dotnet/aspnetcore#53395
924703 dotnet/sdk dotnet-watch.Tests.dll.1.WorkItemExecution dotnet/sdk#46180
924083 dotnet/sdk Microsoft.NET.Sdk.Razor.Tests.dll.14.WorkItemExecution dotnet/sdk#46168
2625568 dotnet-roslyn Microsoft.CodeAnalysis.VisualBasic.Features.UnitTests_Microsoft.CodeAnalysis.VisualBasic.ExpressionEvaluator.ResultProvider.UnitTests_Microsoft.CodeAn..._246.WorkItemExecution
924379 dotnet/sdk dotnet-watch.Tests.dll.6.WorkItemExecution dotnet/sdk#46141
924465 dotnet/sdk Microsoft.NET.Sdk.Razor.Tests.dll.10.WorkItemExecution dotnet/sdk#46179
924554 dotnet/sdk Microsoft.NET.Sdk.Razor.Tests.dll.11.WorkItemExecution dotnet/sdk#46171
924389 dotnet/sdk Microsoft.NET.Sdk.Razor.Tests.dll.11.WorkItemExecution dotnet/sdk#46177
924466 dotnet/sdk dotnet-watch.Tests.dll.1.WorkItemExecution dotnet/sdk#46100
924419 dotnet/sdk dotnet-watch.Tests.dll.1.WorkItemExecution dotnet/sdk#46178
924302 dotnet/sdk Microsoft.NET.Sdk.Razor.Tests.dll.11.WorkItemExecution dotnet/sdk#46108
924106 dotnet/sdk Microsoft.NET.Build.Tests.dll.23.WorkItemExecution dotnet/sdk#45362
923505 dotnet/sdk Microsoft.NET.Sdk.BlazorWebAssembly.Tests.dll.4.WorkItemExecution dotnet/sdk#46002
Displaying 100 of 475 results

Summary

24-Hour Hit Count 7-Day Hit Count 1-Month Count
23 158 520

Known issue validation

Build: 🔎
Result validation: ⚠️ Validation could not be done without an Azure DevOps build URL on the issue. Please add it to the "Build: 🔎" line.
Validation performed at: 2/8/2024 9:24:30 PM UTC

@andriipatsula
Copy link
Member

Hello @fanyang-mono, could you please update the "ErrorMessage" : "" by following the step by step documentation on how to create a known issue

@fanyang-mono
Copy link
Member Author

Updated.

@pavelsavara
Copy link
Member

pavelsavara commented Nov 7, 2023

This is duplicate of dotnet/runtime#89409, except here it's not only wasm debugger

@dougbu
Copy link
Member

dougbu commented Nov 23, 2023

@pavelsavara and @fanyang-mono, should this issue be closed in favour of dotnet/runtime#94560❔ if not, the error message isn't sufficiently precise to understand what (if anything) we (DNCEng) could do about it.

@pavelsavara
Copy link
Member

@pavelsavara and @fanyang-mono, should this issue be closed in favour of

You mean dotnet/runtime#89409

I think we should keep this one, because this one has some hits which are not just wasm debugger related.
Here we see other repo failures, on roslyn and aspnetcore CI.

if not, the error message isn't sufficiently precise to understand what (if anything) we (DNCEng) could do about it.

I have not investigated it myself yet. But got questions which could help to narrow it down

  • are there crash dump files on the machine ?
  • are there some hints on what's going on in the log files before crash ?
  • is this OOM or stack overflow in disguise ?
  • did we run out of disk space or number of inodes ?
  • hit some limit in docker container ?
  • is this always the same OS, version, image ?
  • could this be hardware issue ?

@fanyang-mono
Copy link
Member Author

This issue was originally created to track issues happened to wasm debugger tests (belonging to dotnet/runtime repo). However, the error message was generic enough to include issues from other repos, such as dotnet/roslyn and dotnet/aspnetcore. We need insights from owners from dotnet/roslyn and dotnet/aspnetcore to confirm that they are aware of these issue. And if they are okay to close this issue as well. We could close then.

@mitchdenny
Copy link
Member

We just got an instance of this in the dotnet/aspire repo. It wasn't a WASM related failure for us. Just looks like the test process crashed and took a few tests with it.

@nagilson
Copy link
Member

nagilson commented Dec 3, 2024

@marcpopMSFT was working on getting the test infrastructure to generate crash dumps when it times out to help with investigation. I think it was generating the dumps but not yet correctly uploading them to the artifacts.

We are waiting to hear from the infra team on an issue with this.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

8 participants