Cannot filter/select tests with commas in their name

[natinusala]

I noticed that I am unable to select and run a single test if it contains a comma in its name. XCTest doesn't find the test and it runs 0 tests instead of the one I selected.

The issue comes from this line: https://github.com/apple/swift-corelibs-xctest/blob/release/5.7/Sources/XCTest/Private/ArgumentParser.swift#L69

As you can see it expects tests names to be separated by a comma in the CLI invocation. This breaks tests that contain a comma in their name, as it will split the properly formed test name into multiple non existent tests.

This is a problem when using Quick that creates test names following the pattern test subject, given that, when I do this, it does that.

As you can imagine when running swift test --filter "test subject, given that, when I do this, it does that" (or the testing .xctest binary directly) it tries to run 4 tests: test subject, given that, when I do this, it does that and ends up doing nothing.

I made a minimal project that reproduces this issue: https://github.com/natinusala/xctest-comma

xctest-comma [master] % swift test -l
Building for debugging...
Build complete! (0.13s)
xctest_commaTests.xctest_commaTests/testExample with spaces
xctest_commaTests.xctest_commaTests/testExample, with a comma

xctest-comma [master] % swift test --filter "xctest_commaTests.xctest_commaTests/testExample with spaces"
Building for debugging...
Build complete! (0.07s)
Test Suite 'Selected tests' started at 2022-08-21 13:43:06.766
Test Suite 'xctest_commaTests' started at 2022-08-21 13:43:06.788
Test Case 'xctest_commaTests.testExample with spaces' started at 2022-08-21 13:43:06.788
Test Case 'xctest_commaTests.testExample with spaces' passed (0.003 seconds)
Test Suite 'xctest_commaTests' passed at 2022-08-21 13:43:06.791
Executed 1 test, with 0 failures (0 unexpected) in 0.003 (0.003) seconds
Test Suite 'Selected tests' passed at 2022-08-21 13:43:06.791
Executed 1 test, with 0 failures (0 unexpected) in 0.003 (0.003) seconds

xctest-comma [master] % swift test --filter "xctest_commaTests.xctest_commaTests/testExample, with a comma"
Building for debugging...
Build complete! (0.08s)
Test Suite 'Selected tests' started at 2022-08-21 13:43:22.281
Test Suite 'Selected tests' passed at 2022-08-21 13:43:22.284
Executed 0 tests, with 0 failures (0 unexpected) in 0.0 (0.0) seconds

I would expect the last command to run 1 test instead if 0.

This is on Linux. While I don't have a Mac computer with me to try, I do believe the macOS version of XCTest does not have this issue (since Quick has been around for some time on Xcode and it seems to work there from what I've gathered).

[ysabih]

I've had the same issue on Linux. I reported it here.