I am facing this WEIRD behavior on ci. So what I found is running xcode tests with fastlane on circleci does not run some test cases randomly and I could not found any error or fail cases but just not running even though xcodebuild command is exactly same. So I tested to run fastlane on local but it worked all test cases as it should be.
build machine: xcode 11.5
example output 1
Starting scan #1 with 39 tests for batch #1.
....
[07:00:14]: ▸ [32;1m Executed 39 tests, with 0 failures (0 unexpected) in 89.963 (102.243) seconds
[07:00:14]: ▸ [0m
[07:00:15]: ▸ 2021-05-18 07:00:15.119 xcodebuild[1028:22186] [MT] IDETestOperationsObserverDebug: 143.517 elapsed -- Testing started completed.
[07:00:15]: ▸ 2021-05-18 07:00:15.119 xcodebuild[1028:22186] [MT] IDETestOperationsObserverDebug: 0.000 sec, +0.000 sec -- start
[07:00:15]: ▸ 2021-05-18 07:00:15.119 xcodebuild[1028:22186] [MT] IDETestOperationsObserverDebug: 143.517 sec, +143.517 sec -- end
[07:00:17]: ▸ Test execute Succeeded
+--------------------+----+
| Test Results |
+--------------------+----+
| Number of tests | 37 |
| Number of failures | 0 |
+--------------------+----+
example output 2
Starting scan #1 with 39 tests for batch #1.
...
[11:29:20]: ▸ [32;1m Executed 39 tests, with 0 failures (0 unexpected) in 90.384 (102.720) seconds
[11:29:20]: ▸ [0m
[11:29:20]: ▸ 2021-05-18 11:29:20.469 xcodebuild[1010:15473] [MT] IDETestOperationsObserverDebug: 187.177 elapsed -- Testing started completed.
[11:29:20]: ▸ 2021-05-18 11:29:20.469 xcodebuild[1010:15473] [MT] IDETestOperationsObserverDebug: 0.000 sec, +0.000 sec -- start
[11:29:20]: ▸ 2021-05-18 11:29:20.469 xcodebuild[1010:15473] [MT] IDETestOperationsObserverDebug: 187.177 sec, +187.177 sec -- end
[11:29:21]: ▸ Test execute Succeeded
+--------------------+----+
| Test Results |
+--------------------+----+
| Number of tests | 38 |
| Number of failures | 0 |
+--------------------+----+
example output 3
Starting scan #1 with 39 tests for batch #1.
...
[20:52:01]: ▸ [32;1m Executed 39 tests, with 0 failures (0 unexpected) in 84.788 (96.877) seconds
[20:52:01]: ▸ [0m
[20:52:02]: ▸ 2021-05-17 20:52:02.037 xcodebuild[881:14755] [MT] IDETestOperationsObserverDebug: 148.546 elapsed -- Testing started completed.
[20:52:02]: ▸ 2021-05-17 20:52:02.037 xcodebuild[881:14755] [MT] IDETestOperationsObserverDebug: 0.000 sec, +0.000 sec -- start
[20:52:02]: ▸ 2021-05-17 20:52:02.038 xcodebuild[881:14755] [MT] IDETestOperationsObserverDebug: 148.546 sec, +148.546 sec -- end
[20:52:02]: ▸ Test execute Succeeded
+--------------------+----+
| Test Results |
+--------------------+----+
| Number of tests | 39 |
| Number of failures | 0 |
+--------------------+----+
Any idea…?