We’ve been having an issue with CircleCI executing tests in which tests compile but never get executed and we end up with the following message: Too long with no output (exceeded 30m0s): context deadline exceeded
We’ve tried different things such as increasing the timeout timer to 30 min to no avail. Note that we are able to execute the tests locally via fastlane or Xcode without an issue.
Here is the fastlane action we use to execute tests:
esc 'Run tests'
lane :test do |options|
UI.message "Running on CI, CocoaPods have been installed already"
workspace = "project.xcworkspace"
scheme = "project"
configuration = "Unit Test"
device = "iPad (10th generation)"
configuration = "Unit Test"
UI.message "Running on CI, use Unit Test configuration"
configuration = "Debug"
UI.message "Not running on CI, use Debug configuration"
output_directory = options[:output_directory]
And this is what our config.yml file looks like:
shell: /bin/bash --login -o pipefail
device: "iPad (10th generation)"
- run: xcodebuild -version
- run: brew update && brew install swiftlint
- run: git submodule update --init
name: Set Ruby Version
command: echo "ruby-3.1.4" > ~/.ruby-version
- run: sudo gem install bundler:2.3.26
- run: bundle install
# The following two lines were added to address a bug with cocoapods 1.13.0
# Remove once bug is fixed.
- run: yes | sudo gem uninstall --force activesupport -v 7.1.1 || result=$?
- run: yes | sudo gem install activesupport -v 7.0.8 || result=$?
name: Install CocoaPods
command: pod install --verbose
name: Build and Run Tests
command: bundle exec fastlane test
I increased the timeout to 90 min and still got the same result:
Too long with no output (exceeded 1h30m0s): context deadline exceeded
I have an M1 Macbook Pro with 32GB of memory.
It’s worth noting that in both instances (30 min and 90 min timeout), the last lines of code executing as part of the tests are the same. So somehow it’s hanging when compiling the following Airship (3rd party pod) files:
I do not know much about Mac environments or the applications, but I’m pretty sure that your 10-core, 32GB laptop is going to handle certain tasks better than the 4 core 6GB instance that is being used as the default CircleCI system.
As you have a working local process it would be worth checking the resources it uses as it completes the tests.
Well, a near-constant 100% CPU load from 14m to 47m indicates that you are totally CPU bound. I’m guessing if you were to collect stats on your local system with its 10 cores you would see much the same but it would be doing more work over the same time period.
RAM may also be a limitation as CPU time may be allocated to memory management such as page compression/decompression tasks.
Which CircleCI pricing plan are you using? If you are on a paid plan you could upgrade to the larger M1 instance and see what happens.
As @lukeredpath has stated, this is most likely due to the performance regression in Apple’s latest Xcode (Xcode 15). We have re-released our Xcode 15.1.0 (Xcode 15.1 Beta 2) image, now based on macOS 14 Sonoma, and it has greatly improved the performance of the executors, albeit still not to the level of performance from Xcode 14.
Due to the large number of breaking changes (with a base OS upgrade), we will not be re-releasing Xcode 15.0.1 but rather keep building on macOS 14 going forward.
Please feel free to try out 15.1.0 and you should seem the timeouts decrease dramatically.