Simulators missing from macOS executors?

Curious if anyone else is seeing this today. On a few different branches, xcodebuild commands are failing because the only available simulators are the “Any”/placeholder ones.

When invoking xcodebuild directly with a platform and device name, I get failures like:

xcodebuild: error: Unable to find a device matching the provided destination specifier:
                { platform:iOS Simulator, OS:latest, name:iPhone 16 Pro }

        The requested device could not be found because no available devices matched the request.

        Available destinations for the "Debug" scheme:
                { platform:iOS, id:dvtdevice-DVTiPhonePlaceholder-iphoneos:placeholder, name:Any iOS Device }
                { platform:iOS Simulator, id:dvtdevice-DVTiOSDeviceSimulatorPlaceholder-iphonesimulator:placeholder, name:Any iOS Simulator Device }

Whereas fastlane appears to boot the simulator I want and even find its UUID, but then xcodebuild can’t find the device:

[15:48:38]: Disabling 'Slide to Type' iPhone 16 Pro
[15:48:38]: $ /usr/libexec/PlistBuddy -c "Add :KeyboardContinuousPathEnabled bool false" /Users/distiller/Library/Developer/CoreSimulator/Devices/C9A8F13F-30A8-4696-9274-15F3C5148BD1/data/Library/Preferences/com.apple.keyboard.ContinuousPath.plist >/dev/null 2>&1
[15:48:38]: Booting iPhone 16 Pro
[15:52:13]: $ set -o pipefail && env NSUnbufferedIO=YES xcodebuild -workspace ./Zazu.xcworkspace -scheme Debug -derivedDataPath derived_data -destination 'platform=iOS Simulator,id=C9A8F13F-30A8-4696-9274-15F3C5148BD1' build-for-testing | tee '/Users/distiller/Library/Logs/scan/Chirp-Debug.log' | xcbeautify
[15:52:25]: ▸ Loading...
[15:52:30]: ▸ ----- xcbeautify -----
[15:52:30]: ▸ Version: 2.15.0
[15:52:30]: ▸ ----------------------
[15:53:08]: ▸ 2024-12-03 15:53:08.626 xcodebuild[16933:50789] Writing error result bundle to /var/folders/z5/jct14vd57x7dwp2vbks29_4c0000gn/T/ResultBundle_2024-03-12_15-53-0008.xcresult
[15:53:09]: ▸ xcodebuild: error: Unable to find a device matching the provided destination specifier:
[15:53:09]: ▸           { platform:iOS Simulator, id:C9A8F13F-30A8-4696-9274-15F3C5148BD1 }
[15:53:09]: ▸   The requested device could not be found because no available devices matched the request.
[15:53:09]: ▸   Available destinations for the "Debug" scheme:
[15:53:09]: ▸           { platform:iOS, id:dvtdevice-DVTiPhonePlaceholder-iphoneos:placeholder, name:Any iOS Device }
[15:53:09]: ▸           { platform:iOS Simulator, id:dvtdevice-DVTiOSDeviceSimulatorPlaceholder-iphonesimulator:placeholder, name:Any iOS Simulator Device }

Previously a simulator matching those parameters would always exist. (And the manifest for this image agrees.)

These branches don’t have any changes in common, so I don’t think I’ve caused this. Anyone else seeing this?

This seems to be the case only on certain executor instances. I tried to work around the problem by adding steps that explicitly create the simulator with xcrun. I’ve now (sometimes) seen those steps fail because we now found duplicate devices. But removing those steps can also result in a failure, presumably because we’re back on an instance without the simulators preinstalled.

Hey Ben, I chatted with the team that owns our Mac fleet. A suggestion from them:

Can you check to see if this behavior happens on workflows that have previously succeeded? You can click the “rerun from start” button on an old, successful workflow and if it fails on the re-run, then it’ll help us narrow down that this might be an issue on our end. Let me know your results, thanks

Thanks Sebastian and team! I’ve tried a few re-runs of old workflows like that and I haven’t seen this failure yet. However, I think https://app.circleci.com/pipelines/github/BookBub/zazu-ios/2558 might be almost the same situation. Here, on the first and second run, we had some flaky tests fail (but simulators were present). Then on the third run — same code, rerunning failed tests — the simulators were missing and the job failed without being able to actually run tests. Let me know what other diagnostic info I can provide.

sending that link over to dig in with the team. In the meantime, we were doing some research and found this:

Could it have something to do with the Xcode project’s deployment target? or the version of Xcode the project was worked on with is older than the Xcode version in the Xcode image you’re using (16.1)…

Yeah, I looked into a similar issue but I don’t think that’s affecting us — otherwise I don’t think re-running jobs on the same code could ever succeed, no?

I’ve also played around with prebooting simulators earlier in our workflows, and that may have helped, but it’s hard to tell with a small sample size.

Got it, thanks. I’m still talking with the team, I hope to have an update Monday. If you do end up finding a scenario where you can get a workflow that succeeded to fail on a rerun, feel free to send it over.

1 Like

Update from the investigation today:

I’ll keep you posted when we have more. Thanks for the patience.

Thank you! If there are variants (Xcode version, target OS, etc.) that you’d like me to try, I’m happy to run some sample builds.

More investigations from our side seem to indicate a problem on Apple’s side. We saw this created today Has `xcodebuild -showdestinations`… | Apple Developer Forums which looks similar? At this point, could you check that forum and see if that is similar to the issue you’re seeing and if yes, comment that you’re seeing this too so that we can try to get a response from Apple?

Interesting! That thread theorizes that this is something particular to macOS 15.1. Is it possible that some macOS executors are on Sequoia and some aren’t? We’re specifying a fixed Xcode version (16.1.0) in our config but I’m not sure how/if that affects macOS version.

I’m running Sequoia locally and xcodebuild -showdestinations seems to be working reliably for me, with a dozen invocations or so.

Interesting…any given macOS CircleCI resource class is run by a homogenous set of mac hardware. So as long as you’re keeping that resource class constant, they should all be on the same OS.

Well, that matches my mental model, at least :sweat_smile: But yeah I’m not sure how to interpret the symptoms then. I guess it’s possible that the behavior in that Apple Dev thread is intermittent? Or specific to a patch version of macOS? I’m on 15.1.1 (24B91) locally.