Xcode Exit Code 65 + UI Tests

Hey everyone,

I was traveling yesterday so I couldn’t update this. I think I have identified one of the issues that is causing this and will try some things in the next days and keep you updated:

hint: It’s code signing.

more hint: been working with Xcode 8.1 and 8.2 beta. Both will be ready for you soon!

Best,
Constantin

1 Like

Sorry, the form ate my angle brackets. It should be a subfolder inside the project’s folder where the xcactivity logs are located. Please file with as much information as you can get.

Danielle is in town, I reached out to see if she wants to visit Cupertino so we can triage these problems in the lab.

Filled radar 29108711

Xcode 8.1 didn’t help. Simulator run insists on wanting signed pod frameworks.
https://circleci.com/gh/kitt-browser/adblockplus-ios/1180

@pavel-zdenek Your radar (29108711) is about codesigning errors.

dyld_sim is reporting an error that your framework isn’t signed.

If you take a look at your attached build log, you will see that this is because you have passed CODE_SIGNING_REQUIRED=NO CODE_SIGNING_ALLOWED=NO CODE_SIGN_IDENTITY= to xcodebuild to specifically request that it not be signed. The system is behaving correctly given the way you have configured your build settings.

I strongly suggest you do not modify CODE_SIGNING_ALLOWED and CODE_SIGNING_REQUIRED. They are set by the SDK’s SDKSettings.plist.

Wait a minute. Simulator builds are not required and not supposed to be signed. Those params are just saying it visibly. The current setup was working up to Xcode 7.3, and the build logs never had those those new “Signing Identity: -” lines from Xcode8. I considered those lines an error, and “allowing” the signing even more was the last thing i would do. So, since Xcode8, is “signing” required, even if the identity is “-” (a dash) ? It cannot result in any kind of valid signing anyway.

As of iOS 10, the simulator runtime requires that all images be signed. This isn’t an Xcode 8 change, it’s an iOS 10 change. That is why the iOS 10 SDK’s SDKSettings.plist has CODE_SIGNING_REQUIRED=YES. By setting CODE_SIGNING_REQUIRED=NO, you’re overriding those settings and pushing the error from a build failure to a runtime failure.

Yes, in Xcode 7.3, you got away with it because it was not a requirement. Xcode 7.3 also did not sign simulator products, so if you try to run something built by Xcode 7.3 in the iOS 10 simulator, it will fail to load (unless you sign it afterwards).

“-” indicates that adhoc signing is being used. That is allowed by the simulator runtime but not by device (see the AD_HOC_CODE_SIGNING_ALLOWED build setting in each of the SDK’s SDKSettings.plist).

It is a known issue in Xcode that adhoc is used for the sim when a real identity is requested, but it’s not problematic because we still allow adhoc signatures in the sim.

I considered those lines an error

I think that’s what set you down the wrong path.

So, since Xcode8, is “signing” required, even if the identity is “-” (a dash) ?

iOS 10, watchOS 3, and tvOS 10 simulator runtimes will fail to map executable code that does not contain a valid code signature. Adhoc signatures are allowed.

It cannot result in any kind of valid signing anyway.

What do you mean by that?

If anyone has any data about these error code 65 errors, we’d really appreciate data points. As Russ mentioned above, please file radars that include:

~/Library/Logs/CoreSimulator
The tarball result from sudo sysdiagnose -q. If you cannot get that due to permissions, then at least grab /var/log/system*

how can i get these log files when error code 65 occurs? afaik, when error 65 occurs, the machine is taken offline and the build started again, even if ssh is enabled
 i’d love to be able to ssh in to one of these builds once it fails, but haven’t found a way yet

Hopefully a CircleCI admin can (with your approval) gather than data and hand it off to us for analysis.

be my guest!

I set up a Mac Mini without any credentials and created a regular user to work in. I’ve got a set of fastlane scripts which install signing credentials and download provisioning profiles for the unit test and main target, so code signing should work, just in case it’s an issue. These targets code sign using “-” anyhow, if the Circle default build setup does not do that.

Running in this environment I’m able to reproduce this error but not easily. It requires that the regular user hasn’t approved “debugging” access to the developer tools via admin credentials. This doesn’t appear to pop up the first time I run the fastlane script, but it does happen the second. Subsequent builds work fine.

Do the Circle CI tests run using a regular user on the virtual machine or an administrator?

Hey everyone,

I haven’t updated the this thread in a while, like I wanted to but we are struggling with a lot of things at the moment but things are looking better.

One feature that we rolled out lately is that we know collect xcactivitylogs which give much more information about certain failures. Please take a look in your artifacts.

Best,
Constantin

Just wanted to chime in to say that upgrading our config to use Xcode 8.1 hasn’t fixed this issue for us.

Hey everyone,

I’ve been doing some digging and facepalmed a little when I noticed that the simulators take even longer to boot by now


Could you please try this:

xcrun instruments -w 'sim uuid' || true && sleep 60

I think this might be related to some Exit code 65 issues but not all, and it probably wont fix the UI tests issue. It’s just something to try
 but not yet last resort.

Best,
Constantin

Hey Constantin,

of course these 60 (!) seconds will not be taken into account for our build-minutes, right?
We already wasted quite a lot of minutes due to the problems with the Cocoapods Specs-repo cloning-issue, our builds took up to 45, 50 minutes then.
Since there was no solution in sight from Circles side and the Specs-repo was unfortunately not sharded back then, we created our own Specs repo with only the Specs we needed.
Then, upgrade to Xcode 8.1, the multiple devices issue.
Simulators not starting, crazy 65er errors, tbh, we are currently wasting a lot of time fixing issues with Circle.
This is not your fault, but I have the feeling that Circle should urgently hire more developers, because as far as I see it, you are too few people having a lot of pressure, which inevitably leads to regressions and long times for issues to be resolved.

6 Likes

Is there a chance we get some news one day?

1 Like

Any updates on the ‘exit code 65’ issue?

Up.

My builds are still failing oftenly because of error 65.

Hey Ben,

Could you please open a support ticket? I’d like to take a direct look at your issue.