Xcode Exit Code 65 + UI Tests

be my guest!

I set up a Mac Mini without any credentials and created a regular user to work in. I’ve got a set of fastlane scripts which install signing credentials and download provisioning profiles for the unit test and main target, so code signing should work, just in case it’s an issue. These targets code sign using “-” anyhow, if the Circle default build setup does not do that.

Running in this environment I’m able to reproduce this error but not easily. It requires that the regular user hasn’t approved “debugging” access to the developer tools via admin credentials. This doesn’t appear to pop up the first time I run the fastlane script, but it does happen the second. Subsequent builds work fine.

Do the Circle CI tests run using a regular user on the virtual machine or an administrator?

Hey everyone,

I haven’t updated the this thread in a while, like I wanted to but we are struggling with a lot of things at the moment but things are looking better.

One feature that we rolled out lately is that we know collect xcactivitylogs which give much more information about certain failures. Please take a look in your artifacts.

Best,
Constantin

Just wanted to chime in to say that upgrading our config to use Xcode 8.1 hasn’t fixed this issue for us.

Hey everyone,

I’ve been doing some digging and facepalmed a little when I noticed that the simulators take even longer to boot by now


Could you please try this:

xcrun instruments -w 'sim uuid' || true && sleep 60

I think this might be related to some Exit code 65 issues but not all, and it probably wont fix the UI tests issue. It’s just something to try
 but not yet last resort.

Best,
Constantin

Hey Constantin,

of course these 60 (!) seconds will not be taken into account for our build-minutes, right?
We already wasted quite a lot of minutes due to the problems with the Cocoapods Specs-repo cloning-issue, our builds took up to 45, 50 minutes then.
Since there was no solution in sight from Circles side and the Specs-repo was unfortunately not sharded back then, we created our own Specs repo with only the Specs we needed.
Then, upgrade to Xcode 8.1, the multiple devices issue.
Simulators not starting, crazy 65er errors, tbh, we are currently wasting a lot of time fixing issues with Circle.
This is not your fault, but I have the feeling that Circle should urgently hire more developers, because as far as I see it, you are too few people having a lot of pressure, which inevitably leads to regressions and long times for issues to be resolved.

6 Likes

Is there a chance we get some news one day?

1 Like

Any updates on the ‘exit code 65’ issue?

Up.

My builds are still failing oftenly because of error 65.

Hey Ben,

Could you please open a support ticket? I’d like to take a direct look at your issue.

I would like to but I can’t find how to open support ticket :confused:

Edit: i sent an email to support@circleci.com

we’re going to investigate alternative CI solutions since we’ve lost days of dev time to circle issues. i realise no solution will be without problems, but we don’t seem to be getting anywhere

2 Likes

To complete up my presence in this thread, i declare our build to be working again after several months - essentially since Xcode8 adoption. It was achieved through multiple changes. I don’t claim any of it being the universal fix, but someone still can have an a-ha moment.

  1. cleaned up, rethought and reapplied code signing. Xcode7 setup may not be good enough for Xcode8. Simulator builds do not “just work” anymore. Apple provisioning is damn complicated and clicking the shiny new “Automatically manage signing” will NOT fix your mess. Thanks @jeremyhu for this one.
  2. migrate from Pods to Carthage. Pod’s justification of basically taking over your project is “will do everything for you”. Since Xcode toolchain is being fragmented (and unfortunately also bugged so far) by Swift and its different runtime libs/frameworks philosophy, that justification does not apply anymore. In a complicated project with subprojects, various targets and mixed Swift/ObjC codebase, Pods required so many side hacks already, that Carthage actually made the build simpler.
  3. solved exit code 65 with chruby system. Regarding adoption of upgraded system tools, Xcode toolchain is, uh, stiff somewhat. Anyone remember “resource-rules deprecated” in Xcode6? So thank you Circle for giving us the newest ruby as default, but our Xcode8 ipatool simply won’t work with any ruby>2.0.0. All you get to know is xcodebuild error “No devices found”. Your build containers feel alone, it wants you to ssh in and read its xcactivitylogs.
1 Like

Anybody having problems with iOS UI tests running, but failing on CircleCI? I have some simple XCTest cases that works fine when running locally on my machine, but when running on CircleCI they fail with errors like
"UI Testing Failure - No matches found for ScrollView" or other missing UI objects, and with exit code 65.

We have been having the same issue for a while as well: Exit Status: 65. We normally just do a manual rebuild and then it works after 1 or 2 times. But this really is a time suck.

Any concrete ways to fix this issue? Please?

Hello we are still getting this error very oftenly. We need a fix for that issue. We currently can’t trust circle builds with that error always showing up


1 Like

@ben-j69 @tyler-at-uv @halseth If you’re still experiencing this issue please reach out to support@circleci.com and open a ticket to help get this resolved.

I just got this error this again, so I sent an email to support. Hope it gets resolved. Thank you for the assistance.

If you’re running into the error code 65 (and tests are passing) here’s a command you can pipe on the end of your xcodebuild (assuming you’re running tests) command to get back more consistent results:

(XCODEBUILD_COMMAND_HERE) | awk 'BEGIN {success=0} $0 ~ /.* tests, with 0 failures \(.*/ {success=1} {print $0} END {if(success==0){exit 1}}

This looks for tests, with 0 failures ( in your output text, thus using the text output of xcodebuild instead of the status code of xcodebuild to determine success.

Note: Keep in mind, if you do something like NSLog(' tests, with 0 failures ('); in your code you make get a false positive, it’s very unlikely to happen by accident. You may have to update tests, with 0 failures ( in the awk script between updates of xcodebuild. But, having consistent results with xcodebuild is definitely worth that price.

After long looking for this I’ve just resolved an issue for one of our customers that had the exact same symptoms:

  1. Sometimes exit code 65 during the build

  2. UI test just wont run and fail with a missing element.

You’re most likely running into issues with an UIAlert being on screen which expect user input, while the XCTest daemon quietly timesout in the background and eventually throws an error.

If you run into these issues please open a support ticket I would like to take a closer look at your builds and resolve this with you :slight_smile:

Constantin