Does anyone here have an example app using --split-by=timings --timings-type=name successfully with Rspec/Rails in a parallelized (20 - 30 containers) set up? I can’t seem to get this to work.
@florence023 this is a useless garbage AI response, I have very little belief that you’re actually a human.
I already checked that timing data is available and it is.
“Command Adjustment” are you joking? You just removed a carriage return, otherwise your command is exactly the same.
Ensure parallelism is working correctly? That has nothing to do with this problem. Parallelism of 2 would be fine in an actually working example. The parallelism key is incredibly simple to set up.
Please don’t waste people’s time posting AI-generated responses to this forum. What do you even have to gain from this?
@circleci-community you should probably know your forum has been infiltrated by AI response bots . Sadly nobody from CircleCI has ever been able to provide me a working example of this even after a month and a half+ of back and forth, so I just am using Knapsack Pro instead. Sad that the docs even say this is possible when it clearly isn’t .
Sorry for the tone for any humans that read this, this AI response has just brought up my frustration with this problem all over again.
@zmf-framework sorry about that, i deleted the other post that appears to be spam. Can you send a link to a job where youre seeing the issue to sebastian@circleci.com. The command that you shared looks correct, I want to see what might be going on.
@sebastian-lerner I already have, I have an open support request where the person helping me basically admitted that you all have no working examples of this and nobody on your engineering team apparently could figure this out (even given 6+ weeks to). Sorry if that’s harsh, just trying to share the truth of what’s happened so far.
My work email is zmf (at) frame.work if you’d like to look up the cases.
Again, as I asked on this thread, all I’d like to see is one working example of a Rails/Rspec app splitting by test case.
Got it, I’ll try to track that ticket down. Super strange because I’ve seen plenty of folks do just fine with the syntax from here Rerun failed tests - CircleCI
I guess maybe I’ve only ever looked for splitting by timing, but I would assume that if it works for timing, it would work for filename.
@sebastian-lerner these docs are for re-rerunning failed tests. I’m asking about running Rails Rspec test suites across multiple runners, but by test timing. Not class timing or file timing.
This way if files have a large # of tests, it doesn’t matter to the timing of the test suite :).
Sorry, I should have been more precise. The circleci tests run command works for both test splitting & rerunning failed tests, that’s why I was referencing it.
Hey @zmf-framework . It’s not CircleCI’s fault. I have been trying to accomplish the same thing and the issue is in the rspec_junit_formatter rubygem. It does not include line numbers in the testcase result set, and you cannot run rspec tests by the name of the test, only by file or file/linenumber combo.
When CircleCI goes to split tests, it can only do so on one of the three attributes: classname, name, and file. And when it splits it by one of those, the tool it passes those split values into must also support it. In the case of rspec, passing values like those in classname and name isn’t supported. So it must fallback to using file.
I have filed a ticket with the rspec_junit_formatter team to see if they would be inclined to add support for an option that adds the line numbers into result. The ticket also then demonstrates how we could then utilise this to split per spec, rather than per file.
Until such a thing is supported, looks like we’ll need to use custom hacks/workarounds.
@KieranP you’re right (partially)! CircleCI and I figured this out eventually in our support request thread together, that it’s because of missing library support. What I do think is CircleCI’s fault is 1) saying in their docs this is possible when it is not 2) responding really slowly when I asked about this, taking months to get a solid answer and 3) not understanding the issue after weeks and months of back and forth with them. I think this is mostly because of CircleCI’s transition to only paid support plans. Pretty sad for a company that used to care a lot more about engineers’ individual problems running their specs and jump to fix them as they came up. Now they seem pretty focused on just acquiring other companies and growing revenue at all costs.
To give a more concrete example: after around 6 weeks of back and forth, they tried to have an engineer on their team reproduce the problem. The engineer sent over a sample repo where the tests were not being split by test name. So even their own engineers (who currently work for them) don’t have a great grip on this problem.
Sure, that’s fine, not everyone is going to understand what’s going on here. It took me a while to myself and I still don’t get every part of it with 100% clarity.
My point is: sure, the underlying issue isn’t CircleCI’s fault, but as the provider of CI software and at this point of runners, they should be responsible for understanding their own software and its integration limitations in a reasonable timeframe. Even if they don’t get it, I’d hope within 2 - 3 weeks they could. 2 - 3 months was way too long and had me just using Knapsack Pro to do this instead (for even more money, even though it works quite well) as a workaround. I will give CircleCI credit for at least recommending this service later on in the docs, although I still don’t get why they would if they thought they could do this natively.
@zmf-framework Fair points. Too many support staff at companies these days just copy and paste pre-prepared responses, or worse, are getting AI to give back garbage responses. I’ve lost track how many times I’ve asked an AI bot “I want to talk to a human”
On another sad note, the maintainers of the rspec_junit_formatter gem responded to my ticket and basically said they won’t change anything because adding line numbers to junit isn’t compatible with the junit specification. So without that, and not wanting to spend $100s more on Knapsack (side note: why do so many services charge per developer these days, it’s so frustrating), we’re forced to try roll out our own solution using the rspec json formatter. Not ideal, but some of our CircleCI workers spend half their time idle because CircleCI isn’t balancing the files intelligently enough.
You wouldn’t happen to have open sourced any solution you came up with by chance?
. Sadly nobody from CircleCI has ever been able to provide me a working example of this even after a month and a half+ of back and forth, so I just am using Knapsack Pro instead. Sad that the docs even say this is possible when it clearly isn’t 
.
Not ideal, but some of our CircleCI workers spend half their time idle because CircleCI isn’t balancing the files intelligently enough.