Parallelism messing up with time-based tests

Build Environment Running Tests
, ,
1

Hey everyone,

We have a big old build that is taking too long to run tests and Im trying to setup parallelism to speed things up. After the configuration, I can see the tests are indeed running in parallel and are much faster but Im running into 2 very strange issues:

  • Timestamps for record are set as the same: I have this test that creates two different rows in my database with the created_at column. Later the test picks the latest created one using a order(created_at: :desc).first clause. Running locally it works as expected. Running on CircleCI with parallelism though, both created_at are set to the same value (I printed it) and the test just picks the wrong one. I know I can just also order by id desc, but I would like to understand why is that and how can I avoid it.
  • Tests with sleep simply doesn’t work: Again, time related, we have this class that runs a block of code and logs how long it took to run. It is as simple as the following:
 class BenchmarkLog
   class << self
     def benchmark_log(title, description, &block)
       begin
         start_time = Time.now
         yield
       ensure
         ellapsed_time = Time.now - start_time
         Rails.logger.info("#{title} - #{description} - #{ellapsed_time}")
       end
     end
   end
 end

On the test, we simply use a sleep to wait for 1s and check if that worked, like:

    BenchmarkLog.benchmark_log("MyClass", "desired operation") do
      sleep(1)
    end

    expect(log_message).to match(/MyClass - desired operation - 1/)

Oddly enough, when running locally it passes but when running on circle CI with parallelism only it fails with the message:

  expected "MyClass - desired operation - 0.0" to match /MyClass - desired operation - 1/
  Diff:
  @@ -1,2 +1,2 @@
  -/MyClass - desired operation - 1/
  +"MyClass - desired operation - 0.0"

Indicating that the sleep command was simply ignored.

So, what am I missing on the parallelism that seems to be messing up with these timings?

Thanks!!

2

To make it worse (?), when I pick Run failed tests for those it then works :man_shrugging: …