Containers running out of memory on 14.04 image but runs fine on 12.04

Build Environment
, ,
1

Using Ubuntu some containers run out of memory with capybara-webkit and rspec showing taking all of the memory. The error message is

Your build has exceeded the memory limit of 4G on 6 containers. The results of this build are likely invalid. We have taken a snapshot of the memory usage at the time, which you can find in a build artifact named memory-usage.txt. The RSS column in this file shows the amount of memory used by each process, measured in kilobytes.

In memory-usage.txt:

PID RSS %CPU COMMAND
23546 1809132 24.9 /path/to/ruby/2.3.0/gems/capybara-webkit-1.11.1/bin/webkit_server
22654 1079924 52.4 /path/to/ruby/2.3.0/bin/rspec

Tried running with Capybara.reset_sessions! and Capybara.reset!, no changes.
The tests run fine using Ubuntu 12.04.
Anyone else running into this issue?

2

Can you double check the versions of everything (that is relevant to this) that you are using in both 12.04 and 14.04?

3

Compared the outputs of gem env and bundle show on both builds, they are identical. In circle.yml, the only difference is that on 12.04 we’re using this solution for qt5:

- sudo apt-get update -y; true
- sudo apt-get install apt -y
- sudo add-apt-repository ppa:beineri/opt-qt542 -y
- sudo apt-get update -y; true
- sudo apt-get install -y qt54webkit libwebkit-dev libgstreamer0.10-dev
- echo "/opt/qt54/bin/qt54-env.sh" >> ~/.circlerc

Whereas not doing that on 14.04 since qt5 is pre-installed there. Also different in 14.04 (all were added in hopes to fix this issue):

  • sudo apt-get install libqt5webkit5
  • sudo apt-get install libxmlsec1 xmlsec1 libxmlsec1-openssl libxmlsec1-dev
  • sudo apt-get install gstreamer1.0-plugins-base gstreamer1.0-tools gstreamer1.0-x

Not sure what else to try.

4

As a workaround you can try reducing the number of concurrent compiler processes (if you’re compiling). When compiling with GCC, I frequently run out of memory with make -j32. I have to reduce it to make -j4 to get consistently successful builds. GCC 4.8 can use up to (and more than!) 1 GB per process.

5

Thanks for your response! We don’t compile though. or, at least we don’t use makefile.

As a next step I’ll use ruby-prof to see if the objects we’re creating are being cleared out properly, but if that’s the culprit, it’s very strange that it didn’t come up in 12.04 at all.

6

So, the main difference between my 12.04 and 14.04 runs is the version of Qt5. I mentioned that we were installing Qt5 in 12.04 ourselves in each build, and were not doing that anymore in Trusty cause it’s already installed there by default. However, the default version that comes with Trusty is 5.2.1 (https://circleci.com/docs/build-image-trusty/), which is different from the one we’ve been successfully using with Precise (5.4.2). Weird that nobody else has ran into this, right? But apparently it could happen in certain (rare) cases (one ref here: https://gist.github.com/juniorz/4022134).
As a POC, I installed Qt5.5 on my Trusty build and it solved my memory issues. :tada:

Thanks to Eric, Tom, and Lev at Circle support team for helping troubleshooting this!

7

Hi @maria-v ,
I’m indeed running into the same problem. Do you have the instructions you used to install qt 5.5? Every time I look for how to install qt I find different ways and I remember spending several hours with no luck last year on this so if you could publish what you used it would be tremendously helpful.
Thanks!

8

For those still struggling with this, I was able to finally get it to use qt 5.5 with the following in my circle.yml:

dependencies:
  pre:
    - sudo add-apt-repository -y ppa:beineri/opt-qt551-trusty
    - echo "/opt/qt55/bin/qt55-env.sh" >> ~/.circlerc
    - echo "export PATH=/opt/qt55/bin:$PATH" >> ~/.circlerc
    - qmake --version # this is just here to verify that you're using the correct version
9

Qt 5.5 is preinstalled now on 14.04. But 5.2 is still the default version

5.5 is installed under /opt/qt55 in order to maintain backwards compatibility for people using 5.2. You can enable 5.5 with the following circle.yml.

machine:
  pre:
    - echo 'source /opt/qt55/bin/qt55-env.sh' >> /home/ubuntu/.circlerc

If you are using capybara-webkit and need to re-compile with QT 5.5, you need declare QMAKE env var.

machine:
  environment:
    QMAKE: /opt/qt55/bin/qmake

Please also make sure to expunge cache so that the gem is built from the scratch.

1 Like