https://circleci.com/gh/drazisil/clojure/2
I’m trying to build clojure to get a feel for the language/test, and I’ve exceeded memory on 2 containers so far.
Is there a way around this, or is this project just too big for circle at this time?
I took a look at the memory-usage.txt file that is generated every time a build fails with the out-of-memory error, and here is the process that used the most memory in that build:
PID RSS %CPU COMMAND
9991 4063184 139 /usr/lib/jvm/jdk1.7.0/jre/bin/java -Dclojure.test-clojure.exclude-namespaces=...
This makes the container exceed its memory limit, and it’s therefore getting killed. By default, JVM would try to allocate all the memory it has available, and in this case the available memory it sees is not 4G of the container but the total memory the host machine has, which is far more than 4G.
To work around this, it would be necessary to limit JVM’s memory usage by adding the following to your circle.yml:
machine:
environment:
_JAVA_OPTIONS: "-Xms512m -Xmx1024m"
You can find some more details about this in our OOM doc.
Ah right. I didn’t quite understand that before, thanks for taking a look 
Here’s the working circle.yml for clojue, if anything wants it. It’s great that it builds nearly perfectly out of the box!
machine:
environment:
_JAVA_OPTIONS: "-Xms512m -Xmx1024m"
test:
override:
- mvn package
One thing though, I’ve noticed that circle tries to run the integration tests by default, which fails. How does that differ from the command I have here (which was taken from another build system’s config file?
1 Like
The inferred Clojure test command would use Leiningen to run all the tests—if you prefer to use Maven to do that, overriding the default command is definitely the way to do it.
I’m having a different kind of issue when running ‘lein uberjar’. I am exceeding the 4GB limit. I’ve tried adding the JVM_OPTS above but still no luck.
:jvm-opts ^:replace ["-Xms128m" “-Xmx350m” “-Xss512k” “-XX:MaxMetaspaceSize=150m”]
Looks like you need something like this in profiles.clj
(My ipad wont let my change the values, but that should help)
This unfortunately had no effect. I have additional services running at the same time. The whole thing falls over once I try the uberjar command.
Hm. Not sure then, per https://github.com/technomancy/leiningen/blob/stable/doc/PROFILES.md that really seems like it should work.
So does setting _JAVA_OPTIONS not bring any change to the uberjar process? Does it still run overboard with the memory usage? Can you share the first few lines of the memory-usage.txt file generated in a build where that happened?
It makes no different
PID RSS %CPU COMMAND
9399 3820360 9.4 java -Djava.library.path=./DynamoDBLocal_lib -jar DynamoDBLocal.jar --sharedDb
11309 225016 10.2 /usr/lib/jvm/java-7-openjdk-amd64//bin/java -Xms256m -Xmx1g -Xss256k -Djava.awt.headless=true -XX:+UseParNewGC -XX:+UseConcMarkSweepGC -XX:CMSInitiatingOccupancyFraction=75 -XX:+UseCMSInitiatingOccupancyOnly -XX:+HeapDumpOnOutOfMemoryError -Delasticsearch -Des.pidfile=/var/run/elasticsearch.pid -Des.path.home=/usr/share/elasticsearch -cp :/usr/share/elasticsearch/lib/elasticsearch-0.90.2.jar:/usr/share/elasticsearch/lib/*:/usr/share/elasticsearch/lib/sigar/* -Des.default.config=/etc/elasticsearch/elasticsearch.yml -Des.default.path.home=/usr/share/elasticsearch -Des.default.path.logs=/var/log/elasticsearch -Des.default.path.data=/var/lib/elasticsearch -Des.default.path.work=/tmp/elasticsearch -Des.default.path.conf=/etc/elasticsearch org.elasticsearch.bootstrap.ElasticSearch
1990 64560 0.7 /usr/lib/erlang/erts-5.8.5/bin/beam.smp -W w -A 64 -P 1048576 -K true -B i -- -root /usr/lib/erlang -progname erl -- -home /var/lib/rabbitmq -- -pa /usr/lib/rabbitmq/lib/rabbitmq_server-3.5.6/sbin/../ebin -noshell -noinput -s rabbit boot -sname rabbit@box152 -boot start_sasl -config /etc/rabbitmq/rabbitmq -kernel inet_default_connect_options [{nodelay,true}] -sasl errlog_type error -sasl sasl_error_logger false -rabbit error_logger {file,"/var/log/rabbitmq/rabbit@box152.log"} -rabbit sasl_error_logger {file,"/var/log/rabbitmq/rabbit@box152-sasl.log"} -rabbit enabled_plugins_file "/etc/rabbitmq/enabled_plugins" -rabbit plugins_dir "/usr/lib/rabbitmq/lib/rabbitmq_server-3.5.6/sbin/../plugins" -rabbit plugins_expand_dir "/var/lib/rabbitmq/mnesia/rabbit@box152-plugins-expand" -os_mon start_cpu_sup false -os_mon start_disksup false -os_mon start_memsup false -mnesia dir "/var/lib/rabbitmq/mnesia/rabbit@box152" -kernel inet_dist_listen_min 25672 -kernel inet_dist_listen_max 25672
663 56224 0.3 /usr/bin/mongod --config /etc/mongodb.conf
1393 53244 0.4 /usr/lib/erlang/erts-5.8.5/bin/beam.smp -Bd -K true -A 4 -- -root /usr/lib/erlang -progname erl -- -home /home/couchdb -- -noshell -noinput -os_mon start_memsup false start_cpu_sup false disk_space_check_interval 1 disk_almost_full_threshold 1 -sasl errlog_type error -couch_ini /usr/local/etc/couchdb/default.ini /usr/local/etc/couchdb/local.ini /usr/local/etc/couchdb/default.ini /usr/local/etc/couchdb/local.ini -s couch -pidfile /usr/local/var/run/couchdb/couchdb.pid -heart
659 51268 0.3 /usr/bin/java -cp /etc/zookeeper/conf:/usr/share/java/jline.jar:/usr/share/java/log4j-1.2.jar:/usr/share/java/xercesImpl.jar:/usr/share/java/xmlParserAPIs.jar:/usr/share/java/zookeeper.jar -Dcom.sun.management.jmxremote -Dcom.sun.management.jmxremote.local.only=false -Dzookeeper.log.dir=/var/log/zookeeper -Dzookeeper.root.logger=INFO,ROLLINGFILE org.apache.zookeeper.server.quorum.QuorumPeerMain /etc/zookeeper/conf/zoo.cfg
763 44952 0.0 /usr/sbin/mysqld
2291 21616 0.0 Xvfb :99 -screen 0 1280x1024x24
835 16740 0.0 /usr/lib/postgresql/9.4/bin/postgres -D /var/lib/postgresql/9.4/main -c config_file=/etc/postgresql/9.4/main/postgresql.conf
2340 11352 0.0 xfwm4 --daemon
13288 8796 2.7 convox deploy
2241 7556 0.0 /usr/local/bin/redis-server *:6379
This is very strange—we were assuming that _JAVA_OPTIONS will be picked up by all JVM processes. The only explanation of this to me would be that DynamoDB process doesn’t use Oracle JVM but instead uses OpenJDK (which might or might not have support for _JAVA_OPTIONS built in), but I don’t think that’s the case.
Anyway, it would be necessary to find a way to pass the -Xms256m -Xmx512m flags to the DynamoDB process. How are you launching it right now?
- java -Djava.library.path=./DynamoDBLocal_lib -jar DynamoDBLocal.jar --sharedDb: background: true
Can you please add -Xms256m -Xmx512m flags to that command in your circle.yml, before -Djava.library.path?
That did the trick. Thanks.
It looks like openJDK might use JAVA_OPTS instead.
JAVA_OPTS="-Xms256m -Xmx512m"
Ref: https://community.webfaction.com/questions/7515/heap-memory-java-openjdk