Circle 2.0 doesn't seem to interoperate with Elasticsearch plugins (gradle+java)


#1

We recently migrated our Elasticsearch plugin to circle 2.0 to be able to use Java 9. We have a pretty stock Java configuration.

However, the build consistently fails with error 137:

==> Test Info: seed=D5E8CC90EB0317F8; jvms=4; suites=23
==> Test Summary: 18 suites (1 ignored), 106 tests, 2 ignored (2 assumptions)
[ant:junit4] ERROR: JVM J0 ended with an exception: Forked process returned with error code: 137.
	at com.carrotsearch.ant.tasks.junit4.JUnit4.executeSlave(JUnit4.java:1519)
	at com.carrotsearch.ant.tasks.junit4.JUnit4.access$000(JUnit4.java:126)
	at com.carrotsearch.ant.tasks.junit4.JUnit4$2.call(JUnit4.java:982)
	at com.carrotsearch.ant.tasks.junit4.JUnit4$2.call(JUnit4.java:979)
	at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
	at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1167)
	at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:641)
	at java.base/java.lang.Thread.run(Thread.java:844)
JUnit4 test failed, ant output was:
   [junit4] <JUnit4> says ᐊᐃ! Master seed: D5E8CC90EB0317F8
   [junit4] JVM J1:     0.95 ..    15.66 =    14.71s
   [junit4] JVM J2:     0.95 ..    13.55 =    12.60s
   [junit4] JVM J3:     0.96 ..    13.68 =    12.72s
   [junit4] Execution time total: 15 seconds
   [junit4] ERROR: JVM J0 ended with an exception, command line: /docker-java-home/bin/java -ea -esa -Xmx512m -Xms512m -XX:+HeapDumpOnOutOfMemoryError -XX:HeapDumpPath=/home/circleci/project/build/heapdump -Dtests.prefix=tests -Dtests.seed=D5E8CC90EB0317F8 -Dtests.security.manager=true -Djava.io.tmpdir=./temp -Dtests.gradle=true -Dtests.artifact=ltr -Djava.awt.headless=true -Dtests.ifNoTests=fail -Dtests.task=:test -Djna.nosys=true -Dtests.logger.level=WARN -Djunit4.childvm.cwd=/home/circleci/project/build/testrun/test/J0 -Djunit4.childvm.id=0 -Djunit4.childvm.count=4 -classpath /home/circleci/project/build/classes/java/test:/home/circleci/project/build/resources/test:/home/circleci/project/build/generated-resources:/home/circleci/project/build/classes/java/main:/home/circleci/project/build/resources/main:/home/circleci/.gradle/caches/modules-2/files-2.1/org.apache.lucene/lucene-expressions/7.1.0/714927eb1d1db641bff9aa658e7e112c368f3e6d/lucene-expressions-7.1.0.jar:/home/circleci/.gradle/caches/modules-2/files-2.1/org.antlr/antlr4-runtime/4.5.1-1/66144204f9d6d7d3f3f775622c2dd7e9bd511d97/antlr4-runtime-4.5.1-1.jar:/home/circleci/.gradle/caches/modules-2/files-2.1/org.ow2.asm/asm/5.0.4/da08b8cce7bbf903602a25a3a163ae252435795/asm-5.0.4.jar:/home/circleci/.gradle/caches/modules-2/files-2.1/org.ow2.asm/asm-commons/5.0.4/5a556786086c23cd689a0328f8519db93821c04c/asm-commons-5.0.4.jar:/home/circleci/.gradle/caches/modules-2/files-2.1/org.ow2.asm/asm-tree/5.0.4/396ce0c07ba2b481f25a70195c7c94922f0d1b0b/asm-tree-5.0.4.jar:/home/circleci/.gradle/caches/modules-2/files-2.1/org.elasticsearch.test/framework/6.2.2/439d9e6aff9b485de708cff8f39ffbda1b55d045/framework-6.2.2.jar:/home/circleci/.gradle/caches/modules-2/files-2.1/org.elasticsearch/elasticsearch/6.2.2/16b3f7a7a31b89144723eb85c0583234842f684/elasticsearch-6.2.2.jar:/home/circleci/.gradle/caches/modules-2/files-2.1/com.o19s/RankyMcRankFace/0.1.1/d87cc202693348480baabb19e9faa52d83c4a7e5/RankyMcRankFace-0.1.1.jar:/home/circleci/.gradle/caches/modules-2/files-2.1/com.github.spullara.mustache.java/compiler/0.9.3/2815e016c63bec4f18704ea4f5489106a5b01a99/compiler-0.9.3.jar:/home/circleci/.gradle/caches/modules-2/files-2.1/org.locationtech.spatial4j/spatial4j/0.6/21b15310bddcfd8c72611c180f20cf23279809a3/spatial4j-0.6.jar:/home/circleci/.gradle/caches/modules-2/files-2.1/com.vividsolutions/jts/1.13/3ccfb9b60f04d71add996a666ceb8902904fd805/jts-1.13.jar:/home/circleci/.gradle/caches/modules-2/files-2.1/org.elasticsearch/elasticsearch-core/6.2.2/9c4df6cf94410bec51e1abe1343be9051358eda/elasticsearch-core-6.2.2.jar:/home/circleci/.gradle/caches/modules-2/files-2.1/org.apache.logging.log4j/log4j-api/2.9.1/7a2999229464e7a324aa503c0a52ec0f05efe7bd/log4j-api-2.9.1.jar:/home/circleci/.gradle/caches/modules-2/files-2.1/org.apache.logging.log4j/log4j-core/2.9.1/c041978c686866ee8534f538c6220238db3bb6be/log4j-core-2.9.1.jar:/home/circleci/.gradle/caches/modules-2/files-2.1/org.elasticsearch/jna/4.5.1/da10908ae23dc59b19dc258e63aea1c44621dc3a/jna-4.5.1.jar:/home/circleci/.gradle/caches/modules-2/files-2.1/org.apache.lucene/lucene-core/7.2.1/91897dbbbbada95ccddbd90505f0a0ba6bf7c199/lucene-core-7.2.1.jar:/home/circleci/.gradle/caches/modules-2/files-2.1/org.apache.lucene/lucene-analyzers-common/7.2.1/324c3a090a04136720f4ef612db03b5c14866efa/lucene-analyzers-common-7.2.1.jar:/home/circleci/.gradle/caches/modules-2/files-2.1/org.apache.lucene/lucene-backward-codecs/7.2.1/bc8dc9cc1555543532953d1dff33b67f849e19f9/lucene-backward-codecs-7.2.1.jar:/home/circleci/.gradle/caches/modules-2/files-2.1/org.apache.lucene/lucene-grouping/7.2.1/5dbae570b1a4e54cd978fe5c3ed2d6b2f87be968/lucene-grouping-7.2.1.jar:/home/circleci/.gradle/caches/modules-2/files-2.1/org.apache.lucene/lucene-highlighter/7.2.1/2f4b8c93563409cfebb36d910c4dab4910678689/lucene-highlighter-7.2.1.jar:/home/circleci/.gradle/caches/modules-2/files-2.1/org.apache.lucene/lucene-join/7.2.1/3121a038d472f51087500dd6da9146a9b0031ae4/lucene-join-7.2.1.jar:/home/circleci/.gradle/caches/modules-2/files-2.1/org.apache.lucene/lucene-memory/7.2.1/21233b2baeed2aaa5acf8359bf8c4a90cc6bf553/lucene-memory-7.2.1.jar:/home/circleci/.gradle/caches/modules-2/files-2.1/org.apache.lucene/lucene-misc/7.2.1/478fed6c474c95f6c0c678c04297a3df0c1687e/lucene-misc-7.2.1.jar:/home/circleci/.gradle/caches/modules-2/files-2.1/org.apache.lucene/lucene-queries/7.2.1/2135cf5047409ed1ca6cd098e802b30f9dbd1ff/lucene-queries-7.2.1.jar:/home/circleci/.gradle/caches/modules-2/files-2.1/org.apache.lucene/lucene-queryparser/7.2.1/a87d8b14d1c8045f61cb704955706f6681170be3/lucene-queryparser-7.2.1.jar:/home/circleci/.gradle/caches/modules-2/files-2.1/org.apache.lucene/lucene-sandbox/7.2.1/dc8dd132fd183791dc27591a69974f55b685d0d7/lucene-sandbox-7.2.1.jar:/home/circleci/.gradle/caches/modules-2/files-2.1/org.apache.lucene/lucene-spatial/7.2.1/9c4d96e6ea34292f7cd20c4ff1d16ff31eb7869/lucene-spatial-7.2.1.jar:/home/circleci/.gradle/caches/modules-2/files-2.1/org.apache.lucene/lucene-spatial-extras/7.2.1/8aff7e8a5547c03d0c4e7e1b58cb30773bb1d7d5/lucene-spatial-extras-7.2.1.jar:/home/circleci/.gradle/caches/modules-2/files-2.1/org.apache.lucene/lucene-spatial3d/7.2.1/8b0db8ff795b31994ebe93779c450d17c612590d/lucene-spatial3d-7.2.1.jar:/home/circleci/.gradle/caches/modules-2/files-2.1/org.apache.lucene/lucene-suggest/7.2.1/1c3804602e35589c21b0391fa7088ef012751a22/lucene-suggest-7.2.1.jar:/home/circleci/.gradle/caches/modules-2/files-2.1/org.elasticsearch/securesm/1.2/4c28f5b634497d64b727961430a516f351a099d5/securesm-1.2.jar:/home/circleci/.gradle/caches/modules-2/files-2.1/org.elasticsearch/elasticsearch-cli/6.2.2/b37d7ace8c360f6f244bd654a9271a44e77a8d82/elasticsearch-cli-6.2.2.jar:/home/circleci/.gradle/caches/modules-2/files-2.1/com.carrotsearch/hppc/0.7.1/8b5057f74ea378c0150a1860874a3ebdcb713767/hppc-0.7.1.jar:/home/circleci/.gradle/caches/modules-2/files-2.1/joda-time/joda-time/2.9.9/f7b520c458572890807d143670c9b24f4de90897/joda-time-2.9.9.jar:/home/circleci/.gradle/caches/modules-2/files-2.1/org.yaml/snakeyaml/1.17/7a27ea250c5130b2922b86dea63cbb1cc10a660c/snakeyaml-1.17.jar:/home/circleci/.gradle/caches/modules-2/files-2.1/com.fasterxml.jackson.core/jackson-core/2.8.10/eb21a035c66ad307e66ec8fce37f5d50fd62d039/jackson-core-2.8.10.jar:/home/circleci/.gradle/caches/modules-2/files-2.1/com.fasterxml.jackson.dataformat/jackson-dataformat-smile/2.8.10/e853081fadaad3e98ed801937acc3d8f77580686/jackson-dataformat-smile-2.8.10.jar:/home/circleci/.gradle/caches/modules-2/files-2.1/com.fasterxml.jackson.dataformat/jackson-dataformat-yaml/2.8.10/1e08caf1d787c825307d8cc6362452086020d853/jackson-dataformat-yaml-2.8.10.jar:/home/circleci/.gradle/caches/modules-2/files-2.1/com.fasterxml.jackson.dataformat/jackson-dataformat-cbor/2.8.10/1c58cc9313ddf19f0900cd61ed044874278ce320/jackson-dataformat-cbor-2.8.10.jar:/home/circleci/.gradle/caches/modules-2/files-2.1/com.tdunning/t-digest/3.0/84ccf145ac2215e6bfa63baa3101c0af41017cfc/t-digest-3.0.jar:/home/circleci/.gradle/caches/modules-2/files-2.1/org.hdrhistogram/HdrHistogram/2.1.9/e4631ce165eb400edecfa32e03d3f1be53dee754/HdrHistogram-2.1.9.jar:/home/circleci/.gradle/caches/modules-2/files-2.1/org.elasticsearch.client/elasticsearch-rest-client/6.2.2/d2d12847f08179263d880f329ad8eee748e090e1/elasticsearch-rest-client-6.2.2.jar:/home/circleci/.gradle/caches/modules-2/files-2.1/com.carrotsearch.randomizedtesting/randomizedtesting-runner/2.5.2/91f3284993b44dcb2f003b5f28617abba13971d2/randomizedtesting-runner-2.5.2.jar:/home/circleci/.gradle/caches/modules-2/files-2.1/junit/junit/4.12/2973d150c0dc1fefe998f834810d68f278ea58ec/junit-4.12.jar:/home/circleci/.gradle/caches/modules-2/files-2.1/org.hamcrest/hamcrest-all/1.3/63a21ebc981131004ad02e0434e799fd7f3a8d5a/hamcrest-all-1.3.jar:/home/circleci/.gradle/caches/modules-2/files-2.1/org.apache.lucene/lucene-test-framework/7.2.1/888ce0284818f2cb517090d4e4c6e8c3d397513a/lucene-test-framework-7.2.1.jar:/home/circleci/.gradle/caches/modules-2/files-2.1/org.apache.lucene/lucene-codecs/7.2.1/3eabf801b58f486ca967dd807e98a97efec34a7b/lucene-codecs-7.2.1.jar:/home/circleci/.gradle/caches/modules-2/files-2.1/org.apache.httpcomponents/httpclient/4.5.2/733db77aa8d9b2d68015189df76ab06304406e50/httpclient-4.5.2.jar:/home/circleci/.gradle/caches/modules-2/files-2.1/org.apache.httpcomponents/httpcore/4.4.5/e7501a1b34325abb00d17dde96150604a0658b54/httpcore-4.4.5.jar:/home/circleci/.gradle/caches/modules-2/files-2.1/commons-logging/commons-logging/1.1.3/f6f66e966c70a83ffbdb6f17a0919eaf7c8aca7f/commons-logging-1.1.3.jar:/home/circleci/.gradle/caches/modules-2/files-2.1/commons-codec/commons-codec/1.10/4b95f4897fa13f2cd904aee711aeafc0c5295cd8/commons-codec-1.10.jar:/home/circleci/.gradle/caches/modules-2/files-2.1/org.elasticsearch/securemock/1.2/98201d4ad5ac93f6b415ae9172d52b5e7cda490e/securemock-1.2.jar:/home/circleci/.gradle/caches/modules-2/files-2.1/org.elasticsearch/mocksocket/1.2/190ad3f42fd6ab7367bae0c0e14d1dc4e4a3c361/mocksocket-1.2.jar:/home/circleci/.gradle/caches/modules-2/files-2.1/net.sf.jopt-simple/jopt-simple/5.0.2/98cafc6081d5632b61be2c9e60650b64ddbc637c/jopt-simple-5.0.2.jar:/home/circleci/.gradle/caches/modules-2/files-2.1/org.apache.httpcomponents/httpasyncclient/4.1.2/95aa3e6fb520191a0970a73cf09f62948ee614be/httpasyncclient-4.1.2.jar:/home/circleci/.gradle/caches/modules-2/files-2.1/org.apache.httpcomponents/httpcore-nio/4.4.5/f4be009e7505f6ceddf21e7960c759f413f15056/httpcore-nio-4.4.5.jar:/home/circleci/.gradle/caches/modules-2/files-2.1/com.carrotsearch.randomizedtesting/junit4-ant/2.5.2/b8f91682cfeb8f9196aad56ace9c9a13330acef6/junit4-ant-2.5.2.jar com.carrotsearch.ant.tasks.junit4.slave.SlaveMainSafe -eventsfile /home/circleci/project/build/testrun/test/temp/junit4-J0-20180320_143633_5915605875551591831625.events @/home/circleci/project/build/testrun/test/temp/junit4-J0-20180320_143633_59110358557141992339937.suites -stdin
   [junit4] ERROR: JVM J0 ended with an exception: Forked process returned with error code: 137.
   [junit4] 	at com.carrotsearch.ant.tasks.junit4.JUnit4.executeSlave(JUnit4.java:1519)
   [junit4] 	at com.carrotsearch.ant.tasks.junit4.JUnit4.access$000(JUnit4.java:126)
   [junit4] 	at com.carrotsearch.ant.tasks.junit4.JUnit4$2.call(JUnit4.java:982)
   [junit4] 	at com.carrotsearch.ant.tasks.junit4.JUnit4$2.call(JUnit4.java:979)
   [junit4] 	at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
   [junit4] 	at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1167)
   [junit4] 	at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:641)
   [junit4] 	at java.base/java.lang.Thread.run(Thread.java:844)

:test FAILED
:integTestCluster#stop

FAILURE: Build failed with an exception.

* What went wrong:
Execution failed for task ':test'.
> At least one slave process threw an exception, first: Forked process returned with error code: 137.

* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.

* Get more help at https://help.gradle.org

BUILD FAILED in 1m 3s
30 actionable tasks: 29 executed, 1 up-to-date
Exited with code 1

This build relies on Elasticsearch’s gradle-based plugin building system. This build system forks JVMs to run tests, and probably eats up a good deal of memory.

Thanks for any help


#2

Have you searched for that error on the web?


#3

Yes, and did not see anything relevant or worthwhile to pursue. But if your google-fu is better than mine, please share.

This feels like a simple out-of-the box case that “just worked” in Circle 1, but Circle 2 it’s requiring a bit more work.


#4

We added -Dtests.jvms=1 to restrict memory usage. You can see it in my Circle.yml here


#5

This topic was automatically closed 41 days after the last reply. New replies are no longer allowed.