Elasticsearch image failure

tkwon · May 2, 2019, 11:29pm

In my build I add an image of elasticsearch. It used to work but in the last day or so it fails:

Here is a sample from my config:

    version: 2
    jobs:
      build:
        docker:
          - image: circleci/python:3.6.4
          - image: bitjourney/elasticsearch-ci
            environment:
              xpack.security.enabled: false
              transport.host: localhost

And the error message:

[2019-05-02T23:26:11,921][INFO ][o.e.n.Node ] initializing …
[2019-05-02T23:26:11,971][INFO ][o.e.e.NodeEnvironment ] [4yqwKie] using [1] data paths, mounts [[/ (none)]], net usable_space [574.3gb], net total_space [825gb], spins? [possibly], types [aufs]
[2019-05-02T23:26:11,971][INFO ][o.e.e.NodeEnvironment ] [4yqwKie] heap size [1.9gb], compressed ordinary object pointers [true]
[2019-05-02T23:26:11,972][INFO ][o.e.n.Node ] node name [4yqwKie] derived from node ID [4yqwKie-Sx-ZWBTtEWHAWw]; set [node.name] to override
[2019-05-02T23:26:11,974][INFO ][o.e.n.Node ] version[5.1.2], pid[6], build[c8c4c16/2017-01-11T20:18:39.146Z], OS[Linux/4.15.0-1035-aws/amd64], JVM[Oracle Corporation/OpenJDK 64-Bit Server VM/1.8.0_92-internal/25.92-b14]
[2019-05-02T23:26:12,979][INFO ][o.e.p.PluginsService ] [4yqwKie] loaded module [aggs-matrix-stats]
[2019-05-02T23:26:12,979][INFO ][o.e.p.PluginsService ] [4yqwKie] loaded module [ingest-common]
[2019-05-02T23:26:12,979][INFO ][o.e.p.PluginsService ] [4yqwKie] loaded module [lang-expression]
[2019-05-02T23:26:12,979][INFO ][o.e.p.PluginsService ] [4yqwKie] loaded module [lang-groovy]
[2019-05-02T23:26:12,979][INFO ][o.e.p.PluginsService ] [4yqwKie] loaded module [lang-mustache]
[2019-05-02T23:26:12,979][INFO ][o.e.p.PluginsService ] [4yqwKie] loaded module [lang-painless]
[2019-05-02T23:26:12,979][INFO ][o.e.p.PluginsService ] [4yqwKie] loaded module [percolator]
[2019-05-02T23:26:12,979][INFO ][o.e.p.PluginsService ] [4yqwKie] loaded module [reindex]
[2019-05-02T23:26:12,979][INFO ][o.e.p.PluginsService ] [4yqwKie] loaded module [transport-netty3]
[2019-05-02T23:26:12,980][INFO ][o.e.p.PluginsService ] [4yqwKie] loaded module [transport-netty4]
[2019-05-02T23:26:12,980][INFO ][o.e.p.PluginsService ] [4yqwKie] loaded plugin [analysis-kuromoji]
[2019-05-02T23:26:12,980][INFO ][o.e.p.PluginsService ] [4yqwKie] loaded plugin [x-pack]
[2019-05-02T23:26:13,194][WARN ][o.e.d.s.g.GroovyScriptEngineService] [groovy] scripts are deprecated, use [painless] scripts instead
[2019-05-02T23:26:13,741][WARN ][o.e.b.ElasticsearchUncaughtExceptionHandler] uncaught exception in thread [main]
org.elasticsearch.bootstrap.StartupException: java.lang.IllegalStateException: No match found
at org.elasticsearch.bootstrap.Elasticsearch.init(Elasticsearch.java:125) ~[elasticsearch-5.1.2.jar:5.1.2]
at org.elasticsearch.bootstrap.Elasticsearch.execute(Elasticsearch.java:112) ~[elasticsearch-5.1.2.jar:5.1.2]
at org.elasticsearch.cli.SettingCommand.execute(SettingCommand.java:54) ~[elasticsearch-5.1.2.jar:5.1.2]
at org.elasticsearch.cli.Command.mainWithoutErrorHandling(Command.java:122) ~[elasticsearch-5.1.2.jar:5.1.2]
at org.elasticsearch.cli.Command.main(Command.java:88) ~[elasticsearch-5.1.2.jar:5.1.2]
at org.elasticsearch.bootstrap.Elasticsearch.main(Elasticsearch.java:89) ~[elasticsearch-5.1.2.jar:5.1.2]
at org.elasticsearch.bootstrap.Elasticsearch.main(Elasticsearch.java:82) ~[elasticsearch-5.1.2.jar:5.1.2]
Caused by: java.lang.IllegalStateException: No match found
at java.util.regex.Matcher.group(Matcher.java:536) ~[?:1.8.0_92-internal]
at org.elasticsearch.monitor.os.OsProbe.getControlGroups(OsProbe.java:213) ~[elasticsearch-5.1.2.jar:5.1.2]
at org.elasticsearch.monitor.os.OsProbe.getCgroup(OsProbe.java:402) ~[elasticsearch-5.1.2.jar:5.1.2]
at org.elasticsearch.monitor.os.OsProbe.osStats(OsProbe.java:454) ~[elasticsearch-5.1.2.jar:5.1.2]
at org.elasticsearch.monitor.os.OsService.(OsService.java:45) ~[elasticsearch-5.1.2.jar:5.1.2]
at org.elasticsearch.monitor.MonitorService.(MonitorService.java:45) ~[elasticsearch-5.1.2.jar:5.1.2]
at org.elasticsearch.node.Node.(Node.java:345) ~[elasticsearch-5.1.2.jar:5.1.2]
at org.elasticsearch.node.Node.(Node.java:229) ~[elasticsearch-5.1.2.jar:5.1.2]
at org.elasticsearch.bootstrap.Bootstrap$6.(Bootstrap.java:214) ~[elasticsearch-5.1.2.jar:5.1.2]
at org.elasticsearch.bootstrap.Bootstrap.setup(Bootstrap.java:214) ~[elasticsearch-5.1.2.jar:5.1.2]
at org.elasticsearch.bootstrap.Bootstrap.init(Bootstrap.java:306) ~[elasticsearch-5.1.2.jar:5.1.2]
at org.elasticsearch.bootstrap.Elasticsearch.init(Elasticsearch.java:121) ~[elasticsearch-5.1.2.jar:5.1.2]
… 6 more

Job was canceled

sohaibfarooqi · May 3, 2019, 5:39am

I am facing the same issue as well since yesterday. Did you manage to resolve it?

halfer · May 3, 2019, 7:34am

Has the image digest changed between it working and not working? You can find this in the console output of your first job step.

tkwon · May 3, 2019, 4:06pm

I’m not sure which is the image digest but guess it’s the hash shown here as “A2nLqGz”. It has changed. The failed one is “2GMRlsy”.

Here’s the tops of both console outputs.

Working:

[2019-04-26T15:59:23,880][INFO ][o.e.n.Node [2019-04-26T15:59:23,959][INFO ][o.e.e.NodeEnvironment [2019-04-26T15:59:23,959][INFO ][o.e.e.NodeEnvironment [2019-04-26T15:59:23,961][INFO ][o.e.n.Node [2019-04-26T15:59:23,963][INFO ][o.e.n.Node [2019-04-26T15:59:25,726][INFO ][o.e.p.PluginsService [2019-04-26T15:59:25,727][INFO ][o.e.p.PluginsService [2019-04-26T15:59:25,727][INFO ][o.e.p.PluginsService [2019-04-26T15:59:25,727][INFO ][o.e.p.PluginsService [2019-04-26T15:59:25,727][INFO ][o.e.p.PluginsService [2019-04-26T15:59:25,727][INFO ][o.e.p.PluginsService [2019-04-26T15:59:25,727][INFO ][o.e.p.PluginsService [2019-04-26T15:59:25,727][INFO ][o.e.p.PluginsService [2019-04-26T15:59:25,727][INFO ][o.e.p.PluginsService [2019-04-26T15:59:25,727][INFO ][o.e.p.PluginsService [2019-04-26T15:59:25,727][INFO ][o.e.p.PluginsService [2019-04-26T15:59:25,727][INFO ][o.e.p.PluginsService [2019-04-26T15:59:26,056][WARN ][o.e.d.s.g.GroovyScri [2019-04-26T15:59:28,727][INFO ][o.e.n.Node [2019-04-26T15:59:28,728][INFO ][o.e.n.Node ] initializing …
] [A2nLqGz] using [1] data paths, mounts [[/ (none)]], net usable_space [462gb], net total_space [825gb], spins? [possibly], types [aufs]
] [A2nLqGz] heap size [1.9gb], compressed ordinary object pointers [true]
] node name [A2nLqGz] derived from node ID [A2nLqGzwT8SBb0Er3mp95A]; set [node.name] to override
] version[5.1.2], pid[13], build[c8c4c16/2017-01-11T20:18:39.146Z], OS[Linux/4.4.0-144-generic/amd64], JVM[Oracle Corporation/OpenJDK 64-Bit Server VM/1.8.0_92-internal/25.92-b14]
] [A2nLqGz] loaded module [aggs-matrix-stats]
] [A2nLqGz] loaded module [ingest-common]
] [A2nLqGz] loaded module [lang-expression]
] [A2nLqGz] loaded module [lang-groovy]
] [A2nLqGz] loaded module [lang-mustache]
] [A2nLqGz] loaded module [lang-painless]
] [A2nLqGz] loaded module [percolator]
] [A2nLqGz] loaded module [reindex]
] [A2nLqGz] loaded module [transport-netty3]
] [A2nLqGz] loaded module [transport-netty4]
] [A2nLqGz] loaded plugin [analysis-kuromoji]
] [A2nLqGz] loaded plugin [x-pack]
ptEngineService] [groovy] scripts are deprecated, use [painless] scripts instead
] initialized
] [A2nLqGz] starting …

Failed:

[2019-05-02T23:16:03,538][INFO ][o.e.n.Node ] initializing …
[2019-05-02T23:16:03,583][INFO ][o.e.e.NodeEnvironment ] [2GMRlsy] using [1] data paths, mounts [[/ (none)]], net usable_space [259gb], net total_space [825gb], spins? [possibly], types [aufs]
[2019-05-02T23:16:03,584][INFO ][o.e.e.NodeEnvironment ] [2GMRlsy] heap size [1.9gb], compressed ordinary object pointers [true]
[2019-05-02T23:16:03,585][INFO ][o.e.n.Node ] node name [2GMRlsy] derived from node ID [2GMRlsy7Q0u0Am17qqh5Rg]; set [node.name] to override
[2019-05-02T23:16:03,586][INFO ][o.e.n.Node ] version[5.1.2], pid[32], build[c8c4c16/2017-01-11T20:18:39.146Z], OS[Linux/4.15.0-1035-aws/amd64], JVM[Oracle Corporation/OpenJDK 64-Bit Server VM/1.8.0_92-internal/25.92-b14]
[2019-05-02T23:16:04,415][INFO ][o.e.p.PluginsService ] [2GMRlsy] loaded module [aggs-matrix-stats]
[2019-05-02T23:16:04,415][INFO ][o.e.p.PluginsService ] [2GMRlsy] loaded module [ingest-common]
[2019-05-02T23:16:04,415][INFO ][o.e.p.PluginsService ] [2GMRlsy] loaded module [lang-expression]
[2019-05-02T23:16:04,415][INFO ][o.e.p.PluginsService ] [2GMRlsy] loaded module [lang-groovy]
[2019-05-02T23:16:04,415][INFO ][o.e.p.PluginsService ] [2GMRlsy] loaded module [lang-mustache]
[2019-05-02T23:16:04,416][INFO ][o.e.p.PluginsService ] [2GMRlsy] loaded module [lang-painless]
[2019-05-02T23:16:04,416][INFO ][o.e.p.PluginsService ] [2GMRlsy] loaded module [percolator]
[2019-05-02T23:16:04,416][INFO ][o.e.p.PluginsService ] [2GMRlsy] loaded module [reindex]
[2019-05-02T23:16:04,416][INFO ][o.e.p.PluginsService ] [2GMRlsy] loaded module [transport-netty3]
[2019-05-02T23:16:04,416][INFO ][o.e.p.PluginsService ] [2GMRlsy] loaded module [transport-netty4]
[2019-05-02T23:16:04,416][INFO ][o.e.p.PluginsService ] [2GMRlsy] loaded plugin [analysis-kuromoji]
[2019-05-02T23:16:04,416][INFO ][o.e.p.PluginsService ] [2GMRlsy] loaded plugin [x-pack]
[2019-05-02T23:16:04,593][WARN ][o.e.d.s.g.GroovyScriptEngineService] [groovy] scripts are deprecated, use [painless] scripts instead
[2019-05-02T23:16:04,919][WARN ][o.e.b.ElasticsearchUncaughtExceptionHandler] uncaught exception in thread [main]
org.elasticsearch.bootstrap.StartupException: java.lang.IllegalStateException: No match found

tkwon · May 3, 2019, 6:01pm

I switched images to this one and it seems to be working:

elastic/elasticsearch:6.5.0

halfer · May 3, 2019, 7:26pm

No, those are Elasticsearch logs. The image digest is at the Docker level, and is reported by CircleCI before Elasticsearch is starting or started. It would look like bitjourney/elasticsearch-ci@sha256:hash, where the word “hash” is a 40 character hex value.

I wonder if there was a breaking change in bitjourney/elasticsearch-ci. If so, pinning to a specific hash would have fixed that, but such a fix should be considered temporary, as it locks you to a specific version of the image.

Anyway, if you have fixed it with a different parent image, it’s all good .

system · May 16, 2019, 1:12pm

This topic was automatically closed 10 days after the last reply. New replies are no longer allowed.

Topic		Replies	Views
ElasticSearch exists with code 137 Build Environment docker , elasticsearch	1	14013	August 31, 2021
Build fail with circleci/node:latest Build Environment	5	4479	October 28, 2020
Elasticsearch integration with CI 2.0 Build Environment elasticsearch	8	5585	June 20, 2019
ElasticSearch Docker Build Environment docker	4	1358	July 5, 2019
Recent CircleCI environment changes causing build failure Build Environment	1	351	December 15, 2022

Elasticsearch image failure

Related topics