Elasticsearch image failure

#1

In my build I add an image of elasticsearch. It used to work but in the last day or so it fails:

Here is a sample from my config:

    version: 2
    jobs:
      build:
        docker:
          - image: circleci/python:3.6.4
          - image: bitjourney/elasticsearch-ci
            environment:
              xpack.security.enabled: false
              transport.host: localhost

And the error message:

[2019-05-02T23:26:11,921][INFO ][o.e.n.Node ] initializing …
[2019-05-02T23:26:11,971][INFO ][o.e.e.NodeEnvironment ] [4yqwKie] using [1] data paths, mounts [[/ (none)]], net usable_space [574.3gb], net total_space [825gb], spins? [possibly], types [aufs]
[2019-05-02T23:26:11,971][INFO ][o.e.e.NodeEnvironment ] [4yqwKie] heap size [1.9gb], compressed ordinary object pointers [true]
[2019-05-02T23:26:11,972][INFO ][o.e.n.Node ] node name [4yqwKie] derived from node ID [4yqwKie-Sx-ZWBTtEWHAWw]; set [node.name] to override
[2019-05-02T23:26:11,974][INFO ][o.e.n.Node ] version[5.1.2], pid[6], build[c8c4c16/2017-01-11T20:18:39.146Z], OS[Linux/4.15.0-1035-aws/amd64], JVM[Oracle Corporation/OpenJDK 64-Bit Server VM/1.8.0_92-internal/25.92-b14]
[2019-05-02T23:26:12,979][INFO ][o.e.p.PluginsService ] [4yqwKie] loaded module [aggs-matrix-stats]
[2019-05-02T23:26:12,979][INFO ][o.e.p.PluginsService ] [4yqwKie] loaded module [ingest-common]
[2019-05-02T23:26:12,979][INFO ][o.e.p.PluginsService ] [4yqwKie] loaded module [lang-expression]
[2019-05-02T23:26:12,979][INFO ][o.e.p.PluginsService ] [4yqwKie] loaded module [lang-groovy]
[2019-05-02T23:26:12,979][INFO ][o.e.p.PluginsService ] [4yqwKie] loaded module [lang-mustache]
[2019-05-02T23:26:12,979][INFO ][o.e.p.PluginsService ] [4yqwKie] loaded module [lang-painless]
[2019-05-02T23:26:12,979][INFO ][o.e.p.PluginsService ] [4yqwKie] loaded module [percolator]
[2019-05-02T23:26:12,979][INFO ][o.e.p.PluginsService ] [4yqwKie] loaded module [reindex]
[2019-05-02T23:26:12,979][INFO ][o.e.p.PluginsService ] [4yqwKie] loaded module [transport-netty3]
[2019-05-02T23:26:12,980][INFO ][o.e.p.PluginsService ] [4yqwKie] loaded module [transport-netty4]
[2019-05-02T23:26:12,980][INFO ][o.e.p.PluginsService ] [4yqwKie] loaded plugin [analysis-kuromoji]
[2019-05-02T23:26:12,980][INFO ][o.e.p.PluginsService ] [4yqwKie] loaded plugin [x-pack]
[2019-05-02T23:26:13,194][WARN ][o.e.d.s.g.GroovyScriptEngineService] [groovy] scripts are deprecated, use [painless] scripts instead
[2019-05-02T23:26:13,741][WARN ][o.e.b.ElasticsearchUncaughtExceptionHandler] uncaught exception in thread [main]
org.elasticsearch.bootstrap.StartupException: java.lang.IllegalStateException: No match found
at org.elasticsearch.bootstrap.Elasticsearch.init(Elasticsearch.java:125) ~[elasticsearch-5.1.2.jar:5.1.2]
at org.elasticsearch.bootstrap.Elasticsearch.execute(Elasticsearch.java:112) ~[elasticsearch-5.1.2.jar:5.1.2]
at org.elasticsearch.cli.SettingCommand.execute(SettingCommand.java:54) ~[elasticsearch-5.1.2.jar:5.1.2]
at org.elasticsearch.cli.Command.mainWithoutErrorHandling(Command.java:122) ~[elasticsearch-5.1.2.jar:5.1.2]
at org.elasticsearch.cli.Command.main(Command.java:88) ~[elasticsearch-5.1.2.jar:5.1.2]
at org.elasticsearch.bootstrap.Elasticsearch.main(Elasticsearch.java:89) ~[elasticsearch-5.1.2.jar:5.1.2]
at org.elasticsearch.bootstrap.Elasticsearch.main(Elasticsearch.java:82) ~[elasticsearch-5.1.2.jar:5.1.2]
Caused by: java.lang.IllegalStateException: No match found
at java.util.regex.Matcher.group(Matcher.java:536) ~[?:1.8.0_92-internal]
at org.elasticsearch.monitor.os.OsProbe.getControlGroups(OsProbe.java:213) ~[elasticsearch-5.1.2.jar:5.1.2]
at org.elasticsearch.monitor.os.OsProbe.getCgroup(OsProbe.java:402) ~[elasticsearch-5.1.2.jar:5.1.2]
at org.elasticsearch.monitor.os.OsProbe.osStats(OsProbe.java:454) ~[elasticsearch-5.1.2.jar:5.1.2]
at org.elasticsearch.monitor.os.OsService.(OsService.java:45) ~[elasticsearch-5.1.2.jar:5.1.2]
at org.elasticsearch.monitor.MonitorService.(MonitorService.java:45) ~[elasticsearch-5.1.2.jar:5.1.2]
at org.elasticsearch.node.Node.(Node.java:345) ~[elasticsearch-5.1.2.jar:5.1.2]
at org.elasticsearch.node.Node.(Node.java:229) ~[elasticsearch-5.1.2.jar:5.1.2]
at org.elasticsearch.bootstrap.Bootstrap$6.(Bootstrap.java:214) ~[elasticsearch-5.1.2.jar:5.1.2]
at org.elasticsearch.bootstrap.Bootstrap.setup(Bootstrap.java:214) ~[elasticsearch-5.1.2.jar:5.1.2]
at org.elasticsearch.bootstrap.Bootstrap.init(Bootstrap.java:306) ~[elasticsearch-5.1.2.jar:5.1.2]
at org.elasticsearch.bootstrap.Elasticsearch.init(Elasticsearch.java:121) ~[elasticsearch-5.1.2.jar:5.1.2]
… 6 more

Job was canceled

#2

I am facing the same issue as well since yesterday. Did you manage to resolve it?

#3

Has the image digest changed between it working and not working? You can find this in the console output of your first job step.

#4

I’m not sure which is the image digest but guess it’s the hash shown here as “A2nLqGz”. It has changed. The failed one is “2GMRlsy”.

Here’s the tops of both console outputs.

Working:

[2019-04-26T15:59:23,880][INFO ][o.e.n.Node ] initializing …
[2019-04-26T15:59:23,959][INFO ][o.e.e.NodeEnvironment ] [A2nLqGz] using [1] data paths, mounts [[/ (none)]], net usable_space [462gb], net total_space [825gb], spins? [possibly], types [aufs]
[2019-04-26T15:59:23,959][INFO ][o.e.e.NodeEnvironment ] [A2nLqGz] heap size [1.9gb], compressed ordinary object pointers [true]
[2019-04-26T15:59:23,961][INFO ][o.e.n.Node ] node name [A2nLqGz] derived from node ID [A2nLqGzwT8SBb0Er3mp95A]; set [node.name] to override
[2019-04-26T15:59:23,963][INFO ][o.e.n.Node ] version[5.1.2], pid[13], build[c8c4c16/2017-01-11T20:18:39.146Z], OS[Linux/4.4.0-144-generic/amd64], JVM[Oracle Corporation/OpenJDK 64-Bit Server VM/1.8.0_92-internal/25.92-b14]
[2019-04-26T15:59:25,726][INFO ][o.e.p.PluginsService ] [A2nLqGz] loaded module [aggs-matrix-stats]
[2019-04-26T15:59:25,727][INFO ][o.e.p.PluginsService ] [A2nLqGz] loaded module [ingest-common]
[2019-04-26T15:59:25,727][INFO ][o.e.p.PluginsService ] [A2nLqGz] loaded module [lang-expression]
[2019-04-26T15:59:25,727][INFO ][o.e.p.PluginsService ] [A2nLqGz] loaded module [lang-groovy]
[2019-04-26T15:59:25,727][INFO ][o.e.p.PluginsService ] [A2nLqGz] loaded module [lang-mustache]
[2019-04-26T15:59:25,727][INFO ][o.e.p.PluginsService ] [A2nLqGz] loaded module [lang-painless]
[2019-04-26T15:59:25,727][INFO ][o.e.p.PluginsService ] [A2nLqGz] loaded module [percolator]
[2019-04-26T15:59:25,727][INFO ][o.e.p.PluginsService ] [A2nLqGz] loaded module [reindex]
[2019-04-26T15:59:25,727][INFO ][o.e.p.PluginsService ] [A2nLqGz] loaded module [transport-netty3]
[2019-04-26T15:59:25,727][INFO ][o.e.p.PluginsService ] [A2nLqGz] loaded module [transport-netty4]
[2019-04-26T15:59:25,727][INFO ][o.e.p.PluginsService ] [A2nLqGz] loaded plugin [analysis-kuromoji]
[2019-04-26T15:59:25,727][INFO ][o.e.p.PluginsService ] [A2nLqGz] loaded plugin [x-pack]
[2019-04-26T15:59:26,056][WARN ][o.e.d.s.g.GroovyScriptEngineService] [groovy] scripts are deprecated, use [painless] scripts instead
[2019-04-26T15:59:28,727][INFO ][o.e.n.Node ] initialized
[2019-04-26T15:59:28,728][INFO ][o.e.n.Node ] [A2nLqGz] starting …

Failed:

[2019-05-02T23:16:03,538][INFO ][o.e.n.Node ] initializing …
[2019-05-02T23:16:03,583][INFO ][o.e.e.NodeEnvironment ] [2GMRlsy] using [1] data paths, mounts [[/ (none)]], net usable_space [259gb], net total_space [825gb], spins? [possibly], types [aufs]
[2019-05-02T23:16:03,584][INFO ][o.e.e.NodeEnvironment ] [2GMRlsy] heap size [1.9gb], compressed ordinary object pointers [true]
[2019-05-02T23:16:03,585][INFO ][o.e.n.Node ] node name [2GMRlsy] derived from node ID [2GMRlsy7Q0u0Am17qqh5Rg]; set [node.name] to override
[2019-05-02T23:16:03,586][INFO ][o.e.n.Node ] version[5.1.2], pid[32], build[c8c4c16/2017-01-11T20:18:39.146Z], OS[Linux/4.15.0-1035-aws/amd64], JVM[Oracle Corporation/OpenJDK 64-Bit Server VM/1.8.0_92-internal/25.92-b14]
[2019-05-02T23:16:04,415][INFO ][o.e.p.PluginsService ] [2GMRlsy] loaded module [aggs-matrix-stats]
[2019-05-02T23:16:04,415][INFO ][o.e.p.PluginsService ] [2GMRlsy] loaded module [ingest-common]
[2019-05-02T23:16:04,415][INFO ][o.e.p.PluginsService ] [2GMRlsy] loaded module [lang-expression]
[2019-05-02T23:16:04,415][INFO ][o.e.p.PluginsService ] [2GMRlsy] loaded module [lang-groovy]
[2019-05-02T23:16:04,415][INFO ][o.e.p.PluginsService ] [2GMRlsy] loaded module [lang-mustache]
[2019-05-02T23:16:04,416][INFO ][o.e.p.PluginsService ] [2GMRlsy] loaded module [lang-painless]
[2019-05-02T23:16:04,416][INFO ][o.e.p.PluginsService ] [2GMRlsy] loaded module [percolator]
[2019-05-02T23:16:04,416][INFO ][o.e.p.PluginsService ] [2GMRlsy] loaded module [reindex]
[2019-05-02T23:16:04,416][INFO ][o.e.p.PluginsService ] [2GMRlsy] loaded module [transport-netty3]
[2019-05-02T23:16:04,416][INFO ][o.e.p.PluginsService ] [2GMRlsy] loaded module [transport-netty4]
[2019-05-02T23:16:04,416][INFO ][o.e.p.PluginsService ] [2GMRlsy] loaded plugin [analysis-kuromoji]
[2019-05-02T23:16:04,416][INFO ][o.e.p.PluginsService ] [2GMRlsy] loaded plugin [x-pack]
[2019-05-02T23:16:04,593][WARN ][o.e.d.s.g.GroovyScriptEngineService] [groovy] scripts are deprecated, use [painless] scripts instead
[2019-05-02T23:16:04,919][WARN ][o.e.b.ElasticsearchUncaughtExceptionHandler] uncaught exception in thread [main]
org.elasticsearch.bootstrap.StartupException: java.lang.IllegalStateException: No match found

#5

I switched images to this one and it seems to be working:

elastic/elasticsearch:6.5.0

#6

No, those are Elasticsearch logs. The image digest is at the Docker level, and is reported by CircleCI before Elasticsearch is starting or started. It would look like bitjourney/elasticsearch-ci@sha256:hash, where the word “hash” is a 40 character hex value.

I wonder if there was a breaking change in bitjourney/elasticsearch-ci. If so, pinning to a specific hash would have fixed that, but such a fix should be considered temporary, as it locks you to a specific version of the image.

Anyway, if you have fixed it with a different parent image, it’s all good :relieved:.

1 Like
closed #7

This topic was automatically closed 10 days after the last reply. New replies are no longer allowed.