Running Elasticsearch 5

Build Environment
1

Hi guys,

we’re switching to ES 5, however the service won’t start in the CircleCI VM, complaining

ERROR: bootstrap checks failed
max virtual memory areas vm.max_map_count [65530] is too low, increase to at least [262144]

Locally the workaround was to

$ sudo sysctl -w vm.max_map_count=262144                                                                                                     [12:56:22]
[sudo] password for dtk: 
vm.max_map_count = 262144

however in the container provided by CircleCI, this is not allowed (potentially a kernel setting shared with the host?):

$ sudo sysctl -w vm.max_map_count=262144
error: permission denied on key 'vm.max_map_count'

Has anybody fixed this issue? Assuming that multiple people are going to run into this issue, do you think there is a chance to get the required setting as standard on all containers?

Thanks a lot
dtk

2

Are you able to run the Docker image of ElasticSearch, this should work out of the box.

3

I’m not. That’s actually exactly what we’re trying to do:

database:
  pre:
    - docker run --publish=39200:9200 --detach --volume="$PWD/configs/elasticsearch":/usr/share/elasticsearch/config elasticsearch:5.0.2

In fact the problem is kind of announced in the How to use this image / Host Setup section of that image:

Note: since 5.0, […] Elasticsearch is more strict about the bootstrap checks that it performs, especially when checking the value of vm.max_map_count which is not namespaced and thus must be set to an acceptable value on the host (as opposed to simply using --sysctl on docker run).

Thanks for your help!
dtk

2 Likes
4

Just to add, we’re having this too. I’m currently trying to figure out a workaround.

5

We’re also hitting the same problem while using the Docker image of ES because of the CircleCI host setup.

7

I temporarily fixed the issue by disabling the Bootstrap checks introduced in ES5. On ES5.2.2, you can do so by passing
-Ebootstrap.system_call_filter=false -Etransport.host=localhost
to your docker run command.

8

We’re using the image with the disabled checks that @levlaz kindly provides.

1 Like
9

We are disabling that flag in elasticserch.yml:

bootstrap.system_call_filter: false
transport.host: localhost
10

Unfortunately non of the above solutions in an option for us.
Is there any chance to add the possibility to set vm.max_map_count somehow with circleci 2.0?

11

We’ve raised the value of vm.max_map_count on CircleCI 2.0 at this time. Can you try some new builds?

2 Likes
12

Thanks for working with us on that issue. I just tried a few builds using CircleCI 2.0, but still get the issue that vm.max_map_count is too low:

ERROR: bootstrap checks failed
max virtual memory areas vm.max_map_count [65530] is too low, increase to at least [262144]

Here is an excerpt of the dockerfile I am using:

version: '2'
services:
  # ...
  elasticsearch:
    # need to build elasticsearch, since CircleCI does not allow mounting host files/directories into the container
    build:
      context: ..
      dockerfile: docker/Dockerfile.elasticsearch
    environment:
      - "ES_JAVA_OPTS=-Xms512m -Xmx512m"
    mem_swappiness: 1
    mem_limit: 1g
    volumes:
      - elasticsearch:/usr/share/elasticsearch/data
    ports:
      - 9200:9200
    # ...
  volumes:
    # ...
    elasticsearch:

To which value did you increase vm.max_map_count?

1 Like
13

I thought we raised it to 262144
And that’s not a Dockerfile, that’s a docker-compose.yml

14

I confirm that the following workaround works in my docker-compose.yml:

  elasticsearch:
    image: docker.elastic.co/elasticsearch/elasticsearch:5.3.2
    environment:
      - "transport.host=localhost"
      - "bootstrap.system_call_filter=false"
    ports:
      - "9200:9200"

I think the bootstrap.system_call_filter=false might not even be necessary.

15

We are running the same problem here.

I’ve a docker-compose setup (my base image have a docker installed, that run my docker-compose).

The problem is that querying sysctl for vm.max_map_count, it results the right value:
vm.max_map_count = 262144, but doing the same inside any docker-compose service returns
vm.max_map_count = 65530

I don’t know how to solve this issue.

Disabling bootstrap checks is just a workaround to the issue.

1 Like
16

Trying to run Elasticsearch 6 in docker-compose still fails for us with max virtual memory areas vm.max_map_count [65530] is too low, increase to at least [262144], even with "bootstrap.system_call_filter=false".

17

Are you using Docker 2.0, @ThomWright? You could look at switching to the machine executor to see if the increased RAM would help - you get 8G rather than 4G I believe. Just be aware there may be a cost associated with this config in the future (as per the docs), but it’s a free addition for now.

18

Hi,

We are using the machine executor. I don’t know if this is part of the issue? I don’t think increased RAM would help here, though.

19

Ah, gotcha @ThomWright . It doesn’t look like it was raised in the host (or perhaps it was reverted), as it has gone back to 65530. Can you reset it in the build environment (see here) so that your Docker Compose (effectively Docker in Docker) sees the changed value?

Or does this need to be on the Docker host outside of your build environment?

20

It can’t be set on the host:

root@6f072b9f38bb:~/buildhub2# sysctl -w vm.max_map_count=262144
sysctl: setting key "vm.max_map_count": Read-only file system
21

This topic was automatically closed 90 days after the last reply. New replies are no longer allowed.