Pip install dlib fails

I have a python project that requires dlib . I am trying to setup CircleCI and wrote my config.yml as follows:

# Python CircleCI 2.0 configuration file
#
# Check https://circleci.com/docs/2.0/language-python/ for more details
#
version: 2
jobs:
  build:
    docker:
      # specify the version you desire here
      # use `-browsers` prefix for selenium tests, e.g. `3.6.1-browsers`
      - image: circleci/python:3.6.1

      # Specify service dependencies here if necessary
      # CircleCI maintains a library of pre-built images
      # documented at https://circleci.com/docs/2.0/circleci-images/
      # - image: circleci/postgres:9.4

    working_directory: ~/repo

    steps:
      - checkout

      # Download and cache dependencies
      - restore_cache:
          keys:
            - v1-dependencies-{{ checksum "requirements.txt" }}
            # fallback to using the latest cache if no exact match is found
            - v1-dependencies-

      - run:
          name: install dlib
          command: |
            sudo apt-get update
            sudo apt-get install build-essential cmake pkg-config
            sudo apt-get install libatlas-base-dev
            sudo apt-get install libgtk-3-dev libboost-python-dev
            sudo apt-get install libopenblas-dev liblapack-dev
            sudo apt-get install libboost-all-dev
            sudo apt-get install libx11-dev libgtk-3-dev
            sudo apt-get install python python-dev python-pip
            sudo apt-get install python3 python3-dev python3-pip

      - run:
          name: install dependencies
          command: |
            python3 -m venv venv
            . venv/bin/activate
            pip install -r requirements.txt

      - save_cache:
          paths:
            - ./venv
          key: v1-dependencies-{{ checksum "requirements.txt" }}

requirements.txt has dlib inside. However, when I push this to GitHub to run the CI, I get the following error:

Command "/home/circleci/repo/venv/bin/python3 -u -c "import setuptools, tokenize;__file__='/tmp/pip-build-v87eln0u/dlib/setup.py';f=getattr(tokenize, 'open', open)(__file__);code=f.read().replace('\r\n', '\n');f.close();exec(compile(code, __file__, 'exec'))" install --record /tmp/pip-gpf75fsu-record/install-record.txt --single-version-externally-managed --compile --install-headers /home/circleci/repo/venv/include/site/python3.6/dlib" failed with error code 1 in /tmp/pip-build-v87eln0u/dlib/

I have a python project that requires dlib . I am trying to setup CircleCI and wrote my config.yml as follows:

# Python CircleCI 2.0 configuration file
#
# Check https://circleci.com/docs/2.0/language-python/ for more details
#
version: 2
jobs:
  build:
    docker:
      # specify the version you desire here
      # use `-browsers` prefix for selenium tests, e.g. `3.6.1-browsers`
      - image: circleci/python:3.6.1

      # Specify service dependencies here if necessary
      # CircleCI maintains a library of pre-built images
      # documented at https://circleci.com/docs/2.0/circleci-images/
      # - image: circleci/postgres:9.4

    working_directory: ~/repo

    steps:
      - checkout

      # Download and cache dependencies
      - restore_cache:
          keys:
            - v1-dependencies-{{ checksum "requirements.txt" }}
            # fallback to using the latest cache if no exact match is found
            - v1-dependencies-

      - run:
          name: install dlib
          command: |
            sudo apt-get update
            sudo apt-get install build-essential cmake pkg-config
            sudo apt-get install libatlas-base-dev
            sudo apt-get install libgtk-3-dev libboost-python-dev
            sudo apt-get install libopenblas-dev liblapack-dev
            sudo apt-get install libboost-all-dev
            sudo apt-get install libx11-dev libgtk-3-dev
            sudo apt-get install python3 python3-dev python3-pip

      - run:
          name: install dependencies
          command: |
            python3 -m venv venv
            . venv/bin/activate
            pip install -r requirements.txt

      - save_cache:
          paths:
            - ./venv
          key: v1-dependencies-{{ checksum "requirements.txt" }}

requirements.txt has dlib inside. However, when I push this to GitHub to run the CI, I get the following error:

Command "/home/circleci/repo/venv/bin/python3 -u -c "import setuptools, tokenize;__file__='/tmp/pip-build-v87eln0u/dlib/setup.py';f=getattr(tokenize, 'open', open)(__file__);code=f.read().replace('\r\n', '\n');f.close();exec(compile(code, __file__, 'exec'))" install --record /tmp/pip-gpf75fsu-record/install-record.txt --single-version-externally-managed --compile --install-headers /home/circleci/repo/venv/include/site/python3.6/dlib" failed with error code 1 in /tmp/pip-build-v87eln0u/dlib/

Here’s the traceback:

    Traceback (most recent call last):
      File "<string>", line 1, in <module>
      File "/tmp/pip-build-v87eln0u/dlib/setup.py", line 261, in <module>
        'Topic :: Software Development',
      File "/home/circleci/repo/venv/lib/python3.6/site-packages/setuptools/__init__.py", line 145, in setup
        return distutils.core.setup(**attrs)
      File "/usr/local/lib/python3.6/distutils/core.py", line 148, in setup
        dist.run_commands()
      File "/usr/local/lib/python3.6/distutils/dist.py", line 955, in run_commands
        self.run_command(cmd)
      File "/usr/local/lib/python3.6/distutils/dist.py", line 974, in run_command
        cmd_obj.run()
      File "/home/circleci/repo/venv/lib/python3.6/site-packages/setuptools/command/install.py", line 61, in run
        return orig.install.run(self)
      File "/usr/local/lib/python3.6/distutils/command/install.py", line 545, in run
        self.run_command('build')
      File "/usr/local/lib/python3.6/distutils/cmd.py", line 313, in run_command
        self.distribution.run_command(command)
      File "/usr/local/lib/python3.6/distutils/dist.py", line 974, in run_command
        cmd_obj.run()
      File "/usr/local/lib/python3.6/distutils/command/build.py", line 135, in run
        self.run_command(cmd_name)
      File "/usr/local/lib/python3.6/distutils/cmd.py", line 313, in run_command
        self.distribution.run_command(command)
      File "/usr/local/lib/python3.6/distutils/dist.py", line 974, in run_command
        cmd_obj.run()
      File "/tmp/pip-build-v87eln0u/dlib/setup.py", line 135, in run
        self.build_extension(ext)
      File "/tmp/pip-build-v87eln0u/dlib/setup.py", line 175, in build_extension
        subprocess.check_call(cmake_build, cwd=build_folder)
      File "/usr/local/lib/python3.6/subprocess.py", line 291, in check_call
        raise CalledProcessError(retcode, cmd)
    subprocess.CalledProcessError: Command '['cmake', '--build', '.', '--config', 'Release', '--', '-j34']' returned non-zero exit status 2.

It seems like I’m not setting the linux environment right in order for dlib to be installed successfully. What am I missing here? How can I run CircleCI with dlib properly configured on python?

It seems like the CMake version that’s being installed is 3.0.2.

There’s a related question here, which was not answered.

I’m struggling to see an actual error in that stack trace. Is there any more information available?

Here’s the full error after dlib is downloaded up to 58%. Hope this helps.

    Building CXX object dlib_build/CMakeFiles/dlib.dir/cuda/tensor_tools.cpp.o
    Building CXX object dlib_build/CMakeFiles/dlib.dir/cuda/cpu_dlib.cpp.o
    [ 57%] [ 57%] [ 58%] Building CXX object dlib_build/CMakeFiles/dlib.dir/gui_widgets/widgets.cpp.o
    Building CXX object dlib_build/CMakeFiles/dlib.dir/gui_widgets/drawable.cpp.o
    Building CXX object dlib_build/CMakeFiles/dlib.dir/gui_widgets/fonts.cpp.o
    c++: internal compiler error: Killed (program cc1plus)
    Please submit a full bug report,
    with preprocessed source if appropriate.
    See <file:///usr/share/doc/gcc-4.9/README.Bugs> for instructions.
    dlib_build/CMakeFiles/dlib.dir/build.make:376: recipe for target 'dlib_build/CMakeFiles/dlib.dir/filtering/kalman_filter.cpp.o' failed
    make[2]: *** [dlib_build/CMakeFiles/dlib.dir/filtering/kalman_filter.cpp.o] Error 4
    make[2]: *** Waiting for unfinished jobs....
    c++: internal compiler error: Killed (program cc1plus)
    Please submit a full bug report,
    with preprocessed source if appropriate.
    See <file:///usr/share/doc/gcc-4.9/README.Bugs> for instructions.
    dlib_build/CMakeFiles/dlib.dir/build.make:859: recipe for target 'dlib_build/CMakeFiles/dlib.dir/threads/multithreaded_object_extension.cpp.o' failed
    make[2]: *** [dlib_build/CMakeFiles/dlib.dir/threads/multithreaded_object_extension.cpp.o] Error 4
    c++: internal compiler error: Killed (program cc1plus)
    Please submit a full bug report,
    with preprocessed source if appropriate.
    See <file:///usr/share/doc/gcc-4.9/README.Bugs> for instructions.
    dlib_build/CMakeFiles/dlib.dir/build.make:652: recipe for target 'dlib_build/CMakeFiles/dlib.dir/misc_api/misc_api_kernel_1.cpp.o' failed
    make[2]: *** [dlib_build/CMakeFiles/dlib.dir/misc_api/misc_api_kernel_1.cpp.o] Error 4
    c++: internal compiler error: Killed (program cc1plus)
    Please submit a full bug report,
    with preprocessed source if appropriate.
    See <file:///usr/share/doc/gcc-4.9/README.Bugs> for instructions.
    dlib_build/CMakeFiles/dlib.dir/build.make:836: recipe for target 'dlib_build/CMakeFiles/dlib.dir/server/server_http.cpp.o' failed
    make[2]: *** [dlib_build/CMakeFiles/dlib.dir/server/server_http.cpp.o] Error 4
    c++: internal compiler error: Killed (program cc1plus)
    Please submit a full bug report,
    with preprocessed source if appropriate.
    See <file:///usr/share/doc/gcc-4.9/README.Bugs> for instructions.
    dlib_build/CMakeFiles/dlib.dir/build.make:583: recipe for target 'dlib_build/CMakeFiles/dlib.dir/logger/extra_logger_headers.cpp.o' failed
    make[2]: *** [dlib_build/CMakeFiles/dlib.dir/logger/extra_logger_headers.cpp.o] Error 4
    c++: internal compiler error: Killed (program cc1plus)
    Please submit a full bug report,
    with preprocessed source if appropriate.
    See <file:///usr/share/doc/gcc-4.9/README.Bugs> for instructions.
    dlib_build/CMakeFiles/dlib.dir/build.make:813: recipe for target 'dlib_build/CMakeFiles/dlib.dir/server/server_iostream.cpp.o' failed
    make[2]: *** [dlib_build/CMakeFiles/dlib.dir/server/server_iostream.cpp.o] Error 4
    CMakeFiles/Makefile2:125: recipe for target 'dlib_build/CMakeFiles/dlib.dir/all' failed
    make[1]: *** [dlib_build/CMakeFiles/dlib.dir/all] Error 2
    Makefile:76: recipe for target 'all' failed
    make: *** [all] Error 2
    Traceback (most recent call last):
      File "<string>", line 1, in <module>
      File "/tmp/pip-build-gr52giy4/dlib/setup.py", line 261, in <module>
        'Topic :: Software Development',
      File "/usr/local/lib/python3.6/distutils/core.py", line 148, in setup
        dist.run_commands()
      File "/usr/local/lib/python3.6/distutils/dist.py", line 955, in run_commands
        self.run_command(cmd)
      File "/usr/local/lib/python3.6/distutils/dist.py", line 974, in run_command
        cmd_obj.run()
      File "/home/circleci/repo/venv/lib/python3.6/site-packages/setuptools/command/install.py", line 61, in run
        return orig.install.run(self)
      File "/usr/local/lib/python3.6/distutils/command/install.py", line 545, in run
        self.run_command('build')
      File "/usr/local/lib/python3.6/distutils/cmd.py", line 313, in run_command
        self.distribution.run_command(command)
      File "/usr/local/lib/python3.6/distutils/dist.py", line 974, in run_command
        cmd_obj.run()
      File "/usr/local/lib/python3.6/distutils/command/build.py", line 135, in run
        self.run_command(cmd_name)
      File "/usr/local/lib/python3.6/distutils/cmd.py", line 313, in run_command
        self.distribution.run_command(command)
      File "/usr/local/lib/python3.6/distutils/dist.py", line 974, in run_command
        cmd_obj.run()
      File "/tmp/pip-build-gr52giy4/dlib/setup.py", line 135, in run
        self.build_extension(ext)
      File "/tmp/pip-build-gr52giy4/dlib/setup.py", line 175, in build_extension
        subprocess.check_call(cmake_build, cwd=build_folder)
      File "/usr/local/lib/python3.6/subprocess.py", line 291, in check_call
        raise CalledProcessError(retcode, cmd)
    subprocess.CalledProcessError: Command '['cmake', '--build', '.', '--config', 'Release', '--', '-j34']' returned non-zero exit status 2.
    
    ----------------------------------------
Command "/home/circleci/repo/venv/bin/python3 -u -c "import setuptools, tokenize;__file__='/tmp/pip-build-gr52giy4/dlib/setup.py';f=getattr(tokenize, 'open', open)(__file__);code=f.read().replace('\r\n', '\n');f.close();exec(compile(code, __file__, 'exec'))" install --record /tmp/pip-kqvjg682-record/install-record.txt --single-version-externally-managed --compile --install-headers /home/circleci/repo/venv/include/site/python3.6/dlib" failed with error code 1 in /tmp/pip-build-gr52giy4/dlib/
You are using pip version 9.0.1, however version 19.2.3 is available.
You should consider upgrading via the 'pip install --upgrade pip' command.
Exited with code 1

This isn’t particularly illuminating, but as far as I can see, the most interesting information here is this compiler failure. I would recommend a web search (on Stack Overflow etc) to see what the issue could be. Don’t treat this as a “CircleCI problem” - that is almost certainly a false lead. You have a program that won’t compile on a remote VPS, and you need to find out why.

If you are particularly stuck, you could try Dockerising this, so that when it compiles on your machine, it will also compile on the remote. However, it may be quicker to fix what you have.