So my CircleCI builds were running fine using python setup.py install however now I’m suddenly getting module not found error and I have no idea why. It is incredibly weird as I checked the commit history and nothing has changed regarding import statement or build instructions. It makes no sense.
Here is the log
python3 -m venv venv
. venv/bin/activate
pip install -r requirements.txt
python setup.py install
Requirement already satisfied: scikit-learn in ./venv/lib/python3.7/site-packages (from -r requirements.txt (line 1)) (0.21.3)
Requirement already satisfied: pandas in ./venv/lib/python3.7/site-packages (from -r requirements.txt (line 2)) (0.25.1)
Requirement already satisfied: torch in ./venv/lib/python3.7/site-packages (from -r requirements.txt (line 3)) (1.2.0)
Requirement already satisfied: tb-nightly in ./venv/lib/python3.7/site-packages (from -r requirements.txt (line 4)) (1.15.0a20190902)
Requirement already satisfied: seaborn in ./venv/lib/python3.7/site-packages (from -r requirements.txt (line 5)) (0.9.0)
Requirement already satisfied: future in ./venv/lib/python3.7/site-packages (from -r requirements.txt (line 6)) (0.17.1)
Requirement already satisfied: joblib>=0.11 in ./venv/lib/python3.7/site-packages (from scikit-learn->-r requirements.txt (line 1)) (0.13.2)
Requirement already satisfied: numpy>=1.11.0 in ./venv/lib/python3.7/site-packages (from scikit-learn->-r requirements.txt (line 1)) (1.17.1)
Requirement already satisfied: scipy>=0.17.0 in ./venv/lib/python3.7/site-packages (from scikit-learn->-r requirements.txt (line 1)) (1.3.1)
Requirement already satisfied: pytz>=2017.2 in ./venv/lib/python3.7/site-packages (from pandas->-r requirements.txt (line 2)) (2019.2)
Requirement already satisfied: python-dateutil>=2.6.1 in ./venv/lib/python3.7/site-packages (from pandas->-r requirements.txt (line 2)) (2.8.0)
Requirement already satisfied: wheel>=0.26; python_version >= "3" in ./venv/lib/python3.7/site-packages (from tb-nightly->-r requirements.txt (line 4)) (0.33.6)
Requirement already satisfied: markdown>=2.6.8 in ./venv/lib/python3.7/site-packages (from tb-nightly->-r requirements.txt (line 4)) (3.1.1)
Requirement already satisfied: six>=1.10.0 in ./venv/lib/python3.7/site-packages (from tb-nightly->-r requirements.txt (line 4)) (1.12.0)
Requirement already satisfied: setuptools>=41.0.0 in ./venv/lib/python3.7/site-packages (from tb-nightly->-r requirements.txt (line 4)) (41.2.0)
Requirement already satisfied: absl-py>=0.4 in ./venv/lib/python3.7/site-packages (from tb-nightly->-r requirements.txt (line 4)) (0.8.0)
Requirement already satisfied: protobuf>=3.6.0 in ./venv/lib/python3.7/site-packages (from tb-nightly->-r requirements.txt (line 4)) (3.9.1)
Requirement already satisfied: grpcio>=1.6.3 in ./venv/lib/python3.7/site-packages (from tb-nightly->-r requirements.txt (line 4)) (1.23.0)
Requirement already satisfied: werkzeug>=0.11.15 in ./venv/lib/python3.7/site-packages (from tb-nightly->-r requirements.txt (line 4)) (0.15.5)
Requirement already satisfied: matplotlib>=1.4.3 in ./venv/lib/python3.7/site-packages (from seaborn->-r requirements.txt (line 5)) (3.1.1)
Requirement already satisfied: cycler>=0.10 in ./venv/lib/python3.7/site-packages (from matplotlib>=1.4.3->seaborn->-r requirements.txt (line 5)) (0.10.0)
Requirement already satisfied: kiwisolver>=1.0.1 in ./venv/lib/python3.7/site-packages (from matplotlib>=1.4.3->seaborn->-r requirements.txt (line 5)) (1.1.0)
Requirement already satisfied: pyparsing!=2.0.4,!=2.1.2,!=2.1.6,>=2.0.1 in ./venv/lib/python3.7/site-packages (from matplotlib>=1.4.3->seaborn->-r requirements.txt (line 5)) (2.4.2)
You are using pip version 19.0.3, however version 19.2.3 is available.
You should consider upgrading via the 'pip install --upgrade pip' command.
/home/circleci/repo/venv/lib/python3.7/site-packages/setuptools/dist.py:474: UserWarning: Normalizing '0.01dev' to '0.1.dev0'
normalized_version,
running install
running bdist_egg
running egg_info
creating flood_forecast.egg-info
writing flood_forecast.egg-info/PKG-INFO
writing dependency_links to flood_forecast.egg-info/dependency_links.txt
writing requirements to flood_forecast.egg-info/requires.txt
writing top-level names to flood_forecast.egg-info/top_level.txt
writing manifest file 'flood_forecast.egg-info/SOURCES.txt'
package init file 'flood_forecast/__init__.py' not found (or not a regular file)
package init file 'flood_forecast/transformer_xl/__init__.py' not found (or not a regular file)
package init file 'flood_forecast/preprocessing/__init__.py' not found (or not a regular file)
package init file 'flood_forecast/da_rnn/__init__.py' not found (or not a regular file)
package init file 'flood_forecast/basic/__init__.py' not found (or not a regular file)
reading manifest file 'flood_forecast.egg-info/SOURCES.txt'
writing manifest file 'flood_forecast.egg-info/SOURCES.txt'
installing library code to build/bdist.linux-x86_64/egg
running install_lib
running build_py
creating build
creating build/lib
creating build/lib/flood_forecast
copying flood_forecast/train_model.py -> build/lib/flood_forecast
copying flood_forecast/trainer.py -> build/lib/flood_forecast
creating build/lib/flood_forecast/transformer_xl
copying flood_forecast/transformer_xl/transformer_xl.py -> build/lib/flood_forecast/transformer_xl
creating build/lib/flood_forecast/preprocessing
copying flood_forecast/preprocessing/preprocess_da_rnn.py -> build/lib/flood_forecast/preprocessing
creating build/lib/flood_forecast/da_rnn
copying flood_forecast/da_rnn/modules.py -> build/lib/flood_forecast/da_rnn
copying flood_forecast/da_rnn/utils.py -> build/lib/flood_forecast/da_rnn
copying flood_forecast/da_rnn/main_predict.py -> build/lib/flood_forecast/da_rnn
copying flood_forecast/da_rnn/constants.py -> build/lib/flood_forecast/da_rnn
copying flood_forecast/da_rnn/custom_types.py -> build/lib/flood_forecast/da_rnn
copying flood_forecast/da_rnn/train_da.py -> build/lib/flood_forecast/da_rnn
creating build/bdist.linux-x86_64
creating build/bdist.linux-x86_64/egg
creating build/bdist.linux-x86_64/egg/flood_forecast
copying build/lib/flood_forecast/train_model.py -> build/bdist.linux-x86_64/egg/flood_forecast
copying build/lib/flood_forecast/trainer.py -> build/bdist.linux-x86_64/egg/flood_forecast
creating build/bdist.linux-x86_64/egg/flood_forecast/preprocessing
copying build/lib/flood_forecast/preprocessing/preprocess_da_rnn.py -> build/bdist.linux-x86_64/egg/flood_forecast/preprocessing
creating build/bdist.linux-x86_64/egg/flood_forecast/transformer_xl
copying build/lib/flood_forecast/transformer_xl/transformer_xl.py -> build/bdist.linux-x86_64/egg/flood_forecast/transformer_xl
creating build/bdist.linux-x86_64/egg/flood_forecast/da_rnn
copying build/lib/flood_forecast/da_rnn/modules.py -> build/bdist.linux-x86_64/egg/flood_forecast/da_rnn
copying build/lib/flood_forecast/da_rnn/utils.py -> build/bdist.linux-x86_64/egg/flood_forecast/da_rnn
copying build/lib/flood_forecast/da_rnn/main_predict.py -> build/bdist.linux-x86_64/egg/flood_forecast/da_rnn
copying build/lib/flood_forecast/da_rnn/constants.py -> build/bdist.linux-x86_64/egg/flood_forecast/da_rnn
copying build/lib/flood_forecast/da_rnn/custom_types.py -> build/bdist.linux-x86_64/egg/flood_forecast/da_rnn
copying build/lib/flood_forecast/da_rnn/train_da.py -> build/bdist.linux-x86_64/egg/flood_forecast/da_rnn
byte-compiling build/bdist.linux-x86_64/egg/flood_forecast/train_model.py to train_model.cpython-37.pyc
byte-compiling build/bdist.linux-x86_64/egg/flood_forecast/trainer.py to trainer.cpython-37.pyc
byte-compiling build/bdist.linux-x86_64/egg/flood_forecast/preprocessing/preprocess_da_rnn.py to preprocess_da_rnn.cpython-37.pyc
byte-compiling build/bdist.linux-x86_64/egg/flood_forecast/transformer_xl/transformer_xl.py to transformer_xl.cpython-37.pyc
byte-compiling build/bdist.linux-x86_64/egg/flood_forecast/da_rnn/modules.py to modules.cpython-37.pyc
byte-compiling build/bdist.linux-x86_64/egg/flood_forecast/da_rnn/utils.py to utils.cpython-37.pyc
byte-compiling build/bdist.linux-x86_64/egg/flood_forecast/da_rnn/main_predict.py to main_predict.cpython-37.pyc
byte-compiling build/bdist.linux-x86_64/egg/flood_forecast/da_rnn/constants.py to constants.cpython-37.pyc
byte-compiling build/bdist.linux-x86_64/egg/flood_forecast/da_rnn/custom_types.py to custom_types.cpython-37.pyc
byte-compiling build/bdist.linux-x86_64/egg/flood_forecast/da_rnn/train_da.py to train_da.cpython-37.pyc
creating build/bdist.linux-x86_64/egg/EGG-INFO
copying flood_forecast.egg-info/PKG-INFO -> build/bdist.linux-x86_64/egg/EGG-INFO
copying flood_forecast.egg-info/SOURCES.txt -> build/bdist.linux-x86_64/egg/EGG-INFO
copying flood_forecast.egg-info/dependency_links.txt -> build/bdist.linux-x86_64/egg/EGG-INFO
copying flood_forecast.egg-info/requires.txt -> build/bdist.linux-x86_64/egg/EGG-INFO
copying flood_forecast.egg-info/top_level.txt -> build/bdist.linux-x86_64/egg/EGG-INFO
zip_safe flag not set; analyzing archive contents...
creating dist
creating 'dist/flood_forecast-0.1.dev0-py3.7.egg' and adding 'build/bdist.linux-x86_64/egg' to it
removing 'build/bdist.linux-x86_64/egg' (and everything under it)
Processing flood_forecast-0.1.dev0-py3.7.egg
removing '/home/circleci/repo/venv/lib/python3.7/site-packages/flood_forecast-0.1.dev0-py3.7.egg' (and everything under it)
Copying flood_forecast-0.1.dev0-py3.7.egg to /home/circleci/repo/venv/lib/python3.7/site-packages
flood-forecast 0.1.dev0 is already the active version in easy-install.pth
Installed /home/circleci/repo/venv/lib/python3.7/site-packages/flood_forecast-0.1.dev0-py3.7.egg
Processing dependencies for flood-forecast==0.1.dev0
Searching for pandas==0.25.1
Best match: pandas 0.25.1
Adding pandas 0.25.1 to easy-install.pth file
Using /home/circleci/repo/venv/lib/python3.7/site-packages
Searching for tensorflow==2.0.0b1
Best match: tensorflow 2.0.0b1
Processing tensorflow-2.0.0b1-py3.7-linux-x86_64.egg
tensorflow 2.0.0b1 is already the active version in easy-install.pth
Installing freeze_graph script to /home/circleci/repo/venv/bin
Installing saved_model_cli script to /home/circleci/repo/venv/bin
Installing tensorboard script to /home/circleci/repo/venv/bin
Installing tf_upgrade_v2 script to /home/circleci/repo/venv/bin
Installing tflite_convert script to /home/circleci/repo/venv/bin
Installing toco script to /home/circleci/repo/venv/bin
Installing toco_from_protos script to /home/circleci/repo/venv/bin
Using /home/circleci/repo/venv/lib/python3.7/site-packages/tensorflow-2.0.0b1-py3.7-linux-x86_64.egg
Searching for torch==1.2.0
Best match: torch 1.2.0
Adding torch 1.2.0 to easy-install.pth file
Installing convert-caffe2-to-onnx script to /home/circleci/repo/venv/bin
Installing convert-onnx-to-caffe2 script to /home/circleci/repo/venv/bin
Using /home/circleci/repo/venv/lib/python3.7/site-packages
Searching for scikit-learn==0.21.3
Best match: scikit-learn 0.21.3
Adding scikit-learn 0.21.3 to easy-install.pth file
Using /home/circleci/repo/venv/lib/python3.7/site-packages
Searching for numpy==1.17.1
Best match: numpy 1.17.1
Adding numpy 1.17.1 to easy-install.pth file
Installing f2py script to /home/circleci/repo/venv/bin
Installing f2py3 script to /home/circleci/repo/venv/bin
Installing f2py3.7 script to /home/circleci/repo/venv/bin
Using /home/circleci/repo/venv/lib/python3.7/site-packages
Searching for pytz==2019.2
Best match: pytz 2019.2
Adding pytz 2019.2 to easy-install.pth file
Using /home/circleci/repo/venv/lib/python3.7/site-packages
Searching for python-dateutil==2.8.0
Best match: python-dateutil 2.8.0
Adding python-dateutil 2.8.0 to easy-install.pth file
Using /home/circleci/repo/venv/lib/python3.7/site-packages
Searching for wrapt==1.11.2
Best match: wrapt 1.11.2
Processing wrapt-1.11.2-py3.7-linux-x86_64.egg
wrapt 1.11.2 is already the active version in easy-install.pth
Using /home/circleci/repo/venv/lib/python3.7/site-packages/wrapt-1.11.2-py3.7-linux-x86_64.egg
Searching for wheel==0.33.6
Best match: wheel 0.33.6
Adding wheel 0.33.6 to easy-install.pth file
Installing wheel script to /home/circleci/repo/venv/bin
Using /home/circleci/repo/venv/lib/python3.7/site-packages
Searching for tf-estimator-nightly==1.14.0.dev2019060501
Best match: tf-estimator-nightly 1.14.0.dev2019060501
Processing tf_estimator_nightly-1.14.0.dev2019060501-py3.7.egg
tf-estimator-nightly 1.14.0.dev2019060501 is already the active version in easy-install.pth
Using /home/circleci/repo/venv/lib/python3.7/site-packages/tf_estimator_nightly-1.14.0.dev2019060501-py3.7.egg
Searching for termcolor==1.1.0
Best match: termcolor 1.1.0
Processing termcolor-1.1.0-py3.7.egg
termcolor 1.1.0 is already the active version in easy-install.pth
Using /home/circleci/repo/venv/lib/python3.7/site-packages/termcolor-1.1.0-py3.7.egg
Searching for tb-nightly==1.14.0a20190603
Best match: tb-nightly 1.14.0a20190603
Processing tb_nightly-1.14.0a20190603-py3.7.egg
tb-nightly 1.14.0a20190603 is already the active version in easy-install.pth
Installing tensorboard script to /home/circleci/repo/venv/bin
Using /home/circleci/repo/venv/lib/python3.7/site-packages/tb_nightly-1.14.0a20190603-py3.7.egg
Searching for six==1.12.0
Best match: six 1.12.0
Adding six 1.12.0 to easy-install.pth file
Using /home/circleci/repo/venv/lib/python3.7/site-packages
Searching for protobuf==3.9.1
Best match: protobuf 3.9.1
Adding protobuf 3.9.1 to easy-install.pth file
Using /home/circleci/repo/venv/lib/python3.7/site-packages
Searching for Keras-Preprocessing==1.1.0
Best match: Keras-Preprocessing 1.1.0
Processing Keras_Preprocessing-1.1.0-py3.7.egg
Keras-Preprocessing 1.1.0 is already the active version in easy-install.pth
Using /home/circleci/repo/venv/lib/python3.7/site-packages/Keras_Preprocessing-1.1.0-py3.7.egg
Searching for Keras-Applications==1.0.8
Best match: Keras-Applications 1.0.8
Processing Keras_Applications-1.0.8-py3.7.egg
Keras-Applications 1.0.8 is already the active version in easy-install.pth
Using /home/circleci/repo/venv/lib/python3.7/site-packages/Keras_Applications-1.0.8-py3.7.egg
Searching for grpcio==1.23.0
Best match: grpcio 1.23.0
Adding grpcio 1.23.0 to easy-install.pth file
Using /home/circleci/repo/venv/lib/python3.7/site-packages
Searching for google-pasta==0.1.7
Best match: google-pasta 0.1.7
Processing google_pasta-0.1.7-py3.7.egg
google-pasta 0.1.7 is already the active version in easy-install.pth
Using /home/circleci/repo/venv/lib/python3.7/site-packages/google_pasta-0.1.7-py3.7.egg
Searching for gast==0.2.2
Best match: gast 0.2.2
Processing gast-0.2.2-py3.7.egg
gast 0.2.2 is already the active version in easy-install.pth
Using /home/circleci/repo/venv/lib/python3.7/site-packages/gast-0.2.2-py3.7.egg
Searching for astor==0.8.0
Best match: astor 0.8.0
Processing astor-0.8.0-py3.7.egg
astor 0.8.0 is already the active version in easy-install.pth
Using /home/circleci/repo/venv/lib/python3.7/site-packages/astor-0.8.0-py3.7.egg
Searching for absl-py==0.8.0
Best match: absl-py 0.8.0
Adding absl-py 0.8.0 to easy-install.pth file
Using /home/circleci/repo/venv/lib/python3.7/site-packages
Searching for joblib==0.13.2
Best match: joblib 0.13.2
Adding joblib 0.13.2 to easy-install.pth file
Using /home/circleci/repo/venv/lib/python3.7/site-packages
Searching for scipy==1.3.1
Best match: scipy 1.3.1
Adding scipy 1.3.1 to easy-install.pth file
Using /home/circleci/repo/venv/lib/python3.7/site-packages
Searching for Werkzeug==0.15.5
Best match: Werkzeug 0.15.5
Adding Werkzeug 0.15.5 to easy-install.pth file
Using /home/circleci/repo/venv/lib/python3.7/site-packages
Searching for setuptools==41.2.0
Best match: setuptools 41.2.0
Adding setuptools 41.2.0 to easy-install.pth file
Installing easy_install script to /home/circleci/repo/venv/bin
Installing easy_install-3.6 script to /home/circleci/repo/venv/bin
Using /home/circleci/repo/venv/lib/python3.7/site-packages
Searching for Markdown==3.1.1
Best match: Markdown 3.1.1
Adding Markdown 3.1.1 to easy-install.pth file
Installing markdown_py script to /home/circleci/repo/venv/bin
Using /home/circleci/repo/venv/lib/python3.7/site-packages
Searching for h5py==2.9.0
Best match: h5py 2.9.0
Processing h5py-2.9.0-py3.7-linux-x86_64.egg
h5py 2.9.0 is already the active version in easy-install.pth
Using /home/circleci/repo/venv/lib/python3.7/site-packages/h5py-2.9.0-py3.7-linux-x86_64.egg
Finished processing dependencies for flood-forecast==0.1.dev0
Here is the error
File "tests/test_preprocessing.py", line 3, in <module>
from flood_forecast.preprocessing.preprocess_da_rnn import TrainData, format_data, make_data
ModuleNotFoundError: No module named 'flood_forecast'
Exited with code 1```