AWS CodeDeploy: "can't dup nilClass" in DownloadBundle

Feedback & Bug Reports
1

This is an odd one. I have my circle config set up to deploy to AWS CodeDeploy. It is able to create an application revision in S3, and when I download it everything seems to be fine. But it fails on the DownloadBundle phase of deployment, with the AWS codedeploy-agent logs showing me this cryptic traceback:

2016-03-30 01:09:09 INFO  [codedeploy-agent(4677)]: Version file found in /opt/codedeploy-agent/.version.
2016-03-30 01:09:09 INFO  [codedeploy-agent(4677)]: [Aws::CodeDeployCommand::Client 200 0.05109 0 retries] put_host_command_complete(command
_status:"Failed",diagnostics:{format:"JSON",payload:"{\"error_code\":5,\"script_name\":\"\",\"message\":\"can't dup NilClass\",\"log\":\"\"}
"},host_command_identifier:"WyJjbXXX")

2016-03-30 01:09:09 ERROR [codedeploy-agent(4677)]: InstanceAgent::Plugins::CodeDeployPlugin::CommandPoller: Error during perform: TypeError
 - can't dup NilClass - /opt/codedeploy-agent/vendor/gems/rubyzip-1.1.6/lib/zip/entry_set.rb:56:in `dup'
/opt/codedeploy-agent/vendor/gems/rubyzip-1.1.6/lib/zip/entry_set.rb:56:in `block in dup'
/opt/codedeploy-agent/vendor/gems/rubyzip-1.1.6/lib/zip/entry_set.rb:56:in `each'
/opt/codedeploy-agent/vendor/gems/rubyzip-1.1.6/lib/zip/entry_set.rb:56:in `map'
/opt/codedeploy-agent/vendor/gems/rubyzip-1.1.6/lib/zip/entry_set.rb:56:in `dup'
/opt/codedeploy-agent/vendor/gems/rubyzip-1.1.6/lib/zip/file.rb:84:in `initialize'
/opt/codedeploy-agent/vendor/gems/rubyzip-1.1.6/lib/zip/file.rb:96:in `new'
/opt/codedeploy-agent/vendor/gems/rubyzip-1.1.6/lib/zip/file.rb:96:in `open'
/opt/codedeploy-agent/lib/instance_agent/plugins/codedeploy/command_executor.rb:296:in `unpack_bundle'
/opt/codedeploy-agent/lib/instance_agent/plugins/codedeploy/command_executor.rb:99:in `block in <class:CommandExecutor>'
/opt/codedeploy-agent/lib/instance_agent/plugins/codedeploy/command_executor.rb:63:in `execute_command'
/opt/codedeploy-agent/lib/instance_agent/plugins/codedeploy/command_poller.rb:132:in `process_command'
/opt/codedeploy-agent/lib/instance_agent/plugins/codedeploy/command_poller.rb:65:in `perform'
/opt/codedeploy-agent/lib/instance_agent/agent/base.rb:28:in `run'
/opt/codedeploy-agent/lib/instance_agent/runner/child.rb:38:in `block in run'
/opt/codedeploy-agent/lib/instance_agent/runner/child.rb:55:in `with_error_handling'
/opt/codedeploy-agent/lib/instance_agent/runner/child.rb:37:in `run'
/opt/codedeploy-agent/vendor/gems/process_manager-0.0.13/lib/process_manager/child.rb:70:in `block in run_with_error_handling'
/opt/codedeploy-agent/lib/instance_agent/runner/child.rb:55:in `with_error_handling'
/opt/codedeploy-agent/vendor/gems/process_manager-0.0.13/lib/process_manager/child.rb:69:in `run_with_error_handling'
/opt/codedeploy-agent/vendor/gems/process_manager-0.0.13/lib/process_manager/child.rb:33:in `block in start'
/opt/codedeploy-agent/vendor/gems/process_manager-0.0.13/lib/process_manager/child.rb:22:in `loop'
/opt/codedeploy-agent/vendor/gems/process_manager-0.0.13/lib/process_manager/child.rb:22:in `start'
/opt/codedeploy-agent/vendor/gems/process_manager-0.0.13/lib/process_manager/master.rb:202:in `block in spawn_child'
/opt/codedeploy-agent/vendor/gems/process_manager-0.0.13/lib/process_manager/master.rb:200:in `fork'
/opt/codedeploy-agent/vendor/gems/process_manager-0.0.13/lib/process_manager/master.rb:200:in `spawn_child'
/opt/codedeploy-agent/vendor/gems/process_manager-0.0.13/lib/process_manager/master.rb:192:in `block in spawn_children'
/opt/codedeploy-agent/vendor/gems/process_manager-0.0.13/lib/process_manager/master.rb:191:in `times'
/opt/codedeploy-agent/vendor/gems/process_manager-0.0.13/lib/process_manager/master.rb:191:in `spawn_children'
/opt/codedeploy-agent/vendor/gems/process_manager-0.0.13/lib/process_manager/master.rb:134:in `start'
/opt/codedeploy-agent/vendor/gems/process_manager-0.0.13/lib/process_manager/master.rb:37:in `block in start'
/opt/codedeploy-agent/vendor/gems/process_manager-0.0.13/lib/process_manager/master.rb:36:in `fork'
/opt/codedeploy-agent/vendor/gems/process_manager-0.0.13/lib/process_manager/master.rb:36:in `start'
/opt/codedeploy-agent/bin/codedeploy-agent:37:in `block (2 levels) in <main>'
/opt/codedeploy-agent/vendor/gems/gli-2.5.6/lib/gli/command_support.rb:130:in `call'
/opt/codedeploy-agent/vendor/gems/gli-2.5.6/lib/gli/command_support.rb:130:in `execute'
/opt/codedeploy-agent/vendor/gems/gli-2.5.6/lib/gli/app_support.rb:262:in `block in call_command'
/opt/codedeploy-agent/vendor/gems/gli-2.5.6/lib/gli/app_support.rb:275:in `call'
/opt/codedeploy-agent/vendor/gems/gli-2.5.6/lib/gli/app_support.rb:275:in `call_command'
/opt/codedeploy-agent/vendor/gems/gli-2.5.6/lib/gli/app_support.rb:69:in `run'
/opt/codedeploy-agent/bin/codedeploy-agent:84:in `<main>'
2016-03-30 01:09:12 INFO  [codedeploy-agent(4677)]: Version file found in /opt/codedeploy-agent/.version.
2016-03-30 01:10:12 INFO  [codedeploy-agent(4677)]: [Aws::CodeDeployCommand::Client 200 60.252338 0 retries] poll_host_command(host_identifier:"arn:aws:ec2:us-east-1:XXX:instance/i-XXX")

Looking around, it seems to be a manifestation of this rubyzip bug which seems to be related to “zip64 support”.

I’m running the latest CodeDeploy agent from Amazon on Ubuntu 14.04.4.

Has anyone else seen this?

I’m also wondering if it might be possible to do the zipping myself instead of relying on circle to do it as a workaround for this problem. Is it possible to give circle the path to a .zip or .tar file instead of giving it a directory root in application_root?

2

Follow-up: I haven’t found a definite answer to this yet, but I think the EC2 instance I was deploying to had run out of disk space, which maybe caused this error to occur.

3

Actually I have resolved the disk space issue, but I still get the same error from CodeDeploy.

If I create the application revision on my own via aws deploy push, I am able to deploy the resulting revision from the AWS console, but application revisions that are created from CircleCI consistently give me the can't dup nilClass error on DownloadBundle.

I’ve also seen one other report of this issue on the AWS forums, with no resolution reported.