How To Deploy to AWS efficiently with IaC?

Deployments
1

Hello! I’m currently trying to optimize the build process for my company; however, I have a few kinks that I would want to solve. If anyone has any better suggestions that the way I’m planning on doing it, then please feel free!

Our current build process consists of IaC spread among 5 different repos:

  • aws-{service}-build-setup
  • aws-{service}-build-api
  • aws-{service}-build-services
  • aws-{service}-build-infrastructure
  • aws-{service}-build-images

We perform pull requests each time a build is changed. We have a branch for each separate environment: prod, dev, qa. Each time an environment is changed, we make sure to update its respective branch, from a pull request, to trigger the build process. If the developers made changes in their Dev branch in their applications, we may have to build another lambda, service, etc. in our build process. So, we create a new branch, such as dev-update, and merge the changes into the dev branch for whichever respective build repo. This works well, as it enforces only specific infrastructure to be built- instead of building all the infrastructure again.

I want to condense the build process into something like this:

Repo: aws-service-build (dev branch.. prod / qa will look similar)
/.circleci
    - config.yml
/setup
    - builder.json (Cloudformation template) - manual build
/api
    - aws_deploy.sh
    - api.json
/services
    - aws_deploy.sh
    - services.json
/infrastructure
    - aws_deploy.sh
    - infrastructure.json
/images
    - aws_deploy.sh (Only needs to perform an api call)

Then, I want to have a config.yml file that contains the specific build jobs for whichever build. So:

version: 2
jobs:
  build:
    - run: |
        # How to check if a specific build was changed in the repo?
        # I don't want Circle to run /services if the latest pull request
        # didn't make modifications to it.
        # Or, should I run just use an API Trigger? And manually call?

  buildApi:
    - run: ./api/aws_deploy.sh
 
  buildServices:
    - run: ./services/aws_deploy.sh

As stated in the comments though, how could I use Circle to potentially know what was changed in the repo to only build the specific IaC that was modified? I was thinking a combination of using an env variable provided by Circle and Github’s api to view what was modified-- but I’m not even sure of how that would look.

Any ideas would be greatly appreciated-- sorry for the long question!

2

I guess from the lack of responses, I may not be doing this the best way? :man_shrugging:

For anyone who read this, I do appreciate the time!

3

It’s a little out of my depth, so I was hoping someone else would tackle it. Maybe we have less AWS folks here on the forum then I thought :slight_smile:

If I understand the question, maybe you you can try using git commands to check if the files you need have been modified? I think folks have used git in fancy related ways elsewhere on the forum.

4

Hey @drazisil, I appreciate the timely response! Yeah, that’s more of less what I’m asking. I wanted to know if anyone has used git in such a fancy way to where they could see what was modified and only build that specific job within the config.yml.

repo/services/aws_deploy.sh needs to be triggered only when a change to infrastructure has occured-- such as adding a new lambda to the json template in the services/services.json; however, if that is too challenging to accomplish, then I could definitely just trigger the job via an API call.

Regardless, thanks for your help thus far!

5

Things like How to detect commited file at test phase are what I was thinking of, will some checking of that help?

1 Like
6

Actually, you may have single-handedly answered my question with one link. You’re a forum god. Haha, thanks @drazisil! That’s definitely what I’m looking for! I’m going to accept that as the answer!

1 Like
7

This topic was automatically closed 10 days after the last reply. New replies are no longer allowed.