Infrastructure as Code (IaC) is using code to manage cloud resources to be a better engineer in every sense, from having a simple way to understand the architecture of your cloud to deploying new resources without any mistakes.
The Symptoms
A couple of months ago, I had the mission to fix the project where we used to manage our IaC project but I quickly noticed that I had a very bad time reading or understanding the code because many weak spots I notice in my own and other projects that I've seen:
- Code hurst because usually is overengineered
- Code hurst because of a bad implementation
- Code hurst because we don't fully understand the architecture and therefore we have a black box that no one understands fully how it works
After a few attempts, I think this is the cleanest way that I came up with. I'll try to explain here the principles that I followed to create it (I must say that this is not pretending to be a perfect approach):
The Principles
Zen Mode
Fully understand the architecture of your cloud. Do you use Load Balancers? How many do you have? Which regions are you on?
For having a better understanding you can use Cloudcaraft (non-sponsored) or Seqloud (my own invention). That will allow you to generate diagrams reflecting the actual state of your infrastructure.
From the bottom up
Start with the core: Networking is a good place to start. I'll recommend starting from adding:
- VPC
- Subnets
- Route Tables
- Peering connections (if apply)
- Security Groups
- Load Balancers
- Target Groups
Doing this from the beginning will pay off since the rest of your resources depend on this, for example if you would like to import your EC2 instances you'll need references to at least: VPC, Subnets and Security Groups, and since you already have the references... voilá, should be easier to manage.
After this stage you could start adding your EC2, ECS, EKS or Databases pointing of course to the previous resources. I heavily recommend to leverage pulumi exports, since you can reference stack resources between different projects and you can go even further and create typed references.
A Simple Shared State
Again, this lesson I had it using pulumi that is a wrapper for terraform, you could simply translate this to terraform without so much pain (cannot guarantee).
Why a shared state? Basically I would recommend to approach your project dividing each major service into different projects in a single repository, let me explain with the following project structure:
|
|----- README.md
|----- types
|----- modules
|--------- vpcs
|--------- subnets
|--------- route-tables
|--------- load-balancers
|--------- target-groups
|--------- ec2
Having each divided into multiple projects allows to have easier stack updates, or syncing the state in case that it gets outdated. Also you could have different stacks pointing to different resource stages like: production, staging, development.
Pretty much there is no limit about you could import from the cloud to the pulumi state.