Talking about DevOps, few things must always come to mind, one of which is Infrastructure as Code. Of course, this methodology is based on breaking barriers between Developers and Operations, so a DevOps Engineer needs to develop skills in both directions. Nowadays, there are many different ways to use programming code to create, change and maintain an infrastructure. Here you can check Infrastructure as Code fundamentals.
There is one orchestrating tool, currently gaining popularity among our circles and that is Pulumi. In this blog we will cover how to create your infrastructure with Pulumi and also compare it to the well known orchestrator – Terraform.
What is Pulumi?
Pulumi is an Open Source infrastructure as code tool that allows you to create, deploy, and manage infrastructure using programming languages. Unlike traditional IaC tools, Pulumi treats infrastructure as software.
Pulumi empowers developers and operators to build, deploy, and manage modern cloud applications and infrastructure using familiar languages like TypeScript, Python, Go etc. Pulumi supports multi-cloud setups, enabling users to manage infrastructure on various cloud providers like AWS, Azure, Google Cloud, and more, using a single codebase. This flexibility not only simplifies the management process but also enhances collaboration between development and operations teams.
Pulumi vs Terraform
Where Orchestration is needed, the first option in our heads is Terraform of course. The HashiCorp Open Source software is probably the most popular approach of creating infrastructure in a declarative way, using code. So are there any advantages of using Pulumi instead of sticking with Terraform. Yes indeed. Here you can check all the differences between Pulumi and Terraform Pulumi creators point out, but here are our main and objective picks:
Even though declarative programming is preferred in infrastructure as code, we have to point out that Pulumi can be used to define resources imperatively, whereas in Terraform we have primarily declarative style.
Below is a table that defines the most important differences you should know between these two orchestrators.
Pulumi operates based on a client-server architecture:
Pulumi CLI (Client) is used to interact with Pulumi, where Pulumi Engine (Server) is responsible for coordinating and managing the deployment process. It interacts with cloud providers’ APIs to create and manage resources in the cloud. It takes the Pulumi program, translates it into the appropriate cloud API calls, and ensures the desired state of the infrastructure is achieved.
When you run a Pulumi command to deploy your infrastructure, the CLI sends your program and configuration to the Engine, which then executes the necessary cloud operations to create and manage the specified resources.
Pulumi is security compliant and has PCI, ISO 27001, HIPAA standards.
In Pulumi, stacks are a fundamental concept used to manage different deployments of your infrastructure within the same Pulumi project. A stack represents a distinct instance of your infrastructure, allowing you to have separate deployments for different environments (such as development, staging, and production) or different configurations within the same environment.
Creating infrastructure with pulumi
As you can probably guess, there are two things before starting our infrastructure journey with Pulumi:
1. Install Pulumi: Make sure you have Pulumi installed. You can download and install it from the official Pulumi website and of course based on your Operating System.
2. Cloud Provider Account: You’ll need an account with a cloud provider (like AWS, Azure, or GCP) if you plan to deploy resources on the cloud.
Now we can initialize a Pulumi project. Our suggestion is to use IDE for this purpose and to open an empty folder or create one. Then we simply use:
$ pulumi new
If no other options are provided to this command you will be prompted to a menu to choose a template from:
This command will walk you through creating a new Pulumi project. Enter a value or leave blank to accept the (default), and press <ENTER>. Press ^C at any time to quit. project name: (Testing) project description: (A minimal AWS TypeScript Pulumi program) Created project 'Testing'
Here we are providing a project name for the Pulumi project. After that we are prompted to create our first stack in the project. As we mentioned stacks are logical division in Pulumi mostly used for different environments code within the same project, but it can also be used for defining different parts of the same environment infrastructure like Networking, Data, Compute, Backup etc.
Please enter your desired stack name. To create a stack in an organization, use the format <org-name>/<stack-name> (e.g. `acmecorp/dev`). stack name: (dev) Created stack 'dev'
After picking a name for our first stack, Pulumi usually asks where, in which region, we want to create resources, so we pick the default option again for our purposes:
aws:region: The AWS region to deploy into: (us-east-1) Saved config
Pulumi will download needed dependencies for the selected options and tadaa, we have successfully initialized a new project:
Finished installing dependencies Your new project is ready to go! ✨ To perform an initial deployment, run `pulumi up
Notice how, Pulumi created a bunch of files based on our options and preferences:
Let’s have a closer look at these files. We have Pulumi.yaml, representing our options about the project name and description.
Pulumi.dev.yaml shows the stack configuration on which region we will create resources. If you have any experience with AWS or any Cloud in general, you’ll know that defining only the region is not enough to authenticate Pulumi to create resources to your AWS Account. Yes you can add your credentials to this file, but if you want to do it the secure way, that’s not recommended. When you create resources using Pulumi without explicitly providing AWS credentials, Pulumi uses the default AWS credentials chain to authenticate your requests. The AWS credentials chain typically works in the following order:
Environment Variables: If you have set the AWS_ACCESS_KEY_ID and AWS_SECRET_ACCESS_KEY environment variables, Pulumi will use these credentials.
AWS CLI Configuration: If you have configured the AWS CLI with aws configure, Pulumi will use the credentials stored in the AWS CLI configuration files.
Instance Profile Credentials: If your code is running on an AWS resource with an instance profile (IAM role) attached, Pulumi will use the credentials associated with that role.
So it’s better to leave this file as it is:
And let’s also have a look on the index.ts file:
That’s the default template when a project is based on TypeScript for aws, if you choose another template the file will be based on it. Here we have code for creating S3 bucket in AWS with aws package module. There are 3 modules imported: @pulumi/pulumi, @pulumi/aws and @pulumi/awsx.
@pulumi/awsx is an additional library built on top of @pulumi/aws. It provides higher-level abstractions and patterns for working with AWS resources.
Talking about the index.ts file, or whatever the file is based on language preference, now it’s up to you what resources you want to create and where.
For our example we’ll keep the default file.
Now we can use pulumi up to implement our changes into the Cloud Provider:
$ pulumi up Previewing update (dev) View in Browser (Ctrl+O): https://app.pulumi.com/... Type Name Plan + pulumi:pulumi:Stack Testing-dev create + └─ aws:s3:Bucket my-bucket create Outputs: bucketName: output<string> Resources: + 2 to create Do you want to perform this update? details + pulumi:pulumi:Stack: (create) [urn=urn:pulumi:dev::Testing::pulumi:pulumi:Stack::Testing-dev] + aws:s3/bucket:Bucket: (create) [urn=urn:pulumi:dev::Testing::aws:s3/bucket:Bucket::my-bucket] [provider=urn:pulumi:dev::Testing::pulumi:providers:aws::default_5_42_0::04da6b54-80e4-46f7-96ec-b56ff0331ba9] acl : "private" bucket : "my-bucket-57d6bca" forceDestroy: false --outputs:-- bucketName: output<string> Do you want to perform this update? yes Updating (dev) View in Browser (Ctrl+O): https://app.pulumi.com/... Type Name Status + pulumi:pulumi:Stack Testing-dev created (8s) + └─ aws:s3:Bucket my-bucket created (3s) Outputs: bucketName: "my-bucket-876beba" Resources: + 2 created Duration: 10s
Pulumi shows us an output of what resources will be created, destroyed or updated and when we type yes Pulumi starts creating those resources for us.
Now if we don’t want to keep this Bucket, simply destroy it:
$ pulumi destroy Previewing destroy (dev) View in Browser (Ctrl+O): https://app.pulumi.com/... Type Name Plan - pulumi:pulumi:Stack Testing-dev delete - └─ aws:s3:Bucket my-bucket delete Outputs: - bucketName: "my-bucket-876beba" Resources: - 2 to delete Do you want to perform this destroy? yes Destroying (dev) View in Browser (Ctrl+O): https://app.pulumi.com/... Type Name Status - pulumi:pulumi:Stack Testing-dev deleted - └─ aws:s3:Bucket my-bucket deleted (1s) Outputs: - bucketName: "my-bucket-876beba" Resources: - 2 deleted Duration: 5s The resources in the stack have been deleted, but the history and configuration associated with the stack are still maintained. If you want to remove the stack completely, run `pulumi stack rm dev`.
Also there is a pulumi preview command which only shows us the changes that apply will make.
$ pulumi preview Previewing update (dev) View in Browser (Ctrl+O): https://app.pulumi.com/... Type Name Plan + pulumi:pulumi:Stack Testing-dev create + └─ aws:s3:Bucket my-bucket create Outputs: bucketName: output<string> Resources: + 2 to create
You can manage pulumi stacks, like creating a new one:
$ pulumi stack init prod Created stack 'prod'
list available stacks:
$ pulumi stack ls NAME LAST UPDATE RESOURCE COUNT URL dev 5 minutes ago 0 https://app.pulumi.com/.../Testing/dev prod* n/a n/a https://app.pulumi.com/.../Testing/prod
$ pulumi stack select prod
$ pulumi stack rm dev This will permanently remove the 'dev' stack! Please confirm that this is what you'd like to do by typing `dev`: dev Stack 'dev' has been removed!
and other commands for stacks and more functionalities. You can find them all in the official documentation.
Infrastructure as Code with Pulumi is more than just a technological evolution, it’s a shift in mindset, empowering organizations to achieve greater agility, reliability, and efficiency in their infrastructure management processes.
By enabling DevOps engineers to leverage familiar programming languages, abstracting complexities, and fostering collaboration, Pulumi has emerged as a game-changer in the realm of IaC. So, as businesses continue their digital transformation journeys, considering Pulumi as the tool of choice for Infrastructure as Code can be a decision that propels them toward a more scalable, flexible, and sustainable future.