We were contacted by the financial branch of a German automotive company. The client was looking to set up a multi-environment AWS infrastructure that would facilitate proxying financial transactions from their existing merchant applications towards external third-party payment gateway providers. As a requirement to process financial transactions the Production systems needed to be compliant with the Payment Card Industry Data Security Standard (PCI DSS).
ITGix has a managed services and project services organization to support the desired functionality. We have extensive expertise to design and implement the desired infrastructure. Our team has an extensive experience with projects with complex security and compliance requirements for the financial, fintech, and banking sectors, as well as for digital payment software companies. We focus on automation, infrastructure as code, and designing AWS architectures with the latest security and operational best practices for the financial industry.
Audit and Prerequisites
An initial audit and discovery phase was performed mainly to scope the size of the project, as well as the infrastructure and software requirements. As a result, an architecture design was produced in cooperation with the software development providers. It is comprised of 4 environments – 3 non-production and 1 production CDE (Cardholder Data Environment). The production environment is the one that is PCI DSS certified.
The environment needed to be highly available, scalable, and secured. Moreover, it had to fulfill all security and compliance requirements for the PCI DSS certification.
The project delivery deadlines were also strict as a PCI certification audit was already scheduled for 3 months after the project discovery and design phases started – with the goal of having the production system being completely set up and ready for the certification, penetration testing, as well as application scalability and reliability tests.
Our main goal was to fully automate all components of both the infrastructure and application’s build and deploy lifecycle while considering all the security and compliance requirements and factoring them into the automation to remove the need for any manual configuration.
The AWS infrastructure comprised a set of 9 AWS accounts – 1 common service account where shared repositories were stored; 4 application accounts (1 of which was the production PCI DSS certified environment with actual credit card data); 4 logging accounts – used mainly to aggregate logs from the application accounts into a centralized space.
The access to all AWS accounts is centralized with a SAML integration to a Keycloak Identity Provider. Access to the production system is ephemeral (usually for the duration of the maintenance or deployment that is being performed) and granted after various approvals. Additional security controls are also implemented to limit access to the production system only from PCI DSS compliant laptops and approved networks.
At a later stage of the project, Gitlab was replaced by Azure DevOps repositories and CI/CD pipelines. This step consolidates all automation for the client under their preferred CI/CD tool.
AWS Production Environment
In the diagram below is an overview of the PCI DSS compliant production environment connecting merchant applications with external payment gateways containing actual credit card data in an encrypted format:
This setup has evolved in the lifecycle of the project and mainly after the PCI DSS certification. The most major change that was introduced was that the AWS Network firewall was added to handle blocking all outbound traffic from the infrastructure and whitelisting certain domains for HTTP/TLS traffic.
CI/CD Automation Pipelines
All infrastructure components were developed with Infrastructure as code best practices using Terraform and CI/CD pipelines.
The application components were deployed in Kubernetes using the AWS EKS managed service. Their build and deploy lifecycle was handled using Docker images and Helm charts inside the CI/CD pipelines.
The CI/CD pipelines create self-contained, versioned, Docker images of the infrastructure and application code during the CI process and deploy using those images on each environment. Two of the non-production environments have a Gitlab/Azure DevOps runner in them. Thus all the CI pipelines inside the AWS internal network can be executed. The CD pipelines are configured to deploy using the pre-built images. This is achieved by using AWS-managed CI/CD tools Codebuild and Codepipeline mainly due to PCI DSS compliance requirements that cannot be fulfilled with external tools.
In the diagram below you can see an overview of the log aggregation setup in each AWS account. In summary, all logs are collected via different tools into Cloudwatch log groups. Afterward, logs are restructured by a custom Lambda and sent into an Elasticsearch instance in the other AWS account. That is where logs are aggregated, indexed, and visualized in Kibana.
In addition to the Logging setup, there is also a lot of automation revolving around the monitoring. It factored into all of the Infrastructure automation Terraform modules, Kubernetes helm charts, etc. Each component has its own alerts for AWS managed services, URL monitoring, application monitoring, etc. The main tools used for monitoring are Cloudwatch, Synthetic Canary checks, Prometheus with Alertmanager, and Grafana.
Despite the strict deadlines and complex security and compliance requirements, we were able to complete the project within the requested time frame. Moreover, we have successfully acquired the PCI DSS certification for the client’s infrastructure. The expectations were achieved without having to make any compromise with the infrastructure’s end-to-end automation. We engineered well-architected review best practices, and have also successfully taken on the project for ongoing support.