Ensuring AWS Infrastructure Consistency with Ansible Playbooks

Introduction

Managing cloud infrastructure, especially in AWS, requires regular testing to ensure it remains consistent with your desired state. Changes and misconfigurations can lead to unforeseen issues, impacting performance, security, and overall reliability. Ansible, an open-source automation tool, provides a straightforward way to automate such tests and verify the AWS infrastructure’s integrity. In this blog, we will explore how to leverage Ansible playbooks to test your AWS infrastructure, focusing on EC2 instances and RDS instances. We will use variable files to define the expected state and demonstrate how to compare these expectations against the actual AWS environment.

What is Ansible?

Ansible is a powerful automation tool that allows you to define and manage complex infrastructure configurations through simple, human-readable YAML files called “playbooks.” It enables developers and system administrators to automate repetitive tasks, maintain consistency, and ensure infrastructure compliance across various cloud platforms, including AWS.

Prerequisites

Before diving into the playbook, make sure you have Ansible installed on your system:

# Install Ansible using pip
pip install ansible

Additionally, ensure you have AWS credentials configured to interact with your AWS environment effectively.

Creating the Ansible Playbook

We will create a playbook named aws_infrastructure_test.yml to test the AWS infrastructure. This playbook will focus on two main AWS services: EC2 instances and RDS instances.

Step 1: Define EC2 Instance Variables

Create a variable file named ec2_vars.yml to define the expected state for EC2 instances:

# ec2_vars.yml

ec2_instances:
  - name: "WebServerInstance1"
    count: 2
    security_groups:
      - "WebServerSecurityGroup"
  - name: "AppServerInstance1"
    count: 1
    security_groups:
      - "AppServerSecurityGroup"

Step 2: Define RDS Instance Variables

Create another variable file named rds_vars.yml to define the expected state for RDS instances:

# rds_vars.yml

rds_instances:
  - name: "MyDatabaseInstance1"
    count: 1
    security_groups:
      - "DatabaseSecurityGroup"

Step 3: The Complete Ansible Playbook

Now, let’s update the aws_infrastructure_test.yml playbook to include the variable files and check both EC2 and RDS instances:

# aws_infrastructure_test.yml

- hosts: localhost
  gather_facts: no
  vars:
    aws_region: "your_aws_region_here"
    ec2_vars_file: "ec2_vars.yml"
    rds_vars_file: "rds_vars.yml"
  tasks:
    - name: Include EC2 instance variables
      include_vars:
        file: "{{ ec2_vars_file }}"
      register: expected_ec2_state
    - name: Gather EC2 instance facts
      ec2_instance_facts:
        region: "{{ aws_region }}"
      register: actual_ec2_state
    - name: Check if EC2 instances are unchanged
      assert:
        that:
          - expected_ec2_state.vars.ec2_instances | length == actual_ec2_state.instances | length
          - expected_ec2_state.vars.ec2_instances[0].name == actual_ec2_state.instances[0].tags[0].Value
          - expected_ec2_state.vars.ec2_instances[0].count == actual_ec2_state.instances | length
          - expected_ec2_state.vars.ec2_instances[0].security_groups == actual_ec2_state.instances[0].security_groups | map(attribute='GroupName') | list
        fail_msg: "EC2 instances have changed!"
    - name: Include RDS instance variables
      include_vars:
        file: "{{ rds_vars_file }}"
      register: expected_rds_state
    - name: Gather RDS instance facts
      rds_instance_facts:
        region: "{{ aws_region }}"
      register: actual_rds_state
    - name: Check if RDS instances are unchanged
      assert:
        that:
          - expected_rds_state.vars.rds_instances | length == actual_rds_state.instances | length
          - expected_rds_state.vars.rds_instances[0].name == actual_rds_state.instances[0].db_instance_identifier
          - expected_rds_state.vars.rds_instances[0].count == actual_rds_state.instances | length
          - expected_rds_state.vars.rds_instances[0].security_groups == actual_rds_state.instances[0].vpc_security_groups | map(attribute='vpc_security_group_id') | list
        fail_msg: "RDS instances have changed!"

Let’s break down the Ansible playbook and explain how it works. After that, I’ll provide you with a command to execute the playbook.

Explanation of the Ansible Playbook:

1. The playbook starts by defining the target hosts on which it will run. In this case, we set localhost as the target host since we are only gathering information about the AWS environment on the local machine.
2. gather_facts: no disables the default behavior of Ansible to gather facts about the local system. We don't need local facts for this playbook, as it's designed to interact with AWS resources.
3. The vars section defines variables used throughout the playbook. We specify the aws_region, ec2_vars_file, and rds_vars_file. The aws_region variable should be replaced with your desired AWS region, and the ec2_vars_file and rds_vars_file should point to the respective variable files (ec2_vars.yml and rds_vars.yml).
4. The playbook uses the include_vars module to load the EC2 and RDS instance variables from the respective variable files (ec2_vars.yml and rds_vars.yml). The variables are stored in expected_ec2_state and expected_rds_state, respectively.
5. The playbook gathers facts about the EC2 instances and RDS instances in the specified AWS region using the ec2_instance_facts and rds_instance_facts modules, respectively. The gathered facts are stored in actual_ec2_state and actual_rds_state.
6. The assert module is used to compare the expected state (from the variable files) with the actual AWS state for both EC2 and RDS instances. Each field (name, count, and security groups) is compared separately.
7. If any of the comparisons in the assert module fail, the playbook will stop execution and display the specified error message.

Command to Execute the Playbook:

To execute the Ansible playbook, use the ansible-playbook command followed by the name of the playbook file:

ansible-playbook aws_infrastructure_test.yml

Make sure you are in the directory where the playbook file (aws_infrastructure_test.yml) and the variable files (ec2_vars.yml and rds_vars.yml) are located. The playbook will execute and display the output in the terminal, indicating whether the AWS infrastructure meets the expected state or if there are any discrepancies.

Remember to replace "your_aws_region_here" in the playbook with your desired AWS region before executing the playbook. Additionally, ensure that you have valid AWS credentials set up on your system to interact with your AWS environment.

Conclusion

Ansible playbooks provide a robust and efficient way to test and validate the consistency of your AWS infrastructure. By defining the expected state in variable files and comparing them with the actual AWS environment, you can easily detect any unintended changes or misconfigurations.

Remember to adapt the playbook based on your specific AWS environment’s needs, adding more resources or tests as your infrastructure evolves. Always practice caution when applying changes to your production environment, and test the playbooks in a controlled testing environment before using them in production.

With Ansible, you can rest assured that your AWS infrastructure remains stable, secure, and aligned with your desired state, ensuring a seamless experience for your users and customers. Happy testing!

Thank you for reading my blog! If you enjoyed the content and want to receive timely updates whenever I publish a new article, make sure to follow me on Medium. It’s quick and easy! Just log in to your Medium account (or create one if you haven’t already), visit my profile, and click the “Follow” button. Stay informed and never miss a new post! Your support means a lot to me, and I’m excited to continue sharing valuable insights with you. Happy reading! 🚀