Configuring AWS EC2 Worker Nodes with Ansible Dynamic Inventory: A Step-by-Step Guide

Overview

In this guide, we'll walk through the process of configuring AWS EC2 worker nodes using Ansible. The objective is to set up a user (devuser) on the worker nodes with sudo access and an SSH key, without enabling password authentication. We'll use an existing .pem file to establish the initial connection and then configure devuser for future access.

Prerequisites

1. Ansible installed on your local machine or server.

2. AWS EC2 instances running in the specified region.

3. A .pem file for the initial connection to the EC2 instances.

4. An SSH key pair generated for devuser.

5. AWS Access Key ID and Secret Access Key with administrative access.

Dependencies

Before proceeding, ensure you have the following dependencies installed:

1. pip3: Install pip3 if it is not already installed.

    sudo apt-get update
    sudo apt-get install -y python3-pip
   

2. boto3: Install the Boto3 library for AWS SDK for Python.

    pip3 install boto3
   

3. Ansible AWS Collection: Install the Ansible AWS collection using ansible-galaxy.

    ansible-galaxy collection install amazon.aws
   

4. Upgrade Ansible: If you are using an older version of Ubuntu or other Linux distributions, you may need to upgrade Ansible to ensure it can read the galaxy-installed libraries.

    pip3 install --upgrade ansible
   

5. AWS CLI: Configure the AWS CLI with your access key and secret access key.

    pip3 install awscli
    aws configure
   

Directory Structure

Ensure the following files are in a designated directory, such as ~/aws_setup:

• aws_ec2.yml: Inventory plugin configuration.

• ansible.cfg: Ansible configuration file.

• playbook.yaml: Ansible playbook to configure the EC2 instances.

File Contents

1. aws_ec2.yml

This file configures the AWS EC2 inventory plugin to target the desired region.

plugin: amazon.aws.aws_ec2
regions:
  - us-east-1

2. ansible.cfg

This file specifies the Ansible configuration, including the inventory file, SSH user, and key file. Adjust the remote_user according to the default user for your AMI (e.g., ec2-user, ubuntu, centos, etc.).

[defaults]
inventory = /home/username/aws_setup/aws_ec2.yml
enable_plugins = aws_ec2

remote_user = default-user
private_key_file = /home/username/aws_setup/my-key-pair.pem
host_key_checking = False

3. playbook.yaml

This playbook ensures devuser is configured on the EC2 instances with sudo access and the SSH key.

---
- name: Ensure devuser user is present on EC2 instances with sudo access and SSH key
  hosts: aws_ec2
  become: yes
  vars:
    user: "devuser"
    public_key: "{{ lookup('file', '/home/username/.ssh/id_rsa.pub') }}"

  tasks:
    - name: Ensure devuser user is present
      ansible.builtin.user:
        name: "{{ user }}"
        state: present
        shell: /bin/bash

    - name: Add devuser to sudoers
      ansible.builtin.lineinfile:
        path: /etc/sudoers
        state: present
        regexp: '^{{ user }} ALL='
        line: '{{ user }} ALL=(ALL) NOPASSWD:ALL'
        validate: '/usr/sbin/visudo -cf %s'

    - name: Create .ssh directory for devuser
      ansible.builtin.file:
        path: "/home/{{ user }}/.ssh"
        state: directory
        owner: "{{ user }}"
        group: "{{ user }}"
        mode: '0700'

    - name: Copy authorized keys
      ansible.builtin.copy:
        content: "{{ public_key }}"
        dest: "/home/{{ user }}/.ssh/authorized_keys"
        owner: "{{ user }}"
        group: "{{ user }}"
        mode: '0600'

Steps to Configure EC2 Worker Nodes

1. Set Up the Environment: Ensure that Ansible and the necessary plugins are installed on your local machine or server.

2. Install Dependencies: Follow the steps outlined in the Dependencies section to install pip3, Boto3, the Ansible AWS collection, upgrade Ansible if necessary, and configure AWS CLI.

3. Create the Required Files: Place the aws_ec2.yml, ansible.cfg, and playbook.yaml files in the ~/aws_setup directory.

4. Generate SSH Key Pair: Generate an SSH key pair for devuser and place the public key in /home/username/.ssh/id_rsa.pub.

5. Run the Playbook: Execute the following command to run the playbook and configure the EC2 instances:

    ansible-playbook /home/username/aws_setup/playbook.yaml
   

6. Move the PEM File: After successfully configuring the EC2 instances, move the .pem file to a secure location or delete it from the current directory.

    mv /home/username/aws_setup/my-key-pair.pem /home/username/.ssh/
   

7. Change Directory: Exit the current directory to avoid using the .pem file inadvertently.

    cd ~
   

Creating a New Project to test the configuration of the devuser with playbook

To create a new project with a separate folder for the playbook and configuration files, follow these steps:

1. Create Project Directory: Create a new directory for your project.

    mkdir ~/new_ansible_project
    cd ~/new_ansible_project
   

2. Create Configuration Files: Create new configuration files (ansible.cfg and aws_ec2.yml) in this directory.

    touch ansible.cfg aws_ec2.yml
   

3. Create a New Playbook: Create a new playbook file in the project directory.

    touch playbook.yaml
   

4. Edit Configuration Files: Edit the ansible.cfg and aws_ec2.yml files with the appropriate content.

   • ansible.cfg:

       [defaults]
       inventory = /home/devuser/new_ansible_project/aws_ec2.yml
       enable_plugins = aws_ec2
     
       remote_user = devuser
       private_key_file = /home/devuser/.ssh/id_rsa
       host_key_checking = False
     

   • aws_ec2.yml:

       plugin: amazon.aws.aws_ec2
       regions:
         - us-east-1
     

5. Edit the Playbook: Edit the playbook.yaml file with the appropriate content.

    ---
    - name: Verify devuser configuration on EC2 instances
      hosts: aws_ec2
      become: yes
      tasks:
        - name: Check hostname
          ansible.builtin.command:
            cmd: hostname
   
        - name: Check uptime
          ansible.builtin.command:
            cmd: uptime
   
        - name: Check current user
          ansible.builtin.command:
            cmd: whoami
   
        - name: Check sudo access
          ansible.builtin.command:
            cmd: sudo whoami
   

6. Run the Playbook from the New Project Directory: Execute the following command to run the playbook.

    ansible-playbook /home/username/new_ansible_project/playbook.yaml
   

Testing the Configuration with ad-hoc commands

After running the playbook, verify that the configuration was successful using Ansible ad-hoc commands:

1. Check Hostnames: Run the following command to get the hostname of each worker node.

    ansible aws_ec2 -m command -a "hostname"
   

2. Check Uptime: Run the following command to get the uptime of each worker node.

    ansible aws_ec2 -m command -a "uptime"
   

3. Check User: Run the following command to verify the current user on each worker node.

    ansible aws_ec2 -m command -a "whoami"
   

4. Check Sudo Access: Run the following command to verify sudo access for devuser.

    ansible aws_ec2 -m command -a "sudo whoami"
   

   The output should be root.

Benefits

• Automated Setup: Avoids manual enabling of password authentication and copying SSH keys.

• Secure Access: Uses an existing .pem file for initial setup and configures devuser with an SSH key for secure access.

• Consistency: Ensures all EC2 instances are configured uniformly with the necessary user and permissions.

This guide provides a streamlined approach to setting up devuser on AWS EC2 worker nodes using