Configuring AWS EC2 Worker Nodes with Ansible Dynamic Inventory: A Step-by-Step Guide

Overview

In this guide, we'll walk through the process of configuring AWS EC2 worker nodes using Ansible. The objective is to set up a user (devuser) on the worker nodes with sudo access and an SSH key, without enabling password authentication. We'll use an existing .pem file to establish the initial connection and then configure devuser for future access.

Prerequisites

Ansible installed on your local machine or server.
AWS EC2 instances running in the specified region.
A .pem file for the initial connection to the EC2 instances.
An SSH key pair generated for devuser.
AWS Access Key ID and Secret Access Key with administrative access.

Dependencies

Before proceeding, ensure you have the following dependencies installed:

pip3: Install pip3 if it is not already installed.

 sudo apt-get update
 sudo apt-get install -y python3-pip

boto3: Install the Boto3 library for AWS SDK for Python.
```
 pip3 install boto3
```
Ansible AWS Collection: Install the Ansible AWS collection using ansible-galaxy.
```
 ansible-galaxy collection install amazon.aws
```
Upgrade Ansible: If you are using an older version of Ubuntu or other Linux distributions, you may need to upgrade Ansible to ensure it can read the galaxy-installed libraries.
```
 pip3 install --upgrade ansible
```
AWS CLI: Configure the AWS CLI with your access key and secret access key.
```
 pip3 install awscli
 aws configure
```

Directory Structure

Ensure the following files are in a designated directory, such as ~/aws_setup:

aws_ec2.yml: Inventory plugin configuration.
ansible.cfg: Ansible configuration file.
playbook.yaml: Ansible playbook to configure the EC2 instances.

File Contents

1. `aws_ec2.yml`

This file configures the AWS EC2 inventory plugin to target the desired region.

plugin: amazon.aws.aws_ec2
regions:
  - us-east-1

2. `ansible.cfg`

This file specifies the Ansible configuration, including the inventory file, SSH user, and key file. Adjust the remote_user according to the default user for your AMI (e.g., ec2-user, ubuntu, centos, etc.).

[defaults]
inventory = /home/username/aws_setup/aws_ec2.yml
enable_plugins = aws_ec2

remote_user = default-user
private_key_file = /home/username/aws_setup/my-key-pair.pem
host_key_checking = False

3. `playbook.yaml`

This playbook ensures devuser is configured on the EC2 instances with sudo access and the SSH key.

---
- name: Ensure devuser user is present on EC2 instances with sudo access and SSH key
  hosts: aws_ec2
  become: yes
  vars:
    user: "devuser"
    public_key: "{{ lookup('file', '/home/username/.ssh/id_rsa.pub') }}"

  tasks:
    - name: Ensure devuser user is present
      ansible.builtin.user:
        name: "{{ user }}"
        state: present
        shell: /bin/bash

    - name: Add devuser to sudoers
      ansible.builtin.lineinfile:
        path: /etc/sudoers
        state: present
        regexp: '^{{ user }} ALL='
        line: '{{ user }} ALL=(ALL) NOPASSWD:ALL'
        validate: '/usr/sbin/visudo -cf %s'

    - name: Create .ssh directory for devuser
      ansible.builtin.file:
        path: "/home/{{ user }}/.ssh"
        state: directory
        owner: "{{ user }}"
        group: "{{ user }}"
        mode: '0700'

    - name: Copy authorized keys
      ansible.builtin.copy:
        content: "{{ public_key }}"
        dest: "/home/{{ user }}/.ssh/authorized_keys"
        owner: "{{ user }}"
        group: "{{ user }}"
        mode: '0600'

Steps to Configure EC2 Worker Nodes

Set Up the Environment: Ensure that Ansible and the necessary plugins are installed on your local machine or server.
Install Dependencies: Follow the steps outlined in the Dependencies section to install pip3, Boto3, the Ansible AWS collection, upgrade Ansible if necessary, and configure AWS CLI.
Create the Required Files: Place the aws_ec2.yml, ansible.cfg, and playbook.yaml files in the ~/aws_setup directory.
Generate SSH Key Pair: Generate an SSH key pair for devuser and place the public key in /home/username/.ssh/id_rsa.pub.
Run the Playbook: Execute the following command to run the playbook and configure the EC2 instances:
```
 ansible-playbook /home/username/aws_setup/playbook.yaml
```
Move the PEM File: After successfully configuring the EC2 instances, move the .pem file to a secure location or delete it from the current directory.
```
 mv /home/username/aws_setup/my-key-pair.pem /home/username/.ssh/
```
Change Directory: Exit the current directory to avoid using the .pem file inadvertently.
```
 cd ~
```

Creating a New Project to test the configuration of the devuser with playbook

To create a new project with a separate folder for the playbook and configuration files, follow these steps:

Create Project Directory: Create a new directory for your project.
```
 mkdir ~/new_ansible_project
 cd ~/new_ansible_project
```
Create Configuration Files: Create new configuration files (ansible.cfg and aws_ec2.yml) in this directory.
```
 touch ansible.cfg aws_ec2.yml
```
Create a New Playbook: Create a new playbook file in the project directory.
```
 touch playbook.yaml
```

Edit Configuration Files: Edit the ansible.cfg and aws_ec2.yml files with the appropriate content.

ansible.cfg:

  [defaults]
  inventory = /home/devuser/new_ansible_project/aws_ec2.yml
  enable_plugins = aws_ec2

  remote_user = devuser
  private_key_file = /home/devuser/.ssh/id_rsa
  host_key_checking = False

aws_ec2.yml:

  plugin: amazon.aws.aws_ec2
  regions:
    - us-east-1

Edit the Playbook: Edit the playbook.yaml file with the appropriate content.

 ---
 - name: Verify devuser configuration on EC2 instances
   hosts: aws_ec2
   become: yes
   tasks:
     - name: Check hostname
       ansible.builtin.command:
         cmd: hostname

     - name: Check uptime
       ansible.builtin.command:
         cmd: uptime

     - name: Check current user
       ansible.builtin.command:
         cmd: whoami

     - name: Check sudo access
       ansible.builtin.command:
         cmd: sudo whoami

Run the Playbook from the New Project Directory: Execute the following command to run the playbook.
```
 ansible-playbook /home/username/new_ansible_project/playbook.yaml
```

Testing the Configuration with ad-hoc commands

After running the playbook, verify that the configuration was successful using Ansible ad-hoc commands:

Check Hostnames: Run the following command to get the hostname of each worker node.
```
 ansible aws_ec2 -m command -a "hostname"
```
Check Uptime: Run the following command to get the uptime of each worker node.
```
 ansible aws_ec2 -m command -a "uptime"
```
Check User: Run the following command to verify the current user on each worker node.
```
 ansible aws_ec2 -m command -a "whoami"
```
Check Sudo Access: Run the following command to verify sudo access for devuser.
```
 ansible aws_ec2 -m command -a "sudo whoami"
```
The output should be root.

Benefits

Automated Setup: Avoids manual enabling of password authentication and copying SSH keys.
Secure Access: Uses an existing .pem file for initial setup and configures devuser with an SSH key for secure access.
Consistency: Ensures all EC2 instances are configured uniformly with the necessary user and permissions.

This guide provides a streamlined approach to setting up devuser on AWS EC2 worker nodes using