CloudShovel: scanning public or private AMIs for sensitive files and secrets

CloudShovel

CloudShovel is a tool designed to search for sensitive information within public or private Amazon Machine Images (AMIs). It automates the process of launching instances from target AMIs, mounting their volumes, and scanning for potential secrets or sensitive data.

The tool is a modified version of what was used for the research AWS CloudQuarry: Digging for Secrets in Public AMIs.

How It Works

CloudShovel operates through the following steps:

1. Initialization:

   • Parses command-line arguments and creates an AWS session.
   • Validates the target AMI’s existence.

2. Setup:

   • Creates or verifies the existence of the specified S3 bucket.
   • Creates an IAM role and instance profile for the “secret searcher” EC2 instance.
   • Uploads necessary scripts to the S3 bucket.

3. Secret Searcher Instance:

   • Launches an EC2 instance (the “secret searcher”) based on the latest Amazon Linux 202* AMI.
   • Installs required tools on the secret searcher instance.

4. Target AMI Processing:

   • Launches an EC2 instance from the target AMI.
   • Stops the instance and detaches its volumes.
   • Attaches these volumes to the secret searcher instance.

5. Scanning:

   • Mounts the attached volumes on the secret searcher instance.
   • Executes the mount_and_dig.sh script to search for potential secrets.
   • The script looks for specific file names and patterns that might indicate sensitive information.

6. Results:

   • Uploads the scanning results to the specified S3 bucket.

7. Cleanup:

   • Detaches and deletes the volumes from the target AMI.
   • Terminates instances and removes created IAM resources.

Resources Created

CloudShovel creates the following AWS resources during its operation:

1. S3 Bucket: Stores scanning scripts and results.
2. IAM Role and Instance Profile: Named “minimal-ssm”, used by the secret searcher instance.
3. EC2 Instances:
   • A “secret searcher” instance based on Amazon Linux 2023.
   • A temporary instance launched from the target AMI (terminated after volume detachment).
4. EBS Volumes: Temporary attachments to the secret searcher instance (deleted after scanning).

Required Permissions

To run CloudShovel, your AWS account or IAM identity needs the following permissions:

• EC2:
  • Describe, run, stop, and terminate instances
  • Describe, create, attach, detach, and delete volumes
  • Describe and create tags
• IAM:
  • Create, delete, and manage roles and instance profiles
  • Attach and detach role policies
• S3:
  • Create buckets
  • Put, get, and delete objects
• SSM:
  • Send commands to EC2 instances
  • Get command invocation results

It’s recommended to use the principle of least privilege and create a specific IAM user or role for running CloudShovel with only the necessary permissions.

Install & Use