Dig Deep

Amazon Web Services (AWS) regularly releases brand new services promising an abundance of intriguing features to help make life easier when provisioning your Internet infrastructure. The sheer number of services visible within the AWS Console is simply staggering, which is why AWS is still the dominant cloud provider – admittedly with Google Cloud and Microsoft Azure hot on its heels.

In this article, I demonstrate a tool that is available from a highly respected security specialist, the NCC Group [1], that automatically and safely informs you of misconfigurations and gaping holes in your AWS security posture.

NCC's website describes their focus as "… a global expert in cyber security and risk mitigation, working with businesses to protect their brand, value and reputation against the ever-evolving threat landscape."

A Ballet of Swans

The long journey AWS has taken since around 2006 is of constant innovation. And to think, much of this innovation began with virtual machines (the Xen hypervisor) as their on-demand EC2 (Amazon Elastic Compute Cloud) service combined with their super-reliable S3 storage service (Amazon Simple Storage) to fulfill the meteoric growth of their online retail shop, Amazon.com. The whole being greater than its parts, the collective power of all these mature services makes AWS the cloud behemoth it is today.

As financial institutions, governments, and corporations all over the planet make significant leaps into the cloud infrastructure, AWS innovation continues at a rapid pace. Importantly, the unbridled rate of innovation in the platform-as-a-service (PaaS) and infrastructure-as-a-service (IaaS) aspects of AWS provide a welcome level of maturity that some software-as-a-service (SaaS) provisioning lacks. When it comes to infrastructure, any changes – large or small – can mean instability, and therefore costly downtime, calling for significant testing.

From a United Kingdom perspective alone, 2017 was reportedly another boom year for cloud adoption. According to the Cloud Industry Forum [2]:

… the overall cloud adoption rate in the UK now stands at 88 per cent, with 67 per cent of users expecting to increase their adoption of cloud services over the coming year. However, while organisations are clearly taking a cloud-first approach, the industry body predicts that the vast majority of companies will be maintaining hybrid IT estates for some time to come.

With the incomparable adoption of the cloud infrastructure, integrating on-premises computer rooms or external data center infrastructure has not been easy for some organizations. Suspicion of sharing infrastructure with other organizations still prevails and, from a security perspective, raises concerns of new attack vectors that were previously less important, invisible, or not present at all.

Out of this uncertainty steps a sophisticated security auditing tool, the NCC Group's Scout2, which you might be pleasantly surprised to learn is open source and readily available on the NCC GitHub page [3].

A Cloud of Grasshoppers

I use Scout2 professionally and find that it executes and completes its scans fast and is highly informative and easy to set up. Although you will want to test Scout2 in an AWS sandbox first, over time and with some exposure, you might want to run it only on your preproduction or staging environments (e.g., before trusting it in your production environment).

Whether you prefer to set up NCC's sophisticated scanning tool to run automatically hourly or daily (or even more frequently) or to run it only periodically using a manual method, as in more traditional penetration testing (e.g., one-off audits), the rapid execution of Scout2 means that it adapts perfectly.

The act of inspecting the vast array of AWS configurations using Identity Access Management (IAM) Read/List rules is surprisingly speedy, and once it's completed, a useful HTML summary page points out where issues have been spotted.

A Wreck of Seabirds

Of course, you shouldn't take risks with your potentially complex AWS configuration, so getting the IAM rules correct is the key to getting Scout2 running correctly. You have been warned!

For the example in this article, I will trust (with good reason, I hope you'll agree when the evidence becomes clear) the default IAM policy [4] for Scout2 on the NCC GitHub page. A quick look reveals the number of IAM rules required by Scout2. However, before absolutely committing to using it, you should inspect the human-readable rules the policy creates, which you'll see shortly.

A Murder of Crows

From the size of the policy – roughly 85 actions – you can see why it's helpful first to try an example policy provided by NCC. The services tend to range from describing (e.g., ec2:DescribeSecurityGroups) to listing (e.g., iam:ListAccessKeys) to reading service data (e.g., s3:GetBucketLocation).

To add this custom policy to an AWS user (the user that will need the access and secret keys later on), log in to the AWS console and visit https://console.aws.amazon.com/iam. In the left-hand menu, click the Policies link to open that page (Figure 1). Next on the Policies page, click Create Policy near the top of the screen and then the JSON tab (Figure 2). Make sure you delete the JSON already present before pasting the NCC policy from GitHub; you can save it with a name like NCC-Scout2-policy.

You might need to indulge in a little trial and error if you didn't delete the original JSON properly (e.g., missing a closing curly bracket or making some other error before pasting NCC's example policy), but it's not too tricky to figure out, even if JSON isn't your best friend.

Thankfully, AWS has given some forethought to eye strain by generating a human-readable policy table from the new policy you entered (Figure 3). This table lets you see which keys you're giving to Scout2 and to which kingdom.

When you are back on the IAM screen, look at the left-hand menu (the menu in Figure 1) and choose Users. Next, you should select the user that will run the scan (make sure you definitely have access to that user's access and secret keys).

Finally, you need to attach the NCC-Scout2-policy to your IAM user by simply clicking your username link on the Users page and then clicking Add permissions. On the next page, choose the box that says Attach existing policies directly. Find your freshly created policy with a simple search for NCC, or whatever you called your previously saved policy, and then attach it to your user. Now, via IAM, your user can perform the actions named in the policy.

A Parliament of Owls

The next step is to get the Scout2 auditing tool up and running. Once the scan is complete, Scout2 thoughtfully attempts to open a window in a web browser to show the results of your tests, so you need to be a non-root user to install Scout2 (on my Linux Mint laptop, at least). I'm quite happy navigating as the root user to the directory with the resulting HTML report, if needed, so you should decide at this stage which local Linux user you want to run Scout2. Be aware that if you use a non-root user, your AWS credentials will be on your local machine available to a non-privileged user, which is why I use the superuser to add a little more security and privacy.

One other very minor thing to consider is that if you do use a non-root user, some of the packages might potentially need to be installed as the superuser beforehand.

A Waddle of Penguins

If you've used the Python Pip package manager, you probably already have setuptools and Pip installed. To install Pip on Debian derivatives, use the command:

$ apt install python-pip

On Red Hat Enterprise Linux derivatives, you might need to enable the extra packages repository, which is commonly referred to as EPEL. (You'll find much more about EPEL online [5].) On CentOS, for example, you run the command

$ yum install epel-release

to enable the repo. Refer to the EPEL page for help if you need it.

Once Pip is installed, you can install the setuptools package with your shiny new Pip package manager:

$ pip install setuptools

Next, pull down the Scout2 package:

$ pip install awsscout2

If you want to install Scout2 from source, so you can have a rummage in the package to check for suspicious activity that might compromise your AWS account(s), you can run the following commands, which are also found on the NCC GitHub page:

$ git clone https://github.com/nccgroup/Scout2
$ cd Scout2
$ pip install -r requirements.txt
$ python setup.py install

If you've used the AWS application programming interface (API) before, you've also likely installed the relatively miniscule command-line tool called awscli. My package manager seemed to have a less compatible (possibly older) version of awscli, so I used the faithful Pip package manager yet again to install my preferred version:

$ pip install awscli

If you get stuck, check out the AWS docs [6].

You might need to add awscli to your user's path [7], which Pip put in your home directory, by entering something along the line of:

$ export PATH=~/.local/bin:$PATH

The final easy hurdle to jump over is copying your user's AWS access key and secret key into the credentials file on your local machine. These should be saved in your ~/.aws/credentials path.

Head to the main IAM screen again in your AWS account to set up a new key if you don't have one. Assuming you're using an IAM user and have disabled or aren't using the AWS root account, as advised by AWS, you can use the same menu again (Figure 1) to select Users and click on the correct username link. You're looking for the Security credentials tab shown in Figure 4.

Click the Create Access Key button and you're off. Keep a careful note, encrypted, of the secret key, because this is the only time you will see it in AWS. To copy your credentials file locally, use the format:

[default]
aws_access_key_id = your_access_key_id
aws_secret_access_key = your_secret_access_key

Replace both bits after the equals signs with the relevant info. If you get stuck or you're new to this process, then once you have installed awscli, run the command

$ aws configure

to walk through the configuration; otherwise, you can read the credentials docs on the AWS website [8] or search online for other instructions. Generally, you should just need to copy the access and secret keys into the appropriate files, without having to export variables and other items.

A Thunder of Hippos

Now for the moment of truth: You can run your sophisticated AWS auditing tool over the AWS collective with the command:

$ Scout2 --profile <name-in-AWS-cred-file> --service iam --regions eu-west-1

First, however, replace the profile name from your ~/.aws/credentials file with the profile variable (e.g., default becomes default, in place of <name-in-AWS-cred-file>, and you might need to change your --regions argument if you're not using Dublin, Ireland, as I was.

The resulting report, assuming your browser doesn't open automatically, is found in the file ~/scout2-report/report.html or, if you're root, /root/scout2-report/report.html.

Because I usually run this as root, I tend to copy and chown the whole scout2-report/ directory into my non-privileged user's home directory before clicking on the HTML report, which should pop up in a browser.

Figure 5 shows my report, with the account number redacted for security reasons; the massive number of AWS services is also truncated.

I'm going to leave you to explore the detail offered by the prodigious security tool that is Scout2. Ideally, you should spend some time familiarizing yourself with how AWS grades its security advisories and, more importantly, how you can remediate them. Because of the multitude of AWS services now on offer, there's simply too much to cover here.

To whet your appetite, check out Figure 6, my IAM report dashboard, and Figure 7, a redacted report from another section with a higher level of detail.

The End

Once you've reviewed your results, I hope you will agree that the Scout2 reports are genuinely comprehensive and allow you to drill down into each section to gain some very useful details on the issues that require fixing. Because the NCC Scout2 tool is actively maintained and used by NCC, it's a good bet that additional AWS features and services will be added in the future to keep it current.

When such excellent security tools exist, you have little excuse not to know about the issues that affect your AWS cloud infrastructure, even if you're reluctant to fix some of them.

I trust you will find Scout2 as valuable in keeping your services running round the clock as I do.