Guard on Duty

Organizations, no matter the size, should implement infrastructure for preventing, detecting, and responding to security incidents. In this article, I focus mainly on detection with an AWS-native tool called GuardDuty [1], which sends its findings to a security information and event management (SIEM) system to generate and correlate security alerts to which analysts can react.

Cloud Dangers

One of the principal concerns many companies still have when migrating their assets to the cloud is security and compliance, even though cloud services now take this topic seriously. Cloud vendors have done a brilliant job marketing their security technology as the best and most sophisticated; however, data breaches happen frequently, and bad actors are targeting those large cloud providers.

Securing an on-premises data center is slightly different from securing your assets in the cloud. What is the definition of the cloud? Many would say someone else's computers, but in reality, it is so much more complex than that, although it is true that pure cloud computing involves leasing infrastructure resources from a service provider.

Both models have advantages and disadvantages and likely neither is perfect, so you must carefully choose which model you use depending on the level of security you need for your business. No matter how "big" the cloud provider, the shared responsibility model would be similar.

According to Amazon Web Services (AWS), they are responsible for protecting the underlying infrastructure that runs all of the services offered. This infrastructure comprises the hardware, software, networking, and facilities that run AWS Cloud services, including physical controls. Customers are responsible for the rest of the security controls. For example, the famous Capital One hack happened because of a misconfiguration of the web application and not the underlying cloud-based infrastructure. Other data breaches have occurred because a Security Group was not configured properly.

As a data or application owner, you must protect your company assets, because no one will care more than you or do the job better. Keep in mind the on-premises model – for which you need to take care of the firewall, anti-malware and antivirus defenses, security logs, and other fancy tools – for proper configuration when leveraging cloud-native tools.

GuardDuty

The substantial thing about GuardDuty is that you can have a decent intrusion detection system up and running in less than 15 minutes. In this article, I walk you through enabling the tool, configuring notifications and security alerts, and showing you different ways to bypass detection. Later, I'll also demonstrate how to create threat intelligence, but first, I'll look at what GuardDuty is and what it can do for you.

GuardDuty is an AWS managed service that uses machine learning capabilities to detect unauthorized and unexpected activity in your AWS environment (Figure 1). It detects anomalous behavior, extracts some fields, and finally discards them by analyzing the following logs:

• CloudTrail event logs: Of the two types – management and data events – GuardDuty analyzes only management logs, which are essentially API calls for your account, either through the AWS Management Console, the AWS SDKs, or the AWS command-line interface.
• VPC Flow Logs: These network logs provide visibility into all traffic going to and from Amazon EC2 and are configurable at the virtual private cloud (VPC), subnet, or network interface level. Metadata includes some interesting fields, like source IP, destination port, destination IP, account ID, protocol, and so on.
• DNS logs: An EC2 instance will use a default DNS resolver, for which Amazon has reserved an IP address that ends in 2 (e.g., 192.168.1.2 or 172.16.1.2). However, if you change the default configuration and add a third-party DNS server (e.g., Google, 8.8.8.8), GuardDuty cannot analyze the DNS logs.

Apart from enabling those logs, no other configuration needs to be performed on the GuardDuty side; it will just analyze the logs and trigger findings.

One great and very useful feature is the ability to upload your own threat list. I'll show you how to configure a malicious IP threat list later, but first you have to enable GuardDuty.

Enabling GuardDuty

When you log in on the AWS console and type the magic word GuardDuty in the search bar, you land on the Get Started page.

Note that when you enable GuardDuty for the first time in your account, you have a 30-day free trial, so it is an excellent time to play with the tool and become familiar with the look and feel. The pricing model is based on the quantity of CloudTrail events analyzed and the volume of VPC Flow Logs and DNS logs (per gigabyte). However turning on logging for those services is not required for GuardDuty to work. Please consult the most up-to-date information about volume discounts on the AWS GuardDuty website [2].

You must enable logging in all supported regions on your account. Yes, I know that hurts, but if something happens in a region that is not enabled, you miss the incident detection.

If you are working with AWS Organizations, you can nominate one account as the organization's delegated administrator for GuardDuty, so it becomes the primary (master) account automatically. With the primary account, you enable GuardDuty for any account in the organization and then add that account as a member account.

If you work with several accounts but do not use AWS Organizations, you still have to assign a primary account and then send invitations to member accounts. You just need to know the email address of the root target account and the account ID. Later, you can always add the primary account to the organization, allowing you to take full advantage of the added functionality of managing your GuardDuty accounts with AWS Organizations.

If your Identity and Access Management (IAM) user has administrative permissions into the account, click the orange Enable GuardDuty button. If that is not the case, and you would like to configure more granular permissions, you can always attach a policy (Figure 2) to an IAM user, group, or role. Take into account that GuardDuty will auto-assign itself the role AWSServiceRoleForAmazonGuardDuty to analyze the logs and generate findings.

If some instances expose ports like 22 or 3389 to the world (0.0.0.0/0), you should soon see some findings of malicious IPs trying to get into your systems. GuardDuty threat intelligence findings are based on feeds from third-party vendors CrowdStrike and Proofpoint, but you can add your own custom threat intel feeds, as well.

Currently, GuardDuty only supports IP-based threat lists. Two of my favorites are Tor exit nodes [3] and Open Threat Exchange (OTX) community feeds from AlienVault [4], but you may find some others by just doing some research. The lists are dynamic, so if you want GuardDuty to refresh automatically, you must create some automation scripts or lambda function to grab the source and copy to an S3 bucket with a TXT, CSV, or XML extension.

Formats supported by GuardDuty are STIX (XML) and CSV. If you join the community, you can download pulses that include indicators of compromise (IOCs) in IPv4 address range format, file hashes, file names, and malicious URLs. If you create a threat list in plaintext, IP, or classless interdomain routing (CIDR) format, ranges must be presented one per line, and you have to select Plaintext as the format. It is important to know that all these files need to be in an S3 bucket to which GuardDuty has access, but the good news is that a policy for permissions into S3 is granted automatically once you create a new threat list.

On the main GuardDuty console in the navigation pane under Settings, choose Lists and add a threat list. You can create a maximum of six lists per AWS account per region. Once you successfully upload the list, you must activate it by selecting the checkbox next to it. Now you are ready to have some fun.

Some Fun Now

Now that you have enabled GuardDuty and created some threat lists (Figure 3), you can start building instances as a playground. To get some attention from the bad guys out there, I'll start with an Amazon Linux EC2 instance with its security group configured to allow SSH to 0.0.0.0/0.

If you do not want to wait until a bad guy hits your endpoint, you can add your home public IP into one of the threat lists to generate findings quickly. After waiting for around 20 minutes, I got a hit from a malicious IP; looks like people from the Internet are scanning open ports and trying to get into any network. For now, it is nice to see how GuardDuty triggers a finding (Figure 4). This specific "finding type" is UnauthorizedAccess:EC2/MaliciousIPCaller.Custom, but I'll give more details later on the distinct types available.

Another way to speed the trigger of a finding is by creating a CloudTrail trail from the console and then stop it. GuardDuty will generate a finding type Stealth:IAMUser/CloudTrailLoggingDisabled.

It would be interesting to see whether GuardDuty is able to identify a Kali Linux instance, which is a popular penetration testing tool used by security professionals to perform security checks and attackers to wreak some malicious havoc. I launched a Kali instance from AWS Marketplace, and soon I got another finding; this time it is PenTest:IAMUser/KaliLinux (Figure 5).

Other types of findings can include port scan activities and geolocation (when an IAM user that normally logs in from one place suddenly logs in from another country), for which you can see machine learning in action. Algorithms that detect those behaviors include inspecting signal patterns for heuristics and profiling common behaviors to look for some deviations.

The previous examples have revealed a couple of finding types, but many more fall under the categories shown in Table 1.

Notification and Security Alerts

As you progress with GuardDuty and play with its graphical interface to get more familiar with all the features, you might think the tool would not scale well for a medium or enterprise company if you do not completely integrate with other tools. For instance, you would like to have GuardDuty send logs to a SIEM system for centralization and correlation with other data sources; also, you want to be informed about any malicious activity, or you need to create some actions that trigger on findings. Once the data is ingested by your SIEM tool, you can start creating security alerts and dashboards. You have the option to select findings by severity, category, target account, instance ID, or any other interesting field.

If your company does not have the budget to maintain a tool like a SIEM system, you can always use email as the preferred method to send notifications whenever GuardDuty finds something. For this use case, you might employ a straightforward method that uses AWS CloudWatch Events or Amazon EventBridge with the combination of a Simple Notification Service (SNS) topic to send the notifications to an email inbox. I'll look at both methods.

If you want to implement something simple and economical, you must log in to your AWS account, navigate to the Amazon SNS console, and choose Topics. In the Create topic section, enter a name and a description and click Create subscription. For Protocol, select Email, and in the Endpoint textbox, enter the email address that you want to receive the notifications. You must confirm the subscription by email.

Next, navigate to Amazon EventBridge, click Rules | Create rule and again enter a name and description. In the Define pattern section, choose Event pattern and select Pre-defined pattern by service and AWS as the Service provider. For Service name, type GuardDuty and select GuardDuty Finding as the Event type (Figure 6). The goal is to have any detected findings send notifications to the inbox of the email address specified earlier. You could customize to specific findings, but in this exercise, I notify for all.

After saving the event pattern, the last thing you need to do is select the Target, which is the SNS topic created earlier. In the Configure input options, select all matched events. If you want to be more selective, you must choose to create an input transformer of the data, so only the fields entered would be sent by email.

Everything is ready to receive a notification with findings, but this could sound a little basic, so those companies that have the luck to own their own SIEM system have a number of possibilities to do more complex stuff with this data.

Ingesting GuardDuty Logs

Many SIEM systems, either commercial or open source, are in the market. I'll focus on Sumo Logic, but the same would apply to others like Splunk, QRadar, ArcSight, Elasticsearch, and many others. The concept for any SIEM system is always the same: Provide at a glance a collection of events and logs, normalize the events for field mappings, convert the logs into user-friendly data, correlate with different sources (e.g., firewall data, etc.), report and alert, and last but not least, log management functions.

First, you have to get the data into Sumo Logic, and for that, you follow a similar approach as for the email notifications; however, this time you'll use the AWS Lambda service, which is a serverless application.

In Sumo Logic, you first need to create a Hosted Collector for the HTTP source, which will be the endpoint to which you send those logs. In Splunk, it is called an HTTP Collector. Once it is created, you must copy the endpoint URL provided, because you will need it for the next step. GuardDuty log format requires you to change the format for the timestamp (Figure 7).

The next step is to go to the AWS Serverless Application Repository, search for sumologic-guardduty-events-processor in the Applications searchbar, and select it in the left panel. Scroll down a bit until you see Application settings; paste the HTTP endpoint you copied earlier when you created the Sumo HTTP collector and click the Deploy button at the bottom of the screen.

Security Alerts

If you already had some findings, you should already have some logs. With that information, you are ready now to create security alerts, but you do not want to waste time and money responding to noisy false-positive alerts, so you can focus on high severity only.

According to a security research firm, FireEye, "Of the 17,000 malware alerts an organization receives each week, only 3,230 are considered reliable, and only 680 of the alerts are actually investigated" [5].

GuardDuty high-severity alerts include some of the following:

• Backdoor:EC2/C&CActivity.B!DNS
• Backdoor:EC2/DenialOfService.Tcp, Udp or DNS
• CryptoCurrency:EC2/BitcoinTool.B!DNS
• Trojan:EC2/DriveBySourceTraffic!DNS
• Trojan:EC2/DNSDataExfiltration

To simplify things, start by creating an alert that will trigger when one of those high-security alerts is found; from there, you can be more granular and selective by creating more complex alerts by target account or source IP. Just use your imagination and be creative.

To get familiar with the JSON data fields available, log in to the SIEM system and create a search for all GuardDuty data. Because high-severity alerts in GuardDuty fall within the numeric range 7 to 8.9, you must create the filter accordingly (Figure 8). The alert parses some JSON fields at search time, filters by severity (highlighted), and generates aggregated output by some specific fields, excluding the raw data.

If instead, you would prefer output showing the raw data, the results would look like Figure 9, with the severity highlighted. That specific finding is related to credentials misuse; the credentials were detected as having been used from an external IP address.

You can take things a step further and test your alerts by generating malicious traffic in your environment. Of the many tools out there for this purpose, one was specifically developed for this use case: the amazon-guardduty-tester script [6], which can be used as a proof-of-concept to generate several Amazon GuardDuty findings. Also from AWS is the amazon-guardduty-hands-on repo, which will instruct "by guiding you through enabling the detector, generating a variety of findings, and remediating those findings with Lambda functions" [7].

If you prefer something that is not vendor-specific, you can always use Network Flight Simulator [8], which is an open source utility used to generate malicious network traffic by performing several tests to simulate DNS tunneling, domain generation algorithm (DGA) traffic, requests to known active C2 destinations, and other suspicious traffic.

Bypass GuardDuty Detection

With some decent security alerts now in place, be aware that the tool does a good job detecting some issues, but it is not perfect and may miss some unknown malware, sources, and so on. However, there are still ways to bypass GuardDuty intentionally to evade detection on a network.

One example of detecting Kali Linux on the network is by identifying the user agent being used from the API call. The same happens if you are using Pentoo Linux or Parrot OS. By modifying the user agent, for example,

curl -H 'Boto/1.3.4 Python/3.7.0 MacOs/ Botocore/1.11.0'

GuardDuty will not identify those systems.

GuardDuty uses the default AWS DNS resolvers to find issues, so if someone changes the default configuration and adds a Google DNS (i.e., 8.8.8.8 or 8.8.4.4), the attacker can query any domain without being detected.

As you saw earlier, the alert in Figure 8 triggered because it detected compromised AWS instance credentials outside of its environment. The issue – and I'm not sure whether this has been fixed already by AWS – is that it must also be detected from an internal instance. Even so, a hacker can create a new instance inside the account and use those stolen credentials from there without being detected.

Conclusions

GuardDuty, like many tools, has some gaps. For the price of the service and its simplicity of use, I am confident that it will save you a great deal of time in implementation and deployment. You should definitely enable it in all regions and benefit from the free 30-day trial. Experience it, play with, get familiar, and do some investigations. I have not talked about integrations, but GuardDuty has a large variety of security vendors with which to integrate.