Cloud Police

The Azure cloud offers a variety of access options for resources. Many organizations outsource parts of their data centers to Azure and publish services such as websites, applications, or complex infrastructures as publicly accessible offerings or for users within their own organizations. Rarely are the scenarios so simple that they involve a single website or a single web server operated and published from an Azure virtual machine (VM). "Lift and shift" scenarios or complex architectures, in particular, often map various protected levels that force the isolation of diverse services and components. Classic data centers are rarely connected by any-to-any communication but are grouped into zones and protected.

For demanding applications that comprise multiple components, databases, front ends, back ends, storage, microservices, and loosely connected external data sources, no one likely has the need to talk to everyone else. This means that Azure also needs to support the isolation and distribution of various resources. Imagine a distributed application with multiple tiers distributed across multiple data centers for resilience. Because Azure networks can be connected across data center boundaries and regions, as well, it must be possible to define which requests cross these boundaries. The means of choice goes by the name of Network Security Groups (NSGs).

Rules-Based Traffic Control

On Azure networks, NSGs act like a firewall, examining and filtering incoming and outgoing traffic. Filtering is solved in a classic way, involving definable rules, controlled by Azure admins. The solution is useful for all resources of an Azure network, such as VMs, Azure batch services, HDInsight, Azure AD Domain Services, containers in Kubernetes or container instances, or Azure SQL. These firewall-style capabilities can be applied to multiple resources and are not limited to the outer limits of a virtual network.

For a defense-in-depth strategy, general communication rules can be defined on the virtual network, which in turn can be more granularly defined for individual services within the network. For example, say two virtual networks are connected: One of them will host network infrastructure and the other a publicly available multitiered website. To operate the application, communication must be established and protected both within the website network and beyond its boundaries into the infrastructure network. Although only the front end and back end need to communicate within the web network, the back end has to be able to negotiate with resources from the remote network. Therefore, a multitiered set of rules is required: one NSG at each network level, and several NSGs that include the resource groups of the different servers and allow traffic.

A ruleset of this kind comprises five pieces of information: source, source address, target, target address, and protocol. The ruleset can be used to allow or deny access. NSGs are stateful, so they can remember communication paths and any connections created. You do not have to define response packages for incoming or outgoing requests separately in the ruleset. A rule can contain several target and source addresses and several target and source ports – you do not have to create a rule for each protocol or logical connection. This is especially useful if you can bundle traffic from server groups. NSGs work at the protocol level, making them Layer 4 devices (Figure 1).

Creating NSGs

You can define up to 100 NSGs per Azure subscription, with Azure Support increasing the limit to 200 on request. Although this sounds like a large number, you should still plan your deployment well to avoid ending up with a bottleneck. Microsoft also allows the definition of 200 firewall rules per NSG (which can be extended to 1,000 if you contact Azure support).

When creating new resources in Azure, you can define a default NSG ruleset for a new virtual network as early as the create stage – protecting new resources such as VMs from the start. First, however, you need to create a new resource group for test purposes, to which you add a new NSG by clicking +Create a resource and searching for Network Security Group. Next, name the NSG and be sure to check that the correct resource group is selected.

When you then create a new VM on the Azure portal in an existing resource group, the wizard will guide you through the most important machine setup decisions. If you create a VM manually, you will be asked for the Network Security Group with options Basic or Advanced, in addition to building a new virtual network (VNet).

The Basic option provides the general switch that enables or disables access to resources on the new network from the Internet. Although this is not very granular, it is particularly useful for less complex or small scenarios or test environments. For a proper ruleset, switch to Advanced and create a new NSG by entering a name. A simple naming convention is very important and recommended.

After deploying the VM, you have a resource group, a VM with all its necessary resources, and two NSGs: one for the VM and one for the network. If you select the default subnet while creating a VM in the virtual network settings (Figure 2), you can also configure an NSG there. From the list, select the NSG you previously created, thereby assigning the VM and the entire virtual network their own NSGs with their own rules. If you look at the Networking properties of your VM, you will see the VM's ruleset, which includes the VM's own NSG for incoming and outgoing network traffic, as well as the virtual network's NSG. The blade therefore lists all rulesets that are applied at different levels (the resource itself, resource group, and network). At the head of the blade, you can view the resulting ruleset in Effective security rules – all rules are determined there. Of course, managing a large number of VMs and deployments manually in this way is difficult; to understand the concepts, it helps to examine the effects and relationships on the Azure portal.

Allowing RDP and SSH

To grant remote access to the newly installed VM, create two new rules: one for the network NSG and one for the VM NSG. Access from the Internet to the VM must pass through both NSGs so that the traffic can access the VM end-to-end. If one of the rules is missing, you see an Access denied message, and the connection is not established.

Selecting the properties of the VM and then Networking takes you to the rule overview. Press Add inbound port rule to create a new rule for the VM's NSG. Microsoft offers predefined endpoints for the rules via service tags. For Source you can select Service tag and then Internet. In the list of service tags, you can see some additional definitions that can be helpful when creating complex rules and connecting several Azure services to each other: Internet, Virtual Network, AzureLoadBalancer, EventHub, AzureActiveDirectory, and Storage are just a few of the definitions you can select for Azure to assign the correct source or destination IP addresses dynamically to the rule. As the Destination, select the IP address of the VM; now the rule is as specific as possible. As Destination ports, you can specify 3368, 22 for the two services Remote Desktop Protocol (RDP) and Secure Shell (SSH), which lets you manage Windows and Linux remotely.

Temporary Remote Access

Azure offers just-in-time VM access in the Azure Security Center (ASC) for additional protection of VMs that need ports opened. ASC can protect port releases by checking during connection establishment whether the user to be connected has permissions for the VM in Azure role-based access control (RBAC). If so, access is allowed, if not, access is canceled. ASC creates new NSG access rules for both the virtual network and the VM itself during VM enrollment. By default, access to VMs and management ports is blocked.

If an end user wants access, they first need to request permission by selecting Request access in the properties of the VM under Connect. It is only after doing so that VM access via RDP or SSH is enabled and the RDP file is released for download. Access will then only be enabled for the IP address from which the access request was made. This can also be enabled with NSG rules that ASC manages. Access is automatically rolled back after a predefined time (the default is three hours). In ASC, you can also define whether an access request requires confirmation from another administrator or whether access can be granted immediately after manual activation.

The enrollment of a VM for just-in-time access can be initiated on the Azure portal by selecting the VM properties and then Configuration and pressing Enable just-in-time policy. As soon as you enable this, you will discover that the ASC has already created all the required rules and prioritized them as necessary (Figure 3). Each VM is given an NSG rule that contains an any-to-any block statement. If access to a VM is requested with Request access, the system creates a new NSG rule (for each access path), and access is then permitted specifically with a lower priority value. The lower the priority value, the earlier the rule is evaluated. The first applicable rule wins.

Publishing endpoints on the Internet is highly discouraged, but managing the VMs either from an internal corporate network connected to Azure by ExpressRoute or a VPN, or with a point-to-point VPN connection from the laptop to the target network on which the VMs are located, gives you access to the VMs without actually having to publish ports on the Internet. If neither ExpressRoute nor VPN are available, as might be the case in a test environment, enable remote access only as long as you need it to reduce the attack vectors to the extent possible.

Automate NSGs

Setting up multiple NSGs manually is not particularly efficient; automation with PowerShell or Azure Resource Manager (ARM) templates is the far more convenient and enterprise-ready option. The cmdlets for creating and managing NSGs are available by an Install-Module named AzureRM. One essential advantage is that the scripts can be archived and are thus perfect as documentation, which is an important step toward Infrastructure as Code (IaC).

You will receive an overview of all NSGs in a resource group in Azure as soon as you register with Azure and choose a subscription:

> Login-AzureRMAccount
> Select-AzureRmSubscription -Name "Test-Subscription"

You call the NSGs of a resource group as follows:

> Get-AzureRmNetworkSecurityGroup -ResourceGroupName "NSG-Test" | SELECT Name, SecurityRules

A new rule for an existing NSG is easily added:

> $nsg = Get-AzureRmNetworkSecurityGroup -ResourceGroupName "NSG-Test" -Name "Vnet-NSG"
> $nsgVNet | Add-AzureRmNetworkSecurityRuleConfig -Name "AllowFTP" -Description "Allow FTP port in for FTP server VM" -Protocol Tcp -SourcePortRange "*" -DestinationPortRange "21" -SourceAddressPrefix "*" -DestinationAddressPrefix "10.0.4.4" -Access Allow -Direction Inbound -Priority 200 | Set-AzureRmNetworkSecurityGroup

Here, the Network Security Group to be changed is first stored in a variable and then piped to Add-AzureRMNetworkSecurityRuleConfig before being set to Set-AzureRMNetworkSecurityGroup. Changing a rule follows a similar procedure:

> $nsg = Get-AzureRmNetworkSecurityGroup -Name "Vnet-NSG" -ResourceGroupName "NSG-Test"
< $nsg | Get-AzureRmNetworkSecurityRuleConfig -Name "AllowFTP"
> Set-AzureRmNetworkSecurityRuleConfig -Name "AllowFTP" -NetworkSecurityGroup $nsg -Protocol "UDP" -Priority 150

You just call the rule by name and then change it.

Traffic Analysis with Flow Logs

Where Network Security Groups are used, you are naturally also interested in their content. For this purpose, Microsoft provides the NSG flow logs function as part of the Network Watcher, which lets you record, save, and later evaluate the details of incoming and outgoing IP traffic. Flow logs are not handled like other logfiles; they are stored directly in the storage account. The connections are generated on storage during logging and created in the usual JSON format to enable subsequent standardized reading and evaluation with a security information and event management (SIEM) tool. All relevant information, such the source and target IP addresses, source and target ports, protocol used, and whether access was allowed or blocked, are recorded.

To enable NSG flow recording, enable the Network Watcher, which then logs the NSG connections. You will find the Network Watcher by searching for it on the Azure portal with the All services selection. In the wizard, select the corresponding region (e.g., USA) and enable the Watcher. Next, go to the Azure portal, select Subscriptions and select the target subscription where the NSG is located. For the subscription, search in Resource providers for a provider named microsoft.insights, which you also need to register.

Now you can enable flow logs. As already mentioned, this can be done at different levels. The storage account and the retention period are defined in each case. However, configuring this manually for each NSG will involve a great deal of work; the Azure CLI or PowerShell is very helpful here. A script by Alexandre Verkinderen [1] uses PowerShell to enable NSG flow logs and saves them to the defined storage account. The retention period for the logs can also be defined to suit your requirements; you configure this as a value in days. If you leave the value empty, the logs are not deleted.

If you want to evaluate the logs in a visually appealing way and publish them on a dashboard, you can have the connections processed by Power BI. Microsoft does not have its own dashboard for flow logs but offers a template that displays the connection data [2]. The template requires both the storage account name and the desired number of logfiles to be loaded and included in the analysis. Only the current access key is then required to access the storage account; then, the logfiles are downloaded and displayed in previously prepared visualizations.

Conclusions

Network Security Groups provide multilayered protection for resources in Azure. Defining rules for networks, VMs, or containers enables defense in depth at several levels. If you are outsourcing complex infrastructures to Azure, you especially might need to segment critical data and regulate the data flow to ensure security.