2. Security Groups Monitoring & Automation

What will we build

We will be using custom config rule and a lambda function for remediating security group. The scenario is that ensure a production web server security group should have Inbound rules only for HTTP and HTTPS. If there is any change in this security group rule revert back to original rules

Create a security group with inbound rules only for HTTP and HTTPS

  • Open the Amazon VPC console at VPC Console
  • In the navigation pane, choose Security Groups
  • Select Create Security Group
  • Provide Name Tag, Group Name as WebServerSGDemo
  • Select security group WebServerSGDemo to update INBOUND rules.
  • On the Inbound Rules tab, choose Edit. Select an option for a rule for inbound traffic for Type, and then create two rules:
  • From Type choose HTTP and specify a value for Source as 0.0.0.0/0
  • From Type choose HTTPS and specify a value for Source as 0.0.0.0/0
  • Make note of this security group ID as it will be required in next step

Create IAM policy for lambda function remediate security group

  • Sign in to the AWS Management Console and open the AWS Config console at IAM Console
  • Select Policy from left panel of IAM console and then select Create Policy
  • Select JSON, delete default JSON policy and paste below JSON text for policy
{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Effect": "Allow",
            "Action": [
                "logs:CreateLogGroup",
                "logs:CreateLogStream",
                "logs:PutLogEvents"
            ],
            "Resource": "arn:aws:logs:*:*:*"
        },
        {
            "Effect": "Allow",
            "Action": [
                "config:PutEvaluations",
                "ec2:DescribeSecurityGroups",
                "ec2:AuthorizeSecurityGroupIngress",
                "ec2:RevokeSecurityGroupIngress"
            ],
            "Resource": "*"
        }
    ]
}
  • Click Review Policy
  • Provide Policy name as WebSGMonitorLambdaPolicy
  • Click Create Policy

Create IAM role for your Lambda function

  • On AWS IAM console, select Role from left panel
  • Select Create Role
  • Select Lambda from the list of services that will use this role and then select Next: Permission
  • In search box provide the policy name we created previously - WebSGMonitorLambdaPolicy
  • Select the check box next to the policy you created previously, WebSGMonitorLambdaPolicy and then select Next: Review
  • Name your role WebSGRemediationLambdaRole and provide description
  • Select Create Role

Create Lambda function

  • In the AWS Management Console, under Services, select Lambda to go to the Lambda Console
  • From the Dashboard, select Create Function
  • Provide a name for the function. We’re using WebSGAutoResponder
  • In the Runtime dropdown list, select Python 2.7 as our lambda function in in python>
  • Under Role, select Choose an existing role. Select WebSGRemediationLambdaRole and then select Create function
  • On function Configuration page, scroll down to Basic settings and change Timeout value to 15 sec
  • Scroll up to the Designer section and select the name of your Lambda function and use this code:
#
# This file made available under CC0 1.0 Universal (https://creativecommons.org/publicdomain/zero/1.0/legalcode)
#
# This code is only intended for instructional purposes and should not be used for any other use.

import boto3
import botocore
import json

 
APPLICABLE_RESOURCES = ["AWS::EC2::SecurityGroup"]

# Specify the required ingress permissions using the same key layout as that provided in the
# describe_security_group API response and authorize_security_group_ingress/egress API calls.

REQUIRED_PERMISSIONS = [
{
    "IpProtocol" : "tcp",
    "FromPort" : 80,
    "ToPort" : 80,
    "UserIdGroupPairs" : [],
    "IpRanges" : [{"CidrIp" : "0.0.0.0/0"}],
    "PrefixListIds" : [],
    "Ipv6Ranges": []
},
{
    "IpProtocol" : "tcp",
    "FromPort" : 443,
    "ToPort" : 443,
    "UserIdGroupPairs" : [],
    "IpRanges" : [{"CidrIp" : "0.0.0.0/0"}],
    "PrefixListIds" : [],
    "Ipv6Ranges": []
}]

# normalize_parameters
#
# Normalize all rule parameters so we can handle them consistently.
# All keys are stored in lower case.  Only boolean and numeric keys are stored.

def normalize_parameters(rule_parameters):
    for key, value in rule_parameters.iteritems():
        normalized_key=key.lower()
        normalized_value=value.lower()

        if normalized_value == "true":
            rule_parameters[normalized_key] = True
        elif normalized_value == "false":
            rule_parameters[normalized_key] = False
        elif normalized_value.isdigit():
            rule_parameters[normalized_key] = int(normalized_value)
        else:
            rule_parameters[normalized_key] = True
    return rule_parameters

# evaluate_compliance
#
# This is the main compliance evaluation function.
#
# Arguments:
#
# configuration_item - the configuration item obtained from the AWS Config event
# debug_enabled - debug flag
#
# return values:
#
# compliance_type -
#
#     NOT_APPLICABLE - (1) something other than a security group is being evaluated
#                      (2) the configuration item is being deleted
#     NON_COMPLIANT  - the rules do not match the required rules and we couldn't
#                      fix them
#     COMPLIANT      - the rules match the required rules or we were able to fix
#                      them
#
# annotation         - the annotation message for AWS Config

def evaluate_compliance(configuration_item, debug_enabled):
    if configuration_item["resourceType"] not in APPLICABLE_RESOURCES:
        return {
            "compliance_type" : "NOT_APPLICABLE",
            "annotation" : "The rule doesn't apply to resources of type " +
            configuration_item["resourceType"] + "."
        }

    if configuration_item["configurationItemStatus"] == "ResourceDeleted":
        return {
            "compliance_type": "NOT_APPLICABLE",
            "annotation": "The configurationItem was deleted and therefore cannot be validated."
        }

    group_id = configuration_item["configuration"]["groupId"]
    client = boto3.client("ec2");

    # Call describe_security_groups because the IpPermissions that are returned
    # are in a format that can be used as the basis for input to
    # authorize_security_group_ingress and revoke_security_group_ingress.

    try:
        response = client.describe_security_groups(GroupIds=[group_id])
    except botocore.exceptions.ClientError as e:
        return {
            "compliance_type" : "NON_COMPLIANT",
            "annotation" : "describe_security_groups failure on group " + group_id
        }
        
    if debug_enabled:
        print("security group definition: ", json.dumps(response, indent=2))

    ip_permissions = response["SecurityGroups"][0]["IpPermissions"]
    authorize_permissions = [item for item in REQUIRED_PERMISSIONS if item not in ip_permissions]
    revoke_permissions = [item for item in ip_permissions if item not in REQUIRED_PERMISSIONS]

    if authorize_permissions or revoke_permissions:
        annotation_message = "Permissions were modified."
    else:
        annotation_message = "Permissions are correct."

    if authorize_permissions:
        if debug_enabled:
            print("authorizing for ", group_id, ", ip_permissions ", json.dumps(authorize_permissions, indent=2))

        try:
            client.authorize_security_group_ingress(GroupId=group_id, IpPermissions=authorize_permissions)
            annotation_message += " " + str(len(authorize_permissions)) +" new authorization(s)."
        except botocore.exceptions.ClientError as e:
            return {
                "compliance_type" : "NON_COMPLIANT",
                "annotation" : "authorize_security_group_ingress failure on group " + group_id
            }

    if revoke_permissions:
        if debug_enabled:
            print("revoking for ", group_id, ", ip_permissions ", json.dumps(revoke_permissions, indent=2))

        try:
            client.revoke_security_group_ingress(GroupId=group_id, IpPermissions=revoke_permissions)
            annotation_message += " " + str(len(revoke_permissions)) +" new revocation(s)."
        except botocore.exceptions.ClientError as e:
            return {
                "compliance_type" : "NON_COMPLIANT",
                "annotation" : "revoke_security_group_ingress failure on group " + group_id
            }

    return {
        "compliance_type": "COMPLIANT",
        "annotation": annotation_message
    }

# lambda_handler
# 
# This is the main handle for the Lambda function.  AWS Lambda passes the function an event and a context.
# If "debug" is specified as a rule parameter, then debugging is enabled.

def lambda_handler(event, context):
    invoking_event = json.loads(event['invokingEvent'])
    configuration_item = invoking_event["configurationItem"]
    rule_parameters = normalize_parameters(json.loads(event["ruleParameters"]))

    debug_enabled = False

    if "debug" in rule_parameters:
        debug_enabled = rule_parameters["debug"] 

    if debug_enabled:
        print("Received event: " + json.dumps(event, indent=2))

    evaluation = evaluate_compliance(configuration_item, debug_enabled)

    config = boto3.client('config')

    response = config.put_evaluations(
       Evaluations=[
           {
               'ComplianceResourceType': invoking_event['configurationItem']['resourceType'],
               'ComplianceResourceId': invoking_event['configurationItem']['resourceId'],
               'ComplianceType': evaluation["compliance_type"],
               "Annotation": evaluation["annotation"],
               'OrderingTimestamp': invoking_event['configurationItem']['configurationItemCaptureTime']
           },
       ],
       ResultToken=event['resultToken'])
  • Key: SNSARN
  • Value: The SNS topic ARN we created earlier in Step-1
  • Save function by Selecting Save on top of the page

Create custom config rule to monitor configuration changes for security group and invoke lamnda

  • Open the AWS Config console
  • On the Rules page, choose Add rule
  • On the Add rule page, choose Add custom rule
  • On the Configure rule page, complete the following steps:
    • Name:, type WebSGMonitorRule
    • AWS Lambda function ARN:, specify the ARN of lambda function WebSGAutoResponder we created earlier
    • Trigger type: choose Configuration changes
    • Scope of changes: choose Resources
    • Resources: choose SecurityGroup
    • Resource Identifier: provide group ID of security group WebServerSGDemo we created earlier
  • Select Save

Verify it works

  • In the AWS Management Console, under Services, select VPC
  • Now select security group WebServerSGDemo to update INBOUND rules. The details pane displays the details for the security group, plus tabs for working with its inbound rules and outbound rules
  • On the Inbound Rules tab, choose Edit. Select an option for a rule for inbound traffic for Type, and then add additional rules as per below details
  • From Type choose LDAP and specify a value for Source as 0.0.0.0/0

Your security group inbound rule is currently incompliant. After several minutes, the AWS Config will detect this configuration change and invole lambda function for remediation.

Check the security group rules again after several min, and you would see that it has remediated the rules and removed rules other then HTTP and HTTPS. This is how you can ensure that your security group will run with only desired set of rules