Managing IAM Policies for BCM Cluster Operations in AWS
For many environments, fine-grained IAM (Identity and Access Management) policies are a critical requirement. These policies govern access to key AWS services, ensuring BCM components and users can perform necessary tasks without granting overly broad permissions.
This article outlines the current approach to AWS IAM policy management for BCM clusters. It provides guidance on tagging resources, policy usage, and access setup for various tools.
The JSON documents used as policies for various tools are attached at the end of the article to keep it readable.
Where to Manage Policies
Policies are created and managed in the AWS Console, specific to each region. For the eu-west-1 region, visit: Manage IAM policies in EU-West-1
From there, you can select “Create policy” to define a new policy or modify existing ones.
For more information, please consult AWS documentation.
Scope of the policies
Since the security policies are unique for each cluster, we publish reference documents only. They can be modified as required by the security team overseeing the cloud infrastructure. Users are welcome to modify the attached JSON documents to fit the needs of their environment.
Tagging Resources
To limit the scope of the policies, we identify resources managed by BCM tools by specific AWS tags. Most of the time, actions that create or delete important resources have conditions that validate the tags.
Due to historical reasons, not all resources are created with the same set of tags. We’re working to fix this in future releases, but as of now, we have introduced a non-default tag (managed_by_bcm) to identify resources. Various attached JSON documents refer to this tag. The user may specify a different tag instead, or simply remove the “Condition” section from the desired policy if such granularity is not needed.
Tag key/value: managed_by_bcm=true
Usage: This tag should be passed during cluster creation:
cm-cod-aws cluster create <rest of the arguments> --cluster-tags managed_by_bcm=true
Basic use-case
The user may want to create an HA cluster in AWS. The security team requires a user with minimal permissions needed for the job.
- Create a new IAM user in AWS, and create an access key pair for the user
- Create a new IAM policy, based on the JSON documents found further down in this article. For the given task, the user would need the cm-cod-aws policy and an HA policy.
- Attach the policies to the user created in step 1.
- Create an AWS COD cluster, as outlined in the manual. The key pair created in step 1 can be used to configure the cod tool
- Deploy HA on a new cluster. The key pair created in step 1 can be used to configure the cm-cloud-ha-setup tool
- Based on the environment, the admin may want:
6.1. Use a different IAM user to manage the cluster (e.g., power the nodes on/off). In this case, a new user may be configured with the cmdaemon policy.
6.2. Use the same user created previously. In this case, the cmdaemon policy should be attached to the user created in step 1.
Note: By default, the access key pair used by the cod tool is configured for CMD as well. This can be done from cmsh by setting “Access key ID” and “Secret access key” as shown in the manual.
IAM Policies Overview
We designed several IAM policies with granular scopes, allowing minimal permissions for each task. These policies can be attached to IAM users or roles based on what functionality they need. cmdaemon policy can be used as an instance profile for the head node.
1. cm-cod-aws policy (Cluster Lifecycle Management)
Used by: cm-cod-aws
Purpose: Create, start, stop, and delete clusters
Required for:
cm-cod-aws cluster create
cm-cod-aws cluster delete
cm-cod-aws cluster start
cm-cod-aws cluster stop
2. HA policy (HA Deployment and Recovery)
Used by: Head node
Purpose: Set up and recover high availability
Required for:
cm-cluster-ha-setup
3. Cluster extension policy (On-prem to Cloud Extension)
Used by: Head node
Purpose: Extend a bare-metal cluster to AWS
Required for:
cm-cluster-extension
4. cmdaemon policy (Head Node Cloud Management)
Used by: cmdaemon running on the head node
Purpose: Manage cloud resources and perform initial cluster setup.
Note: This policy has 2 use-cases:
- Can be attached to an IAM user whose API keys are used for the cloud resources management
- Can be attached to a role assumed by the EC2 head node instance, allowing it to operate without embedded AWS credentials. Such roles are known as instance profiles, and the documentation is available here.
A cluster where the head node assumes an instance profile can be created with the command:
cm-cod-aws cc <rest of the arguments> --headnode-instance-profile head-node-instance-profile
Policy Documents
The JSON definitions of each IAM policy are listed below:
1. cm-cod-aws policy:
{
"Version": "2012-10-17",
"Statement": [
{
"Sid": "EC2ReadOperations",
"Effect": "Allow",
"Action": [
"ec2:DescribeAddresses",
"ec2:DescribeAvailabilityZones",
"ec2:DescribeVpcs",
"ec2:DescribeSubnets",
"ec2:DescribeInstances",
"ec2:DescribeNatGateways",
"ec2:DescribePlacementGroups",
"ec2:DescribeImages",
"ec2:DescribeInternetGateways",
"ec2:DescribeNetworkAcls",
"ec2:DescribeRegions",
"ec2:DescribeRouteTables",
"ec2:DescribeSecurityGroups",
"ec2:DescribeInstanceTypes",
"ec2:DescribeKeyPairs",
"ec2:DescribeNetworkInterfaces"
],
"Resource": "*"
},
{
"Sid": "EC2CreateOperationsTaggedRequest",
"Effect": "Allow",
"Action": [
"ec2:CreateVpc",
"ec2:CreateInternetGateway",
"ec2:CreateNatGateway",
"ec2:CreatePlacementGroup",
"ec2:CreateVpc",
"ec2:CreateSubnet",
"ec2:CreateRouteTable",
"ec2:CreateNetworkAcl",
"ec2:CreateNetworkAclEntry",
"ec2:CreateSecurityGroup"
],
"Resource": "*",
"Condition": {
"ForAnyValue:StringEquals": {
"aws:TagKeys": [
"managed_by_bcm"
]
}
}
},
{
"Sid": "EC2CreateOperationsTaggedResource",
"Effect": "Allow",
"Action": [
"ec2:CreateVpc",
"ec2:CreateInternetGateway",
"ec2:CreateNatGateway",
"ec2:CreatePlacementGroup",
"ec2:CreateVpc",
"ec2:CreateSubnet",
"ec2:CreateRouteTable",
"ec2:CreateNetworkAcl",
"ec2:CreateNetworkAclEntry",
"ec2:CreateSecurityGroup"
],
"Resource": "*",
"Condition": {
"ForAnyValue:StringLike": {
"ec2:ResourceTag/managed_by_bcm": "true"
}
}
},
{
"Sid": "EC2CreateRouteOperations",
"Effect": "Allow",
"Action": [
"ec2:CreateRoute"
],
"Resource": "*"
},
{
"Sid": "EC2ModifyOperations",
"Effect": "Allow",
"Action": [
"ec2:AllocateAddress",
"ec2:ModifyInstanceAttribute",
"ec2:AttachInternetGateway",
"ec2:AssociateRouteTable",
"ec2:AuthorizeSecurityGroupIngress",
"ec2:RevokeSecurityGroupIngress",
"ec2:RevokeSecurityGroupEgress",
"ec2:ReplaceNetworkAclAssociation"
],
"Resource": "*"
},
{
"Sid": "EC2CreateTagsForElasticIPs",
"Effect": "Allow",
"Action": [
"ec2:CreateTags"
],
"Resource": "*"
},
{
"Sid": "EC2InstanceManagement",
"Effect": "Allow",
"Action": [
"ec2:TerminateInstances",
"ec2:StopInstances",
"ec2:StartInstances"
],
"Resource": "arn:aws:ec2:*:*:instance/*",
"Condition": {
"ForAnyValue:StringLike": {
"ec2:ResourceTag/managed_by_bcm": "true"
}
}
},
{
"Sid": "EC2RunInstance",
"Effect": "Allow",
"Action": [
"ec2:RunInstances"
],
"Resource": "*",
"Condition": {
"ForAnyValue:StringEquals": {
"aws:TagKeys": [
"managed_by_bcm"
]
}
}
},
{
"Sid": "AllowOtherRequiredResourcesForRunInstances",
"Effect": "Allow",
"Action": [
"ec2:RunInstances"
],
"Resource": [
"arn:aws:ec2:*:*:volume/*",
"arn:aws:ec2:*::image/ami-*",
"arn:aws:ec2:*:*:subnet/*",
"arn:aws:ec2:*:*:network-interface/*",
"arn:aws:ec2:*:*:key-pair/*",
"arn:aws:ec2:*:*:security-group/*",
"arn:aws:ec2:*:*:placement-group/*",
"arn:aws:ec2:*::snapshot/*"
]
},
{
"Sid": "ElasticIPManagement",
"Effect": "Allow",
"Action": [
"ec2:DisassociateAddress",
"ec2:AssociateAddress",
"ec2:ReleaseAddress"
],
"Resource": "*"
},
{
"Sid": "EC2DeleteOperations",
"Effect": "Allow",
"Action": [
"ec2:DeleteSubnet",
"ec2:DeleteRouteTable",
"ec2:DetachInternetGateway",
"ec2:DeleteInternetGateway",
"ec2:DeleteSecurityGroup",
"ec2:DeleteNetworkAcl",
"ec2:DeleteVpc",
"ec2:DeleteNatGateway",
"ec2:DeletePlacementGroup"
],
"Resource": "*",
"Condition": {
"ForAnyValue:StringLike": {
"ec2:ResourceTag/managed_by_bcm": "true"
}
}
},
{
"Sid": "EFSOperations",
"Effect": "Allow",
"Action": [
"elasticfilesystem:DeleteFileSystem",
"elasticfilesystem:DeleteMountTarget",
"elasticfilesystem:DescribeFileSystems",
"elasticfilesystem:DescribeMountTargets"
],
"Resource": "*"
},
{
"Sid": "PricingOperations",
"Effect": "Allow",
"Action": [
"pricing:GetProducts"
],
"Resource": "*"
},
{
"Sid": "PassRoleToEC2Operations",
"Effect": "Allow",
"Action": "iam:PassRole",
"Resource": "arn:aws:iam::*:role/*"
}
]
}
2. HA policy:
{
"Version": "2012-10-17",
"Statement": [
{
"Sid": "EC2ReadOperations",
"Effect": "Allow",
"Action": [
"ec2:DescribeVolumes",
"ec2:DescribeImages",
"ec2:DescribeSnapshots",
"ec2:DescribeInstances",
"ec2:DescribeAddresses",
"ec2:DescribeNatGateways",
"ec2:DescribeNetworkAcls",
"ec2:DescribeNetworkInterfaces",
"ec2:DescribeRouteTables",
"ec2:DescribeSecurityGroups",
"ec2:DescribeSubnets",
"ec2:DescribeTags"
],
"Resource": "*"
},
{
"Sid": "EC2RunInstance",
"Effect": "Allow",
"Action": [
"ec2:RunInstances"
],
"Resource": "*",
"Condition": {
"ForAnyValue:StringEquals": {
"aws:TagKeys": [
"managed_by_bcm"
]
}
}
},
{
"Sid": "EC2InstanceOperations",
"Effect": "Allow",
"Action": [
"ec2:TerminateInstances",
"ec2:ModifyInstanceAttribute"
],
"Resource": "*",
"Condition": {
"ForAnyValue:StringLike": {
"ec2:ResourceTag/managed_by_bcm": "true"
}
}
},
{
"Sid": "EC2NetworkOperations",
"Effect": "Allow",
"Action": [
"ec2:AllocateAddress",
"ec2:AssociateAddress",
"ec2:DisassociateAddress",
"ec2:ReleaseAddress",
"ec2:AssignPrivateIpAddresses",
"ec2:UnassignPrivateIpAddresses",
"ec2:AuthorizeSecurityGroupIngress",
"ec2:ModifyNetworkInterfaceAttribute",
"ec2:DeleteNetworkInterface"
],
"Resource": "*"
},
{
"Sid": "EC2NatGatewayOperations",
"Effect": "Allow",
"Action": [
"ec2:CreateNatGateway",
"ec2:DeleteNatGateway",
"ec2:ReplaceRoute",
"ec2:ReplaceNetworkAclEntry"
],
"Resource": "*"
},
{
"Sid": "EC2ImageAndSnapshotOperations",
"Effect": "Allow",
"Action": [
"ec2:CreateImage",
"ec2:DeregisterImage",
"ec2:CreateSnapshot",
"ec2:DeleteSnapshot",
"ec2:CreateTags"
],
"Resource": "*"
},
{
"Sid": "EFSOperations",
"Effect": "Allow",
"Action": [
"elasticfilesystem:DescribeFileSystems",
"elasticfilesystem:DescribeMountTargets",
"elasticfilesystem:CreateFileSystem",
"elasticfilesystem:CreateMountTarget",
"elasticfilesystem:DeleteFileSystem",
"elasticfilesystem:DeleteMountTarget",
"elasticfilesystem:TagResource"
],
"Resource": "*"
},
{
"Sid": "PassRoleToEC2Operations",
"Effect": "Allow",
"Action": "iam:PassRole",
"Resource": "arn:aws:iam::*:role/*"
}
]
}
3. Cluster extension policy:
{
"Version": "2012-10-17",
"Statement": [
{
"Sid": "EC2ReadOperations",
"Effect": "Allow",
"Action": [
"ec2:DescribeVpcs",
"ec2:DescribeVpcEndpoints",
"ec2:DescribeVpcPeeringConnections",
"ec2:DescribeSubnets",
"ec2:DescribeInstances",
"ec2:DescribeRegions",
"ec2:DescribeRouteTables",
"ec2:DescribeNetworkAcls",
"ec2:DescribeSecurityGroups",
"ec2:DescribeInternetGateways",
"ec2:DescribeInstanceTypes",
"ec2:DescribeSnapshots",
"ec2:DescribeAvailabilityZones",
"ec2:DescribeInstanceTypeOfferings"
],
"Resource": "*"
},
{
"Sid": "EC2CreateOperationsTaggedRequest",
"Effect": "Allow",
"Action": [
"ec2:CreateVpc",
"ec2:CreateNetworkAclEntry",
"ec2:CreateNetworkAcl",
"ec2:CreateSubnet",
"ec2:CreateSecurityGroup",
"ec2:CreateInternetGateway",
"ec2:CreateRouteTable",
"ec2:CreateRoute"
],
"Resource": "*",
"Condition": {
"ForAnyValue:StringEquals": {
"aws:TagKeys": [
"BCM Resource"
]
}
}
},
{
"Sid": "EC2CreateOperationsTaggedResource",
"Effect": "Allow",
"Action": [
"ec2:CreateVpc",
"ec2:CreateNetworkAclEntry",
"ec2:CreateNetworkAcl",
"ec2:CreateSubnet",
"ec2:CreateSecurityGroup",
"ec2:CreateInternetGateway",
"ec2:CreateRouteTable",
"ec2:CreateRoute"
],
"Resource": "*",
"Condition": {
"ForAnyValue:StringEquals": {
"ec2:ResourceTag/BCM Resource": "true"
}
}
},
{
"Sid": "EC2WriteOperations",
"Effect": "Allow",
"Action": [
"ec2:ModifyVpcAttribute",
"ec2:ReplaceNetworkAclAssociation",
"ec2:AttachInternetGateway",
"ec2:DetachInternetGateway",
"ec2:AssociateRouteTable",
"ec2:AuthorizeSecurityGroupIngress",
"ec2:RevokeSecurityGroupIngress"
],
"Resource": "*"
},
{
"Sid": "EC2DeleteOperationsTagged",
"Effect": "Allow",
"Action": [
"ec2:DeleteVpc",
"ec2:DeleteSubnet",
"ec2:DeleteNetworkAcl",
"ec2:DeleteSecurityGroup",
"ec2:DeleteInternetGateway",
"ec2:DeleteRouteTable",
"ec2:DeleteRoute",
"ec2:DeleteVpcEndpoints"
],
"Resource": "*",
"Condition": {
"ForAnyValue:StringEquals": {
"ec2:ResourceTag/BCM Resource": "true"
}
}
},
{
"Sid": "EFSOperations",
"Effect": "Allow",
"Action": [
"elasticfilesystem:CreateFileSystem",
"elasticfilesystem:DeleteFileSystem",
"elasticfilesystem:CreateMountTarget",
"elasticfilesystem:DeleteMountTarget",
"elasticfilesystem:DescribeFileSystems",
"elasticfilesystem:DescribeMountTargets"
],
"Resource": "*"
},
{
"Sid": "FSXOperations",
"Effect": "Allow",
"Action": [
"fsx:DescribeFileSystems",
"fsx:DeleteFileSystem"
],
"Resource": "*"
},
{
"Sid": "ServiceQuotasOperations",
"Effect": "Allow",
"Action": [
"servicequotas:GetServiceQuota"
],
"Resource": "arn:aws:servicequotas:*:*:*"
},
{
"Sid": "EC2TagOperations",
"Effect": "Allow",
"Action": [
"ec2:CreateTags"
],
"Resource": [
"arn:aws:ec2:*:*:vpc/*",
"arn:aws:ec2:*:*:subnet/*",
"arn:aws:ec2:*:*:security-group/*",
"arn:aws:ec2:*:*:internet-gateway/*",
"arn:aws:ec2:*:*:route-table/*",
"arn:aws:ec2:*:*:network-acl/*"
],
"Condition": {
"ForAnyValue:StringEquals": {
"aws:TagKeys": [
"BCM Resource"
]
}
}
}
]
}
4. cmdaemon policy:
{
"Version": "2012-10-17",
"Statement": [
{
"Sid": "EC2ReadOperations",
"Effect": "Allow",
"Action": [
"ec2:DescribeInstances",
"ec2:DescribeInstanceAttribute",
"ec2:DescribeNetworkInterfaces",
"ec2:DescribeVolumes",
"ec2:DescribeAddresses",
"ec2:DescribeImages",
"ec2:DescribeRouteTables",
"ec2:DescribeSubnets",
"ec2:DescribeSecurityGroups",
"ec2:DescribeRegions",
"ec2:DescribeAvailabilityZones",
"ec2:DescribeInstanceTypes",
"ec2:DescribeVpcs",
"ec2:DescribeTags",
"ec2:GetConsoleOutput"
],
"Resource": "*"
},
{
"Sid": "EC2RunInstance",
"Effect": "Allow",
"Action": [
"ec2:RunInstances"
],
"Resource": "*",
"Condition": {
"ForAnyValue:StringEquals": {
"aws:TagKeys": [
"BCM Resource"
]
}
}
},
{
"Sid": "AllowOtherRequiredResourcesForRunInstances",
"Effect": "Allow",
"Action": [
"ec2:RunInstances"
],
"Resource": [
"arn:aws:ec2:*:*:volume/*",
"arn:aws:ec2:*::image/ami-*",
"arn:aws:ec2:*:*:subnet/*",
"arn:aws:ec2:*:*:network-interface/*",
"arn:aws:ec2:*:*:key-pair/*",
"arn:aws:ec2:*:*:security-group/*",
"arn:aws:ec2:*:*:placement-group/*",
"arn:aws:ec2:*::snapshot/*"
]
},
{
"Sid": "EC2InstanceManagement",
"Effect": "Allow",
"Action": [
"ec2:StartInstances",
"ec2:StopInstances",
"ec2:RebootInstances",
"ec2:TerminateInstances"
],
"Resource": "*",
"Condition": {
"ForAnyValue:StringEquals": {
"ec2:ResourceTag/BCM Resource": "true"
}
}
},
{
"Sid": "EC2CreateModifyOperations",
"Effect": "Allow",
"Action": [
"ec2:AssignPrivateIpAddresses",
"ec2:CreateTags",
"ec2:AssociateIamInstanceProfile",
"ec2:ModifyInstanceAttribute",
"ec2:CreateRoute"
],
"Resource": "*"
},
{
"Sid": "EC2SpotInstanceOperations",
"Effect": "Allow",
"Action": [
"ec2:DescribeSpotInstanceRequests",
"ec2:DescribeSpotPriceHistory",
"ec2:CancelSpotInstanceRequests",
"ec2:RequestSpotInstances"
],
"Resource": "*"
},
{
"Sid": "EC2TagOperations",
"Effect": "Allow",
"Action": [
"ec2:CreateTags"
],
"Resource": "*",
"Condition": {
"ForAnyValue:StringEquals": {
"aws:TagKeys": [
"BCM Resource"
]
}
}
},
{
"Sid": "EC2VolumeOperations",
"Effect": "Allow",
"Action": [
"ec2:CreateVolume",
"ec2:AttachVolume"
],
"Resource": [
"arn:aws:ec2:*:*:volume/*",
"arn:aws:ec2:*:*:instance/*"
]
},
{
"Sid": "EC2TagOperationsVolume",
"Effect": "Allow",
"Action": [
"ec2:CreateTags"
],
"Resource": [
"arn:aws:ec2:*:*:volume/*"
]
},
{
"Sid": "EC2DeleteOperations",
"Effect": "Allow",
"Action": [
"ec2:DeleteVolume",
"ec2:DeleteTags",
"ec2:DeleteRoute"
],
"Resource": "*"
},
{
"Sid": "ElasticIPManagement",
"Effect": "Allow",
"Action": [
"ec2:DisassociateAddress",
"ec2:AssociateAddress",
"ec2:ReleaseAddress"
],
"Resource": "*"
},
{
"Sid": "OtherServicesReadOperations",
"Effect": "Allow",
"Action": [
"pricing:GetProducts",
"ssm:GetParameter",
"iam:GetRole"
],
"Resource": "*"
}
]
}