Use your company identities for analytics with Amazon EMR and AWS IAM Identification Middle

To allow your workforce customers for analytics with fine-grained knowledge entry controls and audit knowledge entry, you might need to create a number of AWS Identification and Entry Administration (IAM) roles with completely different knowledge permissions and map the workforce customers to a kind of roles. A number of customers are sometimes mapped to the identical function the place they want comparable privileges to allow knowledge entry controls on the company consumer or group stage and audit knowledge entry.

AWS IAM Identification Middle permits centralized administration of workforce consumer entry to AWS accounts and functions utilizing a neighborhood identification retailer or by connecting company directories by way of identification suppliers (IdPs). IAM Identification Middle now helps trusted identification propagation, a streamlined expertise for customers who require entry to knowledge with AWS analytics companies.

Amazon EMR Studio is an built-in improvement setting (IDE) that makes it easy for knowledge scientists and knowledge engineers to construct knowledge engineering and knowledge science functions. With trusted identification propagation, knowledge entry administration could be based mostly on a consumer’s company identification and could be propagated seamlessly as they entry knowledge with single sign-on to construct analytics functions with Amazon EMR (EMR Studio and Amazon EMR on EC2).

AWS Lake Formation permits knowledge directors to centrally govern, safe, and share knowledge for analytics and machine studying (ML). With trusted identification propagation, knowledge directors can immediately present granular entry to company customers utilizing their identification attributes and simplify the traceability of end-to-end knowledge entry throughout AWS companies. As a result of entry is managed based mostly on a consumer’s company identification, they don’t want to make use of database native consumer credentials or assume an IAM function to entry knowledge.

On this publish, we present how you can convey your workforce identification to EMR Studio for analytics use instances, immediately handle fine-grained permissions for the company customers and teams utilizing Lake Formation, and audit their knowledge entry.

Resolution overview

For our use case, we wish to allow an information analyst consumer named analyst1 to make use of their very own enterprise credentials to question knowledge they’ve been granted permissions to and audit their knowledge entry. We use Okta because the IdP for this demonstration. The next diagram illustrates the answer structure.

This structure relies on the next parts:

Okta is liable for sustaining the company consumer identities, associated teams, and consumer authentication.
IAM Identification Middle connects Okta customers and centrally manages their entry throughout AWS accounts and functions.
Lake Formation supplies fine-grained entry controls on knowledge on to company customers utilizing trusted identification propagation.
EMR Studio is an IDE for customers to construct and run functions. It permits customers to log in immediately with their company credentials with out signing in to the AWS Administration Console.
AWS Service Catalog supplies a product template to create EMR clusters.
EMR cluster is built-in with IAM Identification Middle utilizing a safety configuration.
AWS CloudTrail captures consumer knowledge entry actions.

The next are the high-level steps to implement the answer:

Combine Okta with IAM Identification Middle.
Arrange Amazon EMR Studio.
Create an IAM Identification Middle enabled safety configuration for EMR clusters.
Create a Service Catalog product template to create the EMR clusters.
Use Lake Formation to grant permissions to customers to entry knowledge.
Check the answer by accessing knowledge with a company identification.
Audit consumer knowledge entry.

Stipulations

You need to have the next stipulations:

Combine Okta with IAM Identification Middle

For extra details about configuring Okta with IAM Identification Middle, confer with Configure SAML and SCIM with Okta and IAM Identification Middle.

For this setup, we now have created two customers, analyst1 and engineer1, and assigned them to the corresponding Okta software. You’ll be able to validate the combination is working by navigating to the Customers web page on the IAM Identification Middle console, as proven within the following screenshot. Each enterprise customers from Okta are provisioned in IAM Identification Middle.

The next precise customers won’t be listed in your account. You’ll be able to both create comparable customers or use an present consumer.

Every provisioned consumer in IAM Identification Middle has a singular consumer ID. This ID doesn’t originate from Okta; it’s created in IAM Identification Middle to uniquely establish this consumer. With trusted identification propagation, this consumer ID might be propagated throughout companies and in addition used for traceability functions in CloudTrail. The next screenshot exhibits the IAM Identification Middle consumer matching the provisioned Okta consumer analyst1.

Select the hyperlink below AWS entry portal URL and log in with the analyst1 Okta consumer credentials which might be already assigned to this software.

If you’ll be able to log in and see the touchdown web page, then all of your configurations as much as this step are set accurately. You’ll not see any functions on this web page but.

Arrange EMR Studio

On this step, we display the actions wanted from the info lake administrator to arrange EMR Studio enabled for trusted identification propagation and with IAM Identification Middle integration. This enables customers to immediately entry EMR Studio with their enterprise credentials.

Observe: All Amazon S3 buckets (created after January 5, 2023) have encryption configured by default (Amazon S3 managed keys (SSE-S3)), and all new objects which might be uploaded to an S3 bucket are mechanically encrypted at relaxation. To make use of a distinct kind of encryption, to satisfy your safety wants, please replace the default encryption configuration for the bucket. See Defending knowledge for server-side encryption for additional particulars.

On the Amazon EMR console, select Studios within the navigation pane below EMR Studio.
Select Create Studio.

For Setup choices¸ choose Customized.
For Studio identify, enter a reputation (for this publish, emr-studio-with-tip).
For S3 location for Workspace storage, choose Choose present location and enter an present S3 bucket (if in case you have one). In any other case, choose Create new bucket.

For Service function to let Studio entry your AWS assets, select View permissions particulars to get the belief and IAM coverage info that’s wanted and create a job with these particular insurance policies in IAM. On this case, we create a brand new function known as emr_tip_role.

For Service function to let Studio entry your AWS assets, select the IAM function you created.
For Workspace identify, enter a reputation (for this publish, studio-workspace-with-tip).

For Authentication, choose IAM Identification Middle.
For Person function¸ you possibly can create a brand new function or select an present function. For this publish, we select the function we created (emr_tip_role).
To make use of the identical function, add the next assertion to the belief coverage of the service function:

{
  "Model": "2008-10-17",
  "Assertion": [
    {
      "Effect": "Allow",
      "Principal": {
        "Service": "elasticmapreduce.amazonaws.com",
 "AWS": "arn:aws:iam::xxxxxx:role/emr_tip_role"
      },
      "Action": [
              "sts:AssumeRole",
              "sts:SetContext"
              ]
    }
  ]
}

Choose Allow trusted identification propagation to permit you to management and log consumer entry throughout related functions.

For Select who can entry your software, choose All customers and teams.

Later, we limit entry to assets utilizing Lake Formation. Nonetheless, there’s an choice right here to limit entry to solely assigned customers and teams.

Within the Networking and safety part, you possibly can present non-compulsory particulars on your VPC, subnets, and safety group settings.
Select Create Studio.

On the Studios web page of the Amazon EMR console, find your Studio enabled with IAM Identification Middle.
Copy the hyperlink for Studio Entry URL.

Enter the URL into an internet browser and log in utilizing Okta credentials.

You need to be capable to efficiently check in to the EMR Studio console.

Create an AWS Identification Middle enabled safety configuration for EMR clusters

EMR safety configurations permit you to configure knowledge encryption, Kerberos authentication, and Amazon S3 authorization for the EMR File System (EMRFS) on the clusters. The safety configuration is obtainable to make use of and reuse if you create clusters.

To combine Amazon EMR with IAM Identification Middle, you want to first create an IAM function that authenticates with IAM Identification Middle from the EMR cluster. Amazon EMR makes use of IAM credentials to relay the IAM Identification Middle identification to downstream companies corresponding to Lake Formation. The IAM function must also have the respective permissions to invoke the downstream companies.

Create a job (for this publish, known as emr-idc-application) with the next belief and permission coverage. The function referenced within the belief coverage is the InstanceProfile function for EMR clusters. This enables the EC2 occasion profile to imagine this function and act as an identification dealer on behalf of the federated customers.

{
    "Model": "2012-10-17",
    "Assertion": [
        {
            "Sid": "AssumeRole",
            "Effect": "Allow",
            "Principal": {
                "AWS": "arn:aws:iam::xxxxxxxxxxn:role/service-role/AmazonEMR-InstanceProfile-20240127T102444"
            },
            "Action": [
                "sts:AssumeRole",
                "sts:SetContext"
            ]
        }
    ]
}

{
    "Model": "2012-10-17",
    "Assertion": [
        {
            "Sid": "IdCPermissions",
            "Effect": "Allow",
            "Action": [
                "sso-oauth:*"
            ],
            "Useful resource": "*"
        },
        {
            "Sid": "GlueandLakePermissions",
            "Impact": "Permit",
            "Motion": [
                "glue:*",
                "lakeformation:GetDataAccess"
            ],
            "Useful resource": "*"
        },
        {
            "Sid": "S3Permissions",
            "Impact": "Permit",
            "Motion": [
                "s3:GetDataAccess",
                "s3:GetAccessGrantsInstanceForPrefix"
            ],
            "Useful resource": "*"
        }
    ]
}

Subsequent, you create certificates for encrypting knowledge in transit with Amazon EMR.

For this publish, we use OpenSSL to generate a self-signed X.509 certificates with a 2048-bit RSA non-public key.

The important thing permits entry to the issuer’s EMR cluster cases within the AWS Area getting used. For a whole information on creating and offering a certificates, confer with Offering certificates for encrypting knowledge in transit with Amazon EMR encryption.

Add my-certs.zip to an S3 location that might be used to create the safety configuration.

The EMR service function ought to have entry to the S3 location. The important thing permits entry to the issuer’s EMR cluster cases within the us-west-2 Area as specified by the *.us-west-2.compute.inner area identify because the frequent identify. You’ll be able to change this to the Area your cluster is in.

$ openssl req -x509 -newkey rsa:2048 -keyout privateKey.pem -out certificateChain.pem -days 365 -nodes -subj '/CN=*.us-west-2.compute.inner'
$ cp certificateChain.pem trustedCertificates.pem
$ zip -r -X my-certs.zip certificateChain.pem privateKey.pem trustedCertificates.pem

Create an EMR safety configuration with IAM Identification Middle enabled from the AWS Command Line Interface (AWS CLI) with the next code:

aws emr create-security-configuration --name "IdentityCenterConfiguration-with-lf-tip" --region "us-west-2" --endpoint-url https://elasticmapreduce.us-west-2.amazonaws.com --security-configuration '{
    "AuthenticationConfiguration":{
        "IdentityCenterConfiguration":{
            "EnableIdentityCenter":true,
            "IdentityCenterApplicationAssigmentRequired":false,
            "IdentityCenterInstanceARN": "arn:aws:sso:::occasion/ssoins-7907b0d7d77e3e0d",
            "IAMRoleForEMRIdentityCenterApplicationARN": "arn:aws:iam::1xxxxxxxxx0:function/emr-idc-application"
        }
    },
    "AuthorizationConfiguration": {
        "LakeFormationConfiguration": {
            "EnableLakeFormation": true
        }
    },
    "EncryptionConfiguration": {
        "EnableInTransitEncryption": true,
        "EnableAtRestEncryption": false,
        "InTransitEncryptionConfiguration": {
            "TLSCertificateConfiguration": {
                "CertificateProviderType": "PEM",
                "S3Object": "s3://<<Bucket Title>>/emr-transit-encry-certs/my-certs.zip"
            }
        }
    }
}'

You’ll be able to view the safety configuration on the Amazon EMR console.

Create a Service Catalog product template to create EMR clusters

EMR Studio with trusted identification propagation enabled can solely work with clusters created from a template. Full the next steps to create a product template in Service Catalog:

On the Service Catalog console, select Portfolios below Administration within the navigation pane.
Select Create portfolio.

Enter a reputation on your portfolio (for this publish, EMR Clusters Template) and an non-compulsory description.
Select Create.

On the Portfolios web page, select the portfolio you simply created to view its particulars.

On the Merchandise tab, select Create product.

For Product kind, choose CloudFormation.
For Product identify, enter a reputation (for this publish, EMR-7.0.0).
Use the safety configuration IdentityCenterConfiguration-with-lf-tip you created in earlier steps with the suitable Amazon EMR service roles.
Select Create product.

The next is an instance CloudFormation template. Replace the account-specific values for SecurityConfiguration, JobFlowRole, ServiceRole, LogUri, Ec2KeyName, and Ec2SubnetId. We offer a pattern Amazon EMR service function and belief coverage in Appendix A on the finish of this publish.

'Parameters':
  'ClusterName':
    'Kind': 'String'
    'Default': 'EMR_TIP_Cluster'
  'EmrRelease':
    'Kind': 'String'
    'Default': 'emr-7.0.0'
    'AllowedValues':
    - 'emr-7.0.0'
  'ClusterInstanceType':
    'Kind': 'String'
    'Default': 'm5.xlarge'
    'AllowedValues':
    - 'm5.xlarge'
    - 'm5.2xlarge'
'Assets':
  'EmrCluster':
    'Kind': 'AWS::EMR::Cluster'
    'Properties':
      'Functions':
      - 'Title': 'Spark'
      - 'Title': 'Livy'
      - 'Title': 'Hadoop'
      - 'Title': 'JupyterEnterpriseGateway'       
      'SecurityConfiguration': 'IdentityCenterConfiguration-with-lf-tip'
      'EbsRootVolumeSize': '20'
      'Title':
        'Ref': 'ClusterName'
      'JobFlowRole': <Occasion Profile Position>
      'ServiceRole': <EMR Service Position>
      'ReleaseLabel':
        'Ref': 'EmrRelease'
      'VisibleToAllUsers': !!bool 'true'
      'LogUri':
        'Fn::Sub': <S3 LOG Path>
      'Situations':
        "Ec2KeyName" : <Key Pair Title>
        'TerminationProtected': !!bool 'false'
        'Ec2SubnetId': <subnet-id>
        'MasterInstanceGroup':
          'InstanceCount': !!int '1'
          'InstanceType':
            'Ref': 'ClusterInstanceType'
        'CoreInstanceGroup':
          'InstanceCount': !!int '2'
          'InstanceType':
            'Ref': 'ClusterInstanceType'
          'Market': 'ON_DEMAND'
          'Title': 'Core'
'Outputs':
  'ClusterId':
    'Worth':
      'Ref': 'EmrCluster'
    'Description': 'The ID of the  EMR cluster'
'Metadata':
  'AWS::CloudFormation::Designer': {}
'Guidelines': {}

Trusted identification propagation is supported from Amazon EMR 6.15 onwards. For Amazon EMR 6.15, add the next bootstrap motion to the CloudFormation script:

'BootstrapActions':
- 'Title': 'spark-config'
'ScriptBootstrapAction':
'Path': 's3://emr-data-access-control-<aws-region>/customer-bootstrap-actions/idc-fix/replace-puppet.sh'

The portfolio now ought to have the EMR cluster creation product added.

Grant the EMR Studio function emr_tip_role entry to the portfolio.

Grant Lake Formation permissions to customers to entry knowledge

On this step, we allow Lake Formation integration with IAM Identification Middle and grant permissions to the Identification Middle consumer analyst1. If Lake Formation will not be already enabled, confer with Getting began with Lake Formation.

To make use of Lake Formation with Amazon EMR, create a customized function to register S3 places. That you must create a brand new customized function with Amazon S3 entry and never use the default function AWSServiceRoleForLakeFormationDataAccess. Moreover, allow exterior knowledge filtering in Lake Formation. For extra particulars, confer with Allow Lake Formation with Amazon EMR.

Full the next steps to handle entry permissions in Lake Formation:

On the Lake Formation console, select IAM Identification Middle integration below Administration within the navigation pane.

Lake Formation will mechanically specify the right IAM Identification Middle occasion.

Now you can view the IAM Identification Middle integration particulars.

For this publish, we now have a Advertising and marketing database and a buyer desk on which we grant entry to our enterprise consumer analyst1. You should utilize an present database and desk in your account or create a brand new one. For extra examples, confer with Tutorials.

The next screenshot exhibits the main points of our buyer desk.

Full the next steps to grant analyst1 permissions. For extra info, confer with Granting desk permissions utilizing the named useful resource methodology.

On the Lake Formation console, select Information lake permissions below Permissions within the navigation pane.
Select Grant.

Choose Named Information Catalog assets.
For Databases, select your database (advertising).
For Tables, select your desk (buyer).

For Desk permissions, choose Choose and Describe.
For Information permissions, choose All knowledge entry.
Select Grant.

The next screenshot exhibits a abstract of permissions that consumer analyst1 has. They’ve Choose entry on the desk and Describe permissions on the databases.

Check the answer

To check the answer, we log in to EMR Studio as enterprise consumer analyst1, create a brand new Workspace, create an EMR cluster utilizing a template, and use that cluster to carry out an evaluation. You may additionally use the Workspace that was created throughout the Studio setup. On this demonstration, we create a brand new Workspace.

You want further permissions within the EMR Studio function to create and record Workspaces, use a template, and create EMR clusters. For extra particulars, confer with Configure EMR Studio consumer permissions for Amazon EC2 or Amazon EKS. Appendix B on the finish of this publish accommodates a pattern coverage.

When the cluster is obtainable, we connect the cluster to the Workspace and run queries on the buyer desk, which the consumer has entry to.

Person analyst1 is now capable of run queries for enterprise use instances utilizing their company identification. To open a PySpark pocket book, we select PySpark below Pocket book.

When the pocket book is open, we run a Spark SQL question to record the databases:

On this case, we question the buyer desk within the advertising database. We should always be capable to entry the info.

%%sql
choose * from advertising.buyer

Audit knowledge entry

Lake Formation API actions are logged by CloudTrail. The GetDataAccess motion is logged at any time when a principal or built-in AWS service requests non permanent credentials to entry knowledge in an information lake location that’s registered with Lake Formation. With trusted identification propagation, CloudTrail additionally logs the IAM Identification Middle consumer ID of the company identification who requested entry to the info.

The next screenshot exhibits the main points for the analyst1 consumer.

Select View occasion to view the occasion logs.

The next is an instance of the GetDataAccess occasion log. We are able to hint that consumer analyst1, Identification Middle consumer ID c8c11390-00a1-706e-0c7a-bbcc5a1c9a7f, has accessed the buyer desk.

{
    "eventVersion": "1.09",
    
….
        "onBehalfOf": {
            "userId": "c8c11390-00a1-706e-0c7a-bbcc5a1c9a7f",
            "identityStoreArn": "arn:aws:identitystore::xxxxxxxxx:identitystore/d-XXXXXXXX"
        }
    },
    "eventTime": "2024-01-28T17:56:25Z",
    "eventSource": "lakeformation.amazonaws.com",
    "eventName": "GetDataAccess",
    "awsRegion": "us-west-2",
….
        "requestParameters": {
        "tableArn": "arn:aws:glue:us-west-2:xxxxxxxxxx:desk/advertising/buyer",
        "supportedPermissionTypes": [
            "TABLE_PERMISSION"
        ]
    },
    …..
    }
}

Right here is an finish to finish demonstration video of steps to observe for enabling trusted identification propagation to your analytics movement in Amazon EMR

Clear up

Clear up the next assets if you’re finished utilizing this resolution:

Conclusion

On this publish, we demonstrated how you can arrange and use trusted identification propagation utilizing IAM Identification Middle, EMR Studio, and Lake Formation for analytics. With trusted identification propagation, a consumer’s company identification is seamlessly propagated as they entry knowledge utilizing single sign-on throughout AWS analytics companies to construct analytics functions. Information directors can present fine-grained knowledge entry on to company customers and teams and audit utilization. To study extra, see Combine Amazon EMR with AWS IAM Identification Middle.

Concerning the Authors

Pradeep Misra is a Principal Analytics Options Architect at AWS. He works throughout Amazon to architect and design trendy distributed analytics and AI/ML platform options. He’s keen about fixing buyer challenges utilizing knowledge, analytics, and AI/ML. Outdoors of labor, Pradeep likes exploring new locations, attempting new cuisines, and enjoying board video games along with his household. He additionally likes doing science experiments along with his daughters.

Deepmala Agarwal works as an AWS Information Specialist Options Architect. She is keen about serving to clients construct out scalable, distributed, and data-driven options on AWS. When not at work, Deepmala likes spending time with household, strolling, listening to music, watching films, and cooking!

Abhilash Nagilla is a Senior Specialist Options Architect at Amazon Net Companies (AWS), serving to public sector clients on their cloud journey with a deal with AWS analytics companies. Outdoors of labor, Abhilash enjoys studying new applied sciences, watching films, and visiting new locations.

Appendix A

Pattern Amazon EMR service function and belief coverage:

Observe: It is a pattern service function. Advantageous grained entry management is completed utilizing Lake Formation. Modify the permissions as per your enterprise steerage and to conform together with your safety crew.

Belief coverage:

{
    "Model": "2008-10-17",
    "Assertion": [
        {
            "Sid": "",
            "Effect": "Allow",
            "Principal": {
                "Service": "elasticmapreduce.amazonaws.com",
   "AWS": "arn:aws:iam::xxxxxx:role/emr_tip_role"

            },
            "Action": [
                "sts:AssumeRole",
                "sts:SetContext"
            ]
        }
    ]
}

Permission Coverage:

{
    "Model": "2012-10-17",
    "Assertion": [
        {
            "Sid": "ResourcesToLaunchEC2",
            "Effect": "Allow",
            "Action": [
                "ec2:RunInstances",
                "ec2:CreateFleet",
                "ec2:CreateLaunchTemplate",
                "ec2:CreateLaunchTemplateVersion"
            ],
            "Useful resource": [
                "arn:aws:ec2:*:*:network-interface/*",
                "arn:aws:ec2:*::image/ami-*",
                "arn:aws:ec2:*:*:key-pair/*",
                "arn:aws:ec2:*:*:capacity-reservation/*",
                "arn:aws:ec2:*:*:placement-group/pg-*",
                "arn:aws:ec2:*:*:fleet/*",
                "arn:aws:ec2:*:*:dedicated-host/*",
                "arn:aws:resource-groups:*:*:group/*"
            ]
        },
        {
            "Sid": "TagOnCreateTaggedEMRResources",
            "Impact": "Permit",
            "Motion": [
                "ec2:CreateTags"
            ],
            "Useful resource": [
                "arn:aws:ec2:*:*:network-interface/*",
                "arn:aws:ec2:*:*:instance/*",
                "arn:aws:ec2:*:*:volume/*",
                "arn:aws:ec2:*:*:launch-template/*"
            ],
            "Situation": {
                "StringEquals": {
                    "ec2:CreateAction": [
                        "RunInstances",
                        "CreateFleet",
                        "CreateLaunchTemplate",
                        "CreateNetworkInterface"
                    ]
                }
            }
        },
        {
            "Sid": "ListActionsForEC2Resources",
            "Impact": "Permit",
            "Motion": [
                "ec2:DescribeAccountAttributes",
                "ec2:DescribeCapacityReservations",
                "ec2:DescribeDhcpOptions",
                "ec2:DescribeImages",
                "ec2:DescribeInstances",
                "ec2:DescribeLaunchTemplates",
                "ec2:DescribeNetworkAcls",
                "ec2:DescribeNetworkInterfaces",
                "ec2:DescribePlacementGroups",
                "ec2:DescribeRouteTables",
                "ec2:DescribeSecurityGroups",
                "ec2:DescribeSubnets",
                "ec2:DescribeVolumes",
                "ec2:DescribeVolumeStatus",
                "ec2:DescribeVpcAttribute",
                "ec2:DescribeVpcEndpoints",
                "ec2:DescribeVpcs"
            ],
            "Useful resource": "*"
        },
        {
            "Sid": "AutoScaling",
            "Impact": "Permit",
            "Motion": [
                "application-autoscaling:DeleteScalingPolicy",
                "application-autoscaling:DeregisterScalableTarget",
                "application-autoscaling:DescribeScalableTargets",
                "application-autoscaling:DescribeScalingPolicies",
                "application-autoscaling:PutScalingPolicy",
                "application-autoscaling:RegisterScalableTarget"
            ],
            "Useful resource": "*"
        },
        {
            "Sid": "AutoScalingCloudWatch",
            "Impact": "Permit",
            "Motion": [
                "cloudwatch:PutMetricAlarm",
                "cloudwatch:DeleteAlarms",
                "cloudwatch:DescribeAlarms"
            ],
            "Useful resource": "arn:aws:cloudwatch:*:*:alarm:*_EMR_Auto_Scaling"
        },
        {
            "Sid": "PassRoleForAutoScaling",
            "Impact": "Permit",
            "Motion": "iam:PassRole",
            "Useful resource": "arn:aws:iam::*:function/EMR_AutoScaling_DefaultRole",
            "Situation": {
                "StringLike": {
                    "iam:PassedToService": "application-autoscaling.amazonaws.com*"
                }
            }
        },
        {
            "Sid": "PassRoleForEC2",
            "Impact": "Permit",
            "Motion": "iam:PassRole",
            "Useful resource": "arn:aws:iam::xxxxxxxxxxx:function/service-role/<Occasion-Profile-Position>",
            "Situation": {
                "StringLike": {
                    "iam:PassedToService": "ec2.amazonaws.com*"
                }
            }
        },
        {
            "Impact": "Permit",
            "Motion": [
                "s3:*",
                "s3-object-lambda:*"
            ],
            "Useful resource": [
                "arn:aws:s3:::<bucket>/*",
                "arn:aws:s3:::*logs*/*"
            ]
        },
        {
            "Impact": "Permit",
            "Useful resource": "*",
            "Motion": [
                "ec2:AuthorizeSecurityGroupEgress",
                "ec2:AuthorizeSecurityGroupIngress",
                "ec2:CancelSpotInstanceRequests",
                "ec2:CreateFleet",
                "ec2:CreateLaunchTemplate",
                "ec2:CreateNetworkInterface",
                "ec2:CreateSecurityGroup",
                "ec2:CreateTags",
                "ec2:DeleteLaunchTemplate",
                "ec2:DeleteNetworkInterface",
                "ec2:DeleteSecurityGroup",
                "ec2:DeleteTags",
                "ec2:DescribeAvailabilityZones",
                "ec2:DescribeAccountAttributes",
                "ec2:DescribeDhcpOptions",
                "ec2:DescribeImages",
                "ec2:DescribeInstanceStatus",
                "ec2:DescribeInstances",
                "ec2:DescribeKeyPairs",
                "ec2:DescribeLaunchTemplates",
                "ec2:DescribeNetworkAcls",
                "ec2:DescribeNetworkInterfaces",
                "ec2:DescribePrefixLists",
                "ec2:DescribeRouteTables",
                "ec2:DescribeSecurityGroups",
                "ec2:DescribeSpotInstanceRequests",
                "ec2:DescribeSpotPriceHistory",
                "ec2:DescribeSubnets",
                "ec2:DescribeTags",
                "ec2:DescribeVpcAttribute",
                "ec2:DescribeVpcEndpoints",
                "ec2:DescribeVpcEndpointServices",
                "ec2:DescribeVpcs",
                "ec2:DetachNetworkInterface",
                "ec2:ModifyImageAttribute",
                "ec2:ModifyInstanceAttribute",
                "ec2:RequestSpotInstances",
                "ec2:RevokeSecurityGroupEgress",
                "ec2:RunInstances",
                "ec2:TerminateInstances",
                "ec2:DeleteVolume",
                "ec2:DescribeVolumeStatus",
                "ec2:DescribeVolumes",
                "ec2:DetachVolume",
                "iam:GetRole",
                "iam:GetRolePolicy",
                "iam:ListInstanceProfiles",
                "iam:ListRolePolicies",
                "cloudwatch:PutMetricAlarm",
                "cloudwatch:DescribeAlarms",
                "cloudwatch:DeleteAlarms",
                "application-autoscaling:RegisterScalableTarget",
                "application-autoscaling:DeregisterScalableTarget",
                "application-autoscaling:PutScalingPolicy",
                "application-autoscaling:DeleteScalingPolicy",
                "application-autoscaling:Describe*"
            ]
        }
    ]
}

Appendix B

Pattern EMR Studio function coverage:

{
    "Model": "2012-10-17",
    "Assertion": [
        {
            "Sid": "AllowEMRReadOnlyActions",
            "Effect": "Allow",
            "Action": [
                "elasticmapreduce:ListInstances",
                "elasticmapreduce:DescribeCluster",
                "elasticmapreduce:ListSteps"
            ],
            "Useful resource": "*"
        },
        {
            "Sid": "AllowEC2ENIActionsWithEMRTags",
            "Impact": "Permit",
            "Motion": [
                "ec2:CreateNetworkInterfacePermission",
                "ec2:DeleteNetworkInterface"
            ],
            "Useful resource": [
                "arn:aws:ec2:*:*:network-interface/*"
            ],
            "Situation": {
                "StringEquals": {
                    "aws:ResourceTag/for-use-with-amazon-emr-managed-policies": "true"
                }
            }
        },
        {
            "Sid": "AllowEC2ENIAttributeAction",
            "Impact": "Permit",
            "Motion": [
                "ec2:ModifyNetworkInterfaceAttribute"
            ],
            "Useful resource": [
                "arn:aws:ec2:*:*:instance/*",
                "arn:aws:ec2:*:*:network-interface/*",
                "arn:aws:ec2:*:*:security-group/*"
            ]
        },
        {
            "Sid": "AllowEC2SecurityGroupActionsWithEMRTags",
            "Impact": "Permit",
            "Motion": [
                "ec2:AuthorizeSecurityGroupEgress",
                "ec2:AuthorizeSecurityGroupIngress",
                "ec2:RevokeSecurityGroupEgress",
                "ec2:RevokeSecurityGroupIngress",
                "ec2:DeleteNetworkInterfacePermission"
            ],
            "Useful resource": "*",
            "Situation": {
                "StringEquals": {
                    "aws:ResourceTag/for-use-with-amazon-emr-managed-policies": "true"
                }
            }
        },
        {
            "Sid": "AllowDefaultEC2SecurityGroupsCreationWithEMRTags",
            "Impact": "Permit",
            "Motion": [
                "ec2:CreateSecurityGroup"
            ],
            "Useful resource": [
                "arn:aws:ec2:*:*:security-group/*"
            ],
            "Situation": {
                "StringEquals": {
                    "aws:RequestTag/for-use-with-amazon-emr-managed-policies": "true"
                }
            }
        },
        {
            "Sid": "AllowDefaultEC2SecurityGroupsCreationInVPCWithEMRTags",
            "Impact": "Permit",
            "Motion": [
                "ec2:CreateSecurityGroup"
            ],
            "Useful resource": [
                "arn:aws:ec2:*:*:vpc/*"
            ],
            "Situation": {
                "StringEquals": {
                    "aws:ResourceTag/for-use-with-amazon-emr-managed-policies": "true"
                }
            }
        },
        {
            "Sid": "AllowAddingEMRTagsDuringDefaultSecurityGroupCreation",
            "Impact": "Permit",
            "Motion": [
                "ec2:CreateTags"
            ],
            "Useful resource": "arn:aws:ec2:*:*:security-group/*",
            "Situation": {
                "StringEquals": {
                    "aws:RequestTag/for-use-with-amazon-emr-managed-policies": "true",
                    "ec2:CreateAction": "CreateSecurityGroup"
                }
            }
        },
        {
            "Sid": "AllowEC2ENICreationWithEMRTags",
            "Impact": "Permit",
            "Motion": [
                "ec2:CreateNetworkInterface"
            ],
            "Useful resource": [
                "arn:aws:ec2:*:*:network-interface/*"
            ],
            "Situation": {
                "StringEquals": {
                    "aws:RequestTag/for-use-with-amazon-emr-managed-policies": "true"
                }
            }
        },
        {
            "Sid": "AllowEC2ENICreationInSubnetAndSecurityGroupWithEMRTags",
            "Impact": "Permit",
            "Motion": [
                "ec2:CreateNetworkInterface"
            ],
            "Useful resource": [
                "arn:aws:ec2:*:*:subnet/*",
                "arn:aws:ec2:*:*:security-group/*"
            ],
            "Situation": {
                "StringEquals": {
                    "aws:ResourceTag/for-use-with-amazon-emr-managed-policies": "true"
                }
            }
        },
        {
            "Sid": "AllowAddingTagsDuringEC2ENICreation",
            "Impact": "Permit",
            "Motion": [
                "ec2:CreateTags"
            ],
            "Useful resource": "arn:aws:ec2:*:*:network-interface/*",
            "Situation": {
                "StringEquals": {
                    "ec2:CreateAction": "CreateNetworkInterface"
                }
            }
        },
        {
            "Sid": "AllowEC2ReadOnlyActions",
            "Impact": "Permit",
            "Motion": [
                "ec2:DescribeSecurityGroups",
                "ec2:DescribeNetworkInterfaces",
                "ec2:DescribeTags",
                "ec2:DescribeInstances",
                "ec2:DescribeSubnets",
                "ec2:DescribeVpcs"
            ],
            "Useful resource": "*"
        },
        {
            "Sid": "AllowSecretsManagerReadOnlyActionsWithEMRTags",
            "Impact": "Permit",
            "Motion": [
                "secretsmanager:GetSecretValue"
            ],
            "Useful resource": "arn:aws:secretsmanager:*:*:secret:*",
            "Situation": {
                "StringEquals": {
                    "aws:ResourceTag/for-use-with-amazon-emr-managed-policies": "true"
                }
            }
        },
        {
            "Sid": "AllowWorkspaceCollaboration",
            "Impact": "Permit",
            "Motion": [
                "iam:GetUser",
                "iam:GetRole",
                "iam:ListUsers",
                "iam:ListRoles",
                "sso:GetManagedApplicationInstance",
                "sso-directory:SearchUsers"
            ],
            "Useful resource": "*"
        },
        {
            "Sid": "S3Access",
            "Impact": "Permit",
            "Motion": [
                "s3:PutObject",
                "s3:GetObject",
                "s3:GetEncryptionConfiguration",
                "s3:ListBucket",
                "s3:DeleteObject"
            ],
            "Useful resource": [
                "arn:aws:s3:::<bucket>",
                "arn:aws:s3:::<bucket>/*"
            ]
        },
        {
            "Sid": "EMRStudioWorkspaceAccess",
            "Impact": "Permit",
            "Motion": [
                "elasticmapreduce:CreateEditor",
                "elasticmapreduce:DescribeEditor",
                "elasticmapreduce:ListEditors",
                "elasticmapreduce:DeleteEditor",
                "elasticmapreduce:UpdateEditor",
                "elasticmapreduce:PutWorkspaceAccess",
                "elasticmapreduce:DeleteWorkspaceAccess",
                "elasticmapreduce:ListWorkspaceAccessIdentities",
                "elasticmapreduce:StartEditor",
                "elasticmapreduce:StopEditor",
                "elasticmapreduce:OpenEditorInConsole",
                "elasticmapreduce:AttachEditor",
                "elasticmapreduce:DetachEditor",
                "elasticmapreduce:ListInstanceGroups",
                "elasticmapreduce:ListBootstrapActions",
                "servicecatalog:SearchProducts",
                "servicecatalog:DescribeProduct",
                "servicecatalog:DescribeProductView",
                "servicecatalog:DescribeProvisioningParameters",
                "servicecatalog:ProvisionProduct",
                "servicecatalog:UpdateProvisionedProduct",
                "servicecatalog:ListProvisioningArtifacts",
                "servicecatalog:DescribeRecord",
                "servicecatalog:ListLaunchPaths",
                "elasticmapreduce:RunJobFlow",      
                "elasticmapreduce:ListClusters",
                "elasticmapreduce:DescribeCluster",
                "codewhisperer:GenerateRecommendations",
                "athena:StartQueryExecution",
                "athena:StopQueryExecution",
                "athena:GetQueryExecution",
                "athena:GetQueryRuntimeStatistics",
                "athena:GetQueryResults",
                "athena:ListQueryExecutions",
                "athena:BatchGetQueryExecution",
                "athena:GetNamedQuery",
                "athena:ListNamedQueries",
                "athena:BatchGetNamedQuery",
                "athena:UpdateNamedQuery",
                "athena:DeleteNamedQuery",
                "athena:ListDataCatalogs",
                "athena:GetDataCatalog",
                "athena:ListDatabases",
                "athena:GetDatabase",
                "athena:ListTableMetadata",
                "athena:GetTableMetadata",
                "athena:ListWorkGroups",
                "athena:GetWorkGroup",
                "athena:CreateNamedQuery",
                "athena:GetPreparedStatement",
                "glue:CreateDatabase",
                "glue:DeleteDatabase",
                "glue:GetDatabase",
                "glue:GetDatabases",
                "glue:UpdateDatabase",
                "glue:CreateTable",
                "glue:DeleteTable",
                "glue:BatchDeleteTable",
                "glue:UpdateTable",
                "glue:GetTable",
                "glue:GetTables",
                "glue:BatchCreatePartition",
                "glue:CreatePartition",
                "glue:DeletePartition",
                "glue:BatchDeletePartition",
                "glue:UpdatePartition",
                "glue:GetPartition",
                "glue:GetPartitions",
                "glue:BatchGetPartition",
                "kms:ListAliases",
                "kms:ListKeys",
                "kms:DescribeKey",
                "lakeformation:GetDataAccess",
                "s3:GetBucketLocation",
                "s3:GetObject",
                "s3:ListBucket",
                "s3:ListBucketMultipartUploads",
                "s3:ListMultipartUploadParts",
                "s3:AbortMultipartUpload",
                "s3:PutObject",
                "s3:PutBucketPublicAccessBlock",
                "s3:ListAllMyBuckets",
                "elasticmapreduce:ListStudios",
                "elasticmapreduce:DescribeStudio",
                "cloudformation:GetTemplate",
                "cloudformation:CreateStack",
                "cloudformation:CreateStackSet",
                "cloudformation:DeleteStack",
                "cloudformation:GetTemplateSummary",
                "cloudformation:ValidateTemplate",
                "cloudformation:ListStacks",
                "cloudformation:ListStackSets",
                "elasticmapreduce:AddTags",
                "ec2:CreateNetworkInterface",
                "elasticmapreduce:GetClusterSessionCredentials",
                "elasticmapreduce:GetOnClusterAppUIPresignedURL",
                "cloudformation:DescribeStackResources"
            ],
            "Useful resource": [
                "*"
            ]
        },
        {
            "Sid": "AllowPassingServiceRoleForWorkspaceCreation",
            "Motion": "iam:PassRole",
            "Useful resource": [
                "arn:aws:iam::*:role/<Studio Role>",
                "arn:aws:iam::*:role/<EMR Service Role>",
                "arn:aws:iam::*:role/<EMR Instance Profile Role>"
            ],
            "Impact": "Permit"
        },
{
			"Sid": "Statement1",
			"Impact": "Permit",
			"Motion": [
				"iam:PassRole"
			],
			"Useful resource": [
				"arn:aws:iam::*:role/<EMR Instance Profile Role>"
			]
		}
    ]
}

Use your company identities for analytics with Amazon EMR and AWS IAM Identification Middle

Resolution overview

Stipulations

Combine Okta with IAM Identification Middle

Arrange EMR Studio

Create an AWS Identification Middle enabled safety configuration for EMR clusters

Create a Service Catalog product template to create EMR clusters

Grant Lake Formation permissions to customers to entry knowledge

Check the answer

Audit knowledge entry

Clear up

Conclusion

Concerning the Authors

Appendix A

Appendix B

Recent Articles

The best way to copy a desk from PDF to Excel: 8 strategies defined

Learn how to Flash, Replace and Configure AM32 ESC (Backup & Restore Settings)

Scientific Insights Into Lengthy COVID’s Retreat – NanoApps Medical – Official web site

Google’s 2024 foldable is the Pixel 9 Professional Fold

Sensible Makes use of of AI in Ecommerce

Related Stories

Leave A Reply Cancel reply

Stay on op - Ge the daily news in your inbox