Cloud Integrations

Set up Chef Automate to detect and monitor the nodes in your AWS EC2 and Azure accounts by providing your credentials in the control panel and creating a node manager. Chef Automate creates a node reference for each instance in your account. Associate your EC2 and Azure instances with ssh and WinRM credentials using tags–the values supports wildcard match–in your node manager. Run scan jobs with your node manager reference and you’re suddenly running an inspec exec across your instances. Every two hours, Chef Automate queries your AWS or Azure account to see the current state of all your nodes, if they are running, stopped, or terminated, and then updates Chef Automate accordingly. If the node manager finds an instance that used to be running and reachable, but which no is–if the node is stopped, terminated, or a transition state–it updates the status of that node in Chef Automate accordingly.

Scan AWS EC2 with Chef Automate

Set up Chef Automate to detect and scan the nodes in your AWS EC2 account by providing your AWS Credentials and creating an AWS EC2 Node Manager from the Chef Automate Control Panel. Chef Automate requires your information to detect the nodes in your AWS EC2 account. Chef Automate creates a node reference for each EC2 instance in your account and collects all of the tags associated with each instance.

Adding an AWS EC2 Node Manager

To create an AWS EC2 Node Manager, you need the following information:

  1. A name for your manager
  2. Your AWS credentials (access key id and secret access key)

Chef Automate Create AWS-EC2 Manager

At the bottom of the screen, there is an option to associate ssh or WinRM credentials with your EC2 instances using tag keys or values, which supports wildcard matching, and which is useful for grouping nodes. Chef Automate detects your nodes immediately after any update to the Node Manager, in order to maintain a current list of your node status. The following example uses tag with the key ‘Name’ and the value ‘vj-’ to associate those nodes with the ‘ssh ec22’ credential.

Chef Automate Instance Credentials

Create a Scan Job Targeting Your AWS EC2 Instances

From the Scan Jobs tab, select the “Create new job” button.

Filter instances for scanning by specifying either regions or tags by their keys and values.

Chef Automate Create AWS-EC2 Scan Job

AWS EC2 Node Discovery

The service makes these API calls:

  • STS-GetCallerIdentity
  • EC2-DescribeRegions
  • EC2-DescribeInstances
  • EC2-DescribeInstanceStatus
  • IAM-ListAccountAliases

Chef Automate’s Node Manager discovers EC2 instances by:

  • Polling: Chef Automate’s Node Manager calls out to the AWS DescribeInstanceStatus API every two hours and discovers the state of all the instances in the account. If the node manager finds any instances that aren’t in its database, it adds them. This sometimes results in “bare bones info” and stopped instances in the database. The node mangar updates node information in the database after an instance returnes to a running state and a scan job has run on the node.
  • Scan Jobs: Whenever a scan job is triggered, the node manager queries the AWS API for all nodes. Any scan reports created for nodes that are not already in the database results in creating a new node in the database.

AWS Account Scanning with Chef Automate

Inspec 2+ supports running scan jobs against your AWS account configuration, such as CloudWatch or IAM, see more here. Set up Chef Automate to run these scan jobs by providing your AWS Credentials and creating an AWS API Node Manager in the Control Panel.

Add an AWS API Node Manager

When creating an AWS API Node Manager, you need:

  1. A name for your manager
  2. Your AWS credentials (access key id and secret access key)

This information is required to detect all regions available to your AWS account. Chef Automate creates a nodes reference for each region in your account. Most tests for account configurations are global, but a few are region-specific.

Chef Automate Create AWS-API Manager

Create a Scan Job Targeting Your AWS Account Configuration

Filter the regions for the scan job by specifying regions to include or exclude.

Chef Automate Create AWS-API Scan Job

AWS API Scanning Endpoints

The service makes calls to these API:

  • STS-GetCallerIdentity
  • SEC2-DescribeRegions
  • IAM-ListAccountAliases

Permissions: You’ll need at least a global read permission; arn:aws:iam::aws:policy/ReadOnlyAccess

AWS Credential-less Scanning with Chef Automate

For users running Chef Automate 2 in EC2, we invite you to try out our “AWS-EC2 Credential-less Scanning”!

Ensure Minimum Permissions

Ensure the policy attached to the role used by the instance you have Chef Automate running on has at least these permissions:

{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Effect": "Allow",
            "Action": [
                "ec2:DescribeInstances",
                "ssm:*",
                "ec2:DescribeRegions",
                "sts:GetCallerIdentity",
                "ec2:DescribeInstanceStatus",
                "iam:ListAccountAliases"
            ],
            "Resource": "*"
        }
    ]
}
`"ssm:*"` uses a wildcard match on the AWS EC2 Systems Manager (SSM); You may wish to use a more restrictive policy.

Install AWS EC2 Systems Manager on Instances

Please follow the instructions on AWS

Enable AWS EC2 Systems Manager on Instances

In order to use the SSM scan job functionality, your instances must have access to AmazonEC2RoleforSSM, or arn:aws:iam::aws:policy/service-role/AmazonEC2RoleforSSM.

Adding an AWS EC2 Node Manager Using IAM Credentials

When running in EC2, AWS has the ability to use the IAM role associated with your instance to create and use temporary credentials for accessing the AWS API. If you enable this feature, then you won’t need to provide credentials for your AWS account. You will only be required to provide a name for your node manager. Chef Automate creates a nodes reference for each EC2 instance in your account, collecting all tags associated with each instance. Chef Automate calls out to Amazon System Manager (SSM) describe instance information to get ping status for the SSM agent on all instances. A detect job is not run on the instances; all instances with an SSM ping status of “Online” will be marked as reachable.

Chef Automate Create AWS-EC2 Manager No Creds

Create a Scan Job Targeting Your AWS EC2 Instances using AWS SSM

The ssm Scan Job:

  1. Installs the latest stable InSpec from packages.chef.io
  2. Executes InSpec locally, providing InSpec with the fqdn of Chef Automate and a data collector token, so each instance reports directly back to Chef Automate

Your Automate instance must be reachable (open to incoming traffic) from the instances being scanned in order for the SSM scanning to work. You can filter the instances to be scanned by specifying tag key/value matches or regions.

Chef Automate Create AWS-EC2 Scan Job

AWS Credential-less Scanning

The service makes these API calls:

  • STS-GetCallerIdentity
  • EC2-DescribeRegions
  • EC2-DescribeInstances
  • EC2-DescribeInstanceStatus
  • SSM-DescribeInstanceInformation
  • SSM-SendCommand
  • SSM-ListCommands

Azure VM Scanning with Chef Automate (BETA)

Set up Chef Automate to detect and scan the nodes in your Azure account by providing your Azure Credentials and creating an Azure VM Node Manager. To access the Azure Node Manager, select anywhere on the Automate screen and type ‘beta’. After toggling the Azure Node Manager to the “ON” position, close the helper window and then refresh your browser. Now, when you open node integration on the Control Panel, you should see Azure as one of your node management service options.

Adding an Azure VM Node Manager

When creating an Azure VM Node Manager, you will be required to provide:

  1. A name for your manager
  2. Your Azure credentials (client id, client secret, and tenant id)

This information is required to detect the nodes in your Azure account. Chef Automate creates a nodes reference for each VM in your account, reading in all tags associated with each instance. Chef Automate detects your nodes immediately after any update to the Node Manager, in order to maintain a current list of your node status. The following example uses tag with the key ‘Name’ and the value ‘vj-’ to associate those nodes with the ‘ssh ec22’ credential.

Chef Automate Create Azure-VM Manager

Chef Automate uses Azure’s RunCommand functionality to run scan jobs on instances without needing ssh and WinRM credentials. In order for this functionality to work, the Automate instance must be reachable (open to incoming traffic) from the instances being scanned.

You also have the option of using the traditional ssh and WinRM scanning by providing such credentials for the VMs. At the bottom of the screen, there is an option to associate ssh and WinRM credentials with your VMs using tag key/values (supports wildcard match) to group nodes. Chef Automate detects your nodes immediately after any update to the Node Manager, keeping a current view of your nodes’ reachability.

Chef Automate Instance Credentials

Create a Scan Job Targeting Your Azure VMs

Filter the regions for the scan job by specifying regions to include or exclude.

Chef Automate Create Azure-VM Scan Job

Filter instances for scanning by specifying either regions or tags by their keys and values.

Use Case: Azure Account Scanning with Chef Automate (BETA)

Inspec 2+ supports running scan jobs against your Azure account configuration, such as network security groups and ad users. See Azure resources for more information. Set up Chef Automate to run these scan jobs by providing your Azure credentials and creating an Azure API Node Manager.

Adding an Azure API Node Manager

When creating an Azure API Node Manager, you will be required to provide:

  1. A name for your manager
  2. Your Azure credentials (client id, client secret, and tenant id)

This information is required to detect all subscriptions available to your Azure account. Chef Automate creates a nodes reference for each subscription in your account.

Chef Automate Create Azure-API Manager

Create a Scan Job Targeting Your Azure Account Configuration

From the Scan Jobs tab, select the “Create new job” button.

Filter the regions for the scan job by specifying regions to include or exclude.

Chef Automate Create Azure-API Scan Job

Google Cloud Platform Account Scanning with Chef Automate (BETA - as of 20181020020209)

Run scans against your GCP account infrastructure using Chef Automate. Set up Chef Automate to detect and scan the nodes in your Google Cloud Platform (GCP) account by providing your GCP Credentials and creating a GCP VM Node Manager. To access the GCP Node Manager, select anywhere on the Automate screen and type ‘beta’. After toggling the Google Cloud Node Manager to the “ON” position, close the helper window and then refresh your browser. Now, when you open node integration on the Control Panel, you should see Google Cloud as one of your node management service options. Available resources can be found on the InSpec docs site, https://www.inspec.io/docs/reference/resources/#gcp-resources

To run a GCP scan in Chef Automate:

  1. Add a GCP-API Node Manager using a service account json credential
  2. The service adds a node reference in the database for the project tied to the service account credential
  3. Execute a profile against the project reference with a scan job

This information is required to detect all subscriptions available to your Azure account. Chef Automate creates a nodes reference for each subscription in your account.

Note: The service account json credential requires the following fields: type, project_id, client_id, private_key_id, private_key, client_email, auth_uri, token_uri, auth_provider_x509_cert_url, client_x509_cert_url

Chef Automate Create GCP-API Integration

Create a Scan Job Targeting Your GCP Account Configuration

From the Scan Jobs tab, select the “Create new job” button.

Filter the regions for the scan job by specifying regions to include or exclude.

Chef Automate Create GCP-API Scan Job