POL-960 Update AWS Reserved Instances Utilization - add ability to fi…

…lter by Billing Center (#1602)

* Update utilization_ris.pt

- Added functionality to support filtering on Reservations by Billing Centers
- Neatened up code

* Update README.md

- Added new parameter to README
- Updated Credential Configuration section to reflect correct credentials

* Update CHANGELOG.md

* Update CHANGELOG.md

Fixed missing blank line

* Update utilization_ris.pt

* Update README.md

- Removed reference to RightScale
- Updated language in "What it does" section
- Updated "Cost" section

cost/aws/reserved_instances/utilization/CHANGELOG.md

@@ -1,5 +1,13 @@
 # Changelog
 
+## v2.1
+
+- Added new parameter to allow users to filter reservations by a list of specific billing centers.
+- Updated `param_utilization` parameter to have a clearer description, and to set the default utilization threshold to 100%.
+- Removed `permission` declaration block as no longer needed.
+- Updated README title to reflect policy template name.
+- Updated "Credential Configuration" section in README to reflect the correct credentials required to run the policy.
+
 ## v2.0
 
 - Deprecated `rs` authentication (type: `rightscale`) and replaced with `auth_flexera` (type: `oauth2`).  This is a breaking change which requires a Credential for `auth_flexera` [`provider=flexera`] before the policy can be applied.  Please see docs for setting up [Provider-Specific Credentials](https://docs.flexera.com/flexera/EN/Automation/ProviderCredentials.htm)
@@ -27,7 +35,7 @@
 
 ## v1.9
 
-- update policy name to reflect supported cloud.
+- Updated policy name to reflect supported cloud.
 
 ## v1.8
 

cost/aws/reserved_instances/utilization/README.md

@@ -1,16 +1,15 @@
-# AWS Reserved Instances Utilization Policy Template
-
-**As a best practice, this policy should only be applied to the Master Account, and not to each individual RightScale Account.**
+# AWS Reserved Instances Utilization
 
 ## What it does
 
-This Policy Template leverages the AWS RI report. It will notify only if utilization of a RI falls below the value specified in the `Show RI's with utilization below this value` field. It will email the user specified in `Email addresses of the recipients you wish to notify`
+This Policy Template leverages the AWS Reserved Instances (RI) report. It will notify only if utilization of an RI falls below the value specified in the `Show Reservations with utilization below this value` field. It will email the user specified in `Email addresses of the recipients you wish to notify`
 
 ## Input Parameters
 
 This policy has the following input parameters required when launching the policy.
 
-- *Show RI's with utilization below this value*
+- *Billing Center Name* - Filter reservations for a specific Billing Center/s by entering Billing Center names
+- *Show Reservations with utilization below this value (%)* - Number between 1 and 100
 - *Email addresses of the recipients you wish to notify* - A list of email addresses to notify
 
 ## Policy Actions
@@ -19,16 +18,19 @@ The following policy actions are taken on any resources found to be out of compl
 
 - Send an email report
 
-## Required RightScale Roles
+## Prerequisites
+
+This Policy Template uses [Credentials](https://docs.flexera.com/flexera/EN/Automation/ManagingCredentialsExternal.htm) for authenticating to datasources -- in order to apply this policy you must have a Credential registered in the system that is compatible with this policy. If there are no Credentials listed when you apply the policy, please contact your Flexera Org Admin and ask them to register a Credential that is compatible with this policy. The information below should be consulted when creating the credential(s).
+
+- [**Flexera Credential**](https://docs.flexera.com/flexera/EN/Automation/ProviderCredentials.htm) (*provider=flexera*) which has the following roles:
+  - `billing_center_viewer`
 
-- Cloud Management - Actor
-- Cloud Management - Observer
-- Cloud Management - credential_viewer
+The [Provider-Specific Credentials](https://docs.flexera.com/flexera/EN/Automation/ProviderCredentials.htm) page in the docs has detailed instructions for setting up Credentials for the most common providers.
 
 ## Supported Clouds
 
 - AWS
 
 ## Cost
 
-This Policy Template does not launch any instances, and so does not incur any cloud costs.
+This Policy Template does not incur any cloud costs.

cost/aws/reserved_instances/utilization/utilization_ris.pt

@@ -5,43 +5,177 @@ short_description "A policy that sends email notifications when utilization fall
 long_description ""
 severity "high"
 category "Cost"
-tenancy "single"
 default_frequency "daily"
 info(
-  version: "2.0",
+  version: "2.1",
   provider: "Flexera Optima",
   service: "",
   policy_set: ""
 )
 
-permission "optima_aws_ri" do
-  label "Access Optima Resources"
-  resources "rs_optima.aws_reserved_instances"
-  actions "rs_optima.index"
-end
-
-##################
-# User inputs    #
-##################
+###############################################################################
+# Parameters
+###############################################################################
 
 parameter "param_utilization" do
   category "RI"
-  label "Show RI's with utilization below this value"
+  label "Show Reservations with utilization below this value (%)"
   type "number"
+  min_value 1
+  max_value 100
+  default 100
 end
 
 parameter "param_email" do
   label "Email addresses of the recipients you wish to notify"
   type "list"
 end
 
+parameter "param_billing_centers" do
+  label "Billing Center Name"
+  description "Filter reservations for a specific Billing Center/s by entering Billing Center names"
+  type "list"
+  default []
+end
+
+###############################################################################
+# Authentication
+###############################################################################
+
 credentials "auth_flexera" do
   schemes "oauth2"
-  label "flexera"
+  label "Flexera"
   description "Select Flexera One OAuth2 credentials"
   tags "provider=flexera"
 end
 
+###############################################################################
+# Datasources and Scripts
+###############################################################################
+
+#GET ALL BILLING CENTERS FOR ORG
+datasource "ds_billing_centers" do
+  request do
+    auth $auth_flexera
+    host rs_optima_host
+    path join(["/analytics/orgs/", rs_org_id, "/billing_centers"])
+    header "Api-Version", "1.0"
+    header "User-Agent", "RS Policies"
+    query "view", "allocation_table"
+  end
+  result do
+    encoding "json"
+    collect jmes_path(response, "[*]") do
+      field "href", jmes_path(col_item, "href")
+      field "id", jmes_path(col_item, "id")
+      field "name", jmes_path(col_item, "name")
+      field "parent_id", jmes_path(col_item, "parent_id")
+      field "ancestor_ids", jmes_path(col_item,"ancestor_ids")
+      field "allocation_table", jmes_path(col_item,"allocation_table")
+    end
+  end
+end
+
+#FILTER USING BILLING CENTERS PARAMETER IF APPLICABLE
+datasource "ds_filtered_billing_centers" do
+  run_script $js_filter_billing_centers_with_param, $ds_billing_centers, $param_billing_centers
+end
+
+script "js_filter_billing_centers_with_param", type: "javascript" do
+  parameters "ds_billing_centers", "param_billing_centers"
+  result "result"
+  code <<-EOS
+
+  //Get billing centers. If user specifies no billing centers, retrieve all billing centers. Else get array of billing centers that match the names stated in BC param
+  if (param_billing_centers.length === 0) {
+    result = ds_billing_centers
+  } else {
+		var billing_center_names_ids = _.map(param_billing_centers, function(name) { return name.toLowerCase() })
+    var filtered_bcs = _.filter(ds_billing_centers, function(bc) {
+			return _.contains(billing_center_names_ids, bc.name.toLowerCase()) || _.contains(billing_center_names_ids, bc.id.toLowerCase())
+    })
+
+    //Check that there are no child billing centers specified that conflict with parent billing centers specified
+    var bc_ids = _.map(filtered_bcs, function(bc) { return bc.id.toLowerCase() })
+
+		var conflicting_child_bcs = _.filter(filtered_bcs, function(bc) {
+      if (!(bc.parent_id == null || bc.parent_id == undefined)) {
+        return _.contains(bc_ids, bc.parent_id.toLowerCase())
+      }
+    })
+		
+    if (conflicting_child_bcs != undefined) {
+			conflicting_child_bc_ids = _.pluck(conflicting_child_bcs, "id")
+			result = _.reject(filtered_bcs, function(bc) {
+				return _.contains(conflicting_child_bc_ids, bc.id)
+			})
+    } else {
+			result = filtered_bcs
+		}
+  }
+  EOS
+end
+
+#GET LIST OF ACCOUNTS FOR BILLING CENTERS 
+datasource "ds_accounts_by_billing_center" do
+  request do
+    run_script $js_get_accounts_by_bc, rs_optima_host, rs_org_id, $ds_filtered_billing_centers
+  end
+  result do
+    encoding "json"
+    collect jmes_path(response, "rows[*]") do
+      field "cost", jmes_path(col_item, "metrics.cost_amortized_unblended_adj")
+      field "billing_center_id", jmes_path(col_item, "dimensions.billing_center_id")
+      field "vendor_account_id", jmes_path(col_item, "dimensions.vendor_account")
+      field "vendor_account_name", jmes_path(col_item, "dimensions.vendor_account_name")
+      field "timestamp", jmes_path(col_item, "timestamp")
+    end
+  end
+end
+
+script "js_get_accounts_by_bc", type: "javascript" do
+  parameters "rs_optima_host", "org_id", "ds_filtered_billing_centers"
+  result "request"
+  code <<-EOS
+
+  //Get start and end date for previous month
+	var date = new Date()
+
+	var start_date = new Date(date.getFullYear(), date.getMonth(), 1);  // Set to the 1st of the current month
+	start_date.setMonth( date.getMonth() - 1 )  // Subtract a month
+	var start_month = start_date.toISOString().split("T")[0].split("-")[0] + "-" + start_date.toISOString().split("T")[0].split("-")[1]
+
+	var end_date = new Date(date.getFullYear(), date.getMonth(), 1)  // Set to the 1st of the next month
+	var end_month = end_date.toISOString().split("T")[0].split("-")[0] + "-" + end_date.toISOString().split("T")[0].split("-")[1]
+
+  var billing_center_ids = _.pluck(ds_filtered_billing_centers, "id")
+
+  //Dimensions
+  var dimensions = [
+    "billing_center_id",
+    "vendor_account",
+    "vendor_account_name"
+  ]
+
+  //POST Request
+  var request = {
+    auth: "auth_flexera",
+    verb: "POST",
+    host: rs_optima_host,
+    path: "/bill-analysis/orgs/" + org_id + "/costs/aggregated",
+    body_fields: {
+      "dimensions": dimensions,
+      "granularity": "month",
+      "metrics": [ "cost_amortized_unblended_adj" ],
+      "billing_center_ids": billing_center_ids,
+      "start_at": start_month,
+      "end_at": end_month
+    }
+  }
+  EOS
+end
+
+#GET LIST OF EXISTING RESERVATIONS
 datasource "ds_reservations" do
   request do
     auth $auth_flexera
@@ -65,47 +199,74 @@ datasource "ds_reservations" do
   end
 end
 
-datasource "ds_reservations_rounded" do
-  run_script $js_round_utilization, $ds_reservations
+#FILTER RESERVATIONS BASED ON THEIR ACCOUNT IDS BY MATCHING THEM WITH BILLING CENTER ACCOUNT IDS
+datasource "ds_filtered_reservations" do
+  run_script $js_filter_reservations_by_bc, $ds_accounts_by_billing_center, $ds_reservations
 end
 
-script "js_round_utilization", type: "javascript" do
-  parameters "reservations"
-  result "modified_reservations"
+script "js_filter_reservations_by_bc", type: "javascript" do
+  parameters "ds_accounts_by_billing_center", "ds_reservations"
+  result "result"
   code <<-EOS
-  // This is the list of filtered volumes.
-  var modified_reservations = [];
-  for (var i = 0; i < reservations.length; i++) {
-    modified_reservations.push(
-      {
-        utilization_percentage: Math.round(parseFloat(reservations[i]["utilization_percentage"])),
-        end_datetime: reservations[i]["end_datetime"],
-        start_datetime: reservations[i]["start_datetime"],
-        account_name: reservations[i]["account_name"],
-        account_id: reservations[i]["account_id"],
-        region: reservations[i]["region"],
-        instance_type: reservations[i]["instance_type"],
-        instance_count: reservations[i]["instance_count"],
-        scope: reservations[i]["scope"],
-        type: reservations[i]["type"],
-      }
-    )
-  }
-  modified_reservations = _.sortBy(modified_reservations, 'region');
-EOS
+
+  //Filter reservations for billing centers that contain a matching account ID
+  var bc_account_ids = _.uniq(_.pluck(ds_accounts_by_billing_center, "vendor_account_id"))
+  var filtered_reservations = _.filter(ds_reservations, function(ri) {
+    return _.contains(bc_account_ids, ri.account_id);
+  })
+
+  result = filtered_reservations
+  EOS
 end
 
-escalation "report_reserved_instances_utilization" do
-  automatic true
-  label "Send Email"
-  description "Send incident email"
-  email $param_email
+#ENRICH FILTERED RESERVATION DATA WITH BILLING CENTER DATA
+datasource "ds_enriched_reservation_data" do
+	run_script $js_enrich_reservation_data_with_bc_data, $ds_filtered_reservations, $ds_accounts_by_billing_center, $ds_filtered_billing_centers
+end
+
+script "js_enrich_reservation_data_with_bc_data", type: "javascript" do
+	parameters "ds_filtered_reservations", "ds_accounts_by_billing_center", "ds_filtered_billing_centers"
+	result "result"
+	code <<-EOS
+		_.each(ds_filtered_reservations, function(ri) {
+
+			//Get associated billing centers for reservation
+			var associated_billing_centers = _.filter(ds_accounts_by_billing_center, function(bc) {
+        return bc.vendor_account_id == ri.account_id
+			})
+			
+			//Get list of associated billing center names and convert the list to a string
+			var list_of_associated_bcs_string = ""
+			_.each(associated_billing_centers, function(bc) {
+        var billing_center_name = ""
+        var billing_center_details = _.find(ds_filtered_billing_centers, function(fbc) { return bc.billing_center_id == fbc.id })
+        if (billing_center_details != undefined) {
+          billing_center_name = billing_center_details.name
+        }
+
+				if (list_of_associated_bcs_string == "") {
+					list_of_associated_bcs_string = billing_center_name
+				} else {
+					list_of_associated_bcs_string += ", " + billing_center_name
+				}	
+			})
+
+			//Add list of associated billing center names string to reservation object for policy incident
+			ri["billing_centers"] = list_of_associated_bcs_string
+		})
+
+		result = ds_filtered_reservations
+	EOS
 end
 
-policy "ri_utilization" do
-  validate_each $ds_reservations_rounded do
+###############################################################################
+# Policy
+###############################################################################
+
+policy "pol_ri_utilization" do
+  validate_each $ds_enriched_reservation_data do
     summary_template "{{ rs_project_name }} (Account ID: {{ rs_project_id }}): Reserved Instance Utilization"
-    escalate $report_reserved_instances_utilization
+    escalate $esc_email
     check gt(to_n(val(item,"utilization_percentage")),$param_utilization)
     export do
       resource_level true
@@ -119,6 +280,9 @@ policy "ri_utilization" do
         label "Account Id"
         path "account_id"
       end
+			field "billing_centers" do
+				label "Associated Billing Center(s)"
+			end
       field "scope" do
         label "Scope"
       end
@@ -137,3 +301,14 @@ policy "ri_utilization" do
     end
   end
 end
+
+###############################################################################
+# Escalations
+###############################################################################
+
+escalation "esc_email" do
+  automatic true
+  label "Send Email"
+  description "Send incident email"
+  email $param_email
+end