Prebuilt detection rules: new UI for installing and upgrading (#3552)

* Rename tab, button - "Installed Rules" tab - "Add Elastic rules" button * Fix docs bug: duplicating prebuilt rules outdated instructions * Add Asciidoc frontmatter * Update "Install and enable" section * Add "Update rules" section * Break out prebuilt rules topic, add images * Add frontmatter, li'l cleanup * Fix x-docs link * Add section on tag categories Addresses #3525 * Revise description frontmatter * Update image, add use case * Update ref in Container Workload Protection * Apply suggestions from reviews 1st round Co-authored-by: Nastasha Solomon <79124755+nastasha-solomon@users.noreply.github.com> Co-authored-by: Georgii Gorbachev <banderror@gmail.com> * Explain cost of duplicating rules * Rename page Renamed file for good measure (though not necessary) * Tighten up images * Apply feedback from Kseniia * Remove outdated video * Revise & move download section * Apply suggestions from Janeen's review Co-authored-by: Janeen Mikell Roberts <57149392+jmikell821@users.noreply.github.com> * Remove download section, update xref --------- Co-authored-by: Nastasha Solomon <79124755+nastasha-solomon@users.noreply.github.com> Co-authored-by: Georgii Gorbachev <banderror@gmail.com> Co-authored-by: Janeen Mikell Roberts <57149392+jmikell821@users.noreply.github.com>
elastic · Jul 25, 2023 · 42a3e6c · 42a3e6c
1 parent 986f614
commit 42a3e6c
Show file tree

Hide file tree

Showing 12 changed files with 153 additions and 98 deletions.
diff --git a/docs/cloud-native-security/d4c-get-started.asciidoc b/docs/cloud-native-security/d4c-get-started.asciidoc
@@ -1,6 +1,11 @@
 [[d4c-get-started]]
 = Get started with CWP
 
+:frontmatter-description: Secure your containerized workloads and start detecting threats and vulnerabilities.
+:frontmatter-tags-products: [security]
+:frontmatter-tags-content-type: [how-to]
+:frontmatter-tags-user-goals: [get-started]
+
 This page describes how to set up Container Workload Protection (CWP) for various use cases.
 
 [discrete]
@@ -40,7 +45,9 @@ In order to detect threats using this data, you'll need active <<detection-engin
 
 To install and enable Elastic's prebuilt rules:
 
-. Go to *Security > Manage > Rules*, and click *Load Elastic prebuilt rules and timeline templates* (this may take a few minutes).
+. Go to *Security > Manage > Rules*, then click *Add Elastic rules*.
+. Use the *Tags* selector to search for `container`. Select the `Data Source: Elastic Defend for Containers` tag. The rules table displays relevant rules.
+. Select all the displayed rules and click *Install _x_ selected rule(s)*.
 . Once the rules have loaded, you will see the Rules management page. Use the *Tags* selector to search for `container`. Select the `Container Workload Protection` tag.
 . Select all the rules with the tag, and then click *Bulk actions > Enable*.
 

diff --git a/docs/detections/detections-index.asciidoc b/docs/detections/detections-index.asciidoc
@@ -10,6 +10,8 @@ include::rules-cross-cluster-search.asciidoc[leveloffset=+1]
 
 include::investigation-guide-actions.asciidoc[leveloffset=+1]
 
+include::prebuilt-rules-management.asciidoc[]
+
 include::rules-ui-manage.asciidoc[]
 
 include::rules-ui-monitor.asciidoc[]

diff --git a/docs/detections/images/install-prebuilt-settings.png b/docs/detections/images/install-prebuilt-settings.png
diff --git a/docs/detections/images/prebuilt-rules-add-badge.png b/docs/detections/images/prebuilt-rules-add-badge.png
diff --git a/docs/detections/images/prebuilt-rules-add.png b/docs/detections/images/prebuilt-rules-add.png
diff --git a/docs/detections/images/prebuilt-rules-update.png b/docs/detections/images/prebuilt-rules-update.png
diff --git a/docs/detections/machine-learning/machine-learning.asciidoc b/docs/detections/machine-learning/machine-learning.asciidoc
@@ -2,6 +2,11 @@
 [role="xpack"]
 = Anomaly detection with {ml}
 
+:frontmatter-description: Use the power of machine learning to detect outliers and suspicious events.
+:frontmatter-tags-products: [security]
+:frontmatter-tags-content-type: [overview]
+:frontmatter-tags-user-goals: [manage]
+
 {ml-docs}/ml-ad-overview.html[{ml-cap}] functionality is available when
 you have the appropriate subscription, are using a *{ess-trial}[cloud deployment]*,
 or are testing out a *Free Trial*. Refer to <<ml-requirements>>.
@@ -32,7 +37,7 @@ details*).
 
 You can also check the status of {ml} detection rules, and start or stop their associated {ml} jobs:
 
-* In the *Rules* table, the *Last response* column displays the rule's current <<rule-status,status>>. An indicator icon (image:images/rules-table-error-icon.png[Error icon from Rules table,15,15]) also appears if a required {ml} job isn't running. Click the icon to list the affected jobs, then click *Visit rule details page to investigate* to open the rule's details page.
+* On the *Rules* page, the *Last response* column displays the rule's current <<rule-status,status>>. An indicator icon (image:images/rules-table-error-icon.png[Error icon from rules table,15,15]) also appears if a required {ml} job isn't running. Click the icon to list the affected jobs, then click *Visit rule details page to investigate* to open the rule's details page.
 +
 [role="screenshot"]
 image::images/rules-table-ml-job-error.png[Rules table {ml} job error]

diff --git a/docs/detections/prebuilt-rules-management.asciidoc b/docs/detections/prebuilt-rules-management.asciidoc
@@ -0,0 +1,121 @@
+[[prebuilt-rules-management]]
+== Install and manage Elastic prebuilt rules
+
+:frontmatter-description: Start detections quickly with prebuilt rules designed and updated by Elastic.
+:frontmatter-tags-products: [security]
+:frontmatter-tags-content-type: [how-to]
+:frontmatter-tags-user-goals: [manage]
+
+Follow these guidelines to start using the {es-sec-app}'s <<prebuilt-rules, prebuilt rules>>, keep them updated, and make sure they have the data needed to run successfully. 
+
+* <<load-prebuilt-rules>>
+* <<prebuilt-rule-tags>>
+* <<select-all-prebuilt-rules>>
+* <<update-prebuilt-rules>>
+* <<rule-prerequisites>>
+
+[NOTE]
+====
+* Prebuilt rules don't start running by default. You must first install the rules, then enable them. After installation, only a few prebuilt rules will be enabled by default, such as the Endpoint Security rule.
+
+* You can't modify most settings on Elastic prebuilt rules. You can only edit <<rule-notifications, rule actions>> and <<add-exceptions, add exceptions>>. If you want to modify other settings on a prebuilt rule, you must first duplicate it, then make your changes to the duplicated rule. However, your customized rule is entirely separate from the original prebuilt rule, and will not get updates from Elastic if the prebuilt rule is updated.
+====
+
+[float]
+[[load-prebuilt-rules]]
+=== Install and enable Elastic prebuilt rules
+
+. Go to *Manage* -> *Rules*. The badge next to *Add Elastic rules* shows the number of prebuilt rules available for installation. 
++
+[role="screenshot"]
+image::images/prebuilt-rules-add-badge.png[The Add Elastic Rules page]
+
+. Click *Add Elastic rules*, then do one of the following:
+* Install all available rules: Click *Install all*.
+* Install a single rule: Click *Install rule* for that rule.
+* Install multiple rules: Select the rules and click *Install _x_ selected rule(s)*.
++
+TIP: Use the search bar and *Tags* filter to find the rules you want to install. For example, filter by `OS: Windows` if your environment only includes Windows endpoints. For more on tag categories, refer to <<prebuilt-rule-tags>>.
++
+[role="screenshot"]
+image::images/prebuilt-rules-add.png[The Add Elastic Rules page]
+
+. Go back to the *Rules* page, search or filter for any rules you want to run, and do either of the following:
+
+* Enable a single rule: Turn on the rule's *Enabled* switch.
+* Enable multiple rules: Select the rules, then click *Bulk actions* -> *Enable*.
+
+Once you enable a rule, it starts running on its configured schedule. To confirm that it's running successfully, check its *Last response* status in the rules table, or open the rule's details page and check the <<rule-execution-logs, *Execution results*>> tab.
+
+[float]
+[[prebuilt-rule-tags]]
+=== Prebuilt rule tags
+
+Each prebuilt rule includes several tags identifying the rule's purpose, detection method, associated resources, and other information to help categorize your rules. These tags are category-value pairs; for example, `OS: Windows` indicates rules designed for Windows endpoints. Categories include:
+
+* `Data Source`: The application, cloud provider, data shipper, or Elastic integration providing data for the rule.
+* `Domain`: A general category of data source types (such as cloud, endpoint, or network).
+* `OS`: The host operating system, which could be considered another data source type.
+* `Resources`: Additional rule resources such as investigation guides.
+* `Rule Type`: Identifies if the rule depends on specialized resources (such as machine learning jobs or threat intelligence indicators), or if it's a higher-order rule built from other rules' alerts.
+* `Tactic`: MITRE ATT&CK tactics that the rule addresses.
+* `Threat`: Specific threats the rule detects (such as Cobalt Strike or BPFDoor).
+* `Use Case`: The type of activity the rule detects and its purpose. Use cases include:
+** `Active Directory Monitoring`: Detects changes related to Active Directory.
+** `Asset Visibility`: Detects changes to specified asset types.
+** `Configuration Audit`: Detects undesirable configuration changes.
+** `Guided Onboarding`: Example rule, used for {elastic-sec}'s guided onboarding tour.
+** `Identity and Access Audit`: Detects activity related to identity and access management (IAM).
+** `Log Auditing`: Detects activity on log configurations or storage.
+** `Network Security Monitoring`: Detects network security configuration activity.
+** `Threat Detection`: Detects threats.
+** `Vulnerability`: Detects exploitation of specific vulnerabilities.
+
+[float]
+[[select-all-prebuilt-rules]]
+=== Select and duplicate all prebuilt rules
+
+. Go to *Manage* -> *Rules*, then select the *Elastic rules* filter.
+. Click *Select all _x_ rules* above the rules table.
+. Click *Bulk actions* -> *Duplicate*.
+. Select whether to duplicate the rules' exceptions, then click *Duplicate*.
+
+You can then modify the duplicated rules and, if required, delete the prebuilt ones. However, your customized rules are entirely separate from the original prebuilt rules, and will not get updates from Elastic if the prebuilt rules are updated.
+
+[float]
+[[update-prebuilt-rules]]
+=== Update Elastic prebuilt rules
+
+Elastic regularly updates prebuilt rules to optimize their performance and ensure they detect the latest threats and techniques. When updated versions are available for your installed prebuilt rules, the *Rule Updates* tab appears on the *Rules* page, allowing you to update your installed rules with the latest versions.
+
+. Go to *Manage* -> *Rules*, then select the *Rule Updates* tab.
++
+NOTE: The *Rule Updates* tab doesn't appear if all your installed prebuilt rules are up to date.
++
+[role="screenshot"]
+image::images/prebuilt-rules-update.png[The Rule Updates tab on the Rules page]
+
+. Do one of the following:
+* Update all available rules: Click *Update all*.
+* Update a single rule: Click *Update rule* for that rule.
+* Update multiple rules: Select the rules and click *Update _x_ selected rule(s)*.
++
+TIP: Use the search bar and *Tags* filter to find the rules you want to update. For example, filter by `OS: Windows` if your environment only includes Windows endpoints. For more on tag categories, refer to <<prebuilt-rule-tags>>.
+
+[float]
+[[rule-prerequisites]]
+=== Confirm rule prerequisites
+
+Many Elastic prebuilt rules are designed to work with specific Elastic integrations and data fields. These prerequisites are identified in the *Related integrations* and *Required fields* fields on a rule's details page (*Manage* -> *Rules*, then click a rule's name). *Related integrations* also displays each integration's installation status and includes links for installing and configuring the listed integrations. 
+
+Additionally, the *Setup guide* section provides guidance on setting up the rule's requirements.
+
+[role="screenshot"]
+image::images/rule-details-prerequisites.png[Rule details page with Related integrations, Required fields, and Setup guide highlighted]
+
+You can also check rules' related integrations in the *Installed Rules* and *Rule Monitoring* tables. Click the *integrations* badge to display the related integrations in a popup.
+
+[role="screenshot"]
+image::images/rules-table-related-integrations.png[Rules table with related integrations popup,75%]
+
+TIP: You can hide the *integrations* badge in the rules tables. Go to *{kib}* -> *Stack Management* -> *Advanced Settings*, then turn off `securitySolution:showRelatedIntegrations`.
diff --git a/docs/detections/prebuilt-rules/prebuilt-rules-downloadable-updates.asciidoc b/docs/detections/prebuilt-rules/prebuilt-rules-downloadable-updates.asciidoc
@@ -4,7 +4,7 @@
 
 This section lists all updates to prebuilt detection rules, made available with the *Prebuilt Security Detection Rules* integration in Fleet.
 
-To download the latest updates, follow the instructions in <<download-prebuilt-rules>>
+To update your installed rules to the latest versions, follow the instructions in <<update-prebuilt-rules>>.
 
 
 [width="100%",options="header"]

diff --git a/docs/detections/rules-ui-manage.asciidoc b/docs/detections/rules-ui-manage.asciidoc
@@ -1,7 +1,11 @@
 [[rules-ui-management]]
-[role="xpack"]
 == Manage detection rules
 
+:frontmatter-description: Manage your detection rules and enable Elastic prebuilt rules on the Rules page.
+:frontmatter-tags-products: [security]
+:frontmatter-tags-content-type: [how-to]
+:frontmatter-tags-user-goals: [manage]
+
 The Rules page allows you to view and manage all prebuilt and custom detection rules.
 
 [role="screenshot"]
@@ -11,10 +15,6 @@ On the Rules page, you can:
 
 * <<sort-filter-rules>>
 * <<rule-status>>
-* <<load-prebuilt-rules>>
-* <<select-all-prebuilt-rules>>
-* <<download-prebuilt-rules>>
-* <<rule-prerequisites>>
 * <<edit-rules-settings>>
 * <<manage-rules-ui>>
 * <<snooze-rule-actions>>
@@ -49,75 +49,8 @@ The *Last response* column displays the current status of each rule, based on th
 * *Failed*: The rule encountered an error that prevented it from running. For example, a {ml} rule whose corresponding {ml} job wasn't running.
 * *Warning*: Nothing prevented the rule from running, but it might have returned unexpected results. For example, a custom query rule tried to search an index pattern that couldn't be found in {es}.
 
-For {ml} rules, an indicator icon (image:images/rules-table-error-icon.png[Error icon from Rules table,15,15]) also appears in this column if a required {ml} job isn't running. Click the icon to list the affected jobs, then click *Visit rule details page to investigate* to open the rule's details page, where you can start the {ml} job.
-
-You can filter rules by status using the *Last response* filter.
-
-[float]
-[[load-prebuilt-rules]]
-=== Load and activate Elastic prebuilt rules
-
-To load the {es-sec-app}'s <<prebuilt-rules, prebuilt rules>>, go to *Manage* -> *Rules* -> *Load Elastic prebuilt rules and Timeline templates*.
-
-You can then activate whichever rules you want. If you delete any prebuilt rules, a button appears that enables you to reload all of the deleted ones.
-
-[NOTE]
-==============
-Apart from the Elastic Endpoint rule, prebuilt rules are not activated by
-default. If you want to modify a prebuilt rule, you must first duplicate it, then make your changes to the duplicated rule. All Elastic prebuilt rules are tagged with the word `Elastic`.
-
-To learn how to enable detection rules in Elastic Security, watch the <<enable-detection-rules, tutorial>> at the end of this topic.
-==============
+For {ml} rules, an indicator icon (image:images/rules-table-error-icon.png[Error icon from rules table,15,15]) also appears in this column if a required {ml} job isn't running. Click the icon to list the affected jobs, then click *Visit rule details page to investigate* to open the rule's details page, where you can start the {ml} job.
 
-[float]
-[[select-all-prebuilt-rules]]
-=== Select and duplicate all prebuilt rules
-
-. Go to *Manage* -> *Rules*.
-. Click *Select all _x_ rules* above the rules table.
-. Click *Bulk actions* -> *Duplicate*.
-. Select the *Custom rules* tab.
-
-You can then modify the duplicated rules and, if required, delete the prebuilt ones.
-
-[float]
-[[download-prebuilt-rules]]
-=== Download latest Elastic prebuilt rules
-
-As of {stack} >=7.13.0, you can download the latest version of Elastic prebuilt rules outside of a regular release cycle. This feature ensures you have the latest detection capabilities before upgrading to the latest {stack}.
-
-To download the latest version of prebuilt rules:
-
-. In {kib}, go to *Management* -> *Integrations*.
-. Search for "Prebuilt Security Detection Rules."
-. Select the integration, then select the *Settings* tab. The integration settings page is displayed.
-+
-[role="screenshot"]
-image::images/install-prebuilt-settings.png[]
-+
-. Click *Install Prebuilt Security Detection Rules assets*.
-. Click *Install Prebuilt Security Detection Rules* to confirm the installation.
-+
-[role="screenshot"]
-image::images/install-prebuilt-rules.png[]
-
-[float]
-[[rule-prerequisites]]
-=== Confirm rule prerequisites
-
-Many Elastic prebuilt rules are designed to work with specific Elastic integrations and data fields. These prerequisites are identified in the *Related integrations* and *Required fields* fields on a rule's details page (*Manage* -> *Rules*, then click a rule's name). *Related integrations* also displays each integration's installation status and includes links for installing and configuring the listed integrations. 
-
-Additionally, the *Setup guide* section provides guidance on setting up the rule's requirements.
-
-[role="screenshot"]
-image::images/rule-details-prerequisites.png[Rule details page with Related integrations, Required fields, and Setup guide highlighted]
-
-You can also check rules' related integrations in the *Rules* and *Rule Monitoring* tables. Click the *integrations* badge to display the related integrations in a popup.
-
-[role="screenshot"]
-image::images/rules-table-related-integrations.png[Rules table with related integrations popup,75%]
-
-TIP: You can hide the *integrations* badge in the Rules tables. Go to *{kib}* -> *Stack Management* -> *Advanced Settings*, then turn off `securitySolution:showRelatedIntegrations`.
 
 [float]
 [[edit-rules-settings]]
@@ -177,7 +110,7 @@ Instead of turning rules off to stop alert notifications, you can snooze rule ac
 
 You can snooze notifications temporarily or indefinitely. When actions are snoozed, you can cancel or change the duration of the snoozed state. You can also schedule and manage recurring downtime for actions.
 
-You can snooze rule notifications from the Rules table, the rule details page, or the *Actions* tab when editing a rule.
+You can snooze rule notifications from the *Installed Rules* tab, the rule details page, or the *Actions* tab when editing a rule.
 
 [role="screenshot"]
 image::images/rule-snoozing.png[Rules snooze options,65%]
@@ -226,21 +159,3 @@ NOTE: Imported rules must be in an `.ndjson` file.
 .. (Optional) Select *Overwrite existing connectors with conflicting action "id"* to update existing connectors if they match the `action id` value of any rule actions in the import file. Configuration data included with the actions is also overwritten.
 .. Click *Import rule*.
 .. (Optional) If a connector is missing sensitive information after the import, a warning displays and you're prompted to fix the connector. In the warning, click *Go to connector*. On the Connectors page, find the connector that needs to be updated, click *Fix*, then add the necessary details.  
-
-[float]
-[[enable-detection-rules]]
-=== Tutorial: Enable detection rules
-To learn how to enable detection rules in Elastic Security, watch the following tutorial.
-
-++++
-<script type="text/javascript" async src="https://play.vidyard.com/embed/v4.js"></script>
-<img
-  style="width: 100%; margin: auto; display: block;"
-  class="vidyard-player-embed"
-  src="https://play.vidyard.com/9Kcg8qJcHdcF9bXUc1XEQZ.jpg"
-  data-uuid="9Kcg8qJcHdcF9bXUc1XEQZ"
-  data-v="4"
-  data-type="inline"
-/>
-</br>
-++++
diff --git a/docs/detections/rules-ui-monitor.asciidoc b/docs/detections/rules-ui-monitor.asciidoc
@@ -2,6 +2,11 @@
 [role="xpack"]
 == Monitor and troubleshoot rule executions
 
+:frontmatter-description: Monitor the status of your detection rules and troubleshoot common issues with rules and alerts.
+:frontmatter-tags-products: [security]
+:frontmatter-tags-content-type: [how-to]
+:frontmatter-tags-user-goals: [manage, monitor]
+
 The Rules page offers several ways to gain insight into the status of your detection rules:
 
 * <<rule-monitoring-tab, Rule Monitoring tab>> — The current state of all detection rules and their most recent executions. Go to the *Rule Monitoring* tab to get an overview of which rules are running, how long they're taking, and if they're having any trouble.
@@ -21,11 +26,11 @@ times, select the *Rule Monitoring* tab on the *Rules* page (*Manage* ->
 [role="screenshot"]
 image::images/monitor-table.png[]
 
-On the *Rule Monitoring* tab, you can <<sort-filter-rules, sort and filter rules>> just like you can on the *Rules* tab. 
+On the *Rule Monitoring* tab, you can <<sort-filter-rules, sort and filter rules>> just like you can on the *Installed Rules* tab. 
 
 TIP: To sort the rules list, click any column header. To sort in descending order, click the column header again.
 
-For detailed information on a rule, the alerts it generated, and associated errors, click on its name in the table. This also allows you to perform the same actions that are available on the <<rules-ui-management, **Rules** tab>>, such as modifying or deleting rules, activating or deactivating rules, exporting or importing rules, and duplicating prebuilt rules.
+For detailed information on a rule, the alerts it generated, and associated errors, click on its name in the table. This also allows you to perform the same actions that are available on the <<rules-ui-management, **Installed Rules** tab>>, such as modifying or deleting rules, activating or deactivating rules, exporting or importing rules, and duplicating prebuilt rules.
 
 [float]
 [[rule-execution-logs]]