Documentation: Add Notifications, Escalations, Filters, Delay.

doc/3.02-commands.md

@@ -4,7 +4,7 @@ Icinga 2 uses three different command object types to specify how
 checks should be performed, notifications should be sent and
 events should be handled.
 
-> **Tip**
+> **Note**
 >
 > Define the `$plugindir$` macro in your global `IcingaMacros` variable
 > (located in `/etc/icinga2/conf.d/macros.conf` by default) and use
@@ -12,6 +12,22 @@ events should be handled.
 > Put your plugins and scripts into the directory defined by the `$plugindir$` macro
 > and make sure they are executable by the Icinga 2 user.
 
+### Environment Macros
+
+If your plugins require environment macros instead of command arguments you have
+to define all macros in the `export_macros` attribute as list.
+
+    export_macros = [
+      "HOSTNAME",
+      "SERVICEDESC",
+      "SERVICESTATE"
+    ]
+    
+> **Note**
+>
+> Use templates to define global `export_macros` attributes for the three
+> command types and let each command object inherit the attribute.
+
 ### Check Commands
 
 `CheckCommand` objects define the command line how a check is called.

doc/3.04-notifications.md

@@ -1,13 +1,299 @@
 ## Notifications
 
-TODO
-
 Notifications on alerts are an integral part of your Icinga 2 monitoring application.
+There are many ways of getting a notification to the actual receiver - Email, XMPP,
+IRC, Twitter, etc. The default method for executing a notification command are
+plugin scripts used for notifications.
+These may either be shell commands to invoke a system call to the `mail` binary
+or your own script fetching available macro values and doing proper formatting
+before sending the notification.
+Other mechanism will require writing the notification string into an api processing
+it there (for example ticket system integration).
+
+Such notification plugins are available from community users and professionals for
+example on the [MonitoringExchange](http://www.monitoringexchange.org) or the
+[Icinga Wiki](https://wiki.icinga.org). Or you'll write your own and share it. 
+    
+A notification requires one or more users (and/or user groups) who will be notified
+in case. These users must have all macro attributes defined which will be used in
+the `NotificationCommand` on execution, for example `email` as macro dictionary key
+is referenced as `$USEREMAIL$`.
+
+The user `icingaadmin` in this examples will get notified only on `WARNING` and
+`CRITICAL` states and `problem` and `recovery` notification types.
+    
+    object User "icingaadmin" {
+      display_name = "Icinga 2 Admin",
+      enable_notifications = 1,
+      notification_state_filter = (StateFilterWarning |
+                                   StateFilterCritical),
+      notification_type_filter = (NotificationFilterProblem |
+                                   NotificationFilterRecovery),  
+      macros = {
+        "email" = "icinga@localhost",
+        "pager" = "+49123456789"
+      }
+    }
+    
+> **Note**
+>
+> If you don't set the `notification_state_filter` and `notification_type_filter`
+> configuration attributes for the `User` object, all states and types will be
+> notified.
+
 You should choose which information you (and your notified users) are interested in
 case of emergency, and also which information does not provide any value to you and
 your environment.
 
+> **Note**
+>
+> The chain of attribute inheritance including the (additive) macro dictionary for
+> notifications will allow granular macros for every specific use case, such as
+> `$mail$` or `$mobile$` as `User` macros available in `NotificationCommand`.
+
+    Service -> Notification -> Command -> User
+       
+There are various macros available at runtime execution of the `NotificationCommand`.
+The example below may or may not fit your needs.
+
+    object NotificationCommand "mail-notification" inherits "plugin-notification-command" {
+      command = [
+        "/usr/bin/printf",
+        "\"%b\"",
+        {{{\"***** Icinga  *****
+    
+        Notification Type: $NOTIFICATIONTYPE$
+    
+        Service: $SERVICEDESC$
+        Host: $HOSTALIAS$
+        Address: $HOSTADDRESS$
+        State: $SERVICESTATE$
+    
+        Date/Time: $LONGDATETIME$
+    
+        Additional Info: $SERVICEOUTPUT$
+    
+        Comment: [$NOTIFICATIONAUTHORNAME$] $NOTIFICATIONCOMMENT$\"}}},
+        "/bin/mail",
+        "-s",
+        "\"$NOTIFICATIONTYPE$ - $HOSTDISPLAYNAME$ - $SERVICEDISPLAYNAME$ is $SERVICESTATE$\"",
+        "$USEREMAIL$"
+      ]
+    }
+
+> **Note** 
+>  
+> Alternatively you can use the `export_macros` attributes to export all required
+> macros into the environment.
+
+You can add all shared attributes to a `Notification` template which is inherited
+to the defined notifications. That way you'll save duplicated attributes in each
+`Notification` object. Attributes can be overridden locally. 
+    
+    template Notification "generic-notification" {
+      notification_interval = 15m,
+    
+      notification_command = "mail-service-notification",
+  
+      notification_state_filter = (StateFilterWarning |
+                                   StateFilterCritical |
+                                   StateFilterUnknown),
+      notification_type_filter = (NotificationFilterProblem |
+                                  NotificationFilterAcknowledgement |
+                                  NotificationFilterRecovery |
+                                  NotificationFilterCustom |
+                                  NotificationFilterFlappingStart |
+                                  NotificationFilterFlappingEnd |
+                                  NotificationFilterDowntimeStart |
+                                  NotificationFilterDowntimeEnd |
+                                  NotificationFilterDowntimeRemoved),
+  
+      notification_period = "24x7"
+    }
+    
+> **Note**
+>
+> The `TimePeriod` `24x7` is shipped as example configuration with Icinga 2.
+
+Use the `generic-notification` template for the `mail` notification defined
+inline to the host's service `ping4` and assign the `icingaadmin` user. 
+
+    object Host "localhost" {
+      services["ping4"] = {
+        notifications["mail"] = {
+          templates = [ "generic-notification" ],
+          notification_command = "mail-notification",
+          users = [ "icingaadmin" ],
+        }      
+      }
+    }
+    
+Notifications can be defined in `Service` templates inherited to the objects.
+
+> **Note**
+>
+> Instead of assigning users to notifications, you can also add the `user_groups`
+> attribute with a list of user groups to the `Notification` object. Icinga 2 will
+> resolve all group members and send notifications to all of them.
+
 ### Notification Escalations
 
-TODO
+When a problem notification is sent and a problem still exists after re-notification
+you may want to escalate the problem to the next support level. A different approach
+is to configure the default notification by email, and escalate the problem via sms
+if not already solved.
 
+You can define notification start and end times as additional configuration
+attributes making the `Notification` object a so-called `notification escalation`.
+Using templates you can share the basic notification attributes such as users or the
+`notification_interval` (and override them for the escalation then).
+
+Using the example from above, you can define additional users being escalated for sms
+notifications between start and end time.
+
+> **Note**
+>
+> `notification_state_filter` and `notification_type_filter` configuration attributes
+> are not set in this example.
+
+    object User "icinga-oncall-2nd-level" {
+      display_name = "Icinga 2nd Level",
+      enable_notifications = 1,
+
+      macros = {
+        "mobile" = "+49123456781"
+      }
+    }
+    
+    object User "icinga-oncall-1st-level" {
+      display_name = "Icinga 1st Level",
+      enable_notifications = 1,
+
+      macros = {
+        "mobile" = "+49123456782"
+      }
+    }
+    
+Define an additional `NotificationCommand` for sms notifications.
+
+> **Note**
+>
+> The example is not complete as there are many different sms providers.
+> Please note that sending sms notifications will require an sms provider
+> or local hardware with a sim card active.
+
+    object NotificationCommand "sms-notification" {
+       command = "$plugindir$/send_sms_notification $mobile$ ..."
+    }
+    
+The two new notification escalations are added onto the host `localhost`
+and its service `ping4` using the `generic-notification` template.
+The user `icinga-oncall-2nd-level` will get notified by sms (`sms-notification`
+command) after `30m` until `1h`.
+
+> **Note**
+>
+> The `notification_interval` was set to 15m in the `generic-notification`
+> template example. Lower that value in your escalations by using a secondary
+> template or overriding the attribute directly in the `notifications` array
+> position for `escalation-sms-2nd-level`.
+
+If the problem does not get resolved or acknowledged preventing further notifications
+the `escalation-sms-1st-level` user will be escalated `1h` after the initial problem was
+notified, but only for one hour (`2h` as `end` key for the `times` dictionary). 
+
+    object Host "localhost" {
+      services["ping4"] = {
+        notifications["mail"] = {
+          templates = [ "generic-notification" ],
+          notification_command = "mail-notification",
+          users = [ "icingaadmin" ],
+        },
+        notifications["escalation-sms-2nd-level"] = {
+          templates = [ "generic-notification" ],
+          notification_command = "sms-notification",
+          users = [ "icinga-oncall-2nd-level" ],
+          
+          times = {
+            begin = 30m,
+            end = 1h
+          }
+        },
+        notifications["escalation-sms-1st-level"] = {
+          templates = [ "generic-notification" ],
+          notification_command = "sms-notification",
+          users = [ "icinga-oncall-1st-level" ],
+          
+          times = {
+            begin = 1h,
+            end = 2h
+          }
+        }        
+      }
+    }
+    
+> **Note**
+>
+> Instead of assigning users to notifications, you can also add the `user_groups`
+> attribute with a list of user groups to the `Notification` object. Icinga 2 will
+> resolve all group members and send notifications and notification escalations to
+> all of them.
+
+### First Notification Delay
+
+Sometimes the problem in question should not be notified when the first notification
+happens, but a defined time duration afterwards. In Icinga 2 you can use the `times`
+dictionary and set `begin = 15m` as key and value if you want to suppress notifications
+in the first 15 minutes. Leave out the `end` key - if not set, Icinga 2 will not check against any
+end time for this notification.
+
+    object Host "localhost" {
+      services["ping4"] = {
+        notifications["mail"] = {
+          templates = [ "generic-notification" ],
+          notification_command = "mail-notification",
+          users = [ "icingaadmin" ],
+          
+          times = {
+            begin = 15m // delay first notification
+          }
+        }
+      }
+    }
+    
+> **Note**
+>
+> In Icinga 1.x the attribute is called `first_notification_delay`.
+
+### Notification Filters by State and Type
+
+If there are no notification state and type filter attributes defined at the `Notification`
+or `User` object Icinga 2 assumes that all states and types are being notified.
+
+> **Note**
+>
+> In order to notify on problem states, you still need the type filter `NotificationFilterProblem`.
+
+Available state and type filters for notifications are:
+
+    template Notification "generic-notification" {
+  
+      notification_state_filter = (StateFilterWarning |
+                                   StateFilterCritical |
+                                   StateFilterUnknown),
+      notification_type_filter = (NotificationFilterProblem |
+                                  NotificationFilterAcknowledgement |
+                                  NotificationFilterRecovery |
+                                  NotificationFilterCustom |
+                                  NotificationFilterFlappingStart |
+                                  NotificationFilterFlappingEnd |
+                                  NotificationFilterDowntimeStart |
+                                  NotificationFilterDowntimeEnd |
+                                  NotificationFilterDowntimeRemoved),
+    }
+
+> **Note**
+> 
+> If you are familiar with Icinga 1.x `notification_options` please note that they have been split
+> into type and state, and allow more fine granular filtering for example on downtimes and flapping.
+> You can filter for acknowledgements and custom notifications too. 

doc/8-differences-between-icinga-1x-and-2.md

@@ -1,4 +1,4 @@
-# Differences between Icinga 1.x and 2
+# <a id="differences-1x-2"></a> Differences between Icinga 1.x and 2
 
 ## Configuration Format
 

etc/icinga2/conf.d/notifications.conf

@@ -39,6 +39,7 @@ object NotificationCommand "mail-service-notification" inherits "plugin-notifica
     Additional Info: $SERVICEOUTPUT$
 
     Comment: [$NOTIFICATIONAUTHORNAME$] $NOTIFICATIONCOMMENT$\"}}},
+    "/bin/mail",
     "-s",
     "\"$NOTIFICATIONTYPE$ - $HOSTDISPLAYNAME$ - $SERVICEDISPLAYNAME$ is $SERVICESTATE$\"",
     "$USEREMAIL$"

lib/icinga/notification.cpp

@@ -130,7 +130,7 @@ Dictionary::Ptr Notification::GetTimes(void) const
 unsigned long Notification::GetNotificationTypeFilter(void) const
 {
 	if (m_NotificationTypeFilter.IsEmpty())
-		return ~(unsigned long)0; /* All states. */
+		return ~(unsigned long)0; /* All types. */
 	else
 		return m_NotificationTypeFilter;
 }

lib/icinga/user.cpp

@@ -101,7 +101,7 @@ TimePeriod::Ptr User::GetNotificationPeriod(void) const
 unsigned long User::GetNotificationTypeFilter(void) const
 {
 	if (m_NotificationTypeFilter.IsEmpty())
-		return ~(unsigned long)0; /* All states. */
+		return ~(unsigned long)0; /* All types. */
 	else
 		return m_NotificationTypeFilter;
 }