mirror of https://github.com/Icinga/icinga2.git
231 lines
8.5 KiB
Markdown
231 lines
8.5 KiB
Markdown
## <a id="notifications"></a> Notifications
|
|
|
|
Notifications for service and host problems are an integral part of your
|
|
monitoring setup.
|
|
|
|
There are many ways of sending notifications, e.g. by e-mail, XMPP,
|
|
IRC, Twitter, etc. On its own Icinga 2 does not know how to send notifications.
|
|
Instead it relies on external mechanisms such as shell scripts to notify users.
|
|
|
|
A notification specification requires one or more users (and/or user groups)
|
|
who will be notified in case of problems. These users must have all custom
|
|
attributes defined which will be used in the `NotificationCommand` on execution.
|
|
|
|
TODO
|
|
|
|
The user `icingaadmin` in the example below will get notified only on `WARNING` and
|
|
`CRITICAL` states and `problem` and `recovery` notification types.
|
|
|
|
object User "icingaadmin" {
|
|
display_name = "Icinga 2 Admin"
|
|
enable_notifications = true
|
|
states = [ OK, Warning, Critical ]
|
|
types = [ Problem, Recovery ]
|
|
email = "icinga@localhost"
|
|
}
|
|
|
|
If you don't set the `states` and `types`
|
|
configuration attributes for the `User` object, notifications for all states and types
|
|
will be sent.
|
|
|
|
You should choose which information you (and your notified users) are interested in
|
|
case of emergency, and also which information does not provide any value to you and
|
|
your environment.
|
|
|
|
There are various custom attributes available at runtime execution of the
|
|
`NotificationCommand`. The example below may or may not fit your needs.
|
|
|
|
object NotificationCommand "mail-service-notification" {
|
|
import "plugin-notification-command"
|
|
|
|
command = [ SysconfDir + "/icinga2/scripts/mail-notification.sh" ]
|
|
|
|
env = {
|
|
"NOTIFICATIONTYPE" = "$notification.type$"
|
|
"SERVICEDESC" = "$service.description$"
|
|
"HOSTALIAS" = "$host.displayname$",
|
|
"HOSTADDRESS" = "$host.vars.address$",
|
|
"SERVICESTATE" = "$service.state$",
|
|
"LONGDATETIME" = "$icinga.longdatetime$",
|
|
"SERVICEOUTPUT" = "$service.output$",
|
|
"NOTIFICATIONAUTHORNAME" = "$notification.author$",
|
|
"NOTIFICATIONCOMMENT" = "$notification.comment$",
|
|
"HOSTDISPLAYNAME" = "$host.displayname$",
|
|
"SERVICEDISPLAYNAME" = "$service.displayname$",
|
|
"USEREMAIL" = "$user.vars.email$"
|
|
}
|
|
}
|
|
|
|
The command attribute in the `mail-service-notification` command refers to the
|
|
shell script installed into `/etc/icinga2/scripts/mail-notification.sh`.
|
|
The macros specified in the `env` dictionary are exported as environment
|
|
variables and can be used in the notification script.
|
|
|
|
You can add all shared attributes to a `Notification` template which is inherited
|
|
to the defined notifications. That way you'll save duplicated attributes in each
|
|
`Notification` object. Attributes can be overridden locally.
|
|
|
|
TODO
|
|
|
|
template Notification "generic-notification" {
|
|
interval = 15m
|
|
|
|
_command = "mail-service-notification"
|
|
|
|
states = [ Warning, Critical, Unknown ]
|
|
types = [ Problem, Acknowledgement, Recovery, Custom, FlappingStart,
|
|
FlappingEnd, DowntimeStart,DowntimeEnd, DowntimeRemoved ]
|
|
|
|
period = "24x7"
|
|
}
|
|
|
|
The time period `24x7` is shipped as example configuration with Icinga 2.
|
|
|
|
Use the `apply` keyword to create `Notification` objects for your services:
|
|
|
|
apply Notification "mail" to Service {
|
|
import "generic-notification"
|
|
|
|
command = "mail-notification"
|
|
users = [ "icingaadmin" ]
|
|
|
|
assign where service.name == "mysql"
|
|
}
|
|
|
|
Instead of assigning users to notifications, you can also add the `user_groups`
|
|
attribute with a list of user groups to the `Notification` object. Icinga 2 will
|
|
resolve all group members and send notifications to all of them.
|
|
|
|
### <a id="notification-escalations"></a> Notification Escalations
|
|
|
|
When a problem notification is sent and a problem still exists after re-notification
|
|
you may want to escalate the problem to the next support level. A different approach
|
|
is to configure the default notification by email, and escalate the problem via sms
|
|
if not already solved.
|
|
|
|
You can define notification start and end times as additional configuration
|
|
attributes making the `Notification` object a so-called `notification escalation`.
|
|
Using templates you can share the basic notification attributes such as users or the
|
|
`interval` (and override them for the escalation then).
|
|
|
|
Using the example from above, you can define additional users being escalated for sms
|
|
notifications between start and end time.
|
|
|
|
object User "icinga-oncall-2nd-level" {
|
|
display_name = "Icinga 2nd Level"
|
|
enable_notifications = true
|
|
|
|
vars.mobile = "+1 555 424642"
|
|
}
|
|
|
|
object User "icinga-oncall-1st-level" {
|
|
display_name = "Icinga 1st Level"
|
|
enable_notifications = true
|
|
|
|
vars.mobile = "+1 555 424642"
|
|
}
|
|
}
|
|
|
|
Define an additional `NotificationCommand` for SMS notifications.
|
|
|
|
> **Note**
|
|
>
|
|
> The example is not complete as there are many different SMS providers.
|
|
> Please note that sending SMS notifications will require an SMS provider
|
|
> or local hardware with a sim card active.
|
|
|
|
object NotificationCommand "sms-notification" {
|
|
command = PluginDir + "/send_sms_notification $mobile$ ..."
|
|
}
|
|
|
|
The two new notification escalations are added onto the host `localhost`
|
|
and its service `ping4` using the `generic-notification` template.
|
|
The user `icinga-oncall-2nd-level` will get notified by SMS (`sms-notification`
|
|
command) after `30m` until `1h`.
|
|
|
|
> **Note**
|
|
>
|
|
> The `interval` was set to 15m in the `generic-notification`
|
|
> template example. Lower that value in your escalations by using a secondary
|
|
> template or overriding the attribute directly in the `notifications` array
|
|
> position for `escalation-sms-2nd-level`.
|
|
|
|
If the problem does not get resolved or acknowledged preventing further notifications
|
|
the `escalation-sms-1st-level` user will be escalated `1h` after the initial problem was
|
|
notified, but only for one hour (`2h` as `end` key for the `times` dictionary).
|
|
|
|
apply Notification "mail" to Service {
|
|
import "generic-notification"
|
|
command = "mail-notification"
|
|
users = [ "icingaadmin" ]
|
|
|
|
assign where service.name == "ping4"
|
|
}
|
|
|
|
apply Notification "escalation-sms-2nd-level" to Service {
|
|
import "generic-notification"
|
|
command = "sms-notification"
|
|
users = [ "icinga-oncall-2nd-level" ]
|
|
|
|
times = {
|
|
begin = 30m
|
|
end = 1h
|
|
}
|
|
|
|
assign where service.name == "ping4"
|
|
}
|
|
|
|
apply Notification "escalation-sms-1st-level" to Service {
|
|
import "generic-notification"
|
|
command = "sms-notification"
|
|
users = [ "icinga-oncall-1st-level" ]
|
|
|
|
times = {
|
|
begin = 1h
|
|
end = 2h
|
|
}
|
|
|
|
assign where service.name == "ping4"
|
|
}
|
|
|
|
Instead of assigning users to notifications, you can also add the `user_groups`
|
|
attribute with a list of user groups to the `Notification` object. Icinga 2 will
|
|
resolve all group members and send notifications and notification escalations to
|
|
all of them.
|
|
|
|
### <a id="first-notification-delay"></a> First Notification Delay
|
|
|
|
Sometimes the problem in question should not be notified when the first notification
|
|
happens, but a defined time duration afterwards. In Icinga 2 you can use the `times`
|
|
dictionary and set `begin = 15m` as key and value if you want to suppress notifications
|
|
in the first 15 minutes. Leave out the `end` key - if not set, Icinga 2 will not check against any
|
|
end time for this notification.
|
|
|
|
apply Notification "mail" to Service {
|
|
import "generic-notification"
|
|
command = "mail-notification"
|
|
users = [ "icingaadmin" ]
|
|
|
|
times.begin = 15m // delay first notification
|
|
|
|
assign where service.name == "ping4"
|
|
}
|
|
|
|
### <a id="notification-filters-state-type"></a> Notification Filters by State and Type
|
|
|
|
If there are no notification state and type filter attributes defined at the `Notification`
|
|
or `User` object Icinga 2 assumes that all states and types are being notified.
|
|
|
|
Available state and type filters for notifications are:
|
|
|
|
template Notification "generic-notification" {
|
|
|
|
states = [ Warning, Critical, Unknown ]
|
|
types = [ Problem, Acknowledgement, Recovery, Custom, FlappingStart,
|
|
FlappingEnd, DowntimeStart, DowntimeEnd, DowntimeRemoved ]
|
|
}
|
|
|
|
If you are familiar with Icinga 1.x `notification_options` please note that they have been split
|
|
into type and state, and allow more fine granular filtering for example on downtimes and flapping.
|
|
You can filter for acknowledgements and custom notifications too.
|