From 63c7512fd9e6af9cddcc7749821b575f5c9a18fe Mon Sep 17 00:00:00 2001 From: Michael Friedrich Date: Thu, 29 May 2014 15:14:44 +0200 Subject: [PATCH] Documentation: Monitoring Basics: Add/Fix livestatus and add DB IDO incl health checks. Fixes #6356 Fixes #6358 --- doc/2-getting-started.md | 29 ++--- doc/3-monitoring-basics.md | 214 ++++++++++++++++++++++++++++++++-- doc/5-addons-plugins.md | 7 +- doc/6-configuring-icinga-2.md | 4 +- doc/9-appendix.md | 116 +----------------- 5 files changed, 230 insertions(+), 140 deletions(-) diff --git a/doc/2-getting-started.md b/doc/2-getting-started.md index ab0320174..74d07c69b 100644 --- a/doc/2-getting-started.md +++ b/doc/2-getting-started.md @@ -467,14 +467,13 @@ following command: # mysql -u root -p icinga < /usr/share/doc/icinga2-ido-mysql-*/schema/mysql.sql -The Icinga 2 RPM packages install the schema files into -`/usr/share/doc/icinga2-ido-mysql-*/schema` (`*` means package version). +The schema file location differs by the distribution used: -On SuSE-based distributions the schema files are installed in -`/usr/share/doc/packages/icinga2-ido-mysql/schema`. - -The Debian/Ubuntu packages put the schema files into -`/usr/share/icinga2-ido-mysql/schema`. + Distribution | Schema Files + --------------|--------------------- + RHEL | `/usr/share/doc/icinga2-ido-mysql-*/schema` (`*` means package version). + SUSE | `/usr/share/doc/packages/icinga2-ido-mysql/schema` + Debian/Ubuntu | `/usr/share/icinga2-ido-mysql/schema` #### Upgrading the MySQL database @@ -529,6 +528,10 @@ setting up a PostgreSQL database for Icinga 2: # sudo -u postgres createdb -O icinga -E UTF8 icinga # sudo -u postgres createlang plpgsql icinga +> **Note** +> +> Using PostgreSQL 9.x you can omit the `createlang` command. + Locate your pg_hba.conf (Debian: `/etc/postgresql/*/main/pg_hba.conf`, RHEL/SUSE: `/var/lib/pgsql/data/pg_hba.conf`), add the icinga user with md5 authentication method and restart the postgresql server. @@ -556,14 +559,14 @@ using the following command: # export PGPASSWORD=icinga # psql -U icinga -d icinga < /usr/share/doc/icinga2-ido-pgsql-*/schema/pgsql.sql -The Icinga 2 RPM packages install the schema files into -`/usr/share/doc/icinga2-ido-pgsql-*/schema` (`*` means package version). +The schema file location differs by the distribution used: -On SuSE-based distributions the schema files are installed in -`/usr/share/doc/packages/icinga2-ido-pgsql/schema`. + Distribution | Schema Files + --------------|--------------------- + RHEL | `/usr/share/doc/icinga2-ido-pgsql-*/schema` (`*` means package version). + SUSE | `/usr/share/doc/packages/icinga2-ido-pgsql/schema` + Debian/Ubuntu | `/usr/share/icinga2-ido-pgsql/schema` -The Debian/Ubuntu packages put the schema files into -`/usr/share/icinga2-ido-pgsql/schema`. #### Upgrading the PostgreSQL database diff --git a/doc/3-monitoring-basics.md b/doc/3-monitoring-basics.md index 5716eb6b0..d166086f6 100644 --- a/doc/3-monitoring-basics.md +++ b/doc/3-monitoring-basics.md @@ -1587,7 +1587,6 @@ Icinga 1.x Classic UI requires this data set as part of its backend. > you can safely disable this feature. - ## Compat Logging The Icinga 1.x log format is considered being the `Compat Log` @@ -1596,7 +1595,7 @@ in Icinga 2 provided with the `CompatLogger` object. These logs are not only used for informational representation in external web interfaces parsing the logs, but also to generate SLA reports and trends in Icinga 1.x Classic UI. Futhermore the -`Livestatus` feature uses these logs for answering queries to +[Livestatus](#livestatus) feature uses these logs for answering queries to historical tables. The `CompatLogger` object can be enabled with @@ -1636,6 +1635,211 @@ existing log parsers. + +## DB IDO + +The IDO (Icinga Data Output) modules for Icinga 2 take care of exporting all +configuration and status information into a database. The IDO database is used +by a number of projects including Icinga Web 1.x and 2. + +Details on the installation can be found in the [Getting Started](#configuring-ido) +chapter. Details on the configuration can be found in the +[IdoMysqlConnection](#objecttype-idomysqlconnection) and +[IdoPgsqlConnection](#objecttype-idoPgsqlconnection) +object configuration documentation. + +The following example query checks the health of the current Icinga 2 instance +writing its current status to the DB IDO backend table `icinga_programstatus` +every 10 seconds. By default it checks 60 seconds into the past which is a reasonable +amount of time - adjust it for your requirements. If the condition is not met, +the query returns an empty result. + +> **Tip** +> +> Use [check plugins](#plugins) to monitor the backend. + +Replace the `default` string with your instance name, if different. + +Example for MySQL: + + # mysql -u root -p icinga -e "SELECT status_update_time FROM icinga_programstatus ps + JOIN icinga_instances i ON ps.instance_id=i.instance_id + WHERE (UNIX_TIMESTAMP(ps.status_update_time) > UNIX_TIMESTAMP(NOW())-60) + AND i.instance_name='default';" + + +---------------------+ + | status_update_time | + +---------------------+ + | 2014-05-29 14:29:56 | + +---------------------+ + + +Example for PostgreSQL: + + # export PGPASSWORD=icinga; psql -U icinga -d icinga -c "SELECT ps.status_update_time FROM icinga_programstatus AS ps + JOIN icinga_instances AS i ON ps.instance_id=i.instance_id + WHERE ((SELECT extract(epoch from status_update_time) FROM icinga_programstatus) > (SELECT extract(epoch from now())-60)) + AND i.instance_name='default'"; + + status_update_time + ------------------------ + 2014-05-29 15:11:38+02 + (1 Zeile) + + +A detailed list on the available table attributes can be found in the [DB IDO Schema documentation](#schema-db-ido). + + +## Livestatus + +The [MK Livestatus](http://mathias-kettner.de/checkmk_livestatus.html) project +implements a query protocol that lets users query their Icinga instance for +status information. It can also be used to send commands. + +Details on the installation can be found in the [Getting Started](#setting-up-livestatus) +chapter. + +### Livestatus Sockets + +Other to the Icinga 1.x Addon, Icinga 2 supports two socket types + +* Unix socket (default) +* TCP socket + +Details on the configuration can be found in the [LivestatusListener](#objecttype-livestatuslistener) +object configuration. + +### Livestatus GET Queries + +> **Note** +> +> All Livestatus queries require an additional empty line as query end identifier. +> The `unixcat` tool is either available by the MK Livestatus project or as seperate +> binary. + +There also is a Perl module available in CPAN for accessing the Livestatus socket +programmatically: [Monitoring::Livestatus](http://search.cpan.org/~nierlein/Monitoring-Livestatus-0.74/) + + +Example using the unix socket: + + # echo -e "GET services\n" | unixcat /var/run/icinga2/cmd/livestatus + +Example using the tcp socket listening on port `6558`: + + # echo -e 'GET services\n' | netcat 127.0.0.1 6558 + + # cat servicegroups < Livestatus COMMAND Queries + +A list of available external commands and their parameters can be found [here](#external-commands-list-detail) + + $ echo -e 'COMMAND ' | netcat 127.0.0.1 6558 + + +### Livestatus Filters + +and, or, negate + + Operator | Negate | Description + ----------|------------------------ + = | != | Euqality + ~ | !~ | Regex match + =~ | !=~ | Euqality ignoring case + ~~ | !~~ | Regex ignoring case + > | | Less than + < | | Greater than + >= | | Less than or equal + <= | | Greater than or equal + + +### Livestatus Stats + +Schema: "Stats: aggregatefunction aggregateattribute" + + Aggregate Function | Description + -------------------|-------------- + sum |   + min |   + max |   + avg | sum / count + std | standard deviation + suminv | sum (1 / value) + avginv | suminv / count + count | ordinary default for any stats query if not aggregate function defined + +Example: + + GET hosts + Filter: has_been_checked = 1 + Filter: check_type = 0 + Stats: sum execution_time + Stats: sum latency + Stats: sum percent_state_change + Stats: min execution_time + Stats: min latency + Stats: min percent_state_change + Stats: max execution_time + Stats: max latency + Stats: max percent_state_change + OutputFormat: json + ResponseHeader: fixed16 + +### Livestatus Output + +* CSV + +CSV Output uses two levels of array separators: The members array separator +is a comma (1st level) while extra info and host|service relation separator +is a pipe (2nd level). + +Seperators can be set using ASCII codes like: + + Separators: 10 59 44 124 + +* JSON + +Default separators. + +### Livestatus Error Codes + + Code | Description + ----------|-------------- + 200 | OK + 404 | Table does not exist + 452 | Exception on query + +### Livestatus Tables + + Table | Join |Description + --------------|-----------|---------------------------- + hosts |   | host config and status attributes, services counter + hostgroups |   | hostgroup config, status attributes and host/service counters + services | hosts | service config and status attributes + servicegroups |   | servicegroup config, status attributes and service counters + contacts |   | contact config and status attributes + contactgroups |   | contact config, members + commands |   | command name and line + status |   | programstatus, config and stats + comments | services | status attributes + downtimes | services | status attributes + timeperiods |   | name and is inside flag + endpoints |   | config and status attributes + log | services, hosts, contacts, commands | parses [compatlog](#objecttype-compatlogger) and shows log attributes + statehist | hosts, services | parses [compatlog](#objecttype-compatlogger) and aggregates state change attributes + +The `commands` table is populated with `CheckCommand`, `EventCommand` and `NotificationCommand` objects. + +A detailed list on the available table attributes can be found in the [Livestatus Schema documentation](#schema-livestatus). + + ## Check Result Files Icinga 1.x writes its check result files into a temporary spool directory @@ -1654,9 +1858,3 @@ on-demand in your Icinga 2 objects configuration. object CheckResultReader "reader" { spool_dir = "/data/check-results" } - - - - - - diff --git a/doc/5-addons-plugins.md b/doc/5-addons-plugins.md index 860998738..b41575de6 100644 --- a/doc/5-addons-plugins.md +++ b/doc/5-addons-plugins.md @@ -1,7 +1,9 @@ -# Icinga Addons and Plugins +# Icinga 2 Addons and Plugins ## Addons +[Livestatus] + ### Graphing Addons #### PNP @@ -38,8 +40,7 @@ based on your monitoring configuration and status data using NagVis (http://www. Next to the Icinga supported web interfaces (Classic UI 1.x, Web 1.x, Web 2) there are a couple of community provided web interfaces too: -* Thruk (http://www.thruk.org) based on the Livestatus feature - +* Thruk (http://www.thruk.org) based on the [Livestatus](#livestatus) feature ## Plugins diff --git a/doc/6-configuring-icinga-2.md b/doc/6-configuring-icinga-2.md index 5e9bfedeb..0b62c8407 100644 --- a/doc/6-configuring-icinga-2.md +++ b/doc/6-configuring-icinga-2.md @@ -1191,7 +1191,7 @@ Example: database = "icinga" table_prefix = "icinga_" instance_name = "icinga2" - instance_description = "icinga2 dev instance" + instance_description = "icinga2 instance" cleanup = { downtimehistory_age = 48h @@ -1278,7 +1278,7 @@ Example: database = "icinga" table_prefix = "icinga_" instance_name = "icinga2" - instance_description = "icinga2 dev instance" + instance_description = "icinga2 instance" cleanup = { downtimehistory_age = 48h diff --git a/doc/9-appendix.md b/doc/9-appendix.md index c3cac129e..508b2c9e2 100644 --- a/doc/9-appendix.md +++ b/doc/9-appendix.md @@ -148,7 +148,7 @@ Icinga 2 specific extensions: * command objects support custom variables (added in Classic UI 1.11.2) * host and service objects support 'is_reachable' (added in Classic UI 1.11.3) -### DB IDO +### DB IDO Schema There is a detailed documentation for the Icinga IDOUtils 1.x database schema available on [http://docs.icinga.org/latest/en/db_model.html] @@ -190,119 +190,7 @@ New columns: Additional command custom variables populated from 'vars' dictionary. Additional global custom variables populated from 'Vars' constant (object_id is NULL). - -### Livestatus - -#### Livestatus GET Queries - - $ echo -e 'GET services' | netcat 127.0.0.1 6558 - - $ cat servicegroups < Livestatus COMMAND Queries - -A list of available external commands and their parameters can be found [here](#external-commands-list-detail) - - $ echo -e 'COMMAND ' | netcat 127.0.0.1 6558 - - -#### Livestatus Filters - -and, or, negate - - Operator | Negate | Description - ----------|------------------------ - = | != | Euqality - ~ | !~ | Regex match - =~ | !=~ | Euqality ignoring case - ~~ | !~~ | Regex ignoring case - > | | Less than - < | | Greater than - >= | | Less than or equal - <= | | Greater than or equal - - -#### Livestatus Stats - -Schema: "Stats: aggregatefunction aggregateattribute" - - Aggregate Function | Description - -------------------|-------------- - sum |   - min |   - max |   - avg | sum / count - std | standard deviation - suminv | sum (1 / value) - avginv | suminv / count - count | ordinary default for any stats query if not aggregate function defined - -Example: - - GET hosts - Filter: has_been_checked = 1 - Filter: check_type = 0 - Stats: sum execution_time - Stats: sum latency - Stats: sum percent_state_change - Stats: min execution_time - Stats: min latency - Stats: min percent_state_change - Stats: max execution_time - Stats: max latency - Stats: max percent_state_change - OutputFormat: json - ResponseHeader: fixed16 - -#### Livestatus Output - -* CSV - -CSV Output uses two levels of array separators: The members array separator -is a comma (1st level) while extra info and host|service relation separator -is a pipe (2nd level). - -Seperators can be set using ASCII codes like: - - Separators: 10 59 44 124 - -* JSON - -Default separators. - -#### Livestatus Error Codes - - Code | Description - ----------|-------------- - 200 | OK - 404 | Table does not exist - 452 | Exception on query - -#### Livestatus Tables - - Table | Join |Description - --------------|-----------|---------------------------- - hosts |   | host config and status attributes, services counter - hostgroups |   | hostgroup config, status attributes and host/service counters - services | hosts | service config and status attributes - servicegroups |   | servicegroup config, status attributes and service counters - contacts |   | contact config and status attributes - contactgroups |   | contact config, members - commands |   | command name and line - status |   | programstatus, config and stats - comments | services | status attributes - downtimes | services | status attributes - timeperiods |   | name and is inside flag - endpoints |   | config and status attributes - log | services, hosts, contacts, commands | parses [compatlog](#objecttype-compatlogger) and shows log attributes - statehist | hosts, services | parses [compatlog](#objecttype-compatlogger) and aggregates state change attributes - -The `commands` table is populated with `CheckCommand`, `EventCommand` and `NotificationCommand` objects. +### Livestatus Schema #### Livestatus Schema Extensions