Add troubleshooting chapter, fix addon chapter.

Fixes #5611
Fixes #6275

doc/1-about.md

@@ -27,6 +27,19 @@ is appreciated!
 
 Please get in touch with the Icinga team at [https://www.icinga.org/ecosystem/].
 
+## <a id="development"></a> Icinga 2 Development
+
+You can follow Icinga 2's development closely by checking
+out these resources:
+
+* Development Bug Tracker: [https://dev.icinga.org/projects/i2?jump=issues] ([http://www.icinga.org/faq/how-to-report-a-bug/])
+* Git Repositories: [https://git.icinga.org/?p=icinga2.git;a=summary] (mirror at [https://github.com/Icinga/icinga2])
+* Git Checkins Mailinglist: [https://lists.icinga.org/mailman/listinfo/icinga-checkins]
+* Development Mailinglist: [https://lists.icinga.org/mailman/listinfo/icinga-devel]
+* \#icinga-devel on irc.freenode.net [http://webchat.freenode.net/?channels=icinga-devel] including a Git Commit Bot
+
+For general support questions, please refer to [https://www.icinga.org/support/].
+
 ## <a id="demo-vm"></a> Demo VM
 
 Icinga 2 is available as [Vagrant Demo VM](#vagrant).

doc/2-getting-started.md

@@ -4,6 +4,8 @@ This tutorial is a step-by-step introduction to installing Icinga 2 and
 available Icinga web interfaces. It assumes that you are familiar with
 the system you're installing Icinga 2 on.
 
+Details on troubleshooting problems can be found [here](#troubleshooting).
+
 ## <a id="setting-up-icinga2"></a> Setting up Icinga 2
 
 First of all you will have to install Icinga 2. The preferred way of doing this

doc/3-monitoring-basics.md

@@ -43,6 +43,8 @@ check command.
 The `address` custom attribute is used by check commands to determine which network
 address is associated with the host object.
 
+Details on troubleshooting check problems can be found [here](#troubleshooting).
+
 ### <a id="host-states"></a> Host States
 
 Hosts can be in any of the following states:
@@ -247,9 +249,10 @@ The user `icingaadmin` in the example below will get notified only on `WARNING`
       email = "icinga@localhost"
     }
 
-If you don't set the `states` and `types`
+If you don't set the `states` and `types` configuration attributes for the `User`
-configuration attributes for the `User` object, notifications for all states and types
+object, notifications for all states and types will be sent.
-will be sent.
+
+Details on troubleshooting notification problems can be found [here](#troubleshooting).
 
 You should choose which information you (and your notified users) are interested in
 case of emergency, and also which information does not provide any value to you and

doc/5-addons-plugins.md

@@ -0,0 +1,57 @@
+# <a id="addons-plugins"></a> Icinga Addons and Plugins
+
+## <a id="addons"></a> Addons
+
+### <a id="addons-graphing-reporting"></a> Graphing Addons
+
+#### <a id="addons-graphing-pnp"></a> PNP
+
+PNP (http://www.pnp4nagios.org) must be configured using the "bulk mode with npcd and npcdmod"
+hence Icinga 2's [PerfdataWriter](#performance-data) acts as npcdmod. NPCD will collect
+the rotated performance data files.
+
+#### <a id="addons-graphing-pnp"></a> inGraph
+
+inGraph (https://www.netways.org/projects/ingraph/wiki) requires only the ingraph-collector
+configured pointed to the perfdata files Icinga 2's [PerfdataWriter](#performance-data) will
+write to the performance data spool directory.
+
+#### <a id="addons-graphing-pnp"></a> Graphite
+
+There are Graphite addons available collecting the performance data files as well. But
+natively you just use the [GraphiteWriter](#graphite-carbon-cache-writer) feature.
+
+#### <a id="addons-reporting"></a> Icinga Reporting
+
+By enabling the DB IDO feature you can use the Icinga Reporting package.
+
+
+### <a id="addons-visualization"></a> Visualization
+
+#### <a id="addons-visualization-nagvis"></a> NagVis
+
+Either using Livestatus or DB IDO as backend you can create your own network maps
+based on your monitoring configuration and status data using NagVis (http://www.nagvis.org).
+
+### <a id="addons-web-interfaces"></a> Web Interfaces
+
+Next to the Icinga supported web interfaces (Classic UI 1.x, Web 1.x, Web 2) there are a
+couple of community provided web interfaces too:
+
+* Thruk (http://www.thruk.org) based on the Livestatus feature
+
+
+
+## <a id="plugins"></a> Plugins
+
+Additional plugins next to the [Monitoring Plugins](https://www.monitoring-plugins.org)
+are available at the [Monitoring Exchange](#https://www.monitoringexchange.org) platform.
+
+More details on the plugins can also be found on the Icinga Wiki at https://www.icinga.org
+
+## <a id="plugin-api"></a> Plugin API
+
+Currently Icinga 2 supports the native plugin API specification from the `Monitoring Plugins`
+project.
+
+The `Monitoring Plugin API` is defined in the [Monitoring Plugins Development Guidelines](https://www.monitoring-plugins.org/doc/guidelines.html).

doc/6-addons-plugins.md

@@ -1,8 +0,0 @@
-# <a id="addons-plugins"></a> Icinga Addons and Plugins
-
-## <a id="plugin-api"></a> Plugin API
-
-Currently Icinga 2 supports the native plugin API specification from the `Monitoring Plugins`
-project.
-
-The `Monitoring Plugin API` is defined in the [Monitoring Plugins Development Guidelines](https://www.monitoring-plugins.org/doc/guidelines.html).

doc/7-troubleshooting.md

@@ -0,0 +1,152 @@
+# <a id="troubleshooting"></a> Icinga 2 Troubleshooting
+
+For a more verbose output of the Icinga 2 daemon increase the
+`severity` attribute of the [logs](#logging) to `notice` or
+`debug`.
+
+Additionally you can enable the debug log using
+
+    # icinga2-enable-feature debuglog
+    # service icinga2 restart
+    # tail -f /var/log/icinga2/debug.log
+
+## <a id="troubleshooting-information-required"></a> Which information is required
+
+* Which distribution and version
+* How was Icinga 2 installed (and which repository in case)
+* Run `icinga2 --version`
+* Provide complete configuration snippets explaining your problem in detail
+* Provide complete logs targetting your problem
+* If the check command failed - what's the output of your manual plugin tests?
+* In case of [debugging](#debug) Icinga 2, the full back traces and outputs
+
+## <a id="checks-not-executed"></a> Checks are not executed
+
+* Check the debug log if the check command gets executed
+* Verify that failed depedencies do not prevent the command execution
+* Make sure that the plugin is executable by the Icinga 2 user (run a manual test)
+
+    # sudo -u icinga /usr/lib/nagios/plugins/check_ping -4 -H 127.0.0.1 -c 5000,100% -w 3000,80%
+
+## <a id="notifications-not-sent"></a> Notifications are not sent
+
+* Check the debug log if a notification is triggered
+* If yes, verify that all conditions are satisfied
+* Any errors on the notification command execution logged?
+
+Verify the following configuration
+
+* Is the host/service the `enable_notifications` attribute set, and if, to which value?
+* Do the notification attributes `states`, `types`, `period` match the notification conditions?
+* Do the user attributes `states`, `types`, `period` match the notification conditions?
+* Are there any notification `begin` and `end` times configured?
+
+* Does the referenced NotificationCommand work executed as Icinga user on the shell?
+
+## <a id="feature-not-working"></a> Feature is not working
+
+* Make sure that the feature configuration is enabled by symlink from `features-available/`
+to `features-enabled` and that the latter is included in [icinga2.conf](#icinga2-conf).
+* Are the feature attributes set correctly according to the documentation?
+* Any errors on the logs?
+
+## <a id="configuration-attribute-inheritance"></a> Configuration attributes are inherited from
+
+Icinga 2 allows you to import templates using the [import](#import) keyword. If these template
+contain additional attributes your objects will automatically inherit them. You can override
+or modify these attributes in the current object.
+
+
+## <a id="check-command-definitions"></a> Where are the check command definitions
+
+Icinga 2 ships additional [plugin check command definitions](#plugin-check-commands) which are
+included using
+
+    include <itl>
+    include <plugins>
+
+in [icinga2.conf](#icinga2-conf). These configurations will be overridden on upgrade, so please
+send modifications as proposed patches upstream. The default include path is set to
+`LocalStateDir + "/share/icinga2/includes"`.
+
+You should add your own command definitions to a new file in `conf.d/` called `commands.conf`
+or similar.
+
+
+## <a id="debug"></a> Debug Icinga 2
+
+Make sure that the debug symbols are available for Icinga 2.
+The Icinga 2 packages provide a debug package which must be
+installed separately for all involved binaries, like `icinga2-bin`
+or `icinga2-ido-mysql`.
+
+    # yum install icinga2-bin-debuginfo icinga2-ido-mysql-debuginfo
+
+Compiled binaries require the `-DCMAKE_BUILD_TYPE=RelWithDebInfo` or
+`-DCMAKE_BUILD_TYPE=Debug` cmake build flags.
+
+### <a id="development-debug-gdb"></a> GDB
+
+Call GDB with the binary and all arguments and run it in foreground.
+
+    # gdb --args /usr/sbin/icinga2 -c /etc/icinga2/icinga2.conf -x
+
+### <a id="development-debug-gdb-run"></a> GDB Run
+
+Run the application.
+
+    (gdb) r
+
+Kill the running application.
+
+    (gdb) k
+
+Continue after breakpoint.
+
+    (gdb) c
+
+### <a id="development-debug-gdb-backtrace"></a> GDB Backtrace
+
+If Icinga 2 aborted its operation abnormally, generate a backtrace.
+
+    (gdb) bt
+    (gdb) bt full
+
+>**Tip**
+>
+> If you're opening an issue at [https://dev.icinga.org] make sure
+> to attach as much details as possible.
+
+
+### <a id="development-debug-gdb-backtrace-stepping"></a> GDB Backtrace Stepping
+
+Identifying the problem may require stepping into the backtrace analysing
+the current scope, attributes and possible unmet requirements. `p` prints
+the value of the selected variable or function call result.
+
+    (gdb) up
+    (gdb) down
+    (gdb) p checkable
+    (gdb) p checkable.px->m_Name
+
+
+### <a id="development-debug-gdb-breakpoint"></a> GDB Breakpoints
+
+Set a breakpoint to a specific function call, or file specific line.
+
+    (gdb) b checkable.cpp:125
+    (gdb) b icinga::Checkable::SetEnablePerfdata
+
+GDB will ask about loading the required symbols later, select `yes` instead
+of `no`.
+
+Then run Icinga 2 until it reaches the first breakpoint. Continue with `c`
+afterwards.
+
+    (gdb) run
+    (gdb) c
+
+If you want to delete all breakpoints, use `d` and select `yes`.
+
+    (gdb) d
+

doc/8-development.md

@@ -1,91 +0,0 @@
-# <a id="icinga-2-development"></a> Icinga 2 Development
-
-You can follow Icinga 2's development closely by checking
-out these resources:
-
-* Development Bug Tracker: [https://dev.icinga.org/projects/i2?jump=issues] ([How to report a bug](http://www.icinga.org/faq/how-to-report-a-bug/))
-* Git Repositories: [https://git.icinga.org/?p=icinga2.git;a=summary] (mirror at [https://github.com/Icinga/icinga2])
-* Git Checkins Mailinglist: [https://lists.icinga.org/mailman/listinfo/icinga-checkins]
-* Development Mailinglist: [https://lists.icinga.org/mailman/listinfo/icinga-devel]
-* \#icinga-devel on irc.freenode.net [http://webchat.freenode.net/?channels=icinga-devel] including a Git Commit Bot
-
-For general support questions, please refer to [https://www.icinga.org/support/].
-
-
-## <a id="development-debug"></a> Debug Icinga 2
-
-Make sure that the debug symbols are available for Icinga 2.
-The Icinga 2 packages provide a debug package which must be
-installed separately for all involved binaries, like `icinga2-bin`
-or `icinga2-ido-mysql`.
-
-    # yum install icinga2-bin-debuginfo icinga2-ido-mysql-debuginfo
-
-Compiled binaries require the `-DCMAKE_BUILD_TYPE=RelWithDebInfo` or
-`-DCMAKE_BUILD_TYPE=Debug` cmake build flags.
-
-### <a id="development-debug-gdb"></a> GDB
-
-Call GDB with the binary and all arguments and run it in foreground.
-
-    # gdb --args /usr/sbin/icinga2 -c /etc/icinga2/icinga2.conf -x
-
-### <a id="development-debug-gdb-run"></a> GDB Run
-
-Run the application.
-
-    (gdb) r
-
-Kill the running application.
-
-    (gdb) k
-
-Continue after breakpoint.
-
-    (gdb) c
-
-### <a id="development-debug-gdb-backtrace"></a> GDB Backtrace
-
-If Icinga 2 aborted its operation abnormally, generate a backtrace.
-
-    (gdb) bt
-    (gdb) bt full
-
->**Tip**
->
-> If you're opening an issue at [https://dev.icinga.org] make sure
-> to attach as much details as possible.
-
-
-### <a id="development-debug-gdb-backtrace-stepping"></a> GDB Backtrace Stepping
-
-Identifying the problem may require stepping into the backtrace analysing
-the current scope, attributes and possible unmet requirements. `p` prints
-the value of the selected variable or function call result.
-
-    (gdb) up
-    (gdb) down
-    (gdb) p checkable
-    (gdb) p checkable.px->m_Name
-
-
-### <a id="development-debug-gdb-breakpoint"></a> GDB Breakpoints
-
-Set a breakpoint to a specific function call, or file specific line.
-
-    (gdb) b checkable.cpp:125
-    (gdb) b icinga::Checkable::SetEnablePerfdata
-
-GDB will ask about loading the required symbols later, select `yes` instead
-of `no`.
-
-Then run Icinga 2 until it reaches the first breakpoint. Continue with `c`
-afterwards.
-
-    (gdb) run
-    (gdb) c
-
-If you want to delete all breakpoints, use `d` and select `yes`.
-
-    (gdb) d
-