From 4382b87baa0f4fa433772b2b38814fce0d443200 Mon Sep 17 00:00:00 2001 From: Michael Friedrich Date: Mon, 3 Dec 2018 09:49:27 +0100 Subject: [PATCH 01/11] Docs: First draft for development docs --- doc/21-development.md | 1571 +++++++++++++++++++++++++++++++++++------ 1 file changed, 1350 insertions(+), 221 deletions(-) diff --git a/doc/21-development.md b/doc/21-development.md index d90d50def..1ff183434 100644 --- a/doc/21-development.md +++ b/doc/21-development.md @@ -1,14 +1,19 @@ -# Develop Icinga 2 +# Development This chapter provides hints on Icinga 2 development -especially for debugging purposes. +and debugging. -> **Note** -> -> If you are planning to build your own development environment, -> please consult the `INSTALL.md` file from the source tree. +## Debug Icinga 2 -## Debug Requirements +This chapter targets all users who have been asked by developers to provide +a stack trace or coredump if the application crashed. It is also useful +for developers working with different debuggers. + +> **Note:** +> This is intentionally mentioned before any development insights +> as debugging is a more frequent and commonly asked question. + +### Debug Requirements Make sure that the debug symbols are available for Icinga 2. The Icinga 2 packages provide a debug package which must be @@ -17,43 +22,1364 @@ or `icinga2-ido-mysql`. Debian/Ubuntu: - # apt-get install icinga2-dbg +``` +apt-get install icinga2-dbg +``` RHEL/CentOS: - # yum install icinga2-debuginfo +``` +yum install icinga2-debuginfo +``` Fedora: - # dnf install icinga2-debuginfo icinga2-bin-debuginfo icinga2-ido-mysql-debuginfo +``` +dnf install icinga2-debuginfo icinga2-bin-debuginfo icinga2-ido-mysql-debuginfo +``` SLES/openSUSE: - # zypper install icinga2-bin-debuginfo icinga2-ido-mysql-debuginfo +``` +zypper install icinga2-bin-debuginfo icinga2-ido-mysql-debuginfo +``` - -Furthermore, you may also have to install debug symbols for Boost and your C library. +Furthermore, you may also have to install debug symbols for Boost and your C++ library. If you're building your own binaries, you should use the `-DCMAKE_BUILD_TYPE=Debug` cmake build flag for debug builds. -## GDB +### GDB as Debugger -Install gdb: +Install GDB in your development environment. Debian/Ubuntu: - # apt-get install gdb +``` +apt-get install gdb +``` RHEL/CentOS/Fedora: - # yum install gdb +``` +yum install gdb +``` SLES/openSUSE: - # zypper install gdb +``` +zypper install gdb +``` +#### GDB Run + +Call GDB with the binary (`/usr/sbin/icinga2` is a wrapper script calling +`/usr/lib64/icinga2/sbin/icinga2` since 2.4) and all arguments and run it in foreground. + +``` +gdb --args /usr/lib64/icinga2/sbin/icinga2 daemon -x debug +``` + +The exact path to the Icinga 2 binary differs on each distribution. On Ubuntu +it is installed into `/usr/lib/x86_64-linux-gnu/icinga2/sbin/icinga2` on 64-bit systems +for example. + +> **Note** +> +> If gdb tells you it's missing debug symbols, quit gdb and install +> them: `Missing separate debuginfos, use: debuginfo-install ...` + +Run/restart the application. + +``` +(gdb) r +``` + +Kill the running application. + +``` +(gdb) k +``` + +Continue after breakpoint. + +``` +(gdb) c +``` + +#### GDB Core Dump + +Either attach to the running process using `gdb -p PID` or start +a new gdb run. + +``` +(gdb) r +(gdb) generate-core-file +``` + +#### GDB Backtrace + +If Icinga 2 aborted its operation abnormally, generate a backtrace. +`thread apply all` is important here since this includes all running threads. +We need this information when e.g. debugging dead locks and hanging features. + +``` +(gdb) bt +(gdb) thread apply all bt full +``` + +If gdb stops at a SIGPIPE signal please disable the signal before +running Icinga 2. This isn't an error, but we need to workaround it. + +``` +(gdb) handle SIGPIPE nostop noprint pass +(gdb) r +``` + +If you create a [new issue](https://github.com/Icinga/icinga2/issues), +make sure to attach as much detail as possible. + +#### GDB Backtrace from Running Process + +If Icinga 2 is still running, generate a full backtrace from the running +process and store it into a new file (e.g. for debugging dead locks). + +Icinga 2 runs with 2 processes: main and command executor, therefore generate two backtrace logs +and add them to the GitHub issue. + +``` +for pid in $(pidof icinga2); do gdb -p $pid -batch -ex "thread apply all bt full" -ex "detach" -ex "q" > gdb_bt_${pid}_`date +%s`.log; done +``` + +#### GDB Thread List from Running Process + +Instead of a full backtrace, you sometimes just need a list of running threads. + +``` +for pid in $(pidof icinga2); do gdb -p $pid -batch -ex "info threads" -ex "detach" -ex "q" > gdb_threads_${pid}_`date +%s`.log; done +``` + +#### GDB Backtrace Stepping + +Identifying the problem may require stepping into the backtrace, analysing +the current scope, attributes, and possible unmet requirements. `p` prints +the value of the selected variable or function call result. + +``` +(gdb) up +(gdb) down +(gdb) p checkable +(gdb) p checkable.px->m_Name +``` + +#### GDB Breakpoints + +To set a breakpoint to a specific function call, or file specific line. + +``` +(gdb) b checkable.cpp:125 +(gdb) b icinga::Checkable::SetEnablePerfdata +``` + +GDB will ask about loading the required symbols later, select `yes` instead +of `no`. + +Then run Icinga 2 until it reaches the first breakpoint. Continue with `c` +afterwards. + +``` +(gdb) run +(gdb) c +``` + +In case you want to step into the next line of code, use `n`. If there is a +function call where you want to step into, use `s`. + +``` +(gdb) n + +(gdb) s +``` + +If you want to delete all breakpoints, use `d` and select `yes`. + +``` +(gdb) d +``` + +> **Tip** +> +> When debugging exceptions, set your breakpoint like this: `b __cxa_throw`. + +Breakpoint Example: + + (gdb) b __cxa_throw + (gdb) r + (gdb) up + .... + (gdb) up + #11 0x00007ffff7cbf9ff in icinga::Utility::GlobRecursive(icinga::String const&, icinga::String const&, boost::function const&, int) (path=..., pattern=..., callback=..., type=1) + at /home/michi/coding/icinga/icinga2/lib/base/utility.cpp:609 + 609 callback(cpath); + (gdb) l + 604 + 605 #endif /* _WIN32 */ + 606 + 607 std::sort(files.begin(), files.end()); + 608 BOOST_FOREACH(const String& cpath, files) { + 609 callback(cpath); + 610 } + 611 + 612 std::sort(dirs.begin(), dirs.end()); + 613 BOOST_FOREACH(const String& cpath, dirs) { + (gdb) p files + $3 = std::vector of length 11, capacity 16 = {{static NPos = 18446744073709551615, m_Data = "/etc/icinga2/conf.d/agent.conf"}, {static NPos = 18446744073709551615, + m_Data = "/etc/icinga2/conf.d/commands.conf"}, {static NPos = 18446744073709551615, m_Data = "/etc/icinga2/conf.d/downtimes.conf"}, {static NPos = 18446744073709551615, + m_Data = "/etc/icinga2/conf.d/groups.conf"}, {static NPos = 18446744073709551615, m_Data = "/etc/icinga2/conf.d/notifications.conf"}, {static NPos = 18446744073709551615, + m_Data = "/etc/icinga2/conf.d/satellite.conf"}, {static NPos = 18446744073709551615, m_Data = "/etc/icinga2/conf.d/services.conf"}, {static NPos = 18446744073709551615, + m_Data = "/etc/icinga2/conf.d/templates.conf"}, {static NPos = 18446744073709551615, m_Data = "/etc/icinga2/conf.d/test.conf"}, {static NPos = 18446744073709551615, + m_Data = "/etc/icinga2/conf.d/timeperiods.conf"}, {static NPos = 18446744073709551615, m_Data = "/etc/icinga2/conf.d/users.conf"}} + + +### Core Dump + +When the Icinga 2 daemon crashes with a `SIGSEGV` signal +a core dump file should be written. This will help +developers to analyze and fix the problem. + +#### Core Dump File Size Limit + +This requires setting the core dump file size to `unlimited`. + +Example for Systemd: + +``` +vim /usr/lib/systemd/system/icinga2.service + +[Service] +... +LimitCORE=infinity + +systemctl daemon-reload + +systemctl restart icinga2 +``` + +Example for init script: + +``` +vim /etc/init.d/icinga2 +... +ulimit -c unlimited + +service icinga2 restart +``` + +Verify that the Icinga 2 process core file size limit is set to `unlimited`. + +``` +cat /proc/`pidof icinga2`/limits +... +Max core file size unlimited unlimited bytes +``` + + +#### Core Dump Kernel Format + +The Icinga 2 daemon runs with the SUID bit set. Therefore you need +to explicitly enable core dumps for SUID on Linux. + +``` +sysctl -w fs.suid_dumpable=1 +``` + +Adjust the coredump kernel format and file location on Linux: + +``` +sysctl -w kernel.core_pattern=/var/lib/cores/core.%e.%p + +install -m 1777 -d /var/lib/cores +``` + +MacOS: + +``` +sysctl -w kern.corefile=/cores/core.%P + +chmod 777 /cores +``` + +#### Core Dump Analysis + +Once Icinga 2 crashes again a new coredump file will be written. Please +attach this file to your bug report in addition to the general details. + +Simple test case for a `SIGSEGV` simulation with `sleep`: + +``` +ulimit -c unlimited +sleep 1800& +[1] +kill -SEGV +gdb `which sleep` /var/lib/cores/core.sleep. +(gdb) bt +rm /var/lib/cores/core.sleep.* +``` + +Analyzing Icinga 2: + +``` +gdb /usr/lib64/icinga2/sbin/icinga2 core.icinga2. +(gdb) bt +``` + +### LLDB as Debugger + +LLDB is available on macOS with the Xcode command line tools. + +``` +$ xcode-select --install +``` + +In order to run Icinga 2 with LLDB you need to pass the binary as argument. + +``` +lldb -- /usr/local/icinga2/lib/icinga2/sbin/icinga2 daemon +``` + +Full backtrace: + +``` +> bt all +``` + +Select thread: + +``` +> thr sel 5 +``` + +Step into: + +``` +> s +``` + +Next step: + +``` +> n +``` + +Continue: + +``` +> c +``` + +Up/down in stacktrace: + +``` +> up +> down +``` + + + +## Develop Icinga 2 + +Icinga 2 can be built on many platforms such as Linux, Unix and Windows. +There are limitations in terms of support, e.g. Windows is only supported for clients, +not a full-featured master or satellite. + +Before you start with actual development, there is a couple of pre-requisites. + +### Choose your Editor + +Icinga 2 can be developed with your favorite editor. Icinga developers prefer to use +the tools below: + +- vim +- CLion (macOS, Linux) +- MS Visual Studio (Windows) +- Atom + +Editors differ on the functionality. The more helpers you get for C++ development, +the faster your development workflow will be. + + +#### Whitespace Cleanup + +Patches must be cleaned up and follow the indent style (tabs instead of spaces). +You should also remove any training whitespaces. + +`git diff` allows to highlight such. + +``` +vim $HOME/.gitconfig + +[color "diff"] + whitespace = red reverse +[core] + whitespace=fix,-indent-with-non-tab,trailing-space,cr-at-eol +``` + +`vim` also can match these and visually alert you to remove them. + +``` +vim $HOME/.vimrc + +highlight ExtraWhitespace ctermbg=red guibg=red +match ExtraWhitespace /\s\+$/ +autocmd BufWinEnter * match ExtraWhitespace /\s\+$/ +autocmd InsertEnter * match ExtraWhitespace /\s\+\%#\@ + +Icinga 2 can run standalone or in distributed environments. It contains a whole lot +more than a simple check execution engine. + +Read more about it in the [Technical Concepts](#technical-concepts) chapter. + +### Get to know the code + +First off, you really need to know C++ and portions of C++11 and the boost libraries. +Best is to start with a book or online tutorial to get into the basics. +Icinga developers gained their knowledge through studies, training and self-teaching +code by trying it out and asking senior developers for guidance. + +Here's a few books we can recommend: + +* [Accelerated C++: Practical Programming by Example](https://www.amazon.com/Accelerated-C-Practical-Programming-Example/dp/020170353X) (Andrew Koenig, Barbara E. Moo) +* [Effective C++](https://www.amazon.com/Effective-Specific-Improve-Programs-Designs/dp/0321334876) (Scott Meyers) +* [Der C++ Programmierer](https://www.amazon.de/Programmierer-lernen-Professionell-anwenden-L%C3%B6sungen/dp/3446416447), German (Ulrich Breymann) +* [C++11 programmieren](https://www.amazon.de/gp/product/3836217325/), German (Torsten T. Will) + +In addition, it is a good bet to also know SQL when diving into backend development. + +* [SQL Performance Explained](https://www.amazon.de/gp/product/3950307826/) (Markus Winand) + +Last but not least, if you are developing on Windows, get to know the internals about services and the Win32 API. + + +### Design Patterns + +Icinga 2 heavily relies on object-oriented programming and encapsulates common +functionality into classes and objects. It also uses modern programming techniques +to e.g. work with shared pointer memory management. + +Icinga 2 consists of libraries bundled into the main binary. Therefore you'll +find many code parts in the `lib/` directory wheras the actual application is +built from `icinga-app/`. Accompanied with Icinga 2, there's the Windows plugins +which are standalone and compiled from `plugins/`. + +Library | Description +---------------|------------------------------------ +base | Objects, values, types, streams, tockets, TLS, utilities, etc. +config | Configuration compiler, expressions, etc. +cli | CLI (sub) commands and helpers. +icinga | Icinga specific objects and event handling. +remote | Cluster and HTTP client/server and REST API related code. +checker | Checker feature, check scheduler. +notification | Notification feature, notification scheduler. +methods | Command execution methods, plugins and built-in checks. +perfdata | Performance data related, including Graphite, Elastic, etc. +db\_ido | IDO database abstraction layer. +db\_ido\_mysql | IDO database driver for MySQL. +db\_ido\_pgsql | IDO database driver for PgSQL. +mysql\_shin | Library stub for linking against the MySQL client libraries. +pgsql\_shim | Library stub for linking against the PgSQL client libraries. + +Another thing you will recognize are the `.ti` files which are compiled +by our own class compiler into actual source code. The meta language allows +developers to easily add object attributes and specify their behaviour. + +Some object attributes need to be stored over restarts in the state file +and therefore have the `state` attribute set. Others are treated as `config` +attribute and automatically get configuration validation functions created. +Hidden or read-only REST API attributes are marked with `no_user_view` and +`no_user_modify`. + +The most beneficial thing are getters and setters being generated. The actual object +inherits from `ObjectImpl` and therefore gets them "for free". + +Example: + +``` +vim lib/perfdata/gelfwriter.ti + + [config] enable_tls; + +vim lib/perfdata/gelfwriter.cpp + + if (GetEnableTls()) { +``` + +The logic is hidden in `tools/mkclass/` in case you want to learn more about it. +The first steps during CMake & make also tell you about code generation. + + +### Build Tool: CMake + +In its early development stages in 2012, Icinga 2 was built with autoconf/automake +and separate Windows project files. We've found this very fragile, and have changed +this into CMake as our build tool. + +The most common benefits: + +* Everything is described in CMakeLists.txt in each directory +* CMake only needs to know that a sub directory needs to be included. +* The global CMakeLists.txt acts as main entry point for requirement checks and library/header includes. +* Separate binary build directories, the actual source tree stays clean. +* CMake automatically generates a Visual Studio project file `icinga2.sln` on Windows. + +### Builds: Unity Builds + +Another thing you should be aware of: Unity builds on and off. + +Typically, we already use caching mechanisms to reduce recompile time with ccache. +For release builds, there's always a new build needed as the difference is huge compared +to a previous (major) release. + +Therefore we've invented the Unity builds, which basically concatenates all source files +into one big library source code file. The compiler then doesn't need to load the many small +files but compiles and links this huge one. + +Unity builds require more memory which is why you should disable them for development +builds in small sized VMs (Linux, Windows) and also Docker containers. + +There's a couple of header files which are included everywhere. If you touch/edit them, +the cache is invalidated and you need to recompile a lot more files then. `base/utility.hpp` +and `remote/zone.hpp` are good candidates for this. + + +### Linux Dev Environment + +Based on CentOS 7, we have an early draft available inside the Icinga Vagrant boxes: +[centos7-dev](https://github.com/Icinga/icinga-vagrant/tree/master/centos7-dev). + +If you're compiling Icinga 2 natively without any virtualization layer in between, +this usually is faster. This is also the reason why developers on macOS prefer native builds +over Linux or Windows VMs. Don't forget to test the actual code on Linux later! Socket specific +stuff like `epoll` is not available on Unix kernels. + +Depending on your workstation and environment, you may either develop and run locally, +use a container deployment pipeline or put everything in a high end resource remote VM. + +#### Prepare Linux Dev Environment + +Create two build directories for different binary builds. + +* `debug` contains the debug build binaries. They contain more debug information and run tremendously slower than release builds from packages. Don't use them for benchmarks. +* `release` contains the release build binaries, as you would install them on a live system. This helps comparing specific scenarios for race conditions and more. + +``` +mkdir -p release debug +``` + +Next, check the required packages to build Icinga 2, and install them. + +CentOS: + +``` + +``` + +Debian/Ubuntu: + +``` + +``` + +##### Debian Stretch + +``` +apt-get -y install gdb vim git cmake make ccache build-essential libssl-dev libboost-all-dev bison flex default-libmysqlclient-dev libpq-dev libyajl-dev libedit-dev monitoring-plugins + +ln -s /usr/bin/ccache /usr/local/bin/g++ + +groupadd icinga +groupadd icingacmd +useradd -c "icinga" -s /sbin/nologin -G icingacmd -g icinga icinga + +git clone https://github.com/icinga/icinga2.git && cd icinga2 + +mkdir debug release +cd debug +cmake .. -DCMAKE_BUILD_TYPE=Debug -DICINGA2_UNITY_BUILD=OFF -DCMAKE_INSTALL_PREFIX=/usr/local/icinga2 -DICINGA2_PLUGINDIR=/usr/local/sbin +cd .. +make -j2 install -C debug +``` + + +``` +chown -R icinga:icinga /usr/local/icinga2/var/ + +cd /usr/local/icinga2 +./lib/icinga2/prepare-dirs etc/sysconfig/icinga2 +icinga2 api setup +vim etc/icinga2/conf.d/api-users.conf + +gdb --args ./lib/icinga2/sbin/icinga2 daemon +``` + + + +### macOS Dev Environment + +It is advised to use Homebrew to install all required build dependencies. +Macports have been reported to work as well, typically you'll get more help +with Homebrew. + +We will also build two different flavors. + +``` +mkdir -p release debug +``` + + + +### Windows Dev Environment + +The following sections explain how to setup the required build tools +and how to run and debug the code. + +#### Chocolatey + +Open an administrative command prompt (Win key, type “cmd”, right-click and “run as administrator”) and paste the following instructions: + +``` +@powershell -NoProfile -ExecutionPolicy Bypass -Command "iex ((new-object net.webclient).DownloadString('https://chocolatey.org/install.ps1'))" && SET PATH=%PATH%;%ALLUSERSPROFILE%\chocolatey\bin +``` + +#### Visual Studio + +Thanks to Microsoft they’ll now provide their Professional Edition of Visual Studio 2017 +as community version, free for use for open source projects such as Icinga. +The installation requires ~9GB disk space. [Download](https://www.visualstudio.com/downloads/) +the web installer and start the installation. + +You need a free Microsoft account to download and also store your preferences. + +Choose the following minimal set: + +* .NET Framework 4.x SDK +* C# Compiler +* Visual Studio C++ core features +* VC++ toolset +* Windows 10 SDK (10.0.10240.0 - required) +* Just-in-time debugger +* Windows 8 SDK (includes mscoree.lib required by clrchecktask) +* C++/CLI support +* Windows Universal C Runtime +* Git for Windows +* .NET Framework 3.5 development tools +* Github extension for Visual Studio + +After a while, Visual Studio will be ready. + +#### Flex and Bison + +Install it using [chocolatey](https://www.wireshark.org/docs/wsdg_html_chunked/ChSetupWin32.html): + +``` +choco install -y winflexbison +``` + +Chocolatey installs these tools into the hidden directory `C:\ProgramData\chocolatey\lib\winflexbison\tools`. + +#### OpenSSL + +Icinga 2 requires the OpenSSL library. [Download](http://slproweb.com/products/Win32OpenSSL.html) +and install it into the default path. + +Once asked for `Copy OpenSSLs DLLs to` select `The Windows system directory`. That way CMake/Visual Studio +will automatically detect them for builds and packaging. + +> **Note** +> +> We cannot use the chocolatey package as this one does not provide any development headers. +> +> Choose 1.0.2 LTS from manual downloads for best compatibility if unsure. + +#### Boost + +In order to use the boost development header and library files you need to [download](http://www.boost.org/users/download/) +Boost and then extract it to e.g. `C:\boost_1_65_1`. + +> **Note** +> +> Just use `C:`, the zip file already contains the sub folder. Extraction takes a while, +> the archive contains more than 10k files. + +For integrating Boost into Visual Studio 2017, open the `Developer Command Prompt` from the start menu, +and navigate to `C:\boost_1_65_1`. + +Execute `bootstrap.bat` first. + +``` +cd C:\boost_1_65_1 +bootstrap.bat +``` + +Once finished, specify the required `toolset` to compile boost against Visual Studio. +This takes quite some time in a Windows VM. + +Visual Studio 2015: + +``` +b2 --toolset=msvc-14.0 +``` + +Visual Studio 2017: + +``` +b2 --toolset=msvc-14.1 +``` + +#### TortoiseGit + +TortoiseGit provides a graphical integration into the Windows explorer. This makes it easier to checkout, commit +and whatnot. + +[Download](https://tortoisegit.org/download/) TortoiseGit on your system. + +In order to clone via Git SSH you also need to create a new directory called `.ssh` +inside your user's home directory. +Therefore open a command prompt (win key, type `cmd`, enter) and run `mkdir .ssh`. +Add your `id_rsa` private key and `id_rsa.pub` public key files into that directory. + +Start the setup routine and choose `OpenSSH` as default secure transport when asked. + +Open a Windows Explorer window and navigate into + +Version | Project Location +--------------------|------------------------------ +Visual Studio 2015 | `C:\Users\michi\Documents\Visual Studio 2015\Projects` +Visual Studio 2017+ | `C:\Users\michi\source\repos` + +Right click and select `Git Clone` from the context menu. + +Use `ssh://git@github.com/icinga/icinga2.git` for SSH clones, `https://github.com/icinga/icinga2.git` otherwise. + +#### CMake + +Icinga 2 uses CMake to manage the build environment. You can generate the Visual Studio project files +using CMake. [Download](https://cmake.org/download/) and install CMake. Select to add it to PATH for all users +when asked. + +Once setup is completed, open a command prompt and navigate to + +Visual Studio 2015 + +``` +cd C:\Users\\Documents\Visual Studio 2015\Projects\icinga2 +``` + +Visual Studio 2017 + +``` +cd C:\Users\michi\source\repos +``` + +Run CMake with the following command. This generates a new Visual Studio project file called `icinga2.sln`. + +You need to specify the previously installed component paths: + +Variable | Value | Description +----------------------|----------------------------------------------------------------------| +BOOST_ROOT | `C:\boost_1_65_1` | Root path where you've extracted and compiled Boost. +BISON_EXECUTABLE | `C:\ProgramData\chocolatey\lib\winflexbison\tools\win_bison.exe` | Path to the Bison executable. +FLEX_EXECUTABLE | `C:\ProgramData\chocolatey\lib\winflexbison\tools\win_flex.exe` | Path to the Flex executable. +ICINGA2_WITH_MYSQL | OFF | Requires extra setup for MySQL if set to `ON`. Not supported for client setups. +ICINGA2_WITH_PGSQL | OFF | Requires extra setup for PgSQL if set to `ON`. Not supported for client setups. +ICINGA2_UNITY_BUILD | OFF | Disable unity builds for development environments. + +Tip: If you have previously opened a terminal, run `refreshenv` to re-read updated PATH variables. + +``` +cmake . -DBOOST_ROOT=C:\boost_1_65_1 -DBISON_EXECUTABLE=C:\ProgramData\chocolatey\lib\winflexbison\tools\win_bison.exe -DFLEX_EXECUTABLE=C:\ProgramData\chocolatey\lib\winflexbison\tools\win_flex.exe -DICINGA2_WITH_MYSQL=OFF -DICINGA2_WITH_PGSQL=OFF -DICINGA2_UNITY_BUILD=OFF +``` + +Best is write a small batch/Powershell script which just executes these lines. + + +#### Icinga 2 in Visual Studio + +Navigate to + +Version | Project location +--------------------|------------------------- +Visual Studio 2015 | `C:\Users\michi\Documents\Visual Studio 2015\Projects\icinga2` +Visual Studio 2017+ | `C:\Users\michi\source\repos\icinga2` + +Open `icinga2.sln`. Log into Visual Studio when asked. + +On the right panel, select to build the `Bin/icinga-app` solution. + +The executable binaries are located in `Bin\Release\Debug` in your `icinga2` +project directory. + +Navigate there and run `icinga2.exe --version`. + +Example for Visual Studio 2017: + +``` +cd C:\Users\michi\source\repos\icinga2\Bin\Release\Debug +icinga2.exe --version +``` + + +#### Release Package + +CMake uses CPack and NSIS to create the setup executable including all binaries and libraries +in addition to setup dialogues and configuration. Therefore we’ll need to install [NSIS](http://nsis.sourceforge.net/Download) +first. + +We also need to install the Windows Installer XML (WIX) toolset. + +``` +choco install -y wixtoolset +``` + +Once completed open an administrative shell and navigate to your Visual Studio project. +Let CMake to build a release package. + +``` +cd "c:\Users\michi\Documents\Visual Studio 2015\Projects\icinga2" +cmake --build . --target PACKAGE --config Release +``` + +Note: This will still use the debug builds. A yet more clean approach +is to run CMake with changed release parameters beforehand and then +re-run the release package builder. + +``` +C:\Users\michi\Documents\Visual Studio 2015\Projects\icinga2> +cmake . -DCPACK_GENERATOR=WIX -DCMAKE_BUILD_TYPE=Release -DBOOST_ROOT=C:\boost_1_65_1 -DBISON_EXECUTABLE=C:\ProgramData\chocolatey\lib\winflexbison\tools\win_bison.exe -DFLEX_EXECUTABLE=C:\ProgramData\chocolatey\lib\winflexbison\tools\win_flex.exe -DICINGA2_WITH_MYSQL=OFF -DICINGA2_WITH_PGSQL=OFF -DICINGA2_UNITY_BUILD=OFF + +cmake --build . --target PACKAGE --config Release +``` + +Again, put these lines into a batch/Powershell script and execute that. + + + +### Embedded Dev Env: Pi + +> **Note** +> +> This isn't officially supported yet, just a few hints how you can do it yourself. + +armhf on Raspberry Pi, ccache: + +``` +apt install -y ccache + +/usr/sbin/update-ccache-symlinks + +echo 'export PATH="/usr/lib/ccache:$PATH"' | tee -a ~/.bashrc + +source ~/.bashrc && echo $PATH +``` + +## Package Builds + +This documentation is explicitly meant for packagers and the Icinga +build infrastructure. + +The following requirements need to be fulfilled in order to build the +Icinga application using a dist tarball (including notes for distributions): + +* cmake >= 2.6 +* GNU make (make) or ninja-build +* C++ compiler which supports C++11 + - RHEL/Fedora/SUSE: gcc-c++ >= 4.7 (extra Developer Tools on RHEL5/6 see below) + - Debian/Ubuntu: build-essential + - Alpine: build-base + - you can also use clang++ +* pkg-config +* OpenSSL library and header files >= 1.0.1 + - RHEL/Fedora: openssl-devel + - SUSE: libopenssl-devel (for SLES 11: libopenssl1-devel) + - Debian/Ubuntu: libssl-dev + - Alpine: libressl-dev +* Boost library and header files >= 1.48.0 + - RHEL/Fedora: boost148-devel + - Debian/Ubuntu: libboost-all-dev + - Alpine: boost-dev +* GNU bison (bison) +* GNU flex (flex) >= 2.5.35 +* Systemd headers + - Only required when using Systemd + - Debian/Ubuntu: libsystemd-dev + - RHEL/Fedora: systemd-devel + +### Optional features + +* MySQL (disable with CMake variable `ICINGA2_WITH_MYSQL` to `OFF`) + - RHEL/Fedora: mysql-devel + - SUSE: libmysqlclient-devel + - Debian/Ubuntu: default-libmysqlclient-dev | libmysqlclient-dev + - Alpine: mariadb-dev +* PostgreSQL (disable with CMake variable `ICINGA2_WITH_PGSQL` to `OFF`) + - RHEL/Fedora: postgresql-devel + - Debian/Ubuntu: libpq-dev + - postgresql-dev on Alpine +* YAJL (Faster JSON library) + - RHEL/Fedora: yajl-devel + - Debian: libyajl-dev + - Alpine: yajl-dev +* libedit (CLI console) + - RHEL/Fedora: libedit-devel on CentOS (RHEL requires rhel-7-server-optional-rpms) + - Debian/Ubuntu/Alpine: libedit-dev +* Termcap (only required if libedit doesn't already link against termcap/ncurses) + - RHEL/Fedora: libtermcap-devel + - Debian/Ubuntu: (not necessary) + +### Special requirements + +**FreeBSD**: libexecinfo (automatically used when Icinga 2 is installed via port or package) + +**RHEL6**: Requires a newer boost version which is available on packages.icinga.com +with a version suffixed name. + +### Runtime user environment + +By default Icinga will run as user `icinga` and group `icinga`. Additionally the +external command pipe and livestatus features require a dedicated command group +`icingacmd`. You can choose your own user/group names and pass them to CMake +using the `ICINGA2_USER`, `ICINGA2_GROUP` and `ICINGA2_COMMAND_GROUP` variables. + +``` +# groupadd icinga +# groupadd icingacmd +# useradd -c "icinga" -s /sbin/nologin -G icingacmd -g icinga icinga +``` + +On Alpine (which uses ash busybox) you can run: + +``` +# addgroup -S icinga +# addgroup -S icingacmd +# adduser -S -D -H -h /var/spool/icinga2 -s /sbin/nologin -G icinga -g icinga icinga +# adduser icinga icingacmd +``` + +Add the web server user to the icingacmd group in order to grant it write +permissions to the external command pipe and livestatus socket: +``` +# usermod -a -G icingacmd www-data +``` + +Make sure to replace "www-data" with the name of the user your web server +is running as. + +### Building Icinga 2: Example + +Once you have installed all the necessary build requirements you can build +Icinga 2 using the following commands: + +``` +$ mkdir release && cd release +$ cmake .. +$ cd .. +$ make -C release +$ make install -C release +``` + +You can specify an alternative installation prefix using `-DCMAKE_INSTALL_PREFIX`: + +``` +$ cmake .. -DCMAKE_INSTALL_PREFIX=/tmp/icinga2 +``` + +### CMake Variables + +In addition to `CMAKE_INSTALL_PREFIX` here are most of the supported Icinga-specific cmake variables. + +For all variables regarding defaults paths on in CMake, see +[GNUInstallDirs](https://cmake.org/cmake/help/latest/module/GNUInstallDirs.html). + +Also see `CMakeLists.txt` for details. + +**System Environment** +- `CMAKE_INSTALL_SYSCONFDIR`: The configuration directory; defaults to `CMAKE_INSTALL_PREFIX/etc` +- `CMAKE_INSTALL_LOCALSTATEDIR`: The state directory; defaults to `CMAKE_INSTALL_PREFIX/var` +- `ICINGA2_CONFIGDIR`: Main config directory; defaults to `CMAKE_INSTALL_SYSCONFDIR/icinga2` usually `/etc/icinga2` +- `ICINGA2_CACHEDIR`: Directory for cache files; defaults to `CMAKE_INSTALL_LOCALSTATEDIR/cache/icinga2` usually `/var/cache/icinga2` +- `ICINGA2_DATADIR`: Data directory for the daemon; defaults to `CMAKE_INSTALL_LOCALSTATEDIR/lib/icinga2` usually `/var/lib/icinga2` +- `ICINGA2_LOGDIR`: Logfiles of the daemon; defaults to `CMAKE_INSTALL_LOCALSTATEDIR/log/icinga2 usually `/var/log/icinga2` +- `ICINGA2_SPOOLDIR`: Spooling directory ; defaults to `CMAKE_INSTALL_LOCALSTATEDIR/spool/icinga2` usually `/var/spool/icinga2` +- `ICINGA2_INITRUNDIR`: Runtime data for the init system; defaults to `CMAKE_INSTALL_LOCALSTATEDIR/run/icinga2` usually `/run/icinga2` +- `ICINGA2_GIT_VERSION_INFO`: Whether to use Git to determine the version number; defaults to `ON` +- `ICINGA2_USER`: The user Icinga 2 should run as; defaults to `icinga` +- `ICINGA2_GROUP`: The group Icinga 2 should run as; defaults to `icinga` +- `ICINGA2_COMMAND_GROUP`: The command group Icinga 2 should use; defaults to `icingacmd` +- `ICINGA2_SYSCONFIGFILE`: Where to put the config file the initscript/systemd pulls it's dirs from; + defaults to `CMAKE_INSTALL_PREFIX/etc/sysconfig/icinga2` +- `ICINGA2_PLUGINDIR`: The path for the Monitoring Plugins project binaries; defaults to `/usr/lib/nagios/plugins` + +**Build Optimization** +- `ICINGA2_UNITY_BUILD`: Whether to perform a unity build; defaults to `ON`. Note: This requires additional memory and is not advised for building VMs, Docker for Mac and embedded hardware. +- `ICINGA2_LTO_BUILD`: Whether to use link time optimization (LTO); defaults to `OFF` + +**Init System** +- `USE_SYSTEMD=ON|OFF`: Use systemd or a classic SysV initscript; defaults to `OFF` +- `INSTALL_SYSTEMD_SERVICE_AND_INITSCRIPT=ON|OFF` Force install both the systemd service definition file + and the SysV initscript in parallel, regardless of how `USE_SYSTEMD` is set. + Only use this for special packaging purposes and if you know what you are doing. + Defaults to `OFF`. + +**Features:** +- `ICINGA2_WITH_CHECKER`: Determines whether the checker module is built; defaults to `ON` +- `ICINGA2_WITH_COMPAT`: Determines whether the compat module is built; defaults to `ON` +- `ICINGA2_WITH_DEMO`: Determines whether the demo module is built; defaults to `OFF` +- `ICINGA2_WITH_HELLO`: Determines whether the hello module is built; defaults to `OFF` +- `ICINGA2_WITH_LIVESTATUS`: Determines whether the Livestatus module is built; defaults to `ON` +- `ICINGA2_WITH_NOTIFICATION`: Determines whether the notification module is built; defaults to `ON` +- `ICINGA2_WITH_PERFDATA`: Determines whether the perfdata module is built; defaults to `ON` +- `ICINGA2_WITH_TESTS`: Determines whether the unit tests are built; defaults to `ON` + +**MySQL or MariaDB:** + +The following settings can be tuned for the MySQL / MariaDB IDO feature. + +- `ICINGA2_WITH_MYSQL`: Determines whether the MySQL IDO module is built; defaults to `ON` +- `MYSQL_CLIENT_LIBS`: Client implementation used (mysqlclient / mariadbclient); defaults searches for `mysqlclient` and `mariadbclient` +- `MYSQL_INCLUDE_DIR`: Directory containing include files for the mysqlclient; default empty - + checking multiple paths like `/usr/include/mysql` + +See [FindMySQL.cmake](third-party/cmake/FindMySQL.cmake) for the implementation. + +**PostgreSQL:** + +The following settings can be tuned for the PostgreSQL IDO feature. + +- `ICINGA2_WITH_PGSQL`: Determines whether the PostgreSQL IDO module is built; defaults to `ON` +- `PostgreSQL_INCLUDE_DIR`: Top-level directory containing the PostgreSQL include directories +- `PostgreSQL_LIBRARY`: File path to PostgreSQL library : libpq.so (or libpq.so.[ver] file) + +See [FindMySQL.cmake](third-party/cmake/FindPostgreSQL.cmake) for the implementation. + +**Version detection:** + +CMake determines the Icinga 2 version number using `git describe` if the +source directory is contained in a Git repository. Otherwise the version number +is extracted from the [VERSION](VERSION) file. This behavior can be +overridden by creating a file called `icinga-version.h.force` in the source +directory. Alternatively the `-DICINGA2_GIT_VERSION_INFO=OFF` option for CMake +can be used to disable the usage of `git describe`. + + +### Building RPMs + +#### Build Environment on RHEL, CentOS, Fedora, Amazon Linux + +Setup your build environment: +``` +yum -y install rpmdevtools +``` + +#### Build Environment on SuSE/SLES + +SLES: +``` +zypper addrepo http://download.opensuse.org/repositories/devel:tools/SLE_12_SP3/devel:tools.repo +zypper refresh +zypper install rpmdevtools spectool +``` + +OpenSuSE: +``` +zypper addrepo http://download.opensuse.org/repositories/devel:tools/openSUSE_Leap_15.0/devel:tools.repo +zypper refresh +zypper install rpmdevtools spectool +``` + +#### Package Builds + +Prepare the rpmbuild directory tree: +``` +cd $HOME +rpmdev-setuptree +``` + +Copy the icinga2.spec file to `rpmbuild/SPEC` or fetch the latest version: +``` +curl https://raw.githubusercontent.com/Icinga/rpm-icinga2/master/icinga2.spec -o $HOME/rpmbuild/SPECS/icinga2.spec +``` + +> **Note** +> +> The above command builds snapshot packages. Change to the `release` branch +> for release package builds. + +Copy the tarball to `rpmbuild/SOURCES` e.g. by using the `spectool` binary +provided with `rpmdevtools`: +``` +cd $HOME/rpmbuild/SOURCES +spectool -g ../SPECS/icinga2.spec + +cd $HOME/rpmbuild +``` + +Install the build dependencies. Example for CentOS 7: +``` +yum -y install libedit-devel ncurses-devel gcc-c++ libstdc++-devel openssl-devel \ +cmake flex bison boost-devel systemd mysql-devel postgresql-devel httpd \ +selinux-policy-devel checkpolicy selinux-policy selinux-policy-doc +``` + +Note: If you are using Amazon Linux, systemd is not required. + +A shorter way is available using the `yum-builddep` command on RHEL based systems: +``` +yum-builddep SPECS/icinga2.spec +``` + +Build the RPM: +``` +rpmbuild -ba SPECS/icinga2.spec +``` + +#### Additional Hints + +##### SELinux policy module + +The following packages are required to build the SELinux policy module: + +* checkpolicy +* selinux-policy (selinux-policy on CentOS 6, selinux-policy-devel on CentOS 7) +* selinux-policy-doc + +##### RHEL/CentOS 6 + +The RedHat Developer Toolset is required for building Icinga 2 beforehand. +This contains a modern version of flex and a C++ compiler which supports +C++11 features. +``` +cat >/etc/yum.repos.d/devtools-2.repo <$HOME/.rpmmacros < **WARNING:** This information is outdated! + +Setup your build environment on Debian/Ubuntu, copy the 'debian' directory from +the Debian packaging Git repository (https://github.com/Icinga/deb-icinga2) +into your source tree and run the following command: +``` +$ dpkg-buildpackage -uc -us +``` + +## Build Alpine Linux packages + +A simple way to setup a build environment is installing Alpine in a chroot. +In this way, you can set up an Alpine build environment in a chroot under a +different Linux distro. +There is a script that simplifies these steps with just two commands, and +can be found [here](https://github.com/alpinelinux/alpine-chroot-install). + +Once the build environment is installed, you can setup the system to build +the packages by following [this document](https://wiki.alpinelinux.org/wiki/Creating_an_Alpine_package). + +# Build Post Install Tasks + +After building Icinga 2 yourself, your package build system should at least run the following post +install requirements: + +* enable the `checker`, `notification` and `mainlog` feature by default +* run 'icinga2 api setup' in order to enable the `api` feature and generate SSL certificates for the node + +## Run Icinga 2 + +Icinga 2 comes with a binary that takes care of loading all the relevant +components (e.g. for check execution, notifications, etc.): +``` +# icinga2 daemon +[2016-12-08 16:44:24 +0100] information/cli: Icinga application loader (version: v2.5.4-231-gb10a6b7; debug) +[2016-12-08 16:44:24 +0100] information/cli: Loading configuration file(s). +[2016-12-08 16:44:25 +0100] information/ConfigItem: Committing config item(s). +... +``` + +### Init Script + +Icinga 2 can be started as a daemon using the provided init script: +``` +# /etc/init.d/icinga2 +Usage: /etc/init.d/icinga2 {start|stop|restart|reload|checkconfig|status} +``` + +### Systemd + +If your distribution uses Systemd: +``` +# systemctl {start|stop|reload|status|enable|disable} icinga2 +``` + +In case the distribution is running Systemd >227, you'll also +need to package and install the `etc/initsystem/icinga2.service.limits.conf` +file into `/etc/systemd/system/icinga2.service.d`. + +### openrc + +Or if your distribution uses openrc (like Alpine): +``` +# rc-service icinga2 +Usage: /etc/init.d/icinga2 {start|stop|restart|reload|checkconfig|status} + ``` + +Note: the openrc's init.d is not shipped by default. +A working init.d with openrc can be found here: (https://git.alpinelinux.org/cgit/aports/plain/community/icinga2/icinga2.initd). If you have customized some path, edit the file and adjust it according with your setup. +Those few steps can be followed: +``` +# wget https://git.alpinelinux.org/cgit/aports/plain/community/icinga2/icinga2.initd +# mv icinga2.initd /etc/init.d/icinga2 +# chmod +x /etc/init.d/icinga2 +``` + +Icinga 2 reads a single configuration file which is used to specify all +configuration settings (global settings, hosts, services, etc.). The +configuration format is explained in detail in the [doc/](doc/) directory. + +By default `make install` installs example configuration files in +`/usr/local/etc/icinga2` unless you have specified a different prefix or +sysconfdir. + + +### Windows + +The Windows MSI packages are located at https://packages.icinga.com/windows/ + +#### Requirements + +* 32 or 64-bit system +* Visual Studio >= 14 2015 +* CMake >= 2.6 +* OpenSSL >= 1.0.1 +* Flex and Bison + +##### Visual Studio + +Download the community edition from [visualstudio.com](https://www.visualstudio.com/en/downloads/) + +Workloads to install: +* C++ Desktop +* .NET Desktop + +##### OpenSSL for Icinga + +Download custom OpenSSL builds from [openssl-windows GitHub project](https://github.com/Icinga/openssl-windows/releases). + +You need to install a binary dist version to 'C:\\Program Files\\OpenSSL'. + +The Powershell script `.\tools\win32\download-openssl.ps1` can be used for automated downloads. + +##### Chocolatey + +A simple package manager for Windows, please see [install instructions](https://chocolatey.org/install). + +##### Git + +Use Chocolatey, see [package details](https://chocolatey.org/packages/git). + +``` +choco install git +``` + +##### Flex / Bison + +Use Chocolatey, see [package details](https://chocolatey.org/packages/winflexbison3). + +``` +choco install winflexbison3 +``` + +##### CMake + +Use Chocolatey, see [package details](https://chocolatey.org/packages/cmake) +or download from: [cmake.org](https://cmake.org/download/) + +``` +choco install cmake +``` + +##### WIX + +Use Chocolatey, see [package details](https://chocolatey.org/packages/wixtoolset). + +``` +choco install wixtoolset +``` + +##### Boost + +Download third party Windows binaries from: [boost.org](http://www.boost.org/users/download/) + +For example: `https://dl.bintray.com/boostorg/release/1.65.1/binaries/boost_1_65_1-msvc-14.1-64.exe` + +*Warning:* +* Must match your Visual Studio version! +* CMake might not support the latest Boost version (we used CMake 3.10 and Boost 1_65_1) + +Run the installer exe. + + +#### Build Icinga 2 + +Run with VC Native x64 Command Prompt: + +``` +powershell .\tools\win32\configure.ps1 +powershell .\tools\win32\build.ps1 +powershell .\tools\win32\test.ps1 +``` + +See these scripts for details. + +#### CI: AppVeyor + +We are building [Icinga 2 with AppVeyor](https://ci.appveyor.com/project/icinga/icinga2) for testing and CI integration. + +Please check `appveyor.yml` for instructions. + + + +## Advanced Development Tips + +### GDB Pretty Printers Install the `boost`, `python` and `icinga2` pretty printers. Absolute paths are required, so please make sure to update the installation paths accordingly (`pwd`). @@ -118,217 +1444,20 @@ the duplicate import in your `~/.gdbinit` file. RuntimeError: pretty-printer already registered: libstdc++-v6 -### GDB Run -Call GDB with the binary (`/usr/sbin/icinga2` is a wrapper script calling -`/usr/lib64/icinga2/sbin/icinga2` since 2.4) and all arguments and run it in foreground. +## Development Tests - # gdb --args /usr/lib64/icinga2/sbin/icinga2 daemon -x debug --no-stack-rlimit +Build the binaries and run the tests. -The exact path to the Icinga 2 binary differs on each distribution. On Ubuntu -it is installed into `/usr/lib/x86_64-linux-gnu/icinga2/sbin/icinga2` on 64-bit systems -for example. - -> **Note** -> -> If gdb tells you it's missing debug symbols, quit gdb and install -> them: `Missing separate debuginfos, use: debuginfo-install ...` - -Run the application. - - (gdb) r - -Kill the running application. - - (gdb) k - -Continue after breakpoint. - - (gdb) c - -### GDB Core Dump - -Either attach to the running process using `gdb -p PID` or start -a new gdb run. - - (gdb) r - (gdb) generate-core-file - -### GDB Backtrace - -If Icinga 2 aborted its operation abnormally, generate a backtrace. - - (gdb) bt - (gdb) thread apply all bt full - -If gdb stops at a SIGPIPE signal please disable the signal before -running Icinga 2. - - (gdb) handle SIGPIPE nostop noprint pass - (gdb) r - -If you create a [new issue](https://github.com/Icinga/icinga2/issues), -make sure to attach as much detail as possible. - -### GDB Backtrace from Running Process - -If Icinga 2 is still running, generate a full backtrace from the running -process and store it into a new file (e.g. for debugging dead locks): - -Icinga 2 runs with 2 processes, therefore generate two backtrace logs -and add them to the GitHub issue. ``` -for pid in $(pidof icinga2); do gdb -p $pid -batch -ex "thread apply all bt full" -ex "detach" -ex "q" > gdb_bt_${pid}_`date +%s`.log; done +make -j4 -C debug +make test -C debug ``` -### GDB Thread List from Running Process - -Instead of a full backtrace, you sometimes just need a list of running threads. +Run a specific boost test: ``` -for pid in $(pidof icinga2); do gdb -p $pid -batch -ex "info threads" -ex "detach" -ex "q" > gdb_threads_${pid}_`date +%s`.log; done +debug/Bin/Debug/boosttest-test-base --run_test=remote_url ``` -### GDB Backtrace Stepping - -Identifying the problem may require stepping into the backtrace, analysing -the current scope, attributes, and possible unmet requirements. `p` prints -the value of the selected variable or function call result. - - (gdb) up - (gdb) down - (gdb) p checkable - (gdb) p checkable.px->m_Name - - -### GDB Breakpoints - -To set a breakpoint to a specific function call, or file specific line. - - (gdb) b checkable.cpp:125 - (gdb) b icinga::Checkable::SetEnablePerfdata - -GDB will ask about loading the required symbols later, select `yes` instead -of `no`. - -Then run Icinga 2 until it reaches the first breakpoint. Continue with `c` -afterwards. - - (gdb) run - (gdb) c - -If you want to delete all breakpoints, use `d` and select `yes`. - - (gdb) d - -> **Tip** -> -> When debugging exceptions, set your breakpoint like this: `b __cxa_throw`. - -Breakpoint Example: - - (gdb) b __cxa_throw - (gdb) r - (gdb) up - .... - (gdb) up - #11 0x00007ffff7cbf9ff in icinga::Utility::GlobRecursive(icinga::String const&, icinga::String const&, boost::function const&, int) (path=..., pattern=..., callback=..., type=1) - at /home/michi/coding/icinga/icinga2/lib/base/utility.cpp:609 - 609 callback(cpath); - (gdb) l - 604 - 605 #endif /* _WIN32 */ - 606 - 607 std::sort(files.begin(), files.end()); - 608 BOOST_FOREACH(const String& cpath, files) { - 609 callback(cpath); - 610 } - 611 - 612 std::sort(dirs.begin(), dirs.end()); - 613 BOOST_FOREACH(const String& cpath, dirs) { - (gdb) p files - $3 = std::vector of length 11, capacity 16 = {{static NPos = 18446744073709551615, m_Data = "/etc/icinga2/conf.d/agent.conf"}, {static NPos = 18446744073709551615, - m_Data = "/etc/icinga2/conf.d/commands.conf"}, {static NPos = 18446744073709551615, m_Data = "/etc/icinga2/conf.d/downtimes.conf"}, {static NPos = 18446744073709551615, - m_Data = "/etc/icinga2/conf.d/groups.conf"}, {static NPos = 18446744073709551615, m_Data = "/etc/icinga2/conf.d/notifications.conf"}, {static NPos = 18446744073709551615, - m_Data = "/etc/icinga2/conf.d/satellite.conf"}, {static NPos = 18446744073709551615, m_Data = "/etc/icinga2/conf.d/services.conf"}, {static NPos = 18446744073709551615, - m_Data = "/etc/icinga2/conf.d/templates.conf"}, {static NPos = 18446744073709551615, m_Data = "/etc/icinga2/conf.d/test.conf"}, {static NPos = 18446744073709551615, - m_Data = "/etc/icinga2/conf.d/timeperiods.conf"}, {static NPos = 18446744073709551615, m_Data = "/etc/icinga2/conf.d/users.conf"}} - - -## Core Dump - -When the Icinga 2 daemon crashes with a `SIGSEGV` signal -a core dump file should be written. This will help -developers to analyze and fix the problem. - -### Core Dump File Size Limit - -This requires setting the core dump file size to `unlimited`. - -Example for Systemd: - - vim /usr/lib/systemd/system/icinga2.service - - [Service] - ... - LimitCORE=infinity - - systemctl daemon-reload - - systemctl restart icinga2 - -Example for init script: - - vim /etc/init.d/icinga2 - ... - ulimit -c unlimited - - service icinga2 restart - -Verify that the Icinga 2 process core file size limit is set to `unlimited`. - - cat /proc/`pidof icinga2`/limits - ... - Max core file size unlimited unlimited bytes - - -### Core Dump Kernel Format - -The Icinga 2 daemon runs with the SUID bit set. Therefore you need -to explicitly enable core dumps for SUID on Linux. - - sysctl -w fs.suid_dumpable=1 - -Adjust the coredump kernel format and file location on Linux: - - sysctl -w kernel.core_pattern=/var/lib/cores/core.%e.%p - - install -m 1777 -d /var/lib/cores - -MacOS: - - sysctl -w kern.corefile=/cores/core.%P - - chmod 777 /cores - -### Core Dump Analysis - -Once Icinga 2 crashes again a new coredump file will be written. Please -attach this file to your bug report in addition to the general details. - -Simple test case for a `SIGSEGV` simulation with `sleep`: - - ulimit -c unlimited - sleep 1800& - [1] - kill -SEGV - gdb `which sleep` /var/lib/cores/core.sleep. - (gdb) bt - rm /var/lib/cores/core.sleep.* - -Analyzing Icinga 2: - - gdb /usr/lib64/icinga2/sbin/icinga2 core.icinga2. - (gdb) bt - From c6fc144102eada72ff8146f4be00fd48ceb0d423 Mon Sep 17 00:00:00 2001 From: Michael Friedrich Date: Mon, 3 Dec 2018 09:49:42 +0100 Subject: [PATCH 02/11] Docs: Delete extra docs for windows builds This should be visible in icinga.com/docs, no extra docs needed. --- BUILD_WINDOWS.md | 103 ----------------------------------------------- 1 file changed, 103 deletions(-) delete mode 100644 BUILD_WINDOWS.md diff --git a/BUILD_WINDOWS.md b/BUILD_WINDOWS.md deleted file mode 100644 index 28e75259e..000000000 --- a/BUILD_WINDOWS.md +++ /dev/null @@ -1,103 +0,0 @@ -# Build Icinga 2 on Windows - -The Icinga Project is providing Windows MSI packages under https://packages.icinga.com/windows/ - -> **Note:** -> This is a developer documentation on how to build Icinga 2 on Windows! - -Also see [INSTALL.md](INSTALL.md) for Linux build instructions. - -## Requirements - -* 32 or 64-bit system -* Visual Studio >= 14 2015 -* CMake >= 2.6 -* OpenSSL >= 1.0.1 -* Flex and Bison - -## Install Requirements - -**Visual Studio** - -Download from [visualstudio.com](https://www.visualstudio.com/en/downloads/) - -The Community Edition is available for free, and is what we use to build. - -Workloads to install: -* C++ Desktop -* .NET Desktop - -**OpenSSL for Icinga** - -See our [openssl-windows GitHub project](https://github.com/Icinga/openssl-windows). - -You will need to install a binary dist version to 'C:\\Program Files\\OpenSSL'. - -There is a Powershell script to help you downloading: `.\tools\win32\download-openssl.ps1` - -**Chocolatey** - -A simple package manager for Windows, please see [install instructions](https://chocolatey.org/install). - -**Git** - -Best to use Chocolatey, see [package details](https://chocolatey.org/packages/git). - -``` -choco install git -``` - -**Flex / Bison** - -Best to use Chocolatey, see [package details](https://chocolatey.org/packages/winflexbison3). - -``` -choco install winflexbison3 -``` - -**CMake** - -Best to use Chocolatey, see [package details](https://chocolatey.org/packages/cmake) -or download from: [cmake.org](https://cmake.org/download/) - -``` -choco install cmake -``` - -**WIX** - -Best to use Chocolatey, see [package details](https://chocolatey.org/packages/wixtoolset). - -``` -choco install wixtoolset -``` - -**Boost** - -Download third party Windows binaries from: [boost.org](http://www.boost.org/users/download/) - -For example: `https://dl.bintray.com/boostorg/release/1.65.1/binaries/boost_1_65_1-msvc-14.1-64.exe` - -*Warnings:* -* Must match your Visual Studio version! -* CMake might not support the latest Boost version (we used CMake 3.10 and Boost 1_65_1) - -Run the installer exe. - -## Build Icinga 2 - -Run with VC Native x64 Command Prompt: - -``` -powershell .\tools\win32\configure.ps1 -powershell .\tools\win32\build.ps1 -powershell .\tools\win32\test.ps1 -``` - -See these scripts for details. - -## AppVeyor - -We are building [Icinga 2 with AppVeyor](https://ci.appveyor.com/project/icinga/icinga2) for testing and CI integration. - -Please check `appveyor.yml` for our instructions. From 3827e15c4abf49b4ecdbd23d36e76cb4fdab67a0 Mon Sep 17 00:00:00 2001 From: Michael Friedrich Date: Mon, 3 Dec 2018 09:50:39 +0100 Subject: [PATCH 03/11] Docs: Simplify INSTALL.md and point to icinga.com/docs --- INSTALL.md | 396 +---------------------------------------------------- 1 file changed, 2 insertions(+), 394 deletions(-) diff --git a/INSTALL.md b/INSTALL.md index e0573e0ec..d6121e999 100644 --- a/INSTALL.md +++ b/INSTALL.md @@ -16,398 +16,6 @@ not use the official packages it is advisable to build your own Debian or RPM packages. You can use the source packages from [packages.icinga.com](https://packages.icinga.com) for this purpose. -> **Disclaimer** -> -> This information is intended for developers and packagers. It might be incomplete or unclear -> in some cases. Ensure to check our [packaging scripts on GitHub](https://github.com/Icinga/icinga-packaging) too! +For a more detailed introduction into developing Icinga and building +packages, please visit the [online documentation](https://icinga.com/docs/icinga2/latest/doc/21-development/). -# Build Requirements - -The following requirements need to be fulfilled in order to build the -application using a dist tarball (including notes for distributions): - -* cmake >= 2.6 -* GNU make (make) or ninja-build -* C++ compiler which supports C++11 - - RHEL/Fedora/SUSE: gcc-c++ >= 4.7 (extra Developer Tools on RHEL5/6 see below) - - Debian/Ubuntu: build-essential - - Alpine: build-base - - you can also use clang++ -* pkg-config -* OpenSSL library and header files >= 1.0.1 - - RHEL/Fedora: openssl-devel - - SUSE: libopenssl-devel (for SLES 11: libopenssl1-devel) - - Debian/Ubuntu: libssl-dev - - Alpine: libressl-dev -* Boost library and header files >= 1.48.0 - - RHEL/Fedora: boost148-devel - - Debian/Ubuntu: libboost-all-dev - - Alpine: boost-dev -* GNU bison (bison) -* GNU flex (flex) >= 2.5.35 -* Systemd headers - - Only required when using Systemd - - Debian/Ubuntu: libsystemd-dev - - RHEL/Fedora: systemd-devel - -## Optional features - -* MySQL (disable with CMake variable `ICINGA2_WITH_MYSQL` to `OFF`) - - RHEL/Fedora: mysql-devel - - SUSE: libmysqlclient-devel - - Debian/Ubuntu: default-libmysqlclient-dev | libmysqlclient-dev - - Alpine: mariadb-dev -* PostgreSQL (disable with CMake variable `ICINGA2_WITH_PGSQL` to `OFF`) - - RHEL/Fedora: postgresql-devel - - Debian/Ubuntu: libpq-dev - - postgresql-dev on Alpine -* YAJL (Faster JSON library) - - RHEL/Fedora: yajl-devel - - Debian: libyajl-dev - - Alpine: yajl-dev -* libedit (CLI console) - - RHEL/Fedora: libedit-devel on CentOS (RHEL requires rhel-7-server-optional-rpms) - - Debian/Ubuntu/Alpine: libedit-dev -* Termcap (only required if libedit doesn't already link against termcap/ncurses) - - RHEL/Fedora: libtermcap-devel - - Debian/Ubuntu: (not necessary) - -## Special requirements - -**FreeBSD**: libexecinfo (automatically used when Icinga 2 is installed via port or package) - -**RHEL6**: Requires a newer boost version which is available on packages.icinga.com -with a version suffixed name. - -## Runtime user environment - -By default Icinga will run as user `icinga` and group `icinga`. Additionally the -external command pipe and livestatus features require a dedicated command group -`icingacmd`. You can choose your own user/group names and pass them to CMake -using the `ICINGA2_USER`, `ICINGA2_GROUP` and `ICINGA2_COMMAND_GROUP` variables. - -``` -# groupadd icinga -# groupadd icingacmd -# useradd -c "icinga" -s /sbin/nologin -G icingacmd -g icinga icinga -``` - -On Alpine (which uses ash busybox) you can run: - -``` -# addgroup -S icinga -# addgroup -S icingacmd -# adduser -S -D -H -h /var/spool/icinga2 -s /sbin/nologin -G icinga -g icinga icinga -# adduser icinga icingacmd -``` - -Add the web server user to the icingacmd group in order to grant it write -permissions to the external command pipe and livestatus socket: -``` -# usermod -a -G icingacmd www-data -``` - -Make sure to replace "www-data" with the name of the user your web server -is running as. - -# Building Icinga 2 - -Once you have installed all the necessary build requirements you can build -Icinga 2 using the following commands: - -``` -$ mkdir release && cd release -$ cmake .. -$ cd .. -$ make -C release -$ make install -C release -``` - -You can specify an alternative installation prefix using `-DCMAKE_INSTALL_PREFIX`: - -``` -$ cmake .. -DCMAKE_INSTALL_PREFIX=/tmp/icinga2 -``` - -## CMake Variables - -In addition to `CMAKE_INSTALL_PREFIX` here are most of the supported Icinga-specific cmake variables. - -For all variables regarding defaults paths on in CMake, see -[GNUInstallDirs](https://cmake.org/cmake/help/latest/module/GNUInstallDirs.html). - -Also see `CMakeLists.txt` for details. - -**System Environment** -- `CMAKE_INSTALL_SYSCONFDIR`: The configuration directory; defaults to `CMAKE_INSTALL_PREFIX/etc` -- `CMAKE_INSTALL_LOCALSTATEDIR`: The state directory; defaults to `CMAKE_INSTALL_PREFIX/var` -- `ICINGA2_CONFIGDIR`: Main config directory; defaults to `CMAKE_INSTALL_SYSCONFDIR/icinga2` usually `/etc/icinga2` -- `ICINGA2_CACHEDIR`: Directory for cache files; defaults to `CMAKE_INSTALL_LOCALSTATEDIR/cache/icinga2` usually `/var/cache/icinga2` -- `ICINGA2_DATADIR`: Data directory for the daemon; defaults to `CMAKE_INSTALL_LOCALSTATEDIR/lib/icinga2` usually `/var/lib/icinga2` -- `ICINGA2_LOGDIR`: Logfiles of the daemon; defaults to `CMAKE_INSTALL_LOCALSTATEDIR/log/icinga2 usually `/var/log/icinga2` -- `ICINGA2_SPOOLDIR`: Spooling directory ; defaults to `CMAKE_INSTALL_LOCALSTATEDIR/spool/icinga2` usually `/var/spool/icinga2` -- `ICINGA2_INITRUNDIR`: Runtime data for the init system; defaults to `CMAKE_INSTALL_LOCALSTATEDIR/run/icinga2` usually `/run/icinga2` -- `ICINGA2_GIT_VERSION_INFO`: Whether to use Git to determine the version number; defaults to `ON` -- `ICINGA2_USER`: The user Icinga 2 should run as; defaults to `icinga` -- `ICINGA2_GROUP`: The group Icinga 2 should run as; defaults to `icinga` -- `ICINGA2_COMMAND_GROUP`: The command group Icinga 2 should use; defaults to `icingacmd` -- `ICINGA2_SYSCONFIGFILE`: Where to put the config file the initscript/systemd pulls it's dirs from; - defaults to `CMAKE_INSTALL_PREFIX/etc/sysconfig/icinga2` -- `ICINGA2_PLUGINDIR`: The path for the Monitoring Plugins project binaries; defaults to `/usr/lib/nagios/plugins` - -**Build Optimization** -- `ICINGA2_UNITY_BUILD`: Whether to perform a unity build; defaults to `ON`. Note: This requires additional memory and is not advised for building VMs, Docker for Mac and embedded hardware. -- `ICINGA2_LTO_BUILD`: Whether to use link time optimization (LTO); defaults to `OFF` - -**Init System** -- `USE_SYSTEMD=ON|OFF`: Use systemd or a classic SysV initscript; defaults to `OFF` -- `INSTALL_SYSTEMD_SERVICE_AND_INITSCRIPT=ON|OFF` Force install both the systemd service definition file - and the SysV initscript in parallel, regardless of how `USE_SYSTEMD` is set. - Only use this for special packaging purposes and if you know what you are doing. - Defaults to `OFF`. - -**Features:** -- `ICINGA2_WITH_CHECKER`: Determines whether the checker module is built; defaults to `ON` -- `ICINGA2_WITH_COMPAT`: Determines whether the compat module is built; defaults to `ON` -- `ICINGA2_WITH_DEMO`: Determines whether the demo module is built; defaults to `OFF` -- `ICINGA2_WITH_HELLO`: Determines whether the hello module is built; defaults to `OFF` -- `ICINGA2_WITH_LIVESTATUS`: Determines whether the Livestatus module is built; defaults to `ON` -- `ICINGA2_WITH_NOTIFICATION`: Determines whether the notification module is built; defaults to `ON` -- `ICINGA2_WITH_PERFDATA`: Determines whether the perfdata module is built; defaults to `ON` -- `ICINGA2_WITH_TESTS`: Determines whether the unit tests are built; defaults to `ON` - -**MySQL or MariaDB:** - -The following settings can be tuned for the MySQL / MariaDB IDO feature. - -- `ICINGA2_WITH_MYSQL`: Determines whether the MySQL IDO module is built; defaults to `ON` -- `MYSQL_CLIENT_LIBS`: Client implementation used (mysqlclient / mariadbclient); defaults searches for `mysqlclient` and `mariadbclient` -- `MYSQL_INCLUDE_DIR`: Directory containing include files for the mysqlclient; default empty - - checking multiple paths like `/usr/include/mysql` - -See [FindMySQL.cmake](third-party/cmake/FindMySQL.cmake) for the implementation. - -**PostgreSQL:** - -The following settings can be tuned for the PostgreSQL IDO feature. - -- `ICINGA2_WITH_PGSQL`: Determines whether the PostgreSQL IDO module is built; defaults to `ON` -- `PostgreSQL_INCLUDE_DIR`: Top-level directory containing the PostgreSQL include directories -- `PostgreSQL_LIBRARY`: File path to PostgreSQL library : libpq.so (or libpq.so.[ver] file) - -See [FindMySQL.cmake](third-party/cmake/FindPostgreSQL.cmake) for the implementation. - -**Version detection:** - -CMake determines the Icinga 2 version number using `git describe` if the -source directory is contained in a Git repository. Otherwise the version number -is extracted from the [VERSION](VERSION) file. This behavior can be -overridden by creating a file called `icinga-version.h.force` in the source -directory. Alternatively the `-DICINGA2_GIT_VERSION_INFO=OFF` option for CMake -can be used to disable the usage of `git describe`. - -# Building packages - -> **WARNING:** Some of this information is outdated! - -## Building RPMs - -### Build Environment on RHEL, CentOS, Fedora, Amazon Linux - -Setup your build environment: -``` -yum -y install rpmdevtools -``` - -### Build Environment on SuSE/SLES - -SLES: -``` -zypper addrepo http://download.opensuse.org/repositories/devel:tools/SLE_12_SP2/devel:tools.repo -zypper refresh -zypper install rpmdevtools spectool -``` - -OpenSuSE: -``` -zypper addrepo http://download.opensuse.org/repositories/devel:tools/openSUSE_Leap_42.3/devel:tools.repo -zypper refresh -zypper install rpmdevtools spectool -``` - -### Package Builds - -Prepare the rpmbuild directory tree: -``` -cd $HOME -rpmdev-setuptree -``` - -Copy the icinga2.spec file to `rpmbuild/SPEC` or fetch the latest version: -``` -curl https://raw.githubusercontent.com/Icinga/rpm-icinga2/master/icinga2.spec -o $HOME/rpmbuild/SPECS/icinga2.spec -``` - -> **Note** -> -> The above command builds snapshot packages. Change to the `release` branch -> for release package builds. - -Copy the tarball to `rpmbuild/SOURCES` e.g. by using the `spectool` binary -provided with `rpmdevtools`: -``` -cd $HOME/rpmbuild/SOURCES -spectool -g ../SPECS/icinga2.spec - -cd $HOME/rpmbuild -``` - -Install the build dependencies. Example for CentOS 7: -``` -yum -y install libedit-devel ncurses-devel gcc-c++ libstdc++-devel openssl-devel \ -cmake flex bison boost-devel systemd mysql-devel postgresql-devel httpd \ -selinux-policy-devel checkpolicy selinux-policy selinux-policy-doc -``` - -Note: If you are using Amazon Linux, systemd is not required. - -A shorter way is available using the `yum-builddep` command on RHEL based systems: -``` -yum-builddep SPECS/icinga2.spec -``` - -Build the RPM: -``` -rpmbuild -ba SPECS/icinga2.spec -``` - -### Additional Hints - -#### SELinux policy module - -The following packages are required to build the SELinux policy module: - -* checkpolicy -* selinux-policy (selinux-policy on CentOS 6, selinux-policy-devel on CentOS 7) -* selinux-policy-doc - -#### RHEL/CentOS 6 - -The RedHat Developer Toolset is required for building Icinga 2 beforehand. -This contains a modern version of flex and a C++ compiler which supports -C++11 features. -``` -cat >/etc/yum.repos.d/devtools-2.repo <$HOME/.rpmmacros < **WARNING:** This information is outdated! - -Setup your build environment on Debian/Ubuntu, copy the 'debian' directory from -the Debian packaging Git repository (https://github.com/Icinga/deb-icinga2) -into your source tree and run the following command: -``` -$ dpkg-buildpackage -uc -us -``` - -## Build Alpine Linux packages - -A simple way to setup a build environment is installing Alpine in a chroot. -In this way, you can set up an Alpine build environment in a chroot under a -different Linux distro. -There is a script that simplifies these steps with just two commands, and -can be found [here](https://github.com/alpinelinux/alpine-chroot-install). - -Once the build environment is installed, you can setup the system to build -the packages by following [this document](https://wiki.alpinelinux.org/wiki/Creating_an_Alpine_package). - -# Build Post Install Tasks - -After building Icinga 2 yourself, your package build system should at least run the following post -install requirements: - -* enable the `checker`, `notification` and `mainlog` feature by default -* run 'icinga2 api setup' in order to enable the `api` feature and generate SSL certificates for the node - -## Run Icinga 2 - -Icinga 2 comes with a binary that takes care of loading all the relevant -components (e.g. for check execution, notifications, etc.): -``` -# icinga2 daemon -[2016-12-08 16:44:24 +0100] information/cli: Icinga application loader (version: v2.5.4-231-gb10a6b7; debug) -[2016-12-08 16:44:24 +0100] information/cli: Loading configuration file(s). -[2016-12-08 16:44:25 +0100] information/ConfigItem: Committing config item(s). -... -``` - -### Init Script - -Icinga 2 can be started as a daemon using the provided init script: -``` -# /etc/init.d/icinga2 -Usage: /etc/init.d/icinga2 {start|stop|restart|reload|checkconfig|status} -``` - -### Systemd - -If your distribution uses Systemd: -``` -# systemctl {start|stop|reload|status|enable|disable} icinga2 -``` - -In case the distribution is running Systemd >227, you'll also -need to package and install the `etc/initsystem/icinga2.service.limits.conf` -file into `/etc/systemd/system/icinga2.service.d`. - -### openrc - -Or if your distribution uses openrc (like Alpine): -``` -# rc-service icinga2 -Usage: /etc/init.d/icinga2 {start|stop|restart|reload|checkconfig|status} - ``` - -Note: the openrc's init.d is not shipped by default. -A working init.d with openrc can be found here: (https://git.alpinelinux.org/cgit/aports/plain/community/icinga2/icinga2.initd). If you have customized some path, edit the file and adjust it according with your setup. -Those few steps can be followed: -``` -# wget https://git.alpinelinux.org/cgit/aports/plain/community/icinga2/icinga2.initd -# mv icinga2.initd /etc/init.d/icinga2 -# chmod +x /etc/init.d/icinga2 -``` - -Icinga 2 reads a single configuration file which is used to specify all -configuration settings (global settings, hosts, services, etc.). The -configuration format is explained in detail in the [doc/](doc/) directory. - -By default `make install` installs example configuration files in -`/usr/local/etc/icinga2` unless you have specified a different prefix or -sysconfdir. From ec39a74453f673032cd52edb29eb747d80cb5fe9 Mon Sep 17 00:00:00 2001 From: Michael Friedrich Date: Mon, 3 Dec 2018 11:11:16 +0100 Subject: [PATCH 04/11] Docs: More updates on dev environments, add mini TOC --- doc/21-development.md | 297 +++++++++++++++++++++++++++++++++++------- 1 file changed, 250 insertions(+), 47 deletions(-) diff --git a/doc/21-development.md b/doc/21-development.md index 1ff183434..ff672fe1e 100644 --- a/doc/21-development.md +++ b/doc/21-development.md @@ -1,7 +1,17 @@ # Development -This chapter provides hints on Icinga 2 development -and debugging. +This chapter provides hints on Icinga 2 debugging, +development, package builds and tests. + +* [Debug Icinga 2](#development-debug) +* [Develop Icinga 2](#development-develop) + * [Linux Dev Environment](#development-linux-dev-env) + * [macOS Dev Environment](#development-macos-dev-env) + * [Windows Dev Environment](#development-windows-dev-env) +* [Package Builds](#development-package-builds) +* [Advanced Tips](#development-advanced) +* [Tests](#development-tests) + ## Debug Icinga 2 @@ -348,6 +358,13 @@ In order to run Icinga 2 with LLDB you need to pass the binary as argument. lldb -- /usr/local/icinga2/lib/icinga2/sbin/icinga2 daemon ``` +Breakpoint: + +``` +> b checkable.cpp:57 +> b icinga::Checkable::ProcessCheckResult +``` + Full backtrace: ``` @@ -494,6 +511,8 @@ db\_ido\_pgsql | IDO database driver for PgSQL. mysql\_shin | Library stub for linking against the MySQL client libraries. pgsql\_shim | Library stub for linking against the PgSQL client libraries. +#### Class Compiler + Another thing you will recognize are the `.ti` files which are compiled by our own class compiler into actual source code. The meta language allows developers to easily add object attributes and specify their behaviour. @@ -570,7 +589,7 @@ stuff like `epoll` is not available on Unix kernels. Depending on your workstation and environment, you may either develop and run locally, use a container deployment pipeline or put everything in a high end resource remote VM. -#### Prepare Linux Dev Environment +Fork https://github.com/Icinga/icinga2 into your own repository, e.g. `https://github.com/dnsmichi/icinga2`. Create two build directories for different binary builds. @@ -581,25 +600,55 @@ Create two build directories for different binary builds. mkdir -p release debug ``` -Next, check the required packages to build Icinga 2, and install them. +Proceed with the specific distribution examples below. -CentOS: +* [CentOS 7](#development-linux-dev-env-centos) +* [Debian 9](#development-linux-dev-env-debian) + +#### CentOS 7 + +``` +yum -y install gdb git bash-completion htop rpmdevtools \ + ccache cmake make gcc-c++ flex bison \ + openssl-devel boost-devel systemd-devel mysql-devel \ + postgresql-devel libedit-devel libstdc++-devel + +groupadd icinga +groupadd icingacmd +useradd -c "icinga" -s /sbin/nologin -G icingacmd -g icinga icinga + +ln -s /bin/ccache /usr/local/bin/gcc +ln -s /bin/ccache /usr/local/bin/g++ + +git clone https://github.com/icinga/icinga2.git && cd icinga2 + +mkdir debug release +cd debug +cmake .. -DCMAKE_BUILD_TYPE=Debug -DICINGA2_UNITY_BUILD=OFF -DCMAKE_INSTALL_PREFIX=/usr/local/icinga2 -DICINGA2_PLUGINDIR=/usr/local/sbin +cd .. +make -j2 install -C debug +``` + + +``` +chown -R icinga:icinga /usr/local/icinga2/var/ + +/usr/local/icinga2/lib/icinga2/prepare-dirs /usr/local/icinga2/etc/sysconfig/icinga2 +/usr/local/icinga2/sbin/icinga2 api setup +vim /usr/local/icinga2/etc/icinga2/conf.d/api-users.conf + +gdb --args /usr/local/icinga2/lib/icinga2/sbin/icinga2 daemon ``` ``` -Debian/Ubuntu: - -``` - -``` - -##### Debian Stretch +##### Debian 9 ``` apt-get -y install gdb vim git cmake make ccache build-essential libssl-dev libboost-all-dev bison flex default-libmysqlclient-dev libpq-dev libyajl-dev libedit-dev monitoring-plugins +ln -s /usr/bin/ccache /usr/local/bin/gcc ln -s /usr/bin/ccache /usr/local/bin/g++ groupadd icinga @@ -619,31 +668,154 @@ make -j2 install -C debug ``` chown -R icinga:icinga /usr/local/icinga2/var/ -cd /usr/local/icinga2 -./lib/icinga2/prepare-dirs etc/sysconfig/icinga2 -icinga2 api setup -vim etc/icinga2/conf.d/api-users.conf +/usr/local/icinga2/lib/icinga2/prepare-dirs /usr/local/icinga2/etc/sysconfig/icinga2 +/usr/local/icinga2/sbin/icinga2 api setup +vim /usr/local/icinga2/etc/icinga2/conf.d/api-users.conf -gdb --args ./lib/icinga2/sbin/icinga2 daemon +gdb --args /usr/local/icinga2/lib/icinga2/sbin/icinga2 daemon ``` ### macOS Dev Environment -It is advised to use Homebrew to install all required build dependencies. +It is advised to use Homebrew to install required build dependencies. Macports have been reported to work as well, typically you'll get more help -with Homebrew. +with Homebrew from Icinga developers. -We will also build two different flavors. +#### Users and Groups + +First off, create the following from `Settings - Users & Groups`: + +* Users: `icinga` +* Groups: `icinga` with `icinga` as member +* Groups: `icingaweb2` + +Then disallow login for these users. + +``` +dscl +list Local/Default/Users +read Local/Default/Users/icinga +change Local/Default/Users/icinga UserShell /bin/bash /usr/bin/false +sudo dscl . create /Users/icinga IsHidden 1 +sudo dseditgroup -o edit -a _www -t user icingaweb2 +``` + +#### Requirements + +OpenSSL 1.0.x doesn't build anymore, so we're explicitly using 1.1.x here. + +``` +brew install ccache boost cmake bison flex yajl openssl@1.1 mysql-connector-c++ postgresql libpq +``` + +##### ccache + +``` +sudo mkdir /opt/ccache + +sudo ln -s `which ccache` /opt/ccache/clang +sudo ln -s `which ccache` /opt/ccache/clang++ + +vim $HOME/.bashrc + +# ccache is managed with symlinks to avoid collision with cgo +export PATH="/opt/ccache:$PATH" + +source $HOME/.bashrc +``` + +#### Builds + +We will build two different flavors on macOS. ``` mkdir -p release debug + +cd debug +cmake -DICINGA2_UNITY_BUILD=OFF -DICINGA2_WITH_STUDIO=ON -DCMAKE_INSTALL_PREFIX=/usr/local/icinga2 -DOPENSSL_INCLUDE_DIR=/usr/local/opt/openssl@1.1/include -DOPENSSL_SSL_LIBRARY=/usr/local/opt/openssl@1.1/lib/libssl.dylib -DOPENSSL_CRYPTO_LIBRARY=/usr/local/opt/openssl@1.1/lib/libcrypto.dylib .. +cd .. + +make -j4 -C debug +sudo make -j4 install -C debug +``` + +##### Build Aliases + +This is derived from dnsmichi's flavour and not generally best practice. + +``` +vim $HOME/.bashrc + +export PATH=/usr/local/icinga2/sbin/:$PATH +source /usr/local/icinga2/etc/bash_completion.d/icinga2 + +export I2_GENERIC="-DCMAKE_INSTALL_PREFIX=/usr/local/icinga2 -DOPENSSL_INCLUDE_DIR=/usr/local/opt/openssl@1.1/include -DOPENSSL_SSL_LIBRARY=/usr/local/opt/openssl@1.1/lib/libssl.dylib -DOPENSSL_CRYPTO_LIBRARY=/usr/local/opt/openssl@1.1/lib/libcrypto.dylib -DICINGA2_PLUGINDIR=/usr/local/sbin" + +export I2_DEBUG="-DCMAKE_BUILD_TYPE=Debug -DICINGA2_UNITY_BUILD=OFF $I2_GENERIC" +export I2_RELEASE="-DCMAKE_BUILD_TYPE=RelWithDebInfo -DICINGA2_WITH_TESTS=ON -DICINGA2_UNITY_BUILD=ON $I2_GENERIC" + +alias i2_debug="mkdir -p debug; cd debug; cmake $I2_DEBUG ..; make -j4; sudo make -j4 install; cd .." +alias i2_release="mkdir -p release; cd release; cmake $I2_RELEASE ..; make -j4; sudo make -j4 install; cd .." + +source $HOME/.bashrc +``` + +#### Run + +``` +chown -R icinga:icinga /usr/local/icinga2 +chown -R icinga:_www /usr/local/icinga2/var/run/icinga2/cmd + +icinga2 daemon +``` + +#### Plugins + +``` +brew install nagios-plugins + +sudo vim /usr/local/icinga2/etc/icinga2/constants.conf +const PluginDir = "/usr/local/sbin" +``` + +#### Databases: MariaDB + +``` +brew install mariadb +ln -sfv /usr/local/opt/mariadb/*.plist ~/Library/LaunchAgents +launchctl load ~/Library/LaunchAgents/homebrew.mxcl.mariadb.plist +mysql_secure_installation +``` + +``` +vim $HOME/.my.cnf + +[client] +user = root +password = supersecurerootpassword + +sudo -i +ln -s /Users/michi/.my.cnf $HOME/.my.cnf +exit +``` + +``` +cd $HOME/coding/icinga/icinga2 + +sudo mysql + +CREATE DATABASE icinga; +GRANT SELECT, INSERT, UPDATE, DELETE, DROP, CREATE VIEW, INDEX, EXECUTE ON icinga.* TO 'icinga'@'localhost' IDENTIFIED BY 'icinga'; +quit +sudo mysql icinga < lib/db_ido_mysql/schema/mysql.sql ``` -### Windows Dev Environment + +### Windows Dev Environment The following sections explain how to setup the required build tools and how to run and debug the code. @@ -791,7 +963,7 @@ Run CMake with the following command. This generates a new Visual Studio project You need to specify the previously installed component paths: Variable | Value | Description -----------------------|----------------------------------------------------------------------| +----------------------|----------------------------------------------------------------------|------------------------------------------------------- BOOST_ROOT | `C:\boost_1_65_1` | Root path where you've extracted and compiled Boost. BISON_EXECUTABLE | `C:\ProgramData\chocolatey\lib\winflexbison\tools\win_bison.exe` | Path to the Bison executable. FLEX_EXECUTABLE | `C:\ProgramData\chocolatey\lib\winflexbison\tools\win_flex.exe` | Path to the Flex executable. @@ -875,7 +1047,9 @@ Again, put these lines into a batch/Powershell script and execute that. > > This isn't officially supported yet, just a few hints how you can do it yourself. -armhf on Raspberry Pi, ccache: +The following examples source from armhf on Raspberry Pi. + +#### ccache ``` apt install -y ccache @@ -887,6 +1061,20 @@ echo 'export PATH="/usr/lib/ccache:$PATH"' | tee -a ~/.bashrc source ~/.bashrc && echo $PATH ``` +#### Build + +Copy the icinga2 source code into `$HOME/icinga2`. Clone the `deb-icinga2` repository into `debian/`. + +``` +git clone https://github.com/Icinga/icinga2 $HOME/icinga2 +git clone https://github.com/Icinga/deb-icinga2 $HOME/icinga2/debian +``` + +Then build a Debian package and install it like normal. +``` +dpkg-buildpackage -uc -us +``` + ## Package Builds This documentation is explicitly meant for packagers and the Icinga @@ -919,7 +1107,7 @@ Icinga application using a dist tarball (including notes for distributions): - Debian/Ubuntu: libsystemd-dev - RHEL/Fedora: systemd-devel -### Optional features +### Optional features * MySQL (disable with CMake variable `ICINGA2_WITH_MYSQL` to `OFF`) - RHEL/Fedora: mysql-devel @@ -941,14 +1129,14 @@ Icinga application using a dist tarball (including notes for distributions): - RHEL/Fedora: libtermcap-devel - Debian/Ubuntu: (not necessary) -### Special requirements +### Special requirements **FreeBSD**: libexecinfo (automatically used when Icinga 2 is installed via port or package) **RHEL6**: Requires a newer boost version which is available on packages.icinga.com with a version suffixed name. -### Runtime user environment +### Runtime user environment By default Icinga will run as user `icinga` and group `icinga`. Additionally the external command pipe and livestatus features require a dedicated command group @@ -972,6 +1160,7 @@ On Alpine (which uses ash busybox) you can run: Add the web server user to the icingacmd group in order to grant it write permissions to the external command pipe and livestatus socket: + ``` # usermod -a -G icingacmd www-data ``` @@ -979,7 +1168,7 @@ permissions to the external command pipe and livestatus socket: Make sure to replace "www-data" with the name of the user your web server is running as. -### Building Icinga 2: Example +### Building Icinga 2: Example Once you have installed all the necessary build requirements you can build Icinga 2 using the following commands: @@ -998,7 +1187,7 @@ You can specify an alternative installation prefix using `-DCMAKE_INSTALL_PREFIX $ cmake .. -DCMAKE_INSTALL_PREFIX=/tmp/icinga2 ``` -### CMake Variables +### CMake Variables In addition to `CMAKE_INSTALL_PREFIX` here are most of the supported Icinga-specific cmake variables. @@ -1076,11 +1265,12 @@ directory. Alternatively the `-DICINGA2_GIT_VERSION_INFO=OFF` option for CMake can be used to disable the usage of `git describe`. -### Building RPMs +### Building RPMs #### Build Environment on RHEL, CentOS, Fedora, Amazon Linux Setup your build environment: + ``` yum -y install rpmdevtools ``` @@ -1088,28 +1278,32 @@ yum -y install rpmdevtools #### Build Environment on SuSE/SLES SLES: + ``` -zypper addrepo http://download.opensuse.org/repositories/devel:tools/SLE_12_SP3/devel:tools.repo +zypper addrepo http://download.opensuse.org/repositories/devel:tools/SLE_12_SP4/devel:tools.repo zypper refresh zypper install rpmdevtools spectool ``` OpenSuSE: + ``` zypper addrepo http://download.opensuse.org/repositories/devel:tools/openSUSE_Leap_15.0/devel:tools.repo zypper refresh zypper install rpmdevtools spectool ``` -#### Package Builds +#### Package Builds Prepare the rpmbuild directory tree: + ``` cd $HOME rpmdev-setuptree ``` -Copy the icinga2.spec file to `rpmbuild/SPEC` or fetch the latest version: +Snapshot builds: + ``` curl https://raw.githubusercontent.com/Icinga/rpm-icinga2/master/icinga2.spec -o $HOME/rpmbuild/SPECS/icinga2.spec ``` @@ -1121,6 +1315,7 @@ curl https://raw.githubusercontent.com/Icinga/rpm-icinga2/master/icinga2.spec -o Copy the tarball to `rpmbuild/SOURCES` e.g. by using the `spectool` binary provided with `rpmdevtools`: + ``` cd $HOME/rpmbuild/SOURCES spectool -g ../SPECS/icinga2.spec @@ -1129,6 +1324,7 @@ cd $HOME/rpmbuild ``` Install the build dependencies. Example for CentOS 7: + ``` yum -y install libedit-devel ncurses-devel gcc-c++ libstdc++-devel openssl-devel \ cmake flex bison boost-devel systemd mysql-devel postgresql-devel httpd \ @@ -1138,16 +1334,18 @@ selinux-policy-devel checkpolicy selinux-policy selinux-policy-doc Note: If you are using Amazon Linux, systemd is not required. A shorter way is available using the `yum-builddep` command on RHEL based systems: + ``` yum-builddep SPECS/icinga2.spec ``` Build the RPM: + ``` rpmbuild -ba SPECS/icinga2.spec ``` -#### Additional Hints +#### Additional Hints ##### SELinux policy module @@ -1182,28 +1380,27 @@ cat >$HOME/.rpmmacros < **WARNING:** This information is outdated! +### Build Debian/Ubuntu packages Setup your build environment on Debian/Ubuntu, copy the 'debian' directory from the Debian packaging Git repository (https://github.com/Icinga/deb-icinga2) into your source tree and run the following command: + ``` $ dpkg-buildpackage -uc -us ``` -## Build Alpine Linux packages +### Build Alpine Linux packages A simple way to setup a build environment is installing Alpine in a chroot. In this way, you can set up an Alpine build environment in a chroot under a @@ -1214,7 +1411,7 @@ can be found [here](https://github.com/alpinelinux/alpine-chroot-install). Once the build environment is installed, you can setup the system to build the packages by following [this document](https://wiki.alpinelinux.org/wiki/Creating_an_Alpine_package). -# Build Post Install Tasks +### Build Post Install Tasks After building Icinga 2 yourself, your package build system should at least run the following post install requirements: @@ -1222,29 +1419,33 @@ install requirements: * enable the `checker`, `notification` and `mainlog` feature by default * run 'icinga2 api setup' in order to enable the `api` feature and generate SSL certificates for the node -## Run Icinga 2 +### Run Icinga 2 Icinga 2 comes with a binary that takes care of loading all the relevant components (e.g. for check execution, notifications, etc.): + ``` -# icinga2 daemon +icinga2 daemon + [2016-12-08 16:44:24 +0100] information/cli: Icinga application loader (version: v2.5.4-231-gb10a6b7; debug) [2016-12-08 16:44:24 +0100] information/cli: Loading configuration file(s). [2016-12-08 16:44:25 +0100] information/ConfigItem: Committing config item(s). ... ``` -### Init Script +#### Init Script Icinga 2 can be started as a daemon using the provided init script: + ``` # /etc/init.d/icinga2 Usage: /etc/init.d/icinga2 {start|stop|restart|reload|checkconfig|status} ``` -### Systemd +### Systemd If your distribution uses Systemd: + ``` # systemctl {start|stop|reload|status|enable|disable} icinga2 ``` @@ -1253,9 +1454,10 @@ In case the distribution is running Systemd >227, you'll also need to package and install the `etc/initsystem/icinga2.service.limits.conf` file into `/etc/systemd/system/icinga2.service.d`. -### openrc +### openrc Or if your distribution uses openrc (like Alpine): + ``` # rc-service icinga2 Usage: /etc/init.d/icinga2 {start|stop|restart|reload|checkconfig|status} @@ -1264,6 +1466,7 @@ Usage: /etc/init.d/icinga2 {start|stop|restart|reload|checkconfig|status} Note: the openrc's init.d is not shipped by default. A working init.d with openrc can be found here: (https://git.alpinelinux.org/cgit/aports/plain/community/icinga2/icinga2.initd). If you have customized some path, edit the file and adjust it according with your setup. Those few steps can be followed: + ``` # wget https://git.alpinelinux.org/cgit/aports/plain/community/icinga2/icinga2.initd # mv icinga2.initd /etc/init.d/icinga2 @@ -1279,11 +1482,11 @@ By default `make install` installs example configuration files in sysconfdir. -### Windows +### Windows Builds The Windows MSI packages are located at https://packages.icinga.com/windows/ -#### Requirements +#### Requirements * 32 or 64-bit system * Visual Studio >= 14 2015 From ac9ed41537b36d7a41e26e8535e208579cccbfc2 Mon Sep 17 00:00:00 2001 From: Michael Friedrich Date: Mon, 3 Dec 2018 11:15:04 +0100 Subject: [PATCH 05/11] Docs: Update anchors --- doc/21-development.md | 28 ++++++++++++++-------------- 1 file changed, 14 insertions(+), 14 deletions(-) diff --git a/doc/21-development.md b/doc/21-development.md index ff672fe1e..6bd13e939 100644 --- a/doc/21-development.md +++ b/doc/21-development.md @@ -3,14 +3,14 @@ This chapter provides hints on Icinga 2 debugging, development, package builds and tests. -* [Debug Icinga 2](#development-debug) -* [Develop Icinga 2](#development-develop) - * [Linux Dev Environment](#development-linux-dev-env) - * [macOS Dev Environment](#development-macos-dev-env) - * [Windows Dev Environment](#development-windows-dev-env) -* [Package Builds](#development-package-builds) -* [Advanced Tips](#development-advanced) -* [Tests](#development-tests) +* [Debug Icinga 2](21-development.md#development-debug) +* [Develop Icinga 2](21-development.md#development-develop) + * [Linux Dev Environment](21-development.md#development-linux-dev-env) + * [macOS Dev Environment](21-development.md#development-macos-dev-env) + * [Windows Dev Environment](21-development.md#development-windows-dev-env) +* [Package Builds](21-development.md#development-package-builds) +* [Advanced Tips](21-development.md#development-advanced) +* [Tests](21-development.md#development-tests) ## Debug Icinga 2 @@ -460,7 +460,7 @@ autocmd BufWinLeave * call clearmatches() Icinga 2 can run standalone or in distributed environments. It contains a whole lot more than a simple check execution engine. -Read more about it in the [Technical Concepts](#technical-concepts) chapter. +Read more about it in the [Technical Concepts](19-technical-concepts.md#technical-concepts) chapter. ### Get to know the code @@ -511,7 +511,7 @@ db\_ido\_pgsql | IDO database driver for PgSQL. mysql\_shin | Library stub for linking against the MySQL client libraries. pgsql\_shim | Library stub for linking against the PgSQL client libraries. -#### Class Compiler +#### Class Compiler Another thing you will recognize are the `.ti` files which are compiled by our own class compiler into actual source code. The meta language allows @@ -542,7 +542,7 @@ The logic is hidden in `tools/mkclass/` in case you want to learn more about it. The first steps during CMake & make also tell you about code generation. -### Build Tool: CMake +### Builds: CMake In its early development stages in 2012, Icinga 2 was built with autoconf/automake and separate Windows project files. We've found this very fragile, and have changed @@ -556,7 +556,7 @@ The most common benefits: * Separate binary build directories, the actual source tree stays clean. * CMake automatically generates a Visual Studio project file `icinga2.sln` on Windows. -### Builds: Unity Builds +### Builds: Unity Builds Another thing you should be aware of: Unity builds on and off. @@ -602,8 +602,8 @@ mkdir -p release debug Proceed with the specific distribution examples below. -* [CentOS 7](#development-linux-dev-env-centos) -* [Debian 9](#development-linux-dev-env-debian) +* [CentOS 7](21-development.md#development-linux-dev-env-centos) +* [Debian 9](21-development.md#development-linux-dev-env-debian) #### CentOS 7 From e0b3e05c636db082da264c92fc02593f7ff44ded Mon Sep 17 00:00:00 2001 From: Michael Friedrich Date: Mon, 3 Dec 2018 11:28:39 +0100 Subject: [PATCH 06/11] Docs: Update README.md with a better description and direct URLs to icinga.com/docs --- README.md | 52 ++++++++++++++++++++++++++-------------------------- 1 file changed, 26 insertions(+), 26 deletions(-) diff --git a/README.md b/README.md index 1935d12d4..b0bcd681e 100644 --- a/README.md +++ b/README.md @@ -8,27 +8,42 @@ #### Table of Contents 1. [About][About] -2. [License][License] -3. [Installation][Installation] -4. [Documentation][Documentation] -5. [Support][Support] +2. [Installation][Installation] +3. [Documentation][Documentation] +4. [Support][Support] +5. [License][License] 6. [Contributing][Contributing] ## About -Icinga 2 is an open source monitoring system which checks the availability of your -network resources, notifies users of outages, and generates performance data for reporting. +[Icinga 2](https://icinga.com/products/icinga-2/) is a monitoring system which checks +the availability of your network resources, notifies users of outages, and generates +performance data for reporting. Scalable and extensible, Icinga 2 can monitor large, complex environments across multiple locations. -Icinga 2 as monitoring core works best with [Icinga Web 2](https://icinga.com/products/icinga-web-2/) -as web interface. +Icinga 2 as core requires [Icinga Web 2](https://icinga.com/products/icinga-web-2/) +on top in your Icinga Stack. -More information can be found at [icinga.com](https://icinga.com/products/icinga-2/) -and inside the [documentation](https://icinga.com/docs/icinga2/latest/). +![Icinga Dashboard](https://icinga.com/wp-content/uploads/2017/12/icingaweb2-2.5.0-dashboard.png) -## Licensing +## Installation + +For installing Icinga please check the [installation chapter](https://icinga.com/docs/icinga2/latest/doc/02-getting-started/) +in the documentation. + +## Documentation + +The documentation is available on [icinga.com/docs](https://icinga.com/docs/icinga2/latest/). + +## Support + +Check the [project website](https://icinga.com) for status updates. Join the +[community channels](https://icinga.com/community/) for questions +or ask an Icinga partner for [professional support](https://icinga.com/support/). + +## License Icinga 2 and the Icinga 2 documentation are licensed under the terms of the GNU General Public License Version 2, you will find a copy of this license in the @@ -48,21 +63,6 @@ so, delete this exception statement from your version. If you delete this exception statement from all source files in the program, then also delete it here. -## Installation - -Read the [INSTALL.md](INSTALL.md) file for more information about how to install it. - -## Documentation - -The documentation is located in the [doc/](doc/) directory. The latest documentation -is also available on https://docs.icinga.com - -## Support - -Check the project website at https://icinga.com for status updates. Join the -[community channels](https://icinga.com/community) for questions -or ask an Icinga partner for [professional support](https://icinga.com/services/support/). - ## Contributing There are many ways to contribute to Icinga -- whether it be sending patches, From 8cbe286e676a56c536cee517b898cbe2f25b2547 Mon Sep 17 00:00:00 2001 From: Michael Friedrich Date: Tue, 4 Dec 2018 10:51:04 +0100 Subject: [PATCH 07/11] Fix formatting in dev docs --- doc/21-development.md | 8 +++----- 1 file changed, 3 insertions(+), 5 deletions(-) diff --git a/doc/21-development.md b/doc/21-development.md index 6bd13e939..b6ae7eca4 100644 --- a/doc/21-development.md +++ b/doc/21-development.md @@ -407,15 +407,15 @@ Up/down in stacktrace: ## Develop Icinga 2 Icinga 2 can be built on many platforms such as Linux, Unix and Windows. -There are limitations in terms of support, e.g. Windows is only supported for clients, +There are limitations in terms of support, e.g. Windows is only supported for agents, not a full-featured master or satellite. Before you start with actual development, there is a couple of pre-requisites. ### Choose your Editor -Icinga 2 can be developed with your favorite editor. Icinga developers prefer to use -the tools below: +Icinga 2 can be developed with your favorite editor. Icinga developers prefer +these tools: - vim - CLion (macOS, Linux) @@ -641,8 +641,6 @@ vim /usr/local/icinga2/etc/icinga2/conf.d/api-users.conf gdb --args /usr/local/icinga2/lib/icinga2/sbin/icinga2 daemon ``` -``` - ##### Debian 9 ``` From 9801b8b7a040fd4c91aa05d596ec644ddd294bdc Mon Sep 17 00:00:00 2001 From: Michael Friedrich Date: Tue, 4 Dec 2018 10:54:39 +0100 Subject: [PATCH 08/11] Purge INSTALL.md and add packager info to README.md Everything is located in the online documentation, where the README.md points to. --- INSTALL.md | 21 --------------------- README.md | 3 +++ 2 files changed, 3 insertions(+), 21 deletions(-) delete mode 100644 INSTALL.md diff --git a/INSTALL.md b/INSTALL.md deleted file mode 100644 index d6121e999..000000000 --- a/INSTALL.md +++ /dev/null @@ -1,21 +0,0 @@ -# Installing Icinga 2 - -The recommended way of installing Icinga 2 is to use [packages](https://packages.icinga.com). -The Icinga project provides both release and development packages for a number -of operating systems. - -Please check the documentation in the [doc/](doc/) directory for a current list -of available packages and detailed installation instructions. - -The online documentation is available on [icinga.com/docs](https://icinga.com/docs/) -and will guide you step by step. - -There are a number of known caveats when installing from source such as -incorrect directory and file permissions. So even if you're planning to -not use the official packages it is advisable to build your own Debian -or RPM packages. You can use the source packages from [packages.icinga.com](https://packages.icinga.com) -for this purpose. - -For a more detailed introduction into developing Icinga and building -packages, please visit the [online documentation](https://icinga.com/docs/icinga2/latest/doc/21-development/). - diff --git a/README.md b/README.md index b0bcd681e..f81f6c940 100644 --- a/README.md +++ b/README.md @@ -33,6 +33,9 @@ on top in your Icinga Stack. For installing Icinga please check the [installation chapter](https://icinga.com/docs/icinga2/latest/doc/02-getting-started/) in the documentation. +If you are a packager, please read the [development chapter](https://icinga.com/docs/icinga2/latest/doc/21-development/) +for more details. + ## Documentation The documentation is available on [icinga.com/docs](https://icinga.com/docs/icinga2/latest/). From 01d2a17683659e17b0cc311135937e0d5748a05d Mon Sep 17 00:00:00 2001 From: Michael Friedrich Date: Tue, 4 Dec 2018 10:58:44 +0100 Subject: [PATCH 09/11] Update TOC --- doc/21-development.md | 25 +++++++++++++++++++++---- 1 file changed, 21 insertions(+), 4 deletions(-) diff --git a/doc/21-development.md b/doc/21-development.md index b6ae7eca4..1b43724fe 100644 --- a/doc/21-development.md +++ b/doc/21-development.md @@ -4,11 +4,16 @@ This chapter provides hints on Icinga 2 debugging, development, package builds and tests. * [Debug Icinga 2](21-development.md#development-debug) + * [GDB Backtrace](21-development.md#development-debug-gdb-backtrace) + * [Core Dump](21-development.md#development-debug-core-dump) * [Develop Icinga 2](21-development.md#development-develop) - * [Linux Dev Environment](21-development.md#development-linux-dev-env) - * [macOS Dev Environment](21-development.md#development-macos-dev-env) - * [Windows Dev Environment](21-development.md#development-windows-dev-env) + * [Linux Dev Environment](21-development.md#development-linux-dev-env) + * [macOS Dev Environment](21-development.md#development-macos-dev-env) + * [Windows Dev Environment](21-development.md#development-windows-dev-env) * [Package Builds](21-development.md#development-package-builds) + * [RPM](21-development.md#development-package-builds-rpms) + * [DEB](21-development.md#development-package-builds-deb) + * [Windows](21-development.md#development-package-builds-windows) * [Advanced Tips](21-development.md#development-advanced) * [Tests](21-development.md#development-tests) @@ -20,6 +25,7 @@ a stack trace or coredump if the application crashed. It is also useful for developers working with different debuggers. > **Note:** +> > This is intentionally mentioned before any development insights > as debugging is a more frequent and commonly asked question. @@ -131,6 +137,12 @@ a new gdb run. #### GDB Backtrace If Icinga 2 aborted its operation abnormally, generate a backtrace. + +> **Note** +> +> Please install the [required debug symbols](21-development.md#debug-requirements) +> prior to generating a backtrace. + `thread apply all` is important here since this includes all running threads. We need this information when e.g. debugging dead locks and hanging features. @@ -155,6 +167,11 @@ make sure to attach as much detail as possible. If Icinga 2 is still running, generate a full backtrace from the running process and store it into a new file (e.g. for debugging dead locks). +> **Note** +> +> Please install the [required debug symbols](21-development.md#debug-requirements) +> prior to generating a backtrace. + Icinga 2 runs with 2 processes: main and command executor, therefore generate two backtrace logs and add them to the GitHub issue. @@ -1395,7 +1412,7 @@ the Debian packaging Git repository (https://github.com/Icinga/deb-icinga2) into your source tree and run the following command: ``` -$ dpkg-buildpackage -uc -us +dpkg-buildpackage -uc -us ``` ### Build Alpine Linux packages From 43c171056f826cf08b4155472ad9a6a4d14ba8a9 Mon Sep 17 00:00:00 2001 From: Michael Friedrich Date: Tue, 4 Dec 2018 12:14:01 +0100 Subject: [PATCH 10/11] Minor updates for core dumps The main parts are discussed in #6723 --- doc/21-development.md | 14 +++++++++----- 1 file changed, 9 insertions(+), 5 deletions(-) diff --git a/doc/21-development.md b/doc/21-development.md index 1b43724fe..1d08add4e 100644 --- a/doc/21-development.md +++ b/doc/21-development.md @@ -279,10 +279,11 @@ developers to analyze and fix the problem. This requires setting the core dump file size to `unlimited`. -Example for Systemd: + +##### Systemd ``` -vim /usr/lib/systemd/system/icinga2.service +systemctl edit icinga2.service [Service] ... @@ -293,7 +294,7 @@ systemctl daemon-reload systemctl restart icinga2 ``` -Example for init script: +##### Init Script ``` vim /etc/init.d/icinga2 @@ -303,10 +304,13 @@ ulimit -c unlimited service icinga2 restart ``` +##### Verify + Verify that the Icinga 2 process core file size limit is set to `unlimited`. ``` -cat /proc/`pidof icinga2`/limits +for pid in $(pidof icinga2); do cat /proc/$pid/limits; done + ... Max core file size unlimited unlimited bytes ``` @@ -318,7 +322,7 @@ The Icinga 2 daemon runs with the SUID bit set. Therefore you need to explicitly enable core dumps for SUID on Linux. ``` -sysctl -w fs.suid_dumpable=1 +sysctl -w fs.suid_dumpable=2 ``` Adjust the coredump kernel format and file location on Linux: From 00673cd75911e56cf977fdb5c14c809a97c01e6d Mon Sep 17 00:00:00 2001 From: Michael Friedrich Date: Tue, 4 Dec 2018 12:21:54 +0100 Subject: [PATCH 11/11] Docs: Formatting for dev requirements --- doc/21-development.md | 64 ++++++++++++------------------------------- 1 file changed, 18 insertions(+), 46 deletions(-) diff --git a/doc/21-development.md b/doc/21-development.md index 1d08add4e..9abf3df4c 100644 --- a/doc/21-development.md +++ b/doc/21-development.md @@ -36,29 +36,12 @@ The Icinga 2 packages provide a debug package which must be installed separately for all involved binaries, like `icinga2-bin` or `icinga2-ido-mysql`. -Debian/Ubuntu: - -``` -apt-get install icinga2-dbg -``` - -RHEL/CentOS: - -``` -yum install icinga2-debuginfo -``` - -Fedora: - -``` -dnf install icinga2-debuginfo icinga2-bin-debuginfo icinga2-ido-mysql-debuginfo -``` - -SLES/openSUSE: - -``` -zypper install icinga2-bin-debuginfo icinga2-ido-mysql-debuginfo -``` +Distribution | Command +-------------------|------------------------------------------ +Debian/Ubuntu | `apt-get install icinga2-dbg` +RHEL/CentOS | `yum install icinga2-debuginfo` +Fedora | `dnf install icinga2-debuginfo icinga2-bin-debuginfo icinga2-ido-mysql-debuginfo` +SLES/openSUSE | `zypper install icinga2-bin-debuginfo icinga2-ido-mysql-debuginfo` Furthermore, you may also have to install debug symbols for Boost and your C++ library. @@ -70,23 +53,12 @@ build flag for debug builds. Install GDB in your development environment. -Debian/Ubuntu: - -``` -apt-get install gdb -``` - -RHEL/CentOS/Fedora: - -``` -yum install gdb -``` - -SLES/openSUSE: - -``` -zypper install gdb -``` +Distribution | Command +-------------------|------------------------------------------ +Debian/Ubuntu | `apt-get install gdb` +RHEL/CentOS | `yum install gdb` +Fedora | `dnf install gdb` +SLES/openSUSE | `zypper install gdb` #### GDB Run @@ -983,12 +955,12 @@ You need to specify the previously installed component paths: Variable | Value | Description ----------------------|----------------------------------------------------------------------|------------------------------------------------------- -BOOST_ROOT | `C:\boost_1_65_1` | Root path where you've extracted and compiled Boost. -BISON_EXECUTABLE | `C:\ProgramData\chocolatey\lib\winflexbison\tools\win_bison.exe` | Path to the Bison executable. -FLEX_EXECUTABLE | `C:\ProgramData\chocolatey\lib\winflexbison\tools\win_flex.exe` | Path to the Flex executable. -ICINGA2_WITH_MYSQL | OFF | Requires extra setup for MySQL if set to `ON`. Not supported for client setups. -ICINGA2_WITH_PGSQL | OFF | Requires extra setup for PgSQL if set to `ON`. Not supported for client setups. -ICINGA2_UNITY_BUILD | OFF | Disable unity builds for development environments. +`BOOST_ROOT` | `C:\boost_1_65_1` | Root path where you've extracted and compiled Boost. +`BISON_EXECUTABLE` | `C:\ProgramData\chocolatey\lib\winflexbison\tools\win_bison.exe` | Path to the Bison executable. +`FLEX_EXECUTABLE` | `C:\ProgramData\chocolatey\lib\winflexbison\tools\win_flex.exe` | Path to the Flex executable. +`ICINGA2_WITH_MYSQL` | OFF | Requires extra setup for MySQL if set to `ON`. Not supported for client setups. +`ICINGA2_WITH_PGSQL` | OFF | Requires extra setup for PgSQL if set to `ON`. Not supported for client setups. +`ICINGA2_UNITY_BUILD` | OFF | Disable unity builds for development environments. Tip: If you have previously opened a terminal, run `refreshenv` to re-read updated PATH variables.