Commit Graph

6140 Commits

Author SHA1 Message Date
Alexander A. Klimov a56ad38ad3 Separately handle apply rules targetting only specific parent objects
not to unnecessarily run e.g. the filter assign where host.name=="example.com"
for all hosts being not example.com.
2022-10-28 14:27:53 +02:00
Alexander A. Klimov fd7ac4e5ca Allow hashmaps of String 2022-10-21 10:28:41 +02:00
Alexander A. Klimov 449a3c14cf Allow intrusive pointers to ApplyRule 2022-10-21 10:28:41 +02:00
Julian Brost 987bb22397
Merge pull request #9543 from Icinga/apply-rules-lookup
Lookup apply rules faster by Type*, not String and by map instead of ==/!=
2022-10-21 09:53:35 +02:00
Alexander A. Klimov c7d656716f Remove unused ApplyRule#m_TargetType 2022-10-19 13:43:51 +02:00
Alexander A. Klimov d468d7993c Lookup apply rules faster by Type*, not String and by map instead of ==/!=
1. The lookup of apply rules per source type now implies
   no String(const char*) (no malloc()) and just pointer (uint64) comparisions
2. Apply rules are now also grouped by target type via a nested map, that obsoletes
   checking the target type while iterating over all rules per source type
2022-10-19 13:43:51 +02:00
Alexander A. Klimov 90fe4e5bea ApplyRule::GetTargetTypes(): return by const ref not to malloc() 2022-10-19 13:43:51 +02:00
Julian Brost f2563cc890
Merge pull request #9542 from Icinga/context-evaluating-apply-rules-for-host
Construct string once, not unnecessarily N times
2022-10-17 19:57:09 +02:00
Alexander A. Klimov ce1a122618 Construct string once, not unnecessarily N times 2022-10-17 15:54:02 +02:00
Yonas Habteab 400117e2f6 ConfigItem: Don't add items to the new items vector before committing
This also improves the performance a bit, as we longer have to iterate over the items
and copy them into the new items vector.
2022-10-12 13:27:41 +02:00
Yonas Habteab f7298e85d2 ConfigItem: Fix infinite recursion caused by `ignore_on_error` when committing an item
When committing an item with `ignore_on_error` flag set fails, the `Commit()` method only returns `nullptr`
and the current item is not being dropped from `m_Items`. `CommittNewItems()` also doesn't check the return
value of `Commit()` but just continues and tries to commit all items from `m_Items` in recursive call. Since
this corrupt item is never removed from `m_Items`, it ends up in an endless recursion till it finally crashes.
2022-10-12 13:15:09 +02:00
Julian Brost 91cbb856fe
Merge pull request #9521 from Icinga/noop-log-msgs
Logger: don't render log messages which will be disposed anyway
2022-10-11 19:05:03 +02:00
Alexander Aleksandrovič Klimov 363f4d3fde
Merge pull request #9408 from Icinga/bugfix/match-api-permissions-against-join-relations
ObjectQueryHandler: Check user permissions on joined relations
2022-10-11 13:42:27 +02:00
Yonas Habteab a656444d78 RedisConnection: Don't log queries that are going to be discarded 2022-10-11 13:28:08 +02:00
Alexander A. Klimov 0fbb0332a6 Logger: don't render log messages which will be disposed anyway
by caching the total minimum log severity of all loggers in a
"global variable" and whether a message's severity is large enough for any of
the loggers in a per-message no-op flag.
2022-10-11 13:28:08 +02:00
Julian Brost 9be02e3f04
Merge pull request #9518 from Icinga/9481
StartUnixWorker(): watch forked child via waitpid(), not SIGCHLD handler
2022-10-10 14:36:52 +02:00
Yonas Habteab 72e6894bbb Evaluate permission filters also on all joined relations 2022-10-10 12:33:33 +02:00
Yonas Habteab 607f7ab5ca ObjectQueryHandler: Check user permissions on joined relations 2022-10-10 12:33:33 +02:00
Yonas Habteab 1bb2d65a8d FilterUtility: Outsource permission matching from CheckPermission() to a separate method 2022-10-10 12:33:33 +02:00
Julian Brost 465da17060
Merge pull request #9407 from Icinga/bugfix/don-not-allow-changing-object-relations-at-runtime
Don't allow to change object navigation fields at runtime
2022-10-10 12:27:57 +02:00
Alexander A. Klimov 61f7e029cb Replace two-variants enum with bool 2022-10-07 15:14:33 +02:00
Julian Brost 87a4925997
Merge pull request #9519 from Icinga/utf8cp
Utility::ValidateUTF8(): move a string instead of copying a vector
2022-10-07 10:21:37 +02:00
Julian Brost 2a4dc083ae
Merge pull request #9524 from Icinga/introduce-object-idx
Introduce object identifier attr
2022-10-07 10:19:59 +02:00
Julian Brost 0ed9c09a1d
Merge pull request #9513 from Icinga/9501
Icinga DB: on every check result update state only 1x, not 3x in a row
2022-10-07 10:18:56 +02:00
Yonas Habteab 85c77bd878 IcingaDB: Cache generated object hash 2022-09-12 17:23:06 +02:00
Yonas Habteab 07e60c1961 ConfigObject: Introduce new `icingadb_identifier` attr 2022-09-12 17:22:57 +02:00
Yonas Habteab 28c29c1fbc Don't allow to change object parent,host/service_name at runtime 2022-09-09 18:26:28 +02:00
Alexander A. Klimov a6b36a2d7b Utility::ValidateUTF8(): move a string instead of copying a vector
less malloc() = more speed

Especially as JsonEncode() validates every single input string.
2022-09-09 10:50:42 +02:00
Alexander A. Klimov 22bfcf9ac5 icinga2 daemon: remove no-op SIGCHLD handling
1. Don't set a custom handler for SIGCHLD (in the umbrella process)
   as that handler doesn't actually handle SIGCHLD anymore
2. Don't reset the SIGCHLD handler (in the worker process)
   as there's nothing to reset anymore due to the above change
3. Don't block SIGCHLD across fork(2) as its handler doesn't change anymore
   due to the above changes
2022-09-07 12:12:09 +02:00
Alexander A. Klimov 3de714489c Remove unused UnixWorkerState::Failed 2022-09-07 12:08:33 +02:00
Alexander A. Klimov df9008bfc4 StartUnixWorker(): watch forked child via waitpid(), not SIGCHLD handler
Before:

On SIGCHLD from the forked worker the umbrella process sets a failure flag.
StartUnixWorker() recognises that and does waitpid(), failure message, etc..
On OpenBSD we can't tell the signal source, so we always set the failure flag.
That's not how our IPC shall work, that breaks the IPC sooner or later.

After:

No SIGCHLD handling and no failure flag setting.
Instead StartUnixWorker()'s wait loop uses waitpid(x,y,WNOHANG)
to avoid false positives while watching the forked worker.
2022-09-07 11:46:46 +02:00
Yonas Habteab 31785b48fd Expression: Decrease `frame.Depth` only when calling `IncreaseStackDepth()` succeeds
This ensures that `frame.Depth` is only decreased when preceding `frame.IncreaseStackDepth()` callee was successful.
This way, `frame.Depth` will have the same depth prior to and after evaluating a frame.
2022-09-07 09:41:16 +02:00
Alexander A. Klimov 5e9f95c007 Icinga DB: on every check result update state only 1x, not 3x in a row
Before (time: vertical, stack: horizontal):

* Checkable::ExecuteCheck
  * Checkable::UpdateNextCheck
    * IcingaDB::NextCheckChangedHandler
      * HSET icinga:host:state
      * HSET icinga:checksum:host:state
      * ZADD icinga:nextupdate:host
  * RandomCheckTask::ScriptFunc
    * Checkable::ProcessCheckResult
      * Checkable::UpdateNextCheck
        * IcingaDB::NextCheckChangedHandler
          * HSET icinga:host:state
          * HSET icinga:checksum:host:state
          * ZADD icinga:nextupdate:host
      * IcingaDB::NewCheckResultHandler
        * HSET icinga:host:state
        * HSET icinga:checksum:host:state
        * ZADD icinga:nextupdate:host
  * IcingaDB::StateChangeHandler
    * XADD icinga:runtime:state
    * IcingaDB::ForwardHistoryEntries
      * XADD icinga:history:stream:state

After:

* Checkable::ExecuteCheck
  * Checkable::UpdateNextCheck
  * RandomCheckTask::ScriptFunc
    * Checkable::ProcessCheckResult
      * Checkable::UpdateNextCheck
      * IcingaDB::NewCheckResultHandler
        * HSET icinga:host:state
        * HSET icinga:checksum:host:state
        * ZADD icinga:nextupdate:host
  * IcingaDB::StateChangeHandler
    * XADD icinga:runtime:state
    * IcingaDB::ForwardHistoryEntries
      * XADD icinga:history:stream:state

The first state + nextupdate (for overdue) update comes from next_check being
set to now + interval immediately before doing the actual check (not to trigger
it twice). This update is not only not important for the end user, but even
inappropriate. The end user SHALL see next_check being e.g. in -4s, not 5m, as
the check is running at the moment.

The second one is just redundant as IcingaDB::NewCheckResultHandler (the third
one) is called anyway and will update state + nextupdate as well.
2022-09-06 10:10:14 +02:00
Alexander A. Klimov 790bad9250 Fix compile error on Solaris 11.4
by not using LOG_FTP which is not defined there.
2022-08-16 12:07:05 +02:00
Julian Brost d3cca0e621
Merge pull request #9409 from Icinga/feature/disallow-empty-object-names
Disallow empty object names
2022-08-11 16:53:28 +02:00
Alexander A. Klimov a2362ebf17 IcingaDB::VersionChangedHandler(): don't handle not synced types
not to surprise (and crash) the Icinga DB daemon with unknown types.
2022-08-10 13:24:44 +02:00
Alexander A. Klimov 32871ca40c IcingaDB::SendCustomVarsChanged(): don't delete custom vars of not synced types
not to surprise (and crash) the Icinga DB daemon with unknown types.
2022-08-10 11:40:53 +02:00
Alexander A. Klimov c9d6eecc7f Dump state file atomically not to corrupt it
by using fsync(2) before close(2) and rename(2).
2022-07-28 18:00:37 +02:00
Alexander A. Klimov 600fb0e3c2 Introduce AtomicFile 2022-07-28 18:00:37 +02:00
Julian Brost 913566cbfa Windows: output useful error message for syscall errors 2022-07-28 17:00:57 +02:00
Yonas Habteab 148f5b8416 ConfigObject: Mark object names as required
This prevents a user from creating an object without a valid name such as `"", null`.
2022-07-15 15:51:58 +02:00
Julian Brost a927ba39b7 Windows: only include critical messages in early log messages
The point of logging to the Windows Event Log was to catch errors that happen
before the full logging configuration has been loaded and enabled. Messages
like the number of loaded objects per type just cause noise in the log and
provide little benefit. Therefore raise the required log level at this stage.

Note that this commit removes the (never documented) ability to use the -x flag
to change the level. But doing so would require patching the command line of
the service in the registry anyways.
2022-07-14 14:07:56 +02:00
Julian Brost bd2118c4cd
Merge pull request #9420 from Icinga/IcingaDB-soft_state
Icinga DB: icinga:*:state: rename state to soft_state
2022-06-29 12:24:52 +02:00
Alexander A. Klimov ba9a5c614c Icinga DB: icinga:*:state: rename state to soft_state 2022-06-29 11:49:06 +02:00
Julian Brost 9b24056e05
Merge pull request #9346 from Icinga/icingadb-check
Introduce Icinga DB check (like the IDO one)
2022-06-28 18:24:29 +02:00
Julian Brost 3222fab05a Icinga DB Check: don't check runtime update backlog during full sync 2022-06-28 13:33:00 +02:00
Julian Brost 4f125753bf Icinga DB Check: ignore suppressed queries in Redis backlog check
If some kind of query is not supposed to be processed at the moment, there is
little point in checking it. During a full dump, state updates are suppressed
(i.e. delayed), so when a dump takes very long, this would have resulted in a
false Redis backlog warning.
2022-06-28 13:33:00 +02:00
Julian Brost 5550fb713c Icinga DB Check: include ongoing dumps in OK message
Also use the "current" and "full dump/sync" terminology in the other messages.
2022-06-28 13:33:00 +02:00
Julian Brost 3ded7a9268 Icinga DB Check: rename dump/sync related perfdata values
Scope all values using current/last instead of takes/took.
2022-06-28 13:33:00 +02:00
Julian Brost e36bc92a2c Icinga DB Check: add unit hints to all rates 2022-06-28 13:33:00 +02:00
Julian Brost eaae7d5863 Icinga DB Check: update not connected message
The check makes no attempt to explicitly connect to Redis, it uses the
connection of the IcingaDB feature, so this message better describes the state
in this situation.
2022-06-28 13:33:00 +02:00
Julian Brost 2fafffb85f Icinga DB Check: fix race-condition with IcingaDB::Start()
IcingaDB::GetConnection() uses IcingaDB::m_Rcon which is only initialized in
IcingaDB::Start(), therefore add a nullptr check to the check command.
Additionally, as m_Rcon is potentially accessed concurrently, add a copy of the
value that is safe for concurrent use.
2022-06-28 13:33:00 +02:00
Julian Brost 953e113465 Icinga DB Check: remove markdown headings from output
icingadb-web shows multiple lines from the check output collapsed into a single
line. The lines containing just minuses make this look cluttered and making
making it a heading provides little to no benefit. Even when rendering markdown
in the check output at some point, having the lists labeled using normal
paragraphs would look just fine.
2022-06-28 13:33:00 +02:00
Julian Brost c59d44cd8b Icinga DB Check: rename perfdata values
- Add icinga2_ and icingadb_ prefixes to make clear which component is
  responsible for the value.
- Rename heartbeat_lag to heartbeat_age, describes it better in my opinion and
  sound a bit less like something that should be as close to zero as possible.
- Rename redis_dump/database_sync into full_dump/full_sync as this is how these
  operations are refered to in log messages as well.
- Rename Redis backlog into Redis query backlog, makes it a bit clearer in my
  opinion.
- Rename runtime_backlog into runtime_update_backlog, as the component in
  Icinga DB is called that way and this naming is also exposed in log messages.
- Rename dump_config/state/history into config/state/history_dump, makes it
  sound more natural.
2022-06-28 13:33:00 +02:00
Julian Brost d0382f71ab Icinga DB Check: rename variables from takes to duration
Sounds more natural in my opinion and I doubt that many users would get that
due to the difference between takes/took, this refers to ongoing dumps.
2022-06-28 13:33:00 +02:00
Julian Brost 3c29b15214 Icinga DB Check: use more natural names for sync/cleanup metrics 2022-06-28 13:33:00 +02:00
Julian Brost d70a27b982 Icinga DB Check: report history and runtime update backlog separately
Probably makes little difference for an end-user, but for support and
development it's great to know which of the two is causing problems.
2022-06-28 13:33:00 +02:00
Julian Brost 2a4605f4b7 Icinga DB Check: clearly state Icinga 2 Redis backlog
Should make it easier to understand that this refers to Redis queries issued by
Icinga 2.
2022-06-28 13:33:00 +02:00
Julian Brost 5613412b81 Icinga DB Check: replace nested calls to fmax() with std::max()
Improves readability, even more so after splitting it into separate lines.
2022-06-28 13:33:00 +02:00
Julian Brost f3f1373f83 Icinga DB Check: spell out "error" in perfdata 2022-06-28 13:33:00 +02:00
Julian Brost 31c7dfee53 Icinga DB Check: fix error message on Redis query error
Not only XREAD queries are performed, so the previous error message was incorrect.
2022-06-28 13:33:00 +02:00
Julian Brost 4f1f70f843 Icinga DB Check: remove unused includes 2022-06-28 13:33:00 +02:00
Julian Brost 2b310718e3 Icinga DB Check: rename keys in heartbeat stream
In both C++ and Go, the keys are only used as constant strings, so namespacing
them just adds clutter for the `general:*` keys, therefore remove it.
2022-06-28 13:33:00 +02:00
Julian Brost d74fbbbb82 Icinga DB Check: remove *_1sec metrics
They add no additional information compared to the *_1min values as it's always
the same value divided by 60 anyways. Adding the actual value from the last
second makes little sense for realistic values of check_interval.
2022-06-28 13:33:00 +02:00
Julian Brost 44cbd04088 Icinga DB Check: read performance data string from Redis
Use the already existing format to pass performance data to Icinga 2 rather
than some new JSON structure. Has the additional benefit of doing more things
in Go than in C++.
2022-06-28 13:33:00 +02:00
Yonas Habteab 0ffef02c1d IcingaDB: Adjust some column names according to the DB schema 2022-06-23 14:27:34 +02:00
Alexander A. Klimov e4a36bc217 Introduce Icinga DB check (like the IDO one) 2022-06-23 11:14:31 +02:00
Alexander A. Klimov 88c8d29ee6 Remove Icinga DB perfdata from Icinga check
as the Icinga DB check already yields it.
2022-06-22 13:25:29 +02:00
Julian Brost 6b4681ee9e Icinga DB: make error message more helpful if API isn't set up 2022-06-20 14:57:19 +02:00
Alexander A. Klimov 8eef51afeb Introduce IcingaDB::AddKvsToMap() 2022-06-20 13:47:39 +02:00
Alexander A. Klimov 2c3d2f8b87 RedisConnection::ReadRESP(): *-1\r\n is null, not [ ] 2022-06-20 13:47:39 +02:00
Alexander Aleksandrovič Klimov 4522522444
Merge pull request #9362 from Icinga/bugfix/remove-redundant-serialization
Remove redundant call to Serialize() in ConfigItem::Commit()
2022-06-15 09:34:38 +02:00
Julian Brost ad218c9a12 Icinga DB: initialize environment ID during config validation
IcingaDB may receive callbacks from Boost signals before being fully started.
This resulted in situations where m_EnvironmentId was used before it was
initialized properly. This is fixed by initializing it earlier (during the
config validation stage). However, at this stage, it should not yet write to
disk, therefore, persisting the environment ID to disk is delayed until later
in the startup process.

Initializing at this stage has an extra benefit: if there is an error for some
reason (possibly corrupt icingadb.env file), this now shows up as a nice error
during config validation.

Additionally, this replaces the use of std::call_once with std::mutex due to
bug in libstdc++ (see inline comment for reference).
2022-06-10 14:19:58 +02:00
Yonas Habteab 45f536ca06 Bump Redis schema version to 5 2022-06-07 12:55:12 +02:00
Yonas Habteab 92becec37f IcingaDB: Add `_name` suffix to columns referring to name 2022-05-31 16:41:40 +02:00
Eric Lippmann 18c8b4ad54
Merge pull request #9371 from Icinga/bugfix/icingadb-command-arguments-null
IcingaDB: handle null (Empty) for value/set_if/separator in command arguments
2022-05-23 16:01:49 +02:00
Julian Brost 3220fecd4c
Merge pull request #7919 from Icinga/feature/parameter-delimiters-check-execution-6277
Introduce Command#arguments[].separator
2022-05-23 13:23:36 +02:00
Julian Brost f110e26635 IcingaDB: handle null (Empty) for value/set_if/separator in command arguments
Icinga 2 treats null (Empty) as if the corresponding attribute is not
specified. However, without this commit, it would serialize the value as "null"
(i.e. type string), so that it ends up in the database as this string instead
of NULL. This commit adds handling for ValueEmpty so that is serialized as JSON
null value and ends up in the database as NULL.
2022-05-23 11:53:41 +02:00
Alexander A. Klimov 069c3968d9 Introduce Command#arguments[].sep
... for letting check commands produce argv like --key=value,
not just --key value.

refs #6277
2022-05-11 17:50:12 +02:00
Julian Brost 4184dcd62c
Merge pull request #9354 from WuerthPhoenix/feature/return-correct-status-in-process-check-result-api
Return correct status codes in process-check-result API
2022-05-05 15:30:09 +02:00
Julian Brost abe2dfa763 Replace EventuallyAtomic with AtomicOrLocked which falls back to a mutex
Apparently there was a reason for making the members of generated classes
atomic. However, this was only done for some types, others were still accessed
using non-atomic operations. For members of type T::Ptr (i.e.  intrusive_ptr<T>),
this can result in a double free when multiple threads access the same variable
and at least one of them writes to the variable.

This commit makes use of std::atomic<T> for more T (it removes the additional
constraint sizeof(T) <= sizeof(void*)) and uses a type including a mutex for
load and store operations as a fallback.
2022-05-03 12:02:46 +02:00
Julian Brost 2dcdae4470 Remove redundant call to Serialize() in ConfigItem::Commit()
The very same object is already serialized a few lines above, the result is
even stored in a variable, but that variable was not used before. Simply using
this variable results in a noticeable improvement of config validation times.
2022-04-28 17:09:16 +02:00
Damiano Chini 9d9810b44d Return correct status codes in process-check-result API 2022-04-26 13:33:59 +02:00
Julian Brost 51cd7e7b0b Take host state into account when sending suppressed notifications
Checkable::FireSuppressedNotifications() compares the time of the current
checkable with the last recovery time of parents to avoid notification right
after a parent recovered and before the current checkable was checked.

This commit makes this check also include to host if the checkable is a
service.  This makes the behavior consistent with the documentation that states
there is an implicit dependency on the host (which isn't realized as implicitly
generating a Dependency object unfortunately).
2022-04-19 16:13:15 +02:00
Julian Brost 178aaaeca9
Merge pull request #9332 from Icinga/bugfix/compare-cluster-tickets-in-constant-time
Compare cluster tickets in constant time
2022-04-11 15:32:32 +02:00
Julian Brost b24a2fa2a5
Merge pull request #9179 from Icinga/Al2Klimov-patch-3
Let new cluster certificates expire after 397 days, not 15 years
2022-04-11 15:29:05 +02:00
Julian Brost 0e880048ee
Merge pull request #7961 from Icinga/bugfix/startup-log
Place startup.log and status in /var/lib/icinga2/api, not /var/lib/icinga2/api/zones-stage
2022-04-11 14:41:07 +02:00
Alexander A. Klimov b15763bd86 Compare cluster tickets in constant time
Just to be sure.
2022-04-11 11:17:05 +02:00
Alexander A. Klimov 08a23f4035 Write also /var/lib/icinga2/api/zones-stage-startup-last-failed.log
in addition to /var/lib/icinga2/api/zones-stage-startup.log
to prevent the next success to overwrite the last failure.
2022-04-11 11:14:42 +02:00
Alexander A. Klimov c9e4c016e0 Protect ApiListener#m_SSLContext with a mutex 2022-04-11 11:02:45 +02:00
Alexander A. Klimov e490883577 Renew certificates also periodically 2022-04-11 11:02:39 +02:00
Alexander Aleksandrovič Klimov 39d642af75
Merge pull request #9321 from Icinga/perfdata-resume-signal
Perfdata writers: disconnect handlers from signals in Pause()
2022-04-07 15:51:02 +02:00
Alexander A. Klimov ce6d1b8961 Place startup.log and status in /var/lib/icinga2/api, not /var/lib/icinga2/api/zones-stage
not to loose them.
2022-04-07 11:24:24 +02:00
Alexander Aleksandrovič Klimov b29b95e882
Merge pull request #9267 from Icinga/bugfix/parallel-api-package-calls-do-not-finish-while-reload
Worker process doesn't let parallel API package stage updates to complete when terminated
2022-04-06 13:27:44 +02:00
Alexander A. Klimov 56933b8877 Perfdata writers: disconnect handlers from signals in Pause()
as they would be re-connected in Resume() (HA).

Before they were still connected during pause and connected X+1 times
after X split-brains (the same data was written X+1 times).
2022-04-06 13:09:26 +02:00
Alexander A. Klimov 3753f86c80 ApiListener#Start(): auto-renew own cert if CA owner
otherwise that particular cert would expire.
2022-04-04 12:13:31 +02:00
Alexander A. Klimov 6d470a3ca5 Introduce ApiListener#RenewCert() 2022-04-04 12:12:31 +02:00
Alexander Aleksandrovič Klimov f749c7556e
Merge pull request #9314 from Icinga/latin1
IDO MySQL: reason latin1 charset for actually UTF-8 bytes
2022-04-04 11:05:12 +02:00
Alexander A. Klimov 11b8d0f058 IDO MySQL: reason latin1 charset for actually UTF-8 bytes 2022-03-31 18:10:21 +02:00
Alexander Aleksandrovič Klimov 2fa26961ac
Merge pull request #9311 from Icinga/9308
IDO MySQL: explicitly use latin1
2022-03-31 16:44:11 +02:00