Commit Graph

328 Commits

Author SHA1 Message Date
Julian Brost 3c29b15214 Icinga DB Check: use more natural names for sync/cleanup metrics 2022-06-28 13:33:00 +02:00
Julian Brost d70a27b982 Icinga DB Check: report history and runtime update backlog separately
Probably makes little difference for an end-user, but for support and
development it's great to know which of the two is causing problems.
2022-06-28 13:33:00 +02:00
Julian Brost 2a4605f4b7 Icinga DB Check: clearly state Icinga 2 Redis backlog
Should make it easier to understand that this refers to Redis queries issued by
Icinga 2.
2022-06-28 13:33:00 +02:00
Julian Brost 5613412b81 Icinga DB Check: replace nested calls to fmax() with std::max()
Improves readability, even more so after splitting it into separate lines.
2022-06-28 13:33:00 +02:00
Julian Brost f3f1373f83 Icinga DB Check: spell out "error" in perfdata 2022-06-28 13:33:00 +02:00
Julian Brost 31c7dfee53 Icinga DB Check: fix error message on Redis query error
Not only XREAD queries are performed, so the previous error message was incorrect.
2022-06-28 13:33:00 +02:00
Julian Brost 4f1f70f843 Icinga DB Check: remove unused includes 2022-06-28 13:33:00 +02:00
Julian Brost 2b310718e3 Icinga DB Check: rename keys in heartbeat stream
In both C++ and Go, the keys are only used as constant strings, so namespacing
them just adds clutter for the `general:*` keys, therefore remove it.
2022-06-28 13:33:00 +02:00
Julian Brost d74fbbbb82 Icinga DB Check: remove *_1sec metrics
They add no additional information compared to the *_1min values as it's always
the same value divided by 60 anyways. Adding the actual value from the last
second makes little sense for realistic values of check_interval.
2022-06-28 13:33:00 +02:00
Julian Brost 44cbd04088 Icinga DB Check: read performance data string from Redis
Use the already existing format to pass performance data to Icinga 2 rather
than some new JSON structure. Has the additional benefit of doing more things
in Go than in C++.
2022-06-28 13:33:00 +02:00
Yonas Habteab 0ffef02c1d IcingaDB: Adjust some column names according to the DB schema 2022-06-23 14:27:34 +02:00
Alexander A. Klimov e4a36bc217 Introduce Icinga DB check (like the IDO one) 2022-06-23 11:14:31 +02:00
Alexander A. Klimov 88c8d29ee6 Remove Icinga DB perfdata from Icinga check
as the Icinga DB check already yields it.
2022-06-22 13:25:29 +02:00
Julian Brost 6b4681ee9e Icinga DB: make error message more helpful if API isn't set up 2022-06-20 14:57:19 +02:00
Alexander A. Klimov 8eef51afeb Introduce IcingaDB::AddKvsToMap() 2022-06-20 13:47:39 +02:00
Alexander A. Klimov 2c3d2f8b87 RedisConnection::ReadRESP(): *-1\r\n is null, not [ ] 2022-06-20 13:47:39 +02:00
Julian Brost ad218c9a12 Icinga DB: initialize environment ID during config validation
IcingaDB may receive callbacks from Boost signals before being fully started.
This resulted in situations where m_EnvironmentId was used before it was
initialized properly. This is fixed by initializing it earlier (during the
config validation stage). However, at this stage, it should not yet write to
disk, therefore, persisting the environment ID to disk is delayed until later
in the startup process.

Initializing at this stage has an extra benefit: if there is an error for some
reason (possibly corrupt icingadb.env file), this now shows up as a nice error
during config validation.

Additionally, this replaces the use of std::call_once with std::mutex due to
bug in libstdc++ (see inline comment for reference).
2022-06-10 14:19:58 +02:00
Yonas Habteab 45f536ca06 Bump Redis schema version to 5 2022-06-07 12:55:12 +02:00
Yonas Habteab 92becec37f IcingaDB: Add `_name` suffix to columns referring to name 2022-05-31 16:41:40 +02:00
Eric Lippmann 18c8b4ad54
Merge pull request #9371 from Icinga/bugfix/icingadb-command-arguments-null
IcingaDB: handle null (Empty) for value/set_if/separator in command arguments
2022-05-23 16:01:49 +02:00
Julian Brost f110e26635 IcingaDB: handle null (Empty) for value/set_if/separator in command arguments
Icinga 2 treats null (Empty) as if the corresponding attribute is not
specified. However, without this commit, it would serialize the value as "null"
(i.e. type string), so that it ends up in the database as this string instead
of NULL. This commit adds handling for ValueEmpty so that is serialized as JSON
null value and ends up in the database as NULL.
2022-05-23 11:53:41 +02:00
Alexander A. Klimov 069c3968d9 Introduce Command#arguments[].sep
... for letting check commands produce argv like --key=value,
not just --key value.

refs #6277
2022-05-11 17:50:12 +02:00
Alexander Aleksandrovič Klimov d171301b9d
Merge pull request #9298 from Icinga/bugfix/icingadb-remove-comment-history
Icinga DB: discard comment removals with missing information
2022-03-29 11:25:01 +02:00
Alexander Aleksandrovič Klimov bbc2b59b0d
Merge pull request #9287 from Icinga/9275
Icinga DB: correct ack comments' is_sticky
2022-03-28 22:42:52 +02:00
Julian Brost d139bc31c8 Icinga DB: discard comment removals with missing information
If comments get removed in unintended ways (i.e. not by expiring or by using
the remove-comment API action), the comment object misses information to create
a proper history event for Icinga DB. Therefor, discard these events.
2022-03-28 16:58:05 +02:00
Alexander A. Klimov 1220ad8a2f Icinga DB: correct ack comments' is_sticky
On ack Icinga first adds a comment, then acks the checkable
so the ack event has the comment ID.

But due to the yet missing ack the comment was missing is_sticky.
That's corrected now.
2022-03-24 16:42:18 +01:00
Julian Brost cfa6f1c6a9
Merge pull request #9288 from Icinga/9272
IcingaDB#SendRemovedComment(): ignore ack comments like #SendAddedComment()
2022-03-22 15:06:06 +01:00
Alexander A. Klimov 27966c3c08 IcingaDB#SendRemovedComment(): ignore ack comments like #SendAddedComment()
Icinga DB doesn't expect comment history for ack comments.

Before:

1. Acked checkable recovers
2. Icinga clears ack comments w/o setting removal time
3. Icinga DB gets neither removal time, nor expire time
4. Icinga DB falls back to NULL and violates NOT NULL constraint
2022-03-21 17:06:35 +01:00
Julian Brost 9630e86997 Add missing array locking in IcingaDB::GetArrayDeletedValues()
icinga::Array requires locking by the caller when iterating using Begin() and
End(). This is only checked in debug builds but there it makes this function
fail.
2022-03-09 14:29:44 +01:00
Julian Brost bf5b905707
Merge pull request #9250 from Icinga/feature/fix-compiler-warning-do-not-move-local-variables
Fix compiler warnings don't move local variables
2022-03-08 11:37:09 +01:00
Alexander A. Klimov 6b5106ffdd IcingaDB#Stop(): don't block shutdown, timeout instead 2022-03-02 16:39:44 +01:00
Alexander A. Klimov 3a8efcb4ea IcingaDB#Send*(): don't enqueue any history once stopped 2022-03-02 16:39:44 +01:00
Alexander A. Klimov cac22fe38b RedisConnection#Connect(): wait for all promises to be completed
by the read loop from the previous connection.
2022-03-02 16:39:44 +01:00
Alexander A. Klimov 732d5c472d RedisConnection#ReadLoop(): don't crash (silently) if a promise to be set is already set 2022-03-02 16:39:37 +01:00
Alexander A. Klimov 50fee6aeb9 Icinga DB: include amount of history kept in memory in /v1/status 2022-03-02 16:39:37 +01:00
Alexander A. Klimov ad0fe764f7 Icinga DB: log amount of history kept in memory every 10s 2022-03-02 16:39:37 +01:00
Alexander A. Klimov 8ea62f7fc7 Icinga DB: keep history in memory until written to Redis
by putting the messages into a Bulker and retrying each chunk.
2022-03-02 16:39:37 +01:00
Yonas Habteab a0607aceff Fix compiler warnings don't move local variables 2022-02-22 17:51:43 +01:00
Julian Brost 99008755b5
Merge pull request #9213 from Icinga/feature/icingadb-add-previous_soft_state-to-host_state-and-service_state-9210
IcingaDB: Add previous_soft_state to host_state and service_state
2022-02-18 14:09:35 +01:00
Alexander A. Klimov c613e62454 IcingaDB: Add previous_soft_state to host_state and service_state
refs #9210
2022-02-14 11:32:46 +01:00
Alexander A. Klimov 84d09876b4 Icinga DB: ensure is_reachable and severity don't miss updates
refs #9143
2022-01-25 13:33:46 +01:00
Julian Brost 185fab3761
Merge pull request #9144 from Icinga/bugfix/icingadb-state-history
Icinga DB: don't write state history for ack/downtime/host problem changes
2022-01-20 12:00:24 +01:00
Julian Brost 463b159414
Merge pull request #9171 from Icinga/bugfix/icinga-db-notification-history-might-use-incorrect-previous_hard_state-9132
IcingaDB#SendSentNotification(): make stream deterministic via CheckResult#previous_hard_state
2022-01-18 16:54:16 +01:00
Julian Brost 31da6a56e6 Icinga DB: remove obsolete StateChangeHandler overload
This version of StateChangeHandler is no longer called anywhere as it was the
wrong function for all previous callers anyways.
2022-01-18 12:26:43 +01:00
Julian Brost cf73c6136b Icinga DB: make host problem change events update the state tables but not write state history
StateChangeHandler() is the function used when the actual hard/soft state
changes and thus also writes state history. This is not desired in this case,
instead, a runtime update should be generated, therefore call UpdateState()
instead.

refs #9063
2022-01-18 12:26:43 +01:00
Julian Brost 855e342b63 Icinga DB: make acknowledgement events update the state tables but not write state history
StateChangeHandler() is the function used when the actual hard/soft state
changes and thus also writes state history. This is not desired in this case,
instead, a runtime update should be generated, therefore call UpdateState()
instead.

refs #9063
2022-01-18 12:26:43 +01:00
Julian Brost f63268b0dd Icinga DB: make downtime events update the state tables but not write state history
StateChangeHandler() is the function used when the actual hard/soft state
changes and thus also writes state history. This is not desired in this case,
instead, a runtime update should be generated, therefore call UpdateState()
instead.

refs #9063
2022-01-18 12:26:43 +01:00
Julian Brost 447884be72 Icinga DB: don't reimplement volatile state update in SendConfigUpdate
Sending a volatile state update is already implemented in UpdateState, so just
use that function instead of generating the update queries.
2022-01-18 12:26:43 +01:00
Julian Brost a6d6cb788e Icinga DB: Merge SendStatusUpdate into UpdateState
Previously, both funktions did related operations but had unclear and confusing
naming:
- UpdateState updated the icinga:{host,service}:state Redis keys.
- SendStatusUpdate sent a runtime update for the icinga:{host,service}:state.

This commit merges both functions into one with a new mode parameter. The
following modes are now supported:
- Volatile: Update the icinga:{host,service}:state Redis key.
- Full: Perform the volatile state update and in addition send a corresponding
  runtime update so that this state update gets written through to the
  persistent database by a running icingadb process.
- RuntimeOnly: Special mode for callers that can ensure that a volatile update
  for the current state was already performed but has to be upgraded to a full
  update.

refs #9063
2022-01-18 12:26:43 +01:00
Alexander A. Klimov 1fee3f1b12 IcingaDB#SendSentNotification(): make stream deterministic via CheckResult#previous_hard_state
Now it gets everything from one source, the CheckResult.

refs #9132
2022-01-10 19:18:11 +01:00
Julian Brost e518dc2436
Merge pull request #9112 from Icinga/bugfix/sync-missing-history-information
Icinga DB: ensure consistent history streams in HA setup
2022-01-07 15:14:06 +01:00
Julian Brost 3e73a262cc Sync comment and downtime removal info for Icinga DB history
When a comment or downtime is removed manually, the name of the requestor and
timestamp have to be synced to other nodes in the cluster to allow all of them
to generate a consistent Icinga DB history stream.

refs #9101
2022-01-05 10:27:13 +01:00
Alexander A. Klimov a8c9d19dae Icinga DB: ensure icinga:*command:argument#order is an int
The config parser requires *Command#arguments#order to be a Number, i.e. 42,
4.2 or even "4.2". That's int-casted where needed, now also for Icinga DB.

Before:

```
object CheckCommand "9117" {
	command = [ "true" ]
	arguments = {
		"4.2" = { order = "4.2" }
	}
}
```

2022-01-03T13:25:07.166+0100	FATAL	icingadb	json: cannot unmarshal string into Go value of type int64
2022-01-03 13:28:19 +01:00
Noah Hilverling 4d3b1709fd
Merge pull request #9009 from Icinga/bugfix/icingadb-runtime-updates-delete-relationships
Icinga DB: Make sure object relationships are handled correctly during runtime updates
2021-11-12 17:52:59 +01:00
Julian Brost b9e6273ba0 Icinga DB: only log queries at debug level 2021-11-12 15:41:17 +01:00
Noah Hilverling 7a0796061a IcingaDB::AddObjectDataToRuntimeUpdates(): Copy data before modifying 2021-11-12 13:34:57 +01:00
Noah Hilverling 73e0d6e61b Icinga DB: Make sure object relationships are handled correctly 2021-11-12 13:34:57 +01:00
Noah Hilverling 4e79eb080c
Merge pull request #9058 from Icinga/bugfix/icingadb-prefix-command_id
IcingaDB: Prefix command_id with command type
2021-11-11 11:50:26 +01:00
Noah Hilverling c1098bef35
Merge pull request #9061 from Icinga/add-downtime-duration-and-service-state-host-id-streams
Icinga DB: Add `downtime.duration` & `service_state.host_id` to Redis
2021-11-11 10:19:47 +01:00
Noah Hilverling a9c2304c61 IcingaDB: Prefix command_id with command type 2021-11-09 12:26:30 +01:00
Eric Lippmann 35053ac1dd Icinga DB: Sync groups earlier
Host and service groups are structural information that are used
for Web filters and should therefore be synchronized as soon as
possible.
2021-11-09 11:17:01 +01:00
Alexander A. Klimov 07c8440fd2 Icinga DB: sync checkables along with their states first
`WorkQueue#ParallelFor(x, false, y)` will enqueue x's items in FIFO order,
so x has to start with host and service.
2021-11-09 11:17:01 +01:00
Yonas Habteab fe5aa1e18d Icinga DB: Add `service_state.host_id` to Redis 2021-11-09 11:08:22 +01:00
Yonas Habteab 5dc45baebb Icinga DB: Add `downtime.duration` & `scheduled_duration` to Redis 2021-11-09 11:08:22 +01:00
Julian Brost 848f1ae167
Merge pull request #8998 from Icinga/bugfix/icingadb-program-start-milliseconds
Icinga DB: set value in milliseconds for program_start in stats/heartbeat
2021-11-08 18:18:19 +01:00
Julian Brost 524fe92a1d
Merge pull request #9028 from Icinga/bugfix/icingadb-zone-parent
IcingaDB: actually write parent to parent_id of zones
2021-11-08 18:08:48 +01:00
Julian Brost e46d83b6be Icinga DB: set value in milliseconds for program_start in stats/heartbeat 2021-11-08 14:37:08 +01:00
Noah Hilverling 0b9317a5bf IcingaDB: Remove GetObjectIdentifiersWithoutEnv()
Having the command type be a part of the command ID isn't needed anywhere. Removing this simplifies the way we generate IDs in general, because we don't need Prepend() anymore.

The command type was only needed to prevent ID collisions within the command_envvar and command_argument tables. Those tables have since been separated into {check,event,notification}command_envvar and {check,event,notification}command_argument tables.
2021-11-05 17:01:40 +01:00
Julian Brost 3c8672b4dc Icinga DB: increase Redis schema version
PR #9036 introduces some incompatible changes to the Redis schema, most
importantly where Icinga DB has to read the environment from: now it has to use
a new top-level key of the icinga:stats message instead of a value in the
IcingaApplication part of that message.
2021-11-05 14:14:37 +01:00
Julian Brost 6007848146 IcingaDB: export environment_id via API
Primarily required for Icinga DB integration tests at the moment, but could
also be helpful in other situations.
2021-11-05 14:14:37 +01:00
Julian Brost 4ade4c757b IcingaDB: write new environment to icinga:stats stream 2021-11-05 14:14:37 +01:00
Julian Brost 525dd50859 IcingaDB: introduce a new environment ID derived from the CA public key
In order to avoid changes to the environment ID, it is now no longer derived
from the Environment constant but instead from the public key of the CA
certificate. This ensures that it is different between clusters by default, so
no additional changes have to be done to allow two clusters to use Icinga DB to
write into the same database.

To prevent the ID from changing when the CA certificate is replaced, it is also
persisted into the file /var/lib/icinga2/icingadb.env, so if that file exists,
it takes precedence over the CA certificate.
2021-11-05 14:14:37 +01:00
Julian Brost f976e351f4
Merge pull request #9044 from Icinga/bugfix/idb-dump-buf-lost
Icinga DB init. dump: flush both buffered states and state checksums
2021-11-04 12:26:28 +01:00
Alexander A. Klimov 0ff7d0a06e Icinga DB: raise icinga:schema 1 -> 2 2021-11-02 15:00:55 +01:00
Alexander A. Klimov b1714a10c2 Icinga DB: make icinga:history:stream:*#event_id deterministic
... i.e. UUID -> SHA1(env, eventType, x...) given that SHA1(env, x...) = type-specific ID.
Rationale: allow both masters to write the same history concurrently (while not
in split-brain), so that REPLACE INTO deduplicates the same events written twice.

* ack: SHA1(env, "ack_set"|"ack_clear", checkable.name, setTime)
* comment: SHA1(env, "comment_add"|"comment_remove", comment.name)
* downtime: SHA1(env, "downtime_start"|"downtime_end", downtime.name)
* flapping: SHA1(env, "flapping_start"|"flapping_end", checkable.name, startTime)
* notification: SHA1(env, "notification", notification.name, notificationType, sendTime)
* state: SHA1(env, "state_change", checkable.name, changeTime)
2021-11-02 15:00:03 +01:00
Alexander A. Klimov 5c44365c4e Icinga DB: make icinga:history:stream:notification#id deterministic
... i.e. UUID -> SHA1(x..., send time) given that SHA1(x...) = notification id.
Rationale: allow both masters to write the same notification history concurrently (while
not in split-brain), so that REPLACE INTO deduplicates the same events written twice.
2021-11-02 15:00:03 +01:00
Alexander A. Klimov c2422c56fe Icinga DB: make icinga:history:stream:state#id deterministic
... i.e. UUID -> SHA1(x..., check time) given that SHA1(x...) = checkable id.
Rationale: allow both masters to write the same state history concurrently (while
not in split-brain), so that REPLACE INTO deduplicates the same events written twice.
2021-11-02 15:00:03 +01:00
Alexander Aleksandrovič Klimov f5f8ccb1f4
Merge pull request #9020 from Icinga/feature/icingaeb-schema-version
Icinga DB: publish Redis schema version via XADD icinga:schema
2021-10-25 13:21:37 +02:00
Alexander A. Klimov d8b4768471 Icinga DB init. dump: flush both buffered states and state checksums
not to dump x states, but only x - (x % bulk) state checksums.
2021-10-21 13:49:24 +02:00
Noah Hilverling a7cbf50674
Merge pull request #9030 from Icinga/Al2Klimov-patch-1
Icinga DB: don't include checkable types in history IDs
2021-10-19 14:52:43 +02:00
Alexander A. Klimov 4b0688047e Icinga DB: stream runtime state updates only to icinga:runtime:state
... where they belong to, not to icinga:runtime.
2021-10-18 18:11:30 +02:00
Alexander Aleksandrovič Klimov e0339c387b
Icinga DB: don't include checkable types in history IDs
... as they’re unnecessary for being distinguish across types.
Services always have a ! in the name, hosts never do.
2021-10-11 16:14:30 +02:00
Alexander Aleksandrovič Klimov 30a5ba3961
Merge pull request #9002 from Icinga/feature/icingadb-remove-usernotification-stream
Icinga DB: remove usernotification history stream
2021-10-08 19:16:26 +02:00
Alexander Aleksandrovič Klimov 4190d58668
Merge pull request #9011 from Icinga/bugfix/icingadb-remove-zone-parent-key
Icinga DB: Remove unused Redis key 'icinga:zone:parent'
2021-10-08 17:19:51 +02:00
Noah Hilverling 750e64b974 Icinga DB: Remove unused Redis key 'icinga:zone:parent' 2021-10-08 12:06:14 +02:00
Julian Brost df84a498f4 IcingaDB: actually write parent to parent_id of zones
This fixes that the code used the wrong variable. Previously, it was written to
Redis that each zone is its own parent (if it has a parent at all).
2021-10-08 11:15:54 +02:00
Alexander Aleksandrovič Klimov ed50a9d529
Merge pull request #9001 from Icinga/feature/icingadb-add-user-ids-to-notification-history
Icinga DB: Write IDs of notified users into notification history stream
2021-10-01 17:42:48 +02:00
Alexander A. Klimov 0182d793ac Icinga DB: publish Redis schema version via XADD icinga:schema
... to be able both to subscribe for its change and to just fetch it.
2021-10-01 15:58:57 +02:00
Alexander Aleksandrovič Klimov 6cf0673c11
Merge pull request #9010 from Icinga/feature/icingadb-scheduling_source
Make CheckResult#scheduling_source available to Icinga DB
2021-09-27 16:31:16 +02:00
Alexander A. Klimov 755fc72a66 Make CheckResult#scheduling_source available to Icinga DB 2021-09-22 16:57:49 +02:00
Julian Brost 6fc15449a8
Merge pull request #8953 from Icinga/bugfix/icinga-checksum-state-growing
Icinga DB: clean up vanished objects from icinga:checksum:*:state
2021-09-17 12:04:41 +02:00
Julian Brost 130b22e939 Icinga DB: remove usernotification history stream
These will be added to the normal notification stream so there is no more need
for this extra stream.
2021-09-15 14:47:25 +02:00
Julian Brost 81e5feeb08 Icinga DB: Write IDs of notified users into notification history stream 2021-09-15 14:45:35 +02:00
Noah Hilverling 95cdc00ad4
Merge pull request from GHSA-cxfm-8j5v-5qr2
Add TLS server certificate validation to ElasticsearchWriter, GelfWriter, InfluxdbWriter and Influxdb2Writer (v2)
2021-08-19 13:52:29 +02:00
Julian Brost cb09d6833f RedisConnection: remove now redundant setting of TLS verification parameters
This is now done in UnbufferedAsioTlsStream.
2021-08-13 17:24:24 +02:00
Alexander A. Klimov 70b4558a62 Icinga DB: introduce icinga:history:stream:downtime#scheduled_by
... with the Downtime#scheduled_by attribute.
2021-08-09 20:07:38 +02:00
Alexander A. Klimov e3a5d613aa Icinga DB: clean up vanished objects from icinga:checksum:*:state
... not to let it grow non-stop.
2021-08-05 11:32:47 +02:00
Alexander Aleksandrovič Klimov 40c186515b
Merge pull request #8942 from Icinga/bugfix/idb-hashes
Icinga DB: keep state checksums consistent
2021-07-29 21:54:58 +02:00
Julian Brost 6fa44c8e4e
Merge pull request #8941 from Icinga/bugfix/icingadb-init-all-connections-before-sync
Icinga DB: ensure all connections are ready on first use
2021-07-29 17:33:29 +02:00
Alexander Aleksandrovič Klimov afca6c001e
Merge pull request #8916 from Icinga/feature/icingadb-last_comment_id
Icinga DB: introduce Checkable#last_comment_id
2021-07-29 17:29:51 +02:00