178 Commits

Author SHA1 Message Date
Julian Brost
12293d999c
Merge pull request #9190 from Icinga/bugfix/sync-missing-history-information-213
Icinga DB: ensure consistent history streams in HA setup
2022-03-07 11:32:15 +01:00
Julian Brost
db84b834ca
Merge pull request #9214 from Icinga/feature/icingadb-add-previous_soft_state-to-host_state-and-service_state-9210-213
IcingaDB: Add previous_soft_state to host_state and service_state
2022-03-07 11:19:50 +01:00
Alexander A. Klimov
3c8c248508 Icinga DB: ensure is_reachable and severity don't miss updates
refs #9143
2022-02-23 16:06:31 +01:00
Julian Brost
175fd2df44 Icinga DB: remove obsolete StateChangeHandler overload
This version of StateChangeHandler is no longer called anywhere as it was the
wrong function for all previous callers anyways.
2022-02-23 11:08:56 +01:00
Julian Brost
b60d3e5665 Icinga DB: make host problem change events update the state tables but not write state history
StateChangeHandler() is the function used when the actual hard/soft state
changes and thus also writes state history. This is not desired in this case,
instead, a runtime update should be generated, therefore call UpdateState()
instead.

refs #9063
2022-02-23 11:08:56 +01:00
Julian Brost
3466dacaac Icinga DB: make acknowledgement events update the state tables but not write state history
StateChangeHandler() is the function used when the actual hard/soft state
changes and thus also writes state history. This is not desired in this case,
instead, a runtime update should be generated, therefore call UpdateState()
instead.

refs #9063
2022-02-23 11:08:56 +01:00
Julian Brost
5ea0208b83 Icinga DB: make downtime events update the state tables but not write state history
StateChangeHandler() is the function used when the actual hard/soft state
changes and thus also writes state history. This is not desired in this case,
instead, a runtime update should be generated, therefore call UpdateState()
instead.

refs #9063
2022-02-23 11:08:56 +01:00
Julian Brost
f671374762 Icinga DB: don't reimplement volatile state update in SendConfigUpdate
Sending a volatile state update is already implemented in UpdateState, so just
use that function instead of generating the update queries.
2022-02-23 11:08:56 +01:00
Julian Brost
244575ecbd Icinga DB: Merge SendStatusUpdate into UpdateState
Previously, both funktions did related operations but had unclear and confusing
naming:
- UpdateState updated the icinga:{host,service}:state Redis keys.
- SendStatusUpdate sent a runtime update for the icinga:{host,service}:state.

This commit merges both functions into one with a new mode parameter. The
following modes are now supported:
- Volatile: Update the icinga:{host,service}:state Redis key.
- Full: Perform the volatile state update and in addition send a corresponding
  runtime update so that this state update gets written through to the
  persistent database by a running icingadb process.
- RuntimeOnly: Special mode for callers that can ensure that a volatile update
  for the current state was already performed but has to be upgraded to a full
  update.

refs #9063
2022-02-23 11:08:56 +01:00
Alexander A. Klimov
c613e62454 IcingaDB: Add previous_soft_state to host_state and service_state
refs #9210
2022-02-14 11:32:46 +01:00
Julian Brost
4c2f6faa61 Sync comment and downtime removal info for Icinga DB history
When a comment or downtime is removed manually, the name of the requestor and
timestamp have to be synced to other nodes in the cluster to allow all of them
to generate a consistent Icinga DB history stream.

refs #9101
2022-01-24 18:03:03 +01:00
Julian Brost
cd2ae9a886
Merge pull request #9182 from Icinga/bugfix/icinga-db-notification-history-might-use-incorrect-previous_hard_state-9132-213
IcingaDB#SendSentNotification(): make stream deterministic via CheckResult#previous_hard_state
2022-01-21 18:05:49 +01:00
Alexander A. Klimov
2e7fe4da95 IcingaDB#SendSentNotification(): make stream deterministic via CheckResult#previous_hard_state
Now it gets everything from one source, the CheckResult.

refs #9132
2022-01-20 11:27:19 +01:00
Alexander A. Klimov
50e3e31e04 Icinga DB: ensure icinga:*command:argument#order is an int
The config parser requires *Command#arguments#order to be a Number, i.e. 42,
4.2 or even "4.2". That's int-casted where needed, now also for Icinga DB.

Before:

```
object CheckCommand "9117" {
	command = [ "true" ]
	arguments = {
		"4.2" = { order = "4.2" }
	}
}
```

2022-01-03T13:25:07.166+0100	FATAL	icingadb	json: cannot unmarshal string into Go value of type int64
2022-01-20 11:17:22 +01:00
Noah Hilverling
4fd1b153fa IcingaDB::AddObjectDataToRuntimeUpdates(): Copy data before modifying 2021-11-12 17:55:47 +01:00
Noah Hilverling
2d2b165a79 Icinga DB: Make sure object relationships are handled correctly 2021-11-12 17:55:47 +01:00
Noah Hilverling
73d98cc2ff
Merge pull request #9083 from Icinga/bugfix/icingadb-improve-object-identifier-function-2.13
IcingaDB: Remove GetObjectIdentifiersWithoutEnv()
2021-11-12 13:04:07 +01:00
Noah Hilverling
b55e924e8b IcingaDB: Prefix command_id with command type 2021-11-12 10:19:18 +01:00
Noah Hilverling
d164d66808 IcingaDB: Remove GetObjectIdentifiersWithoutEnv()
Having the command type be a part of the command ID isn't needed anywhere. Removing this simplifies the way we generate IDs in general, because we don't need Prepend() anymore.

The command type was only needed to prevent ID collisions within the command_envvar and command_argument tables. Those tables have since been separated into {check,event,notification}command_envvar and {check,event,notification}command_argument tables.
2021-11-12 10:18:33 +01:00
Julian Brost
9a5ba28e0c Icinga DB: increase Redis schema version
PR #9036 introduces some incompatible changes to the Redis schema, most
importantly where Icinga DB has to read the environment from: now it has to use
a new top-level key of the icinga:stats message instead of a value in the
IcingaApplication part of that message.
2021-11-11 16:58:16 +01:00
Julian Brost
42fc3cdcfc IcingaDB: introduce a new environment ID derived from the CA public key
In order to avoid changes to the environment ID, it is now no longer derived
from the Environment constant but instead from the public key of the CA
certificate. This ensures that it is different between clusters by default, so
no additional changes have to be done to allow two clusters to use Icinga DB to
write into the same database.

To prevent the ID from changing when the CA certificate is replaced, it is also
persisted into the file /var/lib/icinga2/icingadb.env, so if that file exists,
it takes precedence over the CA certificate.
2021-11-11 16:58:16 +01:00
Noah Hilverling
e1dbfa1011
Merge pull request #9076 from Icinga/feature/icingadb-history-deterministic-ids-2.13
Icinga DB: make icinga:history:stream:*#event_id deterministic
2021-11-11 16:57:10 +01:00
Noah Hilverling
20828dc6e4
Merge pull request #9084 from Icinga/add-downtime-duration-and-service-state-host-id-streams-2.13
Icinga DB: Add `downtime.duration` & `service_state.host_id` to Redis
2021-11-11 15:27:52 +01:00
Noah Hilverling
b536caa4dc
Merge pull request #9081 from Icinga/feature/sync-checkables-states-first-2.13
Icinga DB: sync checkables along with their states first
2021-11-11 15:27:42 +01:00
Noah Hilverling
c05866686f
Merge pull request #9079 from Icinga/bugfix/idb-dump-buf-lost-2.13
Icinga DB init. dump: flush both buffered states and state checksums
2021-11-11 15:27:35 +01:00
Noah Hilverling
bf46fc447f
Merge pull request #9080 from Icinga/feature/scheduled_by-2.13
Icinga DB: introduce icinga:history:stream:downtime#scheduled_by
2021-11-11 15:27:28 +01:00
Yonas Habteab
ca2f54352f Icinga DB: Add service_state.host_id to Redis 2021-11-11 10:34:24 +01:00
Yonas Habteab
f470c52819 Icinga DB: Add downtime.duration & scheduled_duration to Redis 2021-11-11 10:34:24 +01:00
Eric Lippmann
8c18708819 Icinga DB: Sync groups earlier
Host and service groups are structural information that are used
for Web filters and should therefore be synchronized as soon as
possible.
2021-11-10 17:34:21 +01:00
Alexander A. Klimov
315a08ed9f Icinga DB: sync checkables along with their states first
`WorkQueue#ParallelFor(x, false, y)` will enqueue x's items in FIFO order,
so x has to start with host and service.
2021-11-10 17:34:14 +01:00
Julian Brost
d774ae1186 IcingaDB: actually write parent to parent_id of zones
This fixes that the code used the wrong variable. Previously, it was written to
Redis that each zone is its own parent (if it has a parent at all).
2021-11-10 17:29:53 +01:00
Alexander A. Klimov
9fa92aff97 Icinga DB: raise icinga:schema 1 -> 2 2021-11-10 17:24:55 +01:00
Alexander A. Klimov
1b06444b8d Icinga DB: make icinga:history:stream:*#event_id deterministic
... i.e. UUID -> SHA1(env, eventType, x...) given that SHA1(env, x...) = type-specific ID.
Rationale: allow both masters to write the same history concurrently (while not
in split-brain), so that REPLACE INTO deduplicates the same events written twice.

* ack: SHA1(env, "ack_set"|"ack_clear", checkable.name, setTime)
* comment: SHA1(env, "comment_add"|"comment_remove", comment.name)
* downtime: SHA1(env, "downtime_start"|"downtime_end", downtime.name)
* flapping: SHA1(env, "flapping_start"|"flapping_end", checkable.name, startTime)
* notification: SHA1(env, "notification", notification.name, notificationType, sendTime)
* state: SHA1(env, "state_change", checkable.name, changeTime)
2021-11-10 17:24:55 +01:00
Alexander A. Klimov
6a75d5121b Icinga DB: make icinga:history:stream:notification#id deterministic
... i.e. UUID -> SHA1(x..., send time) given that SHA1(x...) = notification id.
Rationale: allow both masters to write the same notification history concurrently (while
not in split-brain), so that REPLACE INTO deduplicates the same events written twice.
2021-11-10 17:24:55 +01:00
Alexander A. Klimov
4f6c87402e Icinga DB: make icinga:history:stream:state#id deterministic
... i.e. UUID -> SHA1(x..., check time) given that SHA1(x...) = checkable id.
Rationale: allow both masters to write the same state history concurrently (while
not in split-brain), so that REPLACE INTO deduplicates the same events written twice.
2021-11-10 17:24:55 +01:00
Alexander A. Klimov
2bb5ed0cb3 Icinga DB: introduce icinga:history:stream:downtime#scheduled_by
... with the Downtime#scheduled_by attribute.
2021-11-10 17:16:47 +01:00
Alexander A. Klimov
2afbfa67d3 Icinga DB init. dump: flush both buffered states and state checksums
not to dump x states, but only x - (x % bulk) state checksums.
2021-11-10 17:03:00 +01:00
Noah Hilverling
afb2635e76
Merge pull request #9074 from Icinga/bugfix/icinga-checksum-state-growing-2.13
Icinga DB: clean up vanished objects from icinga:checksum:*:state
2021-11-10 16:48:30 +01:00
Noah Hilverling
1f1c1b39d0
Merge pull request #9073 from Icinga/feature/icingadb-remove-usernotification-stream-2.13
Icinga DB: remove usernotification history stream
2021-11-10 16:48:14 +01:00
Noah Hilverling
c1c2135df1
Merge pull request #9072 from Icinga/feature/icingadb-scheduling_source-2.13
Make CheckResult#scheduling_source available to Icinga DB
2021-11-10 16:47:52 +01:00
Noah Hilverling
268252bd17
Merge pull request #9071 from Icinga/feature/icingadb-add-user-ids-to-notification-history-2.13
Icinga DB: Write IDs of notified users into notification history stream
2021-11-10 15:51:55 +01:00
Noah Hilverling
bb55946367
Merge pull request #9068 from Icinga/bugfix/icinga-db-runtime-state-2.13
Icinga DB: stream runtime state updates only to icinga:runtime:state
2021-11-10 15:51:46 +01:00
Noah Hilverling
e9991bff46
Merge pull request #9069 from Icinga/feature/icingaeb-schema-version-2.13
Icinga DB: publish Redis schema version via XADD icinga:schema
2021-11-10 15:51:20 +01:00
Noah Hilverling
89223a16c9
Merge pull request #9070 from Icinga/bugfix/dont-include-checkable-types-in-history-id-2.13
Icinga DB: don't include checkable types in history IDs
2021-11-10 15:51:04 +01:00
Alexander A. Klimov
5801e548d7 Icinga DB: publish Redis schema version via XADD icinga:schema
... to be able both to subscribe for its change and to just fetch it.
2021-11-10 13:21:46 +01:00
Alexander Aleksandrovič Klimov
998e040b9a Icinga DB: don't include checkable types in history IDs
... as they’re unnecessary for being distinguish across types.
Services always have a ! in the name, hosts never do.
2021-11-10 13:19:54 +01:00
Alexander A. Klimov
51b230f69b Icinga DB: stream runtime state updates only to icinga:runtime:state
... where they belong to, not to icinga:runtime.
2021-11-10 13:17:41 +01:00
Julian Brost
7661bee296 Icinga DB: remove usernotification history stream
These will be added to the normal notification stream so there is no more need
for this extra stream.
2021-11-10 13:16:14 +01:00
Noah Hilverling
c141091d4c Icinga DB: Remove unused Redis key 'icinga:zone:parent' 2021-11-10 13:14:54 +01:00
Julian Brost
f8d4eaab70 Icinga DB: Write IDs of notified users into notification history stream 2021-11-10 13:08:22 +01:00