Commit Graph

5910 Commits

Author SHA1 Message Date
Julian Brost 596fcdc123 Downtime::DowntimesExpireTimerHandler: don't copy vector
`ConfigType::GetObjectsByType<Downtime>()` already returns a
`std::vector<Downtime::Ptr>` so there is no point in copying it into another
vector of the same type just to then iterate the copied vector instead of the
original one.
2021-12-01 13:05:23 +01:00
Yonas Habteab 361807f7a9
Adjust incosistent pki log messages (#8965) 2021-11-22 16:06:55 +01:00
Julian Brost d09925189a
Merge pull request #9037 from Icinga/Al2Klimov-patch-4
InfluxdbCommonWriter#Flush(): fix log message
2021-11-19 17:09:05 +01:00
Julian Brost 2ad0a4b8c3 Add missing include to fix non-unity builds
This commit fixes the following build error:

    [ 55%] Building CXX object lib/icinga/CMakeFiles/icinga.dir/usergroup.cpp.o
    lib/icinga/usergroup.cpp:79:24: error: incomplete type ‘icinga::Notification’ used in nested name specifier
       79 | std::set<Notification::Ptr> UserGroup::GetNotifications() const
          |                        ^~~
2021-11-17 16:11:15 +01:00
Noah Hilverling 4d3b1709fd
Merge pull request #9009 from Icinga/bugfix/icingadb-runtime-updates-delete-relationships
Icinga DB: Make sure object relationships are handled correctly during runtime updates
2021-11-12 17:52:59 +01:00
Julian Brost b9e6273ba0 Icinga DB: only log queries at debug level 2021-11-12 15:41:17 +01:00
Noah Hilverling 7a0796061a IcingaDB::AddObjectDataToRuntimeUpdates(): Copy data before modifying 2021-11-12 13:34:57 +01:00
Noah Hilverling 10bde2075a Dictionary: Make sure underlaying map is ordered 2021-11-12 13:34:57 +01:00
Noah Hilverling 73e0d6e61b Icinga DB: Make sure object relationships are handled correctly 2021-11-12 13:34:57 +01:00
Noah Hilverling 4e79eb080c
Merge pull request #9058 from Icinga/bugfix/icingadb-prefix-command_id
IcingaDB: Prefix command_id with command type
2021-11-11 11:50:26 +01:00
Noah Hilverling c1098bef35
Merge pull request #9061 from Icinga/add-downtime-duration-and-service-state-host-id-streams
Icinga DB: Add `downtime.duration` & `service_state.host_id` to Redis
2021-11-11 10:19:47 +01:00
Noah Hilverling a9c2304c61 IcingaDB: Prefix command_id with command type 2021-11-09 12:26:30 +01:00
Eric Lippmann 35053ac1dd Icinga DB: Sync groups earlier
Host and service groups are structural information that are used
for Web filters and should therefore be synchronized as soon as
possible.
2021-11-09 11:17:01 +01:00
Alexander A. Klimov 07c8440fd2 Icinga DB: sync checkables along with their states first
`WorkQueue#ParallelFor(x, false, y)` will enqueue x's items in FIFO order,
so x has to start with host and service.
2021-11-09 11:17:01 +01:00
Yonas Habteab fe5aa1e18d Icinga DB: Add `service_state.host_id` to Redis 2021-11-09 11:08:22 +01:00
Yonas Habteab 5dc45baebb Icinga DB: Add `downtime.duration` & `scheduled_duration` to Redis 2021-11-09 11:08:22 +01:00
Julian Brost 848f1ae167
Merge pull request #8998 from Icinga/bugfix/icingadb-program-start-milliseconds
Icinga DB: set value in milliseconds for program_start in stats/heartbeat
2021-11-08 18:18:19 +01:00
Julian Brost 524fe92a1d
Merge pull request #9028 from Icinga/bugfix/icingadb-zone-parent
IcingaDB: actually write parent to parent_id of zones
2021-11-08 18:08:48 +01:00
Julian Brost e46d83b6be Icinga DB: set value in milliseconds for program_start in stats/heartbeat 2021-11-08 14:37:08 +01:00
Noah Hilverling 0b9317a5bf IcingaDB: Remove GetObjectIdentifiersWithoutEnv()
Having the command type be a part of the command ID isn't needed anywhere. Removing this simplifies the way we generate IDs in general, because we don't need Prepend() anymore.

The command type was only needed to prevent ID collisions within the command_envvar and command_argument tables. Those tables have since been separated into {check,event,notification}command_envvar and {check,event,notification}command_argument tables.
2021-11-05 17:01:40 +01:00
Julian Brost 3c8672b4dc Icinga DB: increase Redis schema version
PR #9036 introduces some incompatible changes to the Redis schema, most
importantly where Icinga DB has to read the environment from: now it has to use
a new top-level key of the icinga:stats message instead of a value in the
IcingaApplication part of that message.
2021-11-05 14:14:37 +01:00
Julian Brost 6007848146 IcingaDB: export environment_id via API
Primarily required for Icinga DB integration tests at the moment, but could
also be helpful in other situations.
2021-11-05 14:14:37 +01:00
Julian Brost 4ade4c757b IcingaDB: write new environment to icinga:stats stream 2021-11-05 14:14:37 +01:00
Julian Brost 525dd50859 IcingaDB: introduce a new environment ID derived from the CA public key
In order to avoid changes to the environment ID, it is now no longer derived
from the Environment constant but instead from the public key of the CA
certificate. This ensures that it is different between clusters by default, so
no additional changes have to be done to allow two clusters to use Icinga DB to
write into the same database.

To prevent the ID from changing when the CA certificate is replaced, it is also
persisted into the file /var/lib/icinga2/icingadb.env, so if that file exists,
it takes precedence over the CA certificate.
2021-11-05 14:14:37 +01:00
Julian Brost 6cd3a483a0 tlsutility: move hex encoding into a separate function BinaryToHex 2021-11-05 14:14:37 +01:00
Julian Brost f976e351f4
Merge pull request #9044 from Icinga/bugfix/idb-dump-buf-lost
Icinga DB init. dump: flush both buffered states and state checksums
2021-11-04 12:26:28 +01:00
Alexander A. Klimov 0ff7d0a06e Icinga DB: raise icinga:schema 1 -> 2 2021-11-02 15:00:55 +01:00
Alexander A. Klimov b1714a10c2 Icinga DB: make icinga:history:stream:*#event_id deterministic
... i.e. UUID -> SHA1(env, eventType, x...) given that SHA1(env, x...) = type-specific ID.
Rationale: allow both masters to write the same history concurrently (while not
in split-brain), so that REPLACE INTO deduplicates the same events written twice.

* ack: SHA1(env, "ack_set"|"ack_clear", checkable.name, setTime)
* comment: SHA1(env, "comment_add"|"comment_remove", comment.name)
* downtime: SHA1(env, "downtime_start"|"downtime_end", downtime.name)
* flapping: SHA1(env, "flapping_start"|"flapping_end", checkable.name, startTime)
* notification: SHA1(env, "notification", notification.name, notificationType, sendTime)
* state: SHA1(env, "state_change", checkable.name, changeTime)
2021-11-02 15:00:03 +01:00
Alexander A. Klimov 5c44365c4e Icinga DB: make icinga:history:stream:notification#id deterministic
... i.e. UUID -> SHA1(x..., send time) given that SHA1(x...) = notification id.
Rationale: allow both masters to write the same notification history concurrently (while
not in split-brain), so that REPLACE INTO deduplicates the same events written twice.
2021-11-02 15:00:03 +01:00
Alexander A. Klimov c2422c56fe Icinga DB: make icinga:history:stream:state#id deterministic
... i.e. UUID -> SHA1(x..., check time) given that SHA1(x...) = checkable id.
Rationale: allow both masters to write the same state history concurrently (while
not in split-brain), so that REPLACE INTO deduplicates the same events written twice.
2021-11-02 15:00:03 +01:00
Alexander Aleksandrovič Klimov f5f8ccb1f4
Merge pull request #9020 from Icinga/feature/icingaeb-schema-version
Icinga DB: publish Redis schema version via XADD icinga:schema
2021-10-25 13:21:37 +02:00
Alexander A. Klimov d8b4768471 Icinga DB init. dump: flush both buffered states and state checksums
not to dump x states, but only x - (x % bulk) state checksums.
2021-10-21 13:49:24 +02:00
Noah Hilverling a7cbf50674
Merge pull request #9030 from Icinga/Al2Klimov-patch-1
Icinga DB: don't include checkable types in history IDs
2021-10-19 14:52:43 +02:00
Alexander A. Klimov 4b0688047e Icinga DB: stream runtime state updates only to icinga:runtime:state
... where they belong to, not to icinga:runtime.
2021-10-18 18:11:30 +02:00
Alexander Aleksandrovič Klimov 99c5c24a17
InfluxdbCommonWriter#Flush(): fix log message
s/InfluxdbWriter/Influxdb2Writer/

fixes #9035
2021-10-14 12:03:45 +02:00
Alexander Aleksandrovič Klimov e0339c387b
Icinga DB: don't include checkable types in history IDs
... as they’re unnecessary for being distinguish across types.
Services always have a ! in the name, hosts never do.
2021-10-11 16:14:30 +02:00
Alexander Aleksandrovič Klimov 30a5ba3961
Merge pull request #9002 from Icinga/feature/icingadb-remove-usernotification-stream
Icinga DB: remove usernotification history stream
2021-10-08 19:16:26 +02:00
Alexander Aleksandrovič Klimov 4190d58668
Merge pull request #9011 from Icinga/bugfix/icingadb-remove-zone-parent-key
Icinga DB: Remove unused Redis key 'icinga:zone:parent'
2021-10-08 17:19:51 +02:00
Alexander Aleksandrovič Klimov ff60c1af37
Merge pull request #8895 from Icinga/bugfix/typo-8766
Fix typo
2021-10-08 17:19:20 +02:00
Noah Hilverling 750e64b974 Icinga DB: Remove unused Redis key 'icinga:zone:parent' 2021-10-08 12:06:14 +02:00
Julian Brost df84a498f4 IcingaDB: actually write parent to parent_id of zones
This fixes that the code used the wrong variable. Previously, it was written to
Redis that each zone is its own parent (if it has a parent at all).
2021-10-08 11:15:54 +02:00
Alexander A. Klimov 3bf180a341 Fix typo
refs #8766
2021-10-08 10:27:35 +02:00
Alexander Aleksandrovič Klimov ed50a9d529
Merge pull request #9001 from Icinga/feature/icingadb-add-user-ids-to-notification-history
Icinga DB: Write IDs of notified users into notification history stream
2021-10-01 17:42:48 +02:00
Alexander Aleksandrovič Klimov 63fca8faa1
Merge pull request #9000 from haxtibal/feature/journaldlogger
JournaldLogger - log to systemd journal
2021-10-01 17:42:10 +02:00
Alexander A. Klimov 0182d793ac Icinga DB: publish Redis schema version via XADD icinga:schema
... to be able both to subscribe for its change and to just fetch it.
2021-10-01 15:58:57 +02:00
Alexander Aleksandrovič Klimov 6cf0673c11
Merge pull request #9010 from Icinga/feature/icingadb-scheduling_source
Make CheckResult#scheduling_source available to Icinga DB
2021-09-27 16:31:16 +02:00
Tobias Deiminger eb8f67335e Define SD_JOURNAL_SUPPRESS_LOCATION more locally
add_definitions would set SD_JOURNAL_SUPPRESS_LOCATION for all targets
in directory and sub-directories. However, another future target might
want the opposite, so define it as local as possible to journaldlogger.cpp.

To make this work, we must take journaldlogger.cpp out of the unity
build, because all files from a unity of share compiler definitions.
2021-09-23 16:08:39 +02:00
Tobias Deiminger 173caa42aa Add a JournaldLogger
As proposed in #8857, this adds a Logger subclass that writes structured
log messages via journald's native protocol by calling sd_journal_sendv.
The feature therefore depends on the systemd library. sd_journal_sendv is
available since the early days (systemd v38), so a version check is
probably superflous.

We add the following fields to each record:
- MESSAGE: The log message
- PRIORITY (aka severity): Numeric severity as in RFC5424 section 6.2.1
- SYSLOG_FACILITY: Numeric facility as in RFC5424 section 6.2.1
- SYSLOG_IDENTIFIER: If provided, use value from configuration.
  Else use systemd's default behaior, which is to determine the field
  by using libc's program_invocation_short_name, resulting in "icinga2".
- ICINGA2_FACILITY: Facility as in Log::Log(..., String facility, ...),
  e.g. "ApiListener"
- some more fields are added automatically by systemd

Fields are stored indexed, so we can do fast queries for certain field
values. Example:

$ journalctl -t icinga2 ICINGA2_FACILITY=ApiListener -n 5

Syslog compatiblity is ratained because good old tag, severity and facility
is stored along, and systemd can forward to syslog daemons.

See also https://systemd.io/JOURNAL_NATIVE_PROTOCOL/.
2021-09-23 16:08:11 +02:00
Alexander A. Klimov 755fc72a66 Make CheckResult#scheduling_source available to Icinga DB 2021-09-22 16:57:49 +02:00
Julian Brost 6fc15449a8
Merge pull request #8953 from Icinga/bugfix/icinga-checksum-state-growing
Icinga DB: clean up vanished objects from icinga:checksum:*:state
2021-09-17 12:04:41 +02:00