Commit Graph

5884 Commits

Author SHA1 Message Date
Julian Brost 6cd3a483a0 tlsutility: move hex encoding into a separate function BinaryToHex 2021-11-05 14:14:37 +01:00
Julian Brost f976e351f4
Merge pull request #9044 from Icinga/bugfix/idb-dump-buf-lost
Icinga DB init. dump: flush both buffered states and state checksums
2021-11-04 12:26:28 +01:00
Alexander A. Klimov 0ff7d0a06e Icinga DB: raise icinga:schema 1 -> 2 2021-11-02 15:00:55 +01:00
Alexander A. Klimov b1714a10c2 Icinga DB: make icinga:history:stream:*#event_id deterministic
... i.e. UUID -> SHA1(env, eventType, x...) given that SHA1(env, x...) = type-specific ID.
Rationale: allow both masters to write the same history concurrently (while not
in split-brain), so that REPLACE INTO deduplicates the same events written twice.

* ack: SHA1(env, "ack_set"|"ack_clear", checkable.name, setTime)
* comment: SHA1(env, "comment_add"|"comment_remove", comment.name)
* downtime: SHA1(env, "downtime_start"|"downtime_end", downtime.name)
* flapping: SHA1(env, "flapping_start"|"flapping_end", checkable.name, startTime)
* notification: SHA1(env, "notification", notification.name, notificationType, sendTime)
* state: SHA1(env, "state_change", checkable.name, changeTime)
2021-11-02 15:00:03 +01:00
Alexander A. Klimov 5c44365c4e Icinga DB: make icinga:history:stream:notification#id deterministic
... i.e. UUID -> SHA1(x..., send time) given that SHA1(x...) = notification id.
Rationale: allow both masters to write the same notification history concurrently (while
not in split-brain), so that REPLACE INTO deduplicates the same events written twice.
2021-11-02 15:00:03 +01:00
Alexander A. Klimov c2422c56fe Icinga DB: make icinga:history:stream:state#id deterministic
... i.e. UUID -> SHA1(x..., check time) given that SHA1(x...) = checkable id.
Rationale: allow both masters to write the same state history concurrently (while
not in split-brain), so that REPLACE INTO deduplicates the same events written twice.
2021-11-02 15:00:03 +01:00
Alexander Aleksandrovič Klimov f5f8ccb1f4
Merge pull request #9020 from Icinga/feature/icingaeb-schema-version
Icinga DB: publish Redis schema version via XADD icinga:schema
2021-10-25 13:21:37 +02:00
Alexander A. Klimov d8b4768471 Icinga DB init. dump: flush both buffered states and state checksums
not to dump x states, but only x - (x % bulk) state checksums.
2021-10-21 13:49:24 +02:00
Noah Hilverling a7cbf50674
Merge pull request #9030 from Icinga/Al2Klimov-patch-1
Icinga DB: don't include checkable types in history IDs
2021-10-19 14:52:43 +02:00
Alexander A. Klimov 4b0688047e Icinga DB: stream runtime state updates only to icinga:runtime:state
... where they belong to, not to icinga:runtime.
2021-10-18 18:11:30 +02:00
Alexander Aleksandrovič Klimov e0339c387b
Icinga DB: don't include checkable types in history IDs
... as they’re unnecessary for being distinguish across types.
Services always have a ! in the name, hosts never do.
2021-10-11 16:14:30 +02:00
Alexander Aleksandrovič Klimov 30a5ba3961
Merge pull request #9002 from Icinga/feature/icingadb-remove-usernotification-stream
Icinga DB: remove usernotification history stream
2021-10-08 19:16:26 +02:00
Alexander Aleksandrovič Klimov 4190d58668
Merge pull request #9011 from Icinga/bugfix/icingadb-remove-zone-parent-key
Icinga DB: Remove unused Redis key 'icinga:zone:parent'
2021-10-08 17:19:51 +02:00
Alexander Aleksandrovič Klimov ff60c1af37
Merge pull request #8895 from Icinga/bugfix/typo-8766
Fix typo
2021-10-08 17:19:20 +02:00
Noah Hilverling 750e64b974 Icinga DB: Remove unused Redis key 'icinga:zone:parent' 2021-10-08 12:06:14 +02:00
Alexander A. Klimov 3bf180a341 Fix typo
refs #8766
2021-10-08 10:27:35 +02:00
Alexander Aleksandrovič Klimov ed50a9d529
Merge pull request #9001 from Icinga/feature/icingadb-add-user-ids-to-notification-history
Icinga DB: Write IDs of notified users into notification history stream
2021-10-01 17:42:48 +02:00
Alexander Aleksandrovič Klimov 63fca8faa1
Merge pull request #9000 from haxtibal/feature/journaldlogger
JournaldLogger - log to systemd journal
2021-10-01 17:42:10 +02:00
Alexander A. Klimov 0182d793ac Icinga DB: publish Redis schema version via XADD icinga:schema
... to be able both to subscribe for its change and to just fetch it.
2021-10-01 15:58:57 +02:00
Alexander Aleksandrovič Klimov 6cf0673c11
Merge pull request #9010 from Icinga/feature/icingadb-scheduling_source
Make CheckResult#scheduling_source available to Icinga DB
2021-09-27 16:31:16 +02:00
Tobias Deiminger eb8f67335e Define SD_JOURNAL_SUPPRESS_LOCATION more locally
add_definitions would set SD_JOURNAL_SUPPRESS_LOCATION for all targets
in directory and sub-directories. However, another future target might
want the opposite, so define it as local as possible to journaldlogger.cpp.

To make this work, we must take journaldlogger.cpp out of the unity
build, because all files from a unity of share compiler definitions.
2021-09-23 16:08:39 +02:00
Tobias Deiminger 173caa42aa Add a JournaldLogger
As proposed in #8857, this adds a Logger subclass that writes structured
log messages via journald's native protocol by calling sd_journal_sendv.
The feature therefore depends on the systemd library. sd_journal_sendv is
available since the early days (systemd v38), so a version check is
probably superflous.

We add the following fields to each record:
- MESSAGE: The log message
- PRIORITY (aka severity): Numeric severity as in RFC5424 section 6.2.1
- SYSLOG_FACILITY: Numeric facility as in RFC5424 section 6.2.1
- SYSLOG_IDENTIFIER: If provided, use value from configuration.
  Else use systemd's default behaior, which is to determine the field
  by using libc's program_invocation_short_name, resulting in "icinga2".
- ICINGA2_FACILITY: Facility as in Log::Log(..., String facility, ...),
  e.g. "ApiListener"
- some more fields are added automatically by systemd

Fields are stored indexed, so we can do fast queries for certain field
values. Example:

$ journalctl -t icinga2 ICINGA2_FACILITY=ApiListener -n 5

Syslog compatiblity is ratained because good old tag, severity and facility
is stored along, and systemd can forward to syslog daemons.

See also https://systemd.io/JOURNAL_NATIVE_PROTOCOL/.
2021-09-23 16:08:11 +02:00
Alexander A. Klimov 755fc72a66 Make CheckResult#scheduling_source available to Icinga DB 2021-09-22 16:57:49 +02:00
Julian Brost 6fc15449a8
Merge pull request #8953 from Icinga/bugfix/icinga-checksum-state-growing
Icinga DB: clean up vanished objects from icinga:checksum:*:state
2021-09-17 12:04:41 +02:00
Julian Brost 130b22e939 Icinga DB: remove usernotification history stream
These will be added to the normal notification stream so there is no more need
for this extra stream.
2021-09-15 14:47:25 +02:00
Julian Brost 81e5feeb08 Icinga DB: Write IDs of notified users into notification history stream 2021-09-15 14:45:35 +02:00
Tobias Deiminger de7808e32c Make syslog facility handling reusable
The upcoming JournaldLogger will need the same syslog validation and
conversion logic, so factor it out from SyslogLogger to make it
reusable.

Also explicitely include syslog.h, which defines the syslog()
function.
2021-09-15 10:15:22 +02:00
Noah Hilverling 95cdc00ad4
Merge pull request from GHSA-cxfm-8j5v-5qr2
Add TLS server certificate validation to ElasticsearchWriter, GelfWriter, InfluxdbWriter and Influxdb2Writer (v2)
2021-08-19 13:52:29 +02:00
Alexander Aleksandrovič Klimov dfc633074e
Merge pull request #8966 from Icinga/feature/scheduled_by
Icinga DB: introduce icinga:history:stream:downtime#scheduled_by
2021-08-16 16:37:08 +02:00
Julian Brost cb09d6833f RedisConnection: remove now redundant setting of TLS verification parameters
This is now done in UnbufferedAsioTlsStream.
2021-08-13 17:24:24 +02:00
Julian Brost 3ab347bfd4 GelfWriter: show error message of exceptions 2021-08-13 17:24:24 +02:00
Julian Brost 8f3f692ecf InfluxdbCommonWriter: actually verify TLS server certificates
And add a new option ssl_insecure_noverify to explicitly disable it if desired.
2021-08-13 17:24:24 +02:00
Julian Brost 29e9df938c GelfWriter: actually verify TLS server certificates
And add a new option insecure_noverify to explicitly disable it if desired.
2021-08-13 17:24:24 +02:00
Julian Brost 5cada85e54 ElasticsearchWriter: actually verify TLS server certificates
And add a new option insecure_noverify to explicitly disable it if desired.
2021-08-13 17:24:24 +02:00
Julian Brost 396f003c69 Enable hostname verification in UnbufferedAsioTlsStream 2021-08-13 10:58:10 +02:00
Alexander A. Klimov 70b4558a62 Icinga DB: introduce icinga:history:stream:downtime#scheduled_by
... with the Downtime#scheduled_by attribute.
2021-08-09 20:07:38 +02:00
Alexander Aleksandrovič Klimov 852d674ec0
Merge pull request #8957 from Icinga/bugfix/apilistener-detect-ipv6-support
ApiListener: Choose bind host default based on OS IPv6 support
2021-08-09 17:32:40 +02:00
Julian Brost ec73b417f2 ApiListener: Choose bind host default based on OS IPv6 support 2021-08-06 12:19:08 +02:00
Julian Brost 782669f13b IDO PgSQL: always use regular string literals
IdoPgsqlConnection::Escape() internally uses PQescapeStringConn() and its
documentation states the following:

  Furthermore, PQescapeStringConn does not generate the single quotes that must
  surround PostgreSQL string literals; they should be provided in the SQL
  command that the result is inserted into.

So it's intended to use the result in 'string' literals, not in E'string'
literals as Icinga did. This results in problems as the behavior of
PQescapeStringConn() depends on how the current connection will interpret
regular single quoted literals, namely on the value of the
standard_conforming_strings variable.

The E'string' literals were initially introduced in
ac6f3f8acf to fix #1206 where PostgreSQL started
warning about escape sequences in string literals not supported by the SQL
standard (but by PostgreSQL depending on the value of
standard_conforming_strings). In the meantime the oldest PostgreSQL version on
any platform supported by Icinga increased to 9.2 (CentOS 7) and starting with
9.1, standard_conforming_strings is enabled by default, so there will be no
warnings about escape sequences (as the warning is only issued if the escape
sequence is actually interpreted by PostgreSQL).
2021-08-05 11:39:32 +02:00
Alexander A. Klimov e3a5d613aa Icinga DB: clean up vanished objects from icinga:checksum:*:state
... not to let it grow non-stop.
2021-08-05 11:32:47 +02:00
Alexander Aleksandrovič Klimov 3aa2289c59
Merge pull request #8946 from Icinga/bugfix/old-packages
ConfigPackageUtility::ValidatePackageName(): always tolerate already existing packages
2021-08-02 20:27:27 +02:00
Alexander A. Klimov 57df803e35 ConfigPackageUtility::ValidatePackageName(): always tolerate already existing packages
... not to require migrating invalid ones.
2021-08-02 15:40:14 +02:00
Alexander A. Klimov c1df4b70f5 ConfigPackageUtility::PackageExists(): accept invalid package names, too 2021-08-02 15:40:14 +02:00
Alexander A. Klimov c666f81361 De-couple package and stage name validation 2021-08-02 15:40:14 +02:00
Alexander Aleksandrovič Klimov 40c186515b
Merge pull request #8942 from Icinga/bugfix/idb-hashes
Icinga DB: keep state checksums consistent
2021-07-29 21:54:58 +02:00
Julian Brost 6fa44c8e4e
Merge pull request #8941 from Icinga/bugfix/icingadb-init-all-connections-before-sync
Icinga DB: ensure all connections are ready on first use
2021-07-29 17:33:29 +02:00
Alexander Aleksandrovič Klimov afca6c001e
Merge pull request #8916 from Icinga/feature/icingadb-last_comment_id
Icinga DB: introduce Checkable#last_comment_id
2021-07-29 17:29:51 +02:00
Alexander A. Klimov 8476627e91 Icinga DB: keep state checksums consistent
I.e. make hashes in hashmaps and stream the same.
2021-07-29 12:43:40 +02:00
Alexander A. Klimov 5c10fffa3b Icinga DB: introduce Checkable#last_comment_id 2021-07-29 12:22:12 +02:00
Alexander A. Klimov 173a93c487 Split IcingaDB#SendStatusUpdate(), separate stream and history 2021-07-29 12:22:12 +02:00