Commit Graph

862 Commits

Author SHA1 Message Date
Michael Friedrich 2760748d78 Fix and improve logging for runtime object sync
config::UpdateObject would create a new object, but this may
have been silently ignored with 'ignore_on_error' - downtimes, etc.
Since we cannot simply fetch the error from inside the config compiler,
we'd just check whether there's a config object created at this stage.
This happens synchronously, and once there is, log something.

The previous code always logged the creation, even if the downtime
was ignored, e.g. when the first master sent one for local host objects.

This commit also adds more details: identity, endpoint, zone to extract
the MessageOrigin details into log messages for better troubleshooting
and debugging.

refs #7198
2019-08-15 09:29:05 +02:00
Michael Friedrich 7c1f716dad Fix cookie with ActivateItems 2019-08-13 16:09:26 +02:00
Michael Friedrich c30edd0a34 Fix message origin for runtime created config object (create/delete events) 2019-08-13 15:05:47 +02:00
Michael Friedrich 743dcad35d Improve logging for downtime/ack events (add, remove, expire)
fixes #7374
2019-08-06 13:28:58 +02:00
Michael Friedrich c42a2583f0 Cluster sync: Only sync valid UTF8 content (text config, no binaries)
- *.conf files are sanitized automatically.
- Other files detect sanitizing and treat that as unsupported type

refs #7382
2019-08-02 16:06:32 +02:00
Michael Friedrich ebd6735c70 TLS Ciphers: Add compatibility suites for older agents (el7) 2019-07-30 10:55:29 +02:00
Michael Friedrich eff6e7662c Fix style and comments 2019-07-15 09:54:35 +02:00
Diana Flach 5fbc052aba Cluster Sync: Improve log messages 2019-07-15 09:54:26 +02:00
Michael Friedrich b00e1d0c67 Config sync: Count the updates and log them
```
[2019-07-10 12:34:27 +0200] information/ApiListener: Received configuration updates (2) from endpoint 'master1' are equal to production, not triggering reload.
```
2019-07-15 09:54:09 +02:00
Diana Flach 87d4575af8 Cluster Sync: Ensure that files are synced everytime 2019-07-15 09:53:47 +02:00
Alexander A. Klimov 3f4cb0936c Add ApiListener::UpdatedObjectAuthority()
refs #7086
2019-07-11 12:58:07 +02:00
Michael Friedrich 46287c92e6 Cluster: Avoid checking for checksum length with internal files in use
fixes #7282
2019-07-04 13:52:31 +02:00
Janne Heß 3e801fbd5a Fix Path to staged files
The paths in the list are relative, not absolute to the stage directory.
2019-06-26 02:04:06 +02:00
Michael Friedrich 08a47600be Config sync: Only copy paths to prod which are actually there
Stored files may be removed by external sources.
2019-06-19 17:00:50 +02:00
Michael Friedrich db4cc13770 Config Sync: Only log config files for stage, no metadata 2019-06-19 16:09:16 +02:00
Michael Friedrich 577e42e137 Quality: Comments and logs in cluster config sync 2019-06-19 14:46:11 +02:00
Michael Friedrich 3852c51c9f Cluster sync: Don't load/sync the .authoritative config file marker
This would influence everything else, and it isn't needed anywhere
but the master instance (zones.d -> var-zones).
2019-06-19 14:46:11 +02:00
Michael Friedrich 0aa6f1a3b3 Use boost::filesystem & Utility classes for file IO 2019-06-19 14:46:11 +02:00
Michael Friedrich 6add9f9ecb Avoid concurrent cluster config sync transactions
fixes #6660
2019-06-19 14:46:11 +02:00
Michael Friedrich af8624dcf1 Apply ReloadTimeout for 2.11 2019-06-19 14:46:11 +02:00
Michael Friedrich b3b7abdfe8 Spam the log with config file copies from stage to prod 2019-06-19 14:46:11 +02:00
Michael Friedrich 4c6150b254 Improve checksum logic and logging 2019-06-19 14:46:11 +02:00
Michael Friedrich f92f6f7f8c Improve checksum checks for each file content 2019-06-19 14:46:11 +02:00
Michael Friedrich a6ddef17d9 Enhace logging when config change yes/no will trigger further reload actions 2019-06-19 14:46:11 +02:00
Michael Friedrich c230e503e6 Fix global checksum calculation 2019-06-19 14:46:11 +02:00
Michael Friedrich 7a02990ef8 Refactor the client sync, part II (WIP, currently checksums generate an endless loop) 2019-06-19 14:46:11 +02:00
Michael Friedrich 6105ace50f Improve variable names in ApiListener::SendConfigUpdate() 2019-06-19 14:46:11 +02:00
Michael Friedrich a4b48fc7f4 Update code docs 2019-06-19 14:46:11 +02:00
Michael Friedrich fcc1799a5d Split config file sync updates, part I
This commit also introduces a playground for checksums,
whilst refactoring the code in large parts.
2019-06-19 14:46:11 +02:00
Michael Friedrich 9df389a843 Improve logging for ignored config updates where we are authoritative for (config master) 2019-06-19 14:46:11 +02:00
Michael Friedrich efc2289178 Remove duplicated validation paths in function signatures 2019-06-19 14:46:11 +02:00
Michael Friedrich 043824a6a9 Leave partial deletes as is, this is dealt with stage purge later 2019-06-19 14:46:11 +02:00
Michael Friedrich b3fa51a5dc Code Documentation: Config file sync
Adds headers to all functions including parameters. This unveils
certain unused ones too.
2019-06-19 14:46:11 +02:00
Michael Friedrich 604a8a041d Update log message and implement recursive diff delete 2019-06-19 14:46:11 +02:00
Michael Friedrich 2acf3a6941 Indicate a warning in the 'icinga' check when cluster stage validation failed
- success: clear the last failed attribute
- failed: populate it with the output and current timestamp

This can be used to highlight this in the 'icinga' check task.
Since 2.9 we don't have problems with circular library dependencies
with just one linked binary, therefore it is safe to include libremote
in libmethods here.
2019-06-19 14:46:11 +02:00
Michael Friedrich 46cb806b3f Add a note for config updates V1 and V2
Old clients sync !.conf via update_v2 message, we cannot
remove this handling for the time being.
2019-06-19 14:46:11 +02:00
Michael Friedrich 83c11962b2 Only remove directories if they exist during sync 2019-06-19 14:46:11 +02:00
Michael Friedrich 4e9439f2d8 Ensure that config master zones.d -> var-api-zones sync removes deleted files 2019-06-19 14:46:11 +02:00
Michael Friedrich 9d53db1401 Purge stage and production directories before copying files
The cluster-message -> production diff is still intact, we're
just taking care of unwanted/deleted files here.
2019-06-19 14:46:11 +02:00
Michael Friedrich 86108e6a1e Improve logging and code quality 2019-06-19 14:46:11 +02:00
Michael Friedrich fb367e12cc Store the last failed zone stage sync validation as runtime ApiListener attribute 2019-06-19 14:46:11 +02:00
Michael Friedrich a91bbe8acd Fix constant value for zone var override 2019-06-19 14:46:11 +02:00
Michael Friedrich e3e68caaa3 Inherit parent process arguments for defined path constants 2019-06-19 14:46:11 +02:00
Michael Friedrich e545884952 Improve logging for staged config sync 2019-06-19 14:46:11 +02:00
Michael Friedrich 1853254201 Pass the zonesVar override around 2019-06-19 14:46:11 +02:00
Michael Friedrich 2ed56b50a4 Ensure directory paths are created from stage -> prod 2019-06-19 14:46:11 +02:00
Michael Friedrich c2d7063ae7 Better signal for checking the cluster config sync stage (ignore production) 2019-06-19 14:46:11 +02:00
Michael Friedrich 506eee2f7d Fix crash 2019-06-19 14:46:11 +02:00
Michael Friedrich 2c39d69428 Implement first draft for cluster config staged sync 2019-06-19 14:46:11 +02:00
Alexander A. Klimov 42a33cdc7d Fix build errors with Boost v1.70
refs #7237
2019-06-07 16:30:34 +02:00