Squashed Rebase, commit msg below.
convertv1->v2 initial import
refs #2743
itl: fix inclusion of notification.conf
Config Conversion: command macros, templates
refs #2743
WIP: try resolving the host->svc relation from templates and tricks
not finished yet, there's heavy recursion required
WIP configconversion: rewrite recursive lookup of linked host_name/service_description
this is a mess. configs may just use "name" instead of "host_name" on
type "host", same goes for service and service_descriptions. well, and
if it's a template, "name" is populated too.
the algorithm is not clean enough, and does not scale at all. there are
bugs, and the hostgroup linking is not yet taken into account too.
basics host -> service dumping of v2 config is implemented.
refs #2743
ConfigConvert: split into modules, add objects.cache read, dump cmd line
basically it works with
- dumping a host with all services, using templates
- check_command and all arguments/user macros translation
- reading the objects.cache file location from icinga.cfg
=> this is reserved only for getting dedicated relations
=> we do not want to duplicate the work of xodtemplate.c in perl here
- all important functions have been split up into their respective
modules in order to support better re-usage (i.e. lconfexport)
still missing important bits:
- detect service->group<-host links and build templates
- detect other ugly template methods
- dump templates
- the ugly deprecated normal|retry_check_interval mapping
- timeperiod mapping
- contact => user mapping
- notifications rework (dedicated object)
- dependencies, escalations (to be documented/implemented)
- the infamous rest
ConfigConvert: fix hash keys for convert/dump in perl 5.14
ConfigConvert: improved template handling, host/service dump 2x
refactored code again, removed nested loop bottlenecks. still, the
resolval of host_name/service_description requires more love.
templates are now correctly detected, linked, and printed in 2x syntax.
service templates are _not_ linked against host definitions in 2x as
this is not possible.
commands for hosts do not exist in 2x, therefore the command_name of an
1x host will be taken and guessed against its related services. if
there's a match, the service will be linked as 2x 'hostcheck'.
*_interval still needs proper handling, as the default interval is 1m,
but 2x requires the identifier.
ConfigConvert: refactor host_name/service_description link resolving
previous algorithm was buggy, as well get_host_name always looked up
host template objects, even if the template object was a service. this
is now fixed by re-using the __TYPE attribute (set on Icinga 1x config
read).
the algorithm is now like
|---------------------------------|
\/ |
svc_desc? |
0 / \ 1 |
use tmpl? END -------> [svc_desc] |
0 / \ 1 |
[undef] <- BROKEN getobjbyattr(['use'], __TYPE, svcdesc |
| |
getsvcdesc(obj) |
| |
--------------------------------------
refs #2743
ConvertConfig: add basic support for users, timeperiods, *groups
as well as display_name as attribute to print something.
contact => user mapping is added as well, manipulating the __TYPE var
too. furthermore display_name on the host will be overwritten by the
alias, if set.
the groups don't dump much, only the display_name if set. their dump
function is implemented generic, based on the __TYPE
ConfigConvert: fix 4 little big thinkos on import/convert/dump
1) do not skip lines containing ;.* - there may actually be a comment on
attr->val line. rather clean the line safely, and let the magic happen.
2) the type counter was implemented in order to have unique object ids
by type, since 1x config does not provide us with such. problem - the
function had a local var, saving the counter for each file, overwriting
the hash attributes later on, starting again with 0. nasty bug, hard to
figure out. fixed by feeding the type_cnt into the cfg_obj hive for now.
revamped the state machine logic as well, like so
I: define TYPE { --> __TYPE = TYPE
II: attr val --> {type}->{type_cnt->cnt}->{attr} = val
III: } --> type_cnt->cnt++
IV: <empty/comment>
3) also skip service templates (__IS_TEMPLATE) sooner, and do not try to
calculate some host_name/service_description magic out of them - they
just can't go down one level to the host, but only up with 'use'.
4) host and service templates on 2x config dump must check if
__USES_TEMPLATE is actually set to 1, checking for defined is not
enough.
whatthecommit.com:
A long time ago, in a galaxy far far away...
refs #2743
ConfigConvert: add more host/service attributes, groups
*_interval are automatically mapped, while the *groups attribute it
converted into an array in order to easily stash more groups on it.
still a todo - loop over existing groups and re-link their members into
the respective objects, as 2.x only supports that location.
fix for the host->service relation - use the already processed service
2x hashref in order to do some magic with host->{'SERVICE'} = service
all 2x attributes are now encapsulated with "", to stay safe.
todo - more generic config 2x dump.
furthermore, add some more code comments for the respective sections.
refs #2743
ConfigConvert: fix service linking not unique, add hostgroup->members re-link to host->hostgroups
while working on the missing servicegroups and their facepalm members
syntax (host1,svc1,host2,svc2 - awesome to parse), i figured that the
whole detection of services by their attribute 'service_description' is
not uniquely implemented. now the getter functions are duplicated, and
not modular anymore, for the sake of having a service identified by its
host_name + service_description. since that lookup is recursive, we must
pass the host_name from the very beginning - otherwise we could
accidently overwrite that with a template entry.
the hostgroup re-linking works, also thanks to the fact that the inner
object structure is just an array in the hashref, which allows smart
push.
contact/user groups are tricky again due to the different naming.
further todo: sort all macros and hostservice links. right now this
looks like chaos.
refs #2743
ConfigConvert: basic servicegroup parsing method
ConfigConvert: refactor multiple host_name's on service object detection
this is ugly. a service object may contain 'host_name' with
'host1,host2,host3' and so on. treating this as string won't allow us to
look into the template tree for valid host attributes, so we need to
rewrite the getter function, allowing some special treatment on
host_name arrays.
for the service object preparation this means: while in the service
loop, we need to loop again over all hosts, then dclone the hashref
object for the service, and add the unique host_name attribute again.
we might require some loop unrolling later, but for now this works as
expected.
refs #2743
ConvertConfig: disable debug output, some reformatting of dump
ConfigConvert: refactor host-svc links, add full service/usergroup relinking
due to the relinking afterwards we actually modify the service objects
already prepared for 2x. since the *group magic depends on the
host/services/users already prepared, we get a perpetuum mobile here
with the host loop and resolving service relations.
in order to safely fix the issue, there's a seperate host-service link
loop after resolving all the other needed objects, which may cost a
little bit more performance, but is safe and standalone, getting all
previous modifications, migrations and also hacks.
next to the refactored flow, the issue with resolving the correct
servicegroups and stashing them into an array is solved, as well as
converting the existing contactgroups to usergroups, as well as
re-linking existing contactgroup->members to their new home,
user->usergroups.
note: when parsing the servicegroup members into an array, it must not
be sorted afterwards, because we will shift 2 objects (host, service) in
order to identify the correct service object for adding the servicegroup
item.
some code reorganisation too, plus a minor comma fix for the host
address macro.
the used 1x configuration will be addressed in 2743.cfg on
icinga-core.git, with all future additions.
refs #2743
ConfigConvert: *interval, max_check_attempts dumped as numbers
see
https://git.icinga.org/?p=icinga2.git;a=blob_plain;f=lib/icinga/icinga-type.conf;hb=HEAD
refs #2743
ConfigConvert: fix undocumented contact templates conversion
refs #2743
s/DumpIcinga2Cfg/ExportIcinga2Cfg/g
just for Import/Export clarification.
configconvert: use special prefix for internal attributes
__I2CONVERT_ should do the trick, not to interfere with existing custom
variables.
configconvert: fix leftover from rename
configconvert: properly escape strings with double quotes
* command_line and command_args
* notification commands
* action_url, notes_url
* notes
* icon_image, icon_image_alt
fixes#3931
configconvert: fix multiple templates usage, template object lookup error
multiple host/service templates seperated by commas will cause some
irritation, therefore this is treated as array internally, like group
members. for the config export and "inherits", this is easy to fix. for
the actual host_name lookup for service objects, this unveiled another
bug when looping through all available templates - they are unique by
'name' and not by hostname/servicedesc, and require their very own
getter - obj_get_tmpl_obj_by_tmpl_name() - which makes sure to return
the object which is then looked into recursively if there's a host_name
provided.
possible bug for now - multiple service templates used and the host_name is
defined in one of them. but that's not the usual case, and won't be
supported for now.
refs #2743fixes#3932
configconvert: dump modular timperiod attr and values
we need to ignore all other attributes which are to be cleaned sooner,
just for safety.
minor flaw: the keys are not sorted by weekdays now, but dumped like the
hash provides them.
fixes#4013
configconvert: support 'name' for groups' if *group_name is not set
this is a fallback for broken configs only, and since i got some, added
a small fix.
refs #2743
configconvert: add service->hostgroup<-hostmembers conversion to template support
WIP: there's multiple service->hostgroup relations - duplicated hosthg
templates. must be fixed.
refs #2743
refs #4016
configconvert: refactor service-hg-hosts for one hosthg template and multiple services
we need to first loop over all services, getting the key ids, collecting
them for each hostgroup_name (this is the unique identifier, because
this is our hosthg template later on).
then we loop over the collected unique hostgroups and build the 2.x
template logic. the difference here is now, that the code is refactored,
and loops over the service keys (getting their objects) only once,
pushing their templates plus a relation to a single (1) hosthg-template.
there's still some space for enhancements - the service template names
as well as hosthg template names as well as service object names are
generated with some hardcoded strings/numbers.
refs #2743
refs #4016
configconversion: remove debug output
configconvert: only dump macros if there are key-val pairs
configconvert: code cleanup
configconversion: explicitely check __I2CONVERT_USES_TEMPLATE == 1 when dumping service templates
refs #4020
configconvert: refactor how services without host_name are handled
this is mainly the cause when using hostgroup relations, or other object
tricks. we still need the single loop run to convert all 2.x object
attributes in one run, and deal with the missing attributes later on.
configconvert: WIP notification conversion
- refactored notification_commands parsing and storage
- do not add duplicated notification_command names to the 2x
notification object hive
- create notifications based on the notification command linked from
users
- move host->service link magic to the end again
refs #4008
configconvert: WIP notifications, how service/hosts relate to users and notifications
there may be more than one notification for the user, and we already had
the type in place. currently, that type is not taken into account (bug
to fix).
the rest with service -> user -> notification linking works in the first
implementation.
refs #4008
configconvert: final stage, rework notification type handling, fix naming amd template output
the config dump must take care of which notification type is dumped,
otherwise there would be yet another look required. Instead we are just
passing a reference from users to hosts/services with all prepared
notification attributes, and check them lazy on dump then.
further fixe on the notification names - they are not unique by their
command (now used on host/service scope, not user/contact) so we'll add
some magic number as suffix.
fixed template name dumping due to changed conversion vars too.
refs #4008
configconvert: write config files, refactor how objects are exported
structure is currently hardcoded, but may be changed to getopts later
on. all files are overridden into conf/ (testing only right on).
refactored export code, removed everything from Convert.pm, and export
is now called from the script directly, wrapped all dump loops and added
more function layers in betweed. open/close file handles take care of
everything, next to a header on top of the files for getting to know
their origin better (our hard work, ey?).
refs #4036
configconvert: fix service description handling, it's now an inner conversion value
previously, i've just re-used 'service_description' which isn't a valid
2.x attribute either. the lookup of service_description attribute in 1.x
templates works already, but populated __I2CONVERT_SERVICEDESCRIPTION
instead. in favor of having a generated value for all our objects, we'll
use that one, also for dumping the configuration.
even further, this is also changed for the service->hostgroup<-hosts
conversion then.
this also avoids writing empty service names into host->service objects.
fixes#4019
configconvert: add small main config file for standalone tests
refs #2743
configconvert: s/usergroups/groups/ for User config dump
configconvert: add README
configconvert: add ITL templates automatically (for services, notifications)
this currently only makes sense for services and notifications.
refs #4037
configconvert: fix timeperiod ranges, add template support for ITL inherits
refs #4037
refs #4013
configconvert: update requirements, perl deps
refs #2743
configconvert: fix import parser - timeperiods require regex (thx LConfImport)
somehow, the Icinga2 import still fails with us holiday definitions. bug
that's not yet determined whose bug that is.
refs #4039
refs #4013
configconvert: add getopts and pod2usage incl help
fixes#4041
configconvert: add notification_interval|period
and refactor check|retry_interval vs normal|retry_check_interval
refs #4008
configconvert: make host->service linked service a template object, referenced by the host
this is not the best solution, as it it creates a service template for
each host-service relation. if you have many host_name entries defined
in your Icinga 1.x config for a service, this will create duplicates
then. requires some more code refactoring, therefore leaving the issue
open.
refs #3933
configconvert: drop config file prefix 'icinga2'
just makes it hard to work with.
refs #4036
configconvert: code cleanup
configconvert: export check_period for hosts/services
refs #2743
configconvert: WIP dependencies - for host based on host_name and parents
refs #4011
configconvert: s/Timeperiod/TimePeriod/g
refs #4013
configconvert: add hostdependency with hostgroup_name hosts unrolling
refs #4011
configconvert: add servicedependency support (host_name)
missing - the loop fun with hostgroups.
special config is here:
https://git.icinga.org/?p=icinga-core.git;a=blob;f=tests/etc/2743.cfg;hb=refs/heads/next#l376
refs #4011
configconvert: add service dependency support for hostgroups
well, a lot of looping required here:
- loop through all child hostgroup hosts and get their hostnames
- get the servive object based on each hostname (loop again) and
the child service description
- loop through all master hostgroup hosts and get their hostnames
- get the master hostname based on each hostname (loop again) and
the master service description, save that as service dependency
given that implementation, dependencies may be converted for now.
once we decide which options we support, we may add them as well.
refs #4011
configconvert: add support for * as service host_name next to comma seperated list
this is currently a proof of concept for services only, but should
target all the object tricks later on. this will be done on host_name
lookup and stored as array.
http://docs.icinga.org/latest/en/objecttricks.html
refs #4010
configconvert: refactor commaseperated list split
... to list of stripped strings into a function.
fixes#4046
configconvert: handle excluded hosts, ignore them on split to array
exclusions/exceptions cannot be treated by icinga2 the same way as in
1.x so it's better to not create unwanted objects.
fixes#4007
configconvert: replace colon in object names with underscore
it's a special delimiter and therefore forbidden.
fixes#4225
configconvert: detect additive inheritance for *groups and use += on export
i guess this can be used for host/service contact(groups) and other foo
as well, so let's see.
refs #4006
configconvert: add wildcard detection for 'members *' in hostgroup
currently adds itself to the host's hostgroups array, but maybe a global
template where all hosts inherit from, plus additive inheritance fit
better here.
refs #4010
configconvert: add a note about servicedeps wildcards todo
refs #4010
configconvert: svg-hg-hst logic may contain multiple hgnames for services
the object tricks section on the 1.x documentation contains everything
which makes the code less readable even here.
luckily the code is rather modular, so the extension for split string to
array, and looping doesn't hurt much.
refs #4010
configconvert: update README with usage and todos
refs #2743
configconvert: fix perl 5.14, skip non-existing objects
configconvert: escape display_name (may contain quotes)
refs #3931
configconvert: refactor using command objects (check/notification)
eventhandlers are missing.
fixes#4305
configconvert: add volatile attribute
fixes#4321
configconvert: eventhandler as EventCommand objects
fixes#4327
update TODOs
configconvert: remove obsolete itl notification template
and check if defined when used.
refs #2743
configconvert: write compat log too in icinga2 conf
Revert "itl: fix inclusion of notification.conf"
This reverts commit 02eb54dc1a818fdcafa6c85817e9f6341cf2831a.
configconvert: fix thinko with check_command|event_command in service
refs #2743
refs #4327
refs #4305
Michael Friedrich
Gunnar Beutner
Michael Friedrich
Gunnar Beutner